Drupal Norge

This is the question I got at many events where I presented about localize.drupal.org. We've just rolled out support for suggestion exports for translators about a month ago to make quality control and management easier, but that does not help people much who are just looking to download what's available.

I've also announced about three months ago that translation downloads were becoming stable and continually generated in a nice pace. However, the usability of those downloads left a lot to be desired. When looking at project pages on the site, you were shown a bland list of links to major Drupal versions the project was compatible with which all led to a long list of filenames on an FTP browser frontend. Also, some files being months old looked shocking given I've told you the downloads are now stable and up to date.

Well, as of yesterday, some nice big green buttons landed on several areas of localize.drupal.org which all lead to a new downloads center. If you are on the front page, or a language page, you get to the initial download page which shows Drupal core translation downloads. Project pages however prefill the project selection for you appropriately, so you get to go to the downloads respective for the project. Drupal.org project links were also updated to point to these paths.

We actually have some pretty good data about these translation files that was not available on the plain FTP browser. While some files might be as old as three or even five months, we check the translations for updates and sometimes conclude we don't need to generate new files. Unfortunately some translation teams are not (yet) too active, so it can easily happen that months old files are the most up to date possible. So when on the download page, you can hover over the download links and get some additional data like file size, file date and the last up-to date check time.

Finally, while this was about giving more visibility to downloads, we really love those, who actually contribute. So when you visit the download page for a project, the contribution form will be prefilled with the project, letting you to jump to the project's page on localize.drupal.org directly, or even hop to the translation interface if you also select a language. Contribute and feel the Drupal love!

Ps. Usability feedback on the process is more then welcome. While I think we have far better then what we had before, it can definitely do with some improvements. Also, do not forget to check out the Localization update module to automate downloads, so you don't need to do it manually. A huge timesaver!

read more

Three days ago on the 27th of August, Drupal.org's localization service, localize.drupal.org held its one year anniversary. It is worth a look back and a look forward to understand how far we came and what kind of tasks are ahead of us still.

The new web based user interface for Drupal localization came to unseat the usual tools used to translate Drupal itself, and its modules and themes. Over 30 teams joined the first two months, and most others followed later. The site now hosts over 70 language teams and numerous are in the queue discussing best ways they can leverage our toolset.

If you look through http://localize.drupal.org/news, we kept improving our performance, make our user interfaces simpler, give team maintainers more control over their teams, etc. We flip-flopped from backend and bugfix updates to user interface improvements and new features. We were the second site on drupal.org to deploy the new redesigned theme and therefore serve as a good test case for how it works.

With almost 1800 contributors and over 14000 project releases to translate, it is not surprising that we have a staggering 220000 size database of strings to translate. Attracting new contributors is a must in many teams (if you look at the front page highlighting Drupal core translation status, many teams are really far off from even completing a core translation). But a bigger issue might be quality control, and keeping up with the suggestions coming in. Looking at the list of languages, it is not uncommon for teams to have thousands of suggestions sitting waiting for approval. We have done three key changes in the previous couple months to improve on this: the rethought user interface rolled out earlier this year as well as string deeplinking and suggestion exports all attempt to provide different solutions for the issue, giving suggestions more spotlight. Activity streams/feeds for translation teams are also in the works, so moderators can be notified of new input on a real time basis.

Given localize.drupal.org's success to unseat CVS as the de-facto storage for translations, the git migration plans do away with handling .po files in version control altogether. (At least as far as human managed version control goes). We need a unified place to go for translations, and scattered, duplicated efforts don't do our translators and users favors.

That said, localize.drupal.org still needs some infrastructure improvements to be even easier to use for the average Drupal site builder. We did not make as much progress with the Localized Drupal install profile as we wanted, but it is still on our plan as the tool to drive people into localized installations right away when they start to set up Drupal. The Localization update module already works as a client to all the localize.drupal.org downloads, so you don't need to manually hunt down all the translation files for your modules. More testing there is appreciated.

Looking back on this past year, we accomplished a lot. What we have ahead still looks like a mountain to conquer. Let's do it together!

read more

I just had the chance to listen to the latest DrupalEasy Podcast published earlier this week, where Ryan Price and Mike Anello interview Jacob Redding on his work, book, and Drupal's general greatness in many fields. Jacob was an early supporter of the Localization server idea that was built out to eventually power http://localize.drupal.org, so it was great to hear that he gives some exciting coverage of the topic (at about the middle of the podcast). He explains the Localization client and its connection to the server and how these two interact to get as many people submit translations as possible. If you are not using the Localization client yet, this might be a good time to look at it.

In related news, the Backstage with Drupal localization session proposal for Drupalcon Copenhagen passed many levels of the organization committee and was approved for inclusion in the schedule. In that session, two key figures behind features of the Localization server, Gábor Hojtsy and Jose Reyero will talk about how Drupal localization works, and how the client and server fit into the process. Jose will share details of how the server is being used for Open Atrium at the Translate Open Atrium site, and how you can leverage this tool to translate in-house modules as well as client specific modules, themes and customizations. This session is currently scheduled for 2:45pm on the 25th of August in Room 18.

While we'll focus on localization, Amir Helzer and Robert Douglass will present Translation Management for the Enterprise, showcasing the new Translation Management module, which provides a unified user interface on top of the content, navigation, taxonomy, etc. translation process provided by Drupal core and Internationalization module. The module's promise is that "translators can translate everything without having to learn Drupal administration. Node contents, menus, taxonomy, CCK, blocks and strings are all translated from the same interface. Translators need to translate the contents and the module is responsible for putting everything where it should."

At last but not least, the schedule includes a code sprint day right after the main programme. August 27th will not be all about coding only! Documentation contributors, front end techies and translation teams are also welcome! The last European Drupalcon in Paris seen a big gathering of contributors to translations on the then newly launched localize.drupal.org site. Let's repeat that in Copenhagen too! As usual, let us know if you find bugs or missing features crucial for a more effective translation sprint in Copenhagen.

read more

We've launched the Drupal.org redesign theme on localize.drupal.org about six weeks ago, and the reception was great. While other subsites like api.drupal.org are also in the process to migrate to this theme, we could pioneer some fixes and get them into production. We keep tweaking the theme on this site and get fixes in based on your feedback.

Some great feature additions landed since the last update. The most requested new feature is that you can now export all outstanding suggestions with translations. In case of multiple outstanding suggestions for any one string, the suggestions will be in comments. In case of single suggestions, the export uses Gettext's fuzzy facility and just marks the string as "not ready". Look for this option on the language export screen.

Another great feature which should help translation sprints and discussion around concrete string translations in general is the possibility to link to each individual source string. You can reveal the list of links for each string on a translation page by clicking on the # (hashmark) in the source string table header. This will show links with IDs for each source string. You can bookmark these or use them to discuss strings in forum discussions or chats. The "Welcome to Drupal..." string in Hungarian for example can be deeplinked as http://localize.drupal.org/translate/languages/hu/translate?sid=251536

We also put down our legs on drupal.org! Now every project page on drupal.org includes a link to "View project translations". For example http://drupal.org/project/views will link to the corresponding project page at http://localize.drupal.org/translate/projects/views which gives you status information and download links to specific Views translations.

Since the last update, we've opened some new teams. Recent additions include the Portuguese International, the Gujarati and the Haitian Creole team. Welcome! The process for new team additions is explained on the front page.

Finally, on some technical notes, the main drupal.org site was the first to employ a Bazaar and Hudson based deployment system recently to ease our collaboration on new features. This week localize.drupal.org joined this system, so our code is now fully hosted and managed there.

read more

Mark Boulton Design was selected to lead the drupal.org redesign efforts almost two years. While they went on dutifully with the process listening to feedback from the community and designing a modern home for all things Drupal(.org), and the deliverables were in on time, the actual implementation (which was on the drupal.org community in general) lagged behind a bit. I've personally been to multiple sprints where we established the base for the redesign infrastructure on many levels. The big problems are not around applying a different theme to the drupal.org site but reworking the structure and some sections for the redesign. The latest sprint which I was unfortunately unable to attend was in San Francisco, where in a surprising turn of events, it was set that drupal.org subsites should push forward with implementing the new theme and concepts even ahead of drupal.org. After all, we do not have big rearchitecture plans for these sites, so we can sidestep the biggest tasks which keep drupal.org behind. So came http://association.drupal.org/ up in the new theme shortly after. This got me inspired to get deep down into applying the theme to http://localize.drupal.org/ as well.

To see how far we are, I've put the new theme under localize.drupal.org for a test drive a few weeks ago, and it was not looking nice. Turned out that some developments pushed it away from the underlying navigation-sharing infrastructure we built earlier. So I worked on getting those advantages back and making all the shell of the theme work. Then came putting in the localization server pieces to the right places. This mostly involved making some obscure custom page elements actual blocks to put them into the right place with the new theme. I believe this helped push both the theme and the underlying navigation-sharing goodness to get into a better shape for use on other subsites in the hopefully not too far future.

Finally today was the day to finish it all up, fixing some odd issues and switch over to the new theme for your enjoyment. With this, we join the Drupal Association site in adapting the new theme and hope to provide you with an increasingly integrated experience as other subsites adopt the same theme and shared navigation structure.

Drupal.org redesign, here we come!

(Note that anonymous visitors might get half-baked cached pages while the caches are refilled with all the new pages. Let us know if/when you find issues with the new theme via the usual means.)

read more

Just last week, I've decided to finally document the current download methods for translations coming from localize.drupal.org at http://localize.drupal.org/download. We've set up translation packaging earlier this year and let it run while being monitored for tweaking. We found that we can run the packaging as often as every 5 minutes and with allowance to generate as many as a hundred .po files per run. With these numbers in mind, we can generate 100*12 = 1200 files per hour. But we have 67 languages and 12000 releases to generate .po files for, so that is above 800 thousand files to generate. If we'd need to regenerate all files all the time, it would take 28 days.

Looking at the database, we've generated close to 500 thousand files by now and the drush processes looking at generating more found that its time to update the existing ones instead. We do not generate .po files for language and release combinations where no translation exists, so 500 thousands versus 800 thousands is a good amount of files generated. Given we have all these base files now, all our drush jobs do now is they regenerate files as needed. With them being set to look at 100 releases at once (and sometimes still not finding any translations to regenerate), we seem to be on to regenerate files as new translations come in pretty fast (think days or even hours). We are still experimenting with different parameters and ways to speed the process up, but it looks promising.

Let us know if you have any feedback on the downloadables and especially if you have good/bad experience with http://drupal.org/project/l10n_update, which I'd suggest you try out to automate the flow of getting translations for your sites.

read more

Our last major update to localize.drupal.org was about two months ago, bringing in a whole new user interface for translation and easier navigation and collaboration. Since that was a big user interface change and needed some time to settle, I switched gears to work on the backend of the system. The ultimate goal is to get our main backend module called Localization server as general purpose as possible.

Many people noticed that the software on localize.drupal.org would be nice to use for translation of other projects so generalizing the software to get in more collaborators and contributors was a logical step. The 6.x-2.x-beta2 version of the module was released earlier this month with a blog post explaining its general use titled Using Drupal as a collaborative software translation tool. This new release included some pretty big architectural changes, rebasing the module on l10n_server and starting to separate the database backend, the Drupal specific connectors and the exporters more.

There is still quite some work to do around there, but it was time to make the jump and roll out this new backend on localize.drupal.org as well. This time the rollout went well if you do not notice any changes. While there were minimal UI fixes as well, generally nothing should have changed on the front end. However, the backend now provides more tools to administer the server, so in case of the parsing errors we occasionally notice for example, we can easily reset a release for parsing again on the user interface.

Let us know if you notice any new problems. See the having issues page for explanation on how and where to submit bug reports. Thanks!

read more