Re: Starting the 4.2.0 localization
Some more updates. 1. 4.1.0 is the same as 4.1.5: 97 different strings (the same 97) between Pootle and our template files. This means, probably, that Pootle is not perfectly aligned with any 4.1.x release. This also means that, to the only purpose of translators verifying their translations, 4.1.5 is the best match. 2. An important step forward: I've managed to rebuild 4.1.5 in French using strings from Pootle. I've ignored the 97 different strings (this is hardly noticeable; strings are more than 72.000) and used a Pootle export to rebuild French. The French Pootle strings are still broken, but less than the Portuguese ones; and I've managed to fix manually the two errors reported by $ gsicheck -l "" fr/localize.sdf and proceed with the build (see the l10n list for the specific errors). Note: for some reason I had to run a clean build, I tried something like "build --prepare --from helpcontent2 ; build --from helpcontent2" but this didn't clean up enough. So the good news is: we are now able to import translators' work from Pootle and apply it to a branch (AOO415) that is widely matching (not 100%, but still 99.8% matching). And we could, I believe, do the same for trunk and still get a 95-98% match. The bad news is: Pootle is not reliable for checking that strings are formatted in a valid way; both Portuguese and French were 100% translated but had broken strings that did not allow an import as-is. This is a known issue, but since some languages haven't been imported for years I expect virtually all languages to have potential validity issues. Either we go for semi-automated fixes or we'll have to collect all errors and wait for translators to fix strings in Pootle before producing builds. In any case, there is a lot of manual work and we could try to split it and make it accessible to people who can't fully build OpenOffice too. Regards, Andrea. Andrea Pescetti wrote: Further updates, with some numbers based on Portuguese (one of the several languages that are reported as 100% complete in Pootle). The question to answer is "What state of the code does Pootle reflect?". If I generate the en-US.sdf template from each code version and I check how many strings differ when merging pt translations from Pootle, we have: - trunk: 387 different strings - branch AOO415: 97 different strings - tag AOO401: 170 different strings (note for building AOO401 in case someone really wishes to do so: pay attention to warnings at the bootstrap step, since we were using Google Code at the time and it is no longer available; use SourceForge to download the missing dependencies). So the result is: what we have in Pootle now is closer to the 4.1.x series than 4.0.x or trunk, but it is not even perfectly aligned to 4.1.5. If we found a perfect match with 4.1.0 or 4.1.1, we could create a branch and import all Pootle contents there. Thus we could produce (or let the buildbots produce) "vintage" versions that perfectly match Pootle. Of course they would be useless to users and meant just for translators to quickly check that their work is complete, before moving on to updating Pootle with trunk strings. As for the other front: I've tested the import from Pootle, but we are not there yet. Portuguese translators provided examples of recently changed strings in Pootle (in form controls) to check. But unfortunately it seems that their strings in Pootle are not all well-formed. This still requires a lot of manual work since Pootle does not detect malformed HTML errors. We have a tool named gsicheck that can do some basic checks, but we will need to get to a state where a language in Pootle is 100% translated and error-free in order to test a full import. Regards, Andrea. Il 30/12/2017 00:42, Andrea Pescetti ha scritto: Some further notes (then someone else may want to reorganize this, but for the moment I'm just collecting them): - The problem (not a problem actually; just the need to translate some hundreds strings again) I had empirically found for Italian indeed affects all languages. If you download a non-English build from https://ci.apache.org/projects/openoffice/install/linux64/ or https://ci.apache.org/projects/openoffice/install/win/ create a new Impress document and right-click on the slide you will see the context menu in English. Example for German: https://s13.postimg.org/4acjbnr3b/screen.png - Of course, any manual SDF file changes done in the past will be lost. We can at most create a diff for people to review. But it's clear that when somebody edits a file that contains a 10 lines tall "DO NOT EDIT" notice (and I did it too! but it was a Pootle backport, as in many other cases) then he should accept the risk. - Due to the manual fixes above, SDF files in the AOO415 branch are at times different from those in trunk. We can ignore this: manual changes will be lost anyway, and people who edited the "DO NOT EDIT" files knew what they were doing and will redo the
Re: Starting the 4.2.0 localization
Further updates, with some numbers based on Portuguese (one of the several languages that are reported as 100% complete in Pootle). The question to answer is "What state of the code does Pootle reflect?". If I generate the en-US.sdf template from each code version and I check how many strings differ when merging pt translations from Pootle, we have: - trunk: 387 different strings - branch AOO415: 97 different strings - tag AOO401: 170 different strings (note for building AOO401 in case someone really wishes to do so: pay attention to warnings at the bootstrap step, since we were using Google Code at the time and it is no longer available; use SourceForge to download the missing dependencies). So the result is: what we have in Pootle now is closer to the 4.1.x series than 4.0.x or trunk, but it is not even perfectly aligned to 4.1.5. If we found a perfect match with 4.1.0 or 4.1.1, we could create a branch and import all Pootle contents there. Thus we could produce (or let the buildbots produce) "vintage" versions that perfectly match Pootle. Of course they would be useless to users and meant just for translators to quickly check that their work is complete, before moving on to updating Pootle with trunk strings. As for the other front: I've tested the import from Pootle, but we are not there yet. Portuguese translators provided examples of recently changed strings in Pootle (in form controls) to check. But unfortunately it seems that their strings in Pootle are not all well-formed. This still requires a lot of manual work since Pootle does not detect malformed HTML errors. We have a tool named gsicheck that can do some basic checks, but we will need to get to a state where a language in Pootle is 100% translated and error-free in order to test a full import. Regards, Andrea. Il 30/12/2017 00:42, Andrea Pescetti ha scritto: Some further notes (then someone else may want to reorganize this, but for the moment I'm just collecting them): - The problem (not a problem actually; just the need to translate some hundreds strings again) I had empirically found for Italian indeed affects all languages. If you download a non-English build from https://ci.apache.org/projects/openoffice/install/linux64/ or https://ci.apache.org/projects/openoffice/install/win/ create a new Impress document and right-click on the slide you will see the context menu in English. Example for German: https://s13.postimg.org/4acjbnr3b/screen.png - Of course, any manual SDF file changes done in the past will be lost. We can at most create a diff for people to review. But it's clear that when somebody edits a file that contains a 10 lines tall "DO NOT EDIT" notice (and I did it too! but it was a Pootle backport, as in many other cases) then he should accept the risk. - Due to the manual fixes above, SDF files in the AOO415 branch are at times different from those in trunk. We can ignore this: manual changes will be lost anyway, and people who edited the "DO NOT EDIT" files knew what they were doing and will redo the work properly if needed. - I've started testing the other part, since in the end this is what we must do first: importing translations from Pootle to SDF files, in order to preserve as much as possible what translators did. This is a heavily manual process. The basic steps work: 1. Export PO files from Pootle (for this first test I used the UI, but one can use the command-line too); both UI and help; then expand in the same directory "it" 2. Generate the en-US template: in main/, "localize -e -l en-US -f /path/to/en-US.sdf" 3. Generate the new SDF file for Italian: po2oo -l it -t en-US.sdf --keeptimestamp --skipsource it new_it.sdf This produces a bunch of warnings. I think most of them are due to the fact that the strings in the AOO415 branch and trunk are simply different. Next tests (for me or for anyone else who wants to try) would be: 1. Run the same process (2 and 3) on AOO415 and see if the match is perfect there (no warnings) 2. Build both AOO415 and trunk with the new SDF file and see if translations from Pootle are picked up properly. Regards, Andrea. Il 28/12/2017 00:39, Andrea Pescetti ha scritto: Just a few notes on what I've found so far about the 4.2.0 localization, in case others (like Keith, who wrote he is interested) want to follow and/or proceed. Of course, you will need: 1. A full checkout of trunk; I believe some of the tools are built during the build process, but I'm using a fully built tree anyway. 2. Good knowledge of another language than English (I use Italian). The wiki contains instructions for several generations of tools (past, present and even future - but let's fix the current tools before passing to the new ones that were promising but were never used in production). The current process is at https://wiki.openoffice.org/wiki/Localization_for_developers An Italian build of trunk looks quite broken. For example: create a new blank Impress
Re: Starting the 4.2.0 localization
> Impress document and right-click on the slide you will see the > context menu in English. I can confirm with French version too This problem affects all contextual menus for Impress and Draw. - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Starting the 4.2.0 localization
Pedro Lino wrote: Hi Andrea, all Impress document and right-click on the slide you will see the context menu in English. Problem confirmed with Portuguese build Thanks for checking. And actually, as I wrote, I wouldn't consider this a problem. Code refactoring caused some strings to be moved and get a different identifier in trunk, so the translations for 4.1.5 cannot be reused as they are. Of course, it becomes a problem if the Pootle strings, imported in the AOO415 branch, still cause the issue; and this is the next test. Apparently AOO 420 is 100% translated to Portuguese on Pootle so it might be a good test Language. Ah yes, this is a detail I forgot to write. I've repeatedly written in the past that Pootle had the trunk strings and that meant that we couldn't import them to any 4.1.x version (due to the issue above). Well, this was correct but only in part. The last import of the trunk strings into Pootle was done years ago, so (contrary to my expectations) Pootle is probably much closer to 4.1.x than to trunk. In other words, if a language is 100% translated in Pootle now, we can probably use the translation as-is for a 4.1.x release, but not for a trunk (4.2.0) release; exactly the opposite of what I expected. If you can do the build process I would gladly volunteer to check any binaries for untranslated menus/interfaces (after New Year obviously ;) ). I can easily provide Linux builds. But if the above is confirmed by the next tests, then we should probably create a temporary branch named "AOO41L10N" or similar and point the current AOO415 buildbots (now useless) to it. On that branch, we can import strings from Pootle and let teams check that the import is correct. Then we can do the real work, i.e., port this to trunk (4.2.0) and delete the temporary branch. More information after the next tests. Regards, Andrea. - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Starting the 4.2.0 localization
Hi Andrea, all - The problem (not a problem actually; just the need to translate some hundreds strings again) I had empirically found for Italian indeed affects all languages. If you download a non-English build from https://ci.apache.org/projects/openoffice/install/linux64/ or https://ci.apache.org/projects/openoffice/install/win/ create a new Impress document and right-click on the slide you will see the context menu in English. Example for German: https://s13.postimg.org/4acjbnr3b/screen.png Problem confirmed with Portuguese build (AOO420m1(Build:9800) - Rev. 1819451 from BuildBot but also with Matthias' latest build AOO420m1(Build:9800) - Rev. 1819072) under Windows Next tests (for me or for anyone else who wants to try) would be: 1. Run the same process (2 and 3) on AOO415 and see if the match is perfect there (no warnings) 2. Build both AOO415 and trunk with the new SDF file and see if translations from Pootle are picked up properly. Apparently AOO 420 is 100% translated to Portuguese on Pootle so it might be a good test Language. If you can do the build process I would gladly volunteer to check any binaries for untranslated menus/interfaces (after New Year obviously ;) ). Regards, Pedro - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Starting the 4.2.0 localization
Some further notes (then someone else may want to reorganize this, but for the moment I'm just collecting them): - The problem (not a problem actually; just the need to translate some hundreds strings again) I had empirically found for Italian indeed affects all languages. If you download a non-English build from https://ci.apache.org/projects/openoffice/install/linux64/ or https://ci.apache.org/projects/openoffice/install/win/ create a new Impress document and right-click on the slide you will see the context menu in English. Example for German: https://s13.postimg.org/4acjbnr3b/screen.png - Of course, any manual SDF file changes done in the past will be lost. We can at most create a diff for people to review. But it's clear that when somebody edits a file that contains a 10 lines tall "DO NOT EDIT" notice (and I did it too! but it was a Pootle backport, as in many other cases) then he should accept the risk. - Due to the manual fixes above, SDF files in the AOO415 branch are at times different from those in trunk. We can ignore this: manual changes will be lost anyway, and people who edited the "DO NOT EDIT" files knew what they were doing and will redo the work properly if needed. - I've started testing the other part, since in the end this is what we must do first: importing translations from Pootle to SDF files, in order to preserve as much as possible what translators did. This is a heavily manual process. The basic steps work: 1. Export PO files from Pootle (for this first test I used the UI, but one can use the command-line too); both UI and help; then expand in the same directory "it" 2. Generate the en-US template: in main/, "localize -e -l en-US -f /path/to/en-US.sdf" 3. Generate the new SDF file for Italian: po2oo -l it -t en-US.sdf --keeptimestamp --skipsource it new_it.sdf This produces a bunch of warnings. I think most of them are due to the fact that the strings in the AOO415 branch and trunk are simply different. Next tests (for me or for anyone else who wants to try) would be: 1. Run the same process (2 and 3) on AOO415 and see if the match is perfect there (no warnings) 2. Build both AOO415 and trunk with the new SDF file and see if translations from Pootle are picked up properly. Regards, Andrea. Il 28/12/2017 00:39, Andrea Pescetti ha scritto: Just a few notes on what I've found so far about the 4.2.0 localization, in case others (like Keith, who wrote he is interested) want to follow and/or proceed. Of course, you will need: 1. A full checkout of trunk; I believe some of the tools are built during the build process, but I'm using a fully built tree anyway. 2. Good knowledge of another language than English (I use Italian). The wiki contains instructions for several generations of tools (past, present and even future - but let's fix the current tools before passing to the new ones that were promising but were never used in production). The current process is at https://wiki.openoffice.org/wiki/Localization_for_developers An Italian build of trunk looks quite broken. For example: create a new blank Impress document. Right-click anywhere on the slide and you'll see that the context menu is mostly in English (Slide, Insert Snap Point/Line). Is it broken in other languages too? This menu is of course translated in 4.1.5. Note that this is normal. Code rearrangements result in changes of string identifiers, and it already happened that some code changes resulted in the need to re-translate parts that had already been translated. The rest looks as expected. 4.1.5 has COUNTIFS in the formula list, but no help for it; trunk has a help page for it, if you search COUNTIFS (in English). Sp the first step is: regenerate SDF files to replace those in http://svn.apache.org/viewvc/openoffice/trunk/extras/l10n/source/ (then we will have to see what to replace and how, since we'll want to preserve work done in Pootle so far if possible). To this aim, I've run (from within main/ ) $ localize -e -l it=en-US -f /.../my.sdf and compared this to the Italian file we have in trunk. This results in about 1000 strings (counting both the added and removed ones), but it is quite in agreement with what I see in the build. To count the 1000 strings, I stripped away the timestamp from files (otherwise of course all lines would have been different) and sorted both of them. It would be helpful to know if other people are seeing the same in other languages. What I'm seeing now looks reasonable. For example, the "new" SDF file contains the Impress strings that appear in English and the help for the COUNTIFS function. Regards, Andrea. - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Starting the 4.2.0 localization
Just a few notes on what I've found so far about the 4.2.0 localization, in case others (like Keith, who wrote he is interested) want to follow and/or proceed. Of course, you will need: 1. A full checkout of trunk; I believe some of the tools are built during the build process, but I'm using a fully built tree anyway. 2. Good knowledge of another language than English (I use Italian). The wiki contains instructions for several generations of tools (past, present and even future - but let's fix the current tools before passing to the new ones that were promising but were never used in production). The current process is at https://wiki.openoffice.org/wiki/Localization_for_developers An Italian build of trunk looks quite broken. For example: create a new blank Impress document. Right-click anywhere on the slide and you'll see that the context menu is mostly in English (Slide, Insert Snap Point/Line). Is it broken in other languages too? This menu is of course translated in 4.1.5. Note that this is normal. Code rearrangements result in changes of string identifiers, and it already happened that some code changes resulted in the need to re-translate parts that had already been translated. The rest looks as expected. 4.1.5 has COUNTIFS in the formula list, but no help for it; trunk has a help page for it, if you search COUNTIFS (in English). Sp the first step is: regenerate SDF files to replace those in http://svn.apache.org/viewvc/openoffice/trunk/extras/l10n/source/ (then we will have to see what to replace and how, since we'll want to preserve work done in Pootle so far if possible). To this aim, I've run (from within main/ ) $ localize -e -l it=en-US -f /.../my.sdf and compared this to the Italian file we have in trunk. This results in about 1000 strings (counting both the added and removed ones), but it is quite in agreement with what I see in the build. To count the 1000 strings, I stripped away the timestamp from files (otherwise of course all lines would have been different) and sorted both of them. It would be helpful to know if other people are seeing the same in other languages. What I'm seeing now looks reasonable. For example, the "new" SDF file contains the Impress strings that appear in English and the help for the COUNTIFS function. Regards, Andrea. - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org