Re: [translate-pootle] Pootle version control problems with distributed version control systems
Hi everyone I am so rude sometimes. Welcome Asheesh :). From Lars's response: I just wanted to point out, that during step (3) the file file_2.po is also merged with the current state in the VCS upstream repository. I am not sure, if that was important for Wynand's example. That is really the problem. If other files are merged during an update, one is likely to end up with corrupted translation files, since any local changes will cause conflicts. ** I wasn't really clear with the problem I explained. With git, things go wrong if you modify files in your Pootle project which have also been modified in the source repository. You can see the problem below: [EMAIL PROTECTED]:~/workspace/test_area/poupdate/source_repo$ vi abc.po [EMAIL PROTECTED]:~/workspace/test_area/poupdate/source_repo$ git commit -a Created commit 69c409c: adw 1 files changed, 1 insertions(+), 1 deletions(-) [EMAIL PROTECTED]:~/workspace/test_area/poupdate/source_repo$ cd .. [EMAIL PROTECTED]:~/workspace/test_area/poupdate$ cd test_repo/ [EMAIL PROTECTED]:~/workspace/test_area/poupdate/test_repo$ vi abc.po [EMAIL PROTECTED]:~/workspace/test_area/poupdate/test_repo$ git pull remote: Counting objects: 5, done. remote: Compressing objects: 100% (3/3), done. Unpacking objects: 100% (3/3), done. remote: Total 3 (delta 1), reused 0 (delta 0) From /home/wynand/workspace/test_area/poupdate/source_repo// b219733..69c409c master - origin/master Updating b219733..69c409c abc.po: needs update error: Entry 'abc.po' not uptodate. Cannot merge. 1. Here I edited abc.po in the source repository and committed the change. 2. Then I edited abc.po in Pootle's repository. 3. Then I tried to pull changes. 4. git complained. ** Thanks for the feedback Lars Asheesh. I think we'll implement the proposed solution. I hope it won't take too much time to make it work. Cheers Wynand begin:vcard fn:Wynand Winterbach n:Winterbach;Wynand org:Translate.org.za adr:Groenkloof;;63A Wenning Street;Pretoria;Gauteng;0181;South Africa email;internet:[EMAIL PROTECTED] title:Associate tel;work:+27 12 460 1095 tel;fax:+27 12 460 1095 tel;cell:+27 84 268 5340 x-mozilla-html:FALSE url:http://www.translate.org.za version:2.1 end:vcard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
On Thu, 2008-11-13 at 03:21 +0100, Lars Kruse wrote: snip Not that I've seen. I'd appreciate more examples in this discussion, like the one you gave that shows it doesn't seem to be a problem. Furthermore, this issue isn't strictly related to DVCSs, I think. Doesn't conflict2suggest.py handle this in the general case? I lost you here: could you point me to this conflict2suggest.py? conflict2suggest changes PO conflicts i.e. #-#-#-#-# entries into suggestions. It won't resolve VC conflicts I'm afraid. -- Dwayne Bailey Associate +27 12 460 1095 (w) Translate.org.za +27 83 443 7114 (c) Recent blog posts: * FIrefox locale language landing pages http://www.translate.org.za/blogs/dwayne/en/content/firefox-locale-language-landing-pages * Virtaal on Fedora: not just yet, but soon'ish * Fennec in Afrikaans Stop Digital Apartheid! - http://www.digitalapartheid.com Firefox web browser in Afrikaans - http://af.www.mozilla.com/af/ African Network for Localisation (ANLoc) - http://africanlocalisation.net/ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
On Wed, 2008-11-12 at 17:31 -0800, Asheesh Laroia wrote: snip (BTW: Hi all, I'm new to the list. I'm working on internationalization at Creative Commons and spending a lot of time lately talking to Wil Clouser of Mozilla on #pootle, where my nick is paulproteus.) Great to see you here! Welcome aboard. -- Dwayne Bailey Associate +27 12 460 1095 (w) Translate.org.za +27 83 443 7114 (c) Recent blog posts: * FIrefox locale language landing pages http://www.translate.org.za/blogs/dwayne/en/content/firefox-locale-language-landing-pages * Virtaal on Fedora: not just yet, but soon'ish * Fennec in Afrikaans Stop Digital Apartheid! - http://www.digitalapartheid.com Firefox web browser in Afrikaans - http://af.www.mozilla.com/af/ African Network for Localisation (ANLoc) - http://africanlocalisation.net/ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
Hi, From Lars's response: I just wanted to point out, that during step (3) the file file_2.po is also merged with the current state in the VCS upstream repository. I am not sure, if that was important for Wynand's example. That is really the problem. If other files are merged during an update, one is likely to end up with corrupted translation files, since any local changes will cause conflicts. For now there is no locking during merge/update, thus conflicts can arise due to a bad timing. See more below. I wasn't really clear with the problem I explained. With git, things go wrong if you modify files in your Pootle project which have also been modified in the source repository. You can see the problem below: [EMAIL PROTECTED]:~/workspace/test_area/poupdate/source_repo$ vi abc.po [EMAIL PROTECTED]:~/workspace/test_area/poupdate/source_repo$ git commit -a Created commit 69c409c: adw 1 files changed, 1 insertions(+), 1 deletions(-) [EMAIL PROTECTED]:~/workspace/test_area/poupdate/source_repo$ cd .. [EMAIL PROTECTED]:~/workspace/test_area/poupdate$ cd test_repo/ [EMAIL PROTECTED]:~/workspace/test_area/poupdate/test_repo$ vi abc.po [EMAIL PROTECTED]:~/workspace/test_area/poupdate/test_repo$ git pull remote: Counting objects: 5, done. remote: Compressing objects: 100% (3/3), done. Unpacking objects: 100% (3/3), done. remote: Total 3 (delta 1), reused 0 (delta 0) From /home/wynand/workspace/test_area/poupdate/source_repo// b219733..69c409c master - origin/master Updating b219733..69c409c abc.po: needs update error: Entry 'abc.po' not uptodate. Cannot merge. 1. Here I edited abc.po in the source repository and committed the change. 2. Then I edited abc.po in Pootle's repository. 3. Then I tried to pull changes. 4. git complained. Thanks for explaining! The current code in translate/storage/versioncontrol/ does the following during an update: 1) git checkout FILENAME 2) git pull REPO_ROOT_DIR Step (1) reverts local changes. Local modifications are preserved in pootle's memory at this moment. They get merged after step (2). Pootle would do the following for a commit: 1) git add FILENAME 2) git push Your example shows, that it is necessary to update before commit. The same issue is valid for subversion and probably also for the other VCS. Maybe we should change pootle's code to do an update before commit by default? Did I get your point now? Thanks for the feedback Lars Asheesh. I think we'll implement the proposed solution. I hope it won't take too much time to make it work. this would be a great step forward! greetings, Lars - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
Your example shows, that it is necessary to update before commit. The same issue is valid for subversion and probably also for the other VCS. Maybe we should change pootle's code to do an update before commit by default? Did I get your point now? Hey, you really wanted to say another thing, didn't you? If Pootle does as you say all the last work will be lost!! IMHO the things are that way: Suppose you have the version 59 of the file translation.po. The translation.po changes in the vcs, and before you update the local copy with the newest 60 version (created after synchronize the translation.po with a new .pot file), Pootle makes changes to the local 59 copy, so for Pootle this is the new 60 version that it is not the same as the vcs one. So when Pootle tries to commit (in git is pull, isn't it?) the translation.po it fails, because the two files (vcs one and Pootle one) have the same version number, the 60. And of course, before a local update (syncronize with the vcs version), you have to do a commit to save into the vcs the latest changes made in Pootle. If you have a .pot file, there is no problem. If first you make a backup of the local modified translation.po (local 60 version bis), and then you update the local copy downloading the newest .pot file from the vcs, and also the version 60 of translation.po replacing the local 60 version (this should be a way of forcing doing this, but I only know a few commands of svn). Then you could move the backup (local 60 version bis) replacing the downloaded translation.po, making it the new 61 version of translation.po, and finally you locally merge it with your latest .pot file (the vcs one). Finally you only have to do a commit to save the changes in the vcs. But sometimes you don't have any .pot file, so we have a great problem here. The only solution in this case is to use a Translation memory, that saves all the translations made by the Pootle users. Then when updating, if the file is updated (forcing the update, because the two files are 60 version) with a newest version from the vcs, Pootle should run an automatic translation with exact matching using the translations saved in the database in order to try to maintain the translation level of the file. Just after that Pootle should make a commit (or a pull in git), to save the changes in the vcs. I think this is the only way. I hope this will be helpful. Ah. And of course, this things should be automated to avoid human intervention. Another thing, is there any way to making translation memories in Pootle, and importing and exporting them as TMX files?? Bye, Leandro Regueiro - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
Hi, Your example shows, that it is necessary to update before commit. The same issue is valid for subversion and probably also for the other VCS. Maybe we should change pootle's code to do an update before commit by default? Did I get your point now? Hey, you really wanted to say another thing, didn't you? If Pootle does as you say all the last work will be lost!! I guess, we misunderstand each other here. See below. IMHO the things are that way: Suppose you have the version 59 of the file translation.po. The translation.po changes in the vcs, and before you update the local copy with the newest 60 version (created after synchronize the translation.po with a new .pot file), Pootle makes changes to the local 59 copy, so for Pootle this is the new 60 version that it is not the same as the vcs one. So when Pootle tries to commit (in git is pull, isn't it?) ... from my understanding, you mean push. ... the translation.po it fails, because the two files (vcs one and Pootle one) have the same version number, the 60. correct. That's why I proposed, that the update should happen before commit/pull. Otherwise any VCS will refuse to commit an outdated local copy. And of course, before a local update (syncronize with the vcs version), you have to do a commit to save into the vcs the latest changes made in Pootle. as I stated before, an update (in the way it is currently implemented in the translate toolkit) will do this: 1) store the current state of the file in memory 2) revert local changes of the file 3) update the file from the repository 4) merge the previously stored content of the original local file with the new file, that came from the repository This procedure allows us to do the merging with regard to translation units instead of trusting the line-based merging of the respective VCS. See Pootle/projects.py (line 542) and Pootle/pootlefile.py (line 646) for details. I do not see the data loss here, that you mentioned in the beginning. Could you elaborate, please? If you have a .pot file, there is no problem. If first you make a backup of the local modified translation.po (local 60 version bis), and then you update the local copy downloading the newest .pot file from the vcs, and also the version 60 of translation.po replacing the local 60 version (this should be a way of forcing doing this, but I only know a few commands of svn). Then you could move the backup (local 60 version bis) replacing the downloaded translation.po, making it the new 61 version of translation.po, and finally you locally merge it with your latest .pot file (the vcs one). Finally you only have to do a commit to save the changes in the vcs. po files and pot files are currently updated separately. Assuming, that both the po and pot file in the repository are in sync, this should not cause problems. Could you provide a specific recipe to reproduce a potential problem, please? But sometimes you don't have any .pot file, so we have a great problem here. The only solution in this case is to use a Translation memory, that saves all the translations made by the Pootle users. Then when updating, if the file is updated (forcing the update, because the two files are 60 version) with a newest version from the vcs, Pootle should run an automatic translation with exact matching using the translations saved in the database in order to try to maintain the translation level of the file. Just after that Pootle should make a commit (or a pull in git), to save the changes in the vcs. I think, this basically matches the current implementation, that I described above as step 1 to 4. Or am I wrong? I think this is the only way. I hope this will be helpful. Ah. And of course, this things should be automated to avoid human intervention. Yes, I agree, that Pootle should always do an update before commit to avoid the problem, that you described in the beginning of this mail (caused by the outdated local file). Another thing, is there any way to making translation memories in Pootle, and importing and exporting them as TMX files?? This is beyond my knowledge. I guess, someone else will answer this question ... greetings, Lars - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
