Re: [translate-pootle] Pootle version control problems with distributed version control systems
On Mon, Nov 24, 2008 at 10:26:54PM -0600, David Fraser [EMAIL PROTECTED] wrote: Surely the merge has to be done anyway, whether it's done by Pootle or the DVCS? Anyway just thinking aloud... See my other mail in this thread, it was a long one where I described what functionality would I need to write a merge driver for po files. I'm sure Pootle supports merging of po files at some level (ie. the functionality is there, but I have no idea how to use it from commadline). pgpRFp0EkDwmm.pgp Description: PGP signature - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
- Miklos Vajna [EMAIL PROTECTED] wrote: On Thu, Nov 13, 2008 at 11:03:38AM +0200, Wynand Winterbach [EMAIL PROTECTED] wrote: Updating b219733..69c409c abc.po: needs update error: Entry 'abc.po' not uptodate. Cannot merge. There is an easy solution for this: ask your users to commit before they update. The problem is that git expects that local changes are committed or dropped away before you pull, usually. This is a late drop-in to the thread because it got me thinking... Basically the point of distributed version control is that you can easily handle branching and merging in multiple repositories. The ideal way to do this would be to regard: * The current edited Pootle repository as a branch (off of the last update from an upstream server) * The users downloaded files as a branch (from the Pootle branch, as of when the download took place) Then basically the version control system should know how to merge the above. The trick is of course to keep the version control info with the download/upload, and this may well make it so tricky that it doesn't get done. Problems include: * In git, for example, the version control info happens to be the whole repository by default * You can't easily separate this out for subdirectories * The users probably aren't using the version control system to commit their changes (though Pootle could start committing changes locally - this would be a good idea TM) * You'd need to make sure that the right information was included in the download * You'd also need to make sure that the merging was done in a way that didn't interfere with the operation of the server (i.e. it would be nice to do it in a separate directory to the live Pootle one, but checked out from the same repository) Anyway given the insanity of the above I'm not sure it would work but throwing it out there for fun. Incidentally if some of the above are easier in another DVCS than git then it would be an incentive for people to use that to manage their translation data... Cheers David - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
On Mon, Nov 24, 2008 at 08:27:30AM -0600, David Fraser [EMAIL PROTECTED] wrote: Anyway given the insanity of the above I'm not sure it would work but throwing it out there for fun. Incidentally if some of the above are easier in another DVCS than git then it would be an incentive for people to use that to manage their translation data... Given that these require merging and merging po files is not supported (AFAIK) by any DVCS, and given that Git has support for custom merge drivers, I think that would be the easier. But still, there is no merge driver for po files at the moment, so what you outlined is not yet possible IMHO. pgpaVspqspDIx.pgp Description: PGP signature - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
On Thu, Nov 13, 2008 at 08:39:52AM +0200, F Wolff [EMAIL PROTECTED] wrote: conflict2suggest.py is really for handling conflicts caused by msgmerge where the units will contain #-#-#-#-# markers with different conflicting translations all indicated in one translations. This is really unrelated to version control conflicts where the file format is (possibly) broken by an update clashing with local changes. This is interesting. Git supports different merge drivers, so technically it would be possible to handle the following scenario: There is version 'A' on the server. Bob clones the repo, changes string foo to bar, and of course that will update the last modified line in the po file as well. Alice has the server version of the po file on the Pootle server and changes baz to blah (+ the last modified line, of course). Bob does a git push (~ svn commit). Alice can't push, as first she has to merge the server version (it's like when you have to svn up before svn commit). At this point all the VCSes will generate a conflict ATM because the Last modified line is modified from A to 1) B (Bob) and 2) C (Alice), which is a conflict. The nice thing is that with git, it would be possible to handle this scenario. The only requirement is a script that takes 3 parameters: the original version, Bob's one and Alice's one. (3 file paths.) The result is written to Bob's version. Now my question: I know git, but I don't know too much about translation stuff (I'm just a user in the translation world), but I suspect that such a script is fairly easy to do, provided that what tools are available already for po files. :) Can someone please help me what tool could be used to do something like this? The only tool I know myself is msgmerge, but that takes a *template* and the old po file (resulting in an updated po file), not 3 po files. Possibly in case of merging po files, you don't need the original version at all, I guess. A nice thing would be to make difference in case really just trivial stuff conflicts (which is not a real conflict in the po file, like the last modified line) and when we really have a conflict like foo (original version) is changed to bar (Bob) and baz (Alice). Thanks! pgpYfzKnuVOfX.pgp Description: PGP signature - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
On Thu, Nov 13, 2008 at 11:03:38AM +0200, Wynand Winterbach [EMAIL PROTECTED] wrote: Updating b219733..69c409c abc.po: needs update error: Entry 'abc.po' not uptodate. Cannot merge. There is an easy solution for this: ask your users to commit before they update. The problem is that git expects that local changes are committed or dropped away before you pull, usually. pgpQXudrtVSVe.pgp Description: PGP signature - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
[also shortening the reply a bit ...] But I have to point that keeping the local file with changes in the memory could be a bad idea if the computer or server that hosts Pootle unexpectedly loss the energy supply. You are right, this feels a little bit scary. But right now I can't think of a simple way how to avoid this potential problem. I guess, it would involve maintaining a kind of work-in-progress queue - which feels a little bit like overkill considering the small probability of this kind of failure. Any implementation ideas are warmly welcome, of course! This could be storing the file in the hard disk and not in the memory and deleting it after the operation. There is no chance to avoid this, a copy must be saved during the operation to avoid the loss of data. Bye, Leandro Regueiro - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
Your example shows, that it is necessary to update before commit. The same issue is valid for subversion and probably also for the other VCS. Maybe we should change pootle's code to do an update before commit by default? Did I get your point now? Hey, you really wanted to say another thing, didn't you? If Pootle does as you say all the last work will be lost!! I guess, we misunderstand each other here. See below. Yes, you are right. My apologies. IMHO the things are that way: Suppose you have the version 59 of the file translation.po. The translation.po changes in the vcs, and before you update the local copy with the newest 60 version (created after synchronize the translation.po with a new .pot file), Pootle makes changes to the local 59 copy, so for Pootle this is the new 60 version that it is not the same as the vcs one. So when Pootle tries to commit (in git is pull, isn't it?) ... from my understanding, you mean push. Sorry. I also change pull and push. ... the translation.po it fails, because the two files (vcs one and Pootle one) have the same version number, the 60. correct. That's why I proposed, that the update should happen before commit/pull. Otherwise any VCS will refuse to commit an outdated local copy. Here is the problem. I understand that with update you were talking about the vcs action, and not about updating from POT (I usually call this merging with POT). My fault. And of course, before a local update (syncronize with the vcs version), you have to do a commit to save into the vcs the latest changes made in Pootle. as I stated before, an update (in the way it is currently implemented in the translate toolkit) will do this: 1) store the current state of the file in memory 2) revert local changes of the file 3) update the file from the repository 4) merge the previously stored content of the original local file with the new file, that came from the repository This procedure allows us to do the merging with regard to translation units instead of trusting the line-based merging of the respective VCS. See Pootle/projects.py (line 542) and Pootle/pootlefile.py (line 646) for details. I do not see the data loss here, that you mentioned in the beginning. Could you elaborate, please? As I said above I misunderstand you because of the word update. Sorry. But I have to point that keeping the local file with changes in the memory could be a bad idea if the computer or server that hosts Pootle unexpectedly loss the energy supply. If you have a .pot file, there is no problem. If first you make a backup of the local modified translation.po (local 60 version bis), and then you update the local copy downloading the newest .pot file from the vcs, and also the version 60 of translation.po replacing the local 60 version (this should be a way of forcing doing this, but I only know a few commands of svn). Then you could move the backup (local 60 version bis) replacing the downloaded translation.po, making it the new 61 version of translation.po, and finally you locally merge it with your latest .pot file (the vcs one). Finally you only have to do a commit to save the changes in the vcs. po files and pot files are currently updated separately. Assuming, that both the po and pot file in the repository are in sync, this should not cause problems. Could you provide a specific recipe to reproduce a potential problem, please? I don't understand this question. I think I have considered all the potential problems here. But sometimes you don't have any .pot file, so we have a great problem here. The only solution in this case is to use a Translation memory, that saves all the translations made by the Pootle users. Then when updating, if the file is updated (forcing the update, because the two files are 60 version) with a newest version from the vcs, Pootle should run an automatic translation with exact matching using the translations saved in the database in order to try to maintain the translation level of the file. Just after that Pootle should make a commit (or a pull in git), to save the changes in the vcs. I think, this basically matches the current implementation, that I described above as step 1 to 4. Or am I wrong? No, you are right, except for maintaining the local file with changes in memory. I think this is the only way. I hope this will be helpful. Ah. And of course, this things should be automated to avoid human intervention. Yes, I agree, that Pootle should always do an update before commit to avoid the problem, that you described in the beginning of this mail (caused by the outdated local file). Yes. Another thing, is there any way to making translation memories in Pootle, and importing and exporting them as TMX files?? This is beyond my knowledge. I guess, someone else will answer this question ... I've got an answer that not really satisfies me. Bye, Leandro
Re: [translate-pootle] Pootle version control problems with distributed version control systems
Hi, [shortening the reply a bit ...] Assuming, that both the po and pot file in the repository are in sync, this should not cause problems. Could you provide a specific recipe to reproduce a potential problem, please? I don't understand this question. I think I have considered all the potential problems here. My question was based on our misunderstanding. Thus it became obsolete. ... But I have to point that keeping the local file with changes in the memory could be a bad idea if the computer or server that hosts Pootle unexpectedly loss the energy supply. You are right, this feels a little bit scary. But right now I can't think of a simple way how to avoid this potential problem. I guess, it would involve maintaining a kind of work-in-progress queue - which feels a little bit like overkill considering the small probability of this kind of failure. Any implementation ideas are warmly welcome, of course! thanks for your comments! Lars - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
Hi everyone I am so rude sometimes. Welcome Asheesh :). From Lars's response: I just wanted to point out, that during step (3) the file file_2.po is also merged with the current state in the VCS upstream repository. I am not sure, if that was important for Wynand's example. That is really the problem. If other files are merged during an update, one is likely to end up with corrupted translation files, since any local changes will cause conflicts. ** I wasn't really clear with the problem I explained. With git, things go wrong if you modify files in your Pootle project which have also been modified in the source repository. You can see the problem below: [EMAIL PROTECTED]:~/workspace/test_area/poupdate/source_repo$ vi abc.po [EMAIL PROTECTED]:~/workspace/test_area/poupdate/source_repo$ git commit -a Created commit 69c409c: adw 1 files changed, 1 insertions(+), 1 deletions(-) [EMAIL PROTECTED]:~/workspace/test_area/poupdate/source_repo$ cd .. [EMAIL PROTECTED]:~/workspace/test_area/poupdate$ cd test_repo/ [EMAIL PROTECTED]:~/workspace/test_area/poupdate/test_repo$ vi abc.po [EMAIL PROTECTED]:~/workspace/test_area/poupdate/test_repo$ git pull remote: Counting objects: 5, done. remote: Compressing objects: 100% (3/3), done. Unpacking objects: 100% (3/3), done. remote: Total 3 (delta 1), reused 0 (delta 0) From /home/wynand/workspace/test_area/poupdate/source_repo// b219733..69c409c master - origin/master Updating b219733..69c409c abc.po: needs update error: Entry 'abc.po' not uptodate. Cannot merge. 1. Here I edited abc.po in the source repository and committed the change. 2. Then I edited abc.po in Pootle's repository. 3. Then I tried to pull changes. 4. git complained. ** Thanks for the feedback Lars Asheesh. I think we'll implement the proposed solution. I hope it won't take too much time to make it work. Cheers Wynand begin:vcard fn:Wynand Winterbach n:Winterbach;Wynand org:Translate.org.za adr:Groenkloof;;63A Wenning Street;Pretoria;Gauteng;0181;South Africa email;internet:[EMAIL PROTECTED] title:Associate tel;work:+27 12 460 1095 tel;fax:+27 12 460 1095 tel;cell:+27 84 268 5340 x-mozilla-html:FALSE url:http://www.translate.org.za version:2.1 end:vcard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
On Thu, 2008-11-13 at 03:21 +0100, Lars Kruse wrote: snip Not that I've seen. I'd appreciate more examples in this discussion, like the one you gave that shows it doesn't seem to be a problem. Furthermore, this issue isn't strictly related to DVCSs, I think. Doesn't conflict2suggest.py handle this in the general case? I lost you here: could you point me to this conflict2suggest.py? conflict2suggest changes PO conflicts i.e. #-#-#-#-# entries into suggestions. It won't resolve VC conflicts I'm afraid. -- Dwayne Bailey Associate +27 12 460 1095 (w) Translate.org.za +27 83 443 7114 (c) Recent blog posts: * FIrefox locale language landing pages http://www.translate.org.za/blogs/dwayne/en/content/firefox-locale-language-landing-pages * Virtaal on Fedora: not just yet, but soon'ish * Fennec in Afrikaans Stop Digital Apartheid! - http://www.digitalapartheid.com Firefox web browser in Afrikaans - http://af.www.mozilla.com/af/ African Network for Localisation (ANLoc) - http://africanlocalisation.net/ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
On Wed, 2008-11-12 at 17:31 -0800, Asheesh Laroia wrote: snip (BTW: Hi all, I'm new to the list. I'm working on internationalization at Creative Commons and spending a lot of time lately talking to Wil Clouser of Mozilla on #pootle, where my nick is paulproteus.) Great to see you here! Welcome aboard. -- Dwayne Bailey Associate +27 12 460 1095 (w) Translate.org.za +27 83 443 7114 (c) Recent blog posts: * FIrefox locale language landing pages http://www.translate.org.za/blogs/dwayne/en/content/firefox-locale-language-landing-pages * Virtaal on Fedora: not just yet, but soon'ish * Fennec in Afrikaans Stop Digital Apartheid! - http://www.digitalapartheid.com Firefox web browser in Afrikaans - http://af.www.mozilla.com/af/ African Network for Localisation (ANLoc) - http://africanlocalisation.net/ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
Hi, From Lars's response: I just wanted to point out, that during step (3) the file file_2.po is also merged with the current state in the VCS upstream repository. I am not sure, if that was important for Wynand's example. That is really the problem. If other files are merged during an update, one is likely to end up with corrupted translation files, since any local changes will cause conflicts. For now there is no locking during merge/update, thus conflicts can arise due to a bad timing. See more below. I wasn't really clear with the problem I explained. With git, things go wrong if you modify files in your Pootle project which have also been modified in the source repository. You can see the problem below: [EMAIL PROTECTED]:~/workspace/test_area/poupdate/source_repo$ vi abc.po [EMAIL PROTECTED]:~/workspace/test_area/poupdate/source_repo$ git commit -a Created commit 69c409c: adw 1 files changed, 1 insertions(+), 1 deletions(-) [EMAIL PROTECTED]:~/workspace/test_area/poupdate/source_repo$ cd .. [EMAIL PROTECTED]:~/workspace/test_area/poupdate$ cd test_repo/ [EMAIL PROTECTED]:~/workspace/test_area/poupdate/test_repo$ vi abc.po [EMAIL PROTECTED]:~/workspace/test_area/poupdate/test_repo$ git pull remote: Counting objects: 5, done. remote: Compressing objects: 100% (3/3), done. Unpacking objects: 100% (3/3), done. remote: Total 3 (delta 1), reused 0 (delta 0) From /home/wynand/workspace/test_area/poupdate/source_repo// b219733..69c409c master - origin/master Updating b219733..69c409c abc.po: needs update error: Entry 'abc.po' not uptodate. Cannot merge. 1. Here I edited abc.po in the source repository and committed the change. 2. Then I edited abc.po in Pootle's repository. 3. Then I tried to pull changes. 4. git complained. Thanks for explaining! The current code in translate/storage/versioncontrol/ does the following during an update: 1) git checkout FILENAME 2) git pull REPO_ROOT_DIR Step (1) reverts local changes. Local modifications are preserved in pootle's memory at this moment. They get merged after step (2). Pootle would do the following for a commit: 1) git add FILENAME 2) git push Your example shows, that it is necessary to update before commit. The same issue is valid for subversion and probably also for the other VCS. Maybe we should change pootle's code to do an update before commit by default? Did I get your point now? Thanks for the feedback Lars Asheesh. I think we'll implement the proposed solution. I hope it won't take too much time to make it work. this would be a great step forward! greetings, Lars - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
Your example shows, that it is necessary to update before commit. The same issue is valid for subversion and probably also for the other VCS. Maybe we should change pootle's code to do an update before commit by default? Did I get your point now? Hey, you really wanted to say another thing, didn't you? If Pootle does as you say all the last work will be lost!! IMHO the things are that way: Suppose you have the version 59 of the file translation.po. The translation.po changes in the vcs, and before you update the local copy with the newest 60 version (created after synchronize the translation.po with a new .pot file), Pootle makes changes to the local 59 copy, so for Pootle this is the new 60 version that it is not the same as the vcs one. So when Pootle tries to commit (in git is pull, isn't it?) the translation.po it fails, because the two files (vcs one and Pootle one) have the same version number, the 60. And of course, before a local update (syncronize with the vcs version), you have to do a commit to save into the vcs the latest changes made in Pootle. If you have a .pot file, there is no problem. If first you make a backup of the local modified translation.po (local 60 version bis), and then you update the local copy downloading the newest .pot file from the vcs, and also the version 60 of translation.po replacing the local 60 version (this should be a way of forcing doing this, but I only know a few commands of svn). Then you could move the backup (local 60 version bis) replacing the downloaded translation.po, making it the new 61 version of translation.po, and finally you locally merge it with your latest .pot file (the vcs one). Finally you only have to do a commit to save the changes in the vcs. But sometimes you don't have any .pot file, so we have a great problem here. The only solution in this case is to use a Translation memory, that saves all the translations made by the Pootle users. Then when updating, if the file is updated (forcing the update, because the two files are 60 version) with a newest version from the vcs, Pootle should run an automatic translation with exact matching using the translations saved in the database in order to try to maintain the translation level of the file. Just after that Pootle should make a commit (or a pull in git), to save the changes in the vcs. I think this is the only way. I hope this will be helpful. Ah. And of course, this things should be automated to avoid human intervention. Another thing, is there any way to making translation memories in Pootle, and importing and exporting them as TMX files?? Bye, Leandro Regueiro - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
Hi, Your example shows, that it is necessary to update before commit. The same issue is valid for subversion and probably also for the other VCS. Maybe we should change pootle's code to do an update before commit by default? Did I get your point now? Hey, you really wanted to say another thing, didn't you? If Pootle does as you say all the last work will be lost!! I guess, we misunderstand each other here. See below. IMHO the things are that way: Suppose you have the version 59 of the file translation.po. The translation.po changes in the vcs, and before you update the local copy with the newest 60 version (created after synchronize the translation.po with a new .pot file), Pootle makes changes to the local 59 copy, so for Pootle this is the new 60 version that it is not the same as the vcs one. So when Pootle tries to commit (in git is pull, isn't it?) ... from my understanding, you mean push. ... the translation.po it fails, because the two files (vcs one and Pootle one) have the same version number, the 60. correct. That's why I proposed, that the update should happen before commit/pull. Otherwise any VCS will refuse to commit an outdated local copy. And of course, before a local update (syncronize with the vcs version), you have to do a commit to save into the vcs the latest changes made in Pootle. as I stated before, an update (in the way it is currently implemented in the translate toolkit) will do this: 1) store the current state of the file in memory 2) revert local changes of the file 3) update the file from the repository 4) merge the previously stored content of the original local file with the new file, that came from the repository This procedure allows us to do the merging with regard to translation units instead of trusting the line-based merging of the respective VCS. See Pootle/projects.py (line 542) and Pootle/pootlefile.py (line 646) for details. I do not see the data loss here, that you mentioned in the beginning. Could you elaborate, please? If you have a .pot file, there is no problem. If first you make a backup of the local modified translation.po (local 60 version bis), and then you update the local copy downloading the newest .pot file from the vcs, and also the version 60 of translation.po replacing the local 60 version (this should be a way of forcing doing this, but I only know a few commands of svn). Then you could move the backup (local 60 version bis) replacing the downloaded translation.po, making it the new 61 version of translation.po, and finally you locally merge it with your latest .pot file (the vcs one). Finally you only have to do a commit to save the changes in the vcs. po files and pot files are currently updated separately. Assuming, that both the po and pot file in the repository are in sync, this should not cause problems. Could you provide a specific recipe to reproduce a potential problem, please? But sometimes you don't have any .pot file, so we have a great problem here. The only solution in this case is to use a Translation memory, that saves all the translations made by the Pootle users. Then when updating, if the file is updated (forcing the update, because the two files are 60 version) with a newest version from the vcs, Pootle should run an automatic translation with exact matching using the translations saved in the database in order to try to maintain the translation level of the file. Just after that Pootle should make a commit (or a pull in git), to save the changes in the vcs. I think, this basically matches the current implementation, that I described above as step 1 to 4. Or am I wrong? I think this is the only way. I hope this will be helpful. Ah. And of course, this things should be automated to avoid human intervention. Yes, I agree, that Pootle should always do an update before commit to avoid the problem, that you described in the beginning of this mail (caused by the outdated local file). Another thing, is there any way to making translation memories in Pootle, and importing and exporting them as TMX files?? This is beyond my knowledge. I guess, someone else will answer this question ... greetings, Lars - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
[translate-pootle] Pootle version control problems with distributed version control systems
Hi everyone (I'm including OOo here, because you might want to use Mercurial with Pootle) The version control system in Pootle does not work well with distributed version control systems (git, Mercurial, Bazaar, Darcs). We have a proposed solution and would like feedback. ** When a user updates a file from version control in Pootle, Pootle: 1. loads the file into memory (call this A), 2. reverts the file to the version that was checked out from the version control repository, 3. updates the file from the version control repository, 4. loads the updated file into memory (call this B), 5. merges A and B based on translation unit changes, 6. stores the resulting file. This system works for systems like CVS and Subversion, which allow individual files to be updated. ** Now consider what happens if we have two modified files - file_1.po and file_2.po - in our Pootle project. The user attempts to update file_1.po. Pootle: 1. loads file_1.po into memory, 2. reverts file_1.po to the version that was checked out from the version control repository, 3. updates file_1.po from the version control repository, 4. BREAKS This won't work, because file_2.po is modified; making it impossible to reliably pull new revisions from the version control repository. ** Our solution is to keep two project directories: 1. The Pootle project directory, 2. The version control project directory. In this way, Pootle will always be able to update the version control directory. It can also update individual files as the need arises. ** This solution seems wasteful, but there is a good reason: at some point, we *might* (don't take this as a commitment) want to store the projects in a database. Then it will be necessary to have external version control directories to do merging. ** Opinions? Alternative suggestions? Cheers Wynand P.S. Suresh, I'm looking at your patch; in fact, I realized the issue when looking at the version control code. begin:vcard fn:Wynand Winterbach n:Winterbach;Wynand org:Translate.org.za adr:Groenkloof;;63A Wenning Street;Pretoria;Gauteng;0181;South Africa email;internet:[EMAIL PROTECTED] title:Associate tel;work:+27 12 460 1095 tel;fax:+27 12 460 1095 tel;cell:+27 84 268 5340 x-mozilla-html:FALSE url:http://www.translate.org.za version:2.1 end:vcard - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
Hi Wynand, The version control system in Pootle does not work well with distributed version control systems (git, Mercurial, Bazaar, Darcs). Thanks for taking the time to analyze issues with the distributed version control systems! I did some part of the version control code, but I don't have any experience with a distributed VCS. I am glad, you step in, to fill this gap! [...] Now consider what happens if we have two modified files - file_1.po and file_2.po - in our Pootle project. The user attempts to update file_1.po. Pootle: 1. loads file_1.po into memory, 2. reverts file_1.po to the version that was checked out from the version control repository, 3. updates file_1.po from the version control repository, just a note: I think, for mercurial/git/bzr/darcs this causes an update of the whole working copy (not just a single file). 4. BREAKS This won't work, because file_2.po is modified; making it impossible to reliably pull new revisions from the version control repository. I am not sure, if I fully understand the problem. Do distributed VCS really have a problem to run an update, if the local working copy contains modified files? I tried the following here in an example mercurial repository: hg up -r 6142 [replace this with a non-current revision] echo foo SOMEFILE [this file should have changed since above revision] hg up As far as I noticed, this did not cause any unexpected behaviour. beware: I just assume, that I know, what I am doing :) Maybe I do not understand, what exactly you mean with breaks? Our solution is to keep two project directories: 1. The Pootle project directory, 2. The version control project directory. In this way, Pootle will always be able to update the version control directory. It can also update individual files as the need arises. This would solve the potential problem, that you described above. This solution seems wasteful, but there is a good reason: at some point, we *might* (don't take this as a commitment) want to store the projects in a database. Then it will be necessary to have external version control directories to do merging. sounds good to me. Opinions? Alternative suggestions? All in all, your approach should be the right way to allow pootle to connect files to an upstream repository without (console) intervention of the local system administrator (for the initial checkout and regular updates). It would - of course - require a good amount of development effort. But I consider this feature to be essential in the long term. However: it would be great, if you could explain the detail of the break, that you described, so that I can fully understand the problem. thanks for your ideas, Lars - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
On Thu, 13 Nov 2008, Lars Kruse wrote: Now consider what happens if we have two modified files - file_1.po and file_2.po - in our Pootle project. The user attempts to update file_1.po. Pootle: 1. loads file_1.po into memory, 2. reverts file_1.po to the version that was checked out from the version control repository, 3. updates file_1.po from the version control repository, just a note: I think, for mercurial/git/bzr/darcs this causes an update of the whole working copy (not just a single file). git checkout $file can reset a file to the last-in-git version on a per-file basis. I think hg revert and bzr revert do the same there. 4. BREAKS This won't work, because file_2.po is modified; making it impossible to reliably pull new revisions from the version control repository. I am not sure, if I fully understand the problem. Do distributed VCS really have a problem to run an update, if the local working copy contains modified files? Not that I've seen. I'd appreciate more examples in this discussion, like the one you gave that shows it doesn't seem to be a problem. Furthermore, this issue isn't strictly related to DVCSs, I think. Doesn't conflict2suggest.py handle this in the general case? (BTW: Hi all, I'm new to the list. I'm working on internationalization at Creative Commons and spending a lot of time lately talking to Wil Clouser of Mozilla on #pootle, where my nick is paulproteus.) -- Asheesh. -- Go ahead... make my day. -- Dirty Harry - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
Re: [translate-pootle] Pootle version control problems with distributed version control systems
Hi Asheesh, Now consider what happens if we have two modified files - file_1.po and file_2.po - in our Pootle project. The user attempts to update file_1.po. Pootle: 1. loads file_1.po into memory, 2. reverts file_1.po to the version that was checked out from the version control repository, 3. updates file_1.po from the version control repository, just a note: I think, for mercurial/git/bzr/darcs this causes an update of the whole working copy (not just a single file). git checkout $file can reset a file to the last-in-git version on a per-file basis. I think hg revert and bzr revert do the same there. ups - my comment was a little bit ambiguous. Yes - revert/checkout does a file-based reset. But for hg/bzr/darcs/git/cvs (i.e. all except svn) this revert is followed by a directory-based update in the current translate-toolkit code. I just wanted to point out, that during step (3) the file file_2.po is also merged with the current state in the VCS upstream repository. I am not sure, if that was important for Wynand's example. 4. BREAKS This won't work, because file_2.po is modified; making it impossible to reliably pull new revisions from the version control repository. I am not sure, if I fully understand the problem. Do distributed VCS really have a problem to run an update, if the local working copy contains modified files? Not that I've seen. I'd appreciate more examples in this discussion, like the one you gave that shows it doesn't seem to be a problem. Furthermore, this issue isn't strictly related to DVCSs, I think. Doesn't conflict2suggest.py handle this in the general case? I lost you here: could you point me to this conflict2suggest.py? regards, Lars - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Translate-pootle mailing list Translate-pootle@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/translate-pootle
