Re: Bug or undesired behavior when quota exceeded
On Thu, 2007-11-08 at 12:30 +0100, Janusz Jurski wrote: I have just noticed an undesired behavior when using rsync when disk quota is exceeded. 1) I normally execute two following commands to synchronize folders between a local host and a remote host: rsync -aCPvu --existing dir1 [EMAIL PROTECTED]:dir2 rsync -aCPvu --existing [EMAIL PROTECTED]:dir2 dir1 2) Assume that files are in-sync on the hosts. Then we change a file on the local host. When I execute the first rsync command, the file shall be copied to the remote host. When the second command is executed, no copying shall occur as the files are in sync after the first command. 3) Unfortunately, it may happen that the disk quota during the first command execution is exceeded. 5) When the second command is invoked, rsync starts copying the file from the remote host to the local one. This is an undesired behavior, as the file changed on the local host gets overwritten. This is because the time of the file on the remote host is left in an incorrect state during the first command execution (stopped by the quota limit). Any idea on how to avoid such a situation? The trouble is that you are passing --partial (-P = --partial --progress), which makes the receiver in the first command move the partially received copy of the new source file over the old destination file. Remove --partial (by replacing -P with --progress) and you should be fine: the receiver will discard the partially received file, leaving the old destination file as is, and the second command will skip transferring it back to the first host because of --update. For a more robust solution, you might look into a program that is designed for two-way synchronization, such as Unison ( http://www.cis.upenn.edu/~bcpierce/unison/ ). Or maybe we can correct rsync code to set file time to the correct one in case of disk quota error? No. For one thing, this won't solve the problem because rsync checks size as well as mtime and the sizes will be different. Also, we want the source and destination files to appear different so that if the user gets the disk quota on the second host raised and attempts the first command again, the new source file will be retransferred. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Bug or undesired behavior when quota exceeded
On 2007-11-08 at 15:05, Matt McCutchen wrote: On Thu, 2007-11-08 at 12:30 +0100, Janusz Jurski wrote: Any idea on how to avoid such a situation? The trouble is that you are passing --partial (-P = --partial --progress), which makes the receiver in the first command move the partially received copy of the new source file over the old destination file. Remove --partial (by replacing -P with --progress) and you should be fine: the receiver will discard the partially received file, leaving the old destination file as is, Thank you for the explanation. But it seems to me that this is against the description of the partial option in rsync manual. According to the manual: By default, rsync will delete any partially transferred file if the transfer is interrupted. In some circumstances it is more desirable to keep partially transferred files. Using the --partial option tells rsync to keep the partial file which should make a subsequent transfer of the rest of the file much faster. This specifically describes the situation when the transfer is interrupted. In my opinion, the partially transferred file shall be kept for future reuse and shall not overwrite the destination file. Note that the partial option makes no sense otherwise because it actually equals to the in-place option - partial becomes in-place when the transfer is interrupted and any assumption that the transfer is not interrupted is not reasonable in my opinion. Also, the current behavior is inconsistent with the --partial-dir option (the manual says for --partial-dir that such partially copied files are kept in the specified directory to speed up next transfers). These are just my thoughts. What is your opinion? Regards, JJ PS: I use --partial because the files I transfer are very large and the connection speed is not high... I will try using --partial-dir now. Mistrzowie stepowania już w Polsce! Wybierz się na niecodzienne widowisko z Austrii. Zobacz więcej: http://klik.wp.pl/?adr=http%3A%2F%2Fcorto.www.wp.pl%2Fas%2FTap-Dogs.htmlsid=89 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Bug or undesired behavior when quota exceeded
On Thu, 2007-11-08 at 18:36 +0100, Janusz Jurski wrote: But it seems to me that this is against the description of the partial option in rsync manual. In my opinion, the partially transferred file shall be kept for future reuse and shall not overwrite the destination file. Also, the current behavior is inconsistent with the --partial-dir option (the manual says for --partial-dir that such partially copied files are kept in the specified directory to speed up next transfers). The --partial option is behaving as documented. If you want --partial-dir's better behavior, you have to pass --partial-dir (or the environment variable RSYNC_PARTIAL_DIR). Note that the partial option makes no sense otherwise because it actually equals to the in-place option Not exactly. With --inplace, rsync writes new data into the original destination file; the new data will be visible through other hard links to the destination file. Without --inplace, rsync writes data to a new file and moves it over the original on successful completion or (assuming --partial is given) interruption. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
