Bug or undesired behavior when quota exceeded
Hi, I have just noticed an undesired behavior when using rsync when disk quota is exceeded. I do not know how to avoid it (or maybe it can be considered as a bug). The following is a scenario that leads to the undesired behavior: 1) I normally execute two following commands to synchronize folders between a local host and a remote host: rsync -aCPvu --existing dir1 [EMAIL PROTECTED]:dir2 rsync -aCPvu --existing [EMAIL PROTECTED]:dir2 dir1 2) Assume that files are in-sync on the hosts. Then we change a file on the local host. When I execute the first rsync command, the file shall be copied to the remote host. When the second command is executed, no copying shall occur as the files are in sync after the first command. 3) Unfortunately, it may happen that the disk quota during the first command execution is exceeded. I get the following in such a case: rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe (32) rsync: write failed on dir2/file: Disk quota exceeded (122) rsync error: error in file IO (code 11) at receiver.c(259) [receiver=2.6.9] rsync: connection unexpectedly closed (219584 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(453) [generator=2.6.9] rsync: connection unexpectedly closed (25270 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(453) [sender=2.6.9] 4) The above error is normal and expected. 5) When the second command is invoked, rsync starts copying the file from the remote host to the local one. This is an undesired behavior, as the file changed on the local host gets overwritten. This is because the time of the file on the remote host is left in an incorrect state during the first command execution (stopped by the quota limit). Any idea on how to avoid such a situation? Or maybe we can correct rsync code to set file time to the correct one in case of disk quota error? Regards, JJ Wojna z terrorem wkracza w decydującą fazę: Robert Redford, Meryl Streep i Tom Cruise w filmie UKRYTA STRATEGIA - w kinach od 9 listopada! http://klik.wp.pl/?adr=http%3A%2F%2Fcorto.www.wp.pl%2Fas%2Fstrategia.htmlsid=90 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
additional errors from 3.0.0pre4
Hello all, While attempting to run an rsync backup of a remote server (the scenario i posted about a few days ago, in fact), i am occasionally receiving the following error: rsync: writefd_unbuffered failed to write 8 bytes [receiver]: Broken pipe (32) rsync error: error in rsync protocol data stream (code 12) at io.c(1501) [receiver=3.0.0pre4] _exit_cleanup(code=12, file=io.c, line=1501): about to call exit(12) It appears to be reproducible, in that if i run the rsync again, it will error out in the same place each time. :P As before, i am running rsync 3.0.0pre4.matt, with the CVS 1.12 xattrs.c . I am happy to supply any additional information that i can, and will gladly do so (within my capacity) upon request. Thank you, all. -- dan. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Bug or undesired behavior when quota exceeded
On Thu, 2007-11-08 at 12:30 +0100, Janusz Jurski wrote: I have just noticed an undesired behavior when using rsync when disk quota is exceeded. 1) I normally execute two following commands to synchronize folders between a local host and a remote host: rsync -aCPvu --existing dir1 [EMAIL PROTECTED]:dir2 rsync -aCPvu --existing [EMAIL PROTECTED]:dir2 dir1 2) Assume that files are in-sync on the hosts. Then we change a file on the local host. When I execute the first rsync command, the file shall be copied to the remote host. When the second command is executed, no copying shall occur as the files are in sync after the first command. 3) Unfortunately, it may happen that the disk quota during the first command execution is exceeded. 5) When the second command is invoked, rsync starts copying the file from the remote host to the local one. This is an undesired behavior, as the file changed on the local host gets overwritten. This is because the time of the file on the remote host is left in an incorrect state during the first command execution (stopped by the quota limit). Any idea on how to avoid such a situation? The trouble is that you are passing --partial (-P = --partial --progress), which makes the receiver in the first command move the partially received copy of the new source file over the old destination file. Remove --partial (by replacing -P with --progress) and you should be fine: the receiver will discard the partially received file, leaving the old destination file as is, and the second command will skip transferring it back to the first host because of --update. For a more robust solution, you might look into a program that is designed for two-way synchronization, such as Unison ( http://www.cis.upenn.edu/~bcpierce/unison/ ). Or maybe we can correct rsync code to set file time to the correct one in case of disk quota error? No. For one thing, this won't solve the problem because rsync checks size as well as mtime and the sizes will be different. Also, we want the source and destination files to appear different so that if the user gets the disk quota on the second host raised and attempts the first command again, the new source file will be retransferred. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 5064] compile error - SCO 5.0.6a
https://bugzilla.samba.org/show_bug.cgi?id=5064 [EMAIL PROTECTED] changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution||FIXED --- Comment #2 from [EMAIL PROTECTED] 2007-11-08 09:54 CST --- The compile works, if you're picky about warnings, I do still get: gcc -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c lib/getaddrinfo.c -o lib/getaddrinfo.o lib/getaddrinfo.c: In function `get_canon_name_from_addr': lib/getaddrinfo.c:116: warning: passing arg 1 of `gethostbyaddr' from incompatible pointer type Thanks -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 5064] compile error - SCO 5.0.6a
https://bugzilla.samba.org/show_bug.cgi?id=5064 [EMAIL PROTECTED] changed: What|Removed |Added Status|NEW |ASSIGNED --- Comment #1 from [EMAIL PROTECTED] 2007-11-08 09:13 CST --- The latest CVS version fixes all the errors I could discern from that failed compilation. Please add another comment if there are more problems. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Bug or undesired behavior when quota exceeded
On 2007-11-08 at 15:05, Matt McCutchen wrote: On Thu, 2007-11-08 at 12:30 +0100, Janusz Jurski wrote: Any idea on how to avoid such a situation? The trouble is that you are passing --partial (-P = --partial --progress), which makes the receiver in the first command move the partially received copy of the new source file over the old destination file. Remove --partial (by replacing -P with --progress) and you should be fine: the receiver will discard the partially received file, leaving the old destination file as is, Thank you for the explanation. But it seems to me that this is against the description of the partial option in rsync manual. According to the manual: By default, rsync will delete any partially transferred file if the transfer is interrupted. In some circumstances it is more desirable to keep partially transferred files. Using the --partial option tells rsync to keep the partial file which should make a subsequent transfer of the rest of the file much faster. This specifically describes the situation when the transfer is interrupted. In my opinion, the partially transferred file shall be kept for future reuse and shall not overwrite the destination file. Note that the partial option makes no sense otherwise because it actually equals to the in-place option - partial becomes in-place when the transfer is interrupted and any assumption that the transfer is not interrupted is not reasonable in my opinion. Also, the current behavior is inconsistent with the --partial-dir option (the manual says for --partial-dir that such partially copied files are kept in the specified directory to speed up next transfers). These are just my thoughts. What is your opinion? Regards, JJ PS: I use --partial because the files I transfer are very large and the connection speed is not high... I will try using --partial-dir now. Mistrzowie stepowania już w Polsce! Wybierz się na niecodzienne widowisko z Austrii. Zobacz więcej: http://klik.wp.pl/?adr=http%3A%2F%2Fcorto.www.wp.pl%2Fas%2FTap-Dogs.htmlsid=89 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
problems with permissions on windows
Hello, I'm using cwRsync on windows (I've tried a few alternatives, like Delta for example, but all of them behave equally (bad) in my case - they set really weird permissions on binaries - I don't know which ones exactly, but binaries cannot be executed). I have a linux server with binaries for different platforms. When I rsync (copy) linux binaries to linux, or copy mac binaries to Mac OS X, everything works fine, but if I try to rsync (copy) windows binaries to a windows machine, I experience permission problems when trying to run them. All other files work OK (can be read and deleted), but .exe files don't. I use -rpvztl switches for rsync. I need to explicitely execute chmod 777 bin/* after each rsync call and I don't understand why. Original files have their permissions set to 755. - If I use vfat32 partition on a dual boot machine (Mac OS X) and copy (rsync) windows binaries under linux, the binaries work fine when switching back to windows. - If I zip the files on the server, download the zip and extract files on windows, the binaries also work OK. - I should test once more. I experienced problems both on vfat32 and ntfs, but under some circumstances the same thing worked on vfat32, but not on ntfs. Any help or hint about why this is happening would be greately appreciated. Thanks a lot, Mojca -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Bug or undesired behavior when quota exceeded
On Thu, 2007-11-08 at 18:36 +0100, Janusz Jurski wrote: But it seems to me that this is against the description of the partial option in rsync manual. In my opinion, the partially transferred file shall be kept for future reuse and shall not overwrite the destination file. Also, the current behavior is inconsistent with the --partial-dir option (the manual says for --partial-dir that such partially copied files are kept in the specified directory to speed up next transfers). The --partial option is behaving as documented. If you want --partial-dir's better behavior, you have to pass --partial-dir (or the environment variable RSYNC_PARTIAL_DIR). Note that the partial option makes no sense otherwise because it actually equals to the in-place option Not exactly. With --inplace, rsync writes new data into the original destination file; the new data will be visible through other hard links to the destination file. Without --inplace, rsync writes data to a new file and moves it over the original on successful completion or (assuming --partial is given) interruption. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 4912] rsync crashes with a double free or corrupted memory detection in glibc
https://bugzilla.samba.org/show_bug.cgi?id=4912 [EMAIL PROTECTED] changed: What|Removed |Added Status|NEW |RESOLVED Resolution||FIXED --- Comment #4 from [EMAIL PROTECTED] 2007-11-08 22:09 CST --- This appears to be the same crash as bug 4855, caused by the combination of -R (implied by --files-from) and -F and some uninitialized memory in the filter handling. That bug report has a quick fix for 2.6.9, and development version has this fixed. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: additional errors from 3.0.0pre4
On Thu, Nov 08, 2007 at 02:35:48PM +0100, Daniel Maher wrote: rsync: writefd_unbuffered failed to write 8 bytes [receiver]: Broken pipe (32) Are you using 3 or more -v options? If so, that should be fixed in the latest version. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Rsync 3.0.0pre5 released
I've just released rsync 3.0.0pre5, the latest pre-release version of the upcoming 3.0.0 release. Things are progressing well in our efforts to stamp out bugs for the final release. Please keep up the good work in testing this and sending email to the rsync mailing list with your questions, comments, bug reports, etc. Thanks! You can download the tar file and its signature from here: http://rsync.samba.org/ftp/rsync/rsync-3.0.0pre5.tar.gz http://rsync.samba.org/ftp/rsync/rsync-3.0.0pre5.tar.gz.asc This version has the following fixes since pre4 was released: - Output a (DRY RUN) suffix on the last line of the verbose summary when in --dry-run mode. - Fixed a problem with copying multiple mount-point directories when --one-file-system was specified. - The --copy-dest code makes it local-copy update to a temp file when --inplace was not specified. - Fixed a glitch when --copy-dest was combined with --dry-run. - Made some improvements in the parsing of untidy paths in --recursive mode. - Fixed a potential hang when --hard-links mode was copying a large directory structure with few (or no) files. - Fixed a problem with -vvv (or greater) output causing an inc-recursive run to terminate. - Stopped password-file errors from making rsync output that a file transfer was incomplete. - Avoid hard-linking devices together if their preserved mtime differs. - Replaced the lib/getaddrinfo.c code. To see a full summary of the changes since 2.6.9, visit this link: http://rsync.samba.org/ftp/rsync/rsync-3.0.0pre5-NEWS ..wayne.. signature.asc Description: Digital signature -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
CVS update: rsync/lib
Date: Thu Nov 8 14:54:58 2007 Author: wayned Update of /data/cvs/rsync/lib In directory dp.samba.org:/tmp/cvs-serv26936/lib Modified Files: getaddrinfo.c Log Message: Some fixes for SCO in new getaddrinfo() code. Revisions: getaddrinfo.c 1.28 = 1.29 http://www.samba.org/cgi-bin/cvsweb/rsync/lib/getaddrinfo.c?r1=1.28r2=1.29 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync/lib
Date: Thu Nov 8 20:30:17 2007 Author: wayned Update of /data/cvs/rsync/lib In directory dp.samba.org:/tmp/cvs-serv21737/lib Modified Files: getaddrinfo.c Log Message: Got rid of a compiler warning. Revisions: getaddrinfo.c 1.29 = 1.30 http://www.samba.org/cgi-bin/cvsweb/rsync/lib/getaddrinfo.c?r1=1.29r2=1.30 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync/patches
Date: Fri Nov 9 04:12:58 2007 Author: wayned Update of /data/cvs/rsync/patches In directory dp.samba.org:/tmp/cvs-serv30137 Modified Files: drop-cache.diff preallocate.diff Log Message: Fixed failing hunks. Revisions: drop-cache.diff 1.4 = 1.5 http://www.samba.org/cgi-bin/cvsweb/rsync/patches/drop-cache.diff?r1=1.4r2=1.5 preallocate.diff1.13 = 1.14 http://www.samba.org/cgi-bin/cvsweb/rsync/patches/preallocate.diff?r1=1.13r2=1.14 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Fri Nov 9 04:32:58 2007 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv15048 Modified Files: NEWS Log Message: Mention latest changes. Revisions: NEWS1.468 = 1.469 http://www.samba.org/cgi-bin/cvsweb/rsync/NEWS?r1=1.468r2=1.469 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Fri Nov 9 04:48:10 2007 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv27672 Modified Files: NEWS main.c Log Message: When performing a --dry-run, output a (DRY RUN) reminder on the last line of the verbose summary text. Revisions: NEWS1.469 = 1.470 http://www.samba.org/cgi-bin/cvsweb/rsync/NEWS?r1=1.469r2=1.470 main.c 1.353 = 1.354 http://www.samba.org/cgi-bin/cvsweb/rsync/main.c?r1=1.353r2=1.354 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Fri Nov 9 04:49:18 2007 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv28842 Modified Files: configure configure.in rsync.1 rsync.yo rsyncd.conf.5 rsyncd.conf.yo Log Message: Preparing for release of 3.0.0pre5 Revisions: configure 1.277 = 1.278 http://www.samba.org/cgi-bin/cvsweb/rsync/configure?r1=1.277r2=1.278 configure.in1.277 = 1.278 http://www.samba.org/cgi-bin/cvsweb/rsync/configure.in?r1=1.277r2=1.278 rsync.1 1.456 = 1.457 http://www.samba.org/cgi-bin/cvsweb/rsync/rsync.1?r1=1.456r2=1.457 rsync.yo1.442 = 1.443 http://www.samba.org/cgi-bin/cvsweb/rsync/rsync.yo?r1=1.442r2=1.443 rsyncd.conf.5 1.151 = 1.152 http://www.samba.org/cgi-bin/cvsweb/rsync/rsyncd.conf.5?r1=1.151r2=1.152 rsyncd.conf.yo 1.152 = 1.153 http://www.samba.org/cgi-bin/cvsweb/rsync/rsyncd.conf.yo?r1=1.152r2=1.153 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsyncweb
Date: Fri Nov 9 05:30:18 2007 Author: wayned Update of /data/cvs/rsyncweb In directory dp.samba.org:/tmp/cvs-serv4463 Modified Files: index.html Log Message: Mention 3.0.0pre5 release. Revisions: index.html 1.83 = 1.84 http://www.samba.org/cgi-bin/cvsweb/rsyncweb/index.html?r1=1.83r2=1.84 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
