rsync errors
Hi! List, I installed rsync, configured ssh. I need help in configuring rsyncd.conf. Could anyone email me a sample of rsyncd.conf under /etc. Thanks in advance Raj = __ Do You Yahoo!? Make a great connection at Yahoo! Personals. http://personals.yahoo.com
rsync errors
FreeBSD 4.8-Release rsync-2.5.6_1 The error: rsync: writefd_unbuffered failed to write 16385 bytes: phase "unknown": Broken pipe rsync error: error in rsync protocol data stream (code 12) at io.c(515) Log file: (about a dozen of these) Mar 24 07:35:10 vault-a /kernel: (da0:dpt0:0:0:0): READ(10). CDB: 28 0 0 97 e2 7 f 0 0 80 0 Mar 24 07:35:10 vault-a /kernel: (da0:dpt0:0:0:0): RECOVERED ERROR info:97e2a8 a sc:18,f Mar 24 07:35:10 vault-a /kernel: (da0:dpt0:0:0:0): Reserved ASC/ASCQ pair field replaceable unit: ea sks:80,6 Is it the harddisk? Is it recoverable? Thanks in advance for any advice, Sincerely, Mike Lynn -- MJL - Network Consultant Michael J. Lynn Eliot, ME 03903 Phone: 207-451-9433 Pager: 1-800-646-8823 E-Mail: [EMAIL PROTECTED] www.a2nice.net -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Rsync errors
Hello! I'm in trouble with rsync: I'm trying to: sudo rsync -atlrpogH --exclude-from=/usr/home/exclude [EMAIL PROTECTED]::home --password-file=/root/rsync_pass /usr/home/ it does sync some files, but then it stops and outputs these errors: rsync: writefd_unbuffered failed to write 4092 bytes [generator]: Operation not permitted (1) rsync error: error in rsync protocol data stream (code 12) at io.c(1124) [generator=2.6.8] rsync error: received SIGUSR1 (code 19) at main.c(1095) [receiver=2.6.8] I tried to google them, but couldn't locate a similar problem. Please point me to the right direction. -- View this message in context: http://www.nabble.com/Rsync-errors-tf2815812.html#a7858440 Sent from the Samba - rsync mailing list archive at Nabble.com. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync errors
Hi
I am running the command
/usr/bin/rsync -avO /var/lib/mysql [EMAIL PROTECTED]:/var/lib/ >
/tmp/rsync-${TIMESTAMP}.log 2>&1
I am getting the below error in the log file
mysql/
rsync: failed to set permissions on "/var/lib/mysql": Operation not
permitted (1)
rsync: failed to open "/var/lib/mysql/ib_logfile0", continuing: Permission
denied (13)
mysql/ib_logfile0
rsync: failed to open "/var/lib/mysql/ib_logfile1", continuing: Permission
denied (13)
mysql/ib_logfile1
rsync: failed to open "/var/lib/mysql/ibdata1", continuing: Permission
denied (13)
mysql/ibdata1
rsync: failed to open "/var/lib/mysql/master.info", continuing: Permission
denied (13)
mysql/master.info
rsync: failed to open "/var/lib/mysql/SCHEDULER/ScheduleTasks.MYD",
continuing: Permission denied (13)
Any thing I am missing
Thanks and Regards
Kaushal
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Rsync errors
Hi list, I have installed rsync version 2.6.0 on Solaris 9 servers, both the servers have rsync installed in /usr/local/bin directory. When I tried to rsync from one server to other I am getting rsync not found errors, it not the problem with PATH on the source server because rsync by itself coming up with options. bash-2.05$ rsync junk 10.5.2.171:/tmpksh: rsync: not foundrsync: connection unexpectedly closed (0 bytes read so far)rsync error: error in rsync protocol data stream (code 12) at io.c(189) Thanks in advance__Do You Yahoo!?Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync errors
rsync-path fixed those path related errors.__Do You Yahoo!?Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Rsync errors
Hi, I am using Rsync 2.3.1 version to transfer files between 2 unix machines. When I transfer files I am getting a error like the below one. rsync: recv_generator: mkdir "200610/06/2454/4/3827": No such file or directory (2) stat 200610/06/2454/4/3827 : No such file or directory rsync: recv_generator: mkdir "200610/06/2454/4/3828": No such file or directory (2) stat 200610/06/2454/4/3828 : No such file or directory rsync: recv_generator: mkdir "200610/06/2454/4/3829": No such file or directory (2) stat 200610/06/2454/4/3829 : No such file or directory rsync: recv_generator: mkdir "200610/06/2466": No such file or directory (2) stat 200610/06/2466 : No such file or directory rsync: recv_generator: mkdir "200610/06/2466/49": No such file or directory (2) stat 200610/06/2466/49 : No such file or directory and also like the below error 200604/pbstmt-976-05932.200604.pdf mkstemp 200604/.pbstmt-976-05932.200604.pdf.g_aakP failed: Permission denied 200609/pbstmt-106-4.200609.pdf 200609/pbstmt-106-5.200609.pdf 200609/pbstmt-106-6.200609.pdf write failed on 200609/pbstmt-106-4.200609.pdf : Error 0 rsync error: error in file IO (code 11) at receiver.c(244) rsync: connection unexpectedly closed (175586 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(165) Please help me.. Awaiting for a reply. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync errors
Running Ubuntu 10.04.1 and rsync 3.0.7-1. Used rsync successfully for over a year for backups. Recently there was a kernel upgrade to 2.6.32-25. Don't know if that has any bearing. When I tried to run a backup today rsync failed with the error message: sending incremental file list rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32) rsync: mkdir "/media/disk/10.04laptop" failed: No such file or directory (2) rsync error: error in file IO (code 11) at main.c(595) [Receiver=3.0.7] rsync: connection unexpectedly closed (9 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(601) [sender=3.0.7] /var/log/syslog just showed the above message. Searched on the message and got one hit that said it was because a FAT32 partition had a size limit. I doubt that I've come close to that. Couldn't find anything in this list's archives that matched the error message. Did I miss something? The command that has been working until now is: sudo rsync -vaHz --exclude '/proc' --exclude '/sys' --exclude '*.iso' --exclude '/home/holtzm/Documents/*.iso' --exclude '/media' /. /media/disk/10.04laptop I'm at a loss as to how to proceed. any pointers appreciated. -- Bob Holtzman Key ID: 8D549279 "If you think you're getting free lunch, check the price of the beer" -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync errors out
RSYNC keeps giving me "error while reading byte # " It then stops the transfer. Any ideas on this one ? Tim Collins, Server AdministratorGamer's Alliance, Inc.An Euniverse company.http://www.euniverse.com[EMAIL PROTECTED]
Re: rsync errors
I need to use rsync to copy /thirdparty/dynamo/web on host a to /thirdparty/dynamo/web on host b. Could anyone help me how to setup the rsyncd.conf file for this, do I really need to use rsync.conf. I ran ssh to the remote server, the shell is clean but when I tried the following command it gives error ssh rmt_host date (it displays date on the remote host) rsync file rmt_host:/tmp gives following error: Failed to exec remsh :permission denied unexpected EOF in read_timeout I know remsh is used for unsecured hosts using .rhosts that means some where we need to tell rsync to look for ssh not rsh but I don't know where to look. Please help. Thanks --- Raj <[EMAIL PROTECTED]> wrote: > > Hi! List, > I installed rsync, configured ssh. I need help in > configuring rsyncd.conf. Could anyone email me a > sample of rsyncd.conf under /etc. > Thanks in advance > Raj > > = > > > __ > Do You Yahoo!? > Make a great connection at Yahoo! Personals. > http://personals.yahoo.com > = __ Do You Yahoo!? Make a great connection at Yahoo! Personals. http://personals.yahoo.com
Re: rsync errors
List, I also tried the following command, it does not copy any files to remote host but gives a error rsync -Cavze ssh . rmt_host:/backup error:unexpected EOF in read_timeout --- Raj <[EMAIL PROTECTED]> wrote: > I need to use rsync to copy /thirdparty/dynamo/web > on > host a to /thirdparty/dynamo/web on host b. Could > anyone help me how to setup the rsyncd.conf file for > this, do I really need to use rsync.conf. > I ran ssh to the remote server, the shell is clean > but > when I tried the following command it gives error > > ssh rmt_host date (it displays date on the remote > host) > > rsync file rmt_host:/tmp > > gives following error: > Failed to exec remsh :permission denied > unexpected EOF in read_timeout > > I know remsh is used for unsecured hosts using > .rhosts > that means some where we need to tell rsync to look > for ssh not rsh but I don't know where to look. > Please > help. > Thanks > --- Raj <[EMAIL PROTECTED]> wrote: > > > > Hi! List, > > I installed rsync, configured ssh. I need help in > > configuring rsyncd.conf. Could anyone email me a > > sample of rsyncd.conf under /etc. > > Thanks in advance > > Raj > > > > = > > > > > > __ > > Do You Yahoo!? > > Make a great connection at Yahoo! Personals. > > http://personals.yahoo.com > > > > > = > > > __ > Do You Yahoo!? > Make a great connection at Yahoo! Personals. > http://personals.yahoo.com > = __ Do You Yahoo!? Make a great connection at Yahoo! Personals. http://personals.yahoo.com
RE: rsync errors
Hello Raj, I had those errors until I realized that ~/.ssh/authorized_keys were not loaded properly. When executing rsync use -vv and -vvv. You'll get more debug output. Read the following articles: http://www-106.ibm.com/developerworks/linux/library/l-keyc.html http://www-106.ibm.com/developerworks/opensource/library/l-keyc2/?dwzone=ope Cheers, Glenn > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On > Behalf Of Raj > Sent: Thursday, October 18, 2001 6:25 PM > To: [EMAIL PROTECTED] > Subject: Re: rsync errors > > > List, > I also tried the following command, it does not copy > any files to remote host but gives a error > > rsync -Cavze ssh . rmt_host:/backup > > error:unexpected EOF in read_timeout > --- Raj <[EMAIL PROTECTED]> wrote: > > I need to use rsync to copy /thirdparty/dynamo/web > > on > > host a to /thirdparty/dynamo/web on host b. Could > > anyone help me how to setup the rsyncd.conf file for > > this, do I really need to use rsync.conf. > > I ran ssh to the remote server, the shell is clean > > but > > when I tried the following command it gives error > > > > ssh rmt_host date (it displays date on the remote > > host) > > > > rsync file rmt_host:/tmp > > > > gives following error: > > Failed to exec remsh :permission denied > > unexpected EOF in read_timeout > > > > I know remsh is used for unsecured hosts using > > .rhosts > > that means some where we need to tell rsync to look > > for ssh not rsh but I don't know where to look. > > Please > > help. > > Thanks > > --- Raj <[EMAIL PROTECTED]> wrote: > > > > > > Hi! List, > > > I installed rsync, configured ssh. I need help in > > > configuring rsyncd.conf. Could anyone email me a > > > sample of rsyncd.conf under /etc. > > > Thanks in advance > > > Raj > > > > > > = > > > > > > > > > __ > > > Do You Yahoo!? > > > Make a great connection at Yahoo! Personals. > > > http://personals.yahoo.com > > > > > > > > > = > > > > > > __ > > Do You Yahoo!? > > Make a great connection at Yahoo! Personals. > > http://personals.yahoo.com > > > > > = > > > __ > Do You Yahoo!? > Make a great connection at Yahoo! Personals. > http://personals.yahoo.com >
Re: rsync errors
Ah... there's your answer. ssh is working fine, as you say, but you're invoking rsync without telling it to user ssh, so it's using rsh. If you've got this, you don't need rsyncd.conf. The syntax you're using tells rsync to use an external transport. rsync -e ssh file rmt_host:/tmp Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), ".\n" ' "There are some who call me Tim?" Raj <[EMAIL PROTECTED]>@lists.samba.org on 10/18/2001 04:15:16 PM Sent by: [EMAIL PROTECTED] To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: Re: rsync errors Classification: I need to use rsync to copy /thirdparty/dynamo/web on host a to /thirdparty/dynamo/web on host b. Could anyone help me how to setup the rsyncd.conf file for this, do I really need to use rsync.conf. I ran ssh to the remote server, the shell is clean but when I tried the following command it gives error ssh rmt_host date (it displays date on the remote host) rsync file rmt_host:/tmp gives following error: Failed to exec remsh :permission denied unexpected EOF in read_timeout I know remsh is used for unsecured hosts using .rhosts that means some where we need to tell rsync to look for ssh not rsh but I don't know where to look. Please help. Thanks --- Raj <[EMAIL PROTECTED]> wrote: > > Hi! List, > I installed rsync, configured ssh. I need help in > configuring rsyncd.conf. Could anyone email me a > sample of rsyncd.conf under /etc. > Thanks in advance > Raj > > = > > > __ > Do You Yahoo!? > Make a great connection at Yahoo! Personals. > http://personals.yahoo.com > = __ Do You Yahoo!? Make a great connection at Yahoo! Personals. http://personals.yahoo.com
Re: rsync errors
Thanks Tim, I got it working but I noticed rsync does not exit when it is done, i tried to run it from a script and I have exit in the script it still does not exit. Is it a bug? --- [EMAIL PROTECTED] wrote: > Ah... there's your answer. ssh is working fine, as > you say, but you're invoking rsync without telling > it to user ssh, so it's using rsh. If you've got > this, you don't need rsyncd.conf. The syntax you're > using tells rsync to use an external transport. > rsync -e ssh file rmt_host:/tmp > > Tim Conway > [EMAIL PROTECTED] > 303.682.4917 > Philips Semiconductor - Longmont TC > 1880 Industrial Circle, Suite D > Longmont, CO 80501 > Available via SameTime Connect within Philips > Available as n9hmg on AIM > perl -e 'print pack(, > 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), > ".\n" ' > "There are some who call me Tim?" > > > > > > Raj <[EMAIL PROTECTED]>@lists.samba.org on 10/18/2001 > 04:15:16 PM > > Sent by: [EMAIL PROTECTED] > > > To: "[EMAIL PROTECTED]" > <[EMAIL PROTECTED]> > cc: (bcc: Tim Conway/LMT/SC/PHILIPS) > Subject: Re: rsync errors > Classification: > > > > I need to use rsync to copy /thirdparty/dynamo/web > on > host a to /thirdparty/dynamo/web on host b. Could > anyone help me how to setup the rsyncd.conf file for > this, do I really need to use rsync.conf. > I ran ssh to the remote server, the shell is clean > but > when I tried the following command it gives error > > ssh rmt_host date (it displays date on the remote > host) > > rsync file rmt_host:/tmp > > gives following error: > Failed to exec remsh :permission denied > unexpected EOF in read_timeout > > I know remsh is used for unsecured hosts using > .rhosts > that means some where we need to tell rsync to look > for ssh not rsh but I don't know where to look. > Please > help. > Thanks > --- Raj <[EMAIL PROTECTED]> wrote: > > > > Hi! List, > > I installed rsync, configured ssh. I need help in > > configuring rsyncd.conf. Could anyone email me a > > sample of rsyncd.conf under /etc. > > Thanks in advance > > Raj > > > > = > > > > > > __ > > Do You Yahoo!? > > Make a great connection at Yahoo! Personals. > > http://personals.yahoo.com > > > > > = > > > __ > Do You Yahoo!? > Make a great connection at Yahoo! Personals. > http://personals.yahoo.com > > > > > = __ Do You Yahoo!? Make a great connection at Yahoo! Personals. http://personals.yahoo.com
Re: Rsync errors
On 12/13/06, ilushkin <[EMAIL PROTECTED]> wrote: rsync: writefd_unbuffered failed to write 4092 bytes [generator]: Operation not permitted (1) rsync error: error in rsync protocol data stream (code 12) at io.c(1124) [generator=2.6.8] rsync error: received SIGUSR1 (code 19) at main.c(1095) [receiver=2.6.8] This output seems to indicate that a write system call issued by the generator failed with EPERM (Operation not permitted). I don't think write ever fails with EPERM on Linux or BSD. What operating system are you using? I would recommend stracing the client to see exactly what system call is failing. (Run sudo strace -o rsync.log -f rsync (arguments...), or use the strace equivalent for your operating system.) Also try running rsync at a higher verbosity level (-vv or -vvv) to get more information. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync errors
Matt McCutchen-3 wrote: > > On 12/13/06, ilushkin <[EMAIL PROTECTED]> wrote: >> rsync: writefd_unbuffered failed to write 4092 bytes [generator]: >> Operation not permitted (1) >> rsync error: error in rsync protocol data stream (code 12) at >> io.c(1124) [generator=2.6.8] >> rsync error: received SIGUSR1 (code 19) at main.c(1095) [receiver=2.6.8] > > This output seems to indicate that a write system call issued by the > generator failed with EPERM (Operation not permitted). I don't think > write ever fails with EPERM on Linux or BSD. What operating system > are you using? > > I would recommend stracing the client to see exactly what system call > is failing. (Run sudo strace -o rsync.log -f rsync (arguments...), or > use the strace equivalent for your operating system.) Also try > running rsync at a higher verbosity level (-vv or -vvv) to get more > information. > > Matt > -- > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html > > I use freebsd 6.1 this what I get with -vvv (sudo rsync -atlrpogHvvv [EMAIL PROTECTED]::public --password-file=/root/rsync_pass /home/public ) however, some files've been copied. recv_generator(.recycle/P.O.'S 2006.xls~RF7a1972.TMP,2985) recv_generator(.recycle/P.O.'S 2006.xls~RF7baad0.TMP,2986) rsync: writefd_unbuffered failed to write 4092 bytes [generator]: Operation not permitted (1) rsync error: error in rsync protocol data stream (code 12) at io.c(1124) [generator=2.6.8] _exit_cleanup(code=12, file=io.c, line=1124): about to call exit(12) rsync error: received SIGUSR1 (code 19) at main.c(1095) [receiver=2.6.8] _exit_cleanup(code=19, file=main.c, line=1095): about to call exit(19) or sudo rsync -atlrpogHvvv [EMAIL PROTECTED]::username --password-file=/root/rsync_pass /home/username recv_generator(files/Application Data/Microsoft/Media Player,3020) recv_generator(files/Application Data/Microsoft/Media Player/ActivePlaylist.dat,3021) recv_generator(files/Application Data/Microsoft/Media Player/OfflineCDUpdates.dat,3022) rsync: writefd_unbuffered failed to write 4092 bytes [generator]: Operation not permitted (1) rsync error: error in rsync protocol data stream (code 12) at io.c(1124) [generator=2.6.8] _exit_cleanup(code=12, file=io.c, line=1124): about to call exit(12) rsync error: received SIGUSR1 (code 19) at main.c(1095) [receiver=2.6.8] _exit_cleanup(code=19, file=main.c, line=1095): about to call exit(19) -- View this message in context: http://www.nabble.com/Rsync-errors-tf2815812.html#a7896309 Sent from the Samba - rsync mailing list archive at Nabble.com. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync errors
On 12/15/06, ilushkin <[EMAIL PROTECTED]> wrote: recv_generator(files/Application Data/Microsoft/Media Player,3020) recv_generator(files/Application Data/Microsoft/Media Player/ActivePlaylist.dat,3021) recv_generator(files/Application Data/Microsoft/Media Player/OfflineCDUpdates.dat,3022) rsync: writefd_unbuffered failed to write 4092 bytes [generator]: Operation not permitted (1) rsync error: error in rsync protocol data stream (code 12) at io.c(1124) [generator=2.6.8] _exit_cleanup(code=12, file=io.c, line=1124): about to call exit(12) rsync error: received SIGUSR1 (code 19) at main.c(1095) [receiver=2.6.8] _exit_cleanup(code=19, file=main.c, line=1095): about to call exit(19) OK, that level of verbosity didn't help very much. You should trace rsync using strace, truss, or a similar tool to see exactly which write call to which file is failing. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync errors
Matt McCutchen-3 wrote: > > On 12/15/06, ilushkin <[EMAIL PROTECTED]> wrote: >> recv_generator(files/Application Data/Microsoft/Media Player,3020) >> recv_generator(files/Application Data/Microsoft/Media >> Player/ActivePlaylist.dat,3021) >> recv_generator(files/Application Data/Microsoft/Media >> Player/OfflineCDUpdates.dat,3022) >> rsync: writefd_unbuffered failed to write 4092 bytes [generator]: >> Operation >> not permitted (1) >> rsync error: error in rsync protocol data stream (code 12) at io.c(1124) >> [generator=2.6.8] >> _exit_cleanup(code=12, file=io.c, line=1124): about to call exit(12) >> rsync error: received SIGUSR1 (code 19) at main.c(1095) [receiver=2.6.8] >> _exit_cleanup(code=19, file=main.c, line=1095): about to call exit(19) > > OK, that level of verbosity didn't help very much. You should trace > rsync using strace, truss, or a similar tool to see exactly which > write call to which file is failing. > > Matt > -- > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html > > Unfortunately, I'm not such an advance Unix user to do that. So far I figured that strace is not familar to Freebsd, that I'm using. I really appritiate your help. This bug really took most of my time at work. I wish that thing were not that complicated. Sometimes I wish that I had some gui Microsoft like product to play with. Well, guess God wants me to learn things the hard way :) Back to the thread, I noticed that rsync gives me such errors when it meets a directory with the huge amount of files. Sometimes over 6000. I tried to play around with one of those directories by removing the unneeded files, and rsync did the job without any complains. But I still have lots of directories to sychronize and there is no way I can reduce that amount of files. I've heard that rsync can have trouble with the similar issue. Can this be related to my problem? Thank you! Ilya. -- View this message in context: http://www.nabble.com/Rsync-errors-tf2815812.html#a7911305 Sent from the Samba - rsync mailing list archive at Nabble.com. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync errors
On 12/16/06, ilushkin <[EMAIL PROTECTED]> wrote: Unfortunately, I'm not such an advance Unix user to [strace rsync]. So far I figured that strace is not familar to Freebsd, that I'm using. I really appritiate your help. This bug really took most of my time at work. I wish that thing were not that complicated. Sometimes I wish that I had some gui Microsoft like product to play with. Well, guess God wants me to learn things the hard way :) Back to the thread, I noticed that rsync gives me such errors when it meets a directory with the huge amount of files. Sometimes over 6000. I tried to play around with one of those directories by removing the unneeded files, and rsync did the job without any complains. But I still have lots of directories to sychronize and there is no way I can reduce that amount of files. I've heard that rsync can have trouble with the similar issue. Can this be related to my problem? If you're not willing to trace the system calls of rsync, there is nothing more I can do to investigate the problem. I do not know of a connection between directories containing lots of files and write errors. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync errors
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
- Original Message
Subject: Re:Rsync errors
From: Matt McCutchen <[EMAIL PROTECTED]>
To: ilushkin <[EMAIL PROTECTED]>
Cc: rsync@lists.samba.org
Date: Sunday, December 17, 2006 12:22:07 PM
> On 12/16/06, ilushkin <[EMAIL PROTECTED]> wrote:
>> Unfortunately, I'm not such an advance Unix user to [strace
>> rsync]. So far I figured that strace is not familar to Freebsd,
>> that I'm using. I really appritiate your help. This bug really
>> took most of my time at work. I wish that thing were not that
>> complicated. Sometimes I wish that I had some gui Microsoft like
>> product to play with. Well, guess God wants me to learn things
>> the hard way :)
>>
>> Back to the thread, I noticed that rsync gives me such errors
>> when it meets a directory with the huge amount of files.
>> Sometimes over 6000. I tried to play around with one of those
>> directories by removing the unneeded files, and rsync did the job
>> without any complains. But I still have lots of directories to
>> sychronize and there is no way I can reduce that amount of files.
>> I've heard that rsync can have trouble with the similar issue.
>> Can this be related to my problem?
>
> If you're not willing to trace the system calls of rsync, there is
> nothing more I can do to investigate the problem. I do not know of
> a connection between directories containing lots of files and write
> errors.
>
> Matt
>
truss rsync -atlrpogHvv [EMAIL PROTECTED]::public
- --password-file=/root/rsync_pass /home/public
lstat("Company/General/2006 OLD
ARRIVALS/ARRV112106.xls",0x7fffc620) ERR#2 'No such file or directory'
lstat("Company/General/2006 OLD
ARRIVALS/ARRV112206.xls",0x7fffc620) ERR#2 'No such file or directory'
lstat("Company/General/2006 OLD
ARRIVALS/ARRV112706.xls",0x7fffc620) ERR#2 'No such file or directory'
lstat("Company/General/2006 OLD
ARRIVALS/ARRV112806.xls",0x7fffc620) ERR#2 'No such file or directory'
lstat("Company/General/2006 OLD
ARRIVALS/ARRV112906.xls",0x7fffc620) ERR#2 'No such file or directory'
select(5,{4},{3},0x0,{60 0}) = 1 (0x1)
write(3,0xc2,4092) ERR#1 'Operation not permitted'
rsync: writefd_unbuffered failed to write 4092 bytes [generator]:
Operation not permitted (1)write(2,0x7fffa4f0,93) = 93 (0x5d)
write(2,0x80083ce67,1) = 1 (0x1)
sigaction(SIGUSR1,{ SIG_IGN
0x0|ONSTACK|RESTART|RESETHAND|NOCLDSTOP|NODEFER|NOCLDWAIT|SIGINFO ss_t
},0x0) = 0 (0x0)
sigaction(SIGUSR2,{ SIG_IGN
0x0|ONSTACK|RESTART|RESETHAND|NOCLDSTOP|NODEFER|NOCLDWAIT|SIGINFO ss_t
},0x0) = 0 (0x0)
getpid() = 42660 (0xa6a4)
kill(0xa6a5,0x1e)= 0 (0x0)
rsync error: received SIGUSR1 (code 19) at main.c(1095) [receiver=2.6.8]
rsync error: error in rsync protocol data stream (code 12) at
io.c(1124) [generator=2.6.8]write(2,0x7fffa480,90) = 90 (0x5a)
write(2,0x80083ce67,1) = 1 (0x1)
SIGNAL 20 (SIGCHLD)
SIGNAL 20 (SIGCHLD)
wait4(0x,0x7fffb394,0x1,0x0) = 42661 (0xa6a5)
wait4(0x,0x7fffb394,0x1,0x0) ERR#10 'No child
processes'
sigreturn(0x7fffb3b0)= 524288 (0x8)
exit(0xc)
process exit, rval = 3072
thank you.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFiYeibE0+gtjqpZoRAgXiAKCt0sSCu3Cf/nxyqZuPf+SBBcLXRgCeMRKh
izRT7N7aNibhOLapxsPU6vs=
=yWmx
-END PGP SIGNATURE-
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync errors
On 12/20/06, Ilya Vishnyakov <[EMAIL PROTECTED]> wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
- Original Message
Subject: Re:Rsync errors
From: Matt McCutchen <[EMAIL PROTECTED]>
To: ilushkin <[EMAIL PROTECTED]>
Cc: rsync@lists.samba.org
Date: Sunday, December 17, 2006 12:22:07 PM
> On 12/16/06, ilushkin <[EMAIL PROTECTED]> wrote:
>> Unfortunately, I'm not such an advance Unix user to [strace
>> rsync]. So far I figured that strace is not familar to Freebsd,
>> that I'm using. I really appritiate your help. This bug really
>> took most of my time at work. I wish that thing were not that
>> complicated. Sometimes I wish that I had some gui Microsoft like
>> product to play with. Well, guess God wants me to learn things
>> the hard way :)
>>
>> Back to the thread, I noticed that rsync gives me such errors
>> when it meets a directory with the huge amount of files.
>> Sometimes over 6000. I tried to play around with one of those
>> directories by removing the unneeded files, and rsync did the job
>> without any complains. But I still have lots of directories to
>> sychronize and there is no way I can reduce that amount of files.
>> I've heard that rsync can have trouble with the similar issue.
>> Can this be related to my problem?
>
> If you're not willing to trace the system calls of rsync, there is
> nothing more I can do to investigate the problem. I do not know of
> a connection between directories containing lots of files and write
> errors.
>
> Matt
>
truss rsync -atlrpogHvv [EMAIL PROTECTED]::public
- --password-file=/root/rsync_pass /home/public
lstat("Company/General/2006 OLD
ARRIVALS/ARRV112106.xls",0x7fffc620) ERR#2 'No such file or directory'
lstat("Company/General/2006 OLD
ARRIVALS/ARRV112206.xls",0x7fffc620) ERR#2 'No such file or directory'
lstat("Company/General/2006 OLD
ARRIVALS/ARRV112706.xls",0x7fffc620) ERR#2 'No such file or directory'
lstat("Company/General/2006 OLD
ARRIVALS/ARRV112806.xls",0x7fffc620) ERR#2 'No such file or directory'
lstat("Company/General/2006 OLD
ARRIVALS/ARRV112906.xls",0x7fffc620) ERR#2 'No such file or directory'
select(5,{4},{3},0x0,{60 0}) = 1 (0x1)
write(3,0xc2,4092) ERR#1 'Operation not permitted'
rsync: writefd_unbuffered failed to write 4092 bytes [generator]:
Operation not permitted (1)write(2,0x7fffa4f0,93) = 93 (0x5d)
So the write fails on descriptor 3, which I think is the socket to the
daemon. I Googled "FreeBSD write EPERM" and turned up the following
FreeBSD bug, which you might be seeing:
http://www.freebsd.org/cgi/query-pr.cgi?pr=kern%2F95559
I don't have anything more to suggest. The FreeBSD people might be
able to help you on one of their mailing lists:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/eresources.html#ERESOURCES-MAIL
Matt
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[U] Rsync Errors
UNCLASSIFIED I am seeing the following errors occasionally when doing an rsync between servers (Solaris 9) at the local and remote site. Has anyone seen these errors before or know what they are? rsync error: error in rsync protocol data stream (code 12) at io.c(1122) [sender=2.6.9] rsync error: protocol incompatibility (code 2) at io.c(954) [sender=2.6.9] rsync error: protocol incompatibility (code 2) at io.c(960) [sender=2.6.9] rsync error: protocol incompatibility (code 2) at sender.c(174) [sender=2.6.9] rsync error: protocol incompatibility (code 2) at io.c(954) [sender=2.6.9] rsync error: protocol incompatibility (code 2) at io.c(960) [sender=2.6.9] rsync error: protocol incompatibility (code 2) at sender.c(174) [sender=2.6.9] Both servers are using 2.6.9 The only changes that I can think of that have been made is the amount of data is continuously growing. We have had the networking group take a look at their switches and routers bc we see "Broken Pipe" errors soon after we get one of the above errors, but they are not seeing anything on their end. Does anyone have any ideas? Thanks Theresa Peter Unix Team CSI - Data Services ISA-USAAC [EMAIL PROTECTED] (502) 626-0649 UNCLASSIFIED smime.p7s Description: S/MIME cryptographic signature -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync errors
On Tue 25 Mar 2008, Kaushal Shriyan wrote:
>
> I am running the command
>
> /usr/bin/rsync -avO /var/lib/mysql [EMAIL PROTECTED]:/var/lib/ >
> /tmp/rsync-${TIMESTAMP}.log 2>&1
>
> I am getting the below error in the log file
>
> mysql/
> rsync: failed to set permissions on "/var/lib/mysql": Operation not
> permitted (1)
Does the "sms" user on hostaa have permission to set permissions on
/var/lib/mysql? I would guess not, that's probably owned by the "mysql"
user... That's the cause of your error.
Paul Slootman
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync errors
On Tue, Mar 25, 2008 at 10:12 PM, Paul Slootman <[EMAIL PROTECTED]>
wrote:
> On Tue 25 Mar 2008, Kaushal Shriyan wrote:
> >
> > I am running the command
> >
> > /usr/bin/rsync -avO /var/lib/mysql [EMAIL PROTECTED]:/var/lib/ >
> > /tmp/rsync-${TIMESTAMP}.log 2>&1
> >
> > I am getting the below error in the log file
> >
> > mysql/
> > rsync: failed to set permissions on "/var/lib/mysql": Operation not
> > permitted (1)
>
> Does the "sms" user on hostaa have permission to set permissions on
> /var/lib/mysql? I would guess not, that's probably owned by the "mysql"
> user... That's the cause of your error.
>
>
> Paul Slootman
> --
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read:
> http://www.catb.org/~esr/faqs/smart-questions.html
hi
Thanks Paul Slootman
How do i set the "sms" user on hostaa to have permission to set permissions
on
/var/lib/mysql?
I have given #setfacl -R -m u:sms:rwx /var/lib/mysql on hostaa
#setfacl -R -m g:sms:rwx /var/lib/mysql on hostaa
Please let me know if there is anything else to be done
Thanks and Regards
Kaushal
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync errors
On Tue, 2008-03-25 at 23:21 +0530, Kaushal Shriyan wrote: > How do i set the "sms" user on hostaa to have permission to set > permissions on > /var/lib/mysql? > > I have given #setfacl -R -m u:sms:rwx /var/lib/mysql on hostaa > #setfacl -R -m g:sms:rwx /var/lib/mysql on hostaa The second command is unnecessary; granting access to the user "sms" is enough. But you also have to set the default ACL so that "sms" will have permissions on new files created by "mysql". To do this, run the same command plus the -d option: setfacl -R -dm u:sms:rwx /var/lib/mysql You should also add user "mysql" to the access and default ACLs in the same way so that it will have access to files created by "sms". Finally, adding "rwx" entries to the access ACLs of existing files will turn on their S_IXGRP permission bit. To fix that bit, run: chmod -R g-x,g+X /var/lib/mysql At this point, the "failed to open" errors should go away. Rsync may still fail to set the times of /var/lib/mysql because it is owned by user "mysql" and only the owner of a file is allowed to set its times arbitrarily. To get rid of that error, you can pass --omit-dir-times . Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync errors
On Wed, Mar 26, 2008 at 6:21 PM, Matt McCutchen <[EMAIL PROTECTED]> wrote: > On Tue, 2008-03-25 at 23:21 +0530, Kaushal Shriyan wrote: > > How do i set the "sms" user on hostaa to have permission to set > > permissions on > > /var/lib/mysql? > > > > I have given #setfacl -R -m u:sms:rwx /var/lib/mysql on hostaa > > #setfacl -R -m g:sms:rwx /var/lib/mysql on hostaa > > The second command is unnecessary; granting access to the user "sms" is > enough. But you also have to set the default ACL so that "sms" will > have permissions on new files created by "mysql". To do this, run the > same command plus the -d option: > > setfacl -R -dm u:sms:rwx /var/lib/mysql > > You should also add user "mysql" to the access and default ACLs in the > same way so that it will have access to files created by "sms". > Finally, adding "rwx" entries to the access ACLs of existing files will > turn on their S_IXGRP permission bit. To fix that bit, run: > > chmod -R g-x,g+X /var/lib/mysql > > At this point, the "failed to open" errors should go away. Rsync may > still fail to set the times of /var/lib/mysql because it is owned by > user "mysql" and only the owner of a file is allowed to set its times > arbitrarily. To get rid of that error, you can pass --omit-dir-times . > > Matt > > Hi Matt I followed your suggestion, but no luck I did setfacl -R -dm u:sms:rwx /var/lib/mysql on hostaa setfacl -R -dm u:mysql:rwx /var/lib/mysql on hostaa chmod -R g-x,g+X /var/lib/mysql on hostaa I am still getting the below error building file list ... done mysql/ rsync: failed to set permissions on "/var/lib/mysql": Operation not permitted (1) rsync: failed to set permissions on "/var/lib/mysql/.keep_dev-db_mysql-0": Operation not permitted (1) rsync: failed to set permissions on "/var/lib/mysql/host0084-bin.01": Operation not permitted (1) rsync: failed to set permissions on "/var/lib/mysql/host0084-bin.02": Operation not permitted (1) rsync: failed to set permissions on "/var/lib/mysql/host0084-bin.03": Operation not permitted (1) rsync: failed to set permissions on "/var/lib/mysql/host0084-bin.index": Operation not permitted (1) rsync: failed to set permissions on "/var/lib/mysql/host0084-relay-bin.01": Operation not permitted (1) rsync: failed to set permissions on "/var/lib/mysql/host0084-relay-bin.index": Operation not permitted (1) mysql/ib_logfile0 mysql/ib_logfile1 mysql/ibdata1 mysql/master.info rsync: failed to set permissions on "/var/lib/mysql/relay-log.info": Operation not permitted (1) Please let me know if i am missing something Thanks and Regards Kaushal -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync errors
On Thu, Mar 27, 2008 at 7:56 PM, Kaushal Shriyan <[EMAIL PROTECTED]> wrote: > > On Wed, Mar 26, 2008 at 6:21 PM, Matt McCutchen <[EMAIL PROTECTED]> wrote: > > On Tue, 2008-03-25 at 23:21 +0530, Kaushal Shriyan wrote: > > > How do i set the "sms" user on hostaa to have permission to set > > > permissions on > > > /var/lib/mysql? > > > > > > I have given #setfacl -R -m u:sms:rwx /var/lib/mysql on hostaa > > > #setfacl -R -m g:sms:rwx /var/lib/mysql on hostaa > > > > The second command is unnecessary; granting access to the user "sms" is > > enough. But you also have to set the default ACL so that "sms" will > > have permissions on new files created by "mysql". To do this, run the > > same command plus the -d option: > > > > setfacl -R -dm u:sms:rwx /var/lib/mysql > > > > You should also add user "mysql" to the access and default ACLs in the > > same way so that it will have access to files created by "sms". > > Finally, adding "rwx" entries to the access ACLs of existing files will > > turn on their S_IXGRP permission bit. To fix that bit, run: > > > > chmod -R g-x,g+X /var/lib/mysql > > > > At this point, the "failed to open" errors should go away. Rsync may > > still fail to set the times of /var/lib/mysql because it is owned by > > user "mysql" and only the owner of a file is allowed to set its times > > arbitrarily. To get rid of that error, you can pass --omit-dir-times . > > > > Matt > > > > > > Hi Matt > > I followed your suggestion, but no luck > I did setfacl -R -dm u:sms:rwx /var/lib/mysql on hostaa >setfacl -R -dm u:mysql:rwx /var/lib/mysql on hostaa >chmod -R g-x,g+X /var/lib/mysql on hostaa > > I am still getting the below error > > building file list ... done > > mysql/ > rsync: failed to set permissions on "/var/lib/mysql": Operation not > permitted (1) > rsync: failed to set permissions on > "/var/lib/mysql/.keep_dev-db_mysql-0": Operation not permitted (1) > rsync: failed to set permissions on > "/var/lib/mysql/host0084-bin.01": Operation not permitted (1) > rsync: failed to set permissions on > "/var/lib/mysql/host0084-bin.02": Operation not permitted (1) > rsync: failed to set permissions on > "/var/lib/mysql/host0084-bin.03": Operation not permitted (1) > rsync: failed to set permissions on > "/var/lib/mysql/host0084-bin.index": Operation not permitted (1) > rsync: failed to set permissions on > "/var/lib/mysql/host0084-relay-bin.01": Operation not permitted > (1) > rsync: failed to set permissions on > "/var/lib/mysql/host0084-relay-bin.index": Operation not permitted (1) > mysql/ib_logfile0 > mysql/ib_logfile1 > mysql/ibdata1 > mysql/master.info > rsync: failed to set permissions on "/var/lib/mysql/relay-log.info": > Operation not permitted (1) > > Please let me know if i am missing something > > Thanks and Regards > > Kaushal > Hi Matt, Any clue on the issue Thanks Kaushal -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync errors
On Thu, 2008-03-27 at 19:56 +0530, Kaushal Shriyan wrote: > I am still getting the below error > rsync: failed to set permissions on "/var/lib/mysql": Operation not > permitted (1) > [...] User "sms" can't set the permissions of files owned by "mysql"; there's no way to change this policy using ACLs. I think it would make the most sense to stop trying to preserve permissions, since in any event, new destination files need to take on the default permissions you have established (rather than the source permissions) in order for "mysql" to be able to use them. Either pass --no-p or replace -a with the subset of options you do want (perhaps -rlt), and then pass --chmod=ugo=rwX to allow the permissions of new files to increase to the defaults. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync errors
Please ignore this. rsync-path fixed it.Robert <[EMAIL PROTECTED]> wrote: Hi list, I have installed rsync version 2.6.0 on Solaris 9 servers, both the servers have rsync installed in /usr/local/bin directory. When I tried to rsync from one server to other I am getting rsync not found errors, it not the problem with PATH on the source server because rsync by itself coming up with options. bash-2.05$ rsync junk 10.5.2.171:/tmpksh: rsync: not foundrsync: connection unexpectedly closed (0 bytes read so far)rsync error: error in rsync protocol data stream (code 12) at io.c(189) Thanks in advance __Do You Yahoo!?Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsyncBefore posting, read: http://www.catb.org/~esr/faqs/smart-questions.html__Do You Yahoo!?Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: rsync errors
Robert Holtzman wrote: > > Running Ubuntu 10.04.1 and rsync 3.0.7-1. Used rsync successfully for > over a year for backups. Recently there was a kernel upgrade to > 2.6.32-25. Don't know if that has any bearing. When I tried to run a > backup today rsync failed with the error message: > > sending incremental file list > rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: > Broken pipe (32) > rsync: mkdir "/media/disk/10.04laptop" failed: No such file or > directory What does ls -l /media/disk on the target give? rsync must be able to create (ie write) in that directory. > (2) > rsync error: error in file IO (code 11) at main.c(595) [Receiver=3.0.7] > rsync: connection unexpectedly closed (9 bytes received so far) > [sender] > rsync error: error in rsync protocol data stream (code 12) at io.c(601) > [sender=3.0.7] > > /var/log/syslog just showed the above message. Searched on the message > and got one hit that said it was because a FAT32 partition had a size > limit. I doubt that I've come close to that. FAT32 has a 2G file size limit. Some other old file systems have the same limit (32-bit signed) > > Couldn't find anything in this list's archives that matched the error > message. Did I miss something? > > The command that has been working until now is: > > sudo rsync -vaHz --exclude '/proc' --exclude '/sys' --exclude '*.iso' > --exclude '/home/holtzm/Documents/*.iso' --exclude '/media' /. > /media/disk/10.04laptop > > I'm at a loss as to how to proceed. any pointers appreciated. Also check SELinux. It tends to get in the way of anything it does not know about. > > -- > Bob Holtzman > Key ID: 8D549279 > "If you think you're getting free lunch, > check the price of the beer" > -- > Please use reply-all for most replies to avoid omitting the mailing > list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart- > questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync errors out
On Wed, Oct 25, 2000 at 07:55:20AM -0400, Timothy Collins wrote: > RSYNC keeps giving me "error while reading byte # " It then stops the transfer. Any >ideas on this one ? That message appears nowhere in the rsync source code, neither do "error while" nor "while reading". - Dave Dykstra
Re: rsync errors out
I couldn't remember the exact error msg because it only happens on huge transfers so it is hard to reproduce. The error is something like "Error reading byte #". This one happens on 1 of the 2 machines I've worked with. Tim Collins, Server Administrator Gamer's Alliance, Inc. An Euniverse company. http://www.euniverse.com [EMAIL PROTECTED] - Original Message - From: "Dave Dykstra" <[EMAIL PROTECTED]> To: "Timothy Collins" <[EMAIL PROTECTED]> Cc: "rsync list" <[EMAIL PROTECTED]> Sent: Wednesday, October 25, 2000 10:26 AM Subject: Re: rsync errors out > On Wed, Oct 25, 2000 at 07:55:20AM -0400, Timothy Collins wrote: > > RSYNC keeps giving me "error while reading byte # " It then stops the transfer. Any ideas on this one ? > > That message appears nowhere in the rsync source code, neither do "error > while" nor "while reading". > > - Dave Dykstra >
Re: [U] Rsync Errors
On 9/4/07, Peter, Theresa M Ms CONT USAAC <[EMAIL PROTECTED]> wrote: > I am seeing the following errors occasionally when doing an rsync between > servers (Solaris 9) at the local and remote site. Has anyone seen these > errors before or know what they are? > > rsync error: error in rsync protocol data stream (code 12) at io.c(1122) > [sender=2.6.9] > rsync error: protocol incompatibility (code 2) at io.c(954) [sender=2.6.9] > rsync error: protocol incompatibility (code 2) at io.c(960) [sender=2.6.9] > rsync error: protocol incompatibility (code 2) at sender.c(174) > [sender=2.6.9] > rsync error: protocol incompatibility (code 2) at io.c(954) [sender=2.6.9] > rsync error: protocol incompatibility (code 2) at io.c(960) [sender=2.6.9] > rsync error: protocol incompatibility (code 2) at sender.c(174) > [sender=2.6.9] > > Both servers are using 2.6.9 > We have had the networking group take a look at their switches and routers > bc we see "Broken Pipe" errors soon after we get one of the above errors, > but they are not seeing anything on their end. The "Broken Pipe" and the "error in rsync protocol data stream" are both aftermath of a previously occurring error. The other errors all mean that the sender received something invalid from the generator. That could indicate either a bug in rsync or corruption in the network connection. You could dump the data that each side sends and receives over the network using the "savetransfer" program that comes in support/ of the rsync source package ( http://rsync.samba.org/ftp/rsync/rsync-2.6.9.tar.gz ) to determine whether network corruption is occurring. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[U] RE: Rsync Errors
UNCLASSIFIED I am still trying to figure out whether or not we have that savetransfer option already on the box. It may take me a bit to get a new package installed due to software cycle that we have to go through before putting it on the box. Looking into this more... The following errors were the first that we saw: rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe (32) rsync error: error in rsync protocol data stream (code 12) at io.c(1122) [sender=2.6.9] rsync: read errors mapping "/soinc/soinc_std_001" (in CYA): No data available (61) rsync error: some files could not be transferred (code 23) at main.c(683) [sender=2.6.9] Since then we have been seeing the following errors on every rsync: Invalid checksum length 39976960 [sender] Variable lengths listed rsync error: protocol incompatibility (code 2) at io.c(954) [sender=2.6.9] Invalid remainder length 39976960 [sender] rsync error: protocol incompatibility (code 2) at io.c(960) [sender=2.6.9] Invalid file index: 2046101527 (count=1730) [sender] rsync error: protocol incompatibility (code 2) at sender.c(174) [sender=2.6.9] I am hoping that with the additional log entries...someone might have an idea what is going on and the best way to get it back to normal. Theresa Peter Unix Team CSI - Data Services ISA-USAAC [EMAIL PROTECTED] (502) 626-0649 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt McCutchen Sent: Tuesday, September 04, 2007 2:31 PM To: Peter, Theresa M Ms CONT USAAC Cc: rsync@lists.samba.org Subject: Re: [U] Rsync Errors On 9/4/07, Peter, Theresa M Ms CONT USAAC <[EMAIL PROTECTED]> wrote: > I am seeing the following errors occasionally when doing an rsync > between servers (Solaris 9) at the local and remote site. Has anyone > seen these errors before or know what they are? > > rsync error: error in rsync protocol data stream (code 12) at > io.c(1122) [sender=2.6.9] rsync error: protocol incompatibility (code > 2) at io.c(954) [sender=2.6.9] rsync error: protocol incompatibility > (code 2) at io.c(960) [sender=2.6.9] rsync error: protocol > incompatibility (code 2) at sender.c(174) [sender=2.6.9] rsync error: > protocol incompatibility (code 2) at io.c(954) [sender=2.6.9] rsync > error: protocol incompatibility (code 2) at io.c(960) [sender=2.6.9] > rsync error: protocol incompatibility (code 2) at sender.c(174) > [sender=2.6.9] > > Both servers are using 2.6.9 > We have had the networking group take a look at their switches and > routers bc we see "Broken Pipe" errors soon after we get one of the > above errors, but they are not seeing anything on their end. The "Broken Pipe" and the "error in rsync protocol data stream" are both aftermath of a previously occurring error. The other errors all mean that the sender received something invalid from the generator. That could indicate either a bug in rsync or corruption in the network connection. You could dump the data that each side sends and receives over the network using the "savetransfer" program that comes in support/ of the rsync source package ( http://rsync.samba.org/ftp/rsync/rsync-2.6.9.tar.gz ) to determine whether network corruption is occurring. Matt UNCLASSIFIED smime.p7s Description: S/MIME cryptographic signature -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Identifying rsync errors programmatically
Can anyone give me a canonical list of all the possible rsync error messages? I have a script which reads and parses the rsync verbose output with progress so as to output a rather more customer-friendly output and log during the process, and I'm having some difficulty making sure I successfully pass on all error codes WITHOUT passing through the occasional file that has "fail" or "error" in the name. If anybody has already met this challenge and has a really good filter, or can just tell me what all the various rsync errors look like so that I can write one myself with confidence, it would be very helpful indeed. ... I would also humbly suggest STANDARDIZING rsync error outputs in a future version, so that perhaps they all begain with "rsync error: " or some other easily identifiable string. Right now they're, well, all over the place. Thanks all! Jim Salter JRS Systems -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync errors[SOLVED]
On Tue, Sep 21, 2010 at 04:59:39AM -0400, Tony Abernethy wrote: > Robert Holtzman wrote: > > > > Running Ubuntu 10.04.1 and rsync 3.0.7-1. Used rsync successfully for > > over a year for backups. Recently there was a kernel upgrade to > > 2.6.32-25. Don't know if that has any bearing. When I tried to run a > > backup today rsync failed with the error message: > > > > sending incremental file list > > rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: > > Broken pipe (32) > > rsync: mkdir "/media/disk/10.04laptop" failed: No such file or > > directory > > What does > ls -l /media/disk > on the target give? rsync must be able to create (ie write) in that > directory. Sorry to be late with this reply. I didn't think it was a permission problem and it turned out not to be. I had renamed the target directory and forgot to change the backup script. In addition, prior to the upgrade to 10.04, the usb hd showed up as "disk". After the upgrade it had changed to the UUID and, of course the script still reflected "disk". Once this was corrected it ran perfectly. Many thanks. -- Bob Holtzman Key ID: 8D549279 "If you think you're getting free lunch, check the price of the beer" signature.asc Description: Digital signature -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: [U] RE: Rsync Errors
On Tue, Sep 04, 2007 at 05:56:11PM -0400, Peter, Theresa M Ms CONT USAAC wrote: >Invalid checksum length 39976960 [sender] >Invalid remainder length 39976960 [sender] >Invalid file index: 2046101527 (count=1730) [sender] These are all symptoms of the same problem -- the binary data protocol is no longer in sync. Whether it is caused by dropped bytes in the connection or a programatical error on rsync's part is the big question, and can't be determined by us. Are you using the -z (--compress) option? If so, try turning it off. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[U] RE: RE: Rsync Errors
UNCLASSIFIED We are not using the -z or --compress option. I have turned up the logging when running a job but I am not sure what I should be seeing. I have the log from both the machine that is pulling the data and the machine that the data is being pulled from. On the machine pulling data... It looks like that we get an "invalid file index" and then the job dies out. On the machine data is being pulled... I am seeing renaming of files, changing of modtimes, recv_files and recv_generator going at the same time. Then our final error: 2007/09/05 10:28:10 [15233] rsync: connection unexpectedly closed (165735 bytes received so far) [generator] 2007/09/05 10:28:10 [15233] rsync error: error in rsync protocol data stream (code 12) at io.c(453) [generator=2.6.9] 2007/09/05 10:28:10 [15233] _exit_cleanup(code=12, file=io.c, line=453): about to call exit(12) Any more ideas? What is going to be the best way to get us back on track? Should I restart the daemon, would that help anything; I wouldn't think so if it is a file problem with index? Theresa Peter Unix Team CSI - Data Services ISA-USAAC [EMAIL PROTECTED] (502) 626-0649 -Original Message- From: Wayne Davison [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 04, 2007 6:12 PM To: Peter, Theresa M Ms CONT USAAC Cc: rsync@lists.samba.org Subject: Re: [U] RE: Rsync Errors On Tue, Sep 04, 2007 at 05:56:11PM -0400, Peter, Theresa M Ms CONT USAAC wrote: >Invalid checksum length 39976960 [sender] >Invalid remainder length 39976960 [sender] >Invalid file index: 2046101527 (count=1730) [sender] These are all symptoms of the same problem -- the binary data protocol is no longer in sync. Whether it is caused by dropped bytes in the connection or a programatical error on rsync's part is the big question, and can't be determined by us. Are you using the -z (--compress) option? If so, try turning it off. ..wayne.. UNCLASSIFIED smime.p7s Description: S/MIME cryptographic signature -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Detecting reason for rsync errors
Hello guys! I'm doing a middle scale backup job using ccollect [0], which uses rsyn, with about 50 servers, each about 1-10 GiB changes per day and have some problems, that servers do not finish their backup completly: [host1.backup] Read from remote host host1.backup: Connection reset by peer [host1.backup] rsync: connection unexpectedly closed (41306592 bytes received so far) [receiver] [host1.backup] rsync error: error in rsync protocol data stream (code 12) at io.c(458) [receiver=2.6.9] [host1.backup] rsync: connection unexpectedly closed (21302116 bytes received so far) [generator] [host1.backup] rsync error: unexplained error (code 255) at io.c(458) [generator=2.6.9] [host1.backup] 2007-11-19-06:59:24: Finished backup (rsync return code: 255). How can I debug best why the remote server, still running, seems to close the connection? Sincerly Nico [0]: http://unix.schottelius.org/ccollect/ -- Think about Free and Open Source Software (FOSS). http://nico.schottelius.org/documentations/foss/the-term-foss/ PGP: BFE4 C736 ABE5 406F 8F42 F7CF B8BE F92A 9885 188C signature.asc Description: Digital signature -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
capturing or suppressing rsync errors
Hello rsync, I am using rsync to download files from rpm repositories (because yum will not work directly with my [EMAIL PROTECTED] ADSL service). I am calling rsync from a python script. When the rsync server is down, I get error messages, but the message goes to mail, so I get a message you have new mail in /var/spool/mail/rick I would prefer the error message would not go to mail, rather I would like to capture the text of the message in python. However, the things I have tried so far do not work. Any suggestions? Thanks, Rick Graves -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
osx 10.6 strange rsync errors
Hi All, I have been getting reports from users of backuplist+, my wrapper application for rsync (currently with build of 3.0.6), about odd behavior after updating to OS 10.6 Snow Leopard. Basically: the problem occurs backing up a directory to a local mounted network volume. Previously all worked fine but after updating to 10.6 there are reports that no files get copied and the destination fills up with thousands (yes thousands) of .DS-Store files which on OSX are invisible and carry metadata about the directory that contains them (usually just one per directory...) The destination files are visible and labeled ..DS_Store.0aecRT ..DS_Store.0afcRT ..DS_Store.0cecRT ..DS_Store.0cfcRT (Note the "two dots" prefix?) on an on and on. One user reported 97,000 of these with seemingly random names attached. rsync log shows >> file has vanished: "/Volumes/MacMini/Users/rg/..DS_Store.ZxgavW" >> file has vanished: "/Volumes/MacMini/Users/rg/..DS_Store.ZygavW" >> file has vanished: "/Volumes/MacMini/Users/rg/..DS_Store.ZzgavW" So before any files are copied it fails on the "missing" files that aren't there in the first place! The basic command line is pathtoRsync -aHAXN --fileflags --force-change --stats -v -u --progress pathttoSourceDirectory pathToDestinationNetworkDirectory &> pathToOutputLogFile echo $! This works fine when going to an external mounted Volume or a partition, just not to a local mounted Network Volume. I haven't been able to get any more detailed information from users so far. Has anyone seen this or have a clue to what's happening? Something to do with the Network and DS_Store files it seems. I'm not sure where to even begin. Thanks, Rob DuToit -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[U] RE: RE: RE: Rsync Errors
UNCLASSIFIED We were using compression on the network; through using a WAS device. We have turned that off and we are no longer seeing the problem. Why would compression cause these issues? If we turn the WAS (network compression) back on is this going to cause more issues. What do you mean by "the binary data protocol is no longer in sync"? Thanks Theresa Peter Unix Team CSI - Data Services ISA-USAAC [EMAIL PROTECTED] (502) 626-0649 -Original Message- From: Peter, Theresa M Ms CONT USAAC Sent: Wednesday, September 05, 2007 10:46 AM To: 'Wayne Davison'; 'rsync@lists.samba.org' Subject: [U] RE: RE: Rsync Errors UNCLASSIFIED We are not using the -z or --compress option. I have turned up the logging when running a job but I am not sure what I should be seeing. I have the log from both the machine that is pulling the data and the machine that the data is being pulled from. On the machine pulling data... It looks like that we get an "invalid file index" and then the job dies out. On the machine data is being pulled... I am seeing renaming of files, changing of modtimes, recv_files and recv_generator going at the same time. Then our final error: 2007/09/05 10:28:10 [15233] rsync: connection unexpectedly closed (165735 bytes received so far) [generator] 2007/09/05 10:28:10 [15233] rsync error: error in rsync protocol data stream (code 12) at io.c(453) [generator=2.6.9] 2007/09/05 10:28:10 [15233] _exit_cleanup(code=12, file=io.c, line=453): about to call exit(12) Any more ideas? What is going to be the best way to get us back on track? Should I restart the daemon, would that help anything; I wouldn't think so if it is a file problem with index? Theresa Peter Unix Team CSI - Data Services ISA-USAAC [EMAIL PROTECTED] (502) 626-0649 -Original Message- From: Wayne Davison [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 04, 2007 6:12 PM To: Peter, Theresa M Ms CONT USAAC Cc: rsync@lists.samba.org Subject: Re: [U] RE: Rsync Errors On Tue, Sep 04, 2007 at 05:56:11PM -0400, Peter, Theresa M Ms CONT USAAC wrote: >Invalid checksum length 39976960 [sender] >Invalid remainder length 39976960 [sender] >Invalid file index: 2046101527 (count=1730) [sender] These are all symptoms of the same problem -- the binary data protocol is no longer in sync. Whether it is caused by dropped bytes in the connection or a programatical error on rsync's part is the big question, and can't be determined by us. Are you using the -z (--compress) option? If so, try turning it off. ..wayne.. UNCLASSIFIED UNCLASSIFIED smime.p7s Description: S/MIME cryptographic signature -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Detecting reason for rsync errors
On Mon, 2007-11-19 at 11:01 +0100, Nico -telmich- Schottelius wrote: > Hello guys! > > I'm doing a middle scale backup job using ccollect [0], which uses > rsyn, with about 50 servers, each about 1-10 GiB changes per day and have > some problems, that servers do not finish their backup completly: > > [host1.backup] Read from remote host host1.backup: Connection reset by peer > [host1.backup] rsync: connection unexpectedly closed (41306592 bytes received > so far) [receiver] try increase the verbose level at daemon side and when u see this error again, check daemon log. > [host1.backup] rsync error: error in rsync protocol data stream (code 12) at > io.c(458) [receiver=2.6.9] > [host1.backup] rsync: connection unexpectedly closed (21302116 bytes received > so far) [generator] > [host1.backup] rsync error: unexplained error (code 255) at io.c(458) > [generator=2.6.9] > [host1.backup] 2007-11-19-06:59:24: Finished backup (rsync return code: 255). > > How can I debug best why the remote server, still running, seems to close the > connection? > > Sincerly > > Nico > > [0]: http://unix.schottelius.org/ccollect/ > > -- > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: capturing or suppressing rsync errors
On 10/2/06, Rick Graves <[EMAIL PROTECTED]> wrote: I am using rsync to download files from rpm repositories (because yum will not work directly with my [EMAIL PROTECTED] ADSL service). I am calling rsync from a python script. When the rsync server is down, I get error messages, but the message goes to mail, so I get a message you have new mail in /var/spool/mail/rick I would prefer the error message would not go to mail, rather I would like to capture the text of the message in python. However, the things I have tried so far do not work. I suppose the python script is being run through cron and rsync inherits the script's stderr, so cron converts rsync's error messages to mail? How to capture the rsync process's stderr is a Python question, not an rsync one. Invoking rsync with popen3 should trap the error messages and give you a stream from which you can read them. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
With rsync 3.0.7 I have exactly the same problem. I am using rSync to sync my Userdirectory to another Mac in a home network. I am not 100% sure, but the problem occurred after the latest update of the MacOS (Mac OS X 10.6.3 (10D573)) of the Source. My macmini (the Dest of rSync) is also running (Mac OS X 10.6.3 (10D573)). I tried several options but till now without any results. Eric -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
Hi Eric et al, It seems this is not an rsync problem though I am not sure The same thing happens with the Apple supplied rsync as with 3.0.7 (someone emailed me about the same problem using the old apple supplied rsync) and I wonder if other file transfer options might have similar results. I am still mystified about the cause of this except that it started with the latest OS update. Perhaps having to do with security "enhancements?" This is a big problem as administrators have been using rysnc to sync their network machines and everything is failing suddenly. Any one else have a clue? Rob On Apr 4, 2010, at 3:34 PM, Eric Jan Pot wrote: > With rsync 3.0.7 I have exactly the same problem. > > I am using rSync to sync my Userdirectory to another Mac in a home network. > I am not 100% sure, but the problem occurred after the latest update of the > MacOS (Mac OS X 10.6.3 (10D573)) of the Source. > > My macmini (the Dest of rSync) is also running (Mac OS X 10.6.3 (10D573)). > > I tried several options but till now without any results. > > Eric > > > > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
On Sat, 2010-04-03 at 18:12 -0400, Robert DuToit wrote: > I have been getting reports from users of backuplist+, my wrapper > application for rsync (currently with build of 3.0.6), about odd > behavior after updating to OS 10.6 Snow Leopard. > > Basically: the problem occurs backing up a directory to a local > mounted network volume. Previously all worked fine but after updating > to 10.6 there are reports that no files get copied and the destination > fills up with thousands (yes thousands) of .DS-Store files which on > OSX are invisible and carry metadata about the directory that contains > them (usually just one per directory...) In order to investigate this phenomenon from the rsync side, we'll need strace output (or whatever the Mac equivalent is). -- Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
On Apr 5, 2010, at 1:28 PM, Matt McCutchen wrote: > On Sat, 2010-04-03 at 18:12 -0400, Robert DuToit wrote: >> I have been getting reports from users of backuplist+, my wrapper >> application for rsync (currently with build of 3.0.6), about odd >> behavior after updating to OS 10.6 Snow Leopard. >> >> Basically: the problem occurs backing up a directory to a local >> mounted network volume. Previously all worked fine but after updating >> to 10.6 there are reports that no files get copied and the destination >> fills up with thousands (yes thousands) of .DS-Store files which on >> OSX are invisible and carry metadata about the directory that contains >> them (usually just one per directory...) > > In order to investigate this phenomenon from the rsync side, we'll need > strace output (or whatever the Mac equivalent is). Matt Not sure what Strace is... would that be the Mac console and crash reporter logs? I am trying to get some more verbose rsync error reporting from users. So far it seems rsync is reporting missing files (the hidden dot files on the mac) and then mysteriously creating thousands of hidden files into directories. These are files that don't exist - they are being created somehow! It gets worse and worse apparently. I'll rig up a network myself and get back to you with some details of what is happening. Rob > > -- > Matt > -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
Hi Matt, I set up a simple test with a nest of directories ( aa > bb > cc > dd &ee) with 1 file in each. running rsync from OS 10.6 to another Mac with OS10.5 there seems to be no problem. When doing the reverse I am seeing the odd behavior. Below is the log from running the options -aHAXN --fileflags --force-change -stats -vvv note that the lines "rsync: mkstemp "/Volumes/xxx/testaaa/aa/..DS_Store.gnG29l" failed: No such file or directory (2)" are correct in that there are no such files but why does it think there are? They don't exist on source or dest. They do however get created on the destination. It seems in my test that there is one created for every directory. However, the original .DS_Store files are not copied over just the new ones with the odd extra dot prefix and the cryptic extension on the end. The new odd .DS_Store files take the place of the normal ones that should be created by the finder. I don't know if this sheds any light at all on this. Others have reported that these files get created by the thousands on the destination. Not only .DS_Store files but other hidden files. Rob sending incremental file list [sender] make_file(aa,*,2) send_file_list done [sender] make_file(aa/.DS_Store,*,2) [sender] make_file(aa/bb,*,2) [sender] make_file(aa/zwhite.icns,*,2) send_files starting [sender] make_file(aa/bb/.DS_Store,*,2) [sender] make_file(aa/bb/cc,*,2) [sender] make_file(aa/bb/excludes.txt,*,2) [sender] make_file(aa/bb/cc/.DS_Store,*,2) [sender] make_file(aa/bb/cc/dd,*,2) [sender] make_file(aa/bb/cc/ee,*,2) [sender] make_file(aa/bb/cc/size.txt,*,2) [sender] make_file(aa/bb/cc/dd/.DS_Store,*,2) [sender] make_file(aa/bb/cc/dd/splittersize.h,*,2) server_recv(2) starting pid=939 recv_file_name(aa) received 1 names recv_file_list done recv_file_name(aa/.DS_Store) recv_file_name(aa/bb) recv_file_name(aa/zwhite.icns) received 3 names recv_file_list done get_local_name count=4 /Volumes/xxx/testaaa/ generator starting pid=939 delta-transmission disabled for local transfer or --whole-file recv_files(1) starting recv_file_name(aa/bb/.DS_Store) recv_file_name(aa/bb/cc) recv_file_name(aa/bb/excludes.txt) received 3 names recv_file_list done recv_file_name(aa/bb/cc/.DS_Store) recv_file_name(aa/bb/cc/dd) recv_file_name(aa/bb/cc/ee) recv_file_name(aa/bb/cc/size.txt) received 4 names recv_file_list done recv_file_name(aa/bb/cc/dd/.DS_Store) recv_file_name(aa/bb/cc/dd/splittersize.h) received 2 names recv_file_list done received 0 names recv_file_list done recv_generator(aa,1) recv_generator(aa,2) send_files(2, /Users/astrid/Desktop/aa) aa/ set modtime of aa to (1270518365) Mon Apr 5 21:46:05 2010 recv_generator(aa/.DS_Store,3) recv_files(aa) send_files(3, /Users/astrid/Desktop/aa/.DS_Store) send_files mapped /Users/astrid/Desktop/aa/.DS_Store of size 6148 calling match_sums /Users/astrid/Desktop/aa/.DS_Store aa/.DS_Store 6148 100%0.00kB/s0:00:00 sending file_sum false_alarms=0 hash_hits=0 matches=0 6148 100%0.00kB/s0:00:00 (xfer#1, to-check=11/13) sender finished /Users/astrid/Desktop/aa/.DS_Store recv_generator(aa/zwhite.icns,4) send_files(4, /Users/astrid/Desktop/aa/zwhite.icns) send_files mapped /Users/astrid/Desktop/aa/zwhite.icns of size 54611 calling match_sums /Users/astrid/Desktop/aa/zwhite.icns aa/zwhite.icns 32768 60% 820.51kB/s0:00:00 sending file_sum false_alarms=0 hash_hits=0 matches=0 54611 100%1.21MB/s0:00:00 (xfer#2, to-check=10/13) sender finished /Users/astrid/Desktop/aa/zwhite.icns recv_generator(aa/bb,5) recv_generator(aa/bb,6) recv_files(aa/.DS_Store) send_files(6, /Users/astrid/Desktop/aa/bb) aa/bb/ set modtime of aa/bb to (1270518373) Mon Apr 5 21:46:13 2010 recv_generator(aa/bb/.DS_Store,7) rsync: mkstemp "/Volumes/xxx/testaaa/aa/..DS_Store.gnG29l" failed: No such file or directory (2) send_files(7, /Users/astrid/Desktop/aa/bb/.DS_Store) send_files mapped /Users/astrid/Desktop/aa/bb/.DS_Store of size 6148 calling match_sums /Users/astrid/Desktop/aa/bb/.DS_Store aa/bb/.DS_Store 6148 100% 103.52kB/s0:00:00 sending file_sum false_alarms=0 hash_hits=0 matches=0 6148 100% 101.76kB/s0:00:00 (xfer#3, to-check=8/13) sender finished /Users/astrid/Desktop/aa/bb/.DS_Store recv_generator(aa/bb/excludes.txt,8) send_files(8, /Users/astrid/Desktop/aa/bb/excludes.txt) send_files mapped /Users/astrid/Desktop/aa/bb/excludes.txt of size 285 calling match_sums /Users/astrid/Desktop/aa/bb/excludes.txt aa/bb/excludes.txt 285 100%4.42kB/s0:00:00 sending file_sum false_alarms=0 hash_hits=0 matches=0 285 100%4.35kB/s0:00:00 (xfer#4, to-check=7/13) sender finished /Users/astrid/Desktop/aa/bb/excludes.txt got file_sum recv_generator(aa/bb/cc,9) recv_generator(aa/bb/cc,10) send_files(10, /Users/astrid/Desktop/aa/bb/cc) aa/bb/cc/ recv_files(aa/zwhite.icns) set modtime of aa/bb/cc to (1270518390) Mon Apr 5 21:46:30 2010 recv_generator(aa/bb/cc/.DS_Store,11) sen
Re: osx 10.6 strange rsync errors
It's the Easter miracle! :P Sorry about joking in a serious list, hope you won't get angry. Although I don't have suggestions about the problem, I'm following the thread because I'm potentially concerned about rsync on Mac bugs. Cheers! Vitorio Le 6 avr. 10 à 00:09, Robert DuToit a écrit : (...) So far it seems rsync is (...) mysteriously creating thousands of hidden files into directories. These are files that don't exist - they are being created somehow! It gets worse and worse apparently. I'll rig up a network myself and get back to you with some details of what is happening. Rob -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
Matt, I forgot to mention, If I add an exclude for .DS_Store* it doesn't generate the odd .DS_Store files. However, if I copy directories with other types of hidden files like .bash_history then it generates odd versions of those too. So I added an exclude for every hidden (dot prefix) file .* and it seems to work ok. This might be ok as an emergency measure of course but not desirable since a lot of hidden files are needed and at root level indispensable. Rob -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
On Mon, 5 Apr 2010, Robert DuToit wrote: > Hi Matt, > > I set up a simple test with a nest of directories ( aa > bb > cc > dd > &ee) with 1 file in each. running rsync from OS 10.6 to another Mac > with OS10.5 there seems to be no problem. When doing the reverse I am > seeing the odd behavior. > > Below is the log from running the options > > -aHAXN --fileflags --force-change -stats -vvv > > note that the lines > > "rsync: mkstemp "/Volumes/xxx/testaaa/aa/..DS_Store.gnG29l" failed: No > such file or directory (2)" > > are correct in that there are no such files but why does it think > there are? Those files are rsync's temporary files, which it transfers first, then atomically moves into place. Note the many other entries like: /path/to/.ORIGINALNAME.XX e.g.: ORIGINALNAME = splittersize.h and XX = hVRihb in the following: > set modtime of aa/bb/cc/dd/.splittersize.h.hVRihb to (1236212797) Wed Mar 4 > 19:26:37 2009 > renaming aa/bb/cc/dd/.splittersize.h.hVRihb to aa/bb/cc/dd/splittersize.h So, it appears something has changed between 10.5 and 10.6 with how the annoying-to-non-OSX users .DS_Store file is handled. I just ran into __MACOSX/.DS_Store files in a ZIP file the other day. Annoying because they force me to have to filter out the garbage files: unzip file.zip -x '__MACOSX/*' A possibly interesting (probably not) experiment would be to create ZIP files on each machine, then open them on a non-OSX machine to see whether 10.6 creates a file without the .DS_Store files. See: http://en.wikipedia.org/wiki/.DS_Store for the purpose of the files. (basically, per-folder Finder preferences) > They don't exist on source or dest. They do however get created on the > destination. It seems in my test that there is one created for every > directory. However, the original .DS_Store files are not copied over > just the new ones with the odd extra dot prefix and the cryptic > extension on the end. The new odd .DS_Store files take the place of > the normal ones that should be created by the finder. > > I don't know if this sheds any light at all on this. Others have > reported that these files get created by the thousands on the > destination. Not only .DS_Store files but other hidden files. My guess is that one ..DS_Store.XX file is created per-directory per-transfer. Other hidden files are likely created during failed transfers. If something changed between 10.5 and 10.6 regarding default permissions on .DS_Store files, that might explain it. And maybe the files are created automatically at an earlier time (so can't be overwritten by the rename). But I'm not sure what the best solution would be. The .DS_Store files may or may not be something each user would want to keep. -- Best, Ben -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
On Apr 6, 2010, at 9:14 AM, Benjamin R. Haskell wrote: > On Mon, 5 Apr 2010, Robert DuToit wrote: > >> Hi Matt, >> >> I set up a simple test with a nest of directories ( aa > bb > cc > dd >> &ee) with 1 file in each. running rsync from OS 10.6 to another Mac >> with OS10.5 there seems to be no problem. When doing the reverse I am >> seeing the odd behavior. >> >> Below is the log from running the options >> >> -aHAXN --fileflags --force-change -stats -vvv >> >> note that the lines >> >> "rsync: mkstemp "/Volumes/xxx/testaaa/aa/..DS_Store.gnG29l" failed: No >> such file or directory (2)" >> >> are correct in that there are no such files but why does it think >> there are? > > Those files are rsync's temporary files, which it transfers first, then > atomically moves into place. Note the many other entries like: Yes I see now that something in that process of temp files goes awry. > > /path/to/.ORIGINALNAME.XX > > e.g.: ORIGINALNAME = splittersize.h and XX = hVRihb in the following: >> set modtime of aa/bb/cc/dd/.splittersize.h.hVRihb to (1236212797) Wed Mar 4 >> 19:26:37 2009 >> renaming aa/bb/cc/dd/.splittersize.h.hVRihb to aa/bb/cc/dd/splittersize.h > > So, it appears something has changed between 10.5 and 10.6 with how the > annoying-to-non-OSX users .DS_Store file is handled. I just ran into > __MACOSX/.DS_Store files in a ZIP file the other day. Annoying because > they force me to have to filter out the garbage files: > unzip file.zip -x '__MACOSX/*' > > A possibly interesting (probably not) experiment would be to create ZIP > files on each machine, then open them on a non-OSX machine to see > whether 10.6 creates a file without the .DS_Store files. > > See: http://en.wikipedia.org/wiki/.DS_Store for the purpose of the > files. (basically, per-folder Finder preferences) > > >> They don't exist on source or dest. They do however get created on the >> destination. It seems in my test that there is one created for every >> directory. However, the original .DS_Store files are not copied over >> just the new ones with the odd extra dot prefix and the cryptic >> extension on the end. The new odd .DS_Store files take the place of >> the normal ones that should be created by the finder. >> >> I don't know if this sheds any light at all on this. Others have >> reported that these files get created by the thousands on the >> destination. Not only .DS_Store files but other hidden files. > > My guess is that one ..DS_Store.XX file is created per-directory > per-transfer. Other hidden files are likely created during failed > transfers. > > If something changed between 10.5 and 10.6 regarding default permissions > on .DS_Store files, that might explain it. And maybe the files are > created automatically at an earlier time (so can't be overwritten by the > rename). But I'm not sure what the best solution would be. The > .DS_Store files may or may not be something each user would want to > keep. You can live without the .DS_Store files though they do hold directory info but the many root level hidden files are part of the OS. It seems any hidden files (with a dot prefix) cause this to happen - though I haven't tested yet at root level. I checked out dtrace (OSX version of strace) but it seems a bit complicated and I don't know how to use it yet. If anyone has maybe they can let me know. Thanks, Rob > > -- > Best, > Ben -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
On Apr 7, 2010, at 5:59 AM, carson+rs...@taltos.org wrote: > Robert DuToit wrote: >> On Apr 6, 2010, at 9:14 AM, Benjamin R. Haskell wrote: >> >>> On Mon, 5 Apr 2010, Robert DuToit wrote: >>> Hi Matt, I set up a simple test with a nest of directories ( aa > bb > cc > dd &ee) with 1 file in each. running rsync from OS 10.6 to another Mac with OS10.5 there seems to be no problem. When doing the reverse I am seeing the odd behavior. Below is the log from running the options -aHAXN --fileflags --force-change -stats -vvv > > Please specify _exactly_ how you're building rsync. Those options don't exist > without applying some extra patches. I can't replicate the problem with stock > rsync, which means it's either a problem with the patches, or I'm not > triggering the bug properly. The basic OSX patches which are patch -p1 > Can you replicate this without a 10.5 box? Users are all on 10.6 and having the errors. I am only using 10.5 to copy over to my 10.6 box because the old laptop is not intel (no 10.6 possible). But user reports and my test show no issues with earlier OS and its seems that the new release of 10.6.3 is the trouble. 10.6.2 seems ok for users. > Can you replicate it with Apple's rsync? Unpatched 3.0.7 rsync? Patched 3.0.7 > rsync? Nightly snapshot? I really don't want to spend time investigating an > old release, and I need to be able to reproduce the problem before I can help > fix it. > > I have 10.6 and OpenSolaris easily at hand to test with, but no 10.5 boxes. > >> I checked out dtrace (OSX version of strace) but it seems a bit >> complicated and I don't know how to use it yet. If anyone has maybe >> they can let me know. > > sudo dtruss -f /path/to/rsync --whatever src dest 2>/tmp/mydtrusslog > > should dump all the syscalls into /tmp/mydtrusslog I'll give it a try Thanks, Rob > > -- > Carson > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
Hi Carson, OK no differences: Build 3.0.7 with or without patches Build 3.0.6 with or without patches Apple rsync (as far as Users report all have same issues with hidden files on OSX 10.6.3 This is transferring over local ethernet network to another Mac OSX 10.6.3 ( Again I had to use my old 10.5 box to send but all users are reporting issues with all 10.6.3 machines) basic Mac options path/to/rsync -aHAXN --fileflags --force-change -stats - src dst output Rob On Apr 7, 2010, at 5:59 AM, carson+rs...@taltos.org wrote: > Robert DuToit wrote: > > On Apr 6, 2010, at 9:14 AM, Benjamin R. Haskell wrote: > > > >> On Mon, 5 Apr 2010, Robert DuToit wrote: > >> > >>> Hi Matt, > >>> > >>> I set up a simple test with a nest of directories ( aa > bb > cc > >>> > dd &ee) with 1 file in each. running rsync from OS 10.6 to > >>> another Mac with OS10.5 there seems to be no problem. When doing > >>> the reverse I am seeing the odd behavior. > >>> > >>> Below is the log from running the options > >>> > >>> -aHAXN --fileflags --force-change -stats -vvv > > Please specify _exactly_ how you're building rsync. Those options don't exist > without applying some extra patches. I can't replicate the problem with stock > rsync, which means it's either a problem with the patches, or I'm not > triggering the bug properly. > > Can you replicate this without a 10.5 box? Can you replicate it with Apple's > rsync? Unpatched 3.0.7 rsync? Patched 3.0.7 rsync? Nightly snapshot? I really > don't want to spend time investigating an old release, and I need to be able > to reproduce the problem before I can help fix it. > > I have 10.6 and OpenSolaris easily at hand to test with, but no 10.5 boxes. > > > I checked out dtrace (OSX version of strace) but it seems a bit > > complicated and I don't know how to use it yet. If anyone has maybe > > they can let me know. > > sudo dtruss -f /path/to/rsync --whatever src dest 2>/tmp/mydtrusslog > > should dump all the syscalls into /tmp/mydtrusslog > > -- > Carson > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
> Basically: the problem occurs backing up a directory to a local mounted > network volume. If you are using the -a flag then you may have problems preserving the permissions of files on the mounted network directory. This will greatly depend upon you or your users setup. Unfortunately, I do not believe this is the problem you are seeing. -- This email is protected by LBackup http://www.lbackup.org -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
Hi All, I reported a ways back about strange file replication via rsync copying to an OSX 10.6.3 local network volume. Hidden dot files such as .DS-Store and other "dot" files on OSX were copied to the local volume with appended names indicating that the temp files created by rsync were ending up on the destination instead of the originals, often by the thousands. Ok, so I am back on the case but now can not recreate this issue. I am now running OSX 10.6.4. Has anyone else noticed that this problem has ceased with 10.6.4 or is it still happening? I tried builds of rsync 3.0.6 and 3.0.7 and both seem ok now…. If it is fixed with the latest OS release - so much the better. News? Thanks, rob -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
I've recently encountered this issue which was discussed here about a year ago.
I'm not sure if anyone has a fix for this, but I thought I would post my
workaround here.
Since the topic is old, I'm summarising the problem .. basically it involves
rsync creating large numbers of files with a leading ".." when syncing to an
apple network share via afp.
The essence of the problem is that there is an apple bug which causes weird
phantom files to be created whenever an attempt is made to create a file with a
leading ".."
As a workaround I figured the simplest thing to do would be to ensure that
rsync only ever adds a "." when creating tmp files for filenames that do not
already have a leading dot.
I've attached a modified version of the get_tmpname method which simply wraps
the addition of a "." in an if statement.
I'd like to ask the list if you think this is a likely to have any issues (eg
filename comparisons relying on a dot always being added).
On my system this change seems to work around the problem pretty well.
Thanks
Ira
( Look for "Modified Code" so see my change )
int get_tmpname(char *fnametmp, const char *fname)
{
int maxname, added, length = 0;
const char *f;
if (tmpdir) {
/* Note: this can't overflow, so the return value is safe */
length = strlcpy(fnametmp, tmpdir, MAXPATHLEN - 2);
fnametmp[length++] = '/';
}
if ((f = strrchr(fname, '/')) != NULL) {
++f;
if (!tmpdir) {
length = f - fname;
/* copy up to and including the slash */
strlcpy(fnametmp, fname, length + 1);
}
} else
f = fname;
// Modified Code
if ( f[0]!='.'){
fnametmp[length++] = '.';
}
// End Modified Code
/* The maxname value is bufsize, and includes space for the '\0'.
* (Note that NAME_MAX get -8 for the leading '.' above.) */
maxname = MIN(MAXPATHLEN - 7 - length, NAME_MAX - 8);
if (maxname < 1) {
rprintf(FERROR_XFER, "temporary filename too long: %s\n",
fname);
fnametmp[0] = '\0';
return 0;
}
added = strlcpy(fnametmp + length, f, maxname);
if (added >= maxname)
added = maxname - 1;
memcpy(fnametmp + length + added, ".XX", 8);
return 1;
}
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
Hi Vernon,
I don't know why I didn't think of that - I was telling people to exclude all .
files! Thanks. I wonder if this is foolproof enough to create a patch since it
seems to be an ongoing bug with OS releases.
Best, Rob
On Feb 24, 2011, at 4:54 AM, Ira Cooke wrote:
> I've recently encountered this issue which was discussed here about a year
> ago.
>
> I'm not sure if anyone has a fix for this, but I thought I would post my
> workaround here.
>
> Since the topic is old, I'm summarising the problem .. basically it involves
> rsync creating large numbers of files with a leading ".." when syncing to an
> apple network share via afp.
>
> The essence of the problem is that there is an apple bug which causes weird
> phantom files to be created whenever an attempt is made to create a file with
> a leading ".."
>
> As a workaround I figured the simplest thing to do would be to ensure that
> rsync only ever adds a "." when creating tmp files for filenames that do not
> already have a leading dot.
>
> I've attached a modified version of the get_tmpname method which simply wraps
> the addition of a "." in an if statement.
>
> I'd like to ask the list if you think this is a likely to have any issues (eg
> filename comparisons relying on a dot always being added).
>
> On my system this change seems to work around the problem pretty well.
>
> Thanks
> Ira
>
> ( Look for "Modified Code" so see my change )
>
> int get_tmpname(char *fnametmp, const char *fname)
> {
> int maxname, added, length = 0;
> const char *f;
>
> if (tmpdir) {
> /* Note: this can't overflow, so the return value is safe */
> length = strlcpy(fnametmp, tmpdir, MAXPATHLEN - 2);
> fnametmp[length++] = '/';
> }
>
> if ((f = strrchr(fname, '/')) != NULL) {
> ++f;
> if (!tmpdir) {
> length = f - fname;
> /* copy up to and including the slash */
> strlcpy(fnametmp, fname, length + 1);
> }
> } else
> f = fname;
>
> // Modified Code
> if ( f[0]!='.'){
> fnametmp[length++] = '.';
> }
> // End Modified Code
>
> /* The maxname value is bufsize, and includes space for the '\0'.
>* (Note that NAME_MAX get -8 for the leading '.' above.) */
> maxname = MIN(MAXPATHLEN - 7 - length, NAME_MAX - 8);
>
> if (maxname < 1) {
> rprintf(FERROR_XFER, "temporary filename too long: %s\n",
> fname);
> fnametmp[0] = '\0';
> return 0;
> }
>
> added = strlcpy(fnametmp + length, f, maxname);
> if (added >= maxname)
> added = maxname - 1;
> memcpy(fnametmp + length + added, ".XX", 8);
>
> return 1;
> }
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
Hi Rob / Wayne,
Sorry for the slow reply Rob.
I'm not sure of the requirements for patches, but I think it would be useful to
create one for this case as there are reasonably number of people affected.
I created a patch file against the current version (see below).
Would it be possible to include this patch in the official list?
Cheers
Ira
diff -Naur rsync-3.0.8_orig/receiver.c rsync-3.0.8_nodotdot/receiver.c
--- rsync-3.0.8_orig/receiver.c 2011-04-05 17:12:03.0 +1000
+++ rsync-3.0.8_nodotdot/receiver.c 2011-04-05 17:20:40.0 +1000
@@ -103,7 +103,10 @@
}
} else
f = fname;
- fnametmp[length++] = '.';
+
+ if ( f[0]!='.'){
+ fnametmp[length++] = '.';
+ }
/* The maxname value is bufsize, and includes space for the '\0'.
* NAME_MAX needs an extra -1 for the name's leading dot. */
> Hi Vernon,
>
> I don't know why I didn't think of that - I was telling people to exclude all
> . files! Thanks. I wonder if this is foolproof enough to create a patch since
> it seems to be an ongoing bug with OS releases.
>
> Best, Rob
>
> On Feb 24, 2011, at 4:54 AM, Ira Cooke wrote:
>
>> I've recently encountered this issue which was discussed here about a year
>> ago.
>>
>> I'm not sure if anyone has a fix for this, but I thought I would post my
>> workaround here.
>>
>> Since the topic is old, I'm summarising the problem .. basically it involves
>> rsync creating large numbers of files with a leading ".." when syncing to an
>> apple network share via afp.
>>
>> The essence of the problem is that there is an apple bug which causes weird
>> phantom files to be created whenever an attempt is made to create a file
>> with a leading ".."
>>
>> As a workaround I figured the simplest thing to do would be to ensure that
>> rsync only ever adds a "." when creating tmp files for filenames that do not
>> already have a leading dot.
>>
>> I've attached a modified version of the get_tmpname method which simply
>> wraps the addition of a "." in an if statement.
>>
>> I'd like to ask the list if you think this is a likely to have any issues
>> (eg filename comparisons relying on a dot always being added).
>>
>> On my system this change seems to work around the problem pretty well.
>>
>> Thanks
>> Ira
>>
>> ( Look for "Modified Code" so see my change )
>>
>> int get_tmpname(char *fnametmp, const char *fname)
>> {
>> int maxname, added, length = 0;
>> const char *f;
>>
>> if (tmpdir) {
>> /* Note: this can't overflow, so the return value is safe */
>> length = strlcpy(fnametmp, tmpdir, MAXPATHLEN - 2);
>> fnametmp[length++] = '/';
>> }
>>
>> if ((f = strrchr(fname, '/')) != NULL) {
>> ++f;
>> if (!tmpdir) {
>> length = f - fname;
>> /* copy up to and including the slash */
>> strlcpy(fnametmp, fname, length + 1);
>> }
>> } else
>> f = fname;
>>
>> // Modified Code
>> if ( f[0]!='.'){
>> fnametmp[length++] = '.';
>> }
>> // End Modified Code
>>
>> /* The maxname value is bufsize, and includes space for the '\0'.
>> * (Note that NAME_MAX get -8 for the leading '.' above.) */
>> maxname = MIN(MAXPATHLEN - 7 - length, NAME_MAX - 8);
>>
>> if (maxname < 1) {
>> rprintf(FERROR_XFER, "temporary filename too long: %s\n",
>> fname);
>> fnametmp[0] = '\0';
>> return 0;
>> }
>>
>> added = strlcpy(fnametmp + length, f, maxname);
>> if (added >= maxname)
>> added = maxname - 1;
>> memcpy(fnametmp + length + added, ".XX", 8);
>>
>> return 1;
>> }
>> --
>> Please use reply-all for most replies to avoid omitting the mailing list.
>> To unsubscribe or change options:
>> https://lists.samba.org/mailman/listinfo/rsync
>> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
On Tue, Apr 5, 2011 at 3:08 AM, Ira Cooke wrote: > Would it be possible to include this patch in the official list? > I'm making this change in the git repository. Sadly, I didn't remember to review it for inclusion in 3.0.8. ..wayne.. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: osx 10.6 strange rsync errors
Thats great Thanks very much. Ira -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: [U] RE: RE: RE: Rsync Errors
On 9/5/07, Peter, Theresa M Ms CONT USAAC <[EMAIL PROTECTED]> wrote: > We were using compression on the network; through using a WAS device. We > have turned that off and we are no longer seeing the problem. Why would > compression cause these issues? Probably there is a bug in the WAS's compression algorithm that is corrupting the data sent over the rsync connection. > If we turn the WAS (network compression) > back on is this going to cause more issues. Yes. > What do you mean by "the binary > data protocol is no longer in sync"? Rsync sends a continuous stream of bytes over the connection, and each group of bytes has a specific meaning according to the rsync protocol. Rsync depends delicately on the entire stream of bytes arriving intact on the other side so that the receiver interprets each group of bytes in the way the sender intended. In many cases, loss or corruption of a single byte of the stream causes the receiver to misinterpret all the data it receives from that point on, resulting in the errors you saw. This is analogous to a frameshift or nonsense mutation in DNA if you happen to be familiar with those concepts. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: [Kde-mirrors] rsync errors from rsync.kde.org
Hi, On Wed, 9 Jun 2004, Dirk Mueller wrote: > it seems a lot of people get recently rsync protocol errors from > rsync.kde.org. Part of the problem is probably that we upgraded to rsync > 2.6.2 due to security problems with older versions. > > the newer rsync release contains a newer rsync protocol that appears to be not > fully backward compatible. The newer, improved protocol should be faster and > less memory hungry too. > > those of you who experience a protocol error during rsyncing.. are you all > using an older version of rsync? which one? perhaps that helps debugging the > problem. ftp.gwdg.de has senn lots of rsync aborts with master.kde.org - I had the picture that probably master.kde.org got slashdotted since last friday by being a SUSE mirror... Yes, ftp.gwdg.de is still using the "old" rsync: [EMAIL PROTECTED]:0 01:47:48 /mirr/bin > rsync --version rsync version 2.5.7 protocol version 26 and I guess I can't change that due to anasty bug of the new rsync version: If I have configured a "--backup-dir" and the new rsync version finds a second file which has to get deposited there, it is thinking that it is an error 23 that the backup directory already is existing. This is more than nasty for me, it is not tolarable if I like to track my server's activities with the help of rsync return codes - which indeed all the time was a huge improvement over any ftp protocol mirroring, the main advantage beneath the true "inode data mirroring" of rsync in my opinion. Cheers -e -- Eberhard Moenkeberg ([EMAIL PROTECTED], [EMAIL PROTECTED]) -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: [Kde-mirrors] rsync errors from rsync.kde.org
On Wednesday 09 June 2004 01:53, Eberhard Moenkeberg wrote: > ftp.gwdg.de has senn lots of rsync aborts with master.kde.org - I had the > picture that probably master.kde.org got slashdotted since last friday by > being a SUSE mirror... we were talking about rsync.kde.org here.. there are no known problems with master.kde.org (which is used for ftp.kde.org rsyncing). Dirk -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: [Kde-mirrors] rsync errors from rsync.kde.org
On Wed, Jun 09, 2004 at 01:53:52AM +0200, Eberhard Moenkeberg wrote: > If I have configured a "--backup-dir" and the new rsync version finds a > second file which has to get deposited there, it is thinking that it is > an error 23 that the backup directory already is existing. This has been fixed in CVS for a while now. If possible, please try it out. If you'd prefer not to use the CVS version, it shouldn't be too much longer now before I start up a new pre-release cycle, and I'd very much like to get a lot of people testing it when it happens. ..wayne.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Re: osx 10.6 strange rsync errors
Robert DuToit wrote: > On Apr 6, 2010, at 9:14 AM, Benjamin R. Haskell wrote: > >> On Mon, 5 Apr 2010, Robert DuToit wrote: >> >>> Hi Matt, >>> >>> I set up a simple test with a nest of directories ( aa > bb > cc >>> > dd &ee) with 1 file in each. running rsync from OS 10.6 to >>> another Mac with OS10.5 there seems to be no problem. When doing >>> the reverse I am seeing the odd behavior. >>> >>> Below is the log from running the options >>> >>> -aHAXN --fileflags --force-change -stats -vvv Please specify _exactly_ how you're building rsync. Those options don't exist without applying some extra patches. I can't replicate the problem with stock rsync, which means it's either a problem with the patches, or I'm not triggering the bug properly. Can you replicate this without a 10.5 box? Can you replicate it with Apple's rsync? Unpatched 3.0.7 rsync? Patched 3.0.7 rsync? Nightly snapshot? I really don't want to spend time investigating an old release, and I need to be able to reproduce the problem before I can help fix it. I have 10.6 and OpenSolaris easily at hand to test with, but no 10.5 boxes. > I checked out dtrace (OSX version of strace) but it seems a bit > complicated and I don't know how to use it yet. If anyone has maybe > they can let me know. sudo dtruss -f /path/to/rsync --whatever src dest 2>/tmp/mydtrusslog should dump all the syscalls into /tmp/mydtrusslog -- Carson -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: [U] RE: RE: RE: RE: Rsync Errors
On 9/6/07, Peter, Theresa M Ms CONT USAAC <[EMAIL PROTECTED]> wrote: > We had just been running with WAS since April and never > had a problem prior to this. As far as the network group is concerned they > made no changes or upgrades to WAS. So it just seemed odd that out of the > blue we started to see problems. Bugs in compression algorithms tend to be very sensitive to the nature of the data being compressed. It's possible that the files you have been copying recently contain data sequences that trigger the bug in the WAS while files you copied previously did not. In any event, I recommend that you leave the WAS disabled and use rsync's built-in compression (-z option) if reducing network traffic is important. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 4327] Sudden rsync errors
https://bugzilla.samba.org/show_bug.cgi?id=4327 [EMAIL PROTECTED] changed: What|Removed |Added Status|NEW |RESOLVED Resolution||WORKSFORME --- Comment #1 from [EMAIL PROTECTED] 2008-01-12 11:45 CST --- All I see in the strace info is rsync complaining about the connection closing. You'll need to figure out why the connection is closing, perhaps using hints from the issues-and-debugging web page on the rsync website. If you have further info to provide, please feel free to reopen this. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 4327] New: Sudden rsync errors
https://bugzilla.samba.org/show_bug.cgi?id=4327
Summary: Sudden rsync errors
Product: rsync
Version: 2.6.8
Platform: All
URL: http://www.ercbroadband.org
OS/Version: Windows XP
Status: NEW
Severity: normal
Priority: P3
Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
QAContact: [EMAIL PROTECTED]
I'm getting some unknown errors when trying to rsync between one of our servers
and a client of ours. These errors started after this server was upgraded to
Fedora Core 5, but I've verified that the upgrade isn't the cause of the
problem as another server running RHEL AS 4 has the same problem. Below is the
last 70 lines of an strace done earlier today.
31557 fcntl(0, F_GETFL) = 0x802 (flags O_RDWR|O_NONBLOCK)
31557 fcntl(0, F_SETFL, O_RDWR) = 0
31557 ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff272555e0) = -1 EINVAL
(Invalid argument)
31557 fcntl(1, F_GETFL) = 0x802 (flags O_RDWR|O_NONBLOCK)
31557 fcntl(1, F_SETFL, O_RDWR) = 0
31557 ioctl(2, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo
...}) = 0
31557 write(2, "Read from remote host homeguage."..., 63) = 63
31557 gettimeofday({1168304083, 914145}, NULL) = 0
31557 shutdown(3, 2 /* send and receive */) = -1 ENOTCONN (Transport endpoint
is not connected)
31557 close(3) = 0
31557 brk(0x55712000) = 0x55712000
31557 brk(0x55702000) = 0x55702000
31557 brk(0x556f2000) = 0x556f2000
31557 brk(0x556dc000) = 0x556dc000
31557 exit_group(-1)= ?
31556 <... select resumed> )= 1 (out [4], left {21, 278000})
31556 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0Q\332\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
4092) = -1 EPIPE (Broken pipe)
31556 --- SIGPIPE (Broken pipe) @ 0 (0) ---
31556 write(2, "rsync: writefd_unbuffered failed"..., 82) = 82
31556 write(2, "\n", 1) = 1
31556 rt_sigaction(SIGUSR1, {SIG_IGN}, NULL, 8) = 0
31556 rt_sigaction(SIGUSR2, {SIG_IGN}, NULL, 8) = 0
31556 wait4(31557, 0x7fffea7a594c, WNOHANG, NULL) = 0
31556 getpid() = 31556
31556 kill(31557, SIGUSR1) = 0
31556 kill(31622, SIGUSR1) = 0
31622 <... select resumed> )= ? ERESTARTNOHAND (To be restarted)
31622 --- SIGUSR1 (User defined signal 1) @ 0 (0) ---
31556 write(2, "rsync error: error in rsync prot"..., 90
31622 rt_sigaction(SIGUSR1, {SIG_IGN}, NULL, 8) = 0
31622 rt_sigaction(SIGUSR2, {SIG_IGN}, NULL, 8) = 0
31622 wait4(31557, 0x7fffea7a324c, WNOHANG, NULL) = -1 ECHILD (No child
processes)
31622 kill(31557, SIGUSR1) = 0
31622 write(2, "rsync error: received SIGUSR1 (c"..., 72) = 72
31622 write(2, "\n", 1) = 1
31622 exit_group(19)= ?
31556 <... write resumed> ) = ? ERESTARTSYS (To be restarted)
31556 --- SIGCHLD (Child exited) @ 0 (0) ---
31556 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 255}], WNOHANG, NULL) =
31557
31556 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 19}], WNOHANG, NULL) =
31622
31556 wait4(-1, 0x7fffea7a344c, WNOHANG, NULL) = -1 ECHILD (No child processes)
31556 rt_sigreturn(0x) = -1 EINTR (Interrupted system call)
31556 rt_sigaction(SIGUSR1, {SIG_IGN}, NULL, 8) = 0
31556 rt_sigaction(SIGUSR2, {SIG_IGN}, NULL, 8) = 0
31556 wait4(31557, 0x7fffea7a374c, WNOHANG, NULL) = -1 ECHILD (No child
processes)
31556 kill(31557, SIGUSR1) = -1 ESRCH (No such process)
31556 kill(31622, SIGUSR1) = -1 ESRCH (No such process)
31556 write(2, "rsync error: unexplained error ("..., 73) = 73
31556 write(2, "\n", 1) = 1
31556 exit_group(255) = ?
[EMAIL PROTECTED] homeguage]# tail -n 70 /tmp/rsync.log >> strace.txt
[EMAIL PROTECTED] homeguage]# vi strace.txt
[EMAIL PROTECTED] homeguage]# man rsync
[EMAIL PROTECTED] homeguage]# rpm -qa | grep rsync
rsync-2.6.8-1.FC5.1
[EMAIL PROTECTED] homeguage]# vi strace.txt
31557 exit_group(-1)= ?
31556 <... select resumed> )= 1 (out [4], left {21, 278000})
31556 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0Q\332\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
4092) = -1 EPIPE (Broken pipe)
31556 --- SIGPIPE (Broken pipe) @ 0 (0) ---
31556 write(2, "rsync: writefd_unbuffered failed"..., 82) = 82
31556 write(2, "\n", 1) = 1
31556 rt_sigaction(SIGUSR1, {SIG_IGN}, NULL, 8) = 0
31556 rt_sigaction(SIGUSR2, {SIG_IGN}, NULL, 8) = 0
31556 wait4(31557, 0x7fffea7a594c, WNOHANG, NULL) = 0
31556 getpid() = 31556
31556 kill(31557, SIGUSR1) = 0
31556 kill(31622, SIGUSR1) =
