[Bug 8502] New: problem to make snapshot with --link-dest without copy of the files
https://bugzilla.samba.org/show_bug.cgi?id=8502 Summary: problem to make snapshot with --link-dest without copy of the files Product: rsync Version: 3.0.9 Platform: All OS/Version: All Status: NEW Severity: normal Priority: P5 Component: core AssignedTo: way...@samba.org ReportedBy: dieter.ferdin...@gmx.de QAContact: rsync...@samba.org hello, i have tried to make a snapshot with the following command-line: rsync -lHrtDpvP --link-dest=daten/ daten/ snap1/ but some files are copied instad of making hardlinks to the files and i don't know why! the link-dest-dir is the same as the source dir, so it gives no difference between the source file and the link-dest-file! why does rsync then copy some files and don't make hardlinks to it ??? goodby -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 8502] problem to make snapshot with --link-dest without copy of the files
https://bugzilla.samba.org/show_bug.cgi?id=8502 Wayne Davison way...@samba.org changed: What|Removed |Added Status|NEW |RESOLVED Resolution||INVALID --- Comment #1 from Wayne Davison way...@samba.org 2011-09-29 15:43:36 UTC --- The link-dest dir in your command is NOT the same as the source dir, it is a subdir of the destination dir (which presumably doesn't exist). You probably meant to use --link-dest=../daten (or an absolute path to the source dir). See the man page for how the link-dest dir is relative to the destination path unless absolute. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 5478] rsync: writefd_unbuffered failed to write 4092 bytes [sender]: Broken pipe (32)
https://bugzilla.samba.org/show_bug.cgi?id=5478 --- Comment #17 from Eric Shubert e...@shubes.net 2011-09-29 17:16:38 UTC --- (In reply to comment #16) So now I'm beginning to think that the problem is (indeed) outside of the rsync code. I've examined the 2 target hosts (one that worked and one that doesn't), and tried to determine what's different?. These hosts are all VMs that are running under VMware Server2. The host that worked is running CentOS 5.6, while the one that's failing is running CentOS 5.4 (450 days uptime). I'm going to upgrade the failing host OS to CentOS 5.7 and see if that changes anything. I know that some versions of glibc ( 2.5-34, 2.5.58) had mysterious problems with VMware Server 2 (guests terminating for no apparent reason). I'm wondering if there's something else in glibc that's perhaps having a problem with VMware Server 2. (I realize that Server 2 is essentially dead and will be migrating to KVM in the near future, although Server2 has been quite stable for me.) I've upgrade the target host, and still no dice. Same failure. It seems odd to me that the transmission fails at different points in the process, anywhere from 3.5 minutes to 14.75 minutes. I'm presently thinking that the problem is not with rsync itself, but with some aspect of the network connection. I'm seeing no corresponding ethernet errors on either end though, and the network seems to be fine otherwise. I'd have to say at this point that the failures appear to be happening only with large files. I could sure use some pointers with identifying what the cause of this failure is. I have a test file that fails regularly (if not consistently), and should be able to tell easily when it's fixed. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 5478] rsync: writefd_unbuffered failed to write 4092 bytes [sender]: Broken pipe (32)
https://bugzilla.samba.org/show_bug.cgi?id=5478
--- Comment #18 from Eric Shubert e...@shubes.net 2011-09-29 20:39:54 UTC ---
(In reply to comment #17)
I'm presently thinking that the problem is not with rsync itself, but with
some
aspect of the network connection. I'm seeing no corresponding ethernet errors
on either end though, and the network seems to be fine otherwise.
I'd have to say at this point that the failures appear to be happening only
with large files.
I've (re)read through all 5 bug reports that Tim identified. A general
consensus is that this is happening with relatively large files, and relatively
slow writes (ie loaded targets).
Upon closer examination of a pair of straces, I see this on the target side:
6814 11:34:25.018774 read(0,
\3037sE\303{\264\16\377\333\357\375\7\274}1c\21\306\377\215\256\202\7\376\245\26\24\302{\16c...,
4092) = 4092
6814 11:34:25.020201 select(1, [0], [], NULL, {60, 0} unfinished ...
6814 11:35:25.009953 ... select resumed ) = 0 (Timeout)
I found the corresponding block of data in the strace on the source side:
940 11:33:57.494543 write(4,
\3037sE\303{\264\16\377\333\357\375\7\274}1c\21\306\377\215\256\202\7\376\245\26\24\302{\16c...,
4092 unfinished ...
940 11:33:57.494681 ... write resumed ) = 4092
I had previously set both hosts to use ntp with the same time server, so their
times should be very close.
In the source side strace, I counted ~510 (successful) writes of data between
the one listed above, and the point at which the source host timed out:
940 11:34:25.073394 select(5, NULL, [4], [4], {60, 0}) = 1 (out [4], left
{60, 0})
940 11:34:25.073514 write(4, \374\17\0\7, 4) = 4
940 11:34:25.073620 select(5, NULL, [4], [4], {60, 0}) = 0 (Timeout)
Notice, the last successful read on the target side happened .06 seconds before
the last write on the source side, which is pretty much at the same time. At
this point, there were ~510 x 4K blocks of data (or ~2MB) in the pipe, that
had been written, but not read.
So I'm wondering, is it possible that the pipe (buffers) get filled up, and
when they do, that the select function is unable to handle it? I don't (yet)
know how these things work, so I could be totally off base. At the same time,
this would seem to explain things.
I'll continue to poke at this to see what I can come up with. In the meantime,
any feedback from Wayne and Tim will be appreciated.
--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync (or rsync-like tool) provides local metadata cache ?
Let's say I have a slow, or high latency (or both) link between two hosts, and transferring data is expensive. Is it possible with rsync (or any other tool like rsync) to maintain a local metadata cache, such that I can rsync without transferring data to and from the hosts about what files are already in place ? That is, I already know what is out there and don't need to check anything - I can just use the state information on the sending end to send only the data required, and don't need to traverse the remote filesystem for sizes/datestamps/hashes/etc. Does this exist ? -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync (or rsync-like tool) provides local metadata cache ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 You want something real time like lsyncd. On 09/29/11 19:57, Jason Usher wrote: Let's say I have a slow, or high latency (or both) link between two hosts, and transferring data is expensive. Is it possible with rsync (or any other tool like rsync) to maintain a local metadata cache, such that I can rsync without transferring data to and from the hosts about what files are already in place ? That is, I already know what is out there and don't need to check anything - I can just use the state information on the sending end to send only the data required, and don't need to traverse the remote filesystem for sizes/datestamps/hashes/etc. Does this exist ? - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6FBhgACgkQVKC1jlbQAQe4cgCg91qePrvo0j5XvEgOxMOBEoHo P70AnRHby3Rd/wihPZXY3rLOXtCVZBl9 =GiFl -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
