On 1/13/2014 5:34 PM, Perry Smith wrote:
Ok. I can get the Mac up to version 3 but I'm wondering if
I need to rethink my whole strategy. Since the source is on NFS,
doing a stat on all the files each run may cost me too much
time.
I might need to split it into smaller pieces and then rotate
through the pieces via a script. Do you have any suggestions
for this type of situation?
There is no avoiding at least a stat on every file no matter what tool
you use other than say a dd of the whole underlying partition or drive.
One of the main differences rsync 3 will make is if both sides are 3.0+
then rsync will stat a small list of files and start comparing and
transferring those right away, and work in an unlimited number of those
chunks sequentially until the job is done, instead of
parsing/expanding/building the entire job in memory before doing
anything else.
So I don't think breaking up the job into pieces yourself will help any
more than letting rsync 3 do it internally. It might help if you
couldn't upgrade both sides to 3.
Definitely I would try to rsync directly between the nfs server and the
other target instead of rsyncing from a point that accesses the files
via nfs, if that's possible.
If that's not possible, at least I would hope that the rsync client and
the nfs server were at least on a good lan together. Even 100Mb lan
connection to nfs should be good enough to keep your wan connection
saturated.
Or better yet, test the difference nfs makes. Construct a reasonable
test job right on the Mac, copy the same files to nfs, and try the same
exact job both ways:
1) rsync client on the Mac, source from the Macs local drive
2) rsync client on the Mac, source from the nfs mount
You may be nowhere near saturating your wan either way so the point is
to see if nfs is actually hurting very much relative to pure rsync, and
then don't worry about it if it's not.
And you can always add --progress and more -vvv's to see what it's
working on while it's busy.
--
bkw
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
