Re: Rsync over NFS mount sending whole files
Amazing. Connecting to the NFS server directly instead of through a mount results in much better transfers. Thank you. It looks like just the updated portions of the files are zoomed across. A dumb question, but why? Does the rsync daemon on the server side have anything to do with this? Or does the NFS client act as if the mount is a local directory and tries to do read and writes just as it would on disk? Anyways, it works - so I'm happy. Thanks again for your help Anban. On 10/28/05, Wayne Davison [EMAIL PROTECTED] wrote: On Thu, Oct 27, 2005 at 07:27:09PM +0200, Anban Mestry wrote: rsync -avtz --no-whole-file \test1\ \mnt\test2\ You don't want to do that, because --no-whole-file optimizes rsync's socket I/O at the expense of disk I/O, which means that you're making things less efficient when the connection between the sender and the receiver is a local pipe. The use of -z for a local copy is also wasteful because you're using CPU to optimize the transfer of data over a connection that is faster than the disk I/O, even when uncompressed. Your best configuration is to avoid updating via NFS and instead connect to the NFS server directly so that rsync can update the files on a local disk. That allows rsync to optimize the network traffic. rsync -avtz /test1/ remoteNFShost:/test2/ If that is not possible, the method that uses the least disk I/O for a local copy is --whole-file and (to a much smaller extent) --inplace. That still writes out the entire file over NFS for each update, though, but it does at least avoid having rsync do a full-file read followed by a full-file write (which is what occurs with --no-whole-file). ..wayne.. -- Shameless publicity -- http://oranjia.blogspot.com -- -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync over NFS mount sending whole files
On Fri 28 Oct 2005, Anban Mestry wrote: Amazing. Connecting to the NFS server directly instead of through a mount results in much better transfers. Thank you. It looks like just the updated portions of the files are zoomed across. That is the case... A dumb question, but why? Does the rsync daemon on the server side have anything to do with this? Or does the NFS client act as if the Of course; the rsync daemon isn't just a passive data mover. It reads blocks and uses checksums to determine what parts are different. mount is a local directory and tries to do read and writes just as it would on disk? As far as rsync is concerned (and most of all the other utilities you have at your disposal on a Unix / linux system) an NFS-mounted filesystem _is_ local, after all it's just a part of the filesystem space. That's the point of NFS mounts, isn't it? Paul Slootman -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync over NFS mount sending whole files
Hi, On Thu, 27 Oct 2005, Anban Mestry wrote: I'm not sure if anyone has experienced this, and I have searched for it online, with no conclusive, err.. conclusions. Basically, when rsyncing two \test1(local) and \mnt\test2\ (NFS mount) it seems that when using rsync with --no-whole-file entire files (instead of just updated blocks) are sent through. I am using the following command rsync -avtz --no-whole-file \test1\ \mnt\test2\ For example, if say a particular file was 30KB, IPTRAF reports just over 31KB transferred while rsync itself reports just a few bytes (i.e. 200 or so). One of your local only rsync instances has to read the entire file in order to find out that only 200 bytes are to transfer. ;-)) Cheers -e -- Eberhard Moenkeberg ([EMAIL PROTECTED], [EMAIL PROTECTED]) -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync over NFS mount sending whole files
On Thu, Oct 27, 2005 at 07:27:09PM +0200, Anban Mestry wrote: rsync -avtz --no-whole-file \test1\ \mnt\test2\ You don't want to do that, because --no-whole-file optimizes rsync's socket I/O at the expense of disk I/O, which means that you're making things less efficient when the connection between the sender and the receiver is a local pipe. The use of -z for a local copy is also wasteful because you're using CPU to optimize the transfer of data over a connection that is faster than the disk I/O, even when uncompressed. Your best configuration is to avoid updating via NFS and instead connect to the NFS server directly so that rsync can update the files on a local disk. That allows rsync to optimize the network traffic. rsync -avtz /test1/ remoteNFShost:/test2/ If that is not possible, the method that uses the least disk I/O for a local copy is --whole-file and (to a much smaller extent) --inplace. That still writes out the entire file over NFS for each update, though, but it does at least avoid having rsync do a full-file read followed by a full-file write (which is what occurs with --no-whole-file). ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
