Re: bwlimit on rsync locally
Thanks John. I looked into this patch but it doesn't resolve my problem, and we have have posix FADV. I found a simple solution. Just open the file on receiver to be write direct to disk avoid file buffer/cache. Of course the performance sucks but it's what I need. The output block rate cannot be reduced by 2/3 based on my besting on linux 2.6.22. I can post the diff if anybody wants. - Reeve On Sun, Mar 30, 2008 at 10:23 PM, John Van Essen <[EMAIL PROTECTED]> wrote: > On Fri, 28 Mar 2008, Reeve Yang <[EMAIL PROTECTED]> wrote: > > Is there anyway to make rsync use direct I/O bypassing FS buffer? > > I'm not an expert on that by any means, but AFAIK, rsync supports that > only via this patch: > > http://rsync.samba.org/ftp/rsync/patches/drop-cache.diff > > (You'll have to compile your own version of rsync.) > > > > John > > -- > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html > -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: bwlimit on rsync locally
On Fri, 28 Mar 2008, Reeve Yang <[EMAIL PROTECTED]> wrote: > Is there anyway to make rsync use direct I/O bypassing FS buffer? I'm not an expert on that by any means, but AFAIK, rsync supports that only via this patch: http://rsync.samba.org/ftp/rsync/patches/drop-cache.diff (You'll have to compile your own version of rsync.) John -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: bwlimit on rsync locally
Is there anyway to make rsync use direct I/O bypassing FS buffer? On Thu, Mar 27, 2008 at 10:57 PM, John Van Essen <[EMAIL PROTECTED]> wrote: > On Thu, 27 Mar 2008, Reeve Yang <[EMAIL PROTECTED]> wrote: > > Does "bwlimit" option really work on rsync locally? > > Yes, it does. > > > > rsync without --bwlimit: > >> iostat; rsync -a -r --stats swapfile swapfile.rsync; iostat > > > > sent 2147745923 bytes received 42 bytes 23472633.50 bytes/sec > > > > rysnc with --bwlimit: > >> iostat && rsync -a -r --stats --bwlimit=2500 swapfile swapfile.rsync && > iostat > > > > sent 2147745923 bytes received 42 bytes 2543216.06 bytes/sec > > Notice that the second rsync "bytes/sec" was a factor of 10 smaller > than the first one and was, indeed, approximately 2500 kBps. > > If it isn't having the effect you want, keep lowering the bwlimit > value until you get the desired results. > > John > > -- > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html > -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: bwlimit on rsync locally
On Thu, 27 Mar 2008, Reeve Yang <[EMAIL PROTECTED]> wrote: > Does "bwlimit" option really work on rsync locally? Yes, it does. > rsync without --bwlimit: >> iostat; rsync -a -r --stats swapfile swapfile.rsync; iostat > sent 2147745923 bytes received 42 bytes 23472633.50 bytes/sec > rysnc with --bwlimit: >> iostat && rsync -a -r --stats --bwlimit=2500 swapfile swapfile.rsync && >> iostat > sent 2147745923 bytes received 42 bytes 2543216.06 bytes/sec Notice that the second rsync "bytes/sec" was a factor of 10 smaller than the first one and was, indeed, approximately 2500 kBps. If it isn't having the effect you want, keep lowering the bwlimit value until you get the desired results. John -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
bwlimit on rsync locally
Does "bwlimit" option really work on rsync locally? We have one type of harddisk and want to slow down rsync I/O on disk because I don't want the disk head gets too hot. While I'm trying to use --bwlimit option, it looks the rsync speed was slowed down, but iostat is not improved at all. In both case the block written speed is increased by the same amount. How could I really slow down I/O while using rsync? Any help would be greatly appreciated. Regards, - Reeve rsync without --bwlimit: > iostat; rsync -a -r --stats swapfile swapfile.rsync; iostat Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 7.53 103.43 421.27 64549578 262909196 Number of files: 1 Number of files transferred: 1 Total file size: 2147483648 bytes Total transferred file size: 2147483648 bytes Literal data: 2147483648 bytes Matched data: 0 bytes File list size: 73 File list generation time: 0.063 seconds File list transfer time: 0.000 seconds Total bytes sent: 2147745923 Total bytes received: 42 sent 2147745923 bytes received 42 bytes 23472633.50 bytes/sec total size is 2147483648 speedup is 1.00 Linux 2.6.22.1-33.fc7 (rxyang-lx) 03/27/2008 Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 7.57 108.87 427.92 67955562 267094188 rysnc with --bwlimit: > iostat && rsync -a -r --stats --bwlimit=2500 swapfile swapfile.rsync && iostat Linux 2.6.22.1-33.fc7 (rxyang-lx) 03/27/2008 avg-cpu: %user %nice %system %iowait %steal %idle 1.943.394.110.340.00 90.22 Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 7.57 107.96 426.21 68539642 270572036 Number of files: 1 Number of files transferred: 1 Total file size: 2147483648 bytes Total transferred file size: 2147483648 bytes Literal data: 2147483648 bytes Matched data: 0 bytes File list size: 73 File list generation time: 0.061 seconds File list transfer time: 0.000 seconds Total bytes sent: 2147745923 Total bytes received: 42 sent 2147745923 bytes received 42 bytes 2543216.06 bytes/sec total size is 2147483648 speedup is 1.00 Linux 2.6.22.1-33.fc7 (rxyang-lx) 03/27/2008 avg-cpu: %user %nice %system %iowait %steal %idle 1.943.394.110.340.00 90.21 Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 7.61 112.63 432.78 71598154 275107508 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
