I hope this isn't a FAQ.
Per the man page I see ways to control the blocksize for hash
comparison reasons, but no way to control it for i/o performance
reasons.
I'm using rsync to copy folder trees full of large files and I'd like
to have control of how much data is read / written at a time. Maybe
read 10 MB, write 10 MB, etc.
Is there an existing way to do that?
== details ==
When copying a bunch of 1.5 GB files with rsync, I'm only seeing 50%
of the throughput I hope to see.
I haven't looked at the code, or even run strace, but it seems like
the code is doing something like:
while (files) {
read 1.5 GB file to ram
write 1.5 GB file from ram
fsync() ensure 1.5 GB file is on disk
} endwhile
I say that because I see several seconds of high-speed reading, then no reads.
When the reads stop, I see writes kick in, then they stop and reads
start up again.
The end result is I'm only using 50% of the available bandwidth.
Not that I'm copying my source folder tree to a newly created folder
tree, so there is not any reading of the destination needed.
My ultimate would be something like:
while (files) {
while (data_in_file) {
read user_defined_blocksize to ram from file
write user_defined_blocksize from ram to file
}
fsync() ensure 1.5 GB file is on disk
} endwhile
Thanks
Greg
--
Greg Freemyer
www.IntelligentAvatar.net
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
