Your log file indicates that rsync is indeed working as designed finding lots of data matches:
Literal data: 123736377 bytes Matched data: 17889663500 bytes This means that rsync only had to transfer 118 MB instead of 16+ GB. It does this by trading CPU and disk operations for network bytes. Now, it could be that rsync is still slow in terms of "wall clock" time. This can happen if the process gets bottlenecked by something other than the network (CPU, memory, or disk). Rsync does need to read the existing files on the destination side in their entirety to compute checksums. This can make the overall process disk-bound. Rsync also needs to search for hash matches on the sending side, which is CPU-intensive. If your network is fast, you disk is slow-ish, and your files are really big, it can be faster to simply transfer the whole file (with compression). This is in fact the case for one of my rsync jobs... it is faster in terms of wall-clock time to send the whole file without checking for data matches (the -W parameter). But we still use rsync, since we pay per GB for transfer from the colocation facility. Rsync really helps in lower bandwidth or situations where you pay per byte for bandwidth, and wall-clock transfer speed isn't always the most important thing. It would be a big boost for large files if rsync "remembered" the hashes on each end, so it didn't have to re-read the files on every run if the files were unchanged. This is a feature that rsync's developers have rejected, since rsync is designed to be stateless between runs. I believe Unison does keep state at both ends, you might want to look at that. -- RPM -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html