Dick Davies wrote:
On 30/08/06, Matthew Ahrens <[EMAIL PROTECTED]> wrote:
'zfs send' is *incredibly* faster than rsync.
That's interesting. We had considered it as a replacement for a
certain task (publishing a master docroot to multiple webservers)
but a quick test with ~500Mb of data showed the zfs send/recv
to be about 5x slower than rsync for the initial copy.
You're saying subsequent copies (zfs send -i?) should be faster?
There's definitely something anomalous going on if you are seeing 'zfs
send|recv' being 5x slower than rsync. I just did some quick tests on
some mediocre machines and send|recv is much faster than rsync. So it
would be great if you could describe the setup where you are seeing it
be 5x slower, so that we can try to diagnose it.
I used two different source filesystems:
"workspace" has 98,384 files totaling 1.86GB
"big files" has 2 files totaling 2.13GB
I ran zfs send|recv as:
ptime zfs send pool/[EMAIL PROTECTED] | \
ssh -c blowfish hostname zfs recv -dv pool/recvd
I ran rsync as:
ptime rsync -a -e "ssh -c blowfish" /pool/fs hostname:/pool/rsync/fs
And the results:
zfs send|recv rsync
full workspace: 220s (8.6MB/s) 312s (6.1MB/s)
incremental workspace: <1s 51s
full big files 207s (10.6MB/s) 204s (10.7MB/s)
incremental big files: <1s <1s
So as you can see, we are somewhat constrained by the 100mbit/sec link
between these machines, but on the workspace, the full backup is 40%
faster with send|recv, and the incremental > 50x faster.
To eliminate the network, I tried it between two pools on the same
machine (without using ssh):
zfs send|recv rsync
full workspace: 133s (14.9MB/s) 339 (5.6MB/s)
incremental workspace: <1s 56s
full big files 74s (29.6MB/s) 70s (31.1MB/s)
Here, the full workspace backup was 2.5x faster with send|recv, the
incremental >50x faster, and the full big files 5% slower.
--matt
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
