I noticed this issue yesterday when I first started playing around with
zfs send/recv. This is on Solaris 10U6.
It seems that a zfs send of a zvol issues 'volblocksize' reads to the
physical devices. This doesn't make any sense to me, as zfs generally
consolidates read/write requests to improve performance. Even the dd
case with the same snapshot does not exhibit this behavior. It seems to
be specific to zfs send.
I checked with 8k, 64k, and 128k volblocksize, and the reads generated
by zfs send always seem to follow that size, while the reads with dd do not.
The small reads seems to hurt performance of zfs send. I tested with a
mirror, but on another machine with a 7 disk raidz, the performance is
MUCH worse because the 8k reads get broken up into even smaller reads
and spread across the raidz.
Is this a bug, or can someone explain why this is happening?
Thanks
-Brian
Using 8k volblocksize:
-bash-3.00# zfs send pool1/vo...@now > /dev/null
capacity operations bandwidth
pool used avail read write read write
----------- ----- ----- ----- ----- ----- -----
pool1 4.01G 274G 1.88K 0 15.0M 0
mirror 4.01G 274G 1.88K 0 15.0M 0
c0t9d0 - - 961 0 7.46M 0
c0t11d0 - - 968 0 7.53M 0
----------- ----- ----- ----- ----- ----- -----
== ~8k reads to pool and drives
-bash-3.00# dd if=/dev/zvol/dsk/pool1/vo...@now of=/dev/null bs=8k
capacity operations bandwidth
pool used avail read write read write
----------- ----- ----- ----- ----- ----- -----
pool1 4.01G 274G 2.25K 0 17.9M 0
mirror 4.01G 274G 2.25K 0 17.9M 0
c0t9d0 - - 108 0 9.00M 0
c0t11d0 - - 109 0 8.92M 0
----------- ----- ----- ----- ----- ----- -----
== ~8k reads to pool, ~85k reads to drives
Using volblocksize of 64k:
-bash-3.00# zfs send pool1/vol...@now > /dev/null
capacity operations bandwidth
pool used avail read write read write
----------- ----- ----- ----- ----- ----- -----
pool1 6.01G 272G 378 0 23.5M 0
mirror 6.01G 272G 378 0 23.5M 0
c0t9d0 - - 189 0 11.8M 0
c0t11d0 - - 189 0 11.7M 0
----------- ----- ----- ----- ----- ----- -----
== ~64k reads to pool and drives
-bash-3.00# dd if=/dev/zvol/dsk/pool1/vol...@now of=/dev/null bs=64k
capacity operations bandwidth
pool used avail read write read write
----------- ----- ----- ----- ----- ----- -----
pool1 6.01G 272G 414 0 25.7M 0
mirror 6.01G 272G 414 0 25.7M 0
c0t9d0 - - 107 0 12.9M 0
c0t11d0 - - 106 0 12.8M 0
----------- ----- ----- ----- ----- ----- -----
== ~64k reads to pool, ~124k reads to drives
Using volblocksize of 128k:
-bash-3.00# zfs send pool1/vol1...@now > /dev/null
capacity operations bandwidth
pool used avail read write read write
----------- ----- ----- ----- ----- ----- -----
pool1 4.01G 274G 188 0 23.3M 0
mirror 4.01G 274G 188 0 23.3M 0
c0t9d0 - - 94 0 11.7M 0
c0t11d0 - - 93 0 11.7M 0
----------- ----- ----- ----- ----- ----- -----
== ~128k reads to pool and drives
-bash-3.00# dd if=/dev/zvol/dsk/pool1/vol1...@now of=/dev/null bs=128k
capacity operations bandwidth
pool used avail read write read write
----------- ----- ----- ----- ----- ----- -----
pool1 4.01G 274G 247 0 30.8M 0
mirror 4.01G 274G 247 0 30.8M 0
c0t9d0 - - 122 0 15.3M 0
c0t11d0 - - 123 0 15.5M 0
----------- ----- ----- ----- ----- ----- -----
== ~128k reads to pool and drives
--
---------------------------------------------------
Brian H. Nelson Youngstown State University
System Administrator Media and Academic Computing
bnelson[at]cis.ysu.edu
---------------------------------------------------
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
