Nobody can comment on this? -Brian
Brian H. Nelson wrote: > I noticed this issue yesterday when I first started playing around with > zfs send/recv. This is on Solaris 10U6. > > It seems that a zfs send of a zvol issues 'volblocksize' reads to the > physical devices. This doesn't make any sense to me, as zfs generally > consolidates read/write requests to improve performance. Even the dd > case with the same snapshot does not exhibit this behavior. It seems to > be specific to zfs send. > > I checked with 8k, 64k, and 128k volblocksize, and the reads generated > by zfs send always seem to follow that size, while the reads with dd do not. > > The small reads seems to hurt performance of zfs send. I tested with a > mirror, but on another machine with a 7 disk raidz, the performance is > MUCH worse because the 8k reads get broken up into even smaller reads > and spread across the raidz. > > Is this a bug, or can someone explain why this is happening? > > Thanks > -Brian > > Using 8k volblocksize: > > -bash-3.00# zfs send pool1/vo...@now > /dev/null > > capacity operations bandwidth > pool used avail read write read write > ----------- ----- ----- ----- ----- ----- ----- > pool1 4.01G 274G 1.88K 0 15.0M 0 > mirror 4.01G 274G 1.88K 0 15.0M 0 > c0t9d0 - - 961 0 7.46M 0 > c0t11d0 - - 968 0 7.53M 0 > ----------- ----- ----- ----- ----- ----- ----- > == ~8k reads to pool and drives > > -bash-3.00# dd if=/dev/zvol/dsk/pool1/vo...@now of=/dev/null bs=8k > > capacity operations bandwidth > pool used avail read write read write > ----------- ----- ----- ----- ----- ----- ----- > pool1 4.01G 274G 2.25K 0 17.9M 0 > mirror 4.01G 274G 2.25K 0 17.9M 0 > c0t9d0 - - 108 0 9.00M 0 > c0t11d0 - - 109 0 8.92M 0 > ----------- ----- ----- ----- ----- ----- ----- > == ~8k reads to pool, ~85k reads to drives > > > Using volblocksize of 64k: > > -bash-3.00# zfs send pool1/vol...@now > /dev/null > > capacity operations bandwidth > pool used avail read write read write > ----------- ----- ----- ----- ----- ----- ----- > pool1 6.01G 272G 378 0 23.5M 0 > mirror 6.01G 272G 378 0 23.5M 0 > c0t9d0 - - 189 0 11.8M 0 > c0t11d0 - - 189 0 11.7M 0 > ----------- ----- ----- ----- ----- ----- ----- > == ~64k reads to pool and drives > > -bash-3.00# dd if=/dev/zvol/dsk/pool1/vol...@now of=/dev/null bs=64k > > capacity operations bandwidth > pool used avail read write read write > ----------- ----- ----- ----- ----- ----- ----- > pool1 6.01G 272G 414 0 25.7M 0 > mirror 6.01G 272G 414 0 25.7M 0 > c0t9d0 - - 107 0 12.9M 0 > c0t11d0 - - 106 0 12.8M 0 > ----------- ----- ----- ----- ----- ----- ----- > == ~64k reads to pool, ~124k reads to drives > > > Using volblocksize of 128k: > > -bash-3.00# zfs send pool1/vol1...@now > /dev/null > > capacity operations bandwidth > pool used avail read write read write > ----------- ----- ----- ----- ----- ----- ----- > pool1 4.01G 274G 188 0 23.3M 0 > mirror 4.01G 274G 188 0 23.3M 0 > c0t9d0 - - 94 0 11.7M 0 > c0t11d0 - - 93 0 11.7M 0 > ----------- ----- ----- ----- ----- ----- ----- > == ~128k reads to pool and drives > > -bash-3.00# dd if=/dev/zvol/dsk/pool1/vol1...@now of=/dev/null bs=128k > > capacity operations bandwidth > pool used avail read write read write > ----------- ----- ----- ----- ----- ----- ----- > pool1 4.01G 274G 247 0 30.8M 0 > mirror 4.01G 274G 247 0 30.8M 0 > c0t9d0 - - 122 0 15.3M 0 > c0t11d0 - - 123 0 15.5M 0 > ----------- ----- ----- ----- ----- ----- ----- > == ~128k reads to pool and drives > > -- --------------------------------------------------- Brian H. Nelson Youngstown State University System Administrator Media and Academic Computing bnelson[at]cis.ysu.edu --------------------------------------------------- _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss