Hi list! I have seen less-than-stellar ZFS performance on a setup of one main head connected to a JBOD (using SAS, but drives are SATA). There are 16 drives (8 mirrors) in this pool but I'm getting 180ish MB sequential writes (using dd, I know it's not precise, but those numbers should be higher).
With some help on IRC, it seems that part of the reason I'm slowing down is some drives seem to be slower than the others. Initially, I had some drives running at 1.5 mode instead of 3.0 -- They are all running at 3.0 now. While running the following dd command, the output of iostat reflects a much higher %b which seems to say that those drives are slower (but could they really be slowing down everything else that much? --- Or am I looking at the wrong spot here?) -- The pool configuration is also included below dd if=/dev/zero of=4g bs=1M count=4000 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 1.0 0.0 8.0 0.0 0.0 0.0 0.0 0.2 0 0 c1 1.0 0.0 8.0 0.0 0.0 0.0 0.0 0.2 0 0 c1t2d0 8.0 3857.8 64.0 337868.8 0.0 64.5 0.0 16.7 0 704 c5 0.0 259.0 0.0 26386.2 0.0 3.6 0.0 14.0 0 37 c5t50014EE0ACE4AEEFd0 1.0 266.0 8.0 27139.2 0.0 3.6 0.0 13.5 0 37 c5t50014EE056EB0356d0 2.0 276.0 16.0 19315.1 0.0 3.7 0.0 13.3 0 40 c5t50014EE00239C976d0 0.0 279.0 0.0 19699.0 0.0 3.6 0.0 13.0 0 37 c5t50014EE0577C459Cd0 1.0 232.0 8.0 23061.9 0.0 3.6 0.0 15.4 0 37 c5t50014EE0578F60F5d0 0.0 227.0 0.0 22677.9 0.0 3.6 0.0 15.8 0 37 c5t50014EE0AC407BAEd0 0.0 205.0 0.0 24870.2 0.0 3.4 0.0 16.6 0 35 c5t50014EE0AC408605d0 0.0 205.0 0.0 24870.2 0.0 3.4 0.0 16.6 0 35 c5t50014EE056EB0B94d0 1.0 210.0 8.0 15954.2 0.0 4.4 0.0 20.8 0 68 c5t5000C50010C77647d0 0.0 212.0 0.0 16082.2 0.0 4.1 0.0 19.2 0 42 c5t5000C50010C865DEd0 0.0 207.0 0.0 20093.9 0.0 4.2 0.0 20.3 0 45 c5t5000C50010C77679d0 0.0 208.0 0.0 19689.5 0.0 4.1 0.0 19.8 0 44 c5t5000C50010C7672Dd0 0.0 259.0 0.0 14013.7 0.0 5.1 0.0 19.7 0 53 c5t5000C5000A11B600d0 2.0 320.0 16.0 19942.9 0.0 6.9 0.0 21.5 0 84 c5t5000C50008315CE5d0 1.0 259.0 8.0 23380.2 0.0 3.6 0.0 13.9 0 37 c5t50014EE001407113d0 0.0 234.0 0.0 20692.4 0.0 3.6 0.0 15.4 0 38 c5t50014EE00194FB1Bd0 pool: tank state: ONLINE scan: scrub canceled on Mon Jan 30 11:07:02 2012 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c5t50014EE0ACE4AEEFd0 ONLINE 0 0 0 c5t50014EE056EB0356d0 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 c5t50014EE00239C976d0 ONLINE 0 0 0 c5t50014EE0577C459Cd0 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 c5t50014EE0578F60F5d0 ONLINE 0 0 0 c5t50014EE0AC407BAEd0 ONLINE 0 0 0 mirror-4 ONLINE 0 0 0 c5t50014EE056EB0B94d0 ONLINE 0 0 0 c5t50014EE0AC408605d0 ONLINE 0 0 0 mirror-5 ONLINE 0 0 0 c5t5000C50010C77647d0 ONLINE 0 0 0 c5t5000C50010C865DEd0 ONLINE 0 0 0 mirror-6 ONLINE 0 0 0 c5t5000C50010C7672Dd0 ONLINE 0 0 0 c5t5000C50010C77679d0 ONLINE 0 0 0 mirror-7 ONLINE 0 0 0 c5t50014EE001407113d0 ONLINE 0 0 0 c5t50014EE00194FB1Bd0 ONLINE 0 0 0 mirror-8 ONLINE 0 0 0 c5t5000C50008315CE5d0 ONLINE 0 0 0 c5t5000C5000A11B600d0 ONLINE 0 0 0 cache c1t2d0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 spares c5t5000C5000D46F13Dd0 AVAIL From c5t5000C50010C77647d0 to c5t5000C50008315CE5d0 are the 6 Seagate drives, they are 2 ST31000340AS and 4 ST31000340NS. The rest of the drives are all WD RE3 (WD1002FBYS). Could those Seagate's really be slowing down the array that much or there is something else in here that I should be trying to look at? I did the same dd on the main OS pool (2 mirrors) and got 63MB/s .. times 8 mirrors should give me 504MBs reads? tl;dr: My tank of 8 mirrors is giving 180MB writes, how to fix?! -- Mohammed Naser — vexxhost _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss