Under Solaris 10 on a 4 core Sun Ultra 40 with 20GB RAM, I am setting up a Sun StorageTek 2540 with 12 300GB 15K RPM SAS drives and connected via load-shared 4Gbit FC links. This week I have tried many different configurations, using firmware managed RAID, ZFS managed RAID, and with the controller cache enabled or disabled.
My objective is to obtain the best single-file write performance. Unfortunately, I am hitting some sort of write bottleneck and I am not sure how to solve it. I was hoping for a write speed of 300MB/second. With ZFS on top of a firmware managed RAID 0 across all 12 drives, I hit a peak of 200MB/second. With each drive exported as a LUN and a ZFS pool of 6 pairs, I see a write rate of 154MB/second. The number of drives used has not had much effect on write rate. Information on my pool is shown at the end of this email. I am driving the writes using 'iozone' since 'filebench' does not seem to want to install/work on Solaris 10. I am suspecting that the problem is that I am running out of IOPS since the drive array indicates a an average IOPS of 214 for one drive even though the peak write speed is only 26MB/second (peak read is 42MB/second). Can someone share with me what they think the write bottleneck might be and how I can surmount it? Thanks, Bob ====================================== Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ % zpool status pool: Sun_2540 state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM Sun_2540 ONLINE 0 0 0 mirror ONLINE 0 0 0 c4t600A0B80003A8A0B0000096A47B4559Ed0 ONLINE 0 0 0 c4t600A0B80003A8A0B0000096E47B456DAd0 ONLINE 0 0 0 mirror ONLINE 0 0 0 c4t600A0B80003A8A0B0000096147B451BEd0 ONLINE 0 0 0 c4t600A0B80003A8A0B0000096647B453CEd0 ONLINE 0 0 0 mirror ONLINE 0 0 0 c4t600A0B80003A8A0B0000097347B457D4d0 ONLINE 0 0 0 c4t600A0B800039C9B500000A9C47B4522Dd0 ONLINE 0 0 0 mirror ONLINE 0 0 0 c4t600A0B800039C9B500000AA047B4529Bd0 ONLINE 0 0 0 c4t600A0B800039C9B500000AA447B4544Fd0 ONLINE 0 0 0 mirror ONLINE 0 0 0 c4t600A0B800039C9B500000AA847B45605d0 ONLINE 0 0 0 c4t600A0B800039C9B500000AAC47B45739d0 ONLINE 0 0 0 mirror ONLINE 0 0 0 c4t600A0B800039C9B500000AB047B457ADd0 ONLINE 0 0 0 c4t600A0B800039C9B500000AB447B4595Fd0 ONLINE 0 0 0 errors: No known data errors freddy:~% zpool iostat capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- Sun_2540 64.0G 1.57T 808 861 99.8M 105M freddy:~% zpool iostat -v capacity operations bandwidth pool used avail read write read write -------------------------------------- ----- ----- ----- ----- ----- ----- Sun_2540 64.0G 1.57T 809 860 100M 105M mirror 10.7G 267G 135 143 16.7M 17.6M c4t600A0B80003A8A0B0000096A47B4559Ed0 - - 66 141 8.37M 17.6M c4t600A0B80003A8A0B0000096E47B456DAd0 - - 67 141 8.37M 17.6M mirror 10.7G 267G 135 143 16.7M 17.6M c4t600A0B80003A8A0B0000096147B451BEd0 - - 66 141 8.37M 17.6M c4t600A0B80003A8A0B0000096647B453CEd0 - - 66 141 8.37M 17.6M mirror 10.7G 267G 134 143 16.7M 17.6M c4t600A0B80003A8A0B0000097347B457D4d0 - - 66 141 8.34M 17.6M c4t600A0B800039C9B500000A9C47B4522Dd0 - - 66 141 8.32M 17.6M mirror 10.7G 267G 134 143 16.6M 17.6M c4t600A0B800039C9B500000AA047B4529Bd0 - - 66 141 8.32M 17.6M c4t600A0B800039C9B500000AA447B4544Fd0 - - 66 141 8.30M 17.6M mirror 10.7G 267G 134 143 16.6M 17.6M c4t600A0B800039C9B500000AA847B45605d0 - - 66 141 8.31M 17.6M c4t600A0B800039C9B500000AAC47B45739d0 - - 66 141 8.30M 17.6M mirror 10.7G 267G 134 143 16.6M 17.6M c4t600A0B800039C9B500000AB047B457ADd0 - - 66 141 8.30M 17.6M c4t600A0B800039C9B500000AB447B4595Fd0 - - 66 141 8.29M 17.6M -------------------------------------- ----- ----- ----- ----- ----- ----- _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
