Re[2]: [zfs-discuss] ZFS multi-threading
Hello eric, Thursday, February 22, 2007, 5:44:35 PM, you wrote: ek On Feb 9, 2007, at 8:02 AM, Carisdad wrote: I've seen very good performance on streaming large files to ZFS on a T2000. We have been looking at using the T2000 as a disk storage unit for backups. I've been able to push over 500MB/s to the disks. Setup is EMC Clariion CX3 with 84 500GB SATA drives connected w/ 4Gbps all the way to the disk shelves. The 84 drives are presented as raw luns to the T2000 -- no HW RAID enabled on the Clariion. The problem we've seen comes when enabling compression, as that is single threaded per zpool. Enabling compression drops our throughput to 12-15MB/s per pool. This is bugid: 6460622, the fix is apparently set to be put back into Nevada fairly soon. ek This was just putback and will be in snv_59. ek enjoy your unconstrained compressed I/O, That is great news! Now I have to ask thru offician channels for a backport to s10. -- Best regards, Robertmailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS multi-threading
On Feb 9, 2007, at 8:02 AM, Carisdad wrote: I've seen very good performance on streaming large files to ZFS on a T2000. We have been looking at using the T2000 as a disk storage unit for backups. I've been able to push over 500MB/s to the disks. Setup is EMC Clariion CX3 with 84 500GB SATA drives connected w/ 4Gbps all the way to the disk shelves. The 84 drives are presented as raw luns to the T2000 -- no HW RAID enabled on the Clariion. The problem we've seen comes when enabling compression, as that is single threaded per zpool. Enabling compression drops our throughput to 12-15MB/s per pool. This is bugid: 6460622, the fix is apparently set to be put back into Nevada fairly soon. This was just putback and will be in snv_59. enjoy your unconstrained compressed I/O, eric ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS multi-threading
eric kustarz wrote: On Feb 9, 2007, at 8:02 AM, Carisdad wrote: I've seen very good performance on streaming large files to ZFS on a T2000. We have been looking at using the T2000 as a disk storage unit for backups. I've been able to push over 500MB/s to the disks. Setup is EMC Clariion CX3 with 84 500GB SATA drives connected w/ 4Gbps all the way to the disk shelves. The 84 drives are presented as raw luns to the T2000 -- no HW RAID enabled on the Clariion. The problem we've seen comes when enabling compression, as that is single threaded per zpool. Enabling compression drops our throughput to 12-15MB/s per pool. This is bugid: 6460622, the fix is apparently set to be put back into Nevada fairly soon. This was just putback and will be in snv_59. enjoy your unconstrained compressed I/O, eric Awesome, thank you. I almost hate to ask, but... Is there any way to track when the fix will make its way back into Solaris 10 via update4(?) or a patch? Thanks again, -Andy ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS multi-threading
On Feb 22, 2007, at 10:01 AM, Carisdad wrote: eric kustarz wrote: On Feb 9, 2007, at 8:02 AM, Carisdad wrote: I've seen very good performance on streaming large files to ZFS on a T2000. We have been looking at using the T2000 as a disk storage unit for backups. I've been able to push over 500MB/s to the disks. Setup is EMC Clariion CX3 with 84 500GB SATA drives connected w/ 4Gbps all the way to the disk shelves. The 84 drives are presented as raw luns to the T2000 -- no HW RAID enabled on the Clariion. The problem we've seen comes when enabling compression, as that is single threaded per zpool. Enabling compression drops our throughput to 12-15MB/s per pool. This is bugid: 6460622, the fix is apparently set to be put back into Nevada fairly soon. This was just putback and will be in snv_59. enjoy your unconstrained compressed I/O, eric Awesome, thank you. I almost hate to ask, but... Is there any way to track when the fix will make its way back into Solaris 10 via update4(?) or a patch? we're gonna try and squeeze it into s10u4, no guarantees just yet though. If it makes it, i'll update the list... eric ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ZFS multi-threading
With the CPU overhead imposed in checksum of blocks by ZFS, on a large sequential write test, the CPU was heavily loaded in a test that I ran. By turning off the checksum, the CPU load was greatly reduced. Obviously, this caused a tradeoff in reliability for CPU cycles. Would the logic behind ZFS take full advantage of a heavily multicored system, such as on the Sun Niagara platform? Would it utilize of the 32 concurrent threads for generating its checksums? Has anyone compared ZFS on a Sun Tx000, to that of a 2-4 thread x64 machine? This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS multi-threading
With the CPU overhead imposed in checksum of blocks by ZFS, on a large sequential write test, the CPU was heavily loaded in a test that I ran. By turning off the checksum, the CPU load was greatly reduced. Obviously, this caused a tradeoff in reliability for CPU cycles. What hardware platform and what was the I/O throughput at the peak and what was the difference in throughput and CPU utilization between both cases? Casper ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS multi-threading
Would the logic behind ZFS take full advantage of a heavily multicored system, such as on the Sun Niagara platform? Would it utilize of the 32 concurrent threads for generating its checksums? Has anyone compared ZFS on a Sun Tx000, to that of a 2-4 thread x64 machine? Pete and I are working on resolving ZFS scalability issues with Niagara and StarCat right now. I'm not sure if any official numbers about ZFS performance on Niagara have been published. As far as concurrent threads generating checksums goes, the system doesn't work quite the way you have postulated. The checksum is generated in the ZIO_STAGE_CHECKSUM_GENERATE pipeline state for writes, and verified in the ZIO_STAGE_CHECKSUM_VERIFY pipeline stage for reads. Whichever thread happens to advance the pipline to the checksum generate stage is the thread that will actually perform the work. ZFS does not break the work of the checksum into chunks and have multiple CPUs perform the computation. However, it is possible to have concurrent writes simultaneously in the checksum_generate stage. More details about this can be found in zfs/zio.c and zfs/sys/zio_impl.h -j ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
