What more info could you provide? Quite a lot more, actually, like: how many streams of SQL and copy are you running? how are the filesystems/zvols configured (recordsize, etc)? some CPU, VM and network stats would also be nice.
Based on the nexenta iostats you've provided (a tiny window on what's happening), it appears that you have an 8k recordsize for SQL. If you add up all the IOPS for the SQL, it's roughly 2000 reads at around 3ms each. Which might indicate at least 6 reads outstanding at any time. So how many queries do you have running in parallel? If you add more, I'd expect the service times to increase. 3ms isn't much for spinning rust, but isn't this why you are planning to use lots of L2ARC? Could be a similar story on writes. How many parallel streams? How many files? What's the average file size? What's the client filesysyem? How much does it sync to the server? Could it be that your client apps are always waiting for the spinning rust? Does an SSD log make any difference on this pool? Sent from my iPhone On 22 Oct 2010, at 19:57, Ian D <rewar...@hotmail.com> wrote: > Some numbers... > > zpool status > > pool: Pool_sas > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > Pool_sas ONLINE 0 0 0 > c4t5000C5000006A6D3d0 ONLINE 0 0 0 > c4t5000C5000006A777d0 ONLINE 0 0 0 > c4t5000C5000006AA43d0 ONLINE 0 0 0 > c4t5000C5000006AC4Fd0 ONLINE 0 0 0 > c4t5000C5000006AEF7d0 ONLINE 0 0 0 > c4t5000C5000006B27Fd0 ONLINE 0 0 0 > c4t5000C5000006B28Bd0 ONLINE 0 0 0 > c4t5000C5000006B46Bd0 ONLINE 0 0 0 > c4t5000C5000006B563d0 ONLINE 0 0 0 > c4t5000C5000006B643d0 ONLINE 0 0 0 > c4t5000C5000006B6D3d0 ONLINE 0 0 0 > c4t5000C5000006BBE7d0 ONLINE 0 0 0 > c4t5000C5000006C407d0 ONLINE 0 0 0 > c4t5000C5000006C657d0 ONLINE 0 0 0 > > errors: No known data errors > > pool: Pool_test > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > Pool_test ONLINE 0 0 0 > c4t5000C5002103F093d0 ONLINE 0 0 0 > c4t5000C50021101683d0 ONLINE 0 0 0 > c4t5000C50021102AA7d0 ONLINE 0 0 0 > c4t5000C500211034D3d0 ONLINE 0 0 0 > c4t5000C500211035DFd0 ONLINE 0 0 0 > c4t5000C5002110480Fd0 ONLINE 0 0 0 > c4t5000C50021104F0Fd0 ONLINE 0 0 0 > c4t5000C50021119A43d0 ONLINE 0 0 0 > c4t5000C5002112392Fd0 ONLINE 0 0 0 > > errors: No known data errors > > pool: syspool > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > syspool ONLINE 0 0 0 > c0t0d0s0 ONLINE 0 0 0 > > errors: No known data errors > ========================================================= > > Pool_sas is made of 14x 146G 15K SAS Drives in a big stripe. For this test > there is no log device or cache. Connected to it is a RedHat box using iSCSI > through an Intel X520 10GbE NIC. It runs several large MySQL queries at once- > each taking minutes to compute. > > Pool_test is a stripe of 2TB SATA drives and a terrabyte of files is being > copied to it for another box during this test. > > Here's the pastebin of "iostat -xdn 10" on the Linux box: > http://pastebin.com/431ESYaz > > Here's the pastebin of "iostat -xdn 10" on the Nexenta box: > http://pastebin.com/9g7KD3Ku > > Here's the pastebin "zpool iostat -v 10" on the Nexenta box: > http://pastebin.com/05fJL5sw > > From these numbers it looks like the Linux box is waiting for data all the > time while the Nexenta box isn't pulling nearly as much throughput and IOPS > as it could. Where is the bottleneck? > > One thing suspicious is that we notice a slow down of one pool when the other > is under load. How can that be? > > Ian > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
