On Sat, 4 Jul 2009, Jonathan Edwards wrote:
somehow i don't think that reading the first 64MB off (presumably) off a raw
disk device 3 times and picking the middle value is going to give you much
useful information on the overall state of the disks .. i believe this was
more of a quick hack to just validate that there's nothing too far out of the
norm, but with that said - what's the c2 and c3 device above? you've got to
be caching the heck out of that to get that unbelievable 13 GB/s - so you're
really only seeing memory speeds there
Agreed. It is just a quick sanity check. I think that the c2 and c3
devices are speedy USB drives.
more useful information would be something more like the old taz or some of
the disk IO latency tools when you're driving a workload.
What I see from 'iostat -cx' is a low latency (<= 4 ms) and low
workload while the data is being read, and then (periodically) a burst
of write data with much higher latency (40-64ms svc_t). The write
burst does not take long so it is clear that reading is the
bottleneck.
if you're using LUNs off an array - this might be another case of the
zfs_vdev_max_pending being tuned more for direct attach drives .. you could
be trying to queue up too much I/O against the RAID controller, particularly
if the RAID controller is also trying to prefetch out of it's cache.
I have played with zfs_vdev_max_pending before. It does dial down the
latency pretty linearly during the write phase (e.g. 35 queued I/Os
results in 64 ms svc_t).
you might want to dtrace this to break down where the latency is occuring ..
eg: is this a DNLC caching problem, ARC problem, or device level problem
also - is this really coming off a 2540? if so - you should probably
investigate the array throughput numbers and what's happening on the RAID
controller .. i typically find it helpful to understand what the raw hardware
is capable of (hence tools like vdbench to drive an anticipated load before i
configure anything) - and then attempting to configure the various tunables
to match after that
Yes, this comes off of a 2540. I used iozone for testing and see that
through zfs, the hardware is able to write a 64GB file at 380 MB/s and
read at 551 MB/s. Unfortunately, this does not seem to translate well
for the actual task.
Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
