On Thu, 25 Jun 2009, Ross wrote:
But the unintended side effect of this is that ZFS's attempt to
optimize writes will causes jerky read and write behaviour any time
you have a large amount of writes going on, and when you should be
pushing the disks to 100% usage you're never going to reach that as
it's always going to have 5s of inactivity, followed by 5s of
running the disks flat out.
In fact, I wonder if it's a simple as the disks ending up doing 5s
of reads, a delay for processing, 5s of writes, 5s of reads, etc...
It's probably efficient, but it's going to *feel* horrible, a 5s
delay is easily noticeable by the end user, and is a deal breaker
for many applications.
Yes, 5 seconds is a long time. For an application which mixes
computation with I/O it is not really acceptable for read I/O to go
away for up to 5 seconds. This represents time that the CPU is not
being used, and a time that the application may be unresponsive to the
user. When compression is used the impact is different, but the
compression itself consumes considerable CPU (and quite abruptly) so
that other applications (e.g. X11) stop responding during the
compress/write cycle.
The read problem is one of congestion. If I/O is congested with
massive writes, then reads don't work. It does not really matter how
fast your storage system is. If the 5 seconds of buffered writes are
larger than what the device driver and storage system buffering allows
for, then the I/O channel will be congested.
As an example, my storage array is demonstrated to be able to write
359MB/second but ZFS will blast data from memory as fast as it can,
and the storage path can not effectively absorb 1.8GB (359*5) of data
since the StorageTek 2500's internal buffers are much smaller than
that, and fiber channel device drivers are not allowed to consume much
memory either. To make matters worse, I am using ZFS mirrors so the
amount of data written to the array in those five seconds is doubled
to 3.6GB.
Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
