On Aug 27, 2009, at 11:29 AM, Bob Friesenhahn <bfrie...@simple.dallas.tx.us
> wrote:
On Thu, 27 Aug 2009, David Bond wrote:
I just noticed that if the server hasnt hit its target arc size,
the pauses are for maybe .5 seconds, but as soon as it hits its arc
target, the iops drop to around 50% of what it was and then there
are the longer pauses around 4-5 seconds. and then after every
pause the performance slows even more. So it appears it is
definately server side.
This is known behavior of zfs for asynchronous writes. Recent zfs
defers/aggregates writes up to one of these limits:
* 7/8ths of available RAM
* 5 seconds worth of write I/O (full speed write)
* 30 seconds aggregation time
Notice the 5 seconds. This 5 seconds results in the 4-6 second
pause and it seems that the aggregation time is 10 seconds on your
system with this write load. Systems with large amounts of RAM
encounter this issue more than systems with limited RAM.
I encountered the same problem so I put this in /etc/system:
* Set ZFS maximum TXG group size to 3932160000
set zfs:zfs_write_limit_override = 0xea600000
That's the option. When I was experiencing my writes starving reads I
set this to 512MB or the size of my NVRAM cache for my controller and
everything was happy again. Write flushes happened in less then a
second and my IO flattened out nicely.
-Ross
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
