I was all ready to write about my frustrations with
this problem, but I upgraded to snv_117 last night to
fix some iscsi bugs and now it seems that the write
throttling is working as described in that blog.
I may have been a little premature. While everything is much improved for Samba
and
On 02.07.09 22:05, Bob Friesenhahn wrote:
On Thu, 2 Jul 2009, Zhu, Lejun wrote:
Actually it seems to be 3/4:
3/4 is an awful lot. That would be 15 GB on my system, which explains
why the 5 seconds to write rule is dominant.
3/4 is 1/8 * 6, where 6 is worst-case inflation factor (for
Is the system otherwise responsive during the zfs sync cycles?
I ask because I think I'm seeing a similar thing - except that it's not
only other writers that block , it seems like other interrupts are
blocked. Pinging my zfs server in 1s intervals results in large delays
while the system
On Thu, 2 Jul 2009, Zhu, Lejun wrote:
Actually it seems to be 3/4:
3/4 is an awful lot. That would be 15 GB on my system, which explains
why the 5 seconds to write rule is dominant.
It seems that both rules are worthy of re-consideration.
There is also still the little problem that zfs
Note that this issue does not apply at all to NFS
service, database
service, or any other usage which does synchronous
writes.
Bob
Hello Bob,
There is impact for all workloads.
The fact that the write is sync or not, is just a question to write on slog
(SSD) or not.
But the txg
Actually it seems to be 3/4:
dsl_pool.c
391 zfs_write_limit_max = ptob(physmem)
zfs_write_limit_shift;
392 zfs_write_limit_inflated = MAX(zfs_write_limit_min,
393 spa_get_asize(dp-dp_spa, zfs_write_limit_max));
While spa_get_asize
backup windows using primarily iSCSI. When those
writes occur to my RaidZ volume, all activity pauses until the writes
are fully flushed.
The more I read about this, the worse it sounds. The thing is, I can see where
the ZFS developers are coming from - in theory this is a more efficient use
On Tue, 30 Jun 2009, Ross wrote:
However, it completely breaks any process like this that can't
afford 3-5s delays in processing, it makes ZFS a nightmare for
things like audio or video editing (where it would otherwise be a
perfect fit), and it's also horrible from the perspective of the
For what it is worth, I too have seen this behavior when load testing our zfs
box. I used iometer and the RealLife profile (1 worker, 1 target, 65% reads,
60% random, 8k, 32 IOs in the queue). When writes are being dumped, reads drop
close to zero, from 600-700 read IOPS to 15-30 read IOPS.
On Mon, 29 Jun 2009, Lejun Zhu wrote:
With ZFS write throttle, the number 2.5GB is tunable. From what I've
read in the code, it is possible to e.g. set
zfs:zfs_write_limit_override = 0x800 (bytes) to make it write
128M instead.
This works, and the difference in behavior is profound.
On Tue, Jun 30, 2009 at 12:25 PM, Bob
Friesenhahnbfrie...@simple.dallas.tx.us wrote:
On Mon, 29 Jun 2009, Lejun Zhu wrote:
With ZFS write throttle, the number 2.5GB is tunable. From what I've read
in the code, it is possible to e.g. set zfs:zfs_write_limit_override =
0x800 (bytes) to make
On Tue, 30 Jun 2009, Brent Jones wrote:
Maybe there could be a supported ZFS tuneable (per file system even?)
that is optimized for 'background' tasks, or 'foreground'.
Beyond that, I will give this tuneable a shot and see how it impacts
my own workload.
Note that this issue does not apply
CPU is smoothed out quite a lot
yes, but the area under the CPU graph is less, so the
rate of real work performed is less, so the entire
job took longer. (allbeit smoother)
Rob
___
zfs-discuss mailing list
Interesting to see that it makes such a difference, but I wonder what effect it
has on ZFS's write ordering, and it's attempts to prevent fragmentation?
By reducing the write buffer, are you loosing those benefits?
Although on the flip side, I guess this is no worse off than any other
On Tue, 30 Jun 2009, Rob Logan wrote:
CPU is smoothed out quite a lot
yes, but the area under the CPU graph is less, so the
rate of real work performed is less, so the entire
job took longer. (allbeit smoother)
For the purpose of illustration, the case showing the huge sawtooth
was when
On Tue, 30 Jun 2009, Bob Friesenhahn wrote:
Note that this issue does not apply at all to NFS
service, database
service, or any other usage which does synchronous
writes.
I see read starvation with NFS. I was using iometer on a Windows VM, connecting
to an NFS mount on a 2008.11 physical
On Wed, 24 Jun 2009, Lejun Zhu wrote:
There is a bug in the database about reads blocked by writes which may be
related:
http://bugs.opensolaris.org/view_bug.do?bug_id=6471212
The symptom is sometimes reducing queue depth makes read perform better.
I have been banging away at this issue
On Mon, Jun 29, 2009 at 2:48 PM, Bob
Friesenhahnbfrie...@simple.dallas.tx.us wrote:
On Wed, 24 Jun 2009, Lejun Zhu wrote:
There is a bug in the database about reads blocked by writes which may be
related:
http://bugs.opensolaris.org/view_bug.do?bug_id=6471212
The symptom is sometimes
On Wed, 24 Jun 2009, Lejun Zhu wrote:
There is a bug in the database about reads blocked
by writes which may be related:
http://bugs.opensolaris.org/view_bug.do?bug_id=6471212
The symptom is sometimes reducing queue depth makes
read perform better.
I have been banging away at
I am not sure how zfs would know the rate of the
underlying disk storage
Easy: Is the buffer growing? :-)
If the amount of data in the buffer is growing, you need to throttle back a bit
until the disks catch up. Don't stop writes until the buffer is empty, just
slow them down to match
On Wed, 24 Jun 2009, Lejun Zhu wrote:
There is a bug in the database about reads blocked by writes which may be
related:
http://bugs.opensolaris.org/view_bug.do?bug_id=6471212
The symptom is sometimes reducing queue depth makes read perform better.
This one certainly sounds promising.
On Thu, 25 Jun 2009, Ross wrote:
But the unintended side effect of this is that ZFS's attempt to
optimize writes will causes jerky read and write behaviour any time
you have a large amount of writes going on, and when you should be
pushing the disks to 100% usage you're never going to reach
http://opensolaris.org/jive/thread.jspa?threadID=105702tstart=0
Yes, this does sound very similar. It looks to me like data from read
files is clogging the ARC so that there is no more room for more
writes when ZFS periodically goes to commit unwritten data.
I'm wondering if changing
On Wed, 24 Jun 2009, Ethan Erchinger wrote:
http://opensolaris.org/jive/thread.jspa?threadID=105702tstart=0
Yes, this does sound very similar. It looks to me like data from read
files is clogging the ARC so that there is no more room for more
writes when ZFS periodically goes to commit
Wouldn't it make sense for the timing technique to be used if the data is
coming in at a rate slower than the underlying disk storage?
But then if the data starts to come at a faster rate, ZFS needs to start
streaming to disk as quickly as it can, and instead of re-ordering writes in
blocks,
Bob Friesenhahn wrote:
On Wed, 24 Jun 2009, Marcelo Leal wrote:
Hello Bob,
I think that is related with my post about zio_taskq_threads and TXG
sync :
( http://www.opensolaris.org/jive/thread.jspa?threadID=105703tstart=0 )
Roch did say that this is on top of the performance problems, and in
On Wed, 24 Jun 2009, Marcelo Leal wrote:
I think that is the purpose of the current implementation:
http://blogs.sun.com/roch/entry/the_new_zfs_write_throttle But seems
like is not that easy... as i did understand what Roch said, seems
like the cause is not always a hardy writer.
I see
On Thu, 25 Jun 2009, Ian Collins wrote:
I wonder whether a filesystem property streamed might be appropriate? This
could act as hint to ZFS that the data is sequential and should be streamed
direct to disk.
ZFS does not seem to offer an ability to stream direct to disk other
than perhaps
Bob Friesenhahn wrote:
On Wed, 24 Jun 2009, Marcelo Leal wrote:
I think that is the purpose of the current implementation:
http://blogs.sun.com/roch/entry/the_new_zfs_write_throttle But seems
like is not that easy... as i did understand what Roch said, seems
like the cause is not always a
On Wed, 24 Jun 2009, Richard Elling wrote:
The new code keeps track of the amount of data accepted in a TXG and the
time it takes to sync. It dynamically adjusts that amount so that each TXG
sync takes about 5 seconds (txg_time variable). It also clamps the limit to
no more than 1/8th of
On Wed, 24 Jun 2009, Richard Elling wrote:
The new code keeps track of the amount of data
accepted in a TXG and the
time it takes to sync. It dynamically adjusts that
amount so that each TXG
sync takes about 5 seconds (txg_time variable). It
also clamps the limit to
no more than
It has been quite some time (about a year) since I did testing of
batch processing with my software (GraphicsMagick). In between time,
ZFS added write-throttling. I am using Solaris 10 with kernel
141415-03.
Quite a while back I complained that ZFS was periodically stalling the
writing
is this a direct write to a zfs filesystem or is it some kind of zvol export?
anyway, sounds similar to this:
http://opensolaris.org/jive/thread.jspa?threadID=105702tstart=0
On Tue, Jun 23, 2009 at 7:14 PM, Bob
Friesenhahnbfrie...@simple.dallas.tx.us wrote:
It has been quite some time (about a
33 matches
Mail list logo
