Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-14 Thread Miklos Szeredi
> Only if the queue depth is not bound. Queue depths are bound and so > the distance we can go over the threshold is limited. This is the > fundamental principle on which the throttling is based. > > Hence, if the queue is not full, then we will have either written > dirty pages to it (i.e

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-14 Thread Miklos Szeredi
Only if the queue depth is not bound. Queue depths are bound and so the distance we can go over the threshold is limited. This is the fundamental principle on which the throttling is based. Hence, if the queue is not full, then we will have either written dirty pages to it (i.e

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-13 Thread David Chinner
On Tue, Mar 13, 2007 at 09:21:59AM +0100, Miklos Szeredi wrote: > > > read request > > > sys_write > > > mutex_lock(i_mutex) > > > ... > > > balance_dirty_pages > > > submit write requests > > > loop ... write requests completed ... dirty still over limit ... > > > ...

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-13 Thread Miklos Szeredi
> > > IIUC, your problem is that there's another bdi that holds all the > > > dirty pages, and this throttle loop never flushes pages from that > > > other bdi and we sleep instead. It seems to me that the fundamental > > > problem is that to clean the pages we need to flush both bdi's, not > > >

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-13 Thread Miklos Szeredi
IIUC, your problem is that there's another bdi that holds all the dirty pages, and this throttle loop never flushes pages from that other bdi and we sleep instead. It seems to me that the fundamental problem is that to clean the pages we need to flush both bdi's, not just the bdi we

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-13 Thread David Chinner
On Tue, Mar 13, 2007 at 09:21:59AM +0100, Miklos Szeredi wrote: read request sys_write mutex_lock(i_mutex) ... balance_dirty_pages submit write requests loop ... write requests completed ... dirty still over limit ... ... loop forever Hmmm -

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-12 Thread David Chinner
On Mon, Mar 12, 2007 at 11:36:16PM +0100, Miklos Szeredi wrote: > I'll try to explain the reason for the deadlock first. Ah, thanks for that. > > IIUC, your problem is that there's another bdi that holds all the > > dirty pages, and this throttle loop never flushes pages from that > > other bdi

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-12 Thread Miklos Szeredi
I'll try to explain the reason for the deadlock first. > IIUC, your problem is that there's another bdi that holds all the > dirty pages, and this throttle loop never flushes pages from that > other bdi and we sleep instead. It seems to me that the fundamental > problem is that to clean the pages

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-12 Thread David Chinner
On Mon, Mar 12, 2007 at 12:40:47PM +0100, Miklos Szeredi wrote: > > > I have no idea how serious the scalability problems with this are. If > > > they are serious, different solutions can probably be found for the > > > above, but this is certainly the simplest. > > > > Atomic operations to a

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-12 Thread Miklos Szeredi
> > I have no idea how serious the scalability problems with this are. If > > they are serious, different solutions can probably be found for the > > above, but this is certainly the simplest. > > Atomic operations to a single per-backing device from all CPUs at once? > That's a pretty serious

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-12 Thread Miklos Szeredi
I have no idea how serious the scalability problems with this are. If they are serious, different solutions can probably be found for the above, but this is certainly the simplest. Atomic operations to a single per-backing device from all CPUs at once? That's a pretty serious

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-12 Thread David Chinner
On Mon, Mar 12, 2007 at 12:40:47PM +0100, Miklos Szeredi wrote: I have no idea how serious the scalability problems with this are. If they are serious, different solutions can probably be found for the above, but this is certainly the simplest. Atomic operations to a single

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-12 Thread Miklos Szeredi
I'll try to explain the reason for the deadlock first. IIUC, your problem is that there's another bdi that holds all the dirty pages, and this throttle loop never flushes pages from that other bdi and we sleep instead. It seems to me that the fundamental problem is that to clean the pages we

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-12 Thread David Chinner
On Mon, Mar 12, 2007 at 11:36:16PM +0100, Miklos Szeredi wrote: I'll try to explain the reason for the deadlock first. Ah, thanks for that. IIUC, your problem is that there's another bdi that holds all the dirty pages, and this throttle loop never flushes pages from that other bdi and we

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-11 Thread David Chinner
On Tue, Mar 06, 2007 at 07:04:46PM +0100, Miklos Szeredi wrote: > From: Andrew Morton <[EMAIL PROTECTED]> > > [EMAIL PROTECTED]: bugfix] > > Miklos Szeredi <[EMAIL PROTECTED]>: > > Changes: > - updated to apply after clear_page_dirty_for_io() race fix > > This is needed for > > -

Re: [patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-11 Thread David Chinner
On Tue, Mar 06, 2007 at 07:04:46PM +0100, Miklos Szeredi wrote: From: Andrew Morton [EMAIL PROTECTED] [EMAIL PROTECTED]: bugfix] Miklos Szeredi [EMAIL PROTECTED]: Changes: - updated to apply after clear_page_dirty_for_io() race fix This is needed for - balance_dirty_pages()

[patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-06 Thread Miklos Szeredi
From: Andrew Morton <[EMAIL PROTECTED]> [EMAIL PROTECTED]: bugfix] Miklos Szeredi <[EMAIL PROTECTED]>: Changes: - updated to apply after clear_page_dirty_for_io() race fix This is needed for - balance_dirty_pages() deadlock fix - fuse dirty page accounting I have no idea how serious the

[patch 3/8] per backing_dev dirty and writeback page accounting

2007-03-06 Thread Miklos Szeredi
From: Andrew Morton [EMAIL PROTECTED] [EMAIL PROTECTED]: bugfix] Miklos Szeredi [EMAIL PROTECTED]: Changes: - updated to apply after clear_page_dirty_for_io() race fix This is needed for - balance_dirty_pages() deadlock fix - fuse dirty page accounting I have no idea how serious the