Re: [patch] fs: fix deadlocks in writeback_if_idle

2010-12-15 Thread Nick Piggin
On Tue, Nov 30, 2010 at 11:01:18AM +1100, Nick Piggin wrote: > On Mon, Nov 29, 2010 at 02:26:03PM -0800, Andrew Morton wrote: > > On Thu, 25 Nov 2010 14:53:56 +1100 > > Nick Piggin wrote: > > > On Wed, Nov 24, 2010 at 02:10:28PM +0100, Jan Kara wrote: > > &g

Re: [patch] fs: fix deadlocks in writeback_if_idle

2010-11-29 Thread Nick Piggin
On Mon, Nov 29, 2010 at 02:26:03PM -0800, Andrew Morton wrote: > On Thu, 25 Nov 2010 14:53:56 +1100 > Nick Piggin wrote: > > On Wed, Nov 24, 2010 at 02:10:28PM +0100, Jan Kara wrote: > > Well, for now, the easiest and simplest fix is my patch, I think. The > > obje

Re: [patch] fs: fix deadlocks in writeback_if_idle

2010-11-24 Thread Nick Piggin
On Wed, Nov 24, 2010 at 02:51:57PM -0800, Andrew Morton wrote: > On Wed, 24 Nov 2010 12:03:43 +1100 > Nick Piggin wrote: > > > On Tue, Nov 23, 2010 at 01:58:24PM -0500, Chris Mason wrote: > > > > > My original btrfs patch just exported the bdi_ funcs so that btrfs

Re: [patch] fs: fix deadlocks in writeback_if_idle

2010-11-24 Thread Nick Piggin
On Wed, Nov 24, 2010 at 02:10:28PM +0100, Jan Kara wrote: > On Wed 24-11-10 12:03:43, Nick Piggin wrote: > > > For the _nr variant that btrfs uses, it's worse for the filesystems > > > that don't have a 1:1 bdi<->sb mapping. It might not actually write any &

Re: [patch] fs: fix deadlocks in writeback_if_idle

2010-11-24 Thread Nick Piggin
On Tue, Nov 23, 2010 at 02:18:57PM +0100, Jan Kara wrote: > On Tue 23-11-10 21:11:49, Nick Piggin wrote: > > The issue of writeback_inodes_sb being synchronous so far as it has to > > wait until the work has been dequeued is another subtlety. That is a > > funny interface tho

Re: [patch] fs: fix deadlocks in writeback_if_idle

2010-11-23 Thread Nick Piggin
t; > > > functions don't suck any more. > > > > > > > > ext4 now passes extensive stress testing with xfstests, fs_mark, dbench, > > > > with a writeback_inodes_if_idle call added directly into > > > > ext4_da_write_begin > >

Re: [patch] fs: fix deadlocks in writeback_if_idle

2010-11-23 Thread Nick Piggin
; > to trigger the path frequently. Previously it would spew lockdep stuff and > > hang in a number of ways very quickly. > > > > Signed-off-by: Nick Piggin > > > > --- > > fs/fs-writeback.c | 32 ++

Re: [patch] fs: fix deadlocks in writeback_if_idle

2010-11-23 Thread Nick Piggin
On Tue, Nov 23, 2010 at 12:26:31PM +0200, Boaz Harrosh wrote: > > * > > * Invoke writeback_inodes_sb if no writeback is currently underway. > > * Returns 1 if writeback was started, 0 if not. > > + * > > + * Even if 1 is returned, writeback may not be started if memory allocation > > + * fail

Re: [patch] fs: fix deadlocks in writeback_if_idle

2010-11-23 Thread Nick Piggin
On Tue, Nov 23, 2010 at 09:02:39PM +1100, Nick Piggin wrote: > int writeback_inodes_sb_if_idle(struct super_block *sb) > { > if (!writeback_in_progress(sb->s_bdi)) { > - down_read(&sb->s_umount); > - writeback_inodes_sb(sb); > -

[patch] fs: fix deadlocks in writeback_if_idle

2010-11-23 Thread Nick Piggin
tions don't suck any more. ext4 now passes extensive stress testing with xfstests, fs_mark, dbench, with a writeback_inodes_if_idle call added directly into ext4_da_write_begin to trigger the path frequently. Previously it would spew lockdep stuff and hang in a number of ways

Re: [patch] fix up lock order reversal in writeback

2010-11-23 Thread Nick Piggin
Here is another one. Both i_mutex in writeback and s_umount in write(2) underneath i_mutex seem like an interesting idea. The former you can probably get away with (provided you solve the previous AA deadlock), but the latter seems too problematic. I think my trylock patch solves it. [ 409.479214

Re: [patch] fix up lock order reversal in writeback

2010-11-23 Thread Nick Piggin
On Mon, Nov 22, 2010 at 07:16:55PM +0100, Jan Kara wrote: > On Fri 19-11-10 16:16:19, Nick Piggin wrote: > > On Fri, Nov 19, 2010 at 01:45:52AM +0100, Jan Kara wrote: > > > On Wed 17-11-10 22:28:34, Andrew Morton wrote: > > > > The fact that a call to ->wri

Re: [patch] fix up lock order reversal in writeback

2010-11-18 Thread Nick Piggin
On Fri, Nov 19, 2010 at 01:45:52AM +0100, Jan Kara wrote: > On Wed 17-11-10 22:28:34, Andrew Morton wrote: > > The fact that a call to ->write_begin can randomly return with s_umount > > held, to be randomly released at some random time in the future is a > > bit ugly, isn't it? write_begin is a p

Re: [patch] fix up lock order reversal in writeback

2010-11-18 Thread Nick Piggin
On Thu, Nov 18, 2010 at 09:58:31AM -0800, Andrew Morton wrote: > On Thu, 18 Nov 2010 19:18:22 +1100 Nick Piggin wrote: > > > On Wed, Nov 17, 2010 at 10:28:34PM -0800, Andrew Morton wrote: > > > > > Logically I'd expect i_mutex to nest inside s_umount.

Re: [patch] fix up lock order reversal in writeback

2010-11-18 Thread Nick Piggin
On Wed, Nov 17, 2010 at 10:28:34PM -0800, Andrew Morton wrote: > On Thu, 18 Nov 2010 17:00:00 +1100 Nick Piggin wrote: > > > On Wed, Nov 17, 2010 at 07:29:00PM -0800, Andrew Morton wrote: > > > On Wed, 17 Nov 2010 22:06:13 -0500 "Ted Ts'o" wrote: > >

Re: [patch] fix up lock order reversal in writeback

2010-11-17 Thread Nick Piggin
On Wed, Nov 17, 2010 at 07:29:00PM -0800, Andrew Morton wrote: > On Wed, 17 Nov 2010 22:06:13 -0500 "Ted Ts'o" wrote: > > > On Wed, Nov 17, 2010 at 05:10:57PM +1100, Nick Piggin wrote: > > > On Tue, Nov 16, 2010 at 11:05:52PM -0600, Eric Sandeen wrote: >

Re: [patch] fix up lock order reversal in writeback

2010-11-16 Thread Nick Piggin
On Tue, Nov 16, 2010 at 11:05:52PM -0600, Eric Sandeen wrote: > On 11/16/10 10:38 PM, Nick Piggin wrote: > >> as for the locking problems ... sorry about that! > > > > That's no problem. So is that an ack? :) > > > > I'd like to test it with the ori

Re: [patch] fix up lock order reversal in writeback

2010-11-16 Thread Nick Piggin
On Tue, Nov 16, 2010 at 10:30:37PM -0600, Eric Sandeen wrote: > On 11/16/10 7:01 AM, Jan Kara wrote: > > On Tue 16-11-10 22:00:58, Nick Piggin wrote: > >> I saw a lock order warning on ext4 trigger. This should solve it. > >> Raciness shouldn't matter much,

Re: [patch] fix up lock order reversal in writeback

2010-11-16 Thread Nick Piggin
On Tue, Nov 16, 2010 at 12:32:52PM -0800, Andrew Morton wrote: > On Tue, 16 Nov 2010 22:00:58 +1100 > Nick Piggin wrote: > > > I saw a lock order warning on ext4 trigger. This should solve it. > > Send us the trace, please. I lost it, sorry. > The code comment

[patch] fix up lock order reversal in writeback

2010-11-16 Thread Nick Piggin
I saw a lock order warning on ext4 trigger. This should solve it. Raciness shouldn't matter much, because writeback can stop just after we make the test and return anyway (so the API is racy anyway). Signed-off-by: Nick Piggin Index: linux-2.6/fs/fs-writeb

Re: [patch] btrfs: use add_to_page_cache_lru, use __page_cache_alloc

2010-03-17 Thread Nick Piggin
On Wed, Mar 17, 2010 at 05:20:53PM +1100, Nick Piggin wrote: > btrfs: use add_to_page_cache_lru, use __page_cache_alloc > > Pagecache pages should be allocated with __page_cache_alloc, so they > obey pagecache memory policies. > > add_to_page_cache_lru is exported, so it should

[patch] btrfs: use add_to_page_cache_lru, use __page_cache_alloc

2010-03-16 Thread Nick Piggin
, percpu lru ordering is preserved, and finally don't need to flush pagevec before returning so batching may be shared with other LRU insertions. Signed-off-by: Nick Piggin : --- fs/btrfs/compression.c | 20 ++-- fs/btrfs/extent_io.c | 22 +- 2 files ch

[patch] btrfs: fix gfp flags masking

2010-03-15 Thread Nick Piggin
Signed-off-by: Nick Piggin -- Index: linux-2.6/fs/btrfs/compression.c === --- linux-2.6.orig/fs/btrfs/compression.c +++ linux-2.6/fs/btrfs/compression.c @@ -478,7 +478,7 @@ static noinline int add_ra_bio_pages(str

Re: btrfs failing fsx-linux

2009-09-09 Thread Nick Piggin
On Wed, Sep 09, 2009 at 09:37:29AM -0400, Chris Mason wrote: > On Wed, Sep 09, 2009 at 03:34:35PM +0200, Nick Piggin wrote: > > Which cases do set_page_dirty happen without page_mkwrite? For fsx-linux > > I hope there shouldn't be any... > > I've been assuming its

Re: btrfs failing fsx-linux

2009-09-09 Thread Nick Piggin
On Wed, Sep 09, 2009 at 09:13:49AM -0400, Chris Mason wrote: > On Tue, Aug 18, 2009 at 07:26:41PM +0200, Nick Piggin wrote: > > Hi Chris, > > > > Don't know if this is a known issue, but I have btrfs (after my previous > > inode_tree fixup patch) failing fsx-l

Re: btrfs failing fsx-linux

2009-09-03 Thread Nick Piggin
On Wed, Sep 02, 2009 at 03:07:21PM -0400, Chris Mason wrote: > On Wed, Sep 02, 2009 at 07:14:50PM +0200, Nick Piggin wrote: > > On Tue, Aug 18, 2009 at 01:59:50PM -0400, Josef Bacik wrote: > > > I'll take a look at this later today and see what I can come up with.

Re: btrfs failing fsx-linux

2009-09-02 Thread Nick Piggin
On Tue, Aug 18, 2009 at 01:59:50PM -0400, Josef Bacik wrote: > On Tue, Aug 18, 2009 at 07:26:41PM +0200, Nick Piggin wrote: > > Hi Chris, > > > > Don't know if this is a known issue, but I have btrfs (after my previous > > inode_tree fixup patch) failing fsx-l

Re: [patch] btrfs: fix inode rbtree corruption

2009-08-20 Thread Nick Piggin
On Wed, Aug 19, 2009 at 10:59:07AM +0200, Jens Axboe wrote: > On Wed, Aug 19 2009, Nick Piggin wrote: > > On Wed, Aug 19, 2009 at 10:46:59AM +0200, Jens Axboe wrote: > > > On Wed, Aug 19 2009, Nick Piggin wrote: > > > > See my other reply. It *can* work with ke

Re: [patch] btrfs: fix inode rbtree corruption

2009-08-19 Thread Nick Piggin
On Wed, Aug 19, 2009 at 05:34:08PM +0800, Yan, Zheng wrote: > 2009/8/19 Nick Piggin : > > On Wed, Aug 19, 2009 at 04:56:12PM +0800, Yan, Zheng  wrote: > > Firstly, the insert/delete code is wrong for duplicates and it will crash in > > the absense of any search activity. Ag

Re: [patch] btrfs: fix inode rbtree corruption

2009-08-19 Thread Nick Piggin
On Wed, Aug 19, 2009 at 04:56:12PM +0800, Yan, Zheng wrote: > 2009/8/19 Nick Piggin : > > On Tue, Aug 18, 2009 at 11:19:10PM +0200, Jens Axboe wrote: > >> On Wed, Aug 19 2009, Yan, Zheng  wrote: > >> > 2009/8/19 Nick Piggin : > >> It can work with key alia

Re: [patch] btrfs: fix inode rbtree corruption

2009-08-19 Thread Nick Piggin
On Wed, Aug 19, 2009 at 10:46:59AM +0200, Jens Axboe wrote: > On Wed, Aug 19 2009, Nick Piggin wrote: > > See my other reply. It *can* work with key aliases, but this particular > > code does not. > > > > It is pretty easy obviously to put in duplicates because the r

Re: btrfs failing fsx-linux

2009-08-19 Thread Nick Piggin
On Tue, Aug 18, 2009 at 01:59:50PM -0400, Josef Bacik wrote: > On Tue, Aug 18, 2009 at 07:26:41PM +0200, Nick Piggin wrote: > > Hi Chris, > > > > Don't know if this is a known issue, but I have btrfs (after my previous > > inode_tree fixup patch) failing fsx-l

Re: [patch] btrfs: fix inode rbtree corruption

2009-08-19 Thread Nick Piggin
On Tue, Aug 18, 2009 at 11:19:10PM +0200, Jens Axboe wrote: > On Wed, Aug 19 2009, Yan, Zheng wrote: > > 2009/8/19 Nick Piggin : > > > Hi, > > > > > > Ran into a problem stress testing my btrfs truncate conversion attempt... > > > Unfortunately it

Re: [patch] btrfs: fix inode rbtree corruption

2009-08-19 Thread Nick Piggin
On Wed, Aug 19, 2009 at 02:56:54AM +0800, Yan, Zheng wrote: > 2009/8/19 Nick Piggin : > > Hi, > > > > Ran into a problem stress testing my btrfs truncate conversion attempt... > > Unfortunately it was an existing btrfs problem. Fortunately I think I > > w

btrfs failing fsx-linux

2009-08-18 Thread Nick Piggin
Hi Chris, Don't know if this is a known issue, but I have btrfs (after my previous inode_tree fixup patch) failing fsx-linux in Linus's current git tree. This makes it a bit hard for me to test my btrfs truncate conversion patch unfortunately, though it does seem pretty stable so I will probably

[patch] btrfs: fix inode rbtree corruption

2009-08-18 Thread Nick Piggin
working again. Signed-off-by: Nick Piggin --- fs/btrfs/inode.c | 21 ++--- 1 file changed, 14 insertions(+), 7 deletions(-) Index: linux-2.6/fs/btrfs/inode.c === --- linux-2.6.orig/fs/btrfs/inode.c +++ linux-2.6/fs

Re: gcc inlining heuristics was Re: [PATCH -v7][RFC]: mutex: implement adaptive spinning

2009-01-21 Thread Nick Piggin
On Wed, Jan 21, 2009 at 10:54:18AM +0100, Andi Kleen wrote: > > The point is that the compiler is then free to do it. If things > > slow down after the compiler gets *more* information, then that > > is a problem with the compiler heuristics rather than the > > information we give it. > > The poin

Re: gcc inlining heuristics was Re: [PATCH -v7][RFC]: mutex: implement adaptive spinning

2009-01-21 Thread Nick Piggin
On Wed, Jan 21, 2009 at 10:20:49AM +0100, Andi Kleen wrote: > On Wed, Jan 21, 2009 at 09:52:08AM +0100, Nick Piggin wrote: > > On Wed, Jan 21, 2009 at 09:54:02AM +0100, Andi Kleen wrote: > > > > GCC 4.3.2. Maybe i missed something obvious? > > > > > > The

Re: gcc inlining heuristics was Re: [PATCH -v7][RFC]: mutex: implement adaptive spinning

2009-01-21 Thread Nick Piggin
On Wed, Jan 21, 2009 at 09:54:02AM +0100, Andi Kleen wrote: > > GCC 4.3.2. Maybe i missed something obvious? > > The typical use case of restrict is to tell it that multiple given > arrays are independent and then give the loop optimizer > more freedom to handle expressions in the loop that > acc

Re: gcc inlining heuristics was Re: [PATCH -v7][RFC]: mutex: implement adaptive spinning

2009-01-19 Thread Nick Piggin
On Tue, Jan 20, 2009 at 08:01:52AM +1100, Linus Torvalds wrote: > > > On Mon, 19 Jan 2009, Nick Piggin wrote: > > > > I want to know what is the problem with the restrict keyword? > > I'm sure I've read Linus ranting about how bad it is in the > >

Re: gcc inlining heuristics was Re: [PATCH -v7][RFC]: mutex: implement adaptive spinning

2009-01-18 Thread Nick Piggin
On Mon, Jan 19, 2009 at 01:13:45AM +0100, Ingo Molnar wrote: > > * Bernd Schmidt wrote: > > > Linus Torvalds wrote: > > > But you'll need some background to it: > > > > You paint a somewhat one-sided picture bordering on FUD. > > Type based aliasing should at most have been an opt-in for code

Re: [GIT PULL] adaptive spinning mutexes

2009-01-15 Thread Nick Piggin
On Thu, Jan 15, 2009 at 10:16:53AM -0800, Linus Torvalds wrote: > > > On Thu, 15 Jan 2009, Ingo Molnar wrote: > > > > btw., i think spin-mutexes have a design advantage here: in a lot of code > > areas it's quite difficult to use spinlocks - cannot allocate memory, > > cannot call any code tha

Re: [PATCH -v11][RFC] mutex: implement adaptive spinning

2009-01-14 Thread Nick Piggin
On Thu, Jan 15, 2009 at 08:44:03AM +0100, Peter Zijlstra wrote: > On Thu, 2009-01-15 at 01:46 +0100, Nick Piggin wrote: > > > Hmm, well this is rather a slow path, I would say. I'd prefer not to > > modify schedule in this way (if we just get scheduled back on after >

Re: [GIT PULL] adaptive spinning mutexes

2009-01-14 Thread Nick Piggin
On Wed, Jan 14, 2009 at 01:35:29PM -0800, Andrew Morton wrote: > You're taking a whizzy new feature which drastically changes a critical > core kernel feature and jamming it into mainline with a vestigial > amount of testing coverage without giving sufficient care and thought > to the practical les

Re: [PATCH -v8][RFC] mutex: implement adaptive spinning

2009-01-14 Thread Nick Piggin
On Wed, Jan 14, 2009 at 07:23:12PM +0200, Avi Kivity wrote: > Nick Piggin wrote: > >>(no they're not, Nick's ticket locks still spin on a shared cacheline > >>IIRC -- the MCS locks mentioned could fix this) > >> > > > >It reminds me. I wrote

Re: [PATCH -v11][RFC] mutex: implement adaptive spinning

2009-01-14 Thread Nick Piggin
On Wed, Jan 14, 2009 at 06:22:36PM +0100, Peter Zijlstra wrote: > On Wed, 2009-01-14 at 18:18 +0100, Nick Piggin wrote: > > > > @@ -173,21 +237,21 @@ __mutex_lock_common(struct mutex *lock, > > > spin_unlock_mutex

Re: [PATCH -v11][RFC] mutex: implement adaptive spinning

2009-01-14 Thread Nick Piggin
On Wed, Jan 14, 2009 at 06:00:36PM +0100, Peter Zijlstra wrote: > Full series, including changelogs available at: > > http://programming.kicks-ass.net/kernel-patches/mutex-adaptive-spin/ > > and should shortly appear in a git tree near Ingo :-) Linus is going to take a wholesale conversion of m

Re: [PATCH -v8][RFC] mutex: implement adaptive spinning

2009-01-14 Thread Nick Piggin
On Wed, Jan 14, 2009 at 05:46:39PM +0100, Peter Zijlstra wrote: > On Mon, 2009-01-12 at 19:32 +0200, Avi Kivity wrote: > > Peter Zijlstra wrote: > > > Spinlocks can use 'pure' MCS locks. > > > > > > > How about this, then. In mutex_lock(), keep wait_lock locked and only > > release it when sc

Re: generic pagecache to block mapping layer (was Re: Btrfs for mainline)

2009-01-06 Thread Nick Piggin
On Tuesday 06 January 2009 04:10:42 Nick Piggin wrote: > On Tuesday 06 January 2009 03:37:33 Chris Mason wrote: > > So, long term we can have a benchmarking contest, but I've got a little > > ways to go before that is a good idea. > > That would be good. This got me m

Re: [PATCH][RFC]: mutex: adaptive spin

2009-01-06 Thread Nick Piggin
On Tue, Jan 06, 2009 at 12:40:31PM +0100, Peter Zijlstra wrote: > Subject: mutex: adaptive spin > From: Peter Zijlstra > Date: Tue Jan 06 12:32:12 CET 2009 > > Based on the code in -rt, provide adaptive spins on generic mutexes. I guess it would be nice to add another type so you can test/conver

Re: generic pagecache to block mapping layer (was Re: Btrfs for mainline)

2009-01-05 Thread Nick Piggin
On Tuesday 06 January 2009 03:37:33 Chris Mason wrote: > On Tue, 2009-01-06 at 01:39 +1100, Nick Piggin wrote: > > [trim ccs] > > > > Feel free to ignore this diversion ;) I'd like to see btrfs go upstream > > sooner rather than later. But eventually we'll ha

Re: Btrfs for mainline

2009-01-05 Thread Nick Piggin
On Monday 05 January 2009 05:41:03 Matthew Wilcox wrote: > On Sun, Jan 04, 2009 at 07:21:50PM +0100, Peter Zijlstra wrote: > > The -rt tree has adaptive spin patches for the rtmutex code, its really > > not all that hard to do -- the rtmutex code is way more tricky than the > > regular mutexes due

generic pagecache to block mapping layer (was Re: Btrfs for mainline)

2009-01-05 Thread Nick Piggin
[trim ccs] Feel free to ignore this diversion ;) I'd like to see btrfs go upstream sooner rather than later. But eventually we'll have to resurrect fsblock vs extent map discussion. On Tuesday 06 January 2009 00:21:43 Chris Mason wrote: > On Mon, 2009-01-05 at 21:32 +1100, Nick

Re: Btrfs for mainline

2009-01-05 Thread Nick Piggin
On Saturday 03 January 2009 06:38:07 Chris Mason wrote: > On Sat, 2009-01-03 at 01:37 +0900, Ryusuke Konishi wrote: > > Hi, > > > > On Wed, 31 Dec 2008 18:19:09 -0500, Chris Mason wrote: > > > This has only btrfs as a module and would be the fastest way to see > > > the .c files. btrfs doesn't h