Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-09 Thread Stephen C. Tweedie
Hi, On Wed, 2005-03-09 at 13:28, Jan Kara wrote: > Hmm. I see for example a place at jbd/commit.c, line 287 (which you > did not change in your patch) which does this and doesn't seem to be > protected against journal_unmap_buffer() (but maybe I miss something). > Not that I'd find that race pr

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-09 Thread Jan Kara
> On Tue, 2005-03-08 at 15:12, Jan Kara wrote: > > > Isn't also the following scenario dangerous? > > > > __journal_unfile_buffer(jh); > > journal_remove_journal_head(bh); > > It depends. I think the biggest problem here is that there's really no > written rule protecting this stuff unive

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-09 Thread Stephen C. Tweedie
Hi, On Tue, 2005-03-08 at 15:12, Jan Kara wrote: > Isn't also the following scenario dangerous? > > __journal_unfile_buffer(jh); > journal_remove_journal_head(bh); It depends. I think the biggest problem here is that there's really no written rule protecting this stuff universally. But

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-08 Thread Jan Kara
Hello, > On Mon, 2005-03-07 at 21:08, Stephen C. Tweedie wrote: > > > Right, that was what I was thinking might be possible. But for now I've > > just done the simple patch --- make sure we don't clear > > jh->b_transaction when we're just refiling buffers from one list to > > another. That s

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-08 Thread Stephen C. Tweedie
Hi, On Mon, 2005-03-07 at 21:08, Stephen C. Tweedie wrote: > Right, that was what I was thinking might be possible. But for now I've > just done the simple patch --- make sure we don't clear > jh->b_transaction when we're just refiling buffers from one list to > another. That should have the de

[PATCH] invalidate/o_direct livelock {was Re: [RFC] ext3/jbd race: releasing in-use journal_heads}

2005-03-08 Thread Stephen C. Tweedie
Hi, On Tue, 2005-03-08 at 09:28, Stephen C. Tweedie wrote: > I think it should be OK just to move the page->mapping != mapping test > above the page>index > end test. Sure, if all the pages have been > stolen by the time we see them, then we'll repeat without advancing > "next"; but we're still

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-08 Thread Stephen C. Tweedie
Hi, On Mon, 2005-03-07 at 23:50, Andrew Morton wrote: > truncate_inode_pages_range() seems to dtrt here. Can we do it in the same > manner in invalidate_inode_pages2_range()? > > > Something like: > - if (page->mapping != mapping || page->index > end) { > +

Re: [Ext2-devel] Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-08 Thread Suparna Bhattacharya
On Mon, Mar 07, 2005 at 11:37:42PM -0800, Andrew Morton wrote: > Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > > > (let me know if the interface in the patch > > I just posted seems like the right direction to use when we go for the > > cleanup) > > Well what are the semantics? Pass in an

Re: [Ext2-devel] Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Andrew Morton
Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > (let me know if the interface in the patch > I just posted seems like the right direction to use when we go for the > cleanup) Well what are the semantics? Pass in an inclusive max_index and the gang lookup functions terminate when they hit an

Re: [Ext2-devel] Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Suparna Bhattacharya
On Mon, Mar 07, 2005 at 10:46:18PM -0800, Andrew Morton wrote: > Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > > > > > yup, looks like the same issue we hit in wait_on_page_writeback_range > > during AIO work - probably want to break out of the outer loop as well > > when this happens. >

Re: [Ext2-devel] Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Andrew Morton
Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: > > > yup, looks like the same issue we hit in wait_on_page_writeback_range > during AIO work - probably want to break out of the outer loop as well > when this happens. The `next = page_index' before breaking will do that for us. > > How hard w

Re: [Ext2-devel] Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Suparna Bhattacharya
Just as an idea of what I meant (dug up an old WIP patch): --- radix-tree.c2004-04-01 10:32:15.384556136 +0530 +++ radix-tree.c.end2004-04-01 11:11:07.176069944 +0530 @@ -562,7 +562,8 @@ EXPORT_SYMBOL(radix_tree_gang_lookup); */ static unsigned int __lookup_tag(struct radix_tree_

Re: [Ext2-devel] Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Suparna Bhattacharya
yup, looks like the same issue we hit in wait_on_page_writeback_range during AIO work - probably want to break out of the outer loop as well when this happens. >From the old changelog: >> >> wait_on_page_writeback_range shouldn't wait for pages beyond the >> specified range. Ideally, the radix-

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Stephen C. Tweedie
Hi, On Mon, 2005-03-07 at 21:22, Stephen C. Tweedie wrote: > altgr-scrlck is showing a range of EIPs all in ext3_direct_IO-> > invalidate_inode_pages2_range(). I'm seeing > > invalidate_inode_pages2_range()->pagevec_lookup()->find_get_pages() In invalidate_inode_pages2_range(), what happens if

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Andrew Morton
"Stephen C. Tweedie" <[EMAIL PROTECTED]> wrote: > > In invalidate_inode_pages2_range(), what happens if we lookup a pagevec, > get a bunch of pages back, but all the pages in the vec are beyond the > end of the range we want? hmm, yes. Another one :( > @@ -271,12 +271,13 @@ int invalidate_inode_

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Stephen C. Tweedie
Hi, On Mon, 2005-03-07 at 21:11, Andrew Morton wrote: > > I'm having trouble testing it, though --- I seem to be getting livelocks > > in O_DIRECT running 400 fsstress processes in parallel; ring any bells? > > Nope. I dont think anyone has been that cruel to ext3 for a while. > I assume this

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Andrew Morton
"Stephen C. Tweedie" <[EMAIL PROTECTED]> wrote: > > I'm having trouble testing it, though --- I seem to be getting livelocks > in O_DIRECT running 400 fsstress processes in parallel; ring any bells? Nope. I dont think anyone has been that cruel to ext3 for a while. I assume this workload used t

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Stephen C. Tweedie
Hi, On Mon, 2005-03-07 at 20:31, Andrew Morton wrote: > jbd_lock_bh_journal_head() is supposed to be a > finegrained innermost lock whose mandate is purely for atomicity of adding > and removing the journal_head and the b_jcount refcounting. I don't recall > there being any deeper meaning than t

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Andrew Morton
"Stephen C. Tweedie" <[EMAIL PROTECTED]> wrote: > > Trouble is, that's not enough; journal_put_journal_head() can nuke the > buffer with merely the bh_journal_head lock held. In the code above it > would be enough to take the journal_head_lock over the unfile/file pair. > > Andrew, can you rem

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Stephen C. Tweedie
Hi, On Mon, 2005-03-07 at 16:40, Stephen C. Tweedie wrote: > Andrew, can you remember why we ended up with both of those locks in the > first place? If we can do it, the efficient way out here is to abandon > the journal_head_lock and use the bh_state_lock for both. We already > hold that over

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Stephen C. Tweedie
Hi, On Sat, 2005-03-05 at 00:04, Andrew Morton wrote: > Perhaps we could also fix this by elevating b_jcount whenever the jh is > being moved between lists? Possible. But jcount isn't atomic, and it requires the bh_journal_head lock to modify. Taking and dropping the lock twice around the __un

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Stephen C. Tweedie
Hi, On Mon, 2005-03-07 at 14:50, Jan Kara wrote: > I believe the other places should be safe (mostly by luck) as the > caller has made sure that __journal_remove_journal_head() won't do > anything (e.g. set b_transaction, b_next_transaction or such). Right; I've been looking through all the jo

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Jan Kara
> "Stephen C. Tweedie" <[EMAIL PROTECTED]> wrote: > > > > For the past few months there has been a slow but steady trickle of > > reports of oopses in kjournald. > > Yes, really tenuous stuff. Very glad if this is the fix! > > > Recently I got a couple of reports that > > were repeatable enough

Re: [Ext2-devel] [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-07 Thread Stephen C. Tweedie
Hi, On Fri, 2005-03-04 at 23:17, Badari Pulavarty wrote: > I looked at few journalling bugs recently on RHEL4 testing here. > I am wondering if your patch fixes this following BUG also ? > I never got to bottom of some of these journal panics - > since they are not easily reproducible Right...

Re: [Ext2-devel] [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-04 Thread Badari Pulavarty
Stephen, I looked at few journalling bugs recently on RHEL4 testing here. I am wondering if your patch fixes this following BUG also ? I never got to bottom of some of these journal panics - since they are not easily reproducible + I don't understand journal code well enough :( Assertion failure

Re: [RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-04 Thread Andrew Morton
"Stephen C. Tweedie" <[EMAIL PROTECTED]> wrote: > > For the past few months there has been a slow but steady trickle of > reports of oopses in kjournald. Yes, really tenuous stuff. Very glad if this is the fix! > Recently I got a couple of reports that > were repeatable enough to rerun with ext

[RFC] ext3/jbd race: releasing in-use journal_heads

2005-03-04 Thread Stephen C. Tweedie
Hi all, For the past few months there has been a slow but steady trickle of reports of oopses in kjournald. Recently I got a couple of reports that were repeatable enough to rerun with extra debugging code. It turns out that we're releasing a journal_head while it is still linked onto the transa