Re: dcache shrink list corruption?

2014-05-06 Thread Al Viro
On Tue, May 06, 2014 at 10:01:21AM -0700, Linus Torvalds wrote: > On Tue, May 6, 2014 at 9:52 AM, Al Viro wrote: > > > > OK... There's one more thing I would like to put there, if you are going to > > be away for the week. It has sat in -next for a while, and it could stay > > there, except

Re: dcache shrink list corruption?

2014-05-06 Thread Linus Torvalds
On Tue, May 6, 2014 at 9:52 AM, Al Viro wrote: > > OK... There's one more thing I would like to put there, if you are going to > be away for the week. It has sat in -next for a while, and it could stay > there, except that there's a _lot_ of followups touching stuff all over the > tree and I'd

Re: dcache shrink list corruption?

2014-05-06 Thread Al Viro
On Tue, May 06, 2014 at 07:53:19AM -0700, Linus Torvalds wrote: > On Tue, May 6, 2014 at 3:17 AM, Miklos Szeredi wrote: > > > > Patches look okay to me. > > > > Reviewed-by: Miklos Szeredi > > > >> dentry_kill(): don't try to remove from shrink list > > > > Backport of this to 3.12 was

Re: dcache shrink list corruption?

2014-05-06 Thread Linus Torvalds
On Tue, May 6, 2014 at 3:17 AM, Miklos Szeredi wrote: > > Patches look okay to me. > > Reviewed-by: Miklos Szeredi > >> dentry_kill(): don't try to remove from shrink list > > Backport of this to 3.12 was tested by IBM and apparently fixes the > issue for them (I didn't backport the

Re: dcache shrink list corruption?

2014-05-06 Thread Miklos Szeredi
On Sun, May 4, 2014 at 8:29 AM, Al Viro wrote: > On Sat, May 03, 2014 at 07:21:11PM +0100, Al Viro wrote: >> On Sat, May 03, 2014 at 05:26:04AM +0100, Al Viro wrote: >> >> > See vfs.git#dentry_kill-3; warning - this is completely untested and I >> > would >> > really like comments on spinning

Re: dcache shrink list corruption?

2014-05-06 Thread Miklos Szeredi
On Sun, May 4, 2014 at 8:29 AM, Al Viro v...@zeniv.linux.org.uk wrote: On Sat, May 03, 2014 at 07:21:11PM +0100, Al Viro wrote: On Sat, May 03, 2014 at 05:26:04AM +0100, Al Viro wrote: See vfs.git#dentry_kill-3; warning - this is completely untested and I would really like comments on

Re: dcache shrink list corruption?

2014-05-06 Thread Linus Torvalds
On Tue, May 6, 2014 at 3:17 AM, Miklos Szeredi mik...@szeredi.hu wrote: Patches look okay to me. Reviewed-by: Miklos Szeredi mszer...@suse.cz dentry_kill(): don't try to remove from shrink list Backport of this to 3.12 was tested by IBM and apparently fixes the issue for them (I

Re: dcache shrink list corruption?

2014-05-06 Thread Al Viro
On Tue, May 06, 2014 at 07:53:19AM -0700, Linus Torvalds wrote: On Tue, May 6, 2014 at 3:17 AM, Miklos Szeredi mik...@szeredi.hu wrote: Patches look okay to me. Reviewed-by: Miklos Szeredi mszer...@suse.cz dentry_kill(): don't try to remove from shrink list Backport of this

Re: dcache shrink list corruption?

2014-05-06 Thread Linus Torvalds
On Tue, May 6, 2014 at 9:52 AM, Al Viro v...@zeniv.linux.org.uk wrote: OK... There's one more thing I would like to put there, if you are going to be away for the week. It has sat in -next for a while, and it could stay there, except that there's a _lot_ of followups touching stuff all over

Re: dcache shrink list corruption?

2014-05-06 Thread Al Viro
On Tue, May 06, 2014 at 10:01:21AM -0700, Linus Torvalds wrote: On Tue, May 6, 2014 at 9:52 AM, Al Viro v...@zeniv.linux.org.uk wrote: OK... There's one more thing I would like to put there, if you are going to be away for the week. It has sat in -next for a while, and it could stay

Re: dcache shrink list corruption?

2014-05-04 Thread Al Viro
On Sat, May 03, 2014 at 07:21:11PM +0100, Al Viro wrote: > On Sat, May 03, 2014 at 05:26:04AM +0100, Al Viro wrote: > > > See vfs.git#dentry_kill-3; warning - this is completely untested and I would > > really like comments on spinning case there (i.e. the one where > > select_collect() > >

Re: dcache shrink list corruption?

2014-05-04 Thread Al Viro
On Sat, May 03, 2014 at 07:21:11PM +0100, Al Viro wrote: On Sat, May 03, 2014 at 05:26:04AM +0100, Al Viro wrote: See vfs.git#dentry_kill-3; warning - this is completely untested and I would really like comments on spinning case there (i.e. the one where select_collect() finds some

Re: dcache shrink list corruption?

2014-05-03 Thread Al Viro
On Sat, May 03, 2014 at 11:07:57AM -0700, Linus Torvalds wrote: > Sure, umount itself should be serialized by the sb lock, so there > should be only one umount dentry collector. But why wouldn't there be > shrinkers active due to memory pressure? > > generic_unmount_super() is called by

Re: dcache shrink list corruption?

2014-05-03 Thread Al Viro
On Sat, May 03, 2014 at 05:26:04AM +0100, Al Viro wrote: > See vfs.git#dentry_kill-3; warning - this is completely untested and I would > really like comments on spinning case there (i.e. the one where > select_collect() > finds some stuff already on some other shrink list and nothing with zero

Re: dcache shrink list corruption?

2014-05-03 Thread Linus Torvalds
On Fri, May 2, 2014 at 9:26 PM, Al Viro wrote: > > See vfs.git#dentry_kill-3; warning - this is completely untested and I would > really like comments on spinning case there (i.e. the one where > select_collect() > finds some stuff already on some other shrink list and nothing with zero >

Re: dcache shrink list corruption?

2014-05-03 Thread Linus Torvalds
On Fri, May 2, 2014 at 9:26 PM, Al Viro v...@zeniv.linux.org.uk wrote: See vfs.git#dentry_kill-3; warning - this is completely untested and I would really like comments on spinning case there (i.e. the one where select_collect() finds some stuff already on some other shrink list and nothing

Re: dcache shrink list corruption?

2014-05-03 Thread Al Viro
On Sat, May 03, 2014 at 05:26:04AM +0100, Al Viro wrote: See vfs.git#dentry_kill-3; warning - this is completely untested and I would really like comments on spinning case there (i.e. the one where select_collect() finds some stuff already on some other shrink list and nothing with zero

Re: dcache shrink list corruption?

2014-05-03 Thread Al Viro
On Sat, May 03, 2014 at 11:07:57AM -0700, Linus Torvalds wrote: Sure, umount itself should be serialized by the sb lock, so there should be only one umount dentry collector. But why wouldn't there be shrinkers active due to memory pressure? generic_unmount_super() is called by -kill_sb(),

Re: dcache shrink list corruption?

2014-05-02 Thread Al Viro
On Fri, May 02, 2014 at 11:40:22PM +0100, Al Viro wrote: > On Fri, May 02, 2014 at 02:18:43PM -0700, Linus Torvalds wrote: > > On Fri, May 2, 2014 at 2:08 PM, Miklos Szeredi wrote: > > > There's more of the "delete from shrink list not owned by us" in select > > > parent. > > > Proposed patch

Re: dcache shrink list corruption?

2014-05-02 Thread Al Viro
On Fri, May 02, 2014 at 11:40:22PM +0100, Al Viro wrote: > On Fri, May 02, 2014 at 02:18:43PM -0700, Linus Torvalds wrote: > > On Fri, May 2, 2014 at 2:08 PM, Miklos Szeredi wrote: > > > There's more of the "delete from shrink list not owned by us" in select > > > parent. > > > Proposed patch

Re: dcache shrink list corruption?

2014-05-02 Thread Al Viro
On Fri, May 02, 2014 at 02:18:43PM -0700, Linus Torvalds wrote: > On Fri, May 2, 2014 at 2:08 PM, Miklos Szeredi wrote: > > There's more of the "delete from shrink list not owned by us" in select > > parent. > > Proposed patch appended. > > Ahh. Clearly this needs more work before I pull.

Re: dcache shrink list corruption?

2014-05-02 Thread Al Viro
On Fri, May 02, 2014 at 11:08:13PM +0200, Miklos Szeredi wrote: > There's more of the "delete from shrink list not owned by us" in select > parent. > Proposed patch appended. While it certainly looks like dentry_lru_del() should die, I really wonder if "let's pretend that dentry isn't there if

Re: dcache shrink list corruption?

2014-05-02 Thread Linus Torvalds
On Fri, May 2, 2014 at 2:08 PM, Miklos Szeredi wrote: > There's more of the "delete from shrink list not owned by us" in select > parent. > Proposed patch appended. Ahh. Clearly this needs more work before I pull. Linus -- To unsubscribe from this list: send the line "unsubscribe

Re: dcache shrink list corruption?

2014-05-02 Thread Miklos Szeredi
There's more of the "delete from shrink list not owned by us" in select parent. Proposed patch appended. And I'm not sure what umount_collect() is supposed to do. Can other shrinkers still be active at that point? That would present other problems, no? Also somewhat related is the question:

Re: dcache shrink list corruption?

2014-05-02 Thread Linus Torvalds
On Thu, May 1, 2014 at 7:34 AM, Al Viro wrote: > > OK, fixed and pushed (both branches). Al, can you send a real pull request (the "both branches" part in particular makes me worry about which one you think is right), because I suspect by now we just need to get this wider testing.

Re: dcache shrink list corruption?

2014-05-02 Thread Miklos Szeredi
On Fri, May 02, 2014 at 11:00:04AM +0200, Szeredi Miklos wrote: > The bug is private, but I'll ask if I can repost it. The first thing > is a a warning from the D_FLAG_VERIFY() in d_shrink_del() from > shrink_dentry_list(). They added printks that show that the dentry > has

Re: dcache shrink list corruption?

2014-05-02 Thread Szeredi Miklos
On Fri, May 2, 2014 at 7:51 AM, Al Viro wrote: > On Wed, Apr 30, 2014 at 11:15:15AM +0200, Miklos Szeredi wrote: >> IBM is triggering this with the host01 test from the LTP suite. I haven't >> yet >> tried to reproduce it. > > Could you repost their report? So far I hadn't managed to get

Re: dcache shrink list corruption?

2014-05-02 Thread Szeredi Miklos
On Thu, May 1, 2014 at 4:34 PM, Al Viro wrote: > On Thu, May 01, 2014 at 11:42:52AM +0200, Miklos Szeredi wrote: >> - "bool foo = flag & FLAG" looks suspicious. Is this guaranteed not to >> overflow? > > What do you mean, overflow? It's not a 1-bit unsigned int; conversion to > _Bool is

Re: dcache shrink list corruption?

2014-05-02 Thread Szeredi Miklos
On Thu, May 1, 2014 at 4:34 PM, Al Viro v...@zeniv.linux.org.uk wrote: On Thu, May 01, 2014 at 11:42:52AM +0200, Miklos Szeredi wrote: - bool foo = flag FLAG looks suspicious. Is this guaranteed not to overflow? What do you mean, overflow? It's not a 1-bit unsigned int; conversion to

Re: dcache shrink list corruption?

2014-05-02 Thread Szeredi Miklos
On Fri, May 2, 2014 at 7:51 AM, Al Viro v...@zeniv.linux.org.uk wrote: On Wed, Apr 30, 2014 at 11:15:15AM +0200, Miklos Szeredi wrote: IBM is triggering this with the host01 test from the LTP suite. I haven't yet tried to reproduce it. Could you repost their report? So far I hadn't

Re: dcache shrink list corruption?

2014-05-02 Thread Miklos Szeredi
On Fri, May 02, 2014 at 11:00:04AM +0200, Szeredi Miklos wrote: The bug is private, but I'll ask if I can repost it. The first thing is a a warning from the D_FLAG_VERIFY() in d_shrink_del() from shrink_dentry_list(). They added printks that show that the dentry has DCACHE_DENTRY_KILLED.

Re: dcache shrink list corruption?

2014-05-02 Thread Linus Torvalds
On Thu, May 1, 2014 at 7:34 AM, Al Viro v...@zeniv.linux.org.uk wrote: OK, fixed and pushed (both branches). Al, can you send a real pull request (the both branches part in particular makes me worry about which one you think is right), because I suspect by now we just need to get this wider

Re: dcache shrink list corruption?

2014-05-02 Thread Miklos Szeredi
There's more of the delete from shrink list not owned by us in select parent. Proposed patch appended. And I'm not sure what umount_collect() is supposed to do. Can other shrinkers still be active at that point? That would present other problems, no? Also somewhat related is the question: how

Re: dcache shrink list corruption?

2014-05-02 Thread Linus Torvalds
On Fri, May 2, 2014 at 2:08 PM, Miklos Szeredi mik...@szeredi.hu wrote: There's more of the delete from shrink list not owned by us in select parent. Proposed patch appended. Ahh. Clearly this needs more work before I pull. Linus -- To unsubscribe from this list: send the line

Re: dcache shrink list corruption?

2014-05-02 Thread Al Viro
On Fri, May 02, 2014 at 11:08:13PM +0200, Miklos Szeredi wrote: There's more of the delete from shrink list not owned by us in select parent. Proposed patch appended. While it certainly looks like dentry_lru_del() should die, I really wonder if let's pretend that dentry isn't there if it's on

Re: dcache shrink list corruption?

2014-05-02 Thread Al Viro
On Fri, May 02, 2014 at 02:18:43PM -0700, Linus Torvalds wrote: On Fri, May 2, 2014 at 2:08 PM, Miklos Szeredi mik...@szeredi.hu wrote: There's more of the delete from shrink list not owned by us in select parent. Proposed patch appended. Ahh. Clearly this needs more work before I pull.

Re: dcache shrink list corruption?

2014-05-02 Thread Al Viro
On Fri, May 02, 2014 at 11:40:22PM +0100, Al Viro wrote: On Fri, May 02, 2014 at 02:18:43PM -0700, Linus Torvalds wrote: On Fri, May 2, 2014 at 2:08 PM, Miklos Szeredi mik...@szeredi.hu wrote: There's more of the delete from shrink list not owned by us in select parent. Proposed patch

Re: dcache shrink list corruption?

2014-05-02 Thread Al Viro
On Fri, May 02, 2014 at 11:40:22PM +0100, Al Viro wrote: On Fri, May 02, 2014 at 02:18:43PM -0700, Linus Torvalds wrote: On Fri, May 2, 2014 at 2:08 PM, Miklos Szeredi mik...@szeredi.hu wrote: There's more of the delete from shrink list not owned by us in select parent. Proposed patch

Re: dcache shrink list corruption?

2014-05-01 Thread Al Viro
On Wed, Apr 30, 2014 at 11:15:15AM +0200, Miklos Szeredi wrote: > On Tue, Apr 29, 2014 at 07:56:13PM -0700, Linus Torvalds wrote: > > On Tue, Apr 29, 2014 at 7:31 PM, Al Viro wrote: > > > > > > OK, aggregate diff follows, more readable splitup (3 commits) attached. > > > It seems to survive

Re: dcache shrink list corruption?

2014-05-01 Thread Linus Torvalds
On Thu, May 1, 2014 at 2:05 PM, Al Viro wrote: > > ... and so has this one, the bounce being I actually got all three messages despite the bounces. But I seem to have gotten at least the first one through lkml. Your email is lacking SPF or any other authentication, so it may be gmail being

Re: dcache shrink list corruption?

2014-05-01 Thread Al Viro
On Thu, May 01, 2014 at 10:02:33PM +0100, Al Viro wrote: > *grumble* > > Looks like gmail filtering is buggered again. The previous mail got > bounced by gmail, both for Miklos and for Linus ;-/ Hell knows if > this one gets through; the original is at http://lkml.org/lkml/2014/5/1/235, >

Re: dcache shrink list corruption?

2014-05-01 Thread Al Viro
*grumble* Looks like gmail filtering is buggered again. The previous mail got bounced by gmail, both for Miklos and for Linus ;-/ Hell knows if this one gets through; the original is at http://lkml.org/lkml/2014/5/1/235, bounce being bloody vague, as usual. This is starting to get really

Re: dcache shrink list corruption?

2014-05-01 Thread Al Viro
On Thu, May 01, 2014 at 11:42:52AM +0200, Miklos Szeredi wrote: > Two points about latest version (dentry_kill-2): > > - Doing anything with dentry->d_parent in case of DCACHE_DENTRY_KILLED looks > seriously wrong. Parent has been dealt with, at that point, by the other > caller, no? In

Re: dcache shrink list corruption?

2014-05-01 Thread Miklos Szeredi
Two points about latest version (dentry_kill-2): - Doing anything with dentry->d_parent in case of DCACHE_DENTRY_KILLED looks seriously wrong. Parent has been dealt with, at that point, by the other caller, no? - "bool foo = flag & FLAG" looks suspicious. Is this guaranteed not to

Re: dcache shrink list corruption?

2014-05-01 Thread Miklos Szeredi
Two points about latest version (dentry_kill-2): - Doing anything with dentry-d_parent in case of DCACHE_DENTRY_KILLED looks seriously wrong. Parent has been dealt with, at that point, by the other caller, no? - bool foo = flag FLAG looks suspicious. Is this guaranteed not to overflow?

Re: dcache shrink list corruption?

2014-05-01 Thread Al Viro
On Thu, May 01, 2014 at 11:42:52AM +0200, Miklos Szeredi wrote: Two points about latest version (dentry_kill-2): - Doing anything with dentry-d_parent in case of DCACHE_DENTRY_KILLED looks seriously wrong. Parent has been dealt with, at that point, by the other caller, no? In both

Re: dcache shrink list corruption?

2014-05-01 Thread Al Viro
*grumble* Looks like gmail filtering is buggered again. The previous mail got bounced by gmail, both for Miklos and for Linus ;-/ Hell knows if this one gets through; the original is at http://lkml.org/lkml/2014/5/1/235, bounce being bloody vague, as usual. This is starting to get really

Re: dcache shrink list corruption?

2014-05-01 Thread Al Viro
On Thu, May 01, 2014 at 10:02:33PM +0100, Al Viro wrote: *grumble* Looks like gmail filtering is buggered again. The previous mail got bounced by gmail, both for Miklos and for Linus ;-/ Hell knows if this one gets through; the original is at http://lkml.org/lkml/2014/5/1/235, bounce

Re: dcache shrink list corruption?

2014-05-01 Thread Linus Torvalds
On Thu, May 1, 2014 at 2:05 PM, Al Viro v...@zeniv.linux.org.uk wrote: ... and so has this one, the bounce being I actually got all three messages despite the bounces. But I seem to have gotten at least the first one through lkml. Your email is lacking SPF or any other authentication, so it

Re: dcache shrink list corruption?

2014-05-01 Thread Al Viro
On Wed, Apr 30, 2014 at 11:15:15AM +0200, Miklos Szeredi wrote: On Tue, Apr 29, 2014 at 07:56:13PM -0700, Linus Torvalds wrote: On Tue, Apr 29, 2014 at 7:31 PM, Al Viro v...@zeniv.linux.org.uk wrote: OK, aggregate diff follows, more readable splitup (3 commits) attached. It seems to

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 07:59:43PM -0700, Linus Torvalds wrote: > On Wed, Apr 30, 2014 at 7:51 PM, Al Viro wrote: > > > > Help with profiling is needed; the loads to watch are > > the ones where dentry_kill() + dentry_free() are sufficiently high in > > profiles > > for the differences to

Re: dcache shrink list corruption?

2014-04-30 Thread Linus Torvalds
On Wed, Apr 30, 2014 at 7:51 PM, Al Viro wrote: > > Help with profiling is needed; the loads to watch are > the ones where dentry_kill() + dentry_free() are sufficiently high in profiles > for the differences to matter. Any takers? I really hope there are no such loads, my "lock/unlock

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 05:18:23PM -0700, Linus Torvalds wrote: > and then suddenly it looks like we have a common exit sequence from > that dentry_kill() function, no? > > (The earlier "unlock_on_failure" exit case is altogether a different case). > > I dunno. Maybe not a big deal, but one

Re: dcache shrink list corruption?

2014-04-30 Thread Linus Torvalds
On Wed, Apr 30, 2014 at 4:43 PM, Al Viro wrote: > > OK, done and force-pushed. Should propagate in a few... That made it more obvious how the DCACHE_MAY_FREE case ends up working. And in particular, mind rewriting this: if (dentry->d_flags & DCACHE_MAY_FREE) { spin_unlock(>d_lock);

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 04:14:14PM -0700, Linus Torvalds wrote: > On Wed, Apr 30, 2014 at 4:04 PM, Linus Torvalds > wrote: > > > > Let me go talk to the paravirt people. Maybe they don't need this, and > > I don't know exactly *how* they use that lock pointer after the unlock > > in the "kick

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 04:04:58PM -0700, Linus Torvalds wrote: > On Wed, Apr 30, 2014 at 3:12 PM, Al Viro wrote: > > > > I've just pushed the whole thing to vfs.git#for-linus; > > review and testing would be very welcome. > > I have no half-way relevant test-case for this, so I'm hoping people

Re: dcache shrink list corruption?

2014-04-30 Thread Linus Torvalds
On Wed, Apr 30, 2014 at 4:04 PM, Linus Torvalds wrote: > > Let me go talk to the paravirt people. Maybe they don't need this, and > I don't know exactly *how* they use that lock pointer after the unlock > in the "kick waiters" part. Maybe it's all good. .. looking at current users (xen and kvm)

Re: dcache shrink list corruption?

2014-04-30 Thread Linus Torvalds
On Wed, Apr 30, 2014 at 3:12 PM, Al Viro wrote: > > I've just pushed the whole thing to vfs.git#for-linus; > review and testing would be very welcome. I have no half-way relevant test-case for this, so I'm hoping people who have good VFS stress-tests (preferably under memory pressure so that we

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 10:12:06PM +0100, Al Viro wrote: > On Wed, Apr 30, 2014 at 01:57:05PM -0700, Linus Torvalds wrote: > > On Wed, Apr 30, 2014 at 1:38 PM, Al Viro wrote: > > > > > > We do not (and cannot) call dentry_kill() with rcu_read_lock held - it can > > > trigger any amount of IO, for

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 01:57:05PM -0700, Linus Torvalds wrote: > On Wed, Apr 30, 2014 at 1:38 PM, Al Viro wrote: > > > > We do not (and cannot) call dentry_kill() with rcu_read_lock held - it can > > trigger any amount of IO, for one thing. We can take it around the > > couple of places where

Re: dcache shrink list corruption?

2014-04-30 Thread Linus Torvalds
On Wed, Apr 30, 2014 at 1:38 PM, Al Viro wrote: > > We do not (and cannot) call dentry_kill() with rcu_read_lock held - it can > trigger any amount of IO, for one thing. We can take it around the > couple of places where do that spin_unlock(>d_lock) (along with > setting DCACHE_RCUACCESS) -

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 01:23:26PM -0700, Linus Torvalds wrote: > On Wed, Apr 30, 2014 at 12:59 PM, Al Viro wrote: > > > > Another thing: I don't like what's going on with freeing vs. ->d_lock there. > > Had that been a mutex, we'd definitely get a repeat of "vfs: fix subtle > > use-after-free of

Re: dcache shrink list corruption?

2014-04-30 Thread Linus Torvalds
On Wed, Apr 30, 2014 at 12:59 PM, Al Viro wrote: > > Another thing: I don't like what's going on with freeing vs. ->d_lock there. > Had that been a mutex, we'd definitely get a repeat of "vfs: fix subtle > use-after-free of pipe_inode_info". The question is, can spin_unlock(p) > dereference p

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 08:02:27PM +0100, Al Viro wrote: > On Wed, Apr 30, 2014 at 08:42:01PM +0200, Miklos Szeredi wrote: > > > Message-ID: <20140430154958.gc3...@tucsk.piliscsaba.szeredi.hu> > > I see... Several points: > * I still think that it's better to do handling of "we see >

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 08:42:01PM +0200, Miklos Szeredi wrote: > Message-ID: <20140430154958.gc3...@tucsk.piliscsaba.szeredi.hu> I see... Several points: * I still think that it's better to do handling of "we see DCACHE_DENTRY_KILLED already set" in dentry_kill() itself. * in

Re: dcache shrink list corruption?

2014-04-30 Thread Miklos Szeredi
On Wed, Apr 30, 2014 at 8:36 PM, Al Viro wrote: > On Wed, Apr 30, 2014 at 07:33:38PM +0200, Miklos Szeredi wrote: >> On Wed, Apr 30, 2014 at 6:03 PM, Al Viro wrote: >> > On Wed, Apr 30, 2014 at 05:49:58PM +0200, Miklos Szeredi wrote: >> >> > FWIW, the first two are really straightforward

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 07:33:38PM +0200, Miklos Szeredi wrote: > On Wed, Apr 30, 2014 at 6:03 PM, Al Viro wrote: > > On Wed, Apr 30, 2014 at 05:49:58PM +0200, Miklos Szeredi wrote: > >> > FWIW, the first two are really straightforward expanding the function > >> > into its only callsite. The

Re: dcache shrink list corruption?

2014-04-30 Thread Miklos Szeredi
On Wed, Apr 30, 2014 at 6:03 PM, Al Viro wrote: > On Wed, Apr 30, 2014 at 05:49:58PM +0200, Miklos Szeredi wrote: >> > FWIW, the first two are really straightforward expanding the function >> > into its only callsite. The last needs more splitup. Not sure if the >> > following is good enough,

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 05:49:58PM +0200, Miklos Szeredi wrote: > > FWIW, the first two are really straightforward expanding the function > > into its only callsite. The last needs more splitup. Not sure if the > > following is good enough, but it ought to be at least somewhat cleaner. > >

Re: dcache shrink list corruption?

2014-04-30 Thread Miklos Szeredi
And a followup patch for that, removing RCU locking from shrink_dentry_list(). I haven't tested any of this at this point, but I'll ask IBM to do some stress testing. Thanks, Miklos --- From: Miklos Szeredi Subject: dcache: don't need rcu in shrink_dentry_list() Since now the shrink list is

Re: dcache shrink list corruption?

2014-04-30 Thread Miklos Szeredi
On Wed, Apr 30, 2014 at 05:04:36AM +0100, Al Viro wrote: > On Tue, Apr 29, 2014 at 07:56:13PM -0700, Linus Torvalds wrote: > > On Tue, Apr 29, 2014 at 7:31 PM, Al Viro wrote: > > > > > > OK, aggregate diff follows, more readable splitup (3 commits) attached. > > > It seems to survive beating

Re: dcache shrink list corruption?

2014-04-30 Thread Miklos Szeredi
On Tue, Apr 29, 2014 at 07:56:13PM -0700, Linus Torvalds wrote: > On Tue, Apr 29, 2014 at 7:31 PM, Al Viro wrote: > > > > OK, aggregate diff follows, more readable splitup (3 commits) attached. > > It seems to survive beating here; testing, review and comments are > > welcome. > > Miklos, did

Re: dcache shrink list corruption?

2014-04-30 Thread Miklos Szeredi
On Tue, Apr 29, 2014 at 07:56:13PM -0700, Linus Torvalds wrote: On Tue, Apr 29, 2014 at 7:31 PM, Al Viro v...@zeniv.linux.org.uk wrote: OK, aggregate diff follows, more readable splitup (3 commits) attached. It seems to survive beating here; testing, review and comments are welcome.

Re: dcache shrink list corruption?

2014-04-30 Thread Miklos Szeredi
On Wed, Apr 30, 2014 at 05:04:36AM +0100, Al Viro wrote: On Tue, Apr 29, 2014 at 07:56:13PM -0700, Linus Torvalds wrote: On Tue, Apr 29, 2014 at 7:31 PM, Al Viro v...@zeniv.linux.org.uk wrote: OK, aggregate diff follows, more readable splitup (3 commits) attached. It seems to survive

Re: dcache shrink list corruption?

2014-04-30 Thread Miklos Szeredi
And a followup patch for that, removing RCU locking from shrink_dentry_list(). I haven't tested any of this at this point, but I'll ask IBM to do some stress testing. Thanks, Miklos --- From: Miklos Szeredi mszer...@suse.cz Subject: dcache: don't need rcu in shrink_dentry_list() Since now the

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 05:49:58PM +0200, Miklos Szeredi wrote: FWIW, the first two are really straightforward expanding the function into its only callsite. The last needs more splitup. Not sure if the following is good enough, but it ought to be at least somewhat cleaner. Combined

Re: dcache shrink list corruption?

2014-04-30 Thread Miklos Szeredi
On Wed, Apr 30, 2014 at 6:03 PM, Al Viro v...@zeniv.linux.org.uk wrote: On Wed, Apr 30, 2014 at 05:49:58PM +0200, Miklos Szeredi wrote: FWIW, the first two are really straightforward expanding the function into its only callsite. The last needs more splitup. Not sure if the following is

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 07:33:38PM +0200, Miklos Szeredi wrote: On Wed, Apr 30, 2014 at 6:03 PM, Al Viro v...@zeniv.linux.org.uk wrote: On Wed, Apr 30, 2014 at 05:49:58PM +0200, Miklos Szeredi wrote: FWIW, the first two are really straightforward expanding the function into its only

Re: dcache shrink list corruption?

2014-04-30 Thread Miklos Szeredi
On Wed, Apr 30, 2014 at 8:36 PM, Al Viro v...@zeniv.linux.org.uk wrote: On Wed, Apr 30, 2014 at 07:33:38PM +0200, Miklos Szeredi wrote: On Wed, Apr 30, 2014 at 6:03 PM, Al Viro v...@zeniv.linux.org.uk wrote: On Wed, Apr 30, 2014 at 05:49:58PM +0200, Miklos Szeredi wrote: FWIW, the first two

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 08:42:01PM +0200, Miklos Szeredi wrote: Message-ID: 20140430154958.gc3...@tucsk.piliscsaba.szeredi.hu I see... Several points: * I still think that it's better to do handling of we see DCACHE_DENTRY_KILLED already set in dentry_kill() itself. * in

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 08:02:27PM +0100, Al Viro wrote: On Wed, Apr 30, 2014 at 08:42:01PM +0200, Miklos Szeredi wrote: Message-ID: 20140430154958.gc3...@tucsk.piliscsaba.szeredi.hu I see... Several points: * I still think that it's better to do handling of we see

Re: dcache shrink list corruption?

2014-04-30 Thread Linus Torvalds
On Wed, Apr 30, 2014 at 12:59 PM, Al Viro v...@zeniv.linux.org.uk wrote: Another thing: I don't like what's going on with freeing vs. -d_lock there. Had that been a mutex, we'd definitely get a repeat of vfs: fix subtle use-after-free of pipe_inode_info. The question is, can spin_unlock(p)

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 01:23:26PM -0700, Linus Torvalds wrote: On Wed, Apr 30, 2014 at 12:59 PM, Al Viro v...@zeniv.linux.org.uk wrote: Another thing: I don't like what's going on with freeing vs. -d_lock there. Had that been a mutex, we'd definitely get a repeat of vfs: fix subtle

Re: dcache shrink list corruption?

2014-04-30 Thread Linus Torvalds
On Wed, Apr 30, 2014 at 1:38 PM, Al Viro v...@zeniv.linux.org.uk wrote: We do not (and cannot) call dentry_kill() with rcu_read_lock held - it can trigger any amount of IO, for one thing. We can take it around the couple of places where do that spin_unlock(dentry-d_lock) (along with setting

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 01:57:05PM -0700, Linus Torvalds wrote: On Wed, Apr 30, 2014 at 1:38 PM, Al Viro v...@zeniv.linux.org.uk wrote: We do not (and cannot) call dentry_kill() with rcu_read_lock held - it can trigger any amount of IO, for one thing. We can take it around the couple of

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 10:12:06PM +0100, Al Viro wrote: On Wed, Apr 30, 2014 at 01:57:05PM -0700, Linus Torvalds wrote: On Wed, Apr 30, 2014 at 1:38 PM, Al Viro v...@zeniv.linux.org.uk wrote: We do not (and cannot) call dentry_kill() with rcu_read_lock held - it can trigger any amount

Re: dcache shrink list corruption?

2014-04-30 Thread Linus Torvalds
On Wed, Apr 30, 2014 at 3:12 PM, Al Viro v...@zeniv.linux.org.uk wrote: I've just pushed the whole thing to vfs.git#for-linus; review and testing would be very welcome. I have no half-way relevant test-case for this, so I'm hoping people who have good VFS stress-tests (preferably under memory

Re: dcache shrink list corruption?

2014-04-30 Thread Linus Torvalds
On Wed, Apr 30, 2014 at 4:04 PM, Linus Torvalds torva...@linux-foundation.org wrote: Let me go talk to the paravirt people. Maybe they don't need this, and I don't know exactly *how* they use that lock pointer after the unlock in the kick waiters part. Maybe it's all good. .. looking at

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 04:04:58PM -0700, Linus Torvalds wrote: On Wed, Apr 30, 2014 at 3:12 PM, Al Viro v...@zeniv.linux.org.uk wrote: I've just pushed the whole thing to vfs.git#for-linus; review and testing would be very welcome. I have no half-way relevant test-case for this, so I'm

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 04:14:14PM -0700, Linus Torvalds wrote: On Wed, Apr 30, 2014 at 4:04 PM, Linus Torvalds torva...@linux-foundation.org wrote: Let me go talk to the paravirt people. Maybe they don't need this, and I don't know exactly *how* they use that lock pointer after the unlock

Re: dcache shrink list corruption?

2014-04-30 Thread Linus Torvalds
On Wed, Apr 30, 2014 at 4:43 PM, Al Viro v...@zeniv.linux.org.uk wrote: OK, done and force-pushed. Should propagate in a few... That made it more obvious how the DCACHE_MAY_FREE case ends up working. And in particular, mind rewriting this: if (dentry-d_flags DCACHE_MAY_FREE) {

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 05:18:23PM -0700, Linus Torvalds wrote: and then suddenly it looks like we have a common exit sequence from that dentry_kill() function, no? (The earlier unlock_on_failure exit case is altogether a different case). I dunno. Maybe not a big deal, but one reason I

Re: dcache shrink list corruption?

2014-04-30 Thread Linus Torvalds
On Wed, Apr 30, 2014 at 7:51 PM, Al Viro v...@zeniv.linux.org.uk wrote: Help with profiling is needed; the loads to watch are the ones where dentry_kill() + dentry_free() are sufficiently high in profiles for the differences to matter. Any takers? I really hope there are no such loads,

Re: dcache shrink list corruption?

2014-04-30 Thread Al Viro
On Wed, Apr 30, 2014 at 07:59:43PM -0700, Linus Torvalds wrote: On Wed, Apr 30, 2014 at 7:51 PM, Al Viro v...@zeniv.linux.org.uk wrote: Help with profiling is needed; the loads to watch are the ones where dentry_kill() + dentry_free() are sufficiently high in profiles for the

Re: dcache shrink list corruption?

2014-04-29 Thread Al Viro
On Tue, Apr 29, 2014 at 07:56:13PM -0700, Linus Torvalds wrote: > On Tue, Apr 29, 2014 at 7:31 PM, Al Viro wrote: > > > > OK, aggregate diff follows, more readable splitup (3 commits) attached. > > It seems to survive beating here; testing, review and comments are > > welcome. > > Miklos, did

Re: dcache shrink list corruption?

2014-04-29 Thread Linus Torvalds
On Tue, Apr 29, 2014 at 7:31 PM, Al Viro wrote: > > OK, aggregate diff follows, more readable splitup (3 commits) attached. > It seems to survive beating here; testing, review and comments are > welcome. Miklos, did you have some particular load that triggered this, or was it just some reports?

Re: dcache shrink list corruption?

2014-04-29 Thread Al Viro
On Wed, Apr 30, 2014 at 12:20:13AM +0100, Al Viro wrote: > On Tue, Apr 29, 2014 at 04:04:11PM -0700, Linus Torvalds wrote: > > But at a minimum, we have "d_op->d_prune()" that would now be possibly > > be called for the old dentry *after* a new dentry has been allocated. > > Not to mention the

Re: dcache shrink list corruption?

2014-04-29 Thread Al Viro
On Tue, Apr 29, 2014 at 04:04:11PM -0700, Linus Torvalds wrote: > But at a minimum, we have "d_op->d_prune()" that would now be possibly > be called for the old dentry *after* a new dentry has been allocated. > Not to mention the inode not having been dropped. So it looks like a > disaster where

Re: dcache shrink list corruption?

2014-04-29 Thread Linus Torvalds
On Tue, Apr 29, 2014 at 2:48 PM, Al Viro wrote: > > Ummm... You mean, have d_lookup() et.al. fail on something that is on > a shrink list? So I tried to see if that would work just consider it dead by the time it hits the shrink list, and if somebody does a lookup on the dentry, fail on it and

Re: dcache shrink list corruption?

2014-04-29 Thread Al Viro
On Wed, Apr 30, 2014 at 07:18:51AM +1000, Dave Chinner wrote: > Seems like it would work, but it seems fragile to me - I'm > wondering how we can ensure that the private shrink list > manipulations can be kept private. > > We have a similar situation with the inode cache (private shrink > list)

  1   2   >