Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-06-03 Thread Sedat Dilek
On 3/30/16, Peter Zijlstra wrote: > On Wed, Mar 30, 2016 at 11:36:59AM +0200, Peter Zijlstra wrote: >> Furthermore, our hash function has definite room for improvement. > > After a bit of reading, using a 'strong' PRNG as base for a hash > function seems generally suggested. > > --- > kernel/lock

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-05-09 Thread Sedat Dilek
On 4/4/16, Peter Zijlstra wrote: > On Mon, Apr 04, 2016 at 05:31:40PM +0200, Sedat Dilek wrote: >> > +/* https://en.wikipedia.org/wiki/Xorshift#xorshift.2A */ >> > +#define UINT64_C(x) x##ULL >> > +static inline u64 xorshift64star(u64 x) >> > +{ >> > + x ^= x >> 12; // a >> > + x ^= x

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-04-04 Thread Peter Zijlstra
On Mon, Apr 04, 2016 at 05:31:40PM +0200, Sedat Dilek wrote: > > +/* https://en.wikipedia.org/wiki/Xorshift#xorshift.2A */ > > +#define UINT64_C(x) x##ULL > > +static inline u64 xorshift64star(u64 x) > > +{ > > + x ^= x >> 12; // a > > + x ^= x << 25; // b > > + x ^= x >> 27; // c

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-04-04 Thread Sedat Dilek
On Wed, Mar 30, 2016 at 4:06 PM, Peter Zijlstra wrote: > On Wed, Mar 30, 2016 at 11:36:59AM +0200, Peter Zijlstra wrote: >> Furthermore, our hash function has definite room for improvement. > > After a bit of reading, using a 'strong' PRNG as base for a hash > function seems generally suggested. >

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-04-01 Thread Sedat Dilek
On Thu, Mar 31, 2016 at 5:42 PM, Peter Zijlstra wrote: > On Wed, Mar 30, 2016 at 05:59:54PM +0800, Boqun Feng wrote: >> So we should use macro like current_hardirq_context() here? Or >> considering the two helpers introduced in my RFC: >> >> http://lkml.kernel.org/g/1455602265-16490-2-git-send-ema

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-31 Thread Boqun Feng
On Thu, Mar 31, 2016 at 05:42:34PM +0200, Peter Zijlstra wrote: > On Wed, Mar 30, 2016 at 05:59:54PM +0800, Boqun Feng wrote: > > So we should use macro like current_hardirq_context() here? Or > > considering the two helpers introduced in my RFC: > > > > http://lkml.kernel.org/g/1455602265-16490-2

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-31 Thread Peter Zijlstra
On Wed, Mar 30, 2016 at 05:59:54PM +0800, Boqun Feng wrote: > So we should use macro like current_hardirq_context() here? Or > considering the two helpers introduced in my RFC: > > http://lkml.kernel.org/g/1455602265-16490-2-git-send-email-boqun.f...@gmail.com > > if you don't think that overkill

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-30 Thread Sedat Dilek
On Wed, Mar 30, 2016 at 4:06 PM, Peter Zijlstra wrote: > On Wed, Mar 30, 2016 at 11:36:59AM +0200, Peter Zijlstra wrote: >> Furthermore, our hash function has definite room for improvement. > > After a bit of reading, using a 'strong' PRNG as base for a hash > function seems generally suggested. >

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-30 Thread Peter Zijlstra
On Wed, Mar 30, 2016 at 11:36:59AM +0200, Peter Zijlstra wrote: > Furthermore, our hash function has definite room for improvement. After a bit of reading, using a 'strong' PRNG as base for a hash function seems generally suggested. --- kernel/locking/lockdep.c | 19 +++ 1 file c

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-30 Thread Peter Zijlstra
On Wed, Mar 30, 2016 at 02:46:36PM +0200, Sedat Dilek wrote: > dependency chain hlocks: 49608 [max: 327680] OK, so that is still below the u16 limit, so you're seeing an actual hash collision and my patch will not cure that. A different hash function _might_ help, but eventually this

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-30 Thread Sedat Dilek
On Wed, Mar 30, 2016 at 2:43 PM, Peter Zijlstra wrote: > On Wed, Mar 30, 2016 at 11:49:57AM +0200, Sedat Dilek wrote: >> On Wed, Mar 30, 2016 at 11:36 AM, Peter Zijlstra >> wrote: > >> > OK, so while the code in check_no_collision() seems sensible, it relies >> > on borken bits. >> > >> > The wh

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-30 Thread Peter Zijlstra
On Wed, Mar 30, 2016 at 11:49:57AM +0200, Sedat Dilek wrote: > On Wed, Mar 30, 2016 at 11:36 AM, Peter Zijlstra wrote: > > OK, so while the code in check_no_collision() seems sensible, it relies > > on borken bits. > > > > The whole chain_hlocks and /proc/lockdep_chains stuff appears to have > >

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-30 Thread Sedat Dilek
On Wed, Mar 30, 2016 at 12:36 PM, Peter Zijlstra wrote: > On Wed, Mar 30, 2016 at 05:59:54PM +0800, Boqun Feng wrote: >> > @@ -3164,6 +3181,7 @@ static int __lock_acquire(struct lockdep_map *lock, >> > unsigned int subclass, >> > hlock->acquire_ip = ip; >> > hlock->instance = lock; >> >

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-30 Thread Peter Zijlstra
On Wed, Mar 30, 2016 at 05:59:54PM +0800, Boqun Feng wrote: > > @@ -3164,6 +3181,7 @@ static int __lock_acquire(struct lockdep_map *lock, > > unsigned int subclass, > > hlock->acquire_ip = ip; > > hlock->instance = lock; > > hlock->nest_lock = nest_lock; > > + hlock->irq_context = 2*

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-30 Thread Boqun Feng
Hi Peter, On Wed, Mar 30, 2016 at 11:36:59AM +0200, Peter Zijlstra wrote: > On Tue, Mar 29, 2016 at 10:47:02AM +0200, Ingo Molnar wrote: > > > > You are right; this is lockdep running into a hash collision; which is a > > > new > > > DEBUG_LOCKDEP test. See 9e4e7554e755 ("locking/lockdep: Detec

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-30 Thread Peter Zijlstra
On Wed, Mar 30, 2016 at 11:36:59AM +0200, Peter Zijlstra wrote: > On Tue, Mar 29, 2016 at 10:47:02AM +0200, Ingo Molnar wrote: > > > > You are right; this is lockdep running into a hash collision; which is a > > > new > > > DEBUG_LOCKDEP test. See 9e4e7554e755 ("locking/lockdep: Detect chain_key

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-30 Thread Sedat Dilek
On Wed, Mar 30, 2016 at 11:36 AM, Peter Zijlstra wrote: > On Tue, Mar 29, 2016 at 10:47:02AM +0200, Ingo Molnar wrote: > >> > You are right; this is lockdep running into a hash collision; which is a >> > new >> > DEBUG_LOCKDEP test. See 9e4e7554e755 ("locking/lockdep: Detect chain_key >> > collis

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-30 Thread Sedat Dilek
> [0.00] --- > [0.00] Good, all 253 testcases passed! | > [0.00] - > > And it's all good? > ( On shutdown I saw an(other) issue - will investigate, might not be related. > ) > The initial r

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-30 Thread Peter Zijlstra
On Tue, Mar 29, 2016 at 10:47:02AM +0200, Ingo Molnar wrote: > > You are right; this is lockdep running into a hash collision; which is a > > new > > DEBUG_LOCKDEP test. See 9e4e7554e755 ("locking/lockdep: Detect chain_key > > collisions"). > > I've Cc:-ed Alfredo Alvarez Fernandez who added t

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-29 Thread Ingo Molnar
* Peter Zijlstra wrote: > On Sun, Mar 27, 2016 at 05:03:44AM -0700, Linus Torvalds wrote: > > Anyway, I don't think that DEBUG_LOCKS_WARN_ON() in > > > > kernel/locking/lockdep.c:2017 __lock_acquire > > > > would be an ext4 issue, it looks more like an internal lockdep issue. > > > > Adding

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-27 Thread Peter Zijlstra
On Mon, Mar 28, 2016 at 09:05:09AM +0800, Boqun Feng wrote: > On Sun, Mar 27, 2016 at 10:59:00PM +0200, Sedat Dilek wrote: > > [1] says... > > > > "Also tested with lockdep's test suite after applying the patch: > > > > [ 0.00] Good, all 253 testcases passed! |" > > > > Where can I find this

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-27 Thread Boqun Feng
On Sun, Mar 27, 2016 at 10:59:00PM +0200, Sedat Dilek wrote: > On Sun, Mar 27, 2016 at 10:48 PM, Peter Zijlstra wrote: > > On Sun, Mar 27, 2016 at 05:03:44AM -0700, Linus Torvalds wrote: > >> Anyway, I don't think that DEBUG_LOCKS_WARN_ON() in > >> > >> kernel/locking/lockdep.c:2017 __lock_acqui

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-27 Thread Sedat Dilek
On Sun, Mar 27, 2016 at 10:59 PM, Sedat Dilek wrote: > On Sun, Mar 27, 2016 at 10:48 PM, Peter Zijlstra wrote: >> On Sun, Mar 27, 2016 at 05:03:44AM -0700, Linus Torvalds wrote: >>> Anyway, I don't think that DEBUG_LOCKS_WARN_ON() in >>> >>> kernel/locking/lockdep.c:2017 __lock_acquire >>> >>>

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-27 Thread Sedat Dilek
On Sun, Mar 27, 2016 at 10:48 PM, Peter Zijlstra wrote: > On Sun, Mar 27, 2016 at 05:03:44AM -0700, Linus Torvalds wrote: >> Anyway, I don't think that DEBUG_LOCKS_WARN_ON() in >> >> kernel/locking/lockdep.c:2017 __lock_acquire >> >> would be an ext4 issue, it looks more like an internal lockdep

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-27 Thread Peter Zijlstra
On Sun, Mar 27, 2016 at 05:03:44AM -0700, Linus Torvalds wrote: > Anyway, I don't think that DEBUG_LOCKS_WARN_ON() in > > kernel/locking/lockdep.c:2017 __lock_acquire > > would be an ext4 issue, it looks more like an internal lockdep issue. > > Adding in the lockdep people, who will set me rig

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-27 Thread Sedat Dilek
On Sun, Mar 27, 2016 at 9:42 PM, Linus Torvalds wrote: > > On Mar 27, 2016 14:40, "Sedat Dilek" wrote: >> >> So far I can say, that I am *not* seeing this with ext4.git#dev on top >> of v4.6-rc1. > > Mind re-testing just plain 4.6-rc1 again? It might not happen.. > I needed to rebuild a 3rd one

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-27 Thread Sedat Dilek
On Sun, Mar 27, 2016 at 8:23 PM, Theodore Ts'o wrote: > On Sun, Mar 27, 2016 at 05:03:44AM -0700, Linus Torvalds wrote: >> >> Unless you're using overlayfs or per-file encryption, I'm not seeing >> that any of that should make any difference (but it's entirely >> possible I'm missing something). >

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-27 Thread Theodore Ts'o
On Sun, Mar 27, 2016 at 05:03:44AM -0700, Linus Torvalds wrote: > > Unless you're using overlayfs or per-file encryption, I'm not seeing > that any of that should make any difference (but it's entirely > possible I'm missing something). > > Was it entirely repeatable before? Maybe it just happene

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-27 Thread Boqun Feng
On Sun, Mar 27, 2016 at 05:03:44AM -0700, Linus Torvalds wrote: > On Sun, Mar 27, 2016 at 1:57 AM, Sedat Dilek wrote: > > > > I pulled ext4.git#dev on top of Linux v4.6-rc1... > > > > ... and did not see the call-trace. > > Unless you're using overlayfs or per-file encryption, I'm not seeing > th

Re: [Linux-v4.6-rc1] ext4: WARNING: CPU: 2 PID: 2692 at kernel/locking/lockdep.c:2017 __lock_acquire+0x180e/0x2260

2016-03-27 Thread Linus Torvalds
On Sun, Mar 27, 2016 at 1:57 AM, Sedat Dilek wrote: > > I pulled ext4.git#dev on top of Linux v4.6-rc1... > > ... and did not see the call-trace. Unless you're using overlayfs or per-file encryption, I'm not seeing that any of that should make any difference (but it's entirely possible I'm missin