Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-12-04 Thread Tetsuo Handa
I attempted some analysis, but I couldn't find which lock is causing nr_list_entries == 32768. Since "struct lock_list list_entries[MAX_LOCKDEP_ENTRIES]" is marked as "static" variable in lockdep.c , none of /proc/lockdep_stats /proc/lockdep /proc/lockdep_chains can report which lock is responsi

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-12-03 Thread Tetsuo Handa
On Sun, Nov 22, 2020 at 2:56 AM Tetsuo Handa wrote: > Peter, you guessed that the culprit is sysfs at > https://lkml.kernel.org/r/20200916115057.go2...@hirez.programming.kicks-ass.net > , but > syzbot reported at > https://syzkaller.appspot.com/text?tag=MachineInfo&x=99b8f2b092d9714f > that "BU

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-27 Thread Dmitry Vyukov
On Sun, Nov 22, 2020 at 2:56 AM Tetsuo Handa wrote: > > On 2020/11/20 18:27, Dmitry Vyukov wrote: > > Peter, so far it looks like just a very large, but normal graph to me. > > The cheapest from an engineering point of view solution would be just > > to increase the constants. I assume a 2x increa

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-21 Thread Tetsuo Handa
On 2020/11/20 18:27, Dmitry Vyukov wrote: > Peter, so far it looks like just a very large, but normal graph to me. > The cheapest from an engineering point of view solution would be just > to increase the constants. I assume a 2x increase should buy us lots > of time to overflow. > I can think of m

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-20 Thread Dmitry Vyukov
On Fri, Nov 20, 2020 at 10:22 AM Dmitry Vyukov wrote: > > On Thu, Nov 19, 2020 at 7:08 PM Dmitry Vyukov wrote: > > > > > On Thu, Nov 19, 2020 at 2:45 PM Tetsuo Handa > > > > > wrote: > > > > > > > > > > > > On 2020/11/19 22:06, Dmitry Vyukov wrote: > > > > > > > > > > > > I am trying t

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Tetsuo Handa
On 2020/11/19 23:30, Dmitry Vyukov wrote: > p.s. it's indeed huge, full log was 11MB, this probably won't be > chewed by syzbot. Is "cat /proc/lockdep*" output written from userspace? Then, we could try "xz -9" + "base64" for recording. > Peter, are these [hex numbers] needed? Could we strip them

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Dmitry Vyukov
On Thu, Nov 19, 2020 at 3:30 PM Dmitry Vyukov wrote: > > > > On Thu, Nov 19, 2020 at 2:45 PM Tetsuo Handa > > wrote: > > > > > > On 2020/11/19 22:06, Dmitry Vyukov wrote: > > > > > > I am trying to reproduce this locally first. syzbot caims it can > > > reproduce it with a number o

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Dmitry Vyukov
On Thu, Nov 19, 2020 at 2:45 PM Tetsuo Handa wrote: > > On 2020/11/19 22:06, Dmitry Vyukov wrote: > > I am trying to reproduce this locally first. syzbot caims it can > reproduce it with a number of very simpler reproducers (like spawn > process, unshare, create socket): >

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Tetsuo Handa
On 2020/11/19 22:06, Dmitry Vyukov wrote: I am trying to reproduce this locally first. syzbot caims it can reproduce it with a number of very simpler reproducers (like spawn process, unshare, create socket): https://syzkaller.appspot.com/bug?id=8a18efe79140782a88dcd098808d6

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Dmitry Vyukov
On Thu, Nov 19, 2020 at 1:49 PM Dmitry Vyukov wrote: > > On Thu, Nov 19, 2020 at 1:43 PM Dmitry Vyukov wrote: > > > On Wed, Nov 18, 2020 at 4:32 PM Tetsuo Handa > > > wrote: > > > > > > > > On 2020/11/19 0:10, Peter Zijlstra wrote: > > > > > On Wed, Nov 18, 2020 at 11:30:05PM +0900, Tetsuo Handa

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Dmitry Vyukov
On Thu, Nov 19, 2020 at 1:43 PM Dmitry Vyukov wrote: > > On Wed, Nov 18, 2020 at 4:32 PM Tetsuo Handa > > wrote: > > > > > > On 2020/11/19 0:10, Peter Zijlstra wrote: > > > > On Wed, Nov 18, 2020 at 11:30:05PM +0900, Tetsuo Handa wrote: > > > >> The problem is that we can't know what exactly is c

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Dmitry Vyukov
On Thu, Nov 19, 2020 at 1:33 PM Dmitry Vyukov wrote: > > On Wed, Nov 18, 2020 at 4:32 PM Tetsuo Handa > wrote: > > > > On 2020/11/19 0:10, Peter Zijlstra wrote: > > > On Wed, Nov 18, 2020 at 11:30:05PM +0900, Tetsuo Handa wrote: > > >> The problem is that we can't know what exactly is consuming t

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Dmitry Vyukov
On Wed, Nov 18, 2020 at 4:32 PM Tetsuo Handa wrote: > > On 2020/11/19 0:10, Peter Zijlstra wrote: > > On Wed, Nov 18, 2020 at 11:30:05PM +0900, Tetsuo Handa wrote: > >> The problem is that we can't know what exactly is consuming these > >> resources. > >> My question is do you have a plan to make

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-18 Thread Tetsuo Handa
On 2020/11/19 0:10, Peter Zijlstra wrote: > On Wed, Nov 18, 2020 at 11:30:05PM +0900, Tetsuo Handa wrote: >> The problem is that we can't know what exactly is consuming these resources. >> My question is do you have a plan to make it possible to know what exactly is >> consuming these resources. >

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-18 Thread Peter Zijlstra
On Wed, Nov 18, 2020 at 11:30:05PM +0900, Tetsuo Handa wrote: > The problem is that we can't know what exactly is consuming these resources. > My question is do you have a plan to make it possible to know what exactly is > consuming these resources. I'm pretty sure it's in /proc/lockdep* somewhere

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-18 Thread Tetsuo Handa
The problem is that we can't know what exactly is consuming these resources. My question is do you have a plan to make it possible to know what exactly is consuming these resources. On 2020/11/18 23:23, Peter Zijlstra wrote: > On Wed, Nov 18, 2020 at 10:57:08PM +0900, Tetsuo Handa wrote: >> Peter,

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-18 Thread Peter Zijlstra
On Wed, Nov 18, 2020 at 10:57:08PM +0900, Tetsuo Handa wrote: > Peter, do you have a plan to make this problem actionable? I don't even know what the problem is.. nor do you. You can keep sending this bad paper-over-issues patch until you're blue in the face, I'll not take it for reasons I've expl

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-18 Thread Tetsuo Handa
Peter, do you have a plan to make this problem actionable? On 2020/10/18 22:02, Tetsuo Handa wrote: > Peter, what do you think? Can we apply this patch? > > A potential for-now workaround for syzkaller would be to allow syzkaller > not to encounter the BUG: message (by masking BUG: message on the

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-10-18 Thread Tetsuo Handa
Peter, what do you think? Can we apply this patch? A potential for-now workaround for syzkaller would be to allow syzkaller not to encounter the BUG: message (by masking BUG: message on the kernel side) when hitting these limits, for continue testing until the kernel crashes (due to other bugs lik

[PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-10-10 Thread Tetsuo Handa
Since syzkaller continues various test cases until the kernel crashes, syzkaller tends to examine more locking dependencies than normal systems. As a result, syzbot is reporting that the fuzz testing was terminated due to hitting upper limits lockdep can track [1] [2] [3]. Peter Zijlstra does not