Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-12-04 Thread Tetsuo Handa
I attempted some analysis, but I couldn't find which lock is causing nr_list_entries == 32768. Since "struct lock_list list_entries[MAX_LOCKDEP_ENTRIES]" is marked as "static" variable in lockdep.c , none of /proc/lockdep_stats /proc/lockdep /proc/lockdep_chains can report which lock is

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-12-03 Thread Tetsuo Handa
On Sun, Nov 22, 2020 at 2:56 AM Tetsuo Handa wrote: > Peter, you guessed that the culprit is sysfs at > https://lkml.kernel.org/r/20200916115057.go2...@hirez.programming.kicks-ass.net > , but > syzbot reported at > https://syzkaller.appspot.com/text?tag=MachineInfo=99b8f2b092d9714f > that

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-27 Thread Dmitry Vyukov
On Sun, Nov 22, 2020 at 2:56 AM Tetsuo Handa wrote: > > On 2020/11/20 18:27, Dmitry Vyukov wrote: > > Peter, so far it looks like just a very large, but normal graph to me. > > The cheapest from an engineering point of view solution would be just > > to increase the constants. I assume a 2x

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-21 Thread Tetsuo Handa
On 2020/11/20 18:27, Dmitry Vyukov wrote: > Peter, so far it looks like just a very large, but normal graph to me. > The cheapest from an engineering point of view solution would be just > to increase the constants. I assume a 2x increase should buy us lots > of time to overflow. > I can think of

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-20 Thread Dmitry Vyukov
On Fri, Nov 20, 2020 at 10:22 AM Dmitry Vyukov wrote: > > On Thu, Nov 19, 2020 at 7:08 PM Dmitry Vyukov wrote: > > > > > On Thu, Nov 19, 2020 at 2:45 PM Tetsuo Handa > > > > > wrote: > > > > > > > > > > > > On 2020/11/19 22:06, Dmitry Vyukov wrote: > > > > > > > > > > > > I am trying

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Tetsuo Handa
On 2020/11/19 23:30, Dmitry Vyukov wrote: > p.s. it's indeed huge, full log was 11MB, this probably won't be > chewed by syzbot. Is "cat /proc/lockdep*" output written from userspace? Then, we could try "xz -9" + "base64" for recording. > Peter, are these [hex numbers] needed? Could we strip

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Dmitry Vyukov
On Thu, Nov 19, 2020 at 3:30 PM Dmitry Vyukov wrote: > > > > On Thu, Nov 19, 2020 at 2:45 PM Tetsuo Handa > > wrote: > > > > > > On 2020/11/19 22:06, Dmitry Vyukov wrote: > > > > > > I am trying to reproduce this locally first. syzbot caims it can > > > reproduce it with a number

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Dmitry Vyukov
On Thu, Nov 19, 2020 at 2:45 PM Tetsuo Handa wrote: > > On 2020/11/19 22:06, Dmitry Vyukov wrote: > > I am trying to reproduce this locally first. syzbot caims it can > reproduce it with a number of very simpler reproducers (like spawn > process, unshare, create socket): >

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Tetsuo Handa
On 2020/11/19 22:06, Dmitry Vyukov wrote: I am trying to reproduce this locally first. syzbot caims it can reproduce it with a number of very simpler reproducers (like spawn process, unshare, create socket):

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Dmitry Vyukov
On Thu, Nov 19, 2020 at 1:49 PM Dmitry Vyukov wrote: > > On Thu, Nov 19, 2020 at 1:43 PM Dmitry Vyukov wrote: > > > On Wed, Nov 18, 2020 at 4:32 PM Tetsuo Handa > > > wrote: > > > > > > > > On 2020/11/19 0:10, Peter Zijlstra wrote: > > > > > On Wed, Nov 18, 2020 at 11:30:05PM +0900, Tetsuo

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Dmitry Vyukov
On Thu, Nov 19, 2020 at 1:43 PM Dmitry Vyukov wrote: > > On Wed, Nov 18, 2020 at 4:32 PM Tetsuo Handa > > wrote: > > > > > > On 2020/11/19 0:10, Peter Zijlstra wrote: > > > > On Wed, Nov 18, 2020 at 11:30:05PM +0900, Tetsuo Handa wrote: > > > >> The problem is that we can't know what exactly is

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Dmitry Vyukov
On Thu, Nov 19, 2020 at 1:33 PM Dmitry Vyukov wrote: > > On Wed, Nov 18, 2020 at 4:32 PM Tetsuo Handa > wrote: > > > > On 2020/11/19 0:10, Peter Zijlstra wrote: > > > On Wed, Nov 18, 2020 at 11:30:05PM +0900, Tetsuo Handa wrote: > > >> The problem is that we can't know what exactly is consuming

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-19 Thread Dmitry Vyukov
On Wed, Nov 18, 2020 at 4:32 PM Tetsuo Handa wrote: > > On 2020/11/19 0:10, Peter Zijlstra wrote: > > On Wed, Nov 18, 2020 at 11:30:05PM +0900, Tetsuo Handa wrote: > >> The problem is that we can't know what exactly is consuming these > >> resources. > >> My question is do you have a plan to

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-18 Thread Tetsuo Handa
On 2020/11/19 0:10, Peter Zijlstra wrote: > On Wed, Nov 18, 2020 at 11:30:05PM +0900, Tetsuo Handa wrote: >> The problem is that we can't know what exactly is consuming these resources. >> My question is do you have a plan to make it possible to know what exactly is >> consuming these resources. >

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-18 Thread Peter Zijlstra
On Wed, Nov 18, 2020 at 11:30:05PM +0900, Tetsuo Handa wrote: > The problem is that we can't know what exactly is consuming these resources. > My question is do you have a plan to make it possible to know what exactly is > consuming these resources. I'm pretty sure it's in /proc/lockdep*

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-18 Thread Tetsuo Handa
The problem is that we can't know what exactly is consuming these resources. My question is do you have a plan to make it possible to know what exactly is consuming these resources. On 2020/11/18 23:23, Peter Zijlstra wrote: > On Wed, Nov 18, 2020 at 10:57:08PM +0900, Tetsuo Handa wrote: >>

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-18 Thread Peter Zijlstra
On Wed, Nov 18, 2020 at 10:57:08PM +0900, Tetsuo Handa wrote: > Peter, do you have a plan to make this problem actionable? I don't even know what the problem is.. nor do you. You can keep sending this bad paper-over-issues patch until you're blue in the face, I'll not take it for reasons I've

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-11-18 Thread Tetsuo Handa
Peter, do you have a plan to make this problem actionable? On 2020/10/18 22:02, Tetsuo Handa wrote: > Peter, what do you think? Can we apply this patch? > > A potential for-now workaround for syzkaller would be to allow syzkaller > not to encounter the BUG: message (by masking BUG: message on

Re: [PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-10-18 Thread Tetsuo Handa
Peter, what do you think? Can we apply this patch? A potential for-now workaround for syzkaller would be to allow syzkaller not to encounter the BUG: message (by masking BUG: message on the kernel side) when hitting these limits, for continue testing until the kernel crashes (due to other bugs

[PATCH v3] lockdep: Allow tuning tracing capacity constants.

2020-10-10 Thread Tetsuo Handa
Since syzkaller continues various test cases until the kernel crashes, syzkaller tends to examine more locking dependencies than normal systems. As a result, syzbot is reporting that the fuzz testing was terminated due to hitting upper limits lockdep can track [1] [2] [3]. Peter Zijlstra does not