Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Sergey Senozhatsky
On (10/10/18 22:10), Tetsuo Handa wrote: > >> I've found at least 1 place that uses DEFAULT_RATELIMIT_INTERVAL*10: > >> https://elixir.bootlin.com/linux/latest/source/fs/btrfs/extent-tree.c#L8365 > >> Probably we need something similar here. > > Since printk() is a significantly CPU consuming

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Sergey Senozhatsky
On (10/10/18 22:10), Tetsuo Handa wrote: > >> I've found at least 1 place that uses DEFAULT_RATELIMIT_INTERVAL*10: > >> https://elixir.bootlin.com/linux/latest/source/fs/btrfs/extent-tree.c#L8365 > >> Probably we need something similar here. > > Since printk() is a significantly CPU consuming

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Sergey Senozhatsky
On (10/10/18 14:29), Dmitry Vyukov wrote: > >> A bit unrelated, but while we are at it: > >> > >> I like it when we rate-limit printk-s that lookup the system. > >> But it seems that default rate-limit values are not always good enough, > >> DEFAULT_RATELIMIT_INTERVAL / DEFAULT_RATELIMIT_BURST

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Sergey Senozhatsky
On (10/10/18 14:29), Dmitry Vyukov wrote: > >> A bit unrelated, but while we are at it: > >> > >> I like it when we rate-limit printk-s that lookup the system. > >> But it seems that default rate-limit values are not always good enough, > >> DEFAULT_RATELIMIT_INTERVAL / DEFAULT_RATELIMIT_BURST

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Tetsuo Handa
On 2018/10/10 20:35, Michal Hocko wrote: What should we do if memcg-OOM found no killable task because the allocating task was oom_score_adj == -1000 ? Flooding printk() until RCU stall watchdog fires (which seems to be caused by commit 3100dab2aa09dc6e ("mm:

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Tetsuo Handa
On 2018/10/10 20:35, Michal Hocko wrote: What should we do if memcg-OOM found no killable task because the allocating task was oom_score_adj == -1000 ? Flooding printk() until RCU stall watchdog fires (which seems to be caused by commit 3100dab2aa09dc6e ("mm:

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Dmitry Vyukov
On Wed, Oct 10, 2018 at 3:10 PM, Tetsuo Handa wrote: >>> Just flooding out of memory messages can trigger RCU stall problems. >>> For example, a severe skbuff_head_cache or kmalloc-512 leak bug is >>> causing >> >> [...] >> >> Quite some of them, indeed! I guess we

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Dmitry Vyukov
On Wed, Oct 10, 2018 at 3:10 PM, Tetsuo Handa wrote: >>> Just flooding out of memory messages can trigger RCU stall problems. >>> For example, a severe skbuff_head_cache or kmalloc-512 leak bug is >>> causing >> >> [...] >> >> Quite some of them, indeed! I guess we

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Tetsuo Handa
On 2018/10/10 21:36, Dmitry Vyukov wrote: > On Wed, Oct 10, 2018 at 2:29 PM, Dmitry Vyukov wrote: >> On Wed, Oct 10, 2018 at 2:25 PM, Michal Hocko wrote: >>> On Wed 10-10-18 20:48:33, Sergey Senozhatsky wrote: On (10/10/18 13:35), Michal Hocko wrote: >> Just flooding out of memory

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Tetsuo Handa
On 2018/10/10 21:36, Dmitry Vyukov wrote: > On Wed, Oct 10, 2018 at 2:29 PM, Dmitry Vyukov wrote: >> On Wed, Oct 10, 2018 at 2:25 PM, Michal Hocko wrote: >>> On Wed 10-10-18 20:48:33, Sergey Senozhatsky wrote: On (10/10/18 13:35), Michal Hocko wrote: >> Just flooding out of memory

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Dmitry Vyukov
On Wed, Oct 10, 2018 at 2:29 PM, Dmitry Vyukov wrote: > On Wed, Oct 10, 2018 at 2:25 PM, Michal Hocko wrote: >> On Wed 10-10-18 20:48:33, Sergey Senozhatsky wrote: >>> On (10/10/18 13:35), Michal Hocko wrote: >>> > > Just flooding out of memory messages can trigger RCU stall problems. >>> > >

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Dmitry Vyukov
On Wed, Oct 10, 2018 at 2:29 PM, Dmitry Vyukov wrote: > On Wed, Oct 10, 2018 at 2:25 PM, Michal Hocko wrote: >> On Wed 10-10-18 20:48:33, Sergey Senozhatsky wrote: >>> On (10/10/18 13:35), Michal Hocko wrote: >>> > > Just flooding out of memory messages can trigger RCU stall problems. >>> > >

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Dmitry Vyukov
On Wed, Oct 10, 2018 at 2:25 PM, Michal Hocko wrote: > On Wed 10-10-18 20:48:33, Sergey Senozhatsky wrote: >> On (10/10/18 13:35), Michal Hocko wrote: >> > > Just flooding out of memory messages can trigger RCU stall problems. >> > > For example, a severe skbuff_head_cache or kmalloc-512 leak bug

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Dmitry Vyukov
On Wed, Oct 10, 2018 at 2:25 PM, Michal Hocko wrote: > On Wed 10-10-18 20:48:33, Sergey Senozhatsky wrote: >> On (10/10/18 13:35), Michal Hocko wrote: >> > > Just flooding out of memory messages can trigger RCU stall problems. >> > > For example, a severe skbuff_head_cache or kmalloc-512 leak bug

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Michal Hocko
On Wed 10-10-18 20:48:33, Sergey Senozhatsky wrote: > On (10/10/18 13:35), Michal Hocko wrote: > > > Just flooding out of memory messages can trigger RCU stall problems. > > > For example, a severe skbuff_head_cache or kmalloc-512 leak bug is causing > > > > [...] > > > > Quite some of them,

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Michal Hocko
On Wed 10-10-18 20:48:33, Sergey Senozhatsky wrote: > On (10/10/18 13:35), Michal Hocko wrote: > > > Just flooding out of memory messages can trigger RCU stall problems. > > > For example, a severe skbuff_head_cache or kmalloc-512 leak bug is causing > > > > [...] > > > > Quite some of them,

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Sergey Senozhatsky
On (10/10/18 13:35), Michal Hocko wrote: > > Just flooding out of memory messages can trigger RCU stall problems. > > For example, a severe skbuff_head_cache or kmalloc-512 leak bug is causing > > [...] > > Quite some of them, indeed! I guess we want to rate limit the output. > What about the

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Sergey Senozhatsky
On (10/10/18 13:35), Michal Hocko wrote: > > Just flooding out of memory messages can trigger RCU stall problems. > > For example, a severe skbuff_head_cache or kmalloc-512 leak bug is causing > > [...] > > Quite some of them, indeed! I guess we want to rate limit the output. > What about the

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Michal Hocko
On Wed 10-10-18 19:43:38, Tetsuo Handa wrote: > On 2018/10/10 17:59, Michal Hocko wrote: > > On Wed 10-10-18 09:12:45, Tetsuo Handa wrote: > >> syzbot is hitting RCU stall due to memcg-OOM event. > >> https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 > > > > This is

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Michal Hocko
On Wed 10-10-18 19:43:38, Tetsuo Handa wrote: > On 2018/10/10 17:59, Michal Hocko wrote: > > On Wed 10-10-18 09:12:45, Tetsuo Handa wrote: > >> syzbot is hitting RCU stall due to memcg-OOM event. > >> https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 > > > > This is

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Tetsuo Handa
On 2018/10/10 17:59, Michal Hocko wrote: > On Wed 10-10-18 09:12:45, Tetsuo Handa wrote: >> syzbot is hitting RCU stall due to memcg-OOM event. >> https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 > > This is really interesting. If we do not have any eligible oom

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Tetsuo Handa
On 2018/10/10 17:59, Michal Hocko wrote: > On Wed 10-10-18 09:12:45, Tetsuo Handa wrote: >> syzbot is hitting RCU stall due to memcg-OOM event. >> https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 > > This is really interesting. If we do not have any eligible oom

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Dmitry Vyukov
On Wed, Oct 10, 2018 at 11:13 AM, Michal Hocko wrote: > On Wed 10-10-18 09:55:57, Dmitry Vyukov wrote: >> On Wed, Oct 10, 2018 at 6:11 AM, 'David Rientjes' via syzkaller-bugs >> wrote: >> > On Wed, 10 Oct 2018, Tetsuo Handa wrote: >> > >> >> syzbot is hitting RCU stall due to memcg-OOM event. >>

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Dmitry Vyukov
On Wed, Oct 10, 2018 at 11:13 AM, Michal Hocko wrote: > On Wed 10-10-18 09:55:57, Dmitry Vyukov wrote: >> On Wed, Oct 10, 2018 at 6:11 AM, 'David Rientjes' via syzkaller-bugs >> wrote: >> > On Wed, 10 Oct 2018, Tetsuo Handa wrote: >> > >> >> syzbot is hitting RCU stall due to memcg-OOM event. >>

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Michal Hocko
On Wed 10-10-18 09:55:57, Dmitry Vyukov wrote: > On Wed, Oct 10, 2018 at 6:11 AM, 'David Rientjes' via syzkaller-bugs > wrote: > > On Wed, 10 Oct 2018, Tetsuo Handa wrote: > > > >> syzbot is hitting RCU stall due to memcg-OOM event. > >>

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Michal Hocko
On Wed 10-10-18 09:55:57, Dmitry Vyukov wrote: > On Wed, Oct 10, 2018 at 6:11 AM, 'David Rientjes' via syzkaller-bugs > wrote: > > On Wed, 10 Oct 2018, Tetsuo Handa wrote: > > > >> syzbot is hitting RCU stall due to memcg-OOM event. > >>

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Michal Hocko
On Tue 09-10-18 21:11:48, David Rientjes wrote: > On Wed, 10 Oct 2018, Tetsuo Handa wrote: > > > syzbot is hitting RCU stall due to memcg-OOM event. > > https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 > > > > What should we do if memcg-OOM found no killable task

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Michal Hocko
On Tue 09-10-18 21:11:48, David Rientjes wrote: > On Wed, 10 Oct 2018, Tetsuo Handa wrote: > > > syzbot is hitting RCU stall due to memcg-OOM event. > > https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 > > > > What should we do if memcg-OOM found no killable task

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Michal Hocko
On Wed 10-10-18 09:12:45, Tetsuo Handa wrote: > syzbot is hitting RCU stall due to memcg-OOM event. > https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 This is really interesting. If we do not have any eligible oom victim we simply force the charge (allow to proceed

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Michal Hocko
On Wed 10-10-18 09:12:45, Tetsuo Handa wrote: > syzbot is hitting RCU stall due to memcg-OOM event. > https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 This is really interesting. If we do not have any eligible oom victim we simply force the charge (allow to proceed

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Dmitry Vyukov
On Wed, Oct 10, 2018 at 6:11 AM, 'David Rientjes' via syzkaller-bugs wrote: > On Wed, 10 Oct 2018, Tetsuo Handa wrote: > >> syzbot is hitting RCU stall due to memcg-OOM event. >> https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 >> >> What should we do if memcg-OOM

Re: INFO: rcu detected stall in shmem_fault

2018-10-10 Thread Dmitry Vyukov
On Wed, Oct 10, 2018 at 6:11 AM, 'David Rientjes' via syzkaller-bugs wrote: > On Wed, 10 Oct 2018, Tetsuo Handa wrote: > >> syzbot is hitting RCU stall due to memcg-OOM event. >> https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 >> >> What should we do if memcg-OOM

Re: INFO: rcu detected stall in shmem_fault

2018-10-09 Thread David Rientjes
On Wed, 10 Oct 2018, Tetsuo Handa wrote: > syzbot is hitting RCU stall due to memcg-OOM event. > https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 > > What should we do if memcg-OOM found no killable task because the allocating > task > was oom_score_adj == -1000 ?

Re: INFO: rcu detected stall in shmem_fault

2018-10-09 Thread David Rientjes
On Wed, 10 Oct 2018, Tetsuo Handa wrote: > syzbot is hitting RCU stall due to memcg-OOM event. > https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 > > What should we do if memcg-OOM found no killable task because the allocating > task > was oom_score_adj == -1000 ?

Re: INFO: rcu detected stall in shmem_fault

2018-10-09 Thread Tetsuo Handa
syzbot is hitting RCU stall due to memcg-OOM event. https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 What should we do if memcg-OOM found no killable task because the allocating task was oom_score_adj == -1000 ? Flooding printk() until RCU stall watchdog fires

Re: INFO: rcu detected stall in shmem_fault

2018-10-09 Thread Tetsuo Handa
syzbot is hitting RCU stall due to memcg-OOM event. https://syzkaller.appspot.com/bug?id=4ae3fff7fcf4c33a47c1192d2d62d2e03efffa64 What should we do if memcg-OOM found no killable task because the allocating task was oom_score_adj == -1000 ? Flooding printk() until RCU stall watchdog fires

INFO: rcu detected stall in shmem_fault

2018-10-09 Thread syzbot
Hello, syzbot found the following crash on: HEAD commit:570b7bdeaf18 Add linux-next specific files for 20181009 git tree: linux-next console output: https://syzkaller.appspot.com/x/log.txt?x=13eeb68540 kernel config: https://syzkaller.appspot.com/x/.config?x=9b5a60e1381390c4

INFO: rcu detected stall in shmem_fault

2018-10-09 Thread syzbot
Hello, syzbot found the following crash on: HEAD commit:570b7bdeaf18 Add linux-next specific files for 20181009 git tree: linux-next console output: https://syzkaller.appspot.com/x/log.txt?x=13eeb68540 kernel config: https://syzkaller.appspot.com/x/.config?x=9b5a60e1381390c4