Re: general protection fault in put_pid
On Wed, Mar 27, 2019 at 01:27:12PM -0700, Matthew Wilcox wrote: > On Wed, Mar 27, 2019 at 01:10:01PM -0700, syzbot wrote: > > syzbot has bisected this bug to: > > > > commit b9b8a41adeff5666b402996020b698504c927353 > > Author: Dan Carpenter > > Date: Mon Aug 20 08:25:33 2018 + > > > > btrfs: use after free in btrfs_quota_enable > > Not plausible. Try again. Agreed, grep for 'btrfs' in the console log does not show anything, ie. no messages, slab caches nor functions on the stack.
Re: general protection fault in put_pid
On Mon, Jan 7, 2019 at 10:04 AM Manfred Spraul
wrote:
>
> On 1/3/19 11:18 PM, Shakeel Butt wrote:
> > Hi Manfred,
> >
> > On Sun, Dec 23, 2018 at 4:26 AM Manfred Spraul
> > wrote:
> >> Hello Dmitry,
> >>
> >> On 12/23/18 10:57 AM, Dmitry Vyukov wrote:
> >>> I can reproduce this infinite memory consumption with the C
> >>> program:
> >>> https://gist.githubusercontent.com/dvyukov/03ec54b3429ade16fa07bf8b2379aff3/raw/ae4f654e279810de2505e8fa41b73dc1d8e6/gistfile1.txt
> >>>
> >>> But this is working as intended, right? It just creates infinite
> >>> number of large semaphore sets, which reasonably consumes infinite
> >>> amount of memory.
> >>> Except that it also violates the memcg bound and a process can
> >>> have
> >>> effectively unlimited amount of such "drum memory" in semaphores.
> >> Yes, this is as intended:
> >>
> >> If you call semget(), then you can use memory, up to the limits in
> >> /proc/sys/kernel/sem.
> >>
> >> Memcg is not taken into account, an admin must set
> >> /proc/sys/kernel/sem.
> >>
> >> The default are "infinite amount of memory allowed", as this is the
> >> most
> >> sane default: We had a logic that tried to autotune (i.e.: a new
> >> namespace "inherits" a fraction of the parent namespaces memory
> >> limits),
> >> but this we more or less always wrong.
> >>
> >>
> > What's the disadvantage of setting the limits in
> > /proc/sys/kernel/sem
> > high and let the task's memcg limits the number of semaphore a
> > process
> > can create? Please note that the memory underlying shmget and msgget
> > is already accounted to memcg.
>
> Nothing, it it just a question of implementing it.
>
I think it should be something like following:
---
ipc/sem.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/ipc/sem.c b/ipc/sem.c
index 745dc6187e84..ad63df2658aa 100644
--- a/ipc/sem.c
+++ b/ipc/sem.c
@@ -494,7 +494,7 @@ static struct sem_array *sem_alloc(size_t nsems)
return NULL;
size = sizeof(*sma) + nsems * sizeof(sma->sems[0]);
- sma = kvmalloc(size, GFP_KERNEL);
+ sma = kvmalloc(size, GFP_KERNEL_ACCOUNT);
if (unlikely(!sma))
return NULL;
@@ -1897,7 +1897,8 @@ static struct sem_undo *find_alloc_undo(struct
ipc_namespace *ns, int semid)
rcu_read_unlock();
/* step 2: allocate new undo structure */
- new = kzalloc(sizeof(struct sem_undo) + sizeof(short)*nsems,
GFP_KERNEL);
+ new = kzalloc(sizeof(struct sem_undo) + sizeof(short)*nsems,
+ GFP_KERNEL_ACCOUNT);
if (!new) {
ipc_rcu_putref(&sma->sem_perm, sem_rcu_free);
return ERR_PTR(-ENOMEM);
--
2.20.1.97.g81188d93c3-goog
Re: general protection fault in put_pid
On 1/3/19 11:18 PM, Shakeel Butt wrote: Hi Manfred, On Sun, Dec 23, 2018 at 4:26 AM Manfred Spraul wrote: Hello Dmitry, On 12/23/18 10:57 AM, Dmitry Vyukov wrote: I can reproduce this infinite memory consumption with the C program: https://gist.githubusercontent.com/dvyukov/03ec54b3429ade16fa07bf8b2379aff3/raw/ae4f654e279810de2505e8fa41b73dc1d8e6/gistfile1.txt But this is working as intended, right? It just creates infinite number of large semaphore sets, which reasonably consumes infinite amount of memory. Except that it also violates the memcg bound and a process can have effectively unlimited amount of such "drum memory" in semaphores. Yes, this is as intended: If you call semget(), then you can use memory, up to the limits in /proc/sys/kernel/sem. Memcg is not taken into account, an admin must set /proc/sys/kernel/sem. The default are "infinite amount of memory allowed", as this is the most sane default: We had a logic that tried to autotune (i.e.: a new namespace "inherits" a fraction of the parent namespaces memory limits), but this we more or less always wrong. What's the disadvantage of setting the limits in /proc/sys/kernel/sem high and let the task's memcg limits the number of semaphore a process can create? Please note that the memory underlying shmget and msgget is already accounted to memcg. Nothing, it it just a question of implementing it. I'll try to look at it. -- Manfred
Re: general protection fault in put_pid
Hi Manfred, On Sun, Dec 23, 2018 at 4:26 AM Manfred Spraul wrote: > > Hello Dmitry, > > On 12/23/18 10:57 AM, Dmitry Vyukov wrote: > > > > I can reproduce this infinite memory consumption with the C program: > > https://gist.githubusercontent.com/dvyukov/03ec54b3429ade16fa07bf8b2379aff3/raw/ae4f654e279810de2505e8fa41b73dc1d8e6/gistfile1.txt > > > > But this is working as intended, right? It just creates infinite > > number of large semaphore sets, which reasonably consumes infinite > > amount of memory. > > Except that it also violates the memcg bound and a process can have > > effectively unlimited amount of such "drum memory" in semaphores. > > Yes, this is as intended: > > If you call semget(), then you can use memory, up to the limits in > /proc/sys/kernel/sem. > > Memcg is not taken into account, an admin must set /proc/sys/kernel/sem. > > The default are "infinite amount of memory allowed", as this is the most > sane default: We had a logic that tried to autotune (i.e.: a new > namespace "inherits" a fraction of the parent namespaces memory limits), > but this we more or less always wrong. > > What's the disadvantage of setting the limits in /proc/sys/kernel/sem high and let the task's memcg limits the number of semaphore a process can create? Please note that the memory underlying shmget and msgget is already accounted to memcg. thanks, Shakeel
Re: general protection fault in put_pid
On Sun, Dec 30, 2018 at 10:31 AM Dmitry Vyukov wrote: > > On Wed, Dec 26, 2018 at 10:03 AM Dmitry Vyukov wrote: > > > > Hello Dmitry, > > > > > > > > On 12/23/18 11:42 AM, Dmitry Vyukov wrote: > > > > > Actually was able to reproduce this with a syzkaller program: > > > > > ./syz-execprog -repeat=0 -procs=10 prog > > > > > ... > > > > > kasan: CONFIG_KASAN_INLINE enabled > > > > > kasan: GPF could be caused by NULL-ptr deref or user memory access > > > > > general protection fault: [#1] PREEMPT SMP KASAN > > > > > CPU: 1 PID: 8788 Comm: syz-executor8 Not tainted 4.20.0-rc7+ #6 > > > > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 > > > > > 04/01/2014 > > > > > RIP: 0010:__list_del_entry_valid+0x7e/0x150 lib/list_debug.c:51 > > > > > Code: ad de 4c 8b 26 49 39 c4 74 66 48 b8 00 02 00 00 00 00 ad de 48 > > > > > 89 da 48 39 c3 74 65 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c > > > > > 02 00 75 7b 48 8b 13 48 39 f2 75 57 49 8d 7c 24 08 48 b8 00 > > > > > RSP: 0018:88804faef210 EFLAGS: 00010a02 > > > > > RAX: dc00 RBX: f817edba555e1f00 RCX: 831bad5f > > > > > RDX: 1f02fdb74aabc3e0 RSI: 88801b8a0720 RDI: 88801b8a0728 > > > > > RBP: 88804faef228 R08: f52001055401 R09: f52001055401 > > > > > R10: 0001 R11: f52001055400 R12: 88802d52cc98 > > > > > R13: 88801b8a0728 R14: 88801b8a0720 R15: dc00 > > > > > FS: 00d24940() GS:88802d50() > > > > > knlGS: > > > > > CS: 0010 DS: ES: CR0: 80050033 > > > > > CR2: 004bb580 CR3: 11177005 CR4: 003606e0 > > > > > DR0: DR1: DR2: > > > > > DR3: DR6: fffe0ff0 DR7: 0400 > > > > > Call Trace: > > > > > __list_del_entry include/linux/list.h:117 [inline] > > > > > list_del include/linux/list.h:125 [inline] > > > > > unlink_queue ipc/sem.c:786 [inline] > > > > > freeary+0xddb/0x1c90 ipc/sem.c:1164 > > > > > free_ipcs+0xf0/0x160 ipc/namespace.c:112 > > > > > sem_exit_ns+0x20/0x40 ipc/sem.c:237 > > > > > free_ipc_ns ipc/namespace.c:120 [inline] > > > > > put_ipc_ns+0x55/0x160 ipc/namespace.c:152 > > > > > free_nsproxy+0xc0/0x1f0 kernel/nsproxy.c:180 > > > > > switch_task_namespaces+0xa5/0xc0 kernel/nsproxy.c:229 > > > > > exit_task_namespaces+0x17/0x20 kernel/nsproxy.c:234 > > > > > do_exit+0x19e5/0x27d0 kernel/exit.c:866 > > > > > do_group_exit+0x151/0x410 kernel/exit.c:970 > > > > > __do_sys_exit_group kernel/exit.c:981 [inline] > > > > > __se_sys_exit_group kernel/exit.c:979 [inline] > > > > > __x64_sys_exit_group+0x3e/0x50 kernel/exit.c:979 > > > > > do_syscall_64+0x192/0x770 arch/x86/entry/common.c:290 > > > > > entry_SYSCALL_64_after_hwframe+0x49/0xbe > > > > > RIP: 0033:0x4570e9 > > > > > Code: 5d af fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 > > > > > 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d > > > > > 01 f0 ff ff 0f 83 2b af fb ff c3 66 2e 0f 1f 84 00 00 00 00 > > > > > RSP: 002b:7ffe35f12018 EFLAGS: 0246 ORIG_RAX: 00e7 > > > > > RAX: ffda RBX: 0001 RCX: 004570e9 > > > > > RDX: 00410540 RSI: 00a34c00 RDI: 0045 > > > > > RBP: 004a43a4 R08: 000c R09: > > > > > R10: 00d24940 R11: 0246 R12: > > > > > R13: 0001 R14: R15: 0008 > > > > > Modules linked in: > > > > > Dumping ftrace buffer: > > > > > (ftrace buffer empty) > > > > > ---[ end trace 17829b0f00569a59 ]--- > > > > > RIP: 0010:__list_del_entry_valid+0x7e/0x150 lib/list_debug.c:51 > > > > > Code: ad de 4c 8b 26 49 39 c4 74 66 48 b8 00 02 00 00 00 00 ad de 48 > > > > > 89 da 48 39 c3 74 65 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c > > > > > 02 00 75 7b 48 8b 13 48 39 f2 75 57 49 8d 7c 24 08 48 b8 00 > > > > > RSP: 0018:88804faef210 EFLAGS: 00010a02 > > > > > RAX: dc00 RBX: f817edba555e1f00 RCX: 831bad5f > > > > > RDX: 1f02fdb74aabc3e0 RSI: 88801b8a0720 RDI: 88801b8a0728 > > > > > RBP: 88804faef228 R08: f52001055401 R09: f52001055401 > > > > > R10: 0001 R11: f52001055400 R12: 88802d52cc98 > > > > > R13: 88801b8a0728 R14: 88801b8a0720 R15: dc00 > > > > > FS: 00d24940() GS:88802d50() > > > > > knlGS: > > > > > CS: 0010 DS: ES: CR0: 80050033 > > > > > CR2: 004bb580 CR3: 11177005 CR4: 003606e0 > > > > > DR0: DR1: DR2: > > > > > DR3: DR6: fffe0ff0 DR7: 0400 > > > > > > > > > > > > > > > The prog is: > > > > > unshare(0x802) > > > > > semget$private(0x0, 0x4007, 0x0) > > > > > > > > > > kernel is on 9105b8aa5
Re: general protection fault in put_pid
On Wed, Dec 26, 2018 at 10:03 AM Dmitry Vyukov wrote: > > > Hello Dmitry, > > > > > > On 12/23/18 11:42 AM, Dmitry Vyukov wrote: > > > > Actually was able to reproduce this with a syzkaller program: > > > > ./syz-execprog -repeat=0 -procs=10 prog > > > > ... > > > > kasan: CONFIG_KASAN_INLINE enabled > > > > kasan: GPF could be caused by NULL-ptr deref or user memory access > > > > general protection fault: [#1] PREEMPT SMP KASAN > > > > CPU: 1 PID: 8788 Comm: syz-executor8 Not tainted 4.20.0-rc7+ #6 > > > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 > > > > 04/01/2014 > > > > RIP: 0010:__list_del_entry_valid+0x7e/0x150 lib/list_debug.c:51 > > > > Code: ad de 4c 8b 26 49 39 c4 74 66 48 b8 00 02 00 00 00 00 ad de 48 > > > > 89 da 48 39 c3 74 65 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c > > > > 02 00 75 7b 48 8b 13 48 39 f2 75 57 49 8d 7c 24 08 48 b8 00 > > > > RSP: 0018:88804faef210 EFLAGS: 00010a02 > > > > RAX: dc00 RBX: f817edba555e1f00 RCX: 831bad5f > > > > RDX: 1f02fdb74aabc3e0 RSI: 88801b8a0720 RDI: 88801b8a0728 > > > > RBP: 88804faef228 R08: f52001055401 R09: f52001055401 > > > > R10: 0001 R11: f52001055400 R12: 88802d52cc98 > > > > R13: 88801b8a0728 R14: 88801b8a0720 R15: dc00 > > > > FS: 00d24940() GS:88802d50() > > > > knlGS: > > > > CS: 0010 DS: ES: CR0: 80050033 > > > > CR2: 004bb580 CR3: 11177005 CR4: 003606e0 > > > > DR0: DR1: DR2: > > > > DR3: DR6: fffe0ff0 DR7: 0400 > > > > Call Trace: > > > > __list_del_entry include/linux/list.h:117 [inline] > > > > list_del include/linux/list.h:125 [inline] > > > > unlink_queue ipc/sem.c:786 [inline] > > > > freeary+0xddb/0x1c90 ipc/sem.c:1164 > > > > free_ipcs+0xf0/0x160 ipc/namespace.c:112 > > > > sem_exit_ns+0x20/0x40 ipc/sem.c:237 > > > > free_ipc_ns ipc/namespace.c:120 [inline] > > > > put_ipc_ns+0x55/0x160 ipc/namespace.c:152 > > > > free_nsproxy+0xc0/0x1f0 kernel/nsproxy.c:180 > > > > switch_task_namespaces+0xa5/0xc0 kernel/nsproxy.c:229 > > > > exit_task_namespaces+0x17/0x20 kernel/nsproxy.c:234 > > > > do_exit+0x19e5/0x27d0 kernel/exit.c:866 > > > > do_group_exit+0x151/0x410 kernel/exit.c:970 > > > > __do_sys_exit_group kernel/exit.c:981 [inline] > > > > __se_sys_exit_group kernel/exit.c:979 [inline] > > > > __x64_sys_exit_group+0x3e/0x50 kernel/exit.c:979 > > > > do_syscall_64+0x192/0x770 arch/x86/entry/common.c:290 > > > > entry_SYSCALL_64_after_hwframe+0x49/0xbe > > > > RIP: 0033:0x4570e9 > > > > Code: 5d af fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 > > > > 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d > > > > 01 f0 ff ff 0f 83 2b af fb ff c3 66 2e 0f 1f 84 00 00 00 00 > > > > RSP: 002b:7ffe35f12018 EFLAGS: 0246 ORIG_RAX: 00e7 > > > > RAX: ffda RBX: 0001 RCX: 004570e9 > > > > RDX: 00410540 RSI: 00a34c00 RDI: 0045 > > > > RBP: 004a43a4 R08: 000c R09: > > > > R10: 00d24940 R11: 0246 R12: > > > > R13: 0001 R14: R15: 0008 > > > > Modules linked in: > > > > Dumping ftrace buffer: > > > > (ftrace buffer empty) > > > > ---[ end trace 17829b0f00569a59 ]--- > > > > RIP: 0010:__list_del_entry_valid+0x7e/0x150 lib/list_debug.c:51 > > > > Code: ad de 4c 8b 26 49 39 c4 74 66 48 b8 00 02 00 00 00 00 ad de 48 > > > > 89 da 48 39 c3 74 65 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c > > > > 02 00 75 7b 48 8b 13 48 39 f2 75 57 49 8d 7c 24 08 48 b8 00 > > > > RSP: 0018:88804faef210 EFLAGS: 00010a02 > > > > RAX: dc00 RBX: f817edba555e1f00 RCX: 831bad5f > > > > RDX: 1f02fdb74aabc3e0 RSI: 88801b8a0720 RDI: 88801b8a0728 > > > > RBP: 88804faef228 R08: f52001055401 R09: f52001055401 > > > > R10: 0001 R11: f52001055400 R12: 88802d52cc98 > > > > R13: 88801b8a0728 R14: 88801b8a0720 R15: dc00 > > > > FS: 00d24940() GS:88802d50() > > > > knlGS: > > > > CS: 0010 DS: ES: CR0: 80050033 > > > > CR2: 004bb580 CR3: 11177005 CR4: 003606e0 > > > > DR0: DR1: DR2: > > > > DR3: DR6: fffe0ff0 DR7: 0400 > > > > > > > > > > > > The prog is: > > > > unshare(0x802) > > > > semget$private(0x0, 0x4007, 0x0) > > > > > > > > kernel is on 9105b8aa50c182371533fc97db64fc8f26f051b3 > > > > > > > > and again it involved lots of oom kills, the repro eats all memory, a > > > > process getting killed, frees some memory and the process repeats. > > > > > > I was too fast: I can'
Re: general protection fault in put_pid
On Tue, Dec 25, 2018 at 10:35 AM Dmitry Vyukov wrote: > > On Sun, Dec 23, 2018 at 7:38 PM Manfred Spraul > wrote: > > > > Hello Dmitry, > > > > On 12/23/18 11:42 AM, Dmitry Vyukov wrote: > > > Actually was able to reproduce this with a syzkaller program: > > > ./syz-execprog -repeat=0 -procs=10 prog > > > ... > > > kasan: CONFIG_KASAN_INLINE enabled > > > kasan: GPF could be caused by NULL-ptr deref or user memory access > > > general protection fault: [#1] PREEMPT SMP KASAN > > > CPU: 1 PID: 8788 Comm: syz-executor8 Not tainted 4.20.0-rc7+ #6 > > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 > > > 04/01/2014 > > > RIP: 0010:__list_del_entry_valid+0x7e/0x150 lib/list_debug.c:51 > > > Code: ad de 4c 8b 26 49 39 c4 74 66 48 b8 00 02 00 00 00 00 ad de 48 > > > 89 da 48 39 c3 74 65 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c > > > 02 00 75 7b 48 8b 13 48 39 f2 75 57 49 8d 7c 24 08 48 b8 00 > > > RSP: 0018:88804faef210 EFLAGS: 00010a02 > > > RAX: dc00 RBX: f817edba555e1f00 RCX: 831bad5f > > > RDX: 1f02fdb74aabc3e0 RSI: 88801b8a0720 RDI: 88801b8a0728 > > > RBP: 88804faef228 R08: f52001055401 R09: f52001055401 > > > R10: 0001 R11: f52001055400 R12: 88802d52cc98 > > > R13: 88801b8a0728 R14: 88801b8a0720 R15: dc00 > > > FS: 00d24940() GS:88802d50() > > > knlGS: > > > CS: 0010 DS: ES: CR0: 80050033 > > > CR2: 004bb580 CR3: 11177005 CR4: 003606e0 > > > DR0: DR1: DR2: > > > DR3: DR6: fffe0ff0 DR7: 0400 > > > Call Trace: > > > __list_del_entry include/linux/list.h:117 [inline] > > > list_del include/linux/list.h:125 [inline] > > > unlink_queue ipc/sem.c:786 [inline] > > > freeary+0xddb/0x1c90 ipc/sem.c:1164 > > > free_ipcs+0xf0/0x160 ipc/namespace.c:112 > > > sem_exit_ns+0x20/0x40 ipc/sem.c:237 > > > free_ipc_ns ipc/namespace.c:120 [inline] > > > put_ipc_ns+0x55/0x160 ipc/namespace.c:152 > > > free_nsproxy+0xc0/0x1f0 kernel/nsproxy.c:180 > > > switch_task_namespaces+0xa5/0xc0 kernel/nsproxy.c:229 > > > exit_task_namespaces+0x17/0x20 kernel/nsproxy.c:234 > > > do_exit+0x19e5/0x27d0 kernel/exit.c:866 > > > do_group_exit+0x151/0x410 kernel/exit.c:970 > > > __do_sys_exit_group kernel/exit.c:981 [inline] > > > __se_sys_exit_group kernel/exit.c:979 [inline] > > > __x64_sys_exit_group+0x3e/0x50 kernel/exit.c:979 > > > do_syscall_64+0x192/0x770 arch/x86/entry/common.c:290 > > > entry_SYSCALL_64_after_hwframe+0x49/0xbe > > > RIP: 0033:0x4570e9 > > > Code: 5d af fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 > > > 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d > > > 01 f0 ff ff 0f 83 2b af fb ff c3 66 2e 0f 1f 84 00 00 00 00 > > > RSP: 002b:7ffe35f12018 EFLAGS: 0246 ORIG_RAX: 00e7 > > > RAX: ffda RBX: 0001 RCX: 004570e9 > > > RDX: 00410540 RSI: 00a34c00 RDI: 0045 > > > RBP: 004a43a4 R08: 000c R09: > > > R10: 00d24940 R11: 0246 R12: > > > R13: 0001 R14: R15: 0008 > > > Modules linked in: > > > Dumping ftrace buffer: > > > (ftrace buffer empty) > > > ---[ end trace 17829b0f00569a59 ]--- > > > RIP: 0010:__list_del_entry_valid+0x7e/0x150 lib/list_debug.c:51 > > > Code: ad de 4c 8b 26 49 39 c4 74 66 48 b8 00 02 00 00 00 00 ad de 48 > > > 89 da 48 39 c3 74 65 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c > > > 02 00 75 7b 48 8b 13 48 39 f2 75 57 49 8d 7c 24 08 48 b8 00 > > > RSP: 0018:88804faef210 EFLAGS: 00010a02 > > > RAX: dc00 RBX: f817edba555e1f00 RCX: 831bad5f > > > RDX: 1f02fdb74aabc3e0 RSI: 88801b8a0720 RDI: 88801b8a0728 > > > RBP: 88804faef228 R08: f52001055401 R09: f52001055401 > > > R10: 0001 R11: f52001055400 R12: 88802d52cc98 > > > R13: 88801b8a0728 R14: 88801b8a0720 R15: dc00 > > > FS: 00d24940() GS:88802d50() > > > knlGS: > > > CS: 0010 DS: ES: CR0: 80050033 > > > CR2: 004bb580 CR3: 11177005 CR4: 003606e0 > > > DR0: DR1: DR2: > > > DR3: DR6: fffe0ff0 DR7: 0400 > > > > > > > > > The prog is: > > > unshare(0x802) > > > semget$private(0x0, 0x4007, 0x0) > > > > > > kernel is on 9105b8aa50c182371533fc97db64fc8f26f051b3 > > > > > > and again it involved lots of oom kills, the repro eats all memory, a > > > process getting killed, frees some memory and the process repeats. > > > > I was too fast: I can't reproduce the memory leak. > > > > Can you send me the source for prog? > > > Here is the program: > https://g
Re: general protection fault in put_pid
On Sun, Dec 23, 2018 at 1:32 PM Manfred Spraul wrote: > > Hi Dmitry, > > let's simplify the mail, otherwise noone can follow: > > On 12/23/18 11:42 AM, Dmitry Vyukov wrote: > > > >> My naive attempts to re-reproduce this failed so far. > >> But I noticed that _all_ logs for these 3 crashes: > >> https://syzkaller.appspot.com/bug?extid=c92d3646e35bc5d1a909 > >> https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac > >> https://syzkaller.appspot.com/bug?extid=9d8b6fa6ee7636f350c1 > >> involve low memory conditions. My gut feeling says this is not a > >> coincidence. This is also probably the reason why all reproducers > >> create large sem sets. There must be some bad interaction between low > >> memory condition and semaphores/ipc namespaces. > > > > Actually was able to reproduce this with a syzkaller program: > > > > ./syz-execprog -repeat=0 -procs=10 prog > > ... > > kasan: CONFIG_KASAN_INLINE enabled > > kasan: GPF could be caused by NULL-ptr deref or user memory access > > general protection fault: [#1] PREEMPT SMP KASAN > > CPU: 1 PID: 8788 Comm: syz-executor8 Not tainted 4.20.0-rc7+ #6 > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 > > 04/01/2014 > > RIP: 0010:__list_del_entry_valid+0x7e/0x150 lib/list_debug.c:51 > > Code: ad de 4c 8b 26 49 39 c4 74 66 48 b8 00 02 00 00 00 00 ad de 48 > > 89 da 48 39 c3 74 65 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c > > 02 00 75 7b 48 8b 13 48 39 f2 75 57 49 8d 7c 24 08 48 b8 00 > > RSP: 0018:88804faef210 EFLAGS: 00010a02 > > RAX: dc00 RBX: f817edba555e1f00 RCX: 831bad5f > > RDX: 1f02fdb74aabc3e0 RSI: 88801b8a0720 RDI: 88801b8a0728 > > RBP: 88804faef228 R08: f52001055401 R09: f52001055401 > > R10: 0001 R11: f52001055400 R12: 88802d52cc98 > > R13: 88801b8a0728 R14: 88801b8a0720 R15: dc00 > > FS: 00d24940() GS:88802d50() knlGS: > > CS: 0010 DS: ES: CR0: 80050033 > > CR2: 004bb580 CR3: 11177005 CR4: 003606e0 > > DR0: DR1: DR2: > > DR3: DR6: fffe0ff0 DR7: 0400 > > Call Trace: > > __list_del_entry include/linux/list.h:117 [inline] > > list_del include/linux/list.h:125 [inline] > > unlink_queue ipc/sem.c:786 [inline] > > freeary+0xddb/0x1c90 ipc/sem.c:1164 > > free_ipcs+0xf0/0x160 ipc/namespace.c:112 > > sem_exit_ns+0x20/0x40 ipc/sem.c:237 > > free_ipc_ns ipc/namespace.c:120 [inline] > > put_ipc_ns+0x55/0x160 ipc/namespace.c:152 > > free_nsproxy+0xc0/0x1f0 kernel/nsproxy.c:180 > > switch_task_namespaces+0xa5/0xc0 kernel/nsproxy.c:229 > > exit_task_namespaces+0x17/0x20 kernel/nsproxy.c:234 > > do_exit+0x19e5/0x27d0 kernel/exit.c:866 > > do_group_exit+0x151/0x410 kernel/exit.c:970 > > __do_sys_exit_group kernel/exit.c:981 [inline] > > __se_sys_exit_group kernel/exit.c:979 [inline] > > __x64_sys_exit_group+0x3e/0x50 kernel/exit.c:979 > > do_syscall_64+0x192/0x770 arch/x86/entry/common.c:290 > > entry_SYSCALL_64_after_hwframe+0x49/0xbe > > RIP: 0033:0x4570e9 > > Code: 5d af fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 > > 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d > > 01 f0 ff ff 0f 83 2b af fb ff c3 66 2e 0f 1f 84 00 00 00 00 > > RSP: 002b:7ffe35f12018 EFLAGS: 0246 ORIG_RAX: 00e7 > > RAX: ffda RBX: 0001 RCX: 004570e9 > > RDX: 00410540 RSI: 00a34c00 RDI: 0045 > > RBP: 004a43a4 R08: 000c R09: > > R10: 00d24940 R11: 0246 R12: > > R13: 0001 R14: R15: 0008 > > Modules linked in: > > Dumping ftrace buffer: > > (ftrace buffer empty) > > ---[ end trace 17829b0f00569a59 ]--- > > RIP: 0010:__list_del_entry_valid+0x7e/0x150 lib/list_debug.c:51 > > Code: ad de 4c 8b 26 49 39 c4 74 66 48 b8 00 02 00 00 00 00 ad de 48 > > 89 da 48 39 c3 74 65 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c > > 02 00 75 7b 48 8b 13 48 39 f2 75 57 49 8d 7c 24 08 48 b8 00 > > RSP: 0018:88804faef210 EFLAGS: 00010a02 > > RAX: dc00 RBX: f817edba555e1f00 RCX: 831bad5f > > RDX: 1f02fdb74aabc3e0 RSI: 88801b8a0720 RDI: 88801b8a0728 > > RBP: 88804faef228 R08: f52001055401 R09: f52001055401 > > R10: 0001 R11: f52001055400 R12: 88802d52cc98 > > R13: 88801b8a0728 R14: 88801b8a0720 R15: dc00 > > FS: 00d24940() GS:88802d50() knlGS: > > CS: 0010 DS: ES: CR0: 80050033 > > CR2: 004bb580 CR3: 11177005 CR4: 003606e0 > > DR0: DR1: DR2: > > DR3: DR6: fffe0ff0 DR7: 0400 > > > > > > The prog is: > > unshare(0
Re: general protection fault in put_pid
On Sun, Dec 23, 2018 at 7:38 PM Manfred Spraul wrote: > > Hello Dmitry, > > On 12/23/18 11:42 AM, Dmitry Vyukov wrote: > > Actually was able to reproduce this with a syzkaller program: > > ./syz-execprog -repeat=0 -procs=10 prog > > ... > > kasan: CONFIG_KASAN_INLINE enabled > > kasan: GPF could be caused by NULL-ptr deref or user memory access > > general protection fault: [#1] PREEMPT SMP KASAN > > CPU: 1 PID: 8788 Comm: syz-executor8 Not tainted 4.20.0-rc7+ #6 > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 > > 04/01/2014 > > RIP: 0010:__list_del_entry_valid+0x7e/0x150 lib/list_debug.c:51 > > Code: ad de 4c 8b 26 49 39 c4 74 66 48 b8 00 02 00 00 00 00 ad de 48 > > 89 da 48 39 c3 74 65 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c > > 02 00 75 7b 48 8b 13 48 39 f2 75 57 49 8d 7c 24 08 48 b8 00 > > RSP: 0018:88804faef210 EFLAGS: 00010a02 > > RAX: dc00 RBX: f817edba555e1f00 RCX: 831bad5f > > RDX: 1f02fdb74aabc3e0 RSI: 88801b8a0720 RDI: 88801b8a0728 > > RBP: 88804faef228 R08: f52001055401 R09: f52001055401 > > R10: 0001 R11: f52001055400 R12: 88802d52cc98 > > R13: 88801b8a0728 R14: 88801b8a0720 R15: dc00 > > FS: 00d24940() GS:88802d50() knlGS: > > CS: 0010 DS: ES: CR0: 80050033 > > CR2: 004bb580 CR3: 11177005 CR4: 003606e0 > > DR0: DR1: DR2: > > DR3: DR6: fffe0ff0 DR7: 0400 > > Call Trace: > > __list_del_entry include/linux/list.h:117 [inline] > > list_del include/linux/list.h:125 [inline] > > unlink_queue ipc/sem.c:786 [inline] > > freeary+0xddb/0x1c90 ipc/sem.c:1164 > > free_ipcs+0xf0/0x160 ipc/namespace.c:112 > > sem_exit_ns+0x20/0x40 ipc/sem.c:237 > > free_ipc_ns ipc/namespace.c:120 [inline] > > put_ipc_ns+0x55/0x160 ipc/namespace.c:152 > > free_nsproxy+0xc0/0x1f0 kernel/nsproxy.c:180 > > switch_task_namespaces+0xa5/0xc0 kernel/nsproxy.c:229 > > exit_task_namespaces+0x17/0x20 kernel/nsproxy.c:234 > > do_exit+0x19e5/0x27d0 kernel/exit.c:866 > > do_group_exit+0x151/0x410 kernel/exit.c:970 > > __do_sys_exit_group kernel/exit.c:981 [inline] > > __se_sys_exit_group kernel/exit.c:979 [inline] > > __x64_sys_exit_group+0x3e/0x50 kernel/exit.c:979 > > do_syscall_64+0x192/0x770 arch/x86/entry/common.c:290 > > entry_SYSCALL_64_after_hwframe+0x49/0xbe > > RIP: 0033:0x4570e9 > > Code: 5d af fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 > > 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d > > 01 f0 ff ff 0f 83 2b af fb ff c3 66 2e 0f 1f 84 00 00 00 00 > > RSP: 002b:7ffe35f12018 EFLAGS: 0246 ORIG_RAX: 00e7 > > RAX: ffda RBX: 0001 RCX: 004570e9 > > RDX: 00410540 RSI: 00a34c00 RDI: 0045 > > RBP: 004a43a4 R08: 000c R09: > > R10: 00d24940 R11: 0246 R12: > > R13: 0001 R14: R15: 0008 > > Modules linked in: > > Dumping ftrace buffer: > > (ftrace buffer empty) > > ---[ end trace 17829b0f00569a59 ]--- > > RIP: 0010:__list_del_entry_valid+0x7e/0x150 lib/list_debug.c:51 > > Code: ad de 4c 8b 26 49 39 c4 74 66 48 b8 00 02 00 00 00 00 ad de 48 > > 89 da 48 39 c3 74 65 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c > > 02 00 75 7b 48 8b 13 48 39 f2 75 57 49 8d 7c 24 08 48 b8 00 > > RSP: 0018:88804faef210 EFLAGS: 00010a02 > > RAX: dc00 RBX: f817edba555e1f00 RCX: 831bad5f > > RDX: 1f02fdb74aabc3e0 RSI: 88801b8a0720 RDI: 88801b8a0728 > > RBP: 88804faef228 R08: f52001055401 R09: f52001055401 > > R10: 0001 R11: f52001055400 R12: 88802d52cc98 > > R13: 88801b8a0728 R14: 88801b8a0720 R15: dc00 > > FS: 00d24940() GS:88802d50() knlGS: > > CS: 0010 DS: ES: CR0: 80050033 > > CR2: 004bb580 CR3: 11177005 CR4: 003606e0 > > DR0: DR1: DR2: > > DR3: DR6: fffe0ff0 DR7: 0400 > > > > > > The prog is: > > unshare(0x802) > > semget$private(0x0, 0x4007, 0x0) > > > > kernel is on 9105b8aa50c182371533fc97db64fc8f26f051b3 > > > > and again it involved lots of oom kills, the repro eats all memory, a > > process getting killed, frees some memory and the process repeats. > > I was too fast: I can't reproduce the memory leak. > > Can you send me the source for prog? Here is the program: https://gist.githubusercontent.com/dvyukov/03ec54b3429ade16fa07bf8b2379aff3/raw/ae4f654e279810de2505e8fa41b73dc1d8e6/gistfile1.txt But we concluded this is not a leak, right? It just creates large semaphores tied to a persistent ipcns. Once the process is killed, al
Re: general protection fault in put_pid
Hi Dmitry, let's simplify the mail, otherwise noone can follow: On 12/23/18 11:42 AM, Dmitry Vyukov wrote: My naive attempts to re-reproduce this failed so far. But I noticed that _all_ logs for these 3 crashes: https://syzkaller.appspot.com/bug?extid=c92d3646e35bc5d1a909 https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac https://syzkaller.appspot.com/bug?extid=9d8b6fa6ee7636f350c1 involve low memory conditions. My gut feeling says this is not a coincidence. This is also probably the reason why all reproducers create large sem sets. There must be some bad interaction between low memory condition and semaphores/ipc namespaces. Actually was able to reproduce this with a syzkaller program: ./syz-execprog -repeat=0 -procs=10 prog ... kasan: CONFIG_KASAN_INLINE enabled kasan: GPF could be caused by NULL-ptr deref or user memory access general protection fault: [#1] PREEMPT SMP KASAN CPU: 1 PID: 8788 Comm: syz-executor8 Not tainted 4.20.0-rc7+ #6 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 RIP: 0010:__list_del_entry_valid+0x7e/0x150 lib/list_debug.c:51 Code: ad de 4c 8b 26 49 39 c4 74 66 48 b8 00 02 00 00 00 00 ad de 48 89 da 48 39 c3 74 65 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c 02 00 75 7b 48 8b 13 48 39 f2 75 57 49 8d 7c 24 08 48 b8 00 RSP: 0018:88804faef210 EFLAGS: 00010a02 RAX: dc00 RBX: f817edba555e1f00 RCX: 831bad5f RDX: 1f02fdb74aabc3e0 RSI: 88801b8a0720 RDI: 88801b8a0728 RBP: 88804faef228 R08: f52001055401 R09: f52001055401 R10: 0001 R11: f52001055400 R12: 88802d52cc98 R13: 88801b8a0728 R14: 88801b8a0720 R15: dc00 FS: 00d24940() GS:88802d50() knlGS: CS: 0010 DS: ES: CR0: 80050033 CR2: 004bb580 CR3: 11177005 CR4: 003606e0 DR0: DR1: DR2: DR3: DR6: fffe0ff0 DR7: 0400 Call Trace: __list_del_entry include/linux/list.h:117 [inline] list_del include/linux/list.h:125 [inline] unlink_queue ipc/sem.c:786 [inline] freeary+0xddb/0x1c90 ipc/sem.c:1164 free_ipcs+0xf0/0x160 ipc/namespace.c:112 sem_exit_ns+0x20/0x40 ipc/sem.c:237 free_ipc_ns ipc/namespace.c:120 [inline] put_ipc_ns+0x55/0x160 ipc/namespace.c:152 free_nsproxy+0xc0/0x1f0 kernel/nsproxy.c:180 switch_task_namespaces+0xa5/0xc0 kernel/nsproxy.c:229 exit_task_namespaces+0x17/0x20 kernel/nsproxy.c:234 do_exit+0x19e5/0x27d0 kernel/exit.c:866 do_group_exit+0x151/0x410 kernel/exit.c:970 __do_sys_exit_group kernel/exit.c:981 [inline] __se_sys_exit_group kernel/exit.c:979 [inline] __x64_sys_exit_group+0x3e/0x50 kernel/exit.c:979 do_syscall_64+0x192/0x770 arch/x86/entry/common.c:290 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x4570e9 Code: 5d af fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 2b af fb ff c3 66 2e 0f 1f 84 00 00 00 00 RSP: 002b:7ffe35f12018 EFLAGS: 0246 ORIG_RAX: 00e7 RAX: ffda RBX: 0001 RCX: 004570e9 RDX: 00410540 RSI: 00a34c00 RDI: 0045 RBP: 004a43a4 R08: 000c R09: R10: 00d24940 R11: 0246 R12: R13: 0001 R14: R15: 0008 Modules linked in: Dumping ftrace buffer: (ftrace buffer empty) ---[ end trace 17829b0f00569a59 ]--- RIP: 0010:__list_del_entry_valid+0x7e/0x150 lib/list_debug.c:51 Code: ad de 4c 8b 26 49 39 c4 74 66 48 b8 00 02 00 00 00 00 ad de 48 89 da 48 39 c3 74 65 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <80> 3c 02 00 75 7b 48 8b 13 48 39 f2 75 57 49 8d 7c 24 08 48 b8 00 RSP: 0018:88804faef210 EFLAGS: 00010a02 RAX: dc00 RBX: f817edba555e1f00 RCX: 831bad5f RDX: 1f02fdb74aabc3e0 RSI: 88801b8a0720 RDI: 88801b8a0728 RBP: 88804faef228 R08: f52001055401 R09: f52001055401 R10: 0001 R11: f52001055400 R12: 88802d52cc98 R13: 88801b8a0728 R14: 88801b8a0720 R15: dc00 FS: 00d24940() GS:88802d50() knlGS: CS: 0010 DS: ES: CR0: 80050033 CR2: 004bb580 CR3: 11177005 CR4: 003606e0 DR0: DR1: DR2: DR3: DR6: fffe0ff0 DR7: 0400 The prog is: unshare(0x802) semget$private(0x0, 0x4007, 0x0) kernel is on 9105b8aa50c182371533fc97db64fc8f26f051b3 and again it involved lots of oom kills, the repro eats all memory, a process getting killed, frees some memory and the process repeats. Ok, thus the above program triggers two bugs: - a huge memory leak with semaphore arrays - under OOM pressure, an oops. 1) I can reproduce the memory leak, it happens all the t
Re: general protection fault in put_pid
Hello Dmitry, On 12/23/18 10:57 AM, Dmitry Vyukov wrote: I can reproduce this infinite memory consumption with the C program: https://gist.githubusercontent.com/dvyukov/03ec54b3429ade16fa07bf8b2379aff3/raw/ae4f654e279810de2505e8fa41b73dc1d8e6/gistfile1.txt But this is working as intended, right? It just creates infinite number of large semaphore sets, which reasonably consumes infinite amount of memory. Except that it also violates the memcg bound and a process can have effectively unlimited amount of such "drum memory" in semaphores. Yes, this is as intended: If you call semget(), then you can use memory, up to the limits in /proc/sys/kernel/sem. Memcg is not taken into account, an admin must set /proc/sys/kernel/sem. The default are "infinite amount of memory allowed", as this is the most sane default: We had a logic that tried to autotune (i.e.: a new namespace "inherits" a fraction of the parent namespaces memory limits), but this we more or less always wrong. -- Manfred
Re: general protection fault in put_pid
On Sun, Dec 23, 2018 at 11:30 AM Dmitry Vyukov wrote: > > On Sun, Dec 23, 2018 at 10:57 AM Dmitry Vyukov wrote: > > > > On Sun, Dec 23, 2018 at 8:37 AM Dmitry Vyukov wrote: > > > > > > On Sat, Dec 22, 2018 at 8:07 PM Manfred Spraul > > > wrote: > > > > > > > > Hi Dmitry, > > > > > > > > On 12/20/18 4:36 PM, Dmitry Vyukov wrote: > > > > > On Wed, Dec 19, 2018 at 10:04 AM Manfred Spraul > > > > > wrote: > > > > >> Hello Dmitry, > > > > >> > > > > >> On 12/12/18 11:55 AM, Dmitry Vyukov wrote: > > > > >>> On Tue, Dec 11, 2018 at 9:23 PM syzbot > > > > >>> wrote: > > > > Hello, > > > > > > > > syzbot found the following crash on: > > > > > > > > HEAD commit:f5d582777bcb Merge branch 'for-linus' of > > > > git://git.kernel... > > > > git tree: upstream > > > > console output: > > > > https://syzkaller.appspot.com/x/log.txt?x=135bc54740 > > > > kernel config: > > > > https://syzkaller.appspot.com/x/.config?x=c8970c89a0efbb23 > > > > dashboard link: > > > > https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac > > > > compiler: gcc (GCC) 8.0.1 20180413 (experimental) > > > > syz repro: > > > > https://syzkaller.appspot.com/x/repro.syz?x=16803afb40 > > > > >>> +Manfred, this looks similar to the other few crashes related to > > > > >>> semget$private(0x0, 0x4000, 0x3f) that you looked at. > > > > >> I found one unexpected (incorrect?) locking, see the attached patch. > > > > >> > > > > >> But I doubt that this is the root cause of the crashes. > > > > > > > > > > But why? These one-off sporadic crashes reported by syzbot looks > > > > > exactly like a subtle race and your patch touches sem_exit_ns involved > > > > > in all reports. > > > > > So if you don't spot anything else, I would say close these 3 reports > > > > > with this patch (I see you already included Reported-by tags which is > > > > > great!) and then wait for syzbot reaction. Since we got 3 of them, if > > > > > it's still not fixed I would expect that syzbot will be able to > > > > > retrigger this later again. > > > > > > > > As I wrote, unless semop() is used, sma->use_global_lock is always 9 and > > > > nothing can happen. > > > > > > > > Every single-operation semop() reduces use_global_lock by one, i.e a > > > > single semop call as done here cannot trigger the bug: > > > > > > > > https://syzkaller.appspot.com/text?tag=ReproSyz&x=16803afb40 > > > > > > It contains "repeat":true,"procs":6, which means that it run 6 > > > processes running this test in infinite loop. The last mark about > > > number of tests executed was: > > > 2018/12/11 18:38:02 executed programs: 2955 > > > > > > > But, one more finding: > > > > > > > > https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac > > > > > > > > https://syzkaller.appspot.com/text?tag=CrashLog&x=109ecf6e40 > > > > > > > > The log file contain 1080 lines like these: > > > > > > > > > semget$private(..., 0x4003, ...) > > > > > > > > > > semget$private(..., 0x4006, ...) > > > > > > > > > > semget$private(..., 0x4007, ...) > > > > > > > > It ends up as kmalloc(128*0x400x), i.e. slightly more than 2 MB, an > > > > allocation in the 4 MB kmalloc buffer: > > > > > > > > > [ 1201.210245] kmalloc-4194304 4698112KB4698112KB > > > > > > > > > i.e.: 1147 4 MB kmalloc blocks --> are we leaking nearly 100% of the > > > > semaphore arrays?? > > > > > > /\/\/\/\/\/\ > > > > > > Ha, this is definitely not healthy. > > > > I can reproduce this infinite memory consumption with the C program: > > https://gist.githubusercontent.com/dvyukov/03ec54b3429ade16fa07bf8b2379aff3/raw/ae4f654e279810de2505e8fa41b73dc1d8e6/gistfile1.txt > > > > But this is working as intended, right? It just creates infinite > > number of large semaphore sets, which reasonably consumes infinite > > amount of memory. > > Except that it also violates the memcg bound and a process can have > > effectively unlimited amount of such "drum memory" in semaphores. > > > > > > > > > > > > This one looks similar: > > > > > > > > https://syzkaller.appspot.com/bug?extid=c92d3646e35bc5d1a909 > > > > > > > > except that the array sizes are mixed, and thus there are kmalloc-1M and > > > > kmalloc-2M as well. > > > > > > > > (and I did not count the number of semget calls) > > > > > > > > > > > > The test apps use unshare(CLONE_NEWNS) and unshare(CLONE_NEWIPC), > > > > correct? > > > > > > > > I.e. no CLONE_NEWUSER. > > > > > > > > https://github.com/google/syzkaller/blob/master/executor/common_linux.h#L1523 > > > > > > CLONE_NEWUSER is used on some instances as well: > > > https://github.com/google/syzkaller/blob/master/executor/common_linux.h#L1765 > > > This crash happened on 2 different instances and 1 of them uses > > > CLONE_NEWUSER and another does not. > > > If it's important because of CAP_ADMIN in IPC namespace, then all > > > instances should have it (instances that don't use NEWUSER are just > > >
Re: general protection fault in put_pid
On Sun, Dec 23, 2018 at 10:57 AM Dmitry Vyukov wrote: > > On Sun, Dec 23, 2018 at 8:37 AM Dmitry Vyukov wrote: > > > > On Sat, Dec 22, 2018 at 8:07 PM Manfred Spraul > > wrote: > > > > > > Hi Dmitry, > > > > > > On 12/20/18 4:36 PM, Dmitry Vyukov wrote: > > > > On Wed, Dec 19, 2018 at 10:04 AM Manfred Spraul > > > > wrote: > > > >> Hello Dmitry, > > > >> > > > >> On 12/12/18 11:55 AM, Dmitry Vyukov wrote: > > > >>> On Tue, Dec 11, 2018 at 9:23 PM syzbot > > > >>> wrote: > > > Hello, > > > > > > syzbot found the following crash on: > > > > > > HEAD commit:f5d582777bcb Merge branch 'for-linus' of > > > git://git.kernel... > > > git tree: upstream > > > console output: > > > https://syzkaller.appspot.com/x/log.txt?x=135bc54740 > > > kernel config: > > > https://syzkaller.appspot.com/x/.config?x=c8970c89a0efbb23 > > > dashboard link: > > > https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac > > > compiler: gcc (GCC) 8.0.1 20180413 (experimental) > > > syz repro: > > > https://syzkaller.appspot.com/x/repro.syz?x=16803afb40 > > > >>> +Manfred, this looks similar to the other few crashes related to > > > >>> semget$private(0x0, 0x4000, 0x3f) that you looked at. > > > >> I found one unexpected (incorrect?) locking, see the attached patch. > > > >> > > > >> But I doubt that this is the root cause of the crashes. > > > > > > > > But why? These one-off sporadic crashes reported by syzbot looks > > > > exactly like a subtle race and your patch touches sem_exit_ns involved > > > > in all reports. > > > > So if you don't spot anything else, I would say close these 3 reports > > > > with this patch (I see you already included Reported-by tags which is > > > > great!) and then wait for syzbot reaction. Since we got 3 of them, if > > > > it's still not fixed I would expect that syzbot will be able to > > > > retrigger this later again. > > > > > > As I wrote, unless semop() is used, sma->use_global_lock is always 9 and > > > nothing can happen. > > > > > > Every single-operation semop() reduces use_global_lock by one, i.e a > > > single semop call as done here cannot trigger the bug: > > > > > > https://syzkaller.appspot.com/text?tag=ReproSyz&x=16803afb40 > > > > It contains "repeat":true,"procs":6, which means that it run 6 > > processes running this test in infinite loop. The last mark about > > number of tests executed was: > > 2018/12/11 18:38:02 executed programs: 2955 > > > > > But, one more finding: > > > > > > https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac > > > > > > https://syzkaller.appspot.com/text?tag=CrashLog&x=109ecf6e40 > > > > > > The log file contain 1080 lines like these: > > > > > > > semget$private(..., 0x4003, ...) > > > > > > > > semget$private(..., 0x4006, ...) > > > > > > > > semget$private(..., 0x4007, ...) > > > > > > It ends up as kmalloc(128*0x400x), i.e. slightly more than 2 MB, an > > > allocation in the 4 MB kmalloc buffer: > > > > > > > [ 1201.210245] kmalloc-4194304 4698112KB4698112KB > > > > > > > i.e.: 1147 4 MB kmalloc blocks --> are we leaking nearly 100% of the > > > semaphore arrays?? > > > > /\/\/\/\/\/\ > > > > Ha, this is definitely not healthy. > > I can reproduce this infinite memory consumption with the C program: > https://gist.githubusercontent.com/dvyukov/03ec54b3429ade16fa07bf8b2379aff3/raw/ae4f654e279810de2505e8fa41b73dc1d8e6/gistfile1.txt > > But this is working as intended, right? It just creates infinite > number of large semaphore sets, which reasonably consumes infinite > amount of memory. > Except that it also violates the memcg bound and a process can have > effectively unlimited amount of such "drum memory" in semaphores. > > > > > > > This one looks similar: > > > > > > https://syzkaller.appspot.com/bug?extid=c92d3646e35bc5d1a909 > > > > > > except that the array sizes are mixed, and thus there are kmalloc-1M and > > > kmalloc-2M as well. > > > > > > (and I did not count the number of semget calls) > > > > > > > > > The test apps use unshare(CLONE_NEWNS) and unshare(CLONE_NEWIPC), correct? > > > > > > I.e. no CLONE_NEWUSER. > > > > > > https://github.com/google/syzkaller/blob/master/executor/common_linux.h#L1523 > > > > CLONE_NEWUSER is used on some instances as well: > > https://github.com/google/syzkaller/blob/master/executor/common_linux.h#L1765 > > This crash happened on 2 different instances and 1 of them uses > > CLONE_NEWUSER and another does not. > > If it's important because of CAP_ADMIN in IPC namespace, then all > > instances should have it (instances that don't use NEWUSER are just > > root). My naive attempts to re-reproduce this failed so far. But I noticed that _all_ logs for these 3 crashes: https://syzkaller.appspot.com/bug?extid=c92d3646e35bc5d1a909 https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac https://syzkaller.appspot.com/bug?extid=9d8b6fa6ee7636f350c1 involve low me
Re: general protection fault in put_pid
On Sun, Dec 23, 2018 at 8:37 AM Dmitry Vyukov wrote: > > On Sat, Dec 22, 2018 at 8:07 PM Manfred Spraul > wrote: > > > > Hi Dmitry, > > > > On 12/20/18 4:36 PM, Dmitry Vyukov wrote: > > > On Wed, Dec 19, 2018 at 10:04 AM Manfred Spraul > > > wrote: > > >> Hello Dmitry, > > >> > > >> On 12/12/18 11:55 AM, Dmitry Vyukov wrote: > > >>> On Tue, Dec 11, 2018 at 9:23 PM syzbot > > >>> wrote: > > Hello, > > > > syzbot found the following crash on: > > > > HEAD commit:f5d582777bcb Merge branch 'for-linus' of > > git://git.kernel... > > git tree: upstream > > console output: > > https://syzkaller.appspot.com/x/log.txt?x=135bc54740 > > kernel config: > > https://syzkaller.appspot.com/x/.config?x=c8970c89a0efbb23 > > dashboard link: > > https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac > > compiler: gcc (GCC) 8.0.1 20180413 (experimental) > > syz repro: > > https://syzkaller.appspot.com/x/repro.syz?x=16803afb40 > > >>> +Manfred, this looks similar to the other few crashes related to > > >>> semget$private(0x0, 0x4000, 0x3f) that you looked at. > > >> I found one unexpected (incorrect?) locking, see the attached patch. > > >> > > >> But I doubt that this is the root cause of the crashes. > > > > > > But why? These one-off sporadic crashes reported by syzbot looks > > > exactly like a subtle race and your patch touches sem_exit_ns involved > > > in all reports. > > > So if you don't spot anything else, I would say close these 3 reports > > > with this patch (I see you already included Reported-by tags which is > > > great!) and then wait for syzbot reaction. Since we got 3 of them, if > > > it's still not fixed I would expect that syzbot will be able to > > > retrigger this later again. > > > > As I wrote, unless semop() is used, sma->use_global_lock is always 9 and > > nothing can happen. > > > > Every single-operation semop() reduces use_global_lock by one, i.e a > > single semop call as done here cannot trigger the bug: > > > > https://syzkaller.appspot.com/text?tag=ReproSyz&x=16803afb40 > > It contains "repeat":true,"procs":6, which means that it run 6 > processes running this test in infinite loop. The last mark about > number of tests executed was: > 2018/12/11 18:38:02 executed programs: 2955 > > > But, one more finding: > > > > https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac > > > > https://syzkaller.appspot.com/text?tag=CrashLog&x=109ecf6e40 > > > > The log file contain 1080 lines like these: > > > > > semget$private(..., 0x4003, ...) > > > > > > semget$private(..., 0x4006, ...) > > > > > > semget$private(..., 0x4007, ...) > > > > It ends up as kmalloc(128*0x400x), i.e. slightly more than 2 MB, an > > allocation in the 4 MB kmalloc buffer: > > > > > [ 1201.210245] kmalloc-4194304 4698112KB4698112KB > > > > > i.e.: 1147 4 MB kmalloc blocks --> are we leaking nearly 100% of the > > semaphore arrays?? > > /\/\/\/\/\/\ > > Ha, this is definitely not healthy. I can reproduce this infinite memory consumption with the C program: https://gist.githubusercontent.com/dvyukov/03ec54b3429ade16fa07bf8b2379aff3/raw/ae4f654e279810de2505e8fa41b73dc1d8e6/gistfile1.txt But this is working as intended, right? It just creates infinite number of large semaphore sets, which reasonably consumes infinite amount of memory. Except that it also violates the memcg bound and a process can have effectively unlimited amount of such "drum memory" in semaphores. > > This one looks similar: > > > > https://syzkaller.appspot.com/bug?extid=c92d3646e35bc5d1a909 > > > > except that the array sizes are mixed, and thus there are kmalloc-1M and > > kmalloc-2M as well. > > > > (and I did not count the number of semget calls) > > > > > > The test apps use unshare(CLONE_NEWNS) and unshare(CLONE_NEWIPC), correct? > > > > I.e. no CLONE_NEWUSER. > > > > https://github.com/google/syzkaller/blob/master/executor/common_linux.h#L1523 > > CLONE_NEWUSER is used on some instances as well: > https://github.com/google/syzkaller/blob/master/executor/common_linux.h#L1765 > This crash happened on 2 different instances and 1 of them uses > CLONE_NEWUSER and another does not. > If it's important because of CAP_ADMIN in IPC namespace, then all > instances should have it (instances that don't use NEWUSER are just > root).
Re: general protection fault in put_pid
On Sat, Dec 22, 2018 at 8:07 PM Manfred Spraul wrote: > > Hi Dmitry, > > On 12/20/18 4:36 PM, Dmitry Vyukov wrote: > > On Wed, Dec 19, 2018 at 10:04 AM Manfred Spraul > > wrote: > >> Hello Dmitry, > >> > >> On 12/12/18 11:55 AM, Dmitry Vyukov wrote: > >>> On Tue, Dec 11, 2018 at 9:23 PM syzbot > >>> wrote: > Hello, > > syzbot found the following crash on: > > HEAD commit:f5d582777bcb Merge branch 'for-linus' of > git://git.kernel... > git tree: upstream > console output: https://syzkaller.appspot.com/x/log.txt?x=135bc54740 > kernel config: > https://syzkaller.appspot.com/x/.config?x=c8970c89a0efbb23 > dashboard link: > https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac > compiler: gcc (GCC) 8.0.1 20180413 (experimental) > syz repro: > https://syzkaller.appspot.com/x/repro.syz?x=16803afb40 > >>> +Manfred, this looks similar to the other few crashes related to > >>> semget$private(0x0, 0x4000, 0x3f) that you looked at. > >> I found one unexpected (incorrect?) locking, see the attached patch. > >> > >> But I doubt that this is the root cause of the crashes. > > > > But why? These one-off sporadic crashes reported by syzbot looks > > exactly like a subtle race and your patch touches sem_exit_ns involved > > in all reports. > > So if you don't spot anything else, I would say close these 3 reports > > with this patch (I see you already included Reported-by tags which is > > great!) and then wait for syzbot reaction. Since we got 3 of them, if > > it's still not fixed I would expect that syzbot will be able to > > retrigger this later again. > > As I wrote, unless semop() is used, sma->use_global_lock is always 9 and > nothing can happen. > > Every single-operation semop() reduces use_global_lock by one, i.e a > single semop call as done here cannot trigger the bug: > > https://syzkaller.appspot.com/text?tag=ReproSyz&x=16803afb40 It contains "repeat":true,"procs":6, which means that it run 6 processes running this test in infinite loop. The last mark about number of tests executed was: 2018/12/11 18:38:02 executed programs: 2955 > But, one more finding: > > https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac > > https://syzkaller.appspot.com/text?tag=CrashLog&x=109ecf6e40 > > The log file contain 1080 lines like these: > > > semget$private(..., 0x4003, ...) > > > > semget$private(..., 0x4006, ...) > > > > semget$private(..., 0x4007, ...) > > It ends up as kmalloc(128*0x400x), i.e. slightly more than 2 MB, an > allocation in the 4 MB kmalloc buffer: > > > [ 1201.210245] kmalloc-4194304 4698112KB4698112KB > > > i.e.: 1147 4 MB kmalloc blocks --> are we leaking nearly 100% of the > semaphore arrays?? /\/\/\/\/\/\ Ha, this is definitely not healthy. > This one looks similar: > > https://syzkaller.appspot.com/bug?extid=c92d3646e35bc5d1a909 > > except that the array sizes are mixed, and thus there are kmalloc-1M and > kmalloc-2M as well. > > (and I did not count the number of semget calls) > > > The test apps use unshare(CLONE_NEWNS) and unshare(CLONE_NEWIPC), correct? > > I.e. no CLONE_NEWUSER. > > https://github.com/google/syzkaller/blob/master/executor/common_linux.h#L1523 CLONE_NEWUSER is used on some instances as well: https://github.com/google/syzkaller/blob/master/executor/common_linux.h#L1765 This crash happened on 2 different instances and 1 of them uses CLONE_NEWUSER and another does not. If it's important because of CAP_ADMIN in IPC namespace, then all instances should have it (instances that don't use NEWUSER are just root).
Re: general protection fault in put_pid
Hi Dmitry, On 12/20/18 4:36 PM, Dmitry Vyukov wrote: On Wed, Dec 19, 2018 at 10:04 AM Manfred Spraul wrote: Hello Dmitry, On 12/12/18 11:55 AM, Dmitry Vyukov wrote: On Tue, Dec 11, 2018 at 9:23 PM syzbot wrote: Hello, syzbot found the following crash on: HEAD commit:f5d582777bcb Merge branch 'for-linus' of git://git.kernel... git tree: upstream console output: https://syzkaller.appspot.com/x/log.txt?x=135bc54740 kernel config: https://syzkaller.appspot.com/x/.config?x=c8970c89a0efbb23 dashboard link: https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac compiler: gcc (GCC) 8.0.1 20180413 (experimental) syz repro: https://syzkaller.appspot.com/x/repro.syz?x=16803afb40 +Manfred, this looks similar to the other few crashes related to semget$private(0x0, 0x4000, 0x3f) that you looked at. I found one unexpected (incorrect?) locking, see the attached patch. But I doubt that this is the root cause of the crashes. But why? These one-off sporadic crashes reported by syzbot looks exactly like a subtle race and your patch touches sem_exit_ns involved in all reports. So if you don't spot anything else, I would say close these 3 reports with this patch (I see you already included Reported-by tags which is great!) and then wait for syzbot reaction. Since we got 3 of them, if it's still not fixed I would expect that syzbot will be able to retrigger this later again. As I wrote, unless semop() is used, sma->use_global_lock is always 9 and nothing can happen. Every single-operation semop() reduces use_global_lock by one, i.e a single semop call as done here cannot trigger the bug: https://syzkaller.appspot.com/text?tag=ReproSyz&x=16803afb40 But, one more finding: https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac https://syzkaller.appspot.com/text?tag=CrashLog&x=109ecf6e40 The log file contain 1080 lines like these: semget$private(..., 0x4003, ...) semget$private(..., 0x4006, ...) semget$private(..., 0x4007, ...) It ends up as kmalloc(128*0x400x), i.e. slightly more than 2 MB, an allocation in the 4 MB kmalloc buffer: [ 1201.210245] kmalloc-4194304 4698112KB4698112KB i.e.: 1147 4 MB kmalloc blocks --> are we leaking nearly 100% of the semaphore arrays?? This one looks similar: https://syzkaller.appspot.com/bug?extid=c92d3646e35bc5d1a909 except that the array sizes are mixed, and thus there are kmalloc-1M and kmalloc-2M as well. (and I did not count the number of semget calls) The test apps use unshare(CLONE_NEWNS) and unshare(CLONE_NEWIPC), correct? I.e. no CLONE_NEWUSER. https://github.com/google/syzkaller/blob/master/executor/common_linux.h#L1523 -- Manfred
Re: general protection fault in put_pid
On Wed, Dec 19, 2018 at 10:04 AM Manfred Spraul wrote: > > Hello Dmitry, > > On 12/12/18 11:55 AM, Dmitry Vyukov wrote: > > On Tue, Dec 11, 2018 at 9:23 PM syzbot > > wrote: > >> Hello, > >> > >> syzbot found the following crash on: > >> > >> HEAD commit:f5d582777bcb Merge branch 'for-linus' of > >> git://git.kernel... > >> git tree: upstream > >> console output: https://syzkaller.appspot.com/x/log.txt?x=135bc54740 > >> kernel config: https://syzkaller.appspot.com/x/.config?x=c8970c89a0efbb23 > >> dashboard link: > >> https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac > >> compiler: gcc (GCC) 8.0.1 20180413 (experimental) > >> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=16803afb40 > > +Manfred, this looks similar to the other few crashes related to > > semget$private(0x0, 0x4000, 0x3f) that you looked at. > > I found one unexpected (incorrect?) locking, see the attached patch. > > But I doubt that this is the root cause of the crashes. But why? These one-off sporadic crashes reported by syzbot looks exactly like a subtle race and your patch touches sem_exit_ns involved in all reports. So if you don't spot anything else, I would say close these 3 reports with this patch (I see you already included Reported-by tags which is great!) and then wait for syzbot reaction. Since we got 3 of them, if it's still not fixed I would expect that syzbot will be able to retrigger this later again. > Any remarks on the patch? > > I would continue to search, and then send a series with all findings. > > -- > > Manfred >
Re: general protection fault in put_pid
Hello Dmitry,
On 12/12/18 11:55 AM, Dmitry Vyukov wrote:
On Tue, Dec 11, 2018 at 9:23 PM syzbot
wrote:
Hello,
syzbot found the following crash on:
HEAD commit:f5d582777bcb Merge branch 'for-linus' of git://git.kernel...
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=135bc54740
kernel config: https://syzkaller.appspot.com/x/.config?x=c8970c89a0efbb23
dashboard link: https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=16803afb40
+Manfred, this looks similar to the other few crashes related to
semget$private(0x0, 0x4000, 0x3f) that you looked at.
I found one unexpected (incorrect?) locking, see the attached patch.
But I doubt that this is the root cause of the crashes.
Any remarks on the patch?
I would continue to search, and then send a series with all findings.
--
Manfred
>From 733e888993b71fb3c139f71de61534bc603a2bcb Mon Sep 17 00:00:00 2001
From: Manfred Spraul
Date: Wed, 19 Dec 2018 09:26:48 +0100
Subject: [PATCH] ipc/sem.c: ensure proper locking during namespace teardown
free_ipcs() only calls ipc_lock_object() before calling the free callback.
This means:
- There is no exclusion against parallel simple semop() calls.
- sma->use_global_lock may underflow (i.e. jump to UNIT_MAX) when
freeary() calls sem_unlock(,,-1).
The patch fixes that, by adding complexmode_enter() before calling
freeary().
There are multiple syzbot crashes in this code area, but I don't see yet
how a missing complexmode_enter() may cause a crash:
- 1) simple semop() calls are not used by these syzbox tests,
and 2) we are in namespace teardown, noone may run in parallel.
- 1) freeary() is the last call (except parallel operations, which
are impossible due to namespace teardown)
and 2) the underflow of use_global_lock merely delays switching to
parallel simple semop handling for the next UINT_MAX semop() calls.
Thus I think the patch is "only" a cleanup, and does not fix
the observed crashes.
Signed-off-by: Manfred Spraul
Reported-by: syzbot+1145ec2e23165570c...@syzkaller.appspotmail.com
Reported-by: syzbot+c92d3646e35bc5d1a...@syzkaller.appspotmail.com
Reported-by: syzbot+9d8b6fa6ee7636f35...@syzkaller.appspotmail.com
Cc: dvyu...@google.com
Cc: dbu...@suse.de
Cc: Andrew Morton
---
ipc/sem.c | 24 ++--
1 file changed, 22 insertions(+), 2 deletions(-)
diff --git a/ipc/sem.c b/ipc/sem.c
index 745dc6187e84..8ccacd11fb15 100644
--- a/ipc/sem.c
+++ b/ipc/sem.c
@@ -184,6 +184,9 @@ static int sysvipc_sem_proc_show(struct seq_file *s, void *it);
*/
#define USE_GLOBAL_LOCK_HYSTERESIS 10
+static void complexmode_enter(struct sem_array *sma);
+static void complexmode_tryleave(struct sem_array *sma);
+
/*
* Locking:
* a) global sem_lock() for read/write
@@ -232,9 +235,24 @@ void sem_init_ns(struct ipc_namespace *ns)
}
#ifdef CONFIG_IPC_NS
+
+static void freeary_lock(struct ipc_namespace *ns, struct kern_ipc_perm *ipcp)
+{
+ struct sem_array *sma = container_of(ipcp, struct sem_array, sem_perm);
+
+ /*
+ * free_ipcs() isn't aware of sem_lock(), it calls ipc_lock_object()
+ * directly. In order to stay compatible with sem_lock(), we must
+ * upgrade from "simple" ipc_lock_object() to sem_lock(,,-1).
+ */
+ complexmode_enter(sma);
+
+ freeary(ns, ipcp);
+}
+
void sem_exit_ns(struct ipc_namespace *ns)
{
- free_ipcs(ns, &sem_ids(ns), freeary);
+ free_ipcs(ns, &sem_ids(ns), freeary_lock);
idr_destroy(&ns->ids[IPC_SEM_IDS].ipcs_idr);
rhashtable_destroy(&ns->ids[IPC_SEM_IDS].key_ht);
}
@@ -374,7 +392,9 @@ static inline int sem_lock(struct sem_array *sma, struct sembuf *sops,
/* Complex operation - acquire a full lock */
ipc_lock_object(&sma->sem_perm);
- /* Prevent parallel simple ops */
+ /* Prevent parallel simple ops.
+ * This must be identical to freeary_lock().
+ */
complexmode_enter(sma);
return SEM_GLOBAL_LOCK;
}
--
2.17.2
Re: general protection fault in put_pid
On Tue, Dec 11, 2018 at 9:23 PM syzbot wrote: > > Hello, > > syzbot found the following crash on: > > HEAD commit:f5d582777bcb Merge branch 'for-linus' of git://git.kernel... > git tree: upstream > console output: https://syzkaller.appspot.com/x/log.txt?x=135bc54740 > kernel config: https://syzkaller.appspot.com/x/.config?x=c8970c89a0efbb23 > dashboard link: https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac > compiler: gcc (GCC) 8.0.1 20180413 (experimental) > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=16803afb40 +Manfred, this looks similar to the other few crashes related to semget$private(0x0, 0x4000, 0x3f) that you looked at. > IMPORTANT: if you fix the bug, please add the following tag to the commit: > Reported-by: syzbot+1145ec2e23165570c...@syzkaller.appspotmail.com > > kmem_cache 221KB225KB > Out of memory: Kill process 6139 (syz-execprog) score 1 or sacrifice child > Killed process 6164 (syz-executor0) total-vm:37444kB, anon-rss:60kB, > file-rss:0kB, shmem-rss:0kB > oom_reaper: reaped process 6164 (syz-executor0), now anon-rss:0kB, > file-rss:0kB, shmem-rss:0kB > kasan: GPF could be caused by NULL-ptr deref or user memory access > general protection fault: [#1] PREEMPT SMP KASAN > CPU: 0 PID: 6159 Comm: syz-executor3 Not tainted 4.20.0-rc6+ #151 > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS > Google 01/01/2011 > RIP: 0010:put_pid.part.3+0xb6/0x240 kernel/pid.c:108 > Code: d2 0f 85 89 01 00 00 44 8b 63 04 49 8d 44 24 03 48 c1 e0 04 48 8d 7c > 03 10 48 b8 00 00 00 00 00 fc ff df 48 89 fa 48 c1 ea 03 <80> 3c 02 00 0f > 85 66 01 00 00 49 83 c4 03 be 04 00 00 00 48 89 df > RSP: 0018:8881ae116e78 EFLAGS: 00010202 > RAX: dc00 RBX: 816149a0 RCX: 833e420e > RDX: 0001933eab7c RSI: 8152bf8e RDI: 000c99f55be0 > RBP: 8881ae116f08 R08: 8881cdbf2300 R09: f52001507600 > R10: f52001507600 R11: c9000a83b003 R12: d1894120 > R13: 111035c22dd0 R14: 8881ae116ee0 R15: dc00 > FS: 0166d940() GS:8881dae0() knlGS: > CS: 0010 DS: ES: CR0: 80050033 > CR2: 0075c458 CR3: 0001d2bd1000 CR4: 001406f0 > DR0: DR1: DR2: > DR3: DR6: fffe0ff0 DR7: 0400 > Call Trace: > put_pid+0x1f/0x30 kernel/pid.c:105 > ipc_update_pid ipc/util.h:159 [inline] > freeary+0x10c8/0x1a40 ipc/sem.c:1167 > free_ipcs+0x9f/0x1c0 ipc/namespace.c:112 > sem_exit_ns+0x20/0x40 ipc/sem.c:237 > free_ipc_ns ipc/namespace.c:120 [inline] > put_ipc_ns+0x66/0x180 ipc/namespace.c:152 > free_nsproxy+0xcf/0x220 kernel/nsproxy.c:180 > switch_task_namespaces+0xb3/0xd0 kernel/nsproxy.c:229 > exit_task_namespaces+0x17/0x20 kernel/nsproxy.c:234 > do_exit+0x1ad1/0x26d0 kernel/exit.c:866 > do_group_exit+0x177/0x440 kernel/exit.c:970 > get_signal+0x8b0/0x1980 kernel/signal.c:2517 > do_signal+0x9c/0x21c0 arch/x86/kernel/signal.c:816 > exit_to_usermode_loop+0x2e5/0x380 arch/x86/entry/common.c:162 > prepare_exit_to_usermode+0x342/0x3b0 arch/x86/entry/common.c:197 > retint_user+0x8/0x18 > RIP: 0033:0x45a4d0 > Code: 10 44 00 00 00 e8 c0 cc ff ff 0f b6 44 24 18 eb c2 e8 44 ad ff ff e9 > 6f ff ff ff cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc <64> 48 8b 0c 25 > f8 ff ff ff 48 3b 61 10 76 68 48 83 ec 28 48 89 6c > RSP: 002b:7fff9b9d3578 EFLAGS: 00010246 > RAX: RBX: 000a4edb RCX: 00483170 > RDX: RSI: 7fff9b9d3580 RDI: 0001 > RBP: 02ef R08: 0001 R09: 0166d940 > R10: R11: 0246 R12: 0003 > R13: 000a4e70 R14: 01f1 R15: 0003 > Modules linked in: > ---[ end trace 9933854824df8420 ]--- > RIP: 0010:put_pid.part.3+0xb6/0x240 kernel/pid.c:108 > Code: d2 0f 85 89 01 00 00 44 8b 63 04 49 8d 44 24 03 48 c1 e0 04 48 8d 7c > 03 10 48 b8 00 00 00 00 00 fc ff df 48 89 fa 48 c1 ea 03 <80> 3c 02 00 0f > 85 66 01 00 00 49 83 c4 03 be 04 00 00 00 48 89 df > RSP: 0018:8881ae116e78 EFLAGS: 00010202 > RAX: dc00 RBX: 816149a0 RCX: 833e420e > RDX: 0001933eab7c RSI: 8152bf8e RDI: 000c99f55be0 > RBP: 8881ae116f08 R08: 8881cdbf2300 R09: f52001507600 > R10: f52001507600 R11: c9000a83b003 R12: d1894120 > R13: 111035c22dd0 R14: 8881ae116ee0 R15: dc00 > FS: 0166d940() GS:8881dae0() knlGS: > CS: 0010 DS: ES: CR0: 80050033 > CR2: 0075c458 CR3: 0001d2bd1000 CR4: 001406f0 > DR0: DR1: DR2: > DR3: DR6: fffe0ff0 DR7: 0400 > > > --- > This bug is generated by a bot. It may contain errors. > See https
general protection fault in put_pid
Hello, syzbot found the following crash on: HEAD commit:f5d582777bcb Merge branch 'for-linus' of git://git.kernel... git tree: upstream console output: https://syzkaller.appspot.com/x/log.txt?x=135bc54740 kernel config: https://syzkaller.appspot.com/x/.config?x=c8970c89a0efbb23 dashboard link: https://syzkaller.appspot.com/bug?extid=1145ec2e23165570c3ac compiler: gcc (GCC) 8.0.1 20180413 (experimental) syz repro: https://syzkaller.appspot.com/x/repro.syz?x=16803afb40 IMPORTANT: if you fix the bug, please add the following tag to the commit: Reported-by: syzbot+1145ec2e23165570c...@syzkaller.appspotmail.com kmem_cache 221KB225KB Out of memory: Kill process 6139 (syz-execprog) score 1 or sacrifice child Killed process 6164 (syz-executor0) total-vm:37444kB, anon-rss:60kB, file-rss:0kB, shmem-rss:0kB oom_reaper: reaped process 6164 (syz-executor0), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB kasan: GPF could be caused by NULL-ptr deref or user memory access general protection fault: [#1] PREEMPT SMP KASAN CPU: 0 PID: 6159 Comm: syz-executor3 Not tainted 4.20.0-rc6+ #151 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:put_pid.part.3+0xb6/0x240 kernel/pid.c:108 Code: d2 0f 85 89 01 00 00 44 8b 63 04 49 8d 44 24 03 48 c1 e0 04 48 8d 7c 03 10 48 b8 00 00 00 00 00 fc ff df 48 89 fa 48 c1 ea 03 <80> 3c 02 00 0f 85 66 01 00 00 49 83 c4 03 be 04 00 00 00 48 89 df RSP: 0018:8881ae116e78 EFLAGS: 00010202 RAX: dc00 RBX: 816149a0 RCX: 833e420e RDX: 0001933eab7c RSI: 8152bf8e RDI: 000c99f55be0 RBP: 8881ae116f08 R08: 8881cdbf2300 R09: f52001507600 R10: f52001507600 R11: c9000a83b003 R12: d1894120 R13: 111035c22dd0 R14: 8881ae116ee0 R15: dc00 FS: 0166d940() GS:8881dae0() knlGS: CS: 0010 DS: ES: CR0: 80050033 CR2: 0075c458 CR3: 0001d2bd1000 CR4: 001406f0 DR0: DR1: DR2: DR3: DR6: fffe0ff0 DR7: 0400 Call Trace: put_pid+0x1f/0x30 kernel/pid.c:105 ipc_update_pid ipc/util.h:159 [inline] freeary+0x10c8/0x1a40 ipc/sem.c:1167 free_ipcs+0x9f/0x1c0 ipc/namespace.c:112 sem_exit_ns+0x20/0x40 ipc/sem.c:237 free_ipc_ns ipc/namespace.c:120 [inline] put_ipc_ns+0x66/0x180 ipc/namespace.c:152 free_nsproxy+0xcf/0x220 kernel/nsproxy.c:180 switch_task_namespaces+0xb3/0xd0 kernel/nsproxy.c:229 exit_task_namespaces+0x17/0x20 kernel/nsproxy.c:234 do_exit+0x1ad1/0x26d0 kernel/exit.c:866 do_group_exit+0x177/0x440 kernel/exit.c:970 get_signal+0x8b0/0x1980 kernel/signal.c:2517 do_signal+0x9c/0x21c0 arch/x86/kernel/signal.c:816 exit_to_usermode_loop+0x2e5/0x380 arch/x86/entry/common.c:162 prepare_exit_to_usermode+0x342/0x3b0 arch/x86/entry/common.c:197 retint_user+0x8/0x18 RIP: 0033:0x45a4d0 Code: 10 44 00 00 00 e8 c0 cc ff ff 0f b6 44 24 18 eb c2 e8 44 ad ff ff e9 6f ff ff ff cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc <64> 48 8b 0c 25 f8 ff ff ff 48 3b 61 10 76 68 48 83 ec 28 48 89 6c RSP: 002b:7fff9b9d3578 EFLAGS: 00010246 RAX: RBX: 000a4edb RCX: 00483170 RDX: RSI: 7fff9b9d3580 RDI: 0001 RBP: 02ef R08: 0001 R09: 0166d940 R10: R11: 0246 R12: 0003 R13: 000a4e70 R14: 01f1 R15: 0003 Modules linked in: ---[ end trace 9933854824df8420 ]--- RIP: 0010:put_pid.part.3+0xb6/0x240 kernel/pid.c:108 Code: d2 0f 85 89 01 00 00 44 8b 63 04 49 8d 44 24 03 48 c1 e0 04 48 8d 7c 03 10 48 b8 00 00 00 00 00 fc ff df 48 89 fa 48 c1 ea 03 <80> 3c 02 00 0f 85 66 01 00 00 49 83 c4 03 be 04 00 00 00 48 89 df RSP: 0018:8881ae116e78 EFLAGS: 00010202 RAX: dc00 RBX: 816149a0 RCX: 833e420e RDX: 0001933eab7c RSI: 8152bf8e RDI: 000c99f55be0 RBP: 8881ae116f08 R08: 8881cdbf2300 R09: f52001507600 R10: f52001507600 R11: c9000a83b003 R12: d1894120 R13: 111035c22dd0 R14: 8881ae116ee0 R15: dc00 FS: 0166d940() GS:8881dae0() knlGS: CS: 0010 DS: ES: CR0: 80050033 CR2: 0075c458 CR3: 0001d2bd1000 CR4: 001406f0 DR0: DR1: DR2: DR3: DR6: fffe0ff0 DR7: 0400 --- This bug is generated by a bot. It may contain errors. See https://goo.gl/tpsmEJ for more information about syzbot. syzbot engineers can be reached at syzkal...@googlegroups.com. syzbot will keep track of this bug report. See: https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with syzbot. syzbot can test patches for this bug, for details see: https://goo.gl/tpsmEJ#testing-patches
