Re: KASAN: use-after-free Read in alloc_pid
Tetsuo Handawrites: > On 2018/04/10 23:33, Tetsuo Handa wrote: >> syzbot wrote: >>> syzbot has found reproducer for the following crash on upstream commit >>> c18bb396d3d261ebbb4efbc05129c5d354c541e4 (Tue Apr 10 00:04:10 2018 +) >>> Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net >>> syzbot dashboard link: >>> https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c >>> >> While we are waiting for >> >> rpc_pipefs: fix double-dput() >> rpc_pipefs: deal with early sget() failures >> kernfs: deal with early sget() failures >> procfs: deal with early sget() failures >> orangefs_kill_sb(): deal with allocation failures >> nfsd_umount(): deal with early sget() failures >> nfs: avoid double-free on early sget() failures >> jffs2_kill_sb(): deal with failed allocations >> hypfs_kill_super(): deal with failed allocations >> >> in >> https://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git/log/?h=for-linus >> , >> I think the patch at >> >> WARNING in kill_block_super >> >> https://syzkaller.appspot.com/bug?id=588996a25a2587be2e3a54e8646728fb9cae44e7 >> >> is better. >> > > OK. The patch was sent to linux.git as commit 8e04944f0ea8b838. > > #syz fix: mm,vmscan: Allow preallocating memory for > register_shrinker(). Sigh no fixes tag on the commit you sent to Linus, and no cc'ing of stable. Can you please update the stable folks that 9ee332d99e4d ("sget(): handle failures of register_shrinker()") is fixed with the commit you just sent to Linus? Thank you, Eric
Re: KASAN: use-after-free Read in alloc_pid
Tetsuo Handa writes: > On 2018/04/10 23:33, Tetsuo Handa wrote: >> syzbot wrote: >>> syzbot has found reproducer for the following crash on upstream commit >>> c18bb396d3d261ebbb4efbc05129c5d354c541e4 (Tue Apr 10 00:04:10 2018 +) >>> Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net >>> syzbot dashboard link: >>> https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c >>> >> While we are waiting for >> >> rpc_pipefs: fix double-dput() >> rpc_pipefs: deal with early sget() failures >> kernfs: deal with early sget() failures >> procfs: deal with early sget() failures >> orangefs_kill_sb(): deal with allocation failures >> nfsd_umount(): deal with early sget() failures >> nfs: avoid double-free on early sget() failures >> jffs2_kill_sb(): deal with failed allocations >> hypfs_kill_super(): deal with failed allocations >> >> in >> https://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git/log/?h=for-linus >> , >> I think the patch at >> >> WARNING in kill_block_super >> >> https://syzkaller.appspot.com/bug?id=588996a25a2587be2e3a54e8646728fb9cae44e7 >> >> is better. >> > > OK. The patch was sent to linux.git as commit 8e04944f0ea8b838. > > #syz fix: mm,vmscan: Allow preallocating memory for > register_shrinker(). Sigh no fixes tag on the commit you sent to Linus, and no cc'ing of stable. Can you please update the stable folks that 9ee332d99e4d ("sget(): handle failures of register_shrinker()") is fixed with the commit you just sent to Linus? Thank you, Eric
Re: KASAN: use-after-free Read in alloc_pid
On 2018/04/10 23:33, Tetsuo Handa wrote: > syzbot wrote: >> syzbot has found reproducer for the following crash on upstream commit >> c18bb396d3d261ebbb4efbc05129c5d354c541e4 (Tue Apr 10 00:04:10 2018 +) >> Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net >> syzbot dashboard link: >> https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c >> > While we are waiting for > > rpc_pipefs: fix double-dput() > rpc_pipefs: deal with early sget() failures > kernfs: deal with early sget() failures > procfs: deal with early sget() failures > orangefs_kill_sb(): deal with allocation failures > nfsd_umount(): deal with early sget() failures > nfs: avoid double-free on early sget() failures > jffs2_kill_sb(): deal with failed allocations > hypfs_kill_super(): deal with failed allocations > > in > https://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git/log/?h=for-linus > , > I think the patch at > > WARNING in kill_block_super > > https://syzkaller.appspot.com/bug?id=588996a25a2587be2e3a54e8646728fb9cae44e7 > > is better. > OK. The patch was sent to linux.git as commit 8e04944f0ea8b838. #syz fix: mm,vmscan: Allow preallocating memory for register_shrinker().
Re: KASAN: use-after-free Read in alloc_pid
On 2018/04/10 23:33, Tetsuo Handa wrote: > syzbot wrote: >> syzbot has found reproducer for the following crash on upstream commit >> c18bb396d3d261ebbb4efbc05129c5d354c541e4 (Tue Apr 10 00:04:10 2018 +) >> Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net >> syzbot dashboard link: >> https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c >> > While we are waiting for > > rpc_pipefs: fix double-dput() > rpc_pipefs: deal with early sget() failures > kernfs: deal with early sget() failures > procfs: deal with early sget() failures > orangefs_kill_sb(): deal with allocation failures > nfsd_umount(): deal with early sget() failures > nfs: avoid double-free on early sget() failures > jffs2_kill_sb(): deal with failed allocations > hypfs_kill_super(): deal with failed allocations > > in > https://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git/log/?h=for-linus > , > I think the patch at > > WARNING in kill_block_super > > https://syzkaller.appspot.com/bug?id=588996a25a2587be2e3a54e8646728fb9cae44e7 > > is better. > OK. The patch was sent to linux.git as commit 8e04944f0ea8b838. #syz fix: mm,vmscan: Allow preallocating memory for register_shrinker().
Re: KASAN: use-after-free Read in alloc_pid
syzbot wrote: > syzbot has found reproducer for the following crash on upstream commit > c18bb396d3d261ebbb4efbc05129c5d354c541e4 (Tue Apr 10 00:04:10 2018 +) > Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net > syzbot dashboard link: > https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c > While we are waiting for rpc_pipefs: fix double-dput() rpc_pipefs: deal with early sget() failures kernfs: deal with early sget() failures procfs: deal with early sget() failures orangefs_kill_sb(): deal with allocation failures nfsd_umount(): deal with early sget() failures nfs: avoid double-free on early sget() failures jffs2_kill_sb(): deal with failed allocations hypfs_kill_super(): deal with failed allocations in https://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git/log/?h=for-linus , I think the patch at WARNING in kill_block_super https://syzkaller.appspot.com/bug?id=588996a25a2587be2e3a54e8646728fb9cae44e7 is better.
Re: KASAN: use-after-free Read in alloc_pid
syzbot wrote: > syzbot has found reproducer for the following crash on upstream commit > c18bb396d3d261ebbb4efbc05129c5d354c541e4 (Tue Apr 10 00:04:10 2018 +) > Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net > syzbot dashboard link: > https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c > While we are waiting for rpc_pipefs: fix double-dput() rpc_pipefs: deal with early sget() failures kernfs: deal with early sget() failures procfs: deal with early sget() failures orangefs_kill_sb(): deal with allocation failures nfsd_umount(): deal with early sget() failures nfs: avoid double-free on early sget() failures jffs2_kill_sb(): deal with failed allocations hypfs_kill_super(): deal with failed allocations in https://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git/log/?h=for-linus , I think the patch at WARNING in kill_block_super https://syzkaller.appspot.com/bug?id=588996a25a2587be2e3a54e8646728fb9cae44e7 is better.
Re: KASAN: use-after-free Read in alloc_pid
syzbot has found reproducer for the following crash on upstream commit c18bb396d3d261ebbb4efbc05129c5d354c541e4 (Tue Apr 10 00:04:10 2018 +) Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net syzbot dashboard link: https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c So far this crash happened 7 times on upstream. C reproducer: https://syzkaller.appspot.com/x/repro.c?id=544516125184 syzkaller reproducer: https://syzkaller.appspot.com/x/repro.syz?id=5779667084640256 Raw console output: https://syzkaller.appspot.com/x/log.txt?id=5571293525049344 Kernel config: https://syzkaller.appspot.com/x/.config?id=-771321277174894814 compiler: gcc (GCC) 8.0.1 20180301 (experimental) IMPORTANT: if you fix the bug, please add the following tag to the commit: Reported-by: syzbot+7a1cff37dbbef9e7b...@syzkaller.appspotmail.com It will help syzbot understand when the bug is fixed. Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x1b9/0x294 lib/dump_stack.c:113 == fail_dump lib/fault-inject.c:51 [inline] should_fail.cold.4+0xa/0x1a lib/fault-inject.c:149 BUG: KASAN: use-after-free in alloc_pid+0x9e8/0xa50 kernel/pid.c:236 Read of size 4 at addr 8801ad357898 by task syzkaller392486/4543 __should_failslab+0x124/0x180 mm/failslab.c:32 should_failslab+0x9/0x14 mm/slab_common.c:1522 slab_pre_alloc_hook mm/slab.h:423 [inline] slab_alloc mm/slab.c:3378 [inline] kmem_cache_alloc+0x2af/0x760 mm/slab.c:3552 __d_alloc+0xc1/0xc00 fs/dcache.c:1624 d_alloc+0x8e/0x370 fs/dcache.c:1702 d_alloc_name+0xb3/0x110 fs/dcache.c:1756 proc_setup_self+0xbe/0x375 fs/proc/self.c:43 proc_fill_super+0x24d/0x2f5 fs/proc/inode.c:514 mount_ns+0x12a/0x1d0 fs/super.c:1036 proc_mount+0x73/0xa0 fs/proc/root.c:101 mount_fs+0xae/0x328 fs/super.c:1222 vfs_kern_mount.part.34+0xd4/0x4d0 fs/namespace.c:1037 vfs_kern_mount fs/namespace.c:3303 [inline] kern_mount_data+0x50/0xc0 fs/namespace.c:3303 pid_ns_prepare_proc+0x1e/0x90 fs/proc/root.c:222 alloc_pid+0x8cf/0xa50 kernel/pid.c:208 copy_process.part.38+0x36bf/0x6ee0 kernel/fork.c:1809 copy_process kernel/fork.c:1608 [inline] _do_fork+0x291/0x12a0 kernel/fork.c:2089 SYSC_clone kernel/fork.c:2196 [inline] SyS_clone+0x37/0x50 kernel/fork.c:2190 do_syscall_64+0x29e/0x9d0 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x42/0xb7 RIP: 0033:0x442639 RSP: 002b:7ffd890f8138 EFLAGS: 0206 ORIG_RAX: 0038 RAX: ffda RBX: 0001 RCX: 00442639 RDX: 28c0 RSI: 2800 RDI: 2000c100 RBP: 7ffd890f8250 R08: 2940 R09: 0004 R10: 2900 R11: 0206 R12: R13: R14: 1380 R15: 7ffd890f8278 CPU: 1 PID: 4543 Comm: syzkaller392486 Not tainted 4.16.0+ #17 proc_fill_super: can't allocate /proc/self Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x1b9/0x294 lib/dump_stack.c:113 print_address_description+0x6c/0x20b mm/kasan/report.c:256 kasan_report_error mm/kasan/report.c:354 [inline] kasan_report.cold.7+0xac/0x2f5 mm/kasan/report.c:412 __asan_report_load4_noabort+0x14/0x20 mm/kasan/report.c:432 alloc_pid+0x9e8/0xa50 kernel/pid.c:236 copy_process.part.38+0x36bf/0x6ee0 kernel/fork.c:1809 copy_process kernel/fork.c:1608 [inline] _do_fork+0x291/0x12a0 kernel/fork.c:2089 SYSC_clone kernel/fork.c:2196 [inline] SyS_clone+0x37/0x50 kernel/fork.c:2190 do_syscall_64+0x29e/0x9d0 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x42/0xb7 RIP: 0033:0x442639 RSP: 002b:7ffd890f8138 EFLAGS: 0206 ORIG_RAX: 0038 RAX: ffda RBX: RCX: 00442639 RDX: 28c0 RSI: 2800 RDI: 2000c100 RBP: 7ffd890f8250 R08: 2940 R09: R10: 2900 R11: 0206 R12: R13: R14: R15: 7ffd890f8278 Allocated by task 4543: save_stack+0x43/0xd0 mm/kasan/kasan.c:448 set_track mm/kasan/kasan.c:460 [inline] kasan_kmalloc+0xc4/0xe0 mm/kasan/kasan.c:553 kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:490 kmem_cache_alloc+0x12e/0x760 mm/slab.c:3554 kmem_cache_zalloc include/linux/slab.h:691 [inline] create_pid_namespace kernel/pid_namespace.c:97 [inline] copy_pid_ns+0x2c3/0xb40 kernel/pid_namespace.c:156 create_new_namespaces+0x48a/0x8f0 kernel/nsproxy.c:94 copy_namespaces+0x3f7/0x4c0 kernel/nsproxy.c:165 copy_process.part.38+0x353a/0x6ee0 kernel/fork.c:1798 copy_process kernel/fork.c:1608 [inline] _do_fork+0x291/0x12a0 kernel/fork.c:2089 SYSC_clone kernel/fork.c:2196 [inline] SyS_clone+0x37/0x50 kernel/fork.c:2190 do_syscall_64+0x29e/0x9d0
Re: KASAN: use-after-free Read in alloc_pid
syzbot has found reproducer for the following crash on upstream commit c18bb396d3d261ebbb4efbc05129c5d354c541e4 (Tue Apr 10 00:04:10 2018 +) Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net syzbot dashboard link: https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c So far this crash happened 7 times on upstream. C reproducer: https://syzkaller.appspot.com/x/repro.c?id=544516125184 syzkaller reproducer: https://syzkaller.appspot.com/x/repro.syz?id=5779667084640256 Raw console output: https://syzkaller.appspot.com/x/log.txt?id=5571293525049344 Kernel config: https://syzkaller.appspot.com/x/.config?id=-771321277174894814 compiler: gcc (GCC) 8.0.1 20180301 (experimental) IMPORTANT: if you fix the bug, please add the following tag to the commit: Reported-by: syzbot+7a1cff37dbbef9e7b...@syzkaller.appspotmail.com It will help syzbot understand when the bug is fixed. Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x1b9/0x294 lib/dump_stack.c:113 == fail_dump lib/fault-inject.c:51 [inline] should_fail.cold.4+0xa/0x1a lib/fault-inject.c:149 BUG: KASAN: use-after-free in alloc_pid+0x9e8/0xa50 kernel/pid.c:236 Read of size 4 at addr 8801ad357898 by task syzkaller392486/4543 __should_failslab+0x124/0x180 mm/failslab.c:32 should_failslab+0x9/0x14 mm/slab_common.c:1522 slab_pre_alloc_hook mm/slab.h:423 [inline] slab_alloc mm/slab.c:3378 [inline] kmem_cache_alloc+0x2af/0x760 mm/slab.c:3552 __d_alloc+0xc1/0xc00 fs/dcache.c:1624 d_alloc+0x8e/0x370 fs/dcache.c:1702 d_alloc_name+0xb3/0x110 fs/dcache.c:1756 proc_setup_self+0xbe/0x375 fs/proc/self.c:43 proc_fill_super+0x24d/0x2f5 fs/proc/inode.c:514 mount_ns+0x12a/0x1d0 fs/super.c:1036 proc_mount+0x73/0xa0 fs/proc/root.c:101 mount_fs+0xae/0x328 fs/super.c:1222 vfs_kern_mount.part.34+0xd4/0x4d0 fs/namespace.c:1037 vfs_kern_mount fs/namespace.c:3303 [inline] kern_mount_data+0x50/0xc0 fs/namespace.c:3303 pid_ns_prepare_proc+0x1e/0x90 fs/proc/root.c:222 alloc_pid+0x8cf/0xa50 kernel/pid.c:208 copy_process.part.38+0x36bf/0x6ee0 kernel/fork.c:1809 copy_process kernel/fork.c:1608 [inline] _do_fork+0x291/0x12a0 kernel/fork.c:2089 SYSC_clone kernel/fork.c:2196 [inline] SyS_clone+0x37/0x50 kernel/fork.c:2190 do_syscall_64+0x29e/0x9d0 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x42/0xb7 RIP: 0033:0x442639 RSP: 002b:7ffd890f8138 EFLAGS: 0206 ORIG_RAX: 0038 RAX: ffda RBX: 0001 RCX: 00442639 RDX: 28c0 RSI: 2800 RDI: 2000c100 RBP: 7ffd890f8250 R08: 2940 R09: 0004 R10: 2900 R11: 0206 R12: R13: R14: 1380 R15: 7ffd890f8278 CPU: 1 PID: 4543 Comm: syzkaller392486 Not tainted 4.16.0+ #17 proc_fill_super: can't allocate /proc/self Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x1b9/0x294 lib/dump_stack.c:113 print_address_description+0x6c/0x20b mm/kasan/report.c:256 kasan_report_error mm/kasan/report.c:354 [inline] kasan_report.cold.7+0xac/0x2f5 mm/kasan/report.c:412 __asan_report_load4_noabort+0x14/0x20 mm/kasan/report.c:432 alloc_pid+0x9e8/0xa50 kernel/pid.c:236 copy_process.part.38+0x36bf/0x6ee0 kernel/fork.c:1809 copy_process kernel/fork.c:1608 [inline] _do_fork+0x291/0x12a0 kernel/fork.c:2089 SYSC_clone kernel/fork.c:2196 [inline] SyS_clone+0x37/0x50 kernel/fork.c:2190 do_syscall_64+0x29e/0x9d0 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x42/0xb7 RIP: 0033:0x442639 RSP: 002b:7ffd890f8138 EFLAGS: 0206 ORIG_RAX: 0038 RAX: ffda RBX: RCX: 00442639 RDX: 28c0 RSI: 2800 RDI: 2000c100 RBP: 7ffd890f8250 R08: 2940 R09: R10: 2900 R11: 0206 R12: R13: R14: R15: 7ffd890f8278 Allocated by task 4543: save_stack+0x43/0xd0 mm/kasan/kasan.c:448 set_track mm/kasan/kasan.c:460 [inline] kasan_kmalloc+0xc4/0xe0 mm/kasan/kasan.c:553 kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:490 kmem_cache_alloc+0x12e/0x760 mm/slab.c:3554 kmem_cache_zalloc include/linux/slab.h:691 [inline] create_pid_namespace kernel/pid_namespace.c:97 [inline] copy_pid_ns+0x2c3/0xb40 kernel/pid_namespace.c:156 create_new_namespaces+0x48a/0x8f0 kernel/nsproxy.c:94 copy_namespaces+0x3f7/0x4c0 kernel/nsproxy.c:165 copy_process.part.38+0x353a/0x6ee0 kernel/fork.c:1798 copy_process kernel/fork.c:1608 [inline] _do_fork+0x291/0x12a0 kernel/fork.c:2089 SYSC_clone kernel/fork.c:2196 [inline] SyS_clone+0x37/0x50 kernel/fork.c:2190 do_syscall_64+0x29e/0x9d0
Re: KASAN: use-after-free Read in alloc_pid
On 2018/04/03 12:10, Eric Biggers wrote: > On Mon, Apr 02, 2018 at 06:00:57PM -0500, Eric W. Biederman wrote: >> syzbotwrites: >> >>> Hello, >>> >>> syzbot hit the following crash on upstream commit >>> 9dd2326890d89a5179967c947dab2bab34d7ddee (Fri Mar 30 17:29:47 2018 +) >>> Merge tag 'ceph-for-4.16-rc8' of git://github.com/ceph/ceph-client >>> syzbot dashboard link: >>> https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c >>> >>> So far this crash happened 4 times on upstream. >>> >>> Unfortunately, I don't have any reproducer for this crash yet. >> >> Do you have any of the other traces? This looks like a something is >> calling put_pid_ns more than it is calling get_pid_ns causing a >> reference count mismatch. >> >> If this is not: 9ee332d99e4d5a97548943b81c54668450ce641b Yes, that commit is the trigger. Al wrote patches. Let's check them. http://lkml.kernel.org/r/20180402143415.gc30...@zeniv.linux.org.uk http://lkml.kernel.org/r/20180403052009.gh30...@zeniv.linux.org.uk -- struct pid *alloc_pid(struct pid_namespace *ns) { (...snipped...) if (unlikely(is_child_reaper(pid))) { if (pid_ns_prepare_proc(ns)) // ns is freed upon failure. goto out_free; } (...snipped...) out_free: spin_lock_irq(_lock); while (++i <= ns->level) // <= ns is already freed by destroy_pid_namespace() explained below. idr_remove(>idr, (pid->numbers + i)->nr); (...snipped...) } -- -- int pid_ns_prepare_proc(struct pid_namespace *ns) { mnt = kern_mount_data(_fs_type, ns) { // <= ns is passed as ns. mnt = vfs_kern_mount(type, SB_KERNMOUNT, type->name, data) { // <= ns is passed as data. root = mount_fs(type, SB_KERNMOUNT, name, data) { // <= ns is passed as data. root = type->mount(type, SB_KERNMOUNT, name, data) = // <= ns is passed as data. static struct dentry *proc_mount(struct file_system_type *fs_type, int flags, const char *dev_name, void *data) { return mount_ns(fs_type, SB_KERNMOUNT, NULL, ns, ns->user_ns, proc_fill_super) { // <= ns is passed as ns. sb = sget_userns(fs_type, ns_test_super, ns_set_super, SB_KERNMOUNT, user_ns, ns) { // <= ns is passed as ns. err = set(s, data) = // <= ns is passed as data. static int ns_set_super(struct super_block *sb, void *data) { sb->s_fs_info = data; // ns is associated here. } err = register_shrinker(>s_shrink); // <= fail by fault injection. deactivate_locked_super(s) { fs->kill_sb(s) = static void proc_kill_sb(struct super_block *sb) { ns = (struct pid_namespace *)sb->s_fs_info; put_pid_ns(ns) { // <= ns is passed as ns kref_put(>kref, free_pid_ns) { // <= ns refcount becomes 0 destroy_pid_namespace(ns) { call_rcu(>rcu, delayed_free_pidns) { kmem_cache_free(pid_ns_cachep, ns); // <= ns is released here after RCU grace period } } } } } } } } } } } } } -- >> >> I could use a few more hints to help narrow down what is going wrong. >> >> It would be nice to know what the other 3 crashes looked like and >> exactly which upstream they were on. >> > > The other crashes are shown on the syzbot dashboard (link was given in the > original email). > > Eric >
Re: KASAN: use-after-free Read in alloc_pid
On 2018/04/03 12:10, Eric Biggers wrote: > On Mon, Apr 02, 2018 at 06:00:57PM -0500, Eric W. Biederman wrote: >> syzbot writes: >> >>> Hello, >>> >>> syzbot hit the following crash on upstream commit >>> 9dd2326890d89a5179967c947dab2bab34d7ddee (Fri Mar 30 17:29:47 2018 +) >>> Merge tag 'ceph-for-4.16-rc8' of git://github.com/ceph/ceph-client >>> syzbot dashboard link: >>> https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c >>> >>> So far this crash happened 4 times on upstream. >>> >>> Unfortunately, I don't have any reproducer for this crash yet. >> >> Do you have any of the other traces? This looks like a something is >> calling put_pid_ns more than it is calling get_pid_ns causing a >> reference count mismatch. >> >> If this is not: 9ee332d99e4d5a97548943b81c54668450ce641b Yes, that commit is the trigger. Al wrote patches. Let's check them. http://lkml.kernel.org/r/20180402143415.gc30...@zeniv.linux.org.uk http://lkml.kernel.org/r/20180403052009.gh30...@zeniv.linux.org.uk -- struct pid *alloc_pid(struct pid_namespace *ns) { (...snipped...) if (unlikely(is_child_reaper(pid))) { if (pid_ns_prepare_proc(ns)) // ns is freed upon failure. goto out_free; } (...snipped...) out_free: spin_lock_irq(_lock); while (++i <= ns->level) // <= ns is already freed by destroy_pid_namespace() explained below. idr_remove(>idr, (pid->numbers + i)->nr); (...snipped...) } -- -- int pid_ns_prepare_proc(struct pid_namespace *ns) { mnt = kern_mount_data(_fs_type, ns) { // <= ns is passed as ns. mnt = vfs_kern_mount(type, SB_KERNMOUNT, type->name, data) { // <= ns is passed as data. root = mount_fs(type, SB_KERNMOUNT, name, data) { // <= ns is passed as data. root = type->mount(type, SB_KERNMOUNT, name, data) = // <= ns is passed as data. static struct dentry *proc_mount(struct file_system_type *fs_type, int flags, const char *dev_name, void *data) { return mount_ns(fs_type, SB_KERNMOUNT, NULL, ns, ns->user_ns, proc_fill_super) { // <= ns is passed as ns. sb = sget_userns(fs_type, ns_test_super, ns_set_super, SB_KERNMOUNT, user_ns, ns) { // <= ns is passed as ns. err = set(s, data) = // <= ns is passed as data. static int ns_set_super(struct super_block *sb, void *data) { sb->s_fs_info = data; // ns is associated here. } err = register_shrinker(>s_shrink); // <= fail by fault injection. deactivate_locked_super(s) { fs->kill_sb(s) = static void proc_kill_sb(struct super_block *sb) { ns = (struct pid_namespace *)sb->s_fs_info; put_pid_ns(ns) { // <= ns is passed as ns kref_put(>kref, free_pid_ns) { // <= ns refcount becomes 0 destroy_pid_namespace(ns) { call_rcu(>rcu, delayed_free_pidns) { kmem_cache_free(pid_ns_cachep, ns); // <= ns is released here after RCU grace period } } } } } } } } } } } } } -- >> >> I could use a few more hints to help narrow down what is going wrong. >> >> It would be nice to know what the other 3 crashes looked like and >> exactly which upstream they were on. >> > > The other crashes are shown on the syzbot dashboard (link was given in the > original email). > > Eric >
Re: KASAN: use-after-free Read in alloc_pid
On Mon, Apr 02, 2018 at 06:00:57PM -0500, Eric W. Biederman wrote: > syzbotwrites: > > > Hello, > > > > syzbot hit the following crash on upstream commit > > 9dd2326890d89a5179967c947dab2bab34d7ddee (Fri Mar 30 17:29:47 2018 +) > > Merge tag 'ceph-for-4.16-rc8' of git://github.com/ceph/ceph-client > > syzbot dashboard link: > > https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c > > > > So far this crash happened 4 times on upstream. > > > > Unfortunately, I don't have any reproducer for this crash yet. > > Do you have any of the other traces? This looks like a something is > calling put_pid_ns more than it is calling get_pid_ns causing a > reference count mismatch. > > If this is not: 9ee332d99e4d5a97548943b81c54668450ce641b > > I could use a few more hints to help narrow down what is going wrong. > > It would be nice to know what the other 3 crashes looked like and > exactly which upstream they were on. > The other crashes are shown on the syzbot dashboard (link was given in the original email). Eric
Re: KASAN: use-after-free Read in alloc_pid
On Mon, Apr 02, 2018 at 06:00:57PM -0500, Eric W. Biederman wrote: > syzbot writes: > > > Hello, > > > > syzbot hit the following crash on upstream commit > > 9dd2326890d89a5179967c947dab2bab34d7ddee (Fri Mar 30 17:29:47 2018 +) > > Merge tag 'ceph-for-4.16-rc8' of git://github.com/ceph/ceph-client > > syzbot dashboard link: > > https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c > > > > So far this crash happened 4 times on upstream. > > > > Unfortunately, I don't have any reproducer for this crash yet. > > Do you have any of the other traces? This looks like a something is > calling put_pid_ns more than it is calling get_pid_ns causing a > reference count mismatch. > > If this is not: 9ee332d99e4d5a97548943b81c54668450ce641b > > I could use a few more hints to help narrow down what is going wrong. > > It would be nice to know what the other 3 crashes looked like and > exactly which upstream they were on. > The other crashes are shown on the syzbot dashboard (link was given in the original email). Eric
Re: KASAN: use-after-free Read in alloc_pid
syzbotwrites: > Hello, > > syzbot hit the following crash on upstream commit > 9dd2326890d89a5179967c947dab2bab34d7ddee (Fri Mar 30 17:29:47 2018 +) > Merge tag 'ceph-for-4.16-rc8' of git://github.com/ceph/ceph-client > syzbot dashboard link: > https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c > > So far this crash happened 4 times on upstream. > > Unfortunately, I don't have any reproducer for this crash yet. Do you have any of the other traces? This looks like a something is calling put_pid_ns more than it is calling get_pid_ns causing a reference count mismatch. If this is not: 9ee332d99e4d5a97548943b81c54668450ce641b I could use a few more hints to help narrow down what is going wrong. It would be nice to know what the other 3 crashes looked like and exactly which upstream they were on. Eric
Re: KASAN: use-after-free Read in alloc_pid
syzbot writes: > Hello, > > syzbot hit the following crash on upstream commit > 9dd2326890d89a5179967c947dab2bab34d7ddee (Fri Mar 30 17:29:47 2018 +) > Merge tag 'ceph-for-4.16-rc8' of git://github.com/ceph/ceph-client > syzbot dashboard link: > https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c > > So far this crash happened 4 times on upstream. > > Unfortunately, I don't have any reproducer for this crash yet. Do you have any of the other traces? This looks like a something is calling put_pid_ns more than it is calling get_pid_ns causing a reference count mismatch. If this is not: 9ee332d99e4d5a97548943b81c54668450ce641b I could use a few more hints to help narrow down what is going wrong. It would be nice to know what the other 3 crashes looked like and exactly which upstream they were on. Eric
KASAN: use-after-free Read in alloc_pid
Hello, syzbot hit the following crash on upstream commit 9dd2326890d89a5179967c947dab2bab34d7ddee (Fri Mar 30 17:29:47 2018 +) Merge tag 'ceph-for-4.16-rc8' of git://github.com/ceph/ceph-client syzbot dashboard link: https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c So far this crash happened 4 times on upstream. Unfortunately, I don't have any reproducer for this crash yet. Raw console output: https://syzkaller.appspot.com/x/log.txt?id=6588786309857280 Kernel config: https://syzkaller.appspot.com/x/.config?id=-2760467897697295172 compiler: gcc (GCC) 7.1.1 20170620 IMPORTANT: if you fix the bug, please add the following tag to the commit: Reported-by: syzbot+7a1cff37dbbef9e7b...@syzkaller.appspotmail.com It will help syzbot understand when the bug is fixed. See footer for details. If you forward the report, please keep this part and the footer. R10: 2900 R11: 0246 R12: 0015 R13: 0051 R14: 006f2838 R15: 0028 == BUG: KASAN: use-after-free in alloc_pid+0x9a6/0xa00 kernel/pid.c:236 Read of size 4 at addr 8801b56d0088 by task syz-executor4/3533 CPU: 1 PID: 3533 Comm: syz-executor4 Not tainted 4.16.0-rc7+ #372 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:17 [inline] dump_stack+0x194/0x24d lib/dump_stack.c:53 print_address_description+0x73/0x250 mm/kasan/report.c:256 kasan_report_error mm/kasan/report.c:354 [inline] kasan_report+0x23c/0x360 mm/kasan/report.c:412 __asan_report_load4_noabort+0x14/0x20 mm/kasan/report.c:432 alloc_pid+0x9a6/0xa00 kernel/pid.c:236 copy_process.part.38+0x2516/0x4bd0 kernel/fork.c:1807 copy_process kernel/fork.c:1606 [inline] _do_fork+0x1f7/0xf70 kernel/fork.c:2087 SYSC_clone kernel/fork.c:2194 [inline] SyS_clone+0x37/0x50 kernel/fork.c:2188 do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x42/0xb7 RIP: 0033:0x454e79 RSP: 002b:7f6165b9ac68 EFLAGS: 0246 ORIG_RAX: 0038 RAX: ffda RBX: 7f6165b9b6d4 RCX: 00454e79 RDX: 28c0 RSI: 2800 RDI: 2000c100 RBP: 0072bea0 R08: 2940 R09: R10: 2900 R11: 0246 R12: 0015 R13: 0051 R14: 006f2838 R15: 0028 Allocated by task 3533: save_stack+0x43/0xd0 mm/kasan/kasan.c:447 set_track mm/kasan/kasan.c:459 [inline] kasan_kmalloc+0xad/0xe0 mm/kasan/kasan.c:552 kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:489 kmem_cache_alloc+0x12e/0x760 mm/slab.c:3542 kmem_cache_zalloc include/linux/slab.h:691 [inline] create_pid_namespace kernel/pid_namespace.c:116 [inline] copy_pid_ns+0x257/0xc60 kernel/pid_namespace.c:175 create_new_namespaces+0x307/0x880 kernel/nsproxy.c:94 copy_namespaces+0x340/0x400 kernel/nsproxy.c:165 copy_process.part.38+0x2380/0x4bd0 kernel/fork.c:1796 copy_process kernel/fork.c:1606 [inline] _do_fork+0x1f7/0xf70 kernel/fork.c:2087 SYSC_clone kernel/fork.c:2194 [inline] SyS_clone+0x37/0x50 kernel/fork.c:2188 do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x42/0xb7 Freed by task 3539: save_stack+0x43/0xd0 mm/kasan/kasan.c:447 set_track mm/kasan/kasan.c:459 [inline] __kasan_slab_free+0x11a/0x170 mm/kasan/kasan.c:520 kasan_slab_free+0xe/0x10 mm/kasan/kasan.c:527 __cache_free mm/slab.c:3486 [inline] kmem_cache_free+0x83/0x2a0 mm/slab.c:3744 delayed_free_pidns+0x82/0xb0 kernel/pid_namespace.c:157 __rcu_reclaim kernel/rcu/rcu.h:172 [inline] rcu_do_batch kernel/rcu/tree.c:2674 [inline] invoke_rcu_callbacks kernel/rcu/tree.c:2933 [inline] __rcu_process_callbacks kernel/rcu/tree.c:2900 [inline] rcu_process_callbacks+0xd6c/0x17f0 kernel/rcu/tree.c:2917 __do_softirq+0x2d7/0xb85 kernel/softirq.c:285 The buggy address belongs to the object at 8801b56d0040 which belongs to the cache pid_namespace of size 240 The buggy address is located 72 bytes inside of 240-byte region [8801b56d0040, 8801b56d0130) The buggy address belongs to the page: page:ea0006d5b400 count:1 mapcount:0 mapping:8801b56d0040 index:0x0 flags: 0x2fffc000100(slab) raw: 02fffc000100 8801b56d0040 0001000d raw: ea00074458a0 ea00073389e0 8801d40b2c00 page dumped because: kasan: bad access detected Memory state around the buggy address: 8801b56cff80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc 8801b56d: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb 8801b56d0080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ 8801b56d0100: fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb fb 8801b56d0180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb == --- This bug
KASAN: use-after-free Read in alloc_pid
Hello, syzbot hit the following crash on upstream commit 9dd2326890d89a5179967c947dab2bab34d7ddee (Fri Mar 30 17:29:47 2018 +) Merge tag 'ceph-for-4.16-rc8' of git://github.com/ceph/ceph-client syzbot dashboard link: https://syzkaller.appspot.com/bug?extid=7a1cff37dbbef9e7ba4c So far this crash happened 4 times on upstream. Unfortunately, I don't have any reproducer for this crash yet. Raw console output: https://syzkaller.appspot.com/x/log.txt?id=6588786309857280 Kernel config: https://syzkaller.appspot.com/x/.config?id=-2760467897697295172 compiler: gcc (GCC) 7.1.1 20170620 IMPORTANT: if you fix the bug, please add the following tag to the commit: Reported-by: syzbot+7a1cff37dbbef9e7b...@syzkaller.appspotmail.com It will help syzbot understand when the bug is fixed. See footer for details. If you forward the report, please keep this part and the footer. R10: 2900 R11: 0246 R12: 0015 R13: 0051 R14: 006f2838 R15: 0028 == BUG: KASAN: use-after-free in alloc_pid+0x9a6/0xa00 kernel/pid.c:236 Read of size 4 at addr 8801b56d0088 by task syz-executor4/3533 CPU: 1 PID: 3533 Comm: syz-executor4 Not tainted 4.16.0-rc7+ #372 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:17 [inline] dump_stack+0x194/0x24d lib/dump_stack.c:53 print_address_description+0x73/0x250 mm/kasan/report.c:256 kasan_report_error mm/kasan/report.c:354 [inline] kasan_report+0x23c/0x360 mm/kasan/report.c:412 __asan_report_load4_noabort+0x14/0x20 mm/kasan/report.c:432 alloc_pid+0x9a6/0xa00 kernel/pid.c:236 copy_process.part.38+0x2516/0x4bd0 kernel/fork.c:1807 copy_process kernel/fork.c:1606 [inline] _do_fork+0x1f7/0xf70 kernel/fork.c:2087 SYSC_clone kernel/fork.c:2194 [inline] SyS_clone+0x37/0x50 kernel/fork.c:2188 do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x42/0xb7 RIP: 0033:0x454e79 RSP: 002b:7f6165b9ac68 EFLAGS: 0246 ORIG_RAX: 0038 RAX: ffda RBX: 7f6165b9b6d4 RCX: 00454e79 RDX: 28c0 RSI: 2800 RDI: 2000c100 RBP: 0072bea0 R08: 2940 R09: R10: 2900 R11: 0246 R12: 0015 R13: 0051 R14: 006f2838 R15: 0028 Allocated by task 3533: save_stack+0x43/0xd0 mm/kasan/kasan.c:447 set_track mm/kasan/kasan.c:459 [inline] kasan_kmalloc+0xad/0xe0 mm/kasan/kasan.c:552 kasan_slab_alloc+0x12/0x20 mm/kasan/kasan.c:489 kmem_cache_alloc+0x12e/0x760 mm/slab.c:3542 kmem_cache_zalloc include/linux/slab.h:691 [inline] create_pid_namespace kernel/pid_namespace.c:116 [inline] copy_pid_ns+0x257/0xc60 kernel/pid_namespace.c:175 create_new_namespaces+0x307/0x880 kernel/nsproxy.c:94 copy_namespaces+0x340/0x400 kernel/nsproxy.c:165 copy_process.part.38+0x2380/0x4bd0 kernel/fork.c:1796 copy_process kernel/fork.c:1606 [inline] _do_fork+0x1f7/0xf70 kernel/fork.c:2087 SYSC_clone kernel/fork.c:2194 [inline] SyS_clone+0x37/0x50 kernel/fork.c:2188 do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x42/0xb7 Freed by task 3539: save_stack+0x43/0xd0 mm/kasan/kasan.c:447 set_track mm/kasan/kasan.c:459 [inline] __kasan_slab_free+0x11a/0x170 mm/kasan/kasan.c:520 kasan_slab_free+0xe/0x10 mm/kasan/kasan.c:527 __cache_free mm/slab.c:3486 [inline] kmem_cache_free+0x83/0x2a0 mm/slab.c:3744 delayed_free_pidns+0x82/0xb0 kernel/pid_namespace.c:157 __rcu_reclaim kernel/rcu/rcu.h:172 [inline] rcu_do_batch kernel/rcu/tree.c:2674 [inline] invoke_rcu_callbacks kernel/rcu/tree.c:2933 [inline] __rcu_process_callbacks kernel/rcu/tree.c:2900 [inline] rcu_process_callbacks+0xd6c/0x17f0 kernel/rcu/tree.c:2917 __do_softirq+0x2d7/0xb85 kernel/softirq.c:285 The buggy address belongs to the object at 8801b56d0040 which belongs to the cache pid_namespace of size 240 The buggy address is located 72 bytes inside of 240-byte region [8801b56d0040, 8801b56d0130) The buggy address belongs to the page: page:ea0006d5b400 count:1 mapcount:0 mapping:8801b56d0040 index:0x0 flags: 0x2fffc000100(slab) raw: 02fffc000100 8801b56d0040 0001000d raw: ea00074458a0 ea00073389e0 8801d40b2c00 page dumped because: kasan: bad access detected Memory state around the buggy address: 8801b56cff80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc 8801b56d: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb 8801b56d0080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ 8801b56d0100: fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb fb 8801b56d0180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb == --- This bug