Re: KASAN: use-after-free Read in vhost_chr_write_iter
On Mon, May 21, 2018 at 10:38:10AM +0800, Jason Wang wrote:
>
>
> On 2018年05月18日 17:24, Jason Wang wrote:
> >
> >
> > On 2018年05月17日 21:45, DaeRyong Jeong wrote:
> > > We report the crash: KASAN: use-after-free Read in vhost_chr_write_iter
> > >
> > > This crash has been found in v4.17-rc1 using RaceFuzzer (a modified
> > > version of Syzkaller), which we describe more at the end of this
> > > report. Our analysis shows that the race occurs when invoking two
> > > syscalls concurrently, write$vnet and ioctl$VHOST_RESET_OWNER.
> > >
> > >
> > > Analysis:
> > > We think the concurrent execution of vhost_process_iotlb_msg() and
> > > vhost_dev_cleanup() causes the crash.
> > > Both of functions can run concurrently (please see call sequence below),
> > > and possibly, there is a race on dev->iotlb.
> > > If the switch occurs right after vhost_dev_cleanup() frees
> > > dev->iotlb, vhost_process_iotlb_msg() still sees the non-null value
> > > and it
> > > keep executing without returning -EFAULT. Consequently, use-after-free
> > > occures
> > >
> > >
> > > Thread interleaving:
> > > CPU0 (vhost_process_iotlb_msg) CPU1 (vhost_dev_cleanup)
> > > (In the case of both VHOST_IOTLB_UPDATE and
> > > VHOST_IOTLB_INVALIDATE)
> > > = =
> > > vhost_umem_clean(dev->iotlb);
> > > if (!dev->iotlb) {
> > > ret = -EFAULT;
> > > break;
> > > }
> > > dev->iotlb = NULL;
> > >
> > >
> > > Call Sequence:
> > > CPU0
> > > =
> > > vhost_net_chr_write_iter
> > > vhost_chr_write_iter
> > > vhost_process_iotlb_msg
> > >
> > > CPU1
> > > =
> > > vhost_net_ioctl
> > > vhost_net_reset_owner
> > > vhost_dev_reset_owner
> > > vhost_dev_cleanup
> >
> > Thanks a lot for the analysis.
> >
> > This could be addressed by simply protect it with dev mutex.
> >
> > Will post a patch.
> >
>
> Could you please help to test the attached patch? I've done some smoking
> test.
>
> Thanks
Sorry to say this, but we don't have a reproducer for this bug since our
reproducer is being implemented.
This crash had occrued a few times in our fuzzer, so I inspected the code
manually.
It seems the patch is good for me, but we can't test the patch for now.
Sorry.
> From 88328386f3f652e684ee33dc4cf63dcaed871aea Mon Sep 17 00:00:00 2001
> From: Jason Wang
> Date: Fri, 18 May 2018 17:33:27 +0800
> Subject: [PATCH] vhost: synchronize IOTLB message with dev cleanup
>
> DaeRyong Jeong reports a race between vhost_dev_cleanup() and
> vhost_process_iotlb_msg():
>
> Thread interleaving:
> CPU0 (vhost_process_iotlb_msg)CPU1 (vhost_dev_cleanup)
> (In the case of both VHOST_IOTLB_UPDATE and
> VHOST_IOTLB_INVALIDATE)
> = =
> vhost_umem_clean(dev->iotlb);
> if (!dev->iotlb) {
> ret = -EFAULT;
> break;
> }
> dev->iotlb = NULL;
>
> The reason is we don't synchronize between them, fixing by protecting
> vhost_process_iotlb_msg() with dev mutex.
>
> Reported-by: DaeRyong Jeong
> Fixes: 6b1e6cc7855b0 ("vhost: new device IOTLB API")
> Reported-by: DaeRyong Jeong
> ---
> drivers/vhost/vhost.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
> index f3bd8e9..f0be5f3 100644
> --- a/drivers/vhost/vhost.c
> +++ b/drivers/vhost/vhost.c
> @@ -981,6 +981,7 @@ static int vhost_process_iotlb_msg(struct vhost_dev *dev,
> {
> int ret = 0;
>
> + mutex_lock(&dev->mutex);
> vhost_dev_lock_vqs(dev);
> switch (msg->type) {
> case VHOST_IOTLB_UPDATE:
> @@ -1016,6 +1017,8 @@ static int vhost_process_iotlb_msg(struct vhost_dev
> *dev,
> }
>
> vhost_dev_unlock_vqs(dev);
> + mutex_unlock(&dev->mutex);
> +
> return ret;
> }
> ssize_t vhost_chr_write_iter(struct vhost_dev *dev,
> --
> 2.7.4
>
WARNING in ip_recv_error
We report the crash: WARNING in ip_recv_error (I resend the email since I mistakenly missed the subject in my previous email. I'm sorry.) This crash has been found in v4.17-rc1 using RaceFuzzer (a modified version of Syzkaller), which we describe more at the end of this report. Our analysis shows that the race occurs when invoking two syscalls concurrently, setsockopt$inet6_IPV6_ADDRFORM and recvmsg. Diagnosis: We think the concurrent execution of do_ipv6_setsockopt() with optname IPV6_ADDRFORM and inet_recvmsg() causes the crash. do_ipv6_setsockopt() can update sk->prot to &udp_prot and sk->sk_family to PF_INET. But inet_recvmsg() can execute sk->sk_prot->recvmsg() right after that sk->prot is updated and sk->sk_family is not updated by do_ipv6_setsockopt(). This will lead WARN_ON in ip_recv_error(). Thread interleaving: CPU0 (do_ipv6_setsockopt) CPU1 (inet_recvmsg) = = struct proto *prot = &udp_prot; ... sk->sk_prot = prot; sk->sk_socket->ops = &inet_dgram_ops; err = sk->sk_prot->recvmsg(sk, msg, size, flags & MSG_DONTWAIT, flags & ~MSG_DONTWAIT, &addr_len); (in udp_recvmsg) if (flags & MSG_ERRQUEUE) return ip_recv_error(sk, msg, len, addr_len); (in ip_recv_error) WARN_ON_ONCE(sk->sk_family == AF_INET6); sk->sk_family = PF_INET; Call Sequence: CPU0 = udpv6_setsockopt ipv6_setsockopt do_ipv6_setsockopt CPU1 = sock_recvmsg sock_recvmsg_nosec inet_recvmsg udp_recvmsg == WARNING: CPU: 1 PID: 32600 at /home/daeryong/workspace/new-race-fuzzer/kernels_repo/kernel_v4.17-rc1/net/ipv4/ip_sockglue.c:508 ip_recv_error+0x6f2/0x720 net/ipv4/ip_sockglue.c:508 Kernel panic - not syncing: panic_on_warn set ... CPU: 1 PID: 32600 Comm: syz-executor0 Not tainted 4.17.0-rc1 #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x166/0x21c lib/dump_stack.c:113 panic+0x1a0/0x3a7 kernel/panic.c:184 __warn+0x191/0x1a0 kernel/panic.c:536 report_bug+0x132/0x1b0 lib/bug.c:186 fixup_bug.part.11+0x28/0x50 arch/x86/kernel/traps.c:178 fixup_bug arch/x86/kernel/traps.c:247 [inline] do_error_trap+0x28b/0x2d0 arch/x86/kernel/traps.c:296 do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:315 invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:992 RIP: 0010:ip_recv_error+0x6f2/0x720 net/ipv4/ip_sockglue.c:508 RSP: 0018:8801dadff630 EFLAGS: 00010212 RAX: 0004 RBX: 2002 RCX: 8327de12 RDX: 008a RSI: c90001a0c000 RDI: 8801be615010 RBP: 8801dadff720 R08: 2002 R09: 8801dadff918 R10: 8801dadff738 R11: 8801dadffaff R12: 8801be615000 R13: 8801dadffd50 R14: 11003b5bfece R15: 8801dadffb90 udp_recvmsg+0x834/0xa10 net/ipv4/udp.c:1571 inet_recvmsg+0x121/0x420 net/ipv4/af_inet.c:830 sock_recvmsg_nosec net/socket.c:802 [inline] sock_recvmsg+0x7f/0xa0 net/socket.c:809 ___sys_recvmsg+0x1f0/0x430 net/socket.c:2279 __sys_recvmsg+0xfc/0x1c0 net/socket.c:2328 __do_sys_recvmsg net/socket.c:2338 [inline] __se_sys_recvmsg net/socket.c:2335 [inline] __x64_sys_recvmsg+0x48/0x50 net/socket.c:2335 do_syscall_64+0x15f/0x4a0 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x4563f9 RSP: 002b:7f24f6927b28 EFLAGS: 0246 ORIG_RAX: 002f RAX: ffda RBX: 0072bfa0 RCX: 004563f9 RDX: 2002 RSI: 2240 RDI: 0016 RBP: 04e4 R08: R09: R10: R11: 0246 R12: 7f24f69286d4 R13: R14: 006fc600 R15: Dumping ftrace buffer: (ftrace buffer empty) Kernel Offset: disabled Rebooting in 86400 seconds.. == = About RaceFuzzer RaceFuzzer is a customized version of Syzkaller, specifically tailored to find race condition bugs in the Linux kernel. While we leverage many different technique, the notable feature of RaceFuzzer is in leveraging a custom hypervisor (QEMU/KVM) to interleave the scheduling. In particular, we modified the hypervisor to intentionally stall a per-core execution, which is similar to supporting per-core breakpoint functionality. This allows RaceFuzzer to force the kernel to deterministically trigger racy condition (which may rarely happen in practice du
[no subject]
Bcc: Subject: WARNING in ip_recv_error Reply-To: We report the crash: WARNING in ip_recv_error This crash has been found in v4.17-rc1 using RaceFuzzer (a modified version of Syzkaller), which we describe more at the end of this report. Our analysis shows that the race occurs when invoking two syscalls concurrently, do_ipv6_setsockopt and inet_recvmsg. Diagnosis: We think the concurrent execution of do_ipv6_setsockopt() with optname IPV6_ADDRFORM and inet_recvmsg() causes the crash. do_ipv6_setsockopt() can update sk->prot to &udp_prot and sk->sk_family to PF_INET. But inet_recvmsg() can execute sk->sk_prot->recvmsg() right after that sk->prot is updated and sk->sk_family is not updated by do_ipv6_setsockopt(). This will lead WARN_ON in ip_recv_error(). Thread interleaving: CPU0 (do_ipv6_setsockopt) CPU1 (inet_recvmsg) = = struct proto *prot = &udp_prot; ... sk->sk_prot = prot; sk->sk_socket->ops = &inet_dgram_ops; err = sk->sk_prot->recvmsg(sk, msg, size, flags & MSG_DONTWAIT, flags & ~MSG_DONTWAIT, &addr_len); (in udp_recvmsg) if (flags & MSG_ERRQUEUE) return ip_recv_error(sk, msg, len, addr_len); (in ip_recv_error) WARN_ON_ONCE(sk->sk_family == AF_INET6); sk->sk_family = PF_INET; Call Sequence: CPU0 = udpv6_setsockopt ipv6_setsockopt do_ipv6_setsockopt CPU1 = sock_recvmsg sock_recvmsg_nosec inet_recvmsg udp_recvmsg == WARNING: CPU: 1 PID: 32600 at /home/daeryong/workspace/new-race-fuzzer/kernels_repo/kernel_v4.17-rc1/net/ipv4/ip_sockglue.c:508 ip_recv_error+0x6f2/0x720 net/ipv4/ip_sockglue.c:508 Kernel panic - not syncing: panic_on_warn set ... CPU: 1 PID: 32600 Comm: syz-executor0 Not tainted 4.17.0-rc1 #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x166/0x21c lib/dump_stack.c:113 panic+0x1a0/0x3a7 kernel/panic.c:184 __warn+0x191/0x1a0 kernel/panic.c:536 report_bug+0x132/0x1b0 lib/bug.c:186 fixup_bug.part.11+0x28/0x50 arch/x86/kernel/traps.c:178 fixup_bug arch/x86/kernel/traps.c:247 [inline] do_error_trap+0x28b/0x2d0 arch/x86/kernel/traps.c:296 do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:315 invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:992 RIP: 0010:ip_recv_error+0x6f2/0x720 net/ipv4/ip_sockglue.c:508 RSP: 0018:8801dadff630 EFLAGS: 00010212 RAX: 0004 RBX: 2002 RCX: 8327de12 RDX: 008a RSI: c90001a0c000 RDI: 8801be615010 RBP: 8801dadff720 R08: 2002 R09: 8801dadff918 R10: 8801dadff738 R11: 8801dadffaff R12: 8801be615000 R13: 8801dadffd50 R14: 11003b5bfece R15: 8801dadffb90 udp_recvmsg+0x834/0xa10 net/ipv4/udp.c:1571 inet_recvmsg+0x121/0x420 net/ipv4/af_inet.c:830 sock_recvmsg_nosec net/socket.c:802 [inline] sock_recvmsg+0x7f/0xa0 net/socket.c:809 ___sys_recvmsg+0x1f0/0x430 net/socket.c:2279 __sys_recvmsg+0xfc/0x1c0 net/socket.c:2328 __do_sys_recvmsg net/socket.c:2338 [inline] __se_sys_recvmsg net/socket.c:2335 [inline] __x64_sys_recvmsg+0x48/0x50 net/socket.c:2335 do_syscall_64+0x15f/0x4a0 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x4563f9 RSP: 002b:7f24f6927b28 EFLAGS: 0246 ORIG_RAX: 002f RAX: ffda RBX: 0072bfa0 RCX: 004563f9 RDX: 2002 RSI: 2240 RDI: 0016 RBP: 04e4 R08: R09: R10: R11: 0246 R12: 7f24f69286d4 R13: R14: 006fc600 R15: Dumping ftrace buffer: (ftrace buffer empty) Kernel Offset: disabled Rebooting in 86400 seconds.. == = About RaceFuzzer RaceFuzzer is a customized version of Syzkaller, specifically tailored to find race condition bugs in the Linux kernel. While we leverage many different technique, the notable feature of RaceFuzzer is in leveraging a custom hypervisor (QEMU/KVM) to interleave the scheduling. In particular, we modified the hypervisor to intentionally stall a per-core execution, which is similar to supporting per-core breakpoint functionality. This allows RaceFuzzer to force the kernel to deterministically trigger racy condition (which may rarely happen in practice due to randomness in scheduling). RaceFuzzer's C
WARNING in __static_key_slow_dec
We report the crash: WARNING in __static_key_slow_dec
This crash has been found in v4.8 using RaceFuzzer (a modified
version of Syzkaller), which we describe more at the end of this
report.
Even though v4.8 is the relatively old version, we did manual verification
and we think the bug still exists.
Our analysis shows that the race occurs when invoking two syscalls
concurrently, setsockopt() with optname SO_TIMESTAMPING and ioctl() with
cmd SIOCGSTAMPNS.
Diagnosis:
We think if timestamp was previously enabled with
SOCK_TIMESTAMPING_RX_SOFTWARE flag, the concurrent execution of
sock_disable_timestamp() and sock_enable_timestamp() causes the crash.
Thread interleaving:
(Assume sk->flag has the SOCK_TIMESTAMPING_RX_SOFTWARE flag by the
previous setsockopt() call with SO_TIMESTAMPING)
CPU0 (sock_disable_timestamp()) CPU1 (sock_enable_timestamp())
= =
(flag == 1UL << SOCK_TIMESTAMPING_RX_SOFTWARE) (flag == SOCK_TIMESTAMP)
if (!sock_flag(sk, flag)) {
unsigned long
previous_flags = sk->sk_flags;
if (sk->sk_flags & flags) {
sk->sk_flags &= ~flags;
if (sock_needs_netstamp(sk) &&
!(sk->sk_flags & SK_FLAGS_TIMESTAMP))
net_disable_timestamp();
sock_set_flag(sk, flag);
if
(sock_needs_netstamp(sk) &&
!(previous_flags &
SK_FLAGS_TIMESTAMP))
net_enable_timestamp();
/* Here,
net_enable_timestamp() is not called because
* previous_flags has
the SOCK_TIMESTAMPING_RX_SOFTWARE
* flag
*/
/* After the race, sk->sk has the flag SOCK_TIMESTAMP, but
* net_enable_timestamp() is not called one more time.
* Consequently, when the socket is closed, __sk_destruct()
* calls net_disable_timestamp() that leads WARNING.
*/
Call Sequence:
CPU0
=
sock_setsockopt (SO_TIMESTAMPING)
sock_disable_timestamp
CPU1
=
inet_ioctl (SIOCGSTAMPNS)
sock_get_timestampns
sock_enable_timestamp
==
[ cut here ]
WARNING: CPU: 1 PID: 14465 at
/home/blee/project/race-fuzzer/kernels/kernel_v4.8/kernel/jump_label.c:151
__static_key_slow_dec+0xc7/0xd0 kernel/jump_label.c:150
jump label: negative count!
Kernel panic - not syncing: panic_on_warn set ...
CPU: 1 PID: 14465 Comm: syz-executor0 Not tainted 4.8.0 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
8801f2f5f608 81694cc3 82445d20
8801f2f5f6e0 82498700 8801f2f5f6e0 8801f2f5f6d0
812534a0 41b58ab3 829cae04 81253336
Call Trace:
[] __dump_stack lib/dump_stack.c:15 [inline]
[] dump_stack+0xb3/0x110 lib/dump_stack.c:51
[] panic+0x16a/0x336 kernel/panic.c:153
[] __warn+0x164/0x170 kernel/panic.c:509
[] warn_slowpath_fmt+0xac/0xd0 kernel/panic.c:532
[] __static_key_slow_dec+0xc7/0xd0 kernel/jump_label.c:150
[] static_key_slow_dec+0x33/0x60 kernel/jump_label.c:174
[] net_disable_timestamp+0x3b/0x50 net/core/dev.c:1729
[] sock_disable_timestamp+0x6e/0x80 net/core/sock.c:403
[] __sk_destruct+0xd4/0x260 net/core/sock.c:1453
[] sk_destruct+0x32/0x60 net/core/sock.c:1472
[] __sk_free+0x72/0x140 net/core/sock.c:1480
[] sk_free+0x23/0x30 net/core/sock.c:1491
[] sock_put include/net/sock.h:1582 [inline]
[] sk_common_release+0x1ac/0x250 net/core/sock.c:2766
[] raw_close+0x21/0x30 net/ipv4/raw.c:687
[] inet_release+0x80/0xd0 net/ipv4/af_inet.c:420
[] sock_release+0x4e/0xf0 net/socket.c:573
[] sock_close+0x16/0x20 net/socket.c:1020
[] __fput+0x182/0x380 fs/file_table.c:208
[] fput+0x15/0x20 fs/file_table.c:244
[] task_work_run+0xcb/0x100 kernel/task_work.c:116
[] exit_task_work include/linux/task_work.h:21 [inline]
[] do_exit+0x5a1/0x17f0 kernel/exit.c:828
[] do_group_exit+0xa7/0x190 kernel/exit.c:953
[] get_signal+0x465/0xdf0 kernel/signal.c:2307
[] do_signal+0x93/0xad0 arch/x86/kernel/signal.c:805
[] exit_to_usermode_loop+0xe0/0x110
arch/x86/entry/common.c:163
[] prepare_exit_to_usermode arch/x86/entry/common.c:198
[inline]
[] syscall_return_slowpath+0x146/0x150
arch/x86/entry/common.c:267
[] entry_SYSCALL_64_fastpath+0xbb/0xbd
Dumping ftrace buffer:
(ftrace buffer empty)
Kernel Offset: disabled
==
= About RaceFuzzer
RaceFuzzer is a customized version o
KASAN: use-after-free Read in vhost_chr_write_iter
We report the crash: KASAN: use-after-free Read in vhost_chr_write_iter
This crash has been found in v4.17-rc1 using RaceFuzzer (a modified
version of Syzkaller), which we describe more at the end of this
report. Our analysis shows that the race occurs when invoking two
syscalls concurrently, write$vnet and ioctl$VHOST_RESET_OWNER.
Analysis:
We think the concurrent execution of vhost_process_iotlb_msg() and
vhost_dev_cleanup() causes the crash.
Both of functions can run concurrently (please see call sequence below),
and possibly, there is a race on dev->iotlb.
If the switch occurs right after vhost_dev_cleanup() frees
dev->iotlb, vhost_process_iotlb_msg() still sees the non-null value and it
keep executing without returning -EFAULT. Consequently, use-after-free
occures
Thread interleaving:
CPU0 (vhost_process_iotlb_msg) CPU1 (vhost_dev_cleanup)
(In the case of both VHOST_IOTLB_UPDATE and
VHOST_IOTLB_INVALIDATE)
= =
vhost_umem_clean(dev->iotlb);
if (!dev->iotlb) {
ret = -EFAULT;
break;
}
dev->iotlb = NULL;
Call Sequence:
CPU0
=
vhost_net_chr_write_iter
vhost_chr_write_iter
vhost_process_iotlb_msg
CPU1
=
vhost_net_ioctl
vhost_net_reset_owner
vhost_dev_reset_owner
vhost_dev_cleanup
==
BUG: KASAN: use-after-free in vhost_umem_interval_tree_iter_first
drivers/vhost/vhost.c:52 [inline]
BUG: KASAN: use-after-free in vhost_del_umem_range drivers/vhost/vhost.c:936
[inline]
BUG: KASAN: use-after-free in vhost_process_iotlb_msg
drivers/vhost/vhost.c:1010 [inline]
BUG: KASAN: use-after-free in vhost_chr_write_iter+0x44e/0xcd0
drivers/vhost/vhost.c:1037
Read of size 8 at addr 8801d9d7bc00 by task syz-executor0/4997
CPU: 0 PID: 4997 Comm: syz-executor0 Not tainted 4.17.0-rc1 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x166/0x21c lib/dump_stack.c:113
print_address_description+0x73/0x250 mm/kasan/report.c:256
kasan_report_error mm/kasan/report.c:354 [inline]
kasan_report+0x23f/0x360 mm/kasan/report.c:412
check_memory_region_inline mm/kasan/kasan.c:260 [inline]
__asan_load8+0x54/0x90 mm/kasan/kasan.c:699
vhost_umem_interval_tree_iter_first drivers/vhost/vhost.c:52 [inline]
vhost_del_umem_range drivers/vhost/vhost.c:936 [inline]
vhost_process_iotlb_msg drivers/vhost/vhost.c:1010 [inline]
vhost_chr_write_iter+0x44e/0xcd0 drivers/vhost/vhost.c:1037
vhost_net_chr_write_iter+0x38/0x40 drivers/vhost/net.c:1380
call_write_iter include/linux/fs.h:1784 [inline]
new_sync_write fs/read_write.c:474 [inline]
__vfs_write+0x355/0x480 fs/read_write.c:487
vfs_write+0x12d/0x2d0 fs/read_write.c:549
ksys_write+0xca/0x190 fs/read_write.c:598
__do_sys_write fs/read_write.c:610 [inline]
__se_sys_write fs/read_write.c:607 [inline]
__x64_sys_write+0x43/0x50 fs/read_write.c:607
do_syscall_64+0x15f/0x4a0 arch/x86/entry/common.c:287
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4563f9
RSP: 002b:7f4da7ce9b28 EFLAGS: 0246 ORIG_RAX: 0001
RAX: ffda RBX: 0072bee0 RCX: 004563f9
RDX: 0068 RSI: 26c0 RDI: 0015
RBP: 0729 R08: R09:
R10: R11: 0246 R12: 7f4da7cea6d4
R13: R14: 006ffc78 R15:
Allocated by task 4997:
save_stack+0x43/0xd0 mm/kasan/kasan.c:448
set_track mm/kasan/kasan.c:460 [inline]
kasan_kmalloc+0xae/0xe0 mm/kasan/kasan.c:553
__do_kmalloc_node mm/slab.c:3682 [inline]
__kmalloc_node+0x47/0x70 mm/slab.c:3689
kmalloc_node include/linux/slab.h:554 [inline]
kvmalloc_node+0x99/0xd0 mm/util.c:421
kvmalloc include/linux/mm.h:550 [inline]
kvzalloc include/linux/mm.h:558 [inline]
vhost_umem_alloc+0x72/0x120 drivers/vhost/vhost.c:1260
vhost_init_device_iotlb+0x1e/0x160 drivers/vhost/vhost.c:1548
vhost_net_set_features drivers/vhost/net.c:1273 [inline]
vhost_net_ioctl+0x849/0x1040 drivers/vhost/net.c:1338
vfs_ioctl fs/ioctl.c:46 [inline]
do_vfs_ioctl+0x145/0xd00 fs/ioctl.c:686
ksys_ioctl+0x94/0xb0 fs/ioctl.c:701
__do_sys_ioctl fs/ioctl.c:708 [inline]
__se_sys_ioctl fs/ioctl.c:706 [inline]
__x64_sys_ioctl+0x43/0x50 fs/ioctl.c:706
do_syscall_64+0x15f/0x4a0 arch/x86/entry/common.c:287
entry_SYSCALL_64_after_hwframe+0x49/0xbe
Freed by task 5000:
save_stack+0x43/0xd0 mm/kasan/kasan.c:448
set_track mm/kasan/kasan.c:460 [inline]
__kasan_slab_free+0x11a/0x170 mm/kasan/kasan.c:521
kasan_slab_free+0xe/0x10 mm/kasan/kasan.c:528
__cache_free mm/slab.c:3498 [inline]
kfree+0xd9/0x260 mm/slab.
KASAN: null-ptr-deref Read in rds_ib_get_mr
We report the crash: KASAN: null-ptr-deref Read in rds_ib_get_mr
Note that this bug is previously reported by syzkaller.
https://syzkaller.appspot.com/bug?id=0bb56a5a48b000b52aa2b0d8dd20b1f545214d91
Nonetheless, this bug has not fixed yet, and we hope that this report and our
analysis, which gets help by the RaceFuzzer's feature, will helpful to fix the
crash.
This crash has been found in v4.17-rc1 using RaceFuzzer (a modified
version of Syzkaller), which we describe more at the end of this
report. Our analysis shows that the race occurs when invoking two
syscalls concurrently, bind$rds and setsockopt$RDS_GET_MR.
Analysis:
We think the concurrent execution of __rds_rdma_map() and rds_bind()
causes the problem. __rds_rdma_map() checks whether rs->rs_bound_addr is 0
or not. But the concurrent execution with rds_bind() can by-pass this
check. Therefore, __rds_rdmap_map() calls rs->rs_transport->get_mr() and
rds_ib_get_mr() causes the null deref at ib_rdma.c:544 in v4.17-rc1, when
dereferencing rs_conn.
Thread interleaving:
CPU0 (__rds_rdma_map) CPU1 (rds_bind)
// rds_add_bound() sets
rs->bound_addr as none 0
ret = rds_add_bound(rs,
sin->sin_addr.s_addr, &sin->sin_port);
if (rs->rs_bound_addr == 0 || !rs->rs_transport) {
ret = -ENOTCONN; /* XXX not a great errno */
goto out;
}
if (rs->rs_transport) {
/* previously bound */
trans =
rs->rs_transport;
if
(trans->laddr_check(sock_net(sock->sk),
sin->sin_addr.s_addr) != 0) {
ret =
-ENOPROTOOPT;
//
rds_remove_bound() sets rs->bound_addr as 0
rds_remove_bound(rs);
...
trans_private = rs->rs_transport->get_mr(sg, nents, rs,
&mr->r_key);
(in rds_ib_get_mr())
struct rds_ib_connection *ic = rs->rs_conn->c_transport_data;
Call sequence (v4.17-rc1):
CPU0
rds_setsockopt
rds_get_mr
__rds_rdma_map
rds_ib_get_mr
CPU1
rds_bind
rds_add_bound
...
rds_remove_bound
Crash log:
==
BUG: KASAN: null-ptr-deref in rds_ib_get_mr+0x3a/0x150 net/rds/ib_rdma.c:544
Read of size 8 at addr 0068 by task syz-executor0/32067
CPU: 0 PID: 32067 Comm: syz-executor0 Not tainted 4.17.0-rc1 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x166/0x21c lib/dump_stack.c:113
kasan_report_error mm/kasan/report.c:352 [inline]
kasan_report+0x140/0x360 mm/kasan/report.c:412
check_memory_region_inline mm/kasan/kasan.c:260 [inline]
__asan_load8+0x54/0x90 mm/kasan/kasan.c:699
rds_ib_get_mr+0x3a/0x150 net/rds/ib_rdma.c:544
__rds_rdma_map+0x521/0x9d0 net/rds/rdma.c:271
rds_get_mr+0xad/0xf0 net/rds/rdma.c:333
rds_setsockopt+0x57f/0x720 net/rds/af_rds.c:347
__sys_setsockopt+0x147/0x230 net/socket.c:1903
__do_sys_setsockopt net/socket.c:1914 [inline]
__se_sys_setsockopt net/socket.c:1911 [inline]
__x64_sys_setsockopt+0x67/0x80 net/socket.c:1911
do_syscall_64+0x15f/0x4a0 arch/x86/entry/common.c:287
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4563f9
RSP: 002b:7f6a2b3c2b28 EFLAGS: 0246 ORIG_RAX: 0036
RAX: ffda RBX: 0072bee0 RCX: 004563f9
RDX: 0002 RSI: 0114 RDI: 0015
RBP: 0575 R08: 0020 R09:
R10: 2140 R11: 0246 R12: 7f6a2b3c36d4
R13: R14: 006fd398 R15:
==
= About RaceFuzzer
RaceFuzzer is a customized version of Syzkaller, specifically tailored
to find race condition bugs in the Linux kernel. While we leverage
many different technique, the notable feature of RaceFuzzer is in
leveraging a custom hypervisor (QEMU/KVM) to interleave the
scheduling. In particular, we modified the hypervisor to intentionally
stall a per-core execution, which is similar to supporting per-core
breakpoint functionality. This allows RaceFuzzer to force the kernel
to deterministically trigger racy condition (which may rarely happen
in practice due to randomness in scheduling).
RaceFuzzer's C repro always pinpoints two racy syscalls. Since C
repro's scheduling synchronization should be performed
Re: kernel BUG at /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:LINE!
Hello. We have analyzed the cause of the crash, kernel BUG at net/packet/af_packet.c:LINE!, which is found by RaceFuzzer (a modified version of Syzkaller) in v4.16-rc7. Since struct packet_sock's member variables, running, has_vnet_hdr, origdev and auxdata are declared as bitfields, accessing these variables can race if there is no synchronization mechanism. We think racing between following lines in af_packet.c causes the crash. In function __unregister_prot_hook, po->running = 0; In function packet_setsockopt, po->has_vnet_hdr = !!val; Analysis: CPU0 pakcet_setsockopt po->has_vnet_hdr = !!val; CPU1 packet_do_bind __unregister_prot_hook po->running = 0; In the CPU1, the value of po->running should become 0, but because of racing, it is possible that po->running can keep the value 1. Consequently, after returning from __unregister_prot_hook, BUG_ON at net/packet/af_packet.c:3107 can be triggered. Possible interleaving between racy C source lines is as follows (built with gcc-7.1.0). CPU0 (po->has_vnet_hdr = !!val)CPU1 (po->running = 0) movzbl 0x6e0(%r15),%eax andb $0xfe,0x6e0(%r13) shl$0x3,%r12d and$0xfff7,%eax or %r12d,%eax mov%al,0x6e0(%r15) Please, check out the following reproducer. C repro code : https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-repro.c kernel config v4.16-rc3 : https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.16-rc3.config kernel config v4.16-rc7 : https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.16-rc7.config kernel config v4.15.14 : https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.15.14.config = About RaceFuzzer RaceFuzzer is a customized version of Syzkaller, specifically tailored to find race condition bugs in the Linux kernel. While we leverage many different technique, the notable feature of RaceFuzzer is in leveraging a custom hypervisor (QEMU/KVM) to interleave the scheduling. In particular, we modified the hypervisor to intentionally stall a per-core execution, which is similar to supporting per-core breakpoint functionality. This allows RaceFuzzer to force the kernel to deterministically trigger racy condition (which may rarely happen in practice due to randomness in scheduling). RaceFuzzer's C repro always pinpoints two racy syscalls. Since C repro's scheduling synchronization should be performed at the user space, its reproducibility is limited (reproduction may take from 1 second to 10 minutes (or even more), depending on a bug). This is because, while RaceFuzzer precisely interleaves the scheduling at the kernel's instruction level when finding this bug, C repro cannot fully utilize such a feature. Please disregard all code related to "should_hypercall" in the C repro, as this is only for our debugging purposes using our own hypervisor. On Sat, Mar 31, 2018 at 1:33 AM, DaeRyong Jeong wrote: > We report the crash: kernel BUG at > /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:LINE! > > This crash has been found in v4.16-rc3 using RaceFuzzer (a modified > version of Syzkaller), which we describe more at the end of this > report. Our analysis shows that the race occurs when invoking two > syscalls concurrently, (setsockopt$packet_int) and (bind$packet). > We have confirmed that the kernel v4.16-rc3, v4.16-rc7, and v4.15.14 > built with gcc 7.1.0 are crashing by running the provided C repro > program within a few minutes (5 minutes). > Note that this crash can be triggered from the user space. > > C repro code : > https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-repro.c > kernel config v4.16-rc3 : > https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.16-rc3.config > kernel config v4.16-rc7 : > https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.16-rc7.config > kernel config v4.15.14 : > https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.15.14.config > > [ 881.047513] [ cut here ] > [ 881.048416] kernel BUG at > /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:3107! > [ 881.050014] invalid opcode: [#1] SMP KASAN > [ 881.050698] Dumping ftrace buffer: > [ 881.051244](ftrace buffer empty) > [ 881.051768] Modules linked in: > [ 881.052236] CPU: 1 PID: 18247 Comm: syz-executor0 Not tainted 4.16.0-rc3 #1 > [ 881.053247] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014 > [ 881.054880] RIP: 0010:packet_do_bind+0x88d/0x950 > [ 881.03] RSP: 0018:8802231d7b08 EFLAGS: 00010212 > [ 881.056310] RAX: 0001 RBX: 8800af8317
Re: WARNING in refcount_dec
Hello. We have analyzed the cause of the crash in v4.16-rc3, WARNING in refcount_dec, which is found by RaceFuzzer (a modified version of Syzkaller). Since struct packet_sock's member variables, running, has_vnet_hdr, origdev and auxdata are declared as bitfields, accessing these variables can race if there is no synchronization mechanism. We think racing between following lines in af_packet.c causes the crash. In function __unregister_prot_hook, po->running = 0; In function packet_setsockopt, po->has_vnet_hdr = !!val; Analysis: CPU0 pakcet_setsockopt po->has_vnet_hdr = !!val; CPU1 packet_setsockop packet_set_ring __unregister_prot_hook po->running = 0; In the CPU1, the value of po->running should become 0, but because of racing, it is possible that po->running can keep the value 1. Consequently, the followings can happen. - When packet_set_ring calls register_prot_hook, register_prot_hook return immediately without calling sock_hold(sk). - When packet_release is called, __unregister_prot_hook will be called because po->running == 1 and sk->sk_refcnt hits zero. Possible interleaving between racy C source lines is as follows (built with gcc-7.1.0). CPU0 (po->has_vnet_hdr = !!val) CPU1 (po->running = 0) movzbl 0x6e0(%r15),%eax andb $0xfe,0x6e0(%r13) shl$0x3,%r12d and$0xfff7,%eax or %r12d,%eax mov%al,0x6e0(%r15) Please, check out the following reproducer. C repro code : https://kiwi.cs.purdue.edu/static/race-fuzzer/repro-refcount_dec.c kernel config: https://kiwi.cs.purdue.edu/static/race-fuzzer/kernel-config-v4.16-rc3 Since there is a small room to race, it may take a long time to reproduce the crash. = About RaceFuzzer RaceFuzzer is a customized version of Syzkaller, specifically tailored to find race condition bugs in the Linux kernel. While we leverage many different technique, the notable feature of RaceFuzzer is in leveraging a custom hypervisor (QEMU/KVM) to interleave the scheduling. In particular, we modified the hypervisor to intentionally stall a per-core execution, which is similar to supporting per-core breakpoint functionality. This allows RaceFuzzer to force the kernel to deterministically trigger racy condition (which may rarely happen in practice due to randomness in scheduling). RaceFuzzer's C repro always pinpoints two racy syscalls. Since C repro's scheduling synchronization should be performed at the user space, its reproducibility is limited (reproduction may take from 1 second to 10 minutes (or even more), depending on a bug). This is because, while RaceFuzzer precisely interleaves the scheduling at the kernel's instruction level when finding this bug, C repro cannot fully utilize such a feature. Please disregard all code related to "should_hypercall" in the C repro, as this is only for our debugging purposes using our own hypervisor. On Tue, Apr 3, 2018 at 1:12 PM, DaeRyong Jeong wrote: > No. Only the first crash (WARNING in refcount_dec) is reproduced by > the attached reproducer. > > The second crash (kernel bug at af_packet.c:3107) is reproduced by > another reproducer. > We reported it here. > http://lkml.iu.edu/hypermail/linux/kernel/1803.3/05324.html > > On Sun, Apr 1, 2018 at 4:38 PM, Willem de Bruijn > wrote: >> On Thu, Mar 29, 2018 at 1:16 AM, Cong Wang wrote: >>> (Cc'ing netdev and Willem) >>> >>> On Wed, Mar 28, 2018 at 12:03 PM, Byoungyoung Lee >>> wrote: >>>> Another crash patterns observed: race between (setsockopt$packet_int) >>>> and (bind$packet). >>>> >>>> -- >>>> [ 357.731597] kernel BUG at >>>> /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:3107! >>>> [ 357.733382] invalid opcode: [#1] SMP KASAN >>>> [ 357.734017] Modules linked in: >>>> [ 357.734662] CPU: 1 PID: 3871 Comm: repro.exe Not tainted 4.16.0-rc3 #1 >>>> [ 357.735791] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), >>>> BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014 >>>> [ 357.737434] RIP: 0010:packet_do_bind+0x88d/0x950 >>>> [ 357.738121] RSP: 0018:8800b2787b08 EFLAGS: 00010293 >>>> [ 357.738906] RAX: 8800b2fdc780 RBX: 880234358cc0 RCX: >>>> 838b244c >>>> [ 357.739905] RDX: RSI: 838b257d RDI: >>>> 0001 >>>> [ 357.741315] RBP: 8800b2787c10 R08: 8800b2fdc780 R09: >>>> >>>> [ 357.743055] R10: 0001 R11: R12: >>>> 88023352ecc0 >>>>
Re: WARNING in refcount_dec
No. Only the first crash (WARNING in refcount_dec) is reproduced by the attached reproducer. The second crash (kernel bug at af_packet.c:3107) is reproduced by another reproducer. We reported it here. http://lkml.iu.edu/hypermail/linux/kernel/1803.3/05324.html On Sun, Apr 1, 2018 at 4:38 PM, Willem de Bruijn wrote: > On Thu, Mar 29, 2018 at 1:16 AM, Cong Wang wrote: >> (Cc'ing netdev and Willem) >> >> On Wed, Mar 28, 2018 at 12:03 PM, Byoungyoung Lee >> wrote: >>> Another crash patterns observed: race between (setsockopt$packet_int) >>> and (bind$packet). >>> >>> -- >>> [ 357.731597] kernel BUG at >>> /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:3107! >>> [ 357.733382] invalid opcode: [#1] SMP KASAN >>> [ 357.734017] Modules linked in: >>> [ 357.734662] CPU: 1 PID: 3871 Comm: repro.exe Not tainted 4.16.0-rc3 #1 >>> [ 357.735791] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), >>> BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014 >>> [ 357.737434] RIP: 0010:packet_do_bind+0x88d/0x950 >>> [ 357.738121] RSP: 0018:8800b2787b08 EFLAGS: 00010293 >>> [ 357.738906] RAX: 8800b2fdc780 RBX: 880234358cc0 RCX: >>> 838b244c >>> [ 357.739905] RDX: RSI: 838b257d RDI: >>> 0001 >>> [ 357.741315] RBP: 8800b2787c10 R08: 8800b2fdc780 R09: >>> >>> [ 357.743055] R10: 0001 R11: R12: >>> 88023352ecc0 >>> [ 357.744744] R13: R14: 0001 R15: >>> 1d00 >>> [ 357.746377] FS: 7f4b43733700() GS:8800b8b0() >>> knlGS: >>> [ 357.749599] CS: 0010 DS: ES: CR0: 80050033 >>> [ 357.752096] CR2: 20058000 CR3: 0002334b8000 CR4: >>> 06e0 >>> [ 357.755045] Call Trace: >>> [ 357.755822] ? compat_packet_setsockopt+0x100/0x100 >>> [ 357.757324] ? __sanitizer_cov_trace_const_cmp8+0x18/0x20 >>> [ 357.758810] packet_bind+0xa2/0xe0 >>> [ 357.759640] SYSC_bind+0x279/0x2f0 >>> [ 357.760364] ? move_addr_to_kernel.part.19+0xc0/0xc0 >>> [ 357.761491] ? __handle_mm_fault+0x25d0/0x25d0 >>> [ 357.762449] ? __sanitizer_cov_trace_const_cmp4+0x16/0x20 >>> [ 357.763663] ? __do_page_fault+0x417/0xba0 >>> [ 357.764569] ? vmalloc_fault+0x910/0x910 >>> [ 357.765405] ? __sanitizer_cov_trace_const_cmp4+0x16/0x20 >>> [ 357.766525] ? mark_held_locks+0x25/0xb0 >>> [ 357.767336] ? SyS_socketpair+0x4a0/0x4a0 >>> [ 357.768182] SyS_bind+0x24/0x30 >>> [ 357.768851] do_syscall_64+0x209/0x5d0 >>> [ 357.769650] ? syscall_return_slowpath+0x3e0/0x3e0 >>> [ 357.770665] ? __sanitizer_cov_trace_const_cmp4+0x16/0x20 >>> [ 357.771779] ? syscall_return_slowpath+0x260/0x3e0 >>> [ 357.772748] ? mark_held_locks+0x25/0xb0 >>> [ 357.773581] ? __sanitizer_cov_trace_const_cmp4+0x16/0x20 >>> [ 357.774720] ? retint_user+0x18/0x18 >>> [ 357.775493] ? trace_hardirqs_off_caller+0xb5/0x120 >>> [ 357.776567] ? trace_hardirqs_off_thunk+0x1a/0x1c >>> [ 357.777512] entry_SYSCALL_64_after_hwframe+0x42/0xb7 >>> [ 357.778508] RIP: 0033:0x4503a9 >>> [ 357.779156] RSP: 002b:7f4b43732ce8 EFLAGS: 0246 ORIG_RAX: >>> 0031 >>> [ 357.780737] RAX: ffda RBX: RCX: >>> 004503a9 >>> [ 357.782169] RDX: 0014 RSI: 20058000 RDI: >>> 0003 >>> [ 357.783710] RBP: 7f4b43732d10 R08: R09: >>> >>> [ 357.785202] R10: R11: 0246 R12: >>> >>> [ 357.786664] R13: R14: 7f4b437339c0 R15: >>> 7f4b43733700 >>> [ 357.788210] Code: c0 fd 48 c7 c2 00 c8 d9 84 be ab 02 00 00 48 c7 >>> c7 60 c8 d9 84 c6 05 e7 a2 48 02 01 e8 3f 17 af fd e9 60 fb ff ff e8 >>> 43 b3 c0 fd <0f> 0b e8 3c b3 c0 fd 48 8b bd 20 ff ff ff e8 60 1e e7 fd >>> 4c 89 >>> [ 357.792260] RIP: packet_do_bind+0x88d/0x950 RSP: 8800b2787b08 >>> [ 357.793698] ---[ end trace 0c5a2539f0247369 ]--- >>> [ 357.794696] Kernel panic - not syncing: Fatal exception >>> [ 357.795918] Kernel Offset: disabled >>> [ 357.796614] Rebooting in 86400 seconds.. >>> >>> On Wed, Mar 28, 2018 at 1:19 AM, Byoungyoung Lee >>> wrote: We report the crash: WARNING in refcount_dec This crash has been found in v4.16-rc3 using RaceFuzzer (a modified version of Syzkaller), which we describe more at the end of this report. Our analysis shows that the race occurs when invoking two syscalls concurrently, (setsockopt$packet_int) and (setsockopt$packet_rx_ring). C repro code : https://kiwi.cs.purdue.edu/static/race-fuzzer/repro-refcount_dec.c kernel config: https://kiwi.cs.purdue.edu/static/race-fuzzer/kernel-config-v4.16-rc3 >> >> >> I tried your reproducer, no luck here. >> > > Are both crashes with the same reproducer? > > It races setsockopt PACKET_RX_RING wit
kernel BUG at /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:LINE!
We report the crash: kernel BUG at /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:LINE! This crash has been found in v4.16-rc3 using RaceFuzzer (a modified version of Syzkaller), which we describe more at the end of this report. Our analysis shows that the race occurs when invoking two syscalls concurrently, (setsockopt$packet_int) and (bind$packet). We have confirmed that the kernel v4.16-rc3, v4.16-rc7, and v4.15.14 built with gcc 7.1.0 are crashing by running the provided C repro program within a few minutes (5 minutes). Note that this crash can be triggered from the user space. C repro code : https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-repro.c kernel config v4.16-rc3 : https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.16-rc3.config kernel config v4.16-rc7 : https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.16-rc7.config kernel config v4.15.14 : https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.15.14.config [ 881.047513] [ cut here ] [ 881.048416] kernel BUG at /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:3107! [ 881.050014] invalid opcode: [#1] SMP KASAN [ 881.050698] Dumping ftrace buffer: [ 881.051244](ftrace buffer empty) [ 881.051768] Modules linked in: [ 881.052236] CPU: 1 PID: 18247 Comm: syz-executor0 Not tainted 4.16.0-rc3 #1 [ 881.053247] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014 [ 881.054880] RIP: 0010:packet_do_bind+0x88d/0x950 [ 881.03] RSP: 0018:8802231d7b08 EFLAGS: 00010212 [ 881.056310] RAX: 0001 RBX: 8800af831740 RCX: c900025ce000 [ 881.057318] RDX: 00a5 RSI: 838b257d RDI: 0001 [ 881.058301] RBP: 8802231d7c10 R08: 8802342f2480 R09: [ 881.059298] R10: 0001 R11: R12: 8802309f8f00 [ 881.060314] R13: R14: 0001 R15: 1000 [ 881.061320] FS: 7f7fab50d700() GS:88023fc0() knlGS: [ 881.062467] CS: 0010 DS: ES: CR0: 80050033 [ 881.063285] CR2: 20038000 CR3: b11c9000 CR4: 06e0 [ 881.064317] Call Trace: [ 881.064686] ? compat_packet_setsockopt+0x100/0x100 [ 881.065430] ? __sanitizer_cov_trace_const_cmp8+0x18/0x20 [ 881.066188] packet_bind+0xa2/0xe0 [ 881.066690] SYSC_bind+0x279/0x2f0 [ 881.067180] ? move_addr_to_kernel.part.19+0xc0/0xc0 [ 881.067896] ? do_futex+0x1e90/0x1e90 [ 881.068435] ? SyS_sched_getaffinity+0xe3/0x100 [ 881.069112] ? mark_held_locks+0x25/0xb0 [ 881.069677] ? SyS_socketpair+0x4a0/0x4a0 [ 881.070265] SyS_bind+0x24/0x30 [ 881.070732] do_syscall_64+0x209/0x5d0 [ 881.071270] ? syscall_return_slowpath+0x3e0/0x3e0 [ 881.071929] ? __sanitizer_cov_trace_const_cmp4+0x16/0x20 [ 881.072675] ? syscall_return_slowpath+0x260/0x3e0 [ 881.073365] ? mark_held_locks+0x25/0xb0 [ 881.073950] ? entry_SYSCALL_64_after_hwframe+0x52/0xb7 [ 881.074693] ? trace_hardirqs_off_caller+0xb5/0x120 [ 881.075390] ? trace_hardirqs_off_thunk+0x1a/0x1c [ 881.076079] entry_SYSCALL_64_after_hwframe+0x42/0xb7 [ 881.076797] RIP: 0033:0x453909 [ 881.077238] RSP: 002b:7f7fab50caf8 EFLAGS: 0212 ORIG_RAX: 0031 [ 881.078268] RAX: ffda RBX: 007080d8 RCX: 00453909 [ 881.079239] RDX: 0014 RSI: 2001f000 RDI: 0015 [ 881.080268] RBP: 0250 R08: R09: [ 881.081256] R10: R11: 0212 R12: 004a82d3 [ 881.082272] R13: R14: 0015 R15: 2001f000 [ 881.083251] Code: c0 fd 48 c7 c2 00 c8 d9 84 be ab 02 00 00 48 c7 c7 60 c8 d9 84 c6 05 e7 a2 48 02 01 e8 3f 17 af fd e9 60 fb ff ff e8 43 b3 c0 fd <0f> 0b e8 3c b3 c0 fd 48 8b bd 20 ff ff ff e8 60 1e e7 fd 4c 89 [ 881.085828] RIP: packet_do_bind+0x88d/0x950 RSP: 8802231d7b08 [ 881.086619] ---[ end trace 9c461502752b4f3e ]--- [ 881.087181] Kernel panic - not syncing: Fatal exception [ 881.088352] Dumping ftrace buffer: [ 881.088877](ftrace buffer empty) [ 881.089414] Kernel Offset: disabled [ 881.089950] Rebooting in 86400 seconds.. = About RaceFuzzer RaceFuzzer is a customized version of Syzkaller, specifically tailored to find race condition bugs in the Linux kernel. While we leverage many different technique, the notable feature of RaceFuzzer is in leveraging a custom hypervisor (QEMU/KVM) to interleave the scheduling. In particular, we modified the hypervisor to intentionally stall a per-core execution, which is similar to supporting per-core breakpoint functionality. This allows RaceFuzzer to force the kernel to deterministically trigger racy condition (which may rarely happen in practice due to randomness in sched
