Big test regression in -current
Something seriously happened to -current in the last few days. Between June 2 and June 8 the number of test failures jumped from 17 to 80, see: http://www.netbsd.org/~martin/sparc64-atf/ Some rump test crash like this: panic: kernel diagnostic assertion "curcpu() == ci" failed: file "/usr/src/lib/librump/../../sys/rump/librump/rumpkern/intr.c", line 331 with a backtrace similar to: Program terminated with signal 6, Aborted. #0 abort () at /usr/src/lib/libc/stdlib/abort.c:80 80 (void)signal(SIGABRT, SIG_DFL); #0 abort () at /usr/src/lib/libc/stdlib/abort.c:80 #1 0xfcb0a260 in rumpuser_exit (rv=) at /usr/src/lib/librumpuser/rumpuser.c:597 #2 0xfcdb2908 in cpu_reboot (howto=, bootstr=0x0) at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/emul.c:352 #3 0xfcd77840 in vpanic (fmt=0xfcdb5498 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ", ap=0xf92f1b98) at /usr/src/lib/librump/../../sys/rump/../kern/subr_prf.c:284 #4 0xfcd4e450 in kern_assert (fmt=0xfcdb5498 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ") at /usr/src/lib/librump/../../sys/rump/../lib/libkern/kern_assert.c:51 #5 0xfcdb21e4 in softint_schedule_cpu (arg=0xfbf2c580, ci=0xfcef8c00) at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/intr.c:331 #6 0xfd20bb9c in pktq_enqueue (pq=0xfbf65e80, m=0xfa199e00, hash=) at /usr/src/lib/librumpnet/../../sys/rump/../net/pktqueue.c:217 #7 0xfd47e460 in ether_input (ifp=0xfa1ff800, m=0xfa199e00) at /usr/src/sys/rump/net/lib/libnet/../../../../net/if_ethersubr.c:947 #8 0xfd802f3c in shmif_rcv (arg=0xfa1ff800) at /usr/src/sys/rump/net/lib/libshmif/if_shmem.c:760 #9 0xfcdadd50 in threadbouncer (arg=0xfbf2c5a0) at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/threads.c:90 #10 0xfc90c7b8 in pthread__create_tramp (cookie=0xfabf2000) at /usr/src/lib/libpthread/pthread.c:572 or: panic: kernel diagnostic assertion "uid != VNOVAL && gid != VNOVAL && mode != VNOVAL" failed: file "/usr/src/sys/rump/fs/lib/libtmpfs/../../../../fs/tmpfs/tmpfs_subr.c", line 149 with: #0 abort () at /usr/src/lib/libc/stdlib/abort.c:80 80 (void)signal(SIGABRT, SIG_DFL); #0 abort () at /usr/src/lib/libc/stdlib/abort.c:80 #1 0xfe50a260 in rumpuser_exit (rv=) at /usr/src/lib/librumpuser/rumpuser.c:597 #2 0xfe7b2908 in cpu_reboot (howto=, bootstr=0x0) at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/emul.c:352 #3 0xfe777840 in vpanic (fmt=0xff211cf8 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ", ap=0xab78) at /usr/src/lib/librump/../../sys/rump/../kern/subr_prf.c:284 #4 0xfe74e450 in kern_assert (fmt=0xff211cf8 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ") at /usr/src/lib/librump/../../sys/rump/../lib/libkern/kern_assert.c:51 #5 0xff20a108 in tmpfs_alloc_node (tmp=0xfafa8038, type=, uid=, gid=, mode=, target=0x0, rdev=, node=) at /usr/src/sys/rump/fs/lib/libtmpfs/../../../../fs/tmpfs/tmpfs_subr.c:149 #6 0xff21154c in tmpfs_mount (mp=0xfaf84000, path=0x102860 "/stor", data=0xfdb3a730, data_len=) at /usr/src/sys/rump/fs/lib/libtmpfs/../../../../fs/tmpfs/tmpfs_vfsops.c:181 #7 0xfea41f28 in VFS_MOUNT (mp=0xfaf84000, a=0x102860 "/stor", b=0xfdb3a730, c=0xb008) at /usr/src/lib/librumpvfs/../../sys/rump/../kern/vfs_subr.c:914 #8 0xfea44560 in mount_domount (l=0xfafac800, vpp=0xaf28, vfsops=, path=0x102860 "/stor", flags=, data=0xfdb3a730, data_len=) at /usr/src/lib/librumpvfs/../../sys/rump/../kern/vfs_mount.c:663 #9 0xfea3bd0c in do_sys_mount (l=l@entry=0xfafac800, vfsops=, vfsops@entry=0x0, type=, path=0x102860 "/stor", flags=, data=0xb278, data_seg=, data_len=, retval=) at /usr/src/lib/librumpvfs/../../sys/rump/../kern/vfs_syscalls.c:531 #10 0xfea3c210 in sys___mount50 (l=0xfafac800, uap=0xb198, retval=0xb188) at /usr/src/lib/librumpvfs/../../sys/rump/../kern/vfs_syscalls.c:444 #11 0xfe7b4884 in sy_call (rval=0xb188, uap=0xb198, l=0xfafac800, sy=0xfe8ebba0 ) at /usr/src/lib/librump/../../sys/rump/../sys/syscallvar.h:61 #12 sy_invoke (code=, rval=0xb188, uap=0xb198, l=0xfafac800, sy=0xfe8ebba0 ) at /usr/src/lib/librump/../../sys/rump/../sys/syscallvar.h:85 #13 rump_syscall (num=num@entry=410, data=data@entry=0xb198, dlen=dlen@entry=40, retval=retval@entry=0xb188) at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/rump.c:844 #14 0xfe7aa2d4 in rump___sysimpl_mount50 (type=type@entry=0x102878 "tmpfs", path=path@entry=0x102860 "/stor", flags=flags@entry=0,
Re: Big test regression in -current
I see a similar jump in failures on my amd64 system, between June 5 and June 6 ... Source Check-Out TimePassFail XFailSkip 2014-06-06 01:50:16 3781 124 54 80 2014-06-05 18:50:16 3867 33 56 83 On Sun, 8 Jun 2014, Martin Husemann wrote: Something seriously happened to -current in the last few days. Between June 2 and June 8 the number of test failures jumped from 17 to 80, see: http://www.netbsd.org/~martin/sparc64-atf/ Some rump test crash like this: panic: kernel diagnostic assertion "curcpu() == ci" failed: file "/usr/src/lib/librump/../../sys/rump/librump/rumpkern/intr.c", line 331 with a backtrace similar to: Program terminated with signal 6, Aborted. #0 abort () at /usr/src/lib/libc/stdlib/abort.c:80 80 (void)signal(SIGABRT, SIG_DFL); #0 abort () at /usr/src/lib/libc/stdlib/abort.c:80 #1 0xfcb0a260 in rumpuser_exit (rv=) at /usr/src/lib/librumpuser/rumpuser.c:597 #2 0xfcdb2908 in cpu_reboot (howto=, bootstr=0x0) at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/emul.c:352 #3 0xfcd77840 in vpanic (fmt=0xfcdb5498 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ", ap=0xf92f1b98) at /usr/src/lib/librump/../../sys/rump/../kern/subr_prf.c:284 #4 0xfcd4e450 in kern_assert (fmt=0xfcdb5498 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ") at /usr/src/lib/librump/../../sys/rump/../lib/libkern/kern_assert.c:51 #5 0xfcdb21e4 in softint_schedule_cpu (arg=0xfbf2c580, ci=0xfcef8c00) at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/intr.c:331 #6 0xfd20bb9c in pktq_enqueue (pq=0xfbf65e80, m=0xfa199e00, hash=) at /usr/src/lib/librumpnet/../../sys/rump/../net/pktqueue.c:217 #7 0xfd47e460 in ether_input (ifp=0xfa1ff800, m=0xfa199e00) at /usr/src/sys/rump/net/lib/libnet/../../../../net/if_ethersubr.c:947 #8 0xfd802f3c in shmif_rcv (arg=0xfa1ff800) at /usr/src/sys/rump/net/lib/libshmif/if_shmem.c:760 #9 0xfcdadd50 in threadbouncer (arg=0xfbf2c5a0) at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/threads.c:90 #10 0xfc90c7b8 in pthread__create_tramp (cookie=0xfabf2000) at /usr/src/lib/libpthread/pthread.c:572 or: panic: kernel diagnostic assertion "uid != VNOVAL && gid != VNOVAL && mode != VNOVAL" failed: file "/usr/src/sys/rump/fs/lib/libtmpfs/../../../../fs/tmpfs/tmpfs_subr.c", line 149 with: #0 abort () at /usr/src/lib/libc/stdlib/abort.c:80 80 (void)signal(SIGABRT, SIG_DFL); #0 abort () at /usr/src/lib/libc/stdlib/abort.c:80 #1 0xfe50a260 in rumpuser_exit (rv=) at /usr/src/lib/librumpuser/rumpuser.c:597 #2 0xfe7b2908 in cpu_reboot (howto=, bootstr=0x0) at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/emul.c:352 #3 0xfe777840 in vpanic (fmt=0xff211cf8 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ", ap=0xab78) at /usr/src/lib/librump/../../sys/rump/../kern/subr_prf.c:284 #4 0xfe74e450 in kern_assert (fmt=0xff211cf8 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ") at /usr/src/lib/librump/../../sys/rump/../lib/libkern/kern_assert.c:51 #5 0xff20a108 in tmpfs_alloc_node (tmp=0xfafa8038, type=, uid=, gid=, mode=, target=0x0, rdev=, node=) at /usr/src/sys/rump/fs/lib/libtmpfs/../../../../fs/tmpfs/tmpfs_subr.c:149 #6 0xff21154c in tmpfs_mount (mp=0xfaf84000, path=0x102860 "/stor", data=0xfdb3a730, data_len=) at /usr/src/sys/rump/fs/lib/libtmpfs/../../../../fs/tmpfs/tmpfs_vfsops.c:181 #7 0xfea41f28 in VFS_MOUNT (mp=0xfaf84000, a=0x102860 "/stor", b=0xfdb3a730, c=0xb008) at /usr/src/lib/librumpvfs/../../sys/rump/../kern/vfs_subr.c:914 #8 0xfea44560 in mount_domount (l=0xfafac800, vpp=0xaf28, vfsops=, path=0x102860 "/stor", flags=, data=0xfdb3a730, data_len=) at /usr/src/lib/librumpvfs/../../sys/rump/../kern/vfs_mount.c:663 #9 0xfea3bd0c in do_sys_mount (l=l@entry=0xfafac800, vfsops=, vfsops@entry=0x0, type=, path=0x102860 "/stor", flags=, data=0xb278, data_seg=, data_len=, retval=) at /usr/src/lib/librumpvfs/../../sys/rump/../kern/vfs_syscalls.c:531 #10 0xfea3c210 in sys___mount50 (l=0xfafac800, uap=0xb198, retval=0xb188) at /usr/src/lib/librumpvfs/../../sys/rump/../kern/vfs_syscalls.c:444 #11 0xfe7b4884 in sy_call (rval=0xb188, uap=0xb198, l=0xfafac800, sy=0xfe8ebba0 ) at /usr/src/lib/librump/../../sys/rump/../sys/syscallvar.h:61 #12 sy_invoke (code=, rval=0xb188, uap=0xb198, l=0xfafac800, sy=0xfe8ebba0 ) at /usr/src/lib/librump/../../sys/rump/../sys/syscallvar.h:85 #13 rump_syscall (num=num@entry=410, data=data@entry=0x
Re: Big test regression in -current
Martin Husemann wrote: > Something seriously happened to -current in the last few days. Between > June 2 and June 8 the number of test failures jumped from 17 to 80, see: > > http://www.netbsd.org/~martin/sparc64-atf/ > > Some rump test crash like this: > > panic: kernel diagnostic assertion "curcpu() == ci" failed: file > "/usr/src/lib/librump/../../sys/rump/librump/rumpkern/intr.c", line 331 > That is RUMP-specific. Fixed now. -- Mindaugas
Re: Big test regression in -current
On Sun, Jun 8, 2014 at 4:24 PM, Mindaugas Rasiukevicius wrote: > Martin Husemann wrote: >> Something seriously happened to -current in the last few days. Between >> June 2 and June 8 the number of test failures jumped from 17 to 80, see: >> >> http://www.netbsd.org/~martin/sparc64-atf/ >> >> Some rump test crash like this: >> >> panic: kernel diagnostic assertion "curcpu() == ci" failed: file >> "/usr/src/lib/librump/../../sys/rump/librump/rumpkern/intr.c", line 331 >> > > That is RUMP-specific. Fixed now. The rump builds against HEAD are still failing tests http://build.myriabit.eu:8012/waterfall after this fix so there may be other issues, will take a look. Justin
Re: Lockless IP input queue, the pktqueue interface
Hi rmind,
maxlen of ip{,6}_pktq cannot be updated via sysctl.
It seems that we need to do it in sysctl_pktq_count
somehow.
Thanks,
ozaki-r
On Tue, May 27, 2014 at 11:52 AM, Mindaugas Rasiukevicius
wrote:
> Hello,
>
> As we are trying to bring more parallelism in our network stack, I would
> like to make IPv4 and IPv6 input queues lockless. This is implemented by
> replacing struct ifqueue and macros around it with a pktqueue interface.
> This interface also abstracts and handles network ISR scheduling and that
> will allow us to easily add receiver-side packet steering later.
>
> http://www.netbsd.org/~rmind/ip_pktq.diff
>
> The patch makes the following changes:
> - Implements pktqueue interface (see pktqueue.{c,h} files).
> - Replaces ipintrq and ip6intrq with the pktqueue mechanism.
> - Eliminates kernel-lock from ipintr() and ip6intr().
> - Some preparation work to push softnet_lock out of ipintr().
>
> Extra testing would be helpful.
>
> Thanks.
>
> --
> Mindaugas
