Hi. Uwe Toenjes found a bug in ixg(4). See:
http://mail-index.netbsd.org/current-users/2014/10/11/msg025932.html Then, PR#49328 was submitted by him. http://gnats.netbsd.org/49328 This problem is can be reproduced when LOCKDEBUG is defined ifconfig ixg0 up ixg(4) take a spin mutex before accessing MAC core: ------ ixgbe.h ------- #define IXGBE_CORE_LOCK_INIT(_sc, _name) \ mutex_init(&(_sc)->core_mtx, MUTEX_DEFAULT, IPL_NET) ------- ixgbe.c::ixbge_ioctl() ------ case SIOCADDMULTI: case SIOCDELMULTI: case SIOCSIFFLAGS: case SIOCSIFMTU: default: if ((error = ether_ioctl(ifp, command, data)) != ENETRESET) return error; if ((ifp->if_flags & IFF_RUNNING) == 0) ; else if (command == SIOCSIFCAP || command == SIOCSIFMTU) { IXGBE_CORE_LOCK(adapter); ixgbe_init_locked(adapter); IXGBE_CORE_UNLOCK(adapter); In this case, IXGBE_CORE_LOCK() is used to prevent interrupt. I borrowed an old Intel 10G card(82598 based) from my colleague and tested. I got a panic like as follows: -------------------------------------------------------- Mutex error: lockdebug_barrier: spin lock held lock address : 0xffff80001d0eaa08 type : spin initialized : 0xffffffff805cb716 shared holds : 0 exclusive: 1 shares wanted: 0 exclusive: 0 current cpu : 7 last held: 7 current lwp : 0xfffffe840d2049c0 last held: 0xfffffe840d2049c0 last locked* : 0xffffffff805ca63e unlocked : 000000000000000000 owner field : 0x0000000000010600 wait/spin: 0/1 panic: LOCKDEBUG: Mutex error: lockdebug_barrier: spin lock held cpu7: Begin traceback... vpanic() at netbsd:vpanic+0x13c snprintf() at netbsd:snprintf lockdebug_more() at netbsd:lockdebug_more mutex_enter() at netbsd:mutex_enter+0x45e pmap_growkernel() at netbsd:pmap_growkernel+0x2d uvm_map_prepare() at netbsd:uvm_map_prepare+0x24d uvm_map() at netbsd:uvm_map+0x97 uvm_km_alloc() at netbsd:uvm_km_alloc+0xec _bus_dmamem_map.isra.7() at netbsd:_bus_dmamem_map.isra.7+0x69 ixgbe_jcl_reinit() at netbsd:ixgbe_jcl_reinit+0x180 ixgbe_init_locked() at netbsd:ixgbe_init_locked+0x510 ixgbe_init() at netbsd:ixgbe_init+0x22 ether_ioctl() at netbsd:ether_ioctl+0xc8 ixgbe_ioctl() at netbsd:ixgbe_ioctl+0x41 in6_update_ifa1() at netbsd:in6_update_ifa1+0x70e in6_update_ifa() at netbsd:in6_update_ifa+0x36 in6_ifattach_linklocal() at netbsd:in6_ifattach_linklocal+0x2eb in6_ifattach() at netbsd:in6_ifattach+0x15d in6_if_up() at netbsd:in6_if_up+0xf ifioctl_common() at netbsd:ifioctl_common+0x343 ether_ioctl() at netbsd:ether_ioctl+0x162 ixgbe_ioctl() at netbsd:ixgbe_ioctl+0x41 doifioctl() at netbsd:doifioctl+0x493 soo_ioctl() at netbsd:soo_ioctl+0x2af sys_ioctl() at netbsd:sys_ioctl+0x17e syscall() at netbsd:syscall+0x9a --- syscall (number 54) --- 7f7ff6cced0a: cpu7: End traceback... -------------------------------------------------------------- ifconfig ixg0 up -> ioctl() -> take spin mutex -> alloc a lot of dmamem and dmamap. -> it got a shortage of the kernel address space and called pmap_growkernel() -> tried to get adaptive mutex. -> panic My stack trace is different from Uew's traces, but the root of the problem is the same. How should we fix this problem? Some of Ethernet drivers avoid using splnet()/splx() and use mutexes. wm, bnx, ixg, powerpc/booke/dev/pq3etsec.c and others. I think it's better to know the common solution and puse the way. I'm not so familiar with mutex and I have no experience using mutex... Any patches are welcome. I'll try them if you send me :-) Thanks. -- ----------------------------------------------- SAITOH Masanobu (msai...@execsw.org msai...@netbsd.org)