Hi all, After having done a maintainance update of my application server kernel i experience guests to panic with signal 7. The problem is reproducible and seems related to IO traffic as it happens when doing "apt-get update" in guest for example.
New host kernel: 2.6.15.4 + squashfs Different guest kernels all suffer the problem, versions in detail: 2.6.7 + various patches (used to work fine before) 2.6.15.4 + squashfs 2.6.15.4 + squashfs + 2.6.15-bs2 Old host kernel: 2.6.8.1 + various patches + skas3 Short excerpt from crash: [42949509.890000] Kernel panic - not syncing: Kernel mode signal 7 [42949509.890000] [42949509.890000] EIP: 0073:[<402237ee>] CPU: 0 Not tainted ESP: 007b:bf8038e4 EFLAGS: 00000283 [42949509.890000] Not tainted [42949509.890000] EAX: ffffffda EBX: 00000005 ECX: bf8039bc EDX: bf80393c [42949509.890000] ESI: 00000000 EDI: bf803934 EBP: bf803a3c DS: 007b ES: 007b [42949509.890000] 08eb7770: [<0807ce4c>] notifier_call_chain+0x1c/0x40 [42949509.890000] 08eb778c: [<0806ef6b>] panic+0x4b/0xf0 [42949509.890000] 08eb77a0: [<08059e3b>] relay_signal+0x7b/0x80 [42949509.890000] 08eb77bc: [<08059e8f>] bus_handler+0x4f/0x60 [42949509.890000] 08eb77d0: [<0805ce28>] sig_handler_common_skas+0x78/0xd0 [42949509.890000] 08eb77f0: [<0806770f>] sig_handler+0xf/0x20 [42949509.890000] 08eb77fc: [<081a8c08>] __restore+0x0/0x8 [42949509.890000] 08eb783c: [<080b313e>] __pollwait+0x3e/0xb0 [42949509.890000] 08eb789c: [<08057eb4>] change_signals+0x34/0x60 [42949509.890000] 08eb7948: [<08092a57>] __pagevec_lru_add+0x97/0xd0 [42949509.890000] 08eb79a0: [<08057f32>] enable_mask+0x32/0x40 [42949509.890000] 08eb79ac: [<08057fdd>] set_signals+0x5d/0xe0 [42949509.890000] 08eb7a3c: [<0808d61b>] prep_new_page+0x6b/0x80 [42949509.890000] 08eb7a4c: [<0808d9a7>] buffered_rmqueue+0xf7/0x200 [42949509.890000] 08eb7a70: [<0808dbf2>] get_page_from_freelist+0x82/0xc0 [42949509.890000] 08eb7a74: [<0808dc08>] get_page_from_freelist+0x98/0xc0 [42949509.890000] 08eb7a90: [<0808dc7e>] __alloc_pages+0x4e/0x2d0 [42949509.890000] 08eb7ad8: [<0808df20>] __get_free_pages+0x20/0x60 [42949509.890000] 08eb7adc: [<080b3135>] __pollwait+0x35/0xb0 [42949509.890000] 08eb7af8: [<080ad134>] pipe_poll+0x24/0x90 [42949509.890000] 08eb7b14: [<080b3540>] do_select+0x290/0x310 [42949509.890000] 08eb7b6c: [<0805d1f3>] copy_from_user_skas+0x73/0x90 [42949509.890000] 08eb7b74: [<080b3100>] __pollwait+0x0/0xb0 [42949509.890000] 08eb7b94: [<080b38d8>] sys_select+0x2e8/0x530 [42949509.890000] 08eb7bc4: [<080a1a6b>] vfs_write+0xbb/0x130 [42949509.890000] 08eb7bc8: [<080617ed>] mconsole_config+0xad/0xc0 [42949509.890000] 08eb7bcc: [<080a1a7f>] vfs_write+0xcf/0x130 [42949509.890000] 08eb7c1c: [<0805cb99>] handle_syscall+0xb9/0xc0 [42949509.890000] 08eb7c78: [<08068650>] move_registers+0x30/0x50 [42949509.890000] 08eb7c8c: [<0805b934>] handle_trap+0x24/0xe0 [42949509.890000] 08eb7ca8: [<0805bdf0>] userspace+0x170/0x1b0 [42949509.890000] 08eb7ce0: [<0805cd9a>] force_flush_all_skas+0x2a/0x40 [42949509.890000] 08eb7cfc: [<0805c7cf>] fork_handler+0xaf/0xc0 [42949509.890000] 08eb7d1c: [<081a8c08>] __restore+0x0/0x8 [42949509.890000] 08eb7d5c: [<081a8cc1>] __kill+0x11/0x20 [42949509.890000] [42949509.890000] deactivate_all_fds failed, errno = 9 Appended is a full console dump of guest crashing. I'd like to help tracing this down, so if you have ideas then please communicate them. Best regards, Marcus
Checking that ptrace can change system call numbers...OK Checking syscall emulation patch for ptrace...OK Checking advanced syscall emulation patch for ptrace...OK Checking PROT_EXEC mmap in /tmp...OK Checking for the skas3 patch in the host: - /proc/mm...not found - PTRACE_FAULTINFO...not found - PTRACE_LDT...not found UML running in SKAS0 mode [42949372.960000] Linux version 2.6.15.4 ([EMAIL PROTECTED]) (gcc version 3.3.3 20040110 (prerelease) (Debian)) #4 Sun Feb 26 18:31:54 CET 2006 [42949372.960000] Built 1 zonelists [42949372.960000] Kernel command line: mem=16M ubd0=/cow/uml-t0/ubd0,/root/uml-images/uml-t0 eth0=tuntap,tap-uml-t0-0,fe:fd:10:00:00:00 eth1=tuntap,tap-uml-t0-1,fe:fd:10:00:00:01 con=pts con0=fd:0,fd:1 root=98:0 [42949372.960000] PID hash table entries: 128 (order: 7, 2048 bytes) [42949372.960000] Dentry cache hash table entries: 4096 (order: 2, 16384 bytes) [42949372.960000] Inode-cache hash table entries: 2048 (order: 1, 8192 bytes) [42949372.960000] Memory: 13556k available [42949373.210000] Mount-cache hash table entries: 512 [42949373.210000] Checking for host processor cmov support...Yes [42949373.210000] Checking for host processor xmm support...No [42949373.210000] Checking that host ptys support output SIGIO...Yes [42949373.210000] Checking that host ptys support SIGIO on close...No, enabling workaround [42949373.210000] Checking for /dev/anon on the host...Not available (open failed with errno 2) [42949373.210000] /usr/include/linux/aio_abi.h not present during build [42949373.210000] 2.6 host AIO support not used - falling back to I/O thread [42949373.210000] NET: Registered protocol family 16 [42949373.210000] mconsole (version 2) initialized on /home/uml-services//.uml/uml-t0/mconsole [42949373.210000] Netdevice 0 (fe:fd:10:00:00:00) : TUN/TAP backend - [42949373.210000] Netdevice 1 (fe:fd:10:00:00:01) : TUN/TAP backend - [42949373.210000] io scheduler noop registered [42949373.210000] io scheduler cfq registered [42949373.210000] netem: version 1.1 [42949373.210000] u32 classifier [42949373.210000] Perfomance counters on [42949373.210000] input device check on [42949373.210000] NET: Registered protocol family 2 [42949373.330000] IP route cache hash table entries: 256 (order: -2, 1024 bytes) [42949373.330000] TCP established hash table entries: 1024 (order: 0, 4096 bytes) [42949373.330000] TCP bind hash table entries: 1024 (order: 0, 4096 bytes) [42949373.330000] TCP: Hash tables configured (established 1024 bind 1024) [42949373.330000] TCP reno registered [42949373.330000] TCP bic registered [42949373.330000] NET: Registered protocol family 1 [42949373.330000] NET: Registered protocol family 17 [42949373.330000] Ebtables v2.0 registered [42949373.620000] Initialized stdio console driver [42949373.620000] Console initialized on /dev/tty0 [42949373.620000] Initializing software serial port version 1 [42949373.620000] Creating "/cow/uml-t0/ubd0" as COW file for "/root/uml-images/uml-t0" [42949373.620000] ubda: unknown partition table [42949373.680000] kjournald starting. Commit interval 5 seconds [42949373.680000] EXT3-fs: mounted filesystem with ordered data mode. [42949373.680000] VFS: Mounted root (ext3 filesystem) readonly. [42949374.330000] line_ioctl: tty0: ioctl KDSIGACCEPT called INIT: version 2.84 booting + export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin + PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin + mount -n -t devpts devpts /dev/pts + mount -n -t proc proc /proc + mount -n -t tmpfs volatile /tmp + hostname debian + ip addr add dev lo 127.0.0.1/8 + ip link set dev lo up + brctl addbr br0 + ip link set dev br0 up + brctl addif br0 eth0 [42949375.990000] device eth0 entered promiscuous mode + ip link set dev eth0 up [42949375.990000] br0: port 1(eth0) entering learning state + brctl addif br0 eth1 [42949375.990000] device eth1 entered promiscuous mode + ip link set dev eth1 up [42949375.990000] br0: port 2(eth1) entering learning state + ip addr add dev br0 192.168.1.70/24 brd + + ip route add default via 192.168.1.10 INIT: Entering runlevel: 2 [42949376.160000] Virtual console 3 assigned device '/dev/pts/42' [42949376.160000] Virtual console 2 assigned device '/dev/pts/43' [42949376.210000] Virtual console 4 assigned device '/dev/pts/44' [42949376.210000] Virtual console 1 assigned device '/dev/pts/45' [42949376.300000] Virtual console 5 assigned device '/dev/pts/46' PAM_unix[29]: (login) session opened for user root by (uid=0) PAM_unix[32]: (login) session opened for user root by (uid=0) PAM_unix[30]: (login) session opened for user root by (uid=0) PAM_unix[31]: (login) session opened for user root by (uid=0) PAM_unix[28]: (login) session opened for user root by (uid=0) login[29]: ROOT LOGIN on `tty2' login[32]: ROOT LOGIN on `tty5' login[30]: ROOT LOGIN on `tty3' login[31]: ROOT LOGIN on `tty4' login[28]: ROOT LOGIN on `tty1' [42949390.990000] br0: topology change detected, propagating [42949390.990000] br0: port 1(eth0) entering forwarding state [42949390.990000] br0: topology change detected, propagating [42949390.990000] br0: port 2(eth1) entering forwarding state [42949433.910000] EXT3-fs warning: maximal mount count reached, running e2fsck is recommended [42949433.910000] EXT3 FS on ubda, internal journal [42949509.890000] Kernel panic - not syncing: Kernel mode signal 7 [42949509.890000] [42949509.890000] EIP: 0073:[<402237ee>] CPU: 0 Not tainted ESP: 007b:bf8038e4 EFLAGS: 00000283 [42949509.890000] Not tainted [42949509.890000] EAX: ffffffda EBX: 00000005 ECX: bf8039bc EDX: bf80393c [42949509.890000] ESI: 00000000 EDI: bf803934 EBP: bf803a3c DS: 007b ES: 007b [42949509.890000] 08eb7770: [<0807ce4c>] notifier_call_chain+0x1c/0x40 [42949509.890000] 08eb778c: [<0806ef6b>] panic+0x4b/0xf0 [42949509.890000] 08eb77a0: [<08059e3b>] relay_signal+0x7b/0x80 [42949509.890000] 08eb77bc: [<08059e8f>] bus_handler+0x4f/0x60 [42949509.890000] 08eb77d0: [<0805ce28>] sig_handler_common_skas+0x78/0xd0 [42949509.890000] 08eb77f0: [<0806770f>] sig_handler+0xf/0x20 [42949509.890000] 08eb77fc: [<081a8c08>] __restore+0x0/0x8 [42949509.890000] 08eb783c: [<080b313e>] __pollwait+0x3e/0xb0 [42949509.890000] 08eb789c: [<08057eb4>] change_signals+0x34/0x60 [42949509.890000] 08eb7948: [<08092a57>] __pagevec_lru_add+0x97/0xd0 [42949509.890000] 08eb79a0: [<08057f32>] enable_mask+0x32/0x40 [42949509.890000] 08eb79ac: [<08057fdd>] set_signals+0x5d/0xe0 [42949509.890000] 08eb7a3c: [<0808d61b>] prep_new_page+0x6b/0x80 [42949509.890000] 08eb7a4c: [<0808d9a7>] buffered_rmqueue+0xf7/0x200 [42949509.890000] 08eb7a70: [<0808dbf2>] get_page_from_freelist+0x82/0xc0 [42949509.890000] 08eb7a74: [<0808dc08>] get_page_from_freelist+0x98/0xc0 [42949509.890000] 08eb7a90: [<0808dc7e>] __alloc_pages+0x4e/0x2d0 [42949509.890000] 08eb7ad8: [<0808df20>] __get_free_pages+0x20/0x60 [42949509.890000] 08eb7adc: [<080b3135>] __pollwait+0x35/0xb0 [42949509.890000] 08eb7af8: [<080ad134>] pipe_poll+0x24/0x90 [42949509.890000] 08eb7b14: [<080b3540>] do_select+0x290/0x310 [42949509.890000] 08eb7b6c: [<0805d1f3>] copy_from_user_skas+0x73/0x90 [42949509.890000] 08eb7b74: [<080b3100>] __pollwait+0x0/0xb0 [42949509.890000] 08eb7b94: [<080b38d8>] sys_select+0x2e8/0x530 [42949509.890000] 08eb7bc4: [<080a1a6b>] vfs_write+0xbb/0x130 [42949509.890000] 08eb7bc8: [<080617ed>] mconsole_config+0xad/0xc0 [42949509.890000] 08eb7bcc: [<080a1a7f>] vfs_write+0xcf/0x130 [42949509.890000] 08eb7c1c: [<0805cb99>] handle_syscall+0xb9/0xc0 [42949509.890000] 08eb7c78: [<08068650>] move_registers+0x30/0x50 [42949509.890000] 08eb7c8c: [<0805b934>] handle_trap+0x24/0xe0 [42949509.890000] 08eb7ca8: [<0805bdf0>] userspace+0x170/0x1b0 [42949509.890000] 08eb7ce0: [<0805cd9a>] force_flush_all_skas+0x2a/0x40 [42949509.890000] 08eb7cfc: [<0805c7cf>] fork_handler+0xaf/0xc0 [42949509.890000] 08eb7d1c: [<081a8c08>] __restore+0x0/0x8 [42949509.890000] 08eb7d5c: [<081a8cc1>] __kill+0x11/0x20 [42949509.890000] [42949509.890000] deactivate_all_fds failed, errno = 9
