FreeBSD 9 + ZFS + MPS
I'm seeing some odd behavior on FreeBSD 9 with ZFS and an MPS controller. For two or three weeks, this configuration was working like a champ. It's a Bacula storage daemon server and was getting considerable usage - our backup set is in the multiple-terabytes-per-week of data being written to the disk. However, during the last two weeks something has broken. Starting on about July 15th or so we started to loose the data zpool which is connected via the mps driver. The disks are an external array of 32 drives, etc 2TB. When I did a zpool status, I saw that some of the drives had dropped out of the array. Rebooting the server brought them back until the next heavy write time. On July 22nd I did a make world to see if there were any kernel updates that may have fixed the problem, but any updates that may have happened did not seem to help. I'm including the last entry in my dmesg.yesterday file below. Have I stumbled upon a bug? lock order reversal: 1st 0xfe0019cf6db8 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:829 2nd 0xfe0019cf69f8 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2134 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x807 __lockmgr_args() at __lockmgr_args+0xd42 vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x47 vget() at vget+0x7b devfs_allocv() at devfs_allocv+0x13f devfs_root() at devfs_root+0x4d vfs_donmount() at vfs_donmount+0x988 nmount() at nmount+0x63 syscallenter() at syscallenter+0x1aa syscall() at syscall+0x4c Xfast_syscall() at Xfast_syscall+0xdd --- syscall (378, FreeBSD ELF64, nmount), rip = 0x800ab4dfc, rsp = 0x7fffccc8, rbp = 0x801009048 --- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD 9 + ZFS + MPS
On 7/25/11 11:32 AM, Tim Gustafson wrote: I'm seeing some odd behavior on FreeBSD 9 with ZFS and an MPS controller. For two or three weeks, this configuration was working like a champ. It's a Bacula storage daemon server and was getting considerable usage - our backup set is in the multiple-terabytes-per-week of data being written to the disk. However, during the last two weeks something has broken. Starting on about July 15th or so we started to loose the data zpool which is connected via the mps driver. The disks are an external array of 32 drives, etc 2TB. When I did a zpool status, I saw that some of the drives had dropped out of the array. Rebooting the server brought them back until the next heavy write time. On July 22nd I did a make world to see if there were any kernel updates that may have fixed the problem, but any updates that may have happened did not seem to help. I'm including the last entry in my dmesg.yesterday file below. Have I stumbled upon a bug? lock order reversal: 1st 0xfe0019cf6db8 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:829 2nd 0xfe0019cf69f8 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2134 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x807 __lockmgr_args() at __lockmgr_args+0xd42 vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x47 vget() at vget+0x7b devfs_allocv() at devfs_allocv+0x13f devfs_root() at devfs_root+0x4d vfs_donmount() at vfs_donmount+0x988 nmount() at nmount+0x63 syscallenter() at syscallenter+0x1aa syscall() at syscall+0x4c Xfast_syscall() at Xfast_syscall+0xdd --- syscall (378, FreeBSD ELF64, nmount), rip = 0x800ab4dfc, rsp = 0x7fffccc8, rbp = 0x801009048 --- I've seen similar messages on a ZFS-based system, when rebooting, and when unmounting a USB memory stick (curiously, with an msdosfs filesystem, though my hard drive contains ZFS) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
