Re: LOR: vfs_lookup.c:501 ffs_vnops.c:261 vfs_subr.c:2134
Hi Folks, I just updated this particular systems to current as of this evening (2011-07-05 7:22pm EDT) and am still seeing this LOR. lock order reversal: 1st 0xfe003c893818 ufs (ufs) @ /usr/src.2011-07-05_7.22pm_EDT/sys/kern/vfs_lookup.c:501 2nd 0xff9f0c7eeef8 bufwait (bufwait) @ /usr/src.2011-07-05_7.22pm_EDT/sys/ufs/ffs/ffs_vnops.c:261 3rd 0xfe003c8689f8 ufs (ufs) @ /usr/src.2011-07-05_7.22pm_EDT/sys/kern/vfs_subr.c:2134 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x807 __lockmgr_args() at __lockmgr_args+0xd42 ffs_lock() at ffs_lock+0x8c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x47 vget() at vget+0x7b vfs_hash_get() at vfs_hash_get+0xd5 ffs_vgetf() at ffs_vgetf+0x48 softdep_sync_buf() at softdep_sync_buf+0x568 ffs_syncvnode() at ffs_syncvnode+0x293 ffs_truncate() at ffs_truncate+0x4c4 ufs_direnter() at ufs_direnter+0x6ed ufs_makeinode() at ufs_makeinode+0x250 VOP_CREATE_APV() at VOP_CREATE_APV+0x8d vn_open_cred() at vn_open_cred+0x46a kern_openat() at kern_openat+0x17f syscallenter() at syscallenter+0x1aa syscall() at syscall+0x4c Xfast_syscall() at Xfast_syscall+0xdd --- syscall (5, FreeBSD ELF64, open), rip = 0x800936b9c, rsp = 0x7fffdac8, rbp = 0 --- Cross-posting to -fs for more visibility. Thoughts welcome. -John - John's Original Message - > Hi folks, > >I'm seeing the following LOR in dmesg after my latest update this evening. > > # uname -a > FreeBSD zfscarp3p 9.0-CURRENT FreeBSD 9.0-CURRENT #0: Fri Jun 17 22:36:45 EDT > 2011 root@zfscarp3p/usr/obj/usr/src.2011-06-17_9.36pm_EDT/sys/GENERIC amd6 > > WARNING: WITNESS option enabled, expect reduced performance. > Trying to mount root from ufs:/dev/da56s1a [rw]... > lock order reversal: > 1st 0xfe003d2f9278 ufs (ufs) @ > /usr/src.2011-06-17_9.36pm_EDT/sys/kern/vfs_lookup.c:501 > 2nd 0xff9f0c7f0e58 bufwait (bufwait) @ > /usr/src.2011-06-17_9.36pm_EDT/sys/ufs/ffs/ffs_vnops.c:261 > 3rd 0xfe003d371278 ufs (ufs) @ > /usr/src.2011-06-17_9.36pm_EDT/sys/kern/vfs_subr.c:2134 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > kdb_backtrace() at kdb_backtrace+0x37 > _witness_debugger() at _witness_debugger+0x2e > witness_checkorder() at witness_checkorder+0x807 > __lockmgr_args() at __lockmgr_args+0xd42 > ffs_lock() at ffs_lock+0x8c > VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b > _vn_lock() at _vn_lock+0x47 > vget() at vget+0x7b > vfs_hash_get() at vfs_hash_get+0xd5 > ffs_vgetf() at ffs_vgetf+0x48 > softdep_sync_buf() at softdep_sync_buf+0x56a > ffs_syncvnode() at ffs_syncvnode+0x293 > ffs_truncate() at ffs_truncate+0x4c4 > ufs_direnter() at ufs_direnter+0x6ed > ufs_makeinode() at ufs_makeinode+0x250 > VOP_CREATE_APV() at VOP_CREATE_APV+0x8d > vn_open_cred() at vn_open_cred+0x46a > kern_openat() at kern_openat+0x17f > syscallenter() at syscallenter+0x1aa > syscall() at syscall+0x4c > Xfast_syscall() at Xfast_syscall+0xdd > --- syscall (5, FreeBSD ELF64, open), rip = 0x800936b7c, rsp = > 0x7fffdac8, rbp = 0 --- > bce1: Gigabit link up! > >Only seems to happen once at boot time. > > Thanks, > John ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: avl_find() panic
On Wed, Jul 06, 2011 at 12:21:55AM +, John wrote: >I have a system that panic'd this morning, 4 day old current > (2011-07-01_11.45pm). Message typed in from the console immediately > after reboot. OS on ufs, data volumes on zfs. > > ZFS filesystem version 5 > ZFS storage pool version 28 > panic: avl_find() succedded inside avl_find() > >Unfortunately, I don't have a traceback for this. > >The comment in avl.c makes it seem like the avl code is enforcing > uniqueness in calling code, esp. where it talks about kernel vs > userland. > >I'll followup with more info if this replicates. Cross-posting is generally shunned, but since this is a current thing, adding freebsd-current to the CC list. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: named crashes on assertion in rbtdb.c o? sparc64/SMP
On 06/28/2011 08:58, Marius Strobl wrote: Uhm, we once fixed a problem in the MD atomic implementation which still seems to present in the ISC copy. Could you please test whether the following patch makes a difference? http://people.freebsd.org/~marius/sparc64_isc_atomic.h.diff I haven't seen any verification from the OP that this patch solved the problem, however it did pass 'make universe' on both 9-current and RELENG_8, so I've committed it to those 2 branches along with the recent update. I'll also submit it upstream. Thanks, Doug -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: isp(4) timeout
On Tue, 5 Jul 2011, Anton Shterenlikht wrote: I somehow missed ispfw(4). Now added device ispfw to the kernel. Is this sufficient? Overkill, but sufficient. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: isp(4) timeout
On Tue, Jul 05, 2011 at 07:34:44AM -0700, Matthew Jacob wrote: > On 7/5/2011 1:39 AM, Anton Shterenlikht wrote: > > ... > >dmesg: > > > >http://seis.bris.ac.uk/~mexas/freebsd/ia64/rx2620/ZEEV/dmesg.boot > > > >Many thanks > >Anton > > > > >isp0: Board Type 2422, Chip Revision 0x2, resident F/W Revision 4.0.90 > > Add the line > > isp_2400_load="YES" > > to /boot/loader.conf. I somehow missed ispfw(4). Now added device ispfw to the kernel. Is this sufficient? Many thanks Anton -- Anton Shterenlikht Room 2.6, Queen's Building Mech Eng Dept Bristol University University Walk, Bristol BS8 1TR, UK Tel: +44 (0)117 331 5944 Fax: +44 (0)117 929 4423 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: isp(4) timeout
On Tue, Jul 05, 2011 at 08:13:02AM -0700, Marcel Moolenaar wrote: > > On Jul 5, 2011, at 1:39 AM, Anton Shterenlikht wrote: > > > On Thu, Jun 30, 2011 at 05:11:24AM -0700, Matthew Jacob wrote: > >> On 6/30/2011 3:25 AM, Anton Shterenlikht wrote: > >>> I see in my logs: > >>> > >>> isp0: Polled Mailbox Command (0x54) Timeout (50us) (started @ > >>> isp_plogx:2122) > >>> isp0: Mailbox Command 'EXECUTE IOCB A64' failed (TIMEOUT) > >>> isp0: Chan 0 PLOGI 0x010500 failed > >>> isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ > >>> isp_getpdb:2307) > >>> isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) > >> > >> More details please. > >> > >> These errors indicate failures to execute commands that try and figure > >> out what's on a fabric and then log into devices on the fabric. Knowing > >> what hardware you have (QLogic card version), what FreeBSD release you > >> are running, would help. A verbose dmesg would be useful. > > > > I got it again. But this time the network seems fine. > > If you haven't updated to the latest sources, please do so. > If you did already, please make sure you don't have PREEMPTION > configured. updated to r223796 (no PREEMPTION in kernel). Had this hang on reboot: KDB: stack backtrace: getenv with the following non-sleepable locks held: exclusive sleep mutex vnode interlock (vnode interlock) r = 0 (0xe00011587ce8) locked @ /usr/src/sys/fs/devfs/devfs_vnops.c:405 KDB: stack backtrace: getenv with the following non-sleepable locks held: exclusive sleep mutex vnode interlock (vnode interlock) r = 0 (0xe00011587ce8) locked @ /usr/src/sys/fs/devfs/devfs_vnops.c:405 KDB: stack backtrace: (repeated lots of times) no panic, just hang. Had to reset power via MP. Seems fine after reboot. Anything else I should check? Many thanks Anton -- Anton Shterenlikht Room 2.6, Queen's Building Mech Eng Dept Bristol University University Walk, Bristol BS8 1TR, UK Tel: +44 (0)117 331 5944 Fax: +44 (0)117 929 4423 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: isp(4) timeout
On Jul 5, 2011, at 1:39 AM, Anton Shterenlikht wrote: > On Thu, Jun 30, 2011 at 05:11:24AM -0700, Matthew Jacob wrote: >> On 6/30/2011 3:25 AM, Anton Shterenlikht wrote: >>> I see in my logs: >>> >>> isp0: Polled Mailbox Command (0x54) Timeout (50us) (started @ >>> isp_plogx:2122) >>> isp0: Mailbox Command 'EXECUTE IOCB A64' failed (TIMEOUT) >>> isp0: Chan 0 PLOGI 0x010500 failed >>> isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ >>> isp_getpdb:2307) >>> isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) >> >> More details please. >> >> These errors indicate failures to execute commands that try and figure >> out what's on a fabric and then log into devices on the fabric. Knowing >> what hardware you have (QLogic card version), what FreeBSD release you >> are running, would help. A verbose dmesg would be useful. > > I got it again. But this time the network seems fine. If you haven't updated to the latest sources, please do so. If you did already, please make sure you don't have PREEMPTION configured. FYI, -- Marcel Moolenaar mar...@xcllnt.net ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: isp(4) timeout
On 7/5/2011 1:39 AM, Anton Shterenlikht wrote: ... dmesg: http://seis.bris.ac.uk/~mexas/freebsd/ia64/rx2620/ZEEV/dmesg.boot Many thanks Anton isp0: Board Type 2422, Chip Revision 0x2, resident F/W Revision 4.0.90 Add the line isp_2400_load="YES" to /boot/loader.conf. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: isp(4) timeout
On Thu, Jun 30, 2011 at 05:11:24AM -0700, Matthew Jacob wrote: > On 6/30/2011 3:25 AM, Anton Shterenlikht wrote: > >I see in my logs: > > > >isp0: Polled Mailbox Command (0x54) Timeout (50us) (started @ > >isp_plogx:2122) > >isp0: Mailbox Command 'EXECUTE IOCB A64' failed (TIMEOUT) > >isp0: Chan 0 PLOGI 0x010500 failed > >isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ > >isp_getpdb:2307) > >isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) > > More details please. > > These errors indicate failures to execute commands that try and figure > out what's on a fabric and then log into devices on the fabric. Knowing > what hardware you have (QLogic card version), what FreeBSD release you > are running, would help. A verbose dmesg would be useful. I got it again. But this time the network seems fine. ZEEV> uname -a FreeBSD mech-as28.men.bris.ac.uk 9.0-CURRENT FreeBSD 9.0-CURRENT #2 r221340: Mon May 2 23:27:58 BST 2011 r...@mech-as28.men.bris.ac.uk:/usr/obj/usr/src/sys/ZEEV ia64 ZEEV> isp0@pci0:192:1:0: class=0x0c0400 card=0x12d6103c chip=0x24221077 rev=0x02 hdr=0x00 vendor = 'QLogic Corporation' device = 'QLogic PCI to Fibre Channel Host Adapter for QLA2460 (ISP2422)' class = serial bus subclass = Fibre Channel >From the logs: +isp0: Polled Mailbox Command (0x54) Timeout (50us) (started @ isp_plogx:2122) +isp0: Mailbox Command 'EXECUTE IOCB A64' failed (TIMEOUT) +isp0: Chan 0 PLOGI 0x010500 failed +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: mailbox cmd (0x4000) with no waiters +isp0: Polled Mailbox Command (0x54) Timeout (50us) (started @ isp_plogx:2122) +isp0: Mailbox Command 'EXECUTE IOCB A64' failed (TIMEOUT) +isp0: Chan 0 PLOGI 0x010500 failed +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getpdb:2307) +isp0: Mailbox Command 'GET PORT DATABASE' failed (TIMEOUT) +isp0: Polled Mailbox Command (0x64) Timeout (25us) (started @ isp_getp