Re: Major SMP problems with lstat/namei

2008-10-03 Thread Jeff Wheelhouse
On Oct 1, 2008, at 5:17 PM, John Baldwin wrote: It sounds like you missed some of the dirhash changes somehow, as dirhash no longer has any lockmgr stuff in it (and only ever did in HEAD). I've generated a patch though using svn. You can grab it from http://www.FreeBSD.org/~jhb/patches/ufs_lo

Re: panic: lockmgr on FreeBSD 7.0-RELEASE-p4 amd64

2008-09-25 Thread Jeff Wheelhouse
On Sep 25, 2008, at 3:53 PM, John Baldwin wrote: You can use KTR instead of printf perhaps and then use 'show ktr' from DDB. This won't have the same impact on timing as printf(). I would include PIDs in any KTR traces you do so it's easier to parse the interleaved entries from multiple C

Re: Major SMP problems with lstat/namei

2008-09-25 Thread Jeff Wheelhouse
On Sep 24, 2008, at 12:12 PM, John Baldwin wrote: Shared lookups only work on the NFS client in 6.x. I'm about to turn them on for UFS in HEAD (8.x) and will backport the needed fixes to 7.x after 7.1 (too risky to merge to 7.x this close to a release). OK, given all the patches you refer

Re: panic: lockmgr on FreeBSD 7.0-RELEASE-p4 amd64

2008-09-25 Thread Jeff Wheelhouse
On Sep 25, 2008, at 8:45 AM, John Baldwin wrote: It's probably the one just before the NDINIT (note that the return address in the call stack is pointing to the next instruction to be executed after the call to VOP_UNLOCK(), so sometimes it can end up referring to the next line in the sour

Re: Major SMP problems with lstat/namei

2008-09-25 Thread Jeff Wheelhouse
On Sep 25, 2008, at 10:51 AM, Dag-Erling Smørgrav wrote: Jeff Wheelhouse <[EMAIL PROTECTED]> writes: I've written a quick benchmark with a pair of tests to simplify/measure the problem. [...] Care to share? No problem: http://software.wheelhouse.org/rptest.tar.bz2 T

Re: panic: lockmgr on FreeBSD 7.0-RELEASE-p4 amd64

2008-09-24 Thread Jeff Wheelhouse
On Sep 24, 2008, at 12:34 PM, John Baldwin wrote: On Wednesday 24 September 2008 12:17:56 pm Jeff Wheelhouse wrote: panic: lockmgr: thread 0xff0050858350, not exclusive lock holder 0xff00074959f0 unlocking cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at

Re: Major SMP problems with lstat/namei

2008-09-24 Thread Jeff Wheelhouse
On Sep 24, 2008, at 2:10 PM, John Baldwin wrote: You can still use cvs to pull the revisions. All those e-mail msg's have the CVS revisions in them, too. If I'm ever to do anything that will benefit someone besides myself, it's worth my making the effort to learn SVN. We have coasted on

Re: panic: lockmgr on FreeBSD 7.0-RELEASE-p4 amd64

2008-09-24 Thread Jeff Wheelhouse
On Sep 24, 2008, at 2:10 PM, John Baldwin wrote: If possible, get a crashdump. gmirror. :( Thanks, Jeff ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL

Re: Major SMP problems with lstat/namei

2008-09-24 Thread Jeff Wheelhouse
On Sep 24, 2008, at 12:12 PM, John Baldwin wrote: Shared lookups only work on the NFS client in 6.x. I'm about to turn them on for UFS in HEAD (8.x) and will backport the needed fixes to 7.x after 7.1 (too risky to merge to 7.x this close to a release). Testers available, when you get to

Re: panic: lockmgr on FreeBSD 7.0-RELEASE-p4 amd64

2008-09-24 Thread Jeff Wheelhouse
On Sep 24, 2008, at 12:34 PM, John Baldwin wrote: On Wednesday 24 September 2008 12:17:56 pm Jeff Wheelhouse wrote: nullfs_mount() at nullfs_mount+0x244 Can you use gdb or the like to get the souce file/line for the nullfs_mount+0x244 frame? i.e. 'gdb /boot/kernel/kernel'

Re: Major SMP problems with lstat/namei

2008-09-24 Thread Jeff Wheelhouse
On Sep 24, 2008, at 6:12 AM, Ivan Voras wrote: There is nothing that can be done within the 6.x branch. 7.x contains many improvements but I think only 8.x will directly change the lockmgr and the namei cache. The best things you can try right now is to use 7-STABLE (or soon to be released 7.

Re: panic: lockmgr on FreeBSD 7.0-RELEASE-p4 amd64

2008-09-24 Thread Jeff Wheelhouse
--- syscall (378, FreeBSD ELF64, nmount), rip = 0x206845ac, rsp = 0x7fffdfc8, rbp = 0x7fffdfd0 --- Thanks, Jeff On Sep 23, 2008, at 11:51 AM, Jeff Wheelhouse wrote: Got the following panic overnight: panic: lockmgr: thread 0xff0053cda680, not exclusive lock holder

Major SMP problems with lstat/namei

2008-09-23 Thread Jeff Wheelhouse
We have encountered some serious SMP performance/scalability problems that we've tracked back to lstat/namei calls. I've written a quick benchmark with a pair of tests to simplify/measure the problem. Both tests use a tree of directories: the top level directory contains five subdirecto

panic: lockmgr on FreeBSD 7.0-RELEASE-p4 amd64

2008-09-23 Thread Jeff Wheelhouse
Got the following panic overnight: panic: lockmgr: thread 0xff0053cda680, not exclusive lock holder 0xff002d7da680 unlocking cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a panic() at panic+0x17a _lockmgr() at _lockmgr+0x872 VOP_UNLOCK_APV() at VO