Re: [Samba] tdb locking issue - Solaris 10 and Samba 3.0.33
On Fri, Oct 23, 2009 at 03:34:29PM -0700, Volker Lendecke wrote: > You need to find the one holding the lock the others are > waiting for and see what that guy is blocking on. Everything > else is pure speculation. It appears mdb is what I want to use. See this[1] page if anyone is curious. Unfortunately, I kill -9'd one of the processes that was hung. As a result the parent died and all its children, so the fouled up environment I was in is now gone. I'll wait for this to reoccur and then see if I can track down the culprit. Thanks for the guidance, Ray [1] http://utcc.utoronto.ca/~cks/space/blog/solaris/ListingFileLocks -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] tdb locking issue - Solaris 10 and Samba 3.0.33
On Fri, Oct 23, 2009 at 03:28:18PM -0700, Ray Van Dolson wrote: > I bet you're right. So theoretically if I figure out how to track down > that lock, it is likely held by my NFS server... No, the NFS server is not holding a lock. > Hrm. But would it make sense that other smbd processes are apparently > still accessing brlock.tdb OK for different files? Yes, there can be many concurrent accesses to brlock.tdb to different hash chains. > My low level understanding of the locking is lacking a bit. :) > > Anyways, I'm guessing if this turns out to be NFS related as we > suspect, the only fix is to _not_ share with NFS. :) You need to find the one holding the lock the others are waiting for and see what that guy is blocking on. Everything else is pure speculation. Volker pgpmsqF86Hg3t.pgp Description: PGP signature -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] tdb locking issue - Solaris 10 and Samba 3.0.33
On Fri, Oct 23, 2009 at 03:21:35PM -0700, Volker Lendecke wrote: > On Fri, Oct 23, 2009 at 03:19:03PM -0700, Ray Van Dolson wrote: > > Hmm, yes, there is some of that. I guess I was thrown off by the fact > > that the lock appeared to be on the .tdb file and the .tdb file > > wouldn't be accessed by any NFS client. > > > > Could another process be holding the lock on the .tdb file while it > > waits for access to the underlying file within the requested share? > > That's exactly what I'm thinking. An smbd holding the > brlock.tdb lock, waiting for pread while your NFS server has > gone to lunch. > > Volker I bet you're right. So theoretically if I figure out how to track down that lock, it is likely held by my NFS server... Hrm. But would it make sense that other smbd processes are apparently still accessing brlock.tdb OK for different files? My low level understanding of the locking is lacking a bit. :) Anyways, I'm guessing if this turns out to be NFS related as we suspect, the only fix is to _not_ share with NFS. :) Ray -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] tdb locking issue - Solaris 10 and Samba 3.0.33
On Fri, Oct 23, 2009 at 03:19:03PM -0700, Ray Van Dolson wrote: > Hmm, yes, there is some of that. I guess I was thrown off by the fact > that the lock appeared to be on the .tdb file and the .tdb file > wouldn't be accessed by any NFS client. > > Could another process be holding the lock on the .tdb file while it > waits for access to the underlying file within the requested share? That's exactly what I'm thinking. An smbd holding the brlock.tdb lock, waiting for pread while your NFS server has gone to lunch. Volker pgpeHeLhp4qR8.pgp Description: PGP signature -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] tdb locking issue - Solaris 10 and Samba 3.0.33
On Fri, Oct 23, 2009 at 03:15:59PM -0700, Volker Lendecke wrote: > On Fri, Oct 23, 2009 at 03:09:06PM -0700, Ray Van Dolson wrote: > > I'm not sure how to do this either. Guess I could pfiles on every > > other PID on the system. > > See my other mail, this looks like a good hint. > > > > BTW, you don't happen to run something like samfs? > > > > > > > No SamFS on this environment. > > Or maybe re-exporting NFS? > Hmm, yes, there is some of that. I guess I was thrown off by the fact that the lock appeared to be on the .tdb file and the .tdb file wouldn't be accessed by any NFS client. Could another process be holding the lock on the .tdb file while it waits for access to the underlying file within the requested share? Ray -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] tdb locking issue - Solaris 10 and Samba 3.0.33
On Fri, Oct 23, 2009 at 02:56:48PM -0700, Ray Van Dolson wrote: > On Fri, Oct 23, 2009 at 02:41:46PM -0700, Volker Lendecke wrote: > > On Fri, Oct 23, 2009 at 02:18:19PM -0700, Ray Van Dolson wrote: > > > # pfiles -F 25067 > > > 10: S_IFREG mode:0644 dev:85,60 ino:4630 uid:0 gid:0 size:327680 > > > O_RDWR|O_LARGEFILE > > > advisory read lock set by process 21130 > > > /var/samba/locks/brlock.tdb > > > > Ahhh. What does process 21130 do right now? > > > > That is (was) the PID of the parent smbd process -- the one that spawns > all others. Really? Ok, that might be the CLEAR_IF_FIRST indicator lock. That's probably not the one your problem smbd blocks on. Solaris *must* have a way to figure out who is holding an fcntl lock others want Volker pgp2j4jT2wunn.pgp Description: PGP signature -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] tdb locking issue - Solaris 10 and Samba 3.0.33
On Fri, Oct 23, 2009 at 03:09:06PM -0700, Ray Van Dolson wrote: > I'm not sure how to do this either. Guess I could pfiles on every > other PID on the system. See my other mail, this looks like a good hint. > > BTW, you don't happen to run something like samfs? > > > > No SamFS on this environment. Or maybe re-exporting NFS? Volker pgpCrfpTThwXq.pgp Description: PGP signature -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] tdb locking issue - Solaris 10 and Samba 3.0.33
On Fri, Oct 23, 2009 at 02:37:15PM -0700, Volker Lendecke wrote: > On Fri, Oct 23, 2009 at 02:18:19PM -0700, Ray Van Dolson wrote: > > (Yes, I should upgrade Samba to 3.0.35). > > > > We're running the Sun provided Samba daemon (SUNWsmbau and friends) on > > Solaris 10 Generic_13-08 (sparc). > > > > Lots of Windows clients (mixed XP, 2003, 2008) hit this server and > > periodically we'll start seeing smbd processes begin piling up. These > > processes can't be killed with a normal kill -- only kill -9 will do > > the trick. > > Probably someone else is holding the same lock for some > reason and is stuck in a file system syscall. > > Under Linux you would look at /proc/locks to find that info, > no idea how to find the current lock holder under Solaris. > You need to find that one and see what syscall that guy is > stuck in. I'm not sure how to do this either. Guess I could pfiles on every other PID on the system. I know there are other PID's also in the same state, trying to get what appears to be the same lock... > > BTW, you don't happen to run something like samfs? > No SamFS on this environment. > Volker Thanks! Ray -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] tdb locking issue - Solaris 10 and Samba 3.0.33
On Fri, Oct 23, 2009 at 02:41:46PM -0700, Volker Lendecke wrote: > On Fri, Oct 23, 2009 at 02:18:19PM -0700, Ray Van Dolson wrote: > > # pfiles -F 25067 > > 10: S_IFREG mode:0644 dev:85,60 ino:4630 uid:0 gid:0 size:327680 > > O_RDWR|O_LARGEFILE > > advisory read lock set by process 21130 > > /var/samba/locks/brlock.tdb > > Ahhh. What does process 21130 do right now? > That is (was) the PID of the parent smbd process -- the one that spawns all others. Ray -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] tdb locking issue - Solaris 10 and Samba 3.0.33
On Fri, Oct 23, 2009 at 02:18:19PM -0700, Ray Van Dolson wrote: > # pfiles -F 25067 > 10: S_IFREG mode:0644 dev:85,60 ino:4630 uid:0 gid:0 size:327680 > O_RDWR|O_LARGEFILE > advisory read lock set by process 21130 > /var/samba/locks/brlock.tdb Ahhh. What does process 21130 do right now? Volker pgpaQszhczBYj.pgp Description: PGP signature -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] tdb locking issue - Solaris 10 and Samba 3.0.33
On Fri, Oct 23, 2009 at 02:18:19PM -0700, Ray Van Dolson wrote: > (Yes, I should upgrade Samba to 3.0.35). > > We're running the Sun provided Samba daemon (SUNWsmbau and friends) on > Solaris 10 Generic_13-08 (sparc). > > Lots of Windows clients (mixed XP, 2003, 2008) hit this server and > periodically we'll start seeing smbd processes begin piling up. These > processes can't be killed with a normal kill -- only kill -9 will do > the trick. Probably someone else is holding the same lock for some reason and is stuck in a file system syscall. Under Linux you would look at /proc/locks to find that info, no idea how to find the current lock holder under Solaris. You need to find that one and see what syscall that guy is stuck in. BTW, you don't happen to run something like samfs? Volker pgp6Rf0s0VXE9.pgp Description: PGP signature -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
