Following up to Jorgen Lundman's last post about this, I can now reliably
reproduce the problem.
I used the following Perl script, called "lock-test":
#!/usr/bin/env perl
#
# Leak some locks
use Fcntl qw/:flock SEEK_SET/;
use Getopt::Long;
my $iterations=100;
my $uids=1000;
my $result = GetOptions("iterations=i" => \$iterations,
"uids=i" => \$uids);
for my $i (1..$iterations) {
my $pid;
if( ( $pid = fork ) > 0) {
print "Parent ($i/$$): $pid\n";
} else {
my $fh;
$> = $< = int(rand($uids)+1000);
print "$$: changed uid to $<\n";
if(-f "tmp/counter$<") {
open($fh, "+<", "tmp/counter$<")
or die "Couldn't open tmp/counter$<: $!";
} else {
open($fh, "+>", "tmp/counter$<");
}
flock($fh,LOCK_EX);
print "$$: acquired lock\n";
my $count = <$fh>;
$count++;
seek($fh,0,SEEK_SET);
print $fh $count;
print "$$: counted\n";
flock($fh, LOCK_UN);
exit;
}
}
my $child;
do {
$child = waitpid(-1,WNOHANG);
print "PID $child exited\n";
} while $child > 0;
I ran it on a client that mounted a filesystem from the server, within
the filesystem, in a world-writeable directory. The chief thing is that
it does its thing using lots of UIDs.
The internal parameter that leaks on the server is OpenOwner. I start
with a freshly-started nfsd on our NFS server and it looks like this
after its clients have mounted the appropriate filesystems:
x4500# echo '::rfs4_db' | mdb -k | grep OpenOwner
fffffe8a351e4f00 OpenOwner 00000000 0005 2047 fffffe8b2046ef00 0001 0001
As you can see, OpenOwner count is 5.
After running "lock-test" once, it looks like this:
x4500# echo '::rfs4_db' | mdb -k | grep OpenOwner
fffffe8a351e4f00 OpenOwner 00000000 0102 2047 fffffe8b2046ef00 0001 0001
This number stays stable after the test is finished--I would expect it to
gradually subside after a while, like the rest of the values you see when you
do "echo '::rfs4_db | mdb -k'". When I repeat the test, the number goes up
progressively with each iteration:
fffffe8a351e4f00 OpenOwner 00000000 0198 2047 fffffe8b2046ef00 0001 0001
fffffe8a351e4f00 OpenOwner 00000000 0291 2047 fffffe8b2046ef00 0001 0001
fffffe8a351e4f00 OpenOwner 00000000 0385 2047 fffffe8b2046ef00 0001 0001
If I try the test with a smaller number of uids ("lock-test -u 50",
say), the number goes up a little bit, and then stays stable:
fffffe8a351e4f00 OpenOwner 00000000 0425 2047 fffffe8b2046ef00 0001 0001
fffffe8a351e4f00 OpenOwner 00000000 0432 2047 fffffe8b2046ef00 0001 0001
fffffe8a351e4f00 OpenOwner 00000000 0432 2047 fffffe8b2046ef00 0001 0001
fffffe8a351e4f00 OpenOwner 00000000 0432 2047 fffffe8b2046ef00 0001 0001
The point where it starts leaking is with 60 uids:
fffffe8a351e4f00 OpenOwner 00000000 0444 2047 fffffe8b2046ef00 0001 0001
fffffe8a351e4f00 OpenOwner 00000000 0445 2047 fffffe8b2046ef00 0001 0001
fffffe8a351e4f00 OpenOwner 00000000 0448 2047 fffffe8b2046ef00 0001 0001
fffffe8a351e4f00 OpenOwner 00000000 0449 2047 fffffe8b2046ef00 0001 0001
At 59 uids, it stabilizes:
fffffe8a351e4f00 OpenOwner 00000000 0450 2047 fffffe8b2046ef00 0001 0001
fffffe8a351e4f00 OpenOwner 00000000 0451 2047 fffffe8b2046ef00 0001 0001
fffffe8a351e4f00 OpenOwner 00000000 0451 2047 fffffe8b2046ef00 0001 0001
fffffe8a351e4f00 OpenOwner 00000000 0451 2047 fffffe8b2046ef00 0001 0001
Also, the NFS share becomes unusable when the number of OpenOwners hits 1048576.
I hope this is helpful to people trying to track down the problem!
--Dave
_______________________________________________
nfs-discuss mailing list
[email protected]
