On 09.02.2014 13:58, Dennis Jacobfeuerborn wrote:
On 09.02.2014 08:33, Michael Schwartzkopff wrote:
Am Sonntag, 9. Februar 2014, 02:56:55 schrieb Dennis Jacobfeuerborn:
Hi,
i have setup a nfsv3 HA cluster before and that works fine but now I'm
trying to move to v4 and run into problems with the lease grace period.
The grace period on CentOS 6 is 90 seconds and that limits how quickly
the fail-over can happen. The file /etc/sysconfig/nfs contains a
variable NFSD_V4_GRACE to control this but it doesn't get applied. The
reason is that the file /proc/fs/nfsd/nfsv4gracetime is not writable:
[root@nfs1 init.d]# echo 10 > /proc/fs/nfsd/nfsv4gracetime
-bash: echo: write error: Device or resource busy
Apparently the nfs daemon is the culprit here. If it is running you get
this error. The reason for why the value is not set probably even right
after the kernel module is loaded seems to have a different cause (see
below).
You can write the paramerter only after the nfsd kernel module is
loaded. As
far as I can remember the init script sets the leasetime according to the
config. You have to add a config option and patch your init script to
set the
gracetime at the same point in the script.
Does anyone know what the proper way is to reduce this value?
In RHEL 6.5 it is a option in the config file.
Yes this is the NFSD_V4_GRACE option in /etc/sysconfig/nfs I was
mentioning above and in the init script this gets set right after the
kernel module is loaded but this doesn't seem to work. When I uncomment
this option and set it to 10 seconds after the service is started the
value for the grace time is still 90 for /proc/fs/nfsd/nfsv4gracetime.
I've also looked for a parameter for the kernel module itself to set
this when the module is loaded but that doesn't exist and now I'm
wondering how to set this value at all.
Do you know what the criterion is that determines when this value is
settable? The module is loaded the whole time so that alone can't be the
only factor here and there must be some additional constraint that now
prevents me from updating this value.
So I investigated this further and there seem to be two distinct issues:
1) NFSD_V4_GRACE only gets applied the second time the nfs service is
started
Check out the following sequence that I executed right after booting the
system (pacemaker is not running at this point):
[root@nfs2 ~]# cat /etc/sysconfig/nfs |grep GRACE
NFSD_V4_GRACE=10
[root@nfs2 ~]# service nfs start
Starting NFS services: [ OK ]
Starting NFS quotas: [ OK ]
Starting NFS mountd: [ OK ]
Starting NFS daemon: [ OK ]
Starting RPC idmapd: [ OK ]
[root@nfs2 ~]# cat /proc/fs/nfsd/nfsv4gracetime
90
[root@nfs2 ~]# service nfs stop
Shutting down NFS daemon: [ OK ]
Shutting down NFS mountd: [ OK ]
Shutting down NFS quotas: [ OK ]
Shutting down RPC idmapd: [ OK ]
[root@nfs2 ~]# cat /proc/fs/nfsd/nfsv4gracetime
90
[root@nfs2 ~]# service nfs start
Starting NFS services: [ OK ]
Starting NFS quotas: [ OK ]
Starting NFS mountd: [ OK ]
Starting NFS daemon: [ OK ]
Starting RPC idmapd: [ OK ]
[root@nfs2 ~]# cat /proc/fs/nfsd/nfsv4gracetime
10
As you can see the grace time is only applied after I stopped and
started the service the second time. Not sure why this is the case.
2) When started via Pacemaker the value of /proc/fs/nfsd/nfsv4gracetime
is always set to the default value of 90.
It appears that starting the nfs service from the shell vs. starting it
using Pacemaker makes a difference and when started with pacemaker the
value from /etc/sysconfig/nfs is not applied and even when the value has
been set manually to e.g. 20 after pacemaker starts the service it will
be reset to the default 90.
What is strange about this is that the init script simply sources the
sysconfig file so I'm not sure how the environment could prevent the
value from getting applied.
Regards,
Dennis
_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker
Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
