NFS Cache
Greetings, We are having problems with Record Based Log Shipping from our PostGreSQL database server that appears to stem from NFS caching the files. PostGreSQL are continously writing to its WAL files located in the pg_xlog directory which is shared via NFS Our program is in turn continously detecting the changes as described in 24.4.4 at http://www.postgresql.org/docs/8.3/static/warm-standby.html and copying a segment of the WAL file to the local harddisk. The changes written by PostGreSQL however doesn't appear to be reflected through the mapped NFS drive until at some later point in time (not sure how long the delay is, but it appears to be 10+ seconds). The delay causes the transferred WAL files to become corrupt. Running the same program directly on the PostGreSQL machine provides the expected result. The following is the NFS configuration on both the Client and Server machines: sysctl -a |grep nfs vfs.nfs.downdelayinitial: 12 vfs.nfs.downdelayinterval: 30 vfs.nfs.skip_wcc_data_onerr: 1 vfs.nfs.nfs3_jukebox_delay: 10 vfs.nfs.reconnects: 0 vfs.nfs.bufpackets: 4 vfs.nfs.realign_count: 0 vfs.nfs.realign_test: 0 vfs.nfs.defect: 0 vfs.nfs.iodmax: 20 vfs.nfs.iodmin: 0 vfs.nfs.iodmaxidle: 120 vfs.nfs.diskless_rootpath: vfs.nfs.diskless_valid: 0 vfs.nfs.nfs_ip_paranoia: 1 vfs.nfs.nfs_directio_allow_mmap: 1 vfs.nfs.nfs_directio_enable: 1 vfs.nfs.clean_pages_on_close: 1 vfs.nfs.nfsv3_commit_on_close: 0 vfs.nfs.access_cache_timeout: 0 vfs.nfs4.access_cache_timeout: 0 vfs.nfsrv.nfs_privport: 0 vfs.nfsrv.commit_miss: 0 vfs.nfsrv.commit_blks: 0 vfs.nfsrv.async: 0 vfs.nfsrv.realign_count: 0 vfs.nfsrv.realign_test: 4069 vfs.nfsrv.gatherdelay_v3: 0 vfs.nfsrv.gatherdelay: 1 Are there other ways of disabling the NFS' cache than using sysctl? Preferably at mount time so caching is only disabled for the PostGreSQL mount point. Alternatively, how is NFS forced to use Direct IO? Appreciate the input Cheers Jona ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
nfs cache
Greetings, I am trying to get some info on the cache for nfs on the nfs client. I have looked at nfsstat where are the settings located for the nfs cache? the setattr percentages are really high on this server. The is the main reason i want to know what is going on with the cache. Any ideas? thanks, bh # nfsstat -c Client rpc: Connection oriented: calls badcalls badxids timeouts newcreds 143885090 301 232 0 0 badverfs timerscantconn nomem interrupts 0 0 5 0 232 Connectionless: calls badcalls retrans badxids timeouts 203132147 23675 2793 692793 newcreds badverfs timersnomem cantsend 0 0 4330 0 0 Client nfs: calls badcalls clgetscltoomany 346433939 23947 346448014 2 Version 2: (185583924 calls) null getattr setattr root lookup 0 0% 175265837 94% 66975 0% 0 0% 6407008 3% readlink read wrcache write create 18540 0% 1866245 1%0 0% 1356437 0%70530 0% removerenamelink symlink mkdir 72376 0% 10037 0% 1139 0% 9658 0% 3455 0% rmdir readdir statfs 2344 0% 399234 0% 34109 0% Version 3: (143319090 calls) null getattr setattr lookupaccess 0 0% 102639638 71% 796374 0% 9784537 6%6378938 4% readlink read write createmkdir 2017 0% 9299172 6%10831505 7% 374932 0% 14035 0% symlink mknod removermdir rename 5741 0% 2750 0% 528773 0% 15443 0% 45381 0% link readdir readdirplus fsstatfsinfo 95277 0% 249881 0% 2243928 1%513 0%789 0% pathconf commit 9466 0% 0 0% ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: nfs cache
In the last episode (Apr 16), Brian Henning said: I am trying to get some info on the cache for nfs on the nfs client. I have looked at nfsstat where are the settings located for the nfs cache? I think they're just part of the system's regular disk cache, except for the sysctl below. the setattr percentages are really high on this server. The is the main reason i want to know what is going on with the cache. Getattr, you mean? That just means that the client has stat()'ed a file twice within the timeout specified by the vfs.nfs.access_cache_timeout sysctl, and has pulled the value from its cache instead of querying the server. Version 2: (185583924 calls) null getattr setattr root lookup 0 0% 175265837 94% 66975 0% 0 0% 6407008 3% So here, 94% of the getattr calls were pulled from cache. -- Dan Nelson [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: nfs cache
-Original Message- From: Dan Nelson [mailto:[EMAIL PROTECTED] Sent: Friday, April 16, 2004 11:34 AM To: Brian Henning Cc: [EMAIL PROTECTED] Subject: Re: nfs cache In the last episode (Apr 16), Brian Henning said: I am trying to get some info on the cache for nfs on the nfs client. I have looked at nfsstat where are the settings located for the nfs cache? I think they're just part of the system's regular disk cache, except for the sysctl below. the setattr percentages are really high on this server. The is the main reason i want to know what is going on with the cache. Getattr, you mean? That just means that the client has stat()'ed a file twice within the timeout specified by the vfs.nfs.access_cache_timeout sysctl, and has pulled the value from its cache instead of querying the server. Version 2: (185583924 calls) null getattr setattr root lookup 0 0% 175265837 94% 66975 0% 0 0% 6407008 3% So here, 94% of the getattr calls were pulled from cache. -- Dan Nelson [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] Dan, Sorry, you were right I did mean getattr. Just curious, does this mean that 94% of the NFS calls made, where getattr() calls (175265837/185583924 = .944)? Thanks, Brian ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
