We were getting less perf with nfsv4 so we stuck with v3, but perhaps you have other reasons for using v4...
On Mon, Aug 27, 2012 at 11:39 AM, Sabuj Pattanayek <sab...@gmail.com> wrote: > Yes, that's what'll happen if you use sync. One way around using sync > and getting decent write performance is to use rsize and wsize >= > 128kb, but then you'll lose small file performance since you're moving > larger blocks over the wire. The problem I've seen with all the EMC > solutions I've tested is that NFSv3 client (read) data cache usage is > broken unless you use sync or you set the EMC server to force NFS > unstable writes to filesync or datasync rather than the default of an > unstable NFS write (async). > > What I mean by broken client (read) cache usage is that if you write a > file out to the server which fits in memory on the client (since the > client buffers the write to local memory before sending it to the > server), then if you try to read the file back and it's still in local > client memory, the EMC server will do something with the client that > causes it to read the entire file back again over the wire, rather > than just getting it from local memory (really fast). Between Linux > NFS servers and clients or other high end NAS solutions I've tested > the client will read the file out of cache if it exists and if it's > unchanged, which is the correct behavior. > > dd is actually a pretty good way of determining whether several NFS > stacks are "broken" in this regard, I've written a nice wrapper around > it https://code.google.com/p/nfsspeedtest/ > > On Mon, Aug 27, 2012 at 9:19 AM, Gerhardus Geldenhuis > <gerhardus.geldenh...@gmail.com> wrote: >> Hi >> I am debugging NFS performance problems at the moment and going through a >> steep learning curve with regards to all its inner workings. >> >> In short we are mounting EMC provided NFS storage and seeing a massive >> difference when mounting with and without the sync option. >> >> Without the sync option I can achieve 90MB/s without any tuning, with sync >> option turned on that falls to 4MB/s. That seems to slow... any thoughts and >> experience on this would be appreciated. >> These values were obtained by doing a dd if=/dev/zero of=<file onnfs share> >> bs=1M count=1024. I am aware that I need to optimize for specific use case >> with regards to concurrency, file size, writes, rewrites, reads, rereads >> etc. However the performance penalty for sync seems excessive and I wanted >> to know if that is a shared experience. >> >> My understanding of sync is basically that in the case of NFS, it will wait >> for a confirmation from the NFS server that the data has been written to >> disk before acknowledging the write locally, so being "very certain". My >> understanding could be flawed and I would appreciate anyone correcting me on >> that and/or pointing me to more reading which I would happily do. >> >> I am a bit confused however about what the man pages are saying: >> man mount says: >> the sync option today has effect only for ext2, ext3, fat, vfat and ufs >> but when I change it I can see a big difference in performance so the man >> page is out of date... or is there something I am misreading? >> >> man 5 nfs, does not make mention of sync. >> >> OS: RHEL 5.8 >> Kernel: 2.6.18-308.4.1.0.1.el5 >> NFS: >> nfs-utils-lib-1.0.8-7.9.el5 >> nfs-utils-1.0.9-60.0.1.el5 >> mount parameters: >> nfs4 >> rw,bg,hard,nointr,rsize={16384,32768,65536},wsize={16384,32768,65536},suid,timeo=600,_netdev >> 0 0 >> >> I am currently running iozone and changing various mount parameters to see >> how they effect performance. iozone is arguably a better test of performance >> than dd >> >> Regards >> >> -- >> Gerhardus Geldenhuis >> >> _______________________________________________ >> rhelv5-list mailing list >> rhelv5-list@redhat.com >> https://www.redhat.com/mailman/listinfo/rhelv5-list _______________________________________________ rhelv5-list mailing list rhelv5-list@redhat.com https://www.redhat.com/mailman/listinfo/rhelv5-list
