Re: NFS Performance Woes
On 22/07/14 02:39, Peter Skensved wrote: I've been using NFSv4 extensively for several years and I've not had an issue that you describe where everything is fine and then suddenly performance goes to hell in a hand basket. It sounds as if you only have 2 systems to work with? No, tiebreaker so to speak? Have you considered running a VM on your client system to see if it is affected in the same way? DNS problems can do it . Are your /etc/resolv.conf files correct ? You could try running your own nameserver ( dnsmasq ) if the upstream one is too slow or too busy. I'm fairly sure it's not DNS. I run a DNS server actually on the same server, which serves NFS exports with the only DNS server in resolv.conf being itself (over localhost). All clients point to that DNS server and only that one. It's authoritative for my home LAN and both forward and reverse lookups work and resolve correctly and quickly too. Besides, the exports are specified by IP address on the server and the problem still occurs even why I mount an export from the client machine using the server's IP as opposed to its hostname. -- Ian Chapman. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: NFS Performance Woes
On 21/07/14 21:27, Ed Greshko wrote: For all intents and purpose it looks like its working as it should, it's just painfully slow. Any NFS gurus out there, that can tell me what I'm doing wrong? I've been using NFSv4 extensively for several years and I've not had an issue that you describe where everything is fine and then suddenly performance goes to hell in a hand basket. Yeah it's really annoying. I could accept it, if performance was terrible all the time. I'd just figure I had some configuration setting out of whack, but it's the fact it works great.. then wham. To be honest it almost feels as if a cron job overnight it causing something to go screwy. It'll run fine all day, the next day, back to shit. That's without making any configuration changes. It sounds as if you only have 2 systems to work with? No, tiebreaker so to speak? Actually, I have a couple of netbooks which hang off the wifi, although the NFS exports are set to read only on those, it's probably worth changing that and doing some further testing. Have you considered running a VM on your client system to see if it is affected in the same way? I hadn't considered that. Good idea. When the client's having a crisis, I can spin up the VM on the client and see if that's affected. I guess it would it least tell me if it's a hardware/driver quirk or a software issue. -- Ian Chapman. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: NFS Performance Woes
On 21/07/14 20:14, Tom Horsley wrote: Any NFS gurus out there, that can tell me what I'm doing wrong? Not a useful answer, I'm afraid: In my experience, the fundamental problem is caused by using NFS. With all the folks re-writing things that don't need to be replaced, I really wonder why no one seems to be re-writing NFS, which has been utterly unreliable and trouble prone since day one :-(. Sure, it has its issues, but to be honest most network file-systems do. CIFS is no better and in some ways takes more babysitting than NFS. Others don't have wide support amongst many platforms and then there's the differences in implementations. Do you use anything instead of NFS? I toyed with GlusterFS a few months ago and whilst it worked, following various snippets of info on the web, I found the documentation lacking and the admin tools confusing. -- Ian Chapman. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
NFS Performance Woes
Hi, I've been unable to track down an extremely annoying performance issue with NFS. My home server has several exports shared over NFS4. The exports are all secured using Kerberos except one, which can be mounted using the traditional sys security model. The problem is that write performance stalls at 48k/sec (or according to nfsiostat 4k/sec) despite being on a 1Gbit network and once this occurs, nothing I do seems to fix it. The only thing that does, is rebooting the client machine. This has been occurring for a year or so now but is happening so regularly now, it's driving me nuts! :-) I'm fairly sure it's a client side issue, because rebooting the client allows normal transfer speeds once again. If I only reboot the server it makes no difference. My client is F19, the server is F20. After rebooting the client, write transfers can be fine for a day or so and then all of a sudden it hangs around 48k/sec again. Restarting all the NFS daemons on the client makes no difference, including the associated RPC daemons. The NFS exports are mounted via autofs, but manually mounting makes no difference either. When I'm experiencing slow writes to the server of 48k/sec, NFS reads are still in the region of 100MB/sec. Using scp or even copying files to the server via CIFS are well within the range of a 1Gbit network. Nfsstat, wireshark and the system logs do not show anything which screams there's a problem. The network card in the client machine and the server shows no collisions, dropped packets, frame overruns etc. I've tested with the export that isn't using Kerberos and still have the same issue. Messing with the rsize, wsize, async, sync parameters makes no difference either. The server has 32GB RAM, the client 16GB. For all intents and purpose it looks like its working as it should, it's just painfully slow. Any NFS gurus out there, that can tell me what I'm doing wrong? -- Ian Chapman. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: NFS Performance Woes
On Mon, 21 Jul 2014 19:59:40 +0800 Ian Chapman wrote: Any NFS gurus out there, that can tell me what I'm doing wrong? Not a useful answer, I'm afraid: In my experience, the fundamental problem is caused by using NFS. With all the folks re-writing things that don't need to be replaced, I really wonder why no one seems to be re-writing NFS, which has been utterly unreliable and trouble prone since day one :-(. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: NFS Performance Woes
On 07/21/14 19:59, Ian Chapman wrote: Nfsstat, wireshark and the system logs do not show anything which screams there's a problem. The network card in the client machine and the server shows no collisions, dropped packets, frame overruns etc. I've tested with the export that isn't using Kerberos and still have the same issue. Messing with the rsize, wsize, async, sync parameters makes no difference either. The server has 32GB RAM, the client 16GB. For all intents and purpose it looks like its working as it should, it's just painfully slow. Any NFS gurus out there, that can tell me what I'm doing wrong? I've been using NFSv4 extensively for several years and I've not had an issue that you describe where everything is fine and then suddenly performance goes to hell in a hand basket. It sounds as if you only have 2 systems to work with? No, tiebreaker so to speak? Have you considered running a VM on your client system to see if it is affected in the same way? -- If you can't laugh at yourself, others will gladly oblige. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: NFS Performance Woes
On 07/21/14 19:59, Ian Chapman wrote: Nfsstat, wireshark and the system logs do not show anything which screams there's a problem. The network card in the client machine and the server shows no collisions, dropped packets, frame overruns etc. I've tested with the export that isn't using Kerberos and still have the same issue. Messing with the rsize, wsize, async, sync parameters makes no difference either. The server has 32GB RAM, the client 16GB. For all intents and purpose it looks like its working as it should, it's just painfully slow. Any NFS gurus out there, that can tell me what I'm doing wrong? I've been using NFSv4 extensively for several years and I've not had an issue that you describe where everything is fine and then suddenly performance goes to hell in a hand basket. It sounds as if you only have 2 systems to work with? No, tiebreaker so to speak? Have you considered running a VM on your client system to see if it is affected in the same way? DNS problems can do it . Are your /etc/resolv.conf files correct ? You could try running your own nameserver ( dnsmasq ) if the upstream one is too slow or too busy. peter -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
