Kyle, Thanks for the prompt reply. I had been testing with 10-20GB files. So today I used 100GB files. Performance was the same for both PVFS and NFS when using the 20GB files. PVFS reads were slower than writes. NFS reads were faster than NFS writes and way faster than PVFS reads. However, when I tried using multiple NFS clients reading the same filesystem (but different files), performance of NFS dropped 94%. PVFS performance only dropped 25%.
I'm already using the alt-io method which appears to be the default setup. Thanks for your help! -Roger ----------------------------------------------------------- Roger V. Moye Systems Analyst III XSEDE Campus Champion University of Texas - MD Anderson Cancer Center Division of Quantitative Sciences Pickens Academic Tower - FCT4.6109 Houston, Texas (713) 792-2134 ----------------------------------------------------------- From: Kyle Schochenmaier [mailto:[email protected]] Sent: Monday, July 01, 2013 4:02 PM To: Moye,Roger V Cc: [email protected] Subject: Re: [Pvfs2-users] Are writes supposed to be faster than reads? Hi Roger - Depending on your setup I would expect that writes would be faster than reads simply because you'll have a lot of server-side caching of the data on writes and those might be sent out to disk before your reading tests. Are you doing any tests that you're positive you're exceeding the caches on the servers for writes with? (writing out 100GB+ files for example) ? There are some pvfs2 tunables that will force disk flushes on every write in the config file, which would help to see if this really is just caching, Im not sure what the flag is off the top of my head though. I think it is setting troveio to 'directio' : * directio. This uses a direct I/O implementation to perform I/O operations to datafiles. This method may give significant performance improvement if PVFS servers are running over shared storage, especially for large I/O accesses. For local storage, including RAID setups, the alt-aio method is recommended. Also keep in mind that NFS can do client side read caching so reads are almost always going to be faster than writes. Cheers, ~Kyle Kyle Schochenmaier On Mon, Jul 1, 2013 at 3:49 PM, Moye,Roger V <[email protected]<mailto:[email protected]>> wrote: We are currently testing a new PVFS2 setup at our facility. The servers are running on RHEL 6 using XFS as the underlying filesystem. We are using four PVFS servers. Performance is beating NFS, which is what we wanted and expected. While running some simple I/O tests today I discovered that writes are about 30% faster than reads. I expected reads to be faster. Further, when I tried the same I/O tests to the local disk of the client, and to an NFS server running on the same server hardware, I found that reads were indeed faster than writes by 3:1. Is it expected that reads would be slower on PVFS? Is there something I need to tune? Thanks in advance! -Roger Moye ----------------------------------------------------------- Roger V. Moye Systems Analyst III XSEDE Campus Champion University of Texas - MD Anderson Cancer Center Division of Quantitative Sciences Pickens Academic Tower - FCT4.6109 Houston, Texas (713) 792-2134<tel:%28713%29%20792-2134> ----------------------------------------------------------- _______________________________________________ Pvfs2-users mailing list [email protected]<mailto:[email protected]> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
