Another test to copy the 2GB single file confirms that pvfs2 works better with big files. A few snapshots are:
(1) From local to pvfs2 using cp: 4m57s pvfs2-cp: Wrote 2057207616 bytes in 21.67 sceonds. 90.5 MB/seconds (2) From pvfs2 to pvfs2 using pvfs2-cp: Wrote ... in 42.82 seconds. 45.81 MB/seconds cp: 13m4s On Thu, Sep 4, 2008 at 11:37 AM, Kyle Schochenmaier <[EMAIL PROTECTED]>wrote: > It would be interesting to know what the numbers are for copying > pvfs2->local_memory and then back, local_memory->pvfs2 and compare > that to doing pvfs2->pvfs2. > If I were wanting to further look at these results and try to discern > what the slow parts are, some important things to note are cpu > utilization on the servers and clients. > > You should also ideally have a difference in cpu utilization when you > do 'cp -r' vs. the pvfs2-cp which doesnt go through the kernel > module. > > What is the cpu load on the client, I/O, MD servers? > If you find that the MD server is getting hit heavily, you might be > able to startup a second MD server, maybe even on the same box with > multiple-cores. > > I'm also assuming you are using very small files. PVFS2 does not > particularly shine with large numbers of small files in general, > especially when compared to systems that do caching (nfs). > From what I understand about whats going on in the servers during > these operations, these numbers kind of fall in line with what I would > have expected. > > Just some random thoughts, good luck! > > Kyle > > On Thu, Sep 4, 2008 at 4:13 PM, hawaii <[EMAIL PROTECTED]> wrote: > > A pvfs2 I/O test > > (for comments) > > =================== > > > > pvfs version: 2.7.1 > > configuration: 1 meta server + 3 data servre. Each server has 2 cpus > > reserved > > > > Tests: within pvfs2; from pvfs2 to local; nfs > > Use pvfs2-cp-r, pvfs2-cp-r.pl, and cp > > (thanks Mark Bartelt for providing pvfs2-cp-r and pvfs2-cp-r.pl) > > > > Source direcotry: WRF > > df -k WRF > > lots of subdirctories > > 511868 WRF > > > > (1) pvfs2 -> pvfs2 > > > > time pvfs2-cp-r WRF tmp > > real 4m16.414s > > user 0m38.385s > > sys 0m31.874s > > I/O 2 MB/s > > > > time pvfs2-cp-r.pl WRF tmp > > Copying files from "WRF" to "tmp/WRF" using pvfs2-cp > > real 4m21.573s > > user 0m38.741s > > sys 0m32.293s > > I/O 1.96 MB/s > > > > time cp -rp WRF tmp > > Lots of 0 files (unreliable) > > real 7m21.473s > > user 0m0.100s > > sys 0m5.792s > > I/O 1.16 MB/s > > > > (2) pvfs2 -> local hard drive > > > > time pvfs2-cp-r.pl /pvfs2-dir/WRF /home/user > > Copying files from "/pvfs2-dir/WRF" to "/home/user/WRF" using pvfs2-cp > > real 1m19.311s > > user 0m19.008s > > sys 0m29.327s > > I/O 6.4 MB/s > > > > time pvfs2-cp-r /pvfs2-dir/WRF /home/user > > real 1m10.739s > > user 0m20.609s > > sys 0m30.021s > > I/O 7.25 MB/s > > > > time cp -rp /pvfs2-dir/WRF /home/user > > Lots of 0 files (unreliable) > > real 1m34.071s > > user 0m0.113s > > sys 0m3.906s > > I/O 5.45 MB/s > > > > (3) cp an nfs file to local disk > > 2057207616 t.dat > > time cp -p /nfs-dir/t.dat /home/usr/. > > real 2m55.894s > > user 0m0.257s > > sys 0m6.946s > > Read 11.7 MB/s > > > > Conclusion from this test: > > (1) pvfs2-cp-r is the fastest and cp -r is the slowest > > (2) it is slower to copy directory inside pvfs2 than to local disk > > (3) I/O pvfs2 is much slower than nfs !!! > > Any comments on the results? > > _______________________________________________ > > Pvfs2-users mailing list > > [email protected] > > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > > > > > > > > -- > Kyle Schochenmaier >
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
