Hi Rob,
Doing a non-disk test with Lustre, I hit 1170 MB/s using 3-4 threads
(single client to/from single server).
Using a single thread with Lustre, I get 675 MB/s.
Using the same machines as below, my Lustre results are at:
https://mail.clusterfs.com/wikis/lustre/MX-10G
Scott
On Jan 8, 2007, at 2:08 PM, Rob Ross wrote:
Nice Scott! What's the realistic BW limit for this hardware?
Rob
Scott Atchley wrote:
On Jan 5, 2007, at 11:47 AM, Scott Atchley wrote:
I modified perf.c to write 10 times a buffer of 128 MBs. I get:
% mpiexec -n 1 ./perf -fname pvfs2:/mnt/pvfs2/x1
Access size per process = 134217728 bytes, ntimes = 10
Write bandwidth without file sync = 827.824860 Mbytes/sec
Read bandwidth without prior file sync = 870.979531 Mbytes/sec
Write bandwidth including file sync = 827.166730 Mbytes/sec
Read bandwidth after file sync = 865.624838 Mbytes/sec
using a single client. The client used about 25% CPU.
Scott
The above was with 1 MB buffers. With 16 MB buffers:
% mpiexec -n 1 ./perf -fname pvfs2:/mnt/pvfs2/x1
Access size per process = 268435456 bytes, ntimes = 10
Write bandwidth without file sync = 822.867597 Mbytes/sec
Read bandwidth without prior file sync = 844.154572 Mbytes/sec
Write bandwidth including file sync = 823.773517 Mbytes/sec
Read bandwidth after file sync = 844.393557 Mbytes/sec
With 4 MB buffers:
% mpiexec -n 1 ./perf -fname pvfs2:/mnt/pvfs2/x1
Access size per process = 268435456 bytes, ntimes = 10
Write bandwidth without file sync = 847.215804 Mbytes/sec
Read bandwidth without prior file sync = 862.068630 Mbytes/sec
Write bandwidth including file sync = 847.861378 Mbytes/sec
Read bandwidth after file sync = 862.307479 Mbytes/sec
The client CPU usage was 25% for both values. I do not see any
benefit for larger than 1 MB buffers.
Scott
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
