Dean Hildebrand wrote:
Thanks for the info Phil. While I'm running an I/O performance
experiment, I never want the attributes to expire, so I'm using the -a
option to set it to a really high value.
But for the average pvfs2 setup, what is a useful value? With 5msec,
the cache was continually expiring, and my data servers were contacting
the metadata server multiple times when writing a single file (which
takes on the order of seconds, if not minutes for large files), just to
retrieve the same information over and over.
I've seen single client file write throughput (with < 10 of data
servers) achieve roughly 40-60MB/s (with gigabit ethernet). So taking a
1G file around 20s to write to disk. Of course there is no typical
setup or typical file size, but maybe a better default acache timeout
value is in the 10s of seconds?
Dean
I'm really not sure, to be honest with you. Someone else may have more
input. I ran some limited tests at one point (very different from your
workload, though) and found that there was a point of diminishing
returns before the timeout got that high. At some point the cost of
occasional attribute retrieval just becomes noise.
I think that in your case you are probably triggering a few getattrs at
open time, and then 1 getattr every 4 MB of the file (because that is
the default buffer size the kernel module is breaking the file into). I
guess that works out to 10-15 getattr operations per second for your
case that you could be avoiding. Reducing the getattr traffic may also
have a beneficial impact on the server request scheduling too.
-Phil
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
