Brian Bockelman wrote:
Do you have any graphs you can share showing 50k opens / second (could
be publicly or privately)? The more external benchmarking data I have,
the more I can encourage adoption amongst my university...
The 50k opens/second is from some internal benchmarks run at Y! nearly a
year ago. (It doesn't look like Y! runs that benchmark regularly
anymore, as far as I can tell.) I copied the graph to:
http://people.apache.org/~cutting/nn500.png
Note that all of the operations that modify the namespace top out at
around 5k/second, since these are logged & flushed to disk.
I found some more recent micro namenode benchmarks at:
http://tinyurl.com/6bxoxz
These indicate that actual use doesn't hit these levels, but would
still, on large clusters, be adversely affected by moving to a
disk-based namespace.
Doug
