Bench-marking Hadoop Performance

2014-07-22 Thread Charley Newtonne
This is a new cluster I'm putting up and I need to get an idea on what to
expect from a performance standpoint.

Older docs point to gridmix and TestDFSIO . However, most of this doc is
obsolete and no longer applies on 2.4.

Where can I find benchmarking docs for 2.4? What are my options?
Also, I have searched safari books online including rough cuts, but not
seeing books for the 2.4 release. If you know of a book for this release,
please share.

Thank you.


Re: Bench-marking Hadoop Performance

2014-07-22 Thread jay vyas
There are alot of tests out there and it can be tough to determine what is
a standard.

- TeraGen/TearSort and testdfsio are starting points.

- Various other non apache projects (such as ycsb or hibench) will have
good benchmarks for certain type sof cases.

-If looking for a more comprehensive long term strategy, I'd suggest the
you ask on the  bigtop mailing list, where we are
building a broader community around uniform smoke testing and benchmarking
of hadoop, hadoop compatible file systems, and YARN applications.







On Tue, Jul 22, 2014 at 11:23 AM, Charley Newtonne cnewto...@gmail.com
wrote:

 This is a new cluster I'm putting up and I need to get an idea on what to
 expect from a performance standpoint.

 Older docs point to gridmix and TestDFSIO . However, most of this doc is
 obsolete and no longer applies on 2.4.

 Where can I find benchmarking docs for 2.4? What are my options?
 Also, I have searched safari books online including rough cuts, but not
 seeing books for the 2.4 release. If you know of a book for this release,
 please share.

 Thank you.





-- 
jay vyas