On 23/04/10 15:43, Todd Lipcon wrote:
Hi Stephen,
Can you try mounting ext4 with the nodelalloc option? I've seen the same
improvement due to delayed allocation butbeen a little nervous about that
option (especially in the NN where we currently follow what the kernel
people call an antipattern for image rotation).
Hi Todd,
Sorry for the delayed response - I had to wait for another test window
before trying this out.
To clarify, my namename and secondary namenode have been using ext4 in
all tests - reconfiguring the datanodes is a fast operation, the nn and
2nn less so. I figure any big performance benefit would appear on the
data nodes anyway and can then apply it back to the nn and 2nn if
testing shows any benefits in changing.
So I tried running our datanodes with their ext4 filesystems mounted
using "noatime,nodelalloc" and after 6 runs of the TeraSort, it seems it
runs SLOWER with those options by between 5-8%. The TeraGen itself
seemed to run about 5% faster but it was only a single run so I'm not
sure how reliable that is.
hth,
-stephen
--
Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
http://di2.deri.ie http://webstar.deri.ie http://sindice.com
