On Jul 21, 2010, at 6:29 PM, Bobby Dennett wrote: > The team that manages our Hadoop clusters is currently being pressured > to reduce block replication from 3 to 2 in our production cluster. This > request is for various reasons -- particularly the reduction of used > space in the cluster and potential of reduced write operations -- but > from what I've read previously, it seems to be strongly discouraged.
I'm trying to understand 'potential of reduced write operations'. Does this mean you want to reduce the amount of wear and tear on a drive by reducing how many writes are happening to it? If so, you might want to research what your jobs are actually doing. Since we split our drives such that we have dedicated MR temp space and dedicated HDFS space, it was pretty obvious for us to see that the vast majority of our IO (especially writes) is in spills, not from HDFS....
