We are currently modifying the configuration of our hadoop grid (250 machines). The machines are homogeneous and the specs are
dual quad core cpu 18Gb ram 8x1tb drives currently we have set this up - 8 reduce slots at 800mb 8 map slots at 800mb raised our io.sort.mb to 256mb we see a lot of spilling on both maps and reduces and I am wondering what other configs I should be looking into Thanks