Team, can someone please clarify the following question? In the map phase, the map output is written to the local disk. And in the shuffle phase, the map output partitions are transferred to reduce nodes using http. So, my question is assuming there are no spills (data set is small enough to accommodate this), will the map output be transferred directly from memory to the reduce nodes using http without a disk access to write the map output? Or, is the map output always flushed to the disk before transferred to reduce nodes?
Appreciate the help. Thanks, Satheesh
