How reduce tasks know which partition they should read?

2015-03-09 Thread xeonmailinglist-gmail
Hi, I am looking to the Yarn mapreduce internals to try to understand how reduce tasks know which partition of the map output they should read. Even, when they re-execute after a crash? I am also looking to the mapreduce source code. Is there any class that I should look to try to

Re: How reduce tasks know which partition they should read?

2015-03-09 Thread Vinod Kumar Vavilapalli
The reducers(Fetcher.java) simply ask the Shuffle Service (ShuffleHandler.java) to give them output corresponding to a specific map. The partitioning detail is hidden from the reducers. Thanks, +Vinod On Mar 9, 2015, at 7:56 AM, xeonmailinglist-gmail xeonmailingl...@gmail.com wrote: Hi,