Re: On the topic of task scheduling

2012-09-02 Thread Arun C Murthy
Vasco, Welcome to Hadoop! You observations are all correct - in simplest case you launch all reduces up front (we used to do that initially) and get a good 'pipeline' between maps, shuffle (i.e. moving map-outputs to reduces) and the reduce itself. However, one thing to remember is that kee

On the topic of task scheduling

2012-09-02 Thread Vasco Visser
Hi, I am new to the list, I am working with hadoop in the context of my MSc graduation project (has nothing to do with task scheduling per se). I came across task scheduling because I ran into the fifo starvation bug (MAPREDUCE-4613). Now, I am running 2.1.0 branch where the fifo starvation issue

Hadoop-Mapreduce-trunk - Build # 1184 - Failure

2012-09-02 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1184/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 30429 lines...] Tests run: 1, Failures: 0, Errors: 0