Hi Abhishek, thanks for your answer. Thus this is the reason for the 1s and 3s raster in the whole plot.
Do you (or someone else) have any ideas what maybe is causing the few outliers downwards? The outliers upwards can be caused by some latencies in the network or in the some of the nodes. No problem so far. But are the few (but recurring) outliers downwards just a chain of (very) lucky accidents? Or a bug in my measurement :-) At least they fall into the 1s and 3s raster, too. Or is there another bigger (about 6s) polling interval which is overlaying the 1s and 3s raster. Christian Am Montag, den 11.04.2011, 12:38 -0400 schrieb abhishek sharma: > Christian, > > The TaskTrackers send "heartbeat" messages to the JobTracker. The > default interval for these messages is 3 seconds. > This is one reason why you see the 3 second steps. > > Abhishek > > On Mon, Apr 11, 2011 at 3:19 AM, Christian Kumpe <christ...@kumpe.de> wrote: > > Hi, > > > > I'm doing some measurement on hadoop's execution time for my theses. I > > discovered some steps in the jobs execution time when raising the > > mappers' execution time continuously. > > Here is a plot of the execution times with 1, 30 and 60 parallel > > executing mappers: > > http://baumbart.drhuim.de/executiontime.gif > > > > The cluster consists of 10 nodes and an overall map task capacity of 60. > > I'm using hadoop 0.20.2. > > > > The many small steps about 1 second are probably caused by the polling > > interval of the JobClient in monitorAndPrintJob(..) which is about 1 > > second. > > > > But can anyone give me a hint, what's causing the bigger 3 second steps? > > I suppose it's a polling interval, too. But where? In the Jobtracker? In > > the Tasktrackers? > > > > Thanks for some help and comments, > > Christian > > > >
