On 04/04/2012 11:20 PM, bikash sharma wrote:
Hi,
Is it possible to get the execution time of the constituent map/reduce
tasks of a MapReduce job (say sort) at the end of a job run?
Preferably, can we obtain this programatically?
Thanks,
Bikash
You could parse the logs inside the job's output directory (<output
dir>/_logs)
You may also find this tool to be interesting:
http://www.cs.duke.edu/starfish/
It can extract all that task info for you, after doing a profiling run.
--
Luca Pireddu
CRS4 - Distributed Computing Group
Loc. Pixina Manna Edificio 1
09010 Pula (CA), Italy
Tel: +39 0709250452
