Thanks for the info. I have quickly draft this bash script in case it can help someone... You just neeed to make sure the IP inside is replaced. To call it, you need to give the job task page.
./showLogs.sh "http://192.168.23.7:50030/jobtasks.jsp?jobid=job_201211211408_0001&type=map&pagenum=1"; Then you can redirect the output, or do what ever you want. I was wondering if there was a "nicer" solution... :~/test$ cat showLogs.sh #!/bin/bash rm -f tasks.html wget --quiet --output-document tasks.html $1 for i in `cat tasks.html | grep taskdetails | cut -d"\"" -f2 | grep taskdetails`; do rm -f tasksdetails.html wget --quiet --output-document tasksdetails.html http://192.168.23.7:50030/$i for j in `cat tasksdetails.html | grep "all=true" | cut -d"\"" -f6`; do printf "*"%.0s {1..80} echo echo $j printf "*"%.0s {1..80} echo rm -f logs.txt wget --quiet --output-document logs.txt $j tail -n +31 logs.txt | head -n -2 done done rm -f tasks.html rm -f tasksdetails.html rm -f logs.txt 2012/11/21, Dino Kečo <dino.k...@gmail.com>: > Hi, > > We had similar requirement and we built small Java application which gets > information about task nodes from Job Tracker and download logs into one > file using URLs of each task tracker. > > For huge logs this becomes slow and time consuming. > > Hope this helps. > > Regards, > Dino Kečo > msn: xdi...@hotmail.com > mail: dino.k...@gmail.com > skype: dino.keco > phone: +387 61 507 851 > > > On Wed, Nov 21, 2012 at 7:55 PM, Jean-Marc Spaggiari < > jean-m...@spaggiari.org> wrote: > >> Hi, >> >> When we run a MapReduce job, the logs are stored on all the tasktracker >> nodes. >> >> Is there an easy way to agregate all those logs together and see them >> in a single place instead of going to the tasks one by one and open >> the file? >> >> Thanks, >> >> JM >> >
