[
https://issues.apache.org/jira/browse/HADOOP-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12486435
]
Arun C Murthy commented on HADOOP-1181:
---------------------------------------
Actually (we probably should take this discussion elsewhere, so +1 for a
separate issue), the idea of starting another mapred job to get the logs of a
previous mapred job doesn't seem right ...
How about extending my previous idea to get logs of all tasks via tasklog.jsp
like so:
Get all tasklogs for a given jobid
$ hadoop job <id> -tasklogs
Get all tasklogs for all maps/reduces
$ hadoop job <id> -tasklogs maps
$ hadoop job <id> -tasklogs reduces
(extend this for succesfull/failed maps/reduces?)
Get tasklog for given <jobid,taskid>
$ hadoop job <id> -tasklogs <taskid>
Thoughts?
> userlogs reader
> ---------------
>
> Key: HADOOP-1181
> URL: https://issues.apache.org/jira/browse/HADOOP-1181
> Project: Hadoop
> Issue Type: Improvement
> Reporter: [EMAIL PROTECTED]
> Attachments: hadoop1181-v2.patch, hadoop1181.patch
>
>
> My jobs output lots of logging. I want to be able to quickly parse the logs
> across the cluster for anomalies. org.apache.hadoop.tool.Logalyzer looks
> promising at first but it does not know how to deal with the userlog format
> and it wants to first copy all logs local. Digging, there does not seem to
> currently be a reader for hadoop userlog format. TaskLog$Reader is not
> generally accessible and it too expects logs to be on the local filesystem
> (The latter is of little good if I want to run the analysis as a mapreduce
> job).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
