[
https://issues.apache.org/jira/browse/HADOOP-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
[EMAIL PROTECTED] updated HADOOP-1181:
--------------------------------------
Attachment: hadoop1181-v3.patch
I like both of the above suggestions. Arun's integrates the pulling of remote
userlogs nicely into the command-line and if Dougs' HADOOP-1199 had been in
place, I wouldn't have filed this issue in the first place.
I would suggest though that the attached patch may still have some (minor)
merit: it makes it possible to get a stream on a remote hosts' userlogs dir
for grepping, sorting, etc., w/o having to first copy all logs local (or to
HDFS) or go via a head node (as per Arun's suggestion) or w/o having to write a
mapreduce job as per the Doug suggestion.
Thanks lads
(Redoing patch because v2 had some pollution).
> userlogs reader
> ---------------
>
> Key: HADOOP-1181
> URL: https://issues.apache.org/jira/browse/HADOOP-1181
> Project: Hadoop
> Issue Type: Improvement
> Reporter: [EMAIL PROTECTED]
> Attachments: hadoop1181-v2.patch, hadoop1181-v3.patch,
> hadoop1181.patch
>
>
> My jobs output lots of logging. I want to be able to quickly parse the logs
> across the cluster for anomalies. org.apache.hadoop.tool.Logalyzer looks
> promising at first but it does not know how to deal with the userlog format
> and it wants to first copy all logs local. Digging, there does not seem to
> currently be a reader for hadoop userlog format. TaskLog$Reader is not
> generally accessible and it too expects logs to be on the local filesystem
> (The latter is of little good if I want to run the analysis as a mapreduce
> job).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
