[
https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12555417#action_12555417
]
eric baldeschwieler commented on HADOOP-2178:
---------------------------------------------
Why copy files once a job is complete? Why not just always write them directly
to the HDFS (or any other URL the user configures)?
I'm not a real fan of hidden directories like these. The user will not know of
them and potentially will fill a lot of disk/name space with never viewed
material. I'd be much happier if job history were considered part of the
output of a job, unless configured otherwise. IE put it in the map-reduce
output directory in a file or directories prefixed with an underscore. So
<output>/_jobHistory or perhaps <output>/_logs/history.
We added the convention that map-reduce ignores underscore prefixed files
specifically to allow this use case...
This also reduces jobid/name confusion, since the history is directly
associated with the job's output.
We could then provide an option to put it in another location if the user
desires.
thoughts?
> Job history on HDFS
> -------------------
>
> Key: HADOOP-2178
> URL: https://issues.apache.org/jira/browse/HADOOP-2178
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Amareshwari Sri Ramadasu
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1. Check for accuracy of job tracker history logs.
> 2. After completion of the job, copy the JobHistory.log(Master index file)
> and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
