[
https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558010#action_12558010
]
Amareshwari Sri Ramadasu commented on HADOOP-2178:
--------------------------------------------------
To address use cases 1 and 2 suggested by Eric, I propose the following
approach.
If the job tracker is static, we will store history logs in a location
specified by hadoop.job.history.location, by default it is local file system.
If the job tracker is not static (like HOD JT) we will store log files in user
specified location, by default it is job output directory.
We will not have index file any more, because appending becomes an issue in
DFS. And we dont need one in case of non-static JT.
For static JT, We can do listing of files in the log directory to show the
first page.
> Job history on HDFS
> -------------------
>
> Key: HADOOP-2178
> URL: https://issues.apache.org/jira/browse/HADOOP-2178
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Amareshwari Sri Ramadasu
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1. Check for accuracy of job tracker history logs.
> 2. After completion of the job, copy the JobHistory.log(Master index file)
> and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
