[
https://issues.apache.org/jira/browse/MAPREDUCE-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882320#action_12882320
]
Krishna Ramachandran commented on MAPREDUCE-1699:
-------------------------------------------------
yes - this is good. I have added a map and logSubmitted will initialize in case
of an error for that job and use that for subsequent logging (failed, killed
etc.,)
Will update patch once testing is completed
Am not able to entirely take out disableHistory because of this code (incorrect
javadoc - there is no param - jobID)
/**
* Logs history meta-info to the history file. This needs to be called once
* per history file.
* @param jobId job id, assigned by jobtracker.
*/
static void logMetaInfo(ArrayList<PrintWriter> writers){
if (!disableHistory){
if (null != writers){
JobHistory.log(writers, RecordTypes.Meta,
new Keys[] {Keys.VERSION},
new String[] {String.valueOf(VERSION)});
}
}
}
also there are couple of (public) methods that use this flag. though I do not
see where they are called
/**
* Returns history disable status. by default history is enabled so this
* method returns false.
* @return true if history logging is disabled, false otherwise.
*/
public static boolean isDisableHistory() {
return disableHistory;
}
/**
* Enable/disable history logging. Default value is false, so history
* is enabled by default.
* @param disableHistory true if history should be disabled, false otherwise.
*/
public static void setDisableHistory(boolean disableHistory) {
JobHistory.disableHistory = disableHistory;
}
> JobHistory shouldn't be disabled for any reason
> -----------------------------------------------
>
> Key: MAPREDUCE-1699
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1699
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2
> Reporter: Arun C Murthy
> Assignee: Krishna Ramachandran
> Fix For: 0.20.3
>
> Attachments: mapred-1699-1.patch, mapred-1699-2.patch,
> mapred-1699-3.patch, mapred-1699.patch
>
>
> Recently we have had issues with JobTracker silently disabling job-history
> and starting to keep all completed jobs in memory. This leads to OOM on the
> JobTracker. We should never do this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
