[jira] [Commented] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415384#comment-15415384
]
Tatyana But commented on MAPREDUCE-6737:
[~sjlee0], could you please check the patch and commit if everything is ok ?
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.0, 2.5.1
>Reporter: Roman Gavryliuk
> Attachments: MAPREDUCE-6737.patch
>
>
> The problem shows itself while recovering old apps information:
> 2016-07-18 16:08:35,031 WARN
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
> start time from job history file
> job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
> : java.lang.NumberFormatException: For input string:
> ""
> The problem is in JobHistoryEventHandler.java class in the
> following part of code:
> //initialize the launchTime in the JobIndexInfo of MetaInfo
> if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
> JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
> mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());
> Because of job was not initialized properly, the 'if' statement takes value
> 'false' and .setJobStartTime() is not called.
> In JobIndexInfo constructor, we have a default value for jobStartTime:
> this.jobStartTime = -1;
> When history server recovers any application's info, it passes all parameters
> to array of strings jobDetails:
> String[] jobDetails = fileName.split(DELIMITER);
> Please note, DELIMETER is initialized in the following way:
> static final String DELIMITER = "-";
> So, jobDetails array has 10 elements - job ID, submit time, username, job
> name,
> finish time, number of maps, number of reducers, job status, queue, and start
> time).
> If jobStartTime = -1, the minus symbol is considered as delimeter and the code
> will assign an empty string "" as a value for 9-th element in jobDetails
> array.
> In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
> NumberFormatException will appear while trying to parse empty string to long
> type.
> Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));
> The most simple fix is to change the value this.jobStartTime to 0 in
> JobIndexInfo constructor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387755#comment-15387755
]
Tatyana But commented on MAPREDUCE-6737:
[~vinodkv], could you please check the patch and commit if everything is ok ?
I'm asking you because you was the last person who patched JobIndexInfo class.
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.0, 2.5.1
>Reporter: Roman Gavryliuk
> Attachments: MAPREDUCE-6737.patch
>
>
> The problem shows itself while recovering old apps information:
> 2016-07-18 16:08:35,031 WARN
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
> start time from job history file
> job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
> : java.lang.NumberFormatException: For input string:
> ""
> The problem is in JobHistoryEventHandler.java class in the
> following part of code:
> //initialize the launchTime in the JobIndexInfo of MetaInfo
> if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
> JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
> mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());
> Because of job was not initialized properly, the 'if' statement takes value
> 'false' and .setJobStartTime() is not called.
> In JobIndexInfo constructor, we have a default value for jobStartTime:
> this.jobStartTime = -1;
> When history server recovers any application's info, it passes all parameters
> to array of strings jobDetails:
> String[] jobDetails = fileName.split(DELIMITER);
> Please note, DELIMETER is initialized in the following way:
> static final String DELIMITER = "-";
> So, jobDetails array has 10 elements - job ID, submit time, username, job
> name,
> finish time, number of maps, number of reducers, job status, queue, and start
> time).
> If jobStartTime = -1, the minus symbol is considered as delimeter and the code
> will assign an empty string "" as a value for 9-th element in jobDetails
> array.
> In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
> NumberFormatException will appear while trying to parse empty string to long
> type.
> Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));
> The most simple fix is to change the value this.jobStartTime to 0 in
> JobIndexInfo constructor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
