[
https://issues.apache.org/jira/browse/MAPREDUCE-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135171#comment-16135171
]
Peter Bacsko edited comment on MAPREDUCE-6892 at 8/21/17 1:25 PM:
------------------------------------------------------------------
[~haibochen] I uploaded patch v4. I fixed everything you mentioned, but there
are some things which I didn't change:
1) Checkstyle, Javadoc errors: we would end up with a messy, inconsistent code
style if we decide to eliminate these. I kept the style of the existing code,
which unfortunately introduced new warnings. Refactoring/re-indenting the
existing code would be way too much change and it's unnecessary IMHO.
2) Job20LineHistoryEventEmitter - I kept the "-1" values. I was looking for the
code which generates the stuff that this class parses. I couldn't find it and I
believe it does not exist anymore. However I found this on an old branch:
https://github.com/apache/hadoop/blob/branch-1.2/src/mapred/org/apache/hadoop/mapred/JobHistory.java
I think this was the class which generated the events that Apache Rumen is
meant to parse. This makes me wonder whether it makes sense to have Rumen on
the trunk. Looking at the latest changes, the vast majority of the source files
haven't been touched for 5-6 years (and newer changes are completely unrelated
to its functionality).
was (Author: pbacsko):
[~haibochen] I uploaded patch v4. I fixed everything you mentioned, but there
are some things which I didn't change:
1) Checkstyle, Javadoc errors: we would end up with a messy, inconsistent code
style if we decide to eliminate these. I kept the style of the existing code,
which unfortunately introduced new warnings. Refactoring/re-indenting the
existing code would be way too much change and it's unnecessary IMHO.
2) Job20LineHistoryEventEmitter - I kept the "-1" values. I was looking for the
code which generates the stuff that this class parses. I couldn't find it and I
believe it does not exist anymore. However I found this on an old branch:
https://github.com/apache/hadoop/blob/branch-1.2/src/mapred/org/apache/hadoop/mapred/JobHistory.java
I think this was the class which generated the events that Apache Rumen is
meant parse. This makes me wonder whether it makes sense to have Rumen on the
trunk. Looking at the latest changes, the vast majority of the source files
haven't been touched for 5-6 years (and newer changes are completely unrelated
to its functionality).
> Issues with the count of failed/killed tasks in the jhist file
> --------------------------------------------------------------
>
> Key: MAPREDUCE-6892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6892
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, jobhistoryserver
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Attachments: MAPREDUCE-6892-001.patch, MAPREDUCE-6892-002.PATCH,
> MAPREDUCE-6892-003.patch, MAPREDUCE-6892-004.patch
>
>
> Recently we encountered some issues with the value of failed tasks. After
> parsing the jhist file, {{JobInfo.getFailedMaps()}} returned 0, but actually
> there were failures.
> Another minor thing is that you cannot get the number of killed tasks
> (although this can be calculated).
> The root cause is that {{JobUnsuccessfulCompletionEvent}} contains only the
> successful map/reduce task counts. Number of failed (or killed) tasks are not
> stored.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
