[
https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582675#comment-13582675
]
Xuan Gong commented on MAPREDUCE-4693:
--------------------------------------
bq:TaskAttempt20LineEventEmitter, Task20LineHistoryEventEmitter don't need to
be changed - unless this change is being made in branch-1 as well.
I did some changes on TaskFailedEvent and
TaskAttemptUnsuccessfulCompletionEvent, add new constructer without counter
parameter. Otherwise, there will be error on these two
TaskAttempt20LineEventEmitter, Task20LineHistoryEventEmitter since we have
added counter as new parameter at the old patch.
bq:JobBuilder should be able to handle null counters.
At the new patch, handle the null counters, use EMPTY_COUNTERS when the
counters is null.
bq:TaskFailedEvent and TaskAttemptUnsuccessfulCompletionEvent should store
counters as
org.apache.hadoop.mapreduce.Counters, and convert to jobhistory.JhCounters only
while serializing. (See MapAttemptFinishedEvent). That's to lower the AM memory
overhead in case the history events processor falls behind.
I think this has already been handled. Already made changes on Event.avro, and
the TaskFailed and TaskAttemptUnsuccessfulCompletion will be automatically
generated by avro, and the counters is converted to jobhistory.JhCounters while
serializing.
bq:Needs a unit test.
We already have a testcase to test it, I made a simple change just make sure
the counters we got back will neither be null nor be empty.
> Historyserver should provide counters for failed tasks
> ------------------------------------------------------
>
> Key: MAPREDUCE-4693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver, mrv2
> Affects Versions: 2.0.3-alpha, 0.23.6
> Reporter: Jason Lowe
> Assignee: Xuan Gong
> Labels: usability
> Attachments: MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch
>
>
> Currently the historyserver is not providing counters for failed tasks, even
> though they are available via the AM as long as the job is still running.
> Those counters are lost when the client needs to redirect to the
> historyserver after the job completes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
