[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589528#comment-13589528 ] Hudson commented on MAPREDUCE-4693: --- Integrated in Hadoop-Mapreduce-trunk #1358 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1358/]) MAPREDUCE-4693. Historyserver should provide counters for failed tasks. Contributed by Xuan Gong. (Revision 1450956) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1450956 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFailedEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/JobBuilder.java > Historyserver should provide counters for failed tasks > -- > > Key: MAPREDUCE-4693 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 2.0.3-alpha, 0.23.6 >Reporter: Jason Lowe >Assignee: Xuan Gong > Labels: usability > Fix For: 2.0.4-beta > > Attachments: > job_1361915610508_0012-1361920250120-xuan-random-writer-136192043-9-0-FAILED-default.jhist, > MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch, MAPREDUCE-4693.3.patch, > MAPREDUCE-4693.4.patch > > > Currently the historyserver is not providing counters for failed tasks, even > though they are available via the AM as long as the job is still running. > Those counters are lost when the client needs to redirect to the > historyserver after the job completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589504#comment-13589504 ] Hudson commented on MAPREDUCE-4693: --- Integrated in Hadoop-Hdfs-trunk #1330 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1330/]) MAPREDUCE-4693. Historyserver should provide counters for failed tasks. Contributed by Xuan Gong. (Revision 1450956) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1450956 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFailedEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/JobBuilder.java > Historyserver should provide counters for failed tasks > -- > > Key: MAPREDUCE-4693 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 2.0.3-alpha, 0.23.6 >Reporter: Jason Lowe >Assignee: Xuan Gong > Labels: usability > Fix For: 2.0.4-beta > > Attachments: > job_1361915610508_0012-1361920250120-xuan-random-writer-136192043-9-0-FAILED-default.jhist, > MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch, MAPREDUCE-4693.3.patch, > MAPREDUCE-4693.4.patch > > > Currently the historyserver is not providing counters for failed tasks, even > though they are available via the AM as long as the job is still running. > Those counters are lost when the client needs to redirect to the > historyserver after the job completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589429#comment-13589429 ] Hudson commented on MAPREDUCE-4693: --- Integrated in Hadoop-Yarn-trunk #141 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/141/]) MAPREDUCE-4693. Historyserver should provide counters for failed tasks. Contributed by Xuan Gong. (Revision 1450956) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1450956 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFailedEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/JobBuilder.java > Historyserver should provide counters for failed tasks > -- > > Key: MAPREDUCE-4693 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 2.0.3-alpha, 0.23.6 >Reporter: Jason Lowe >Assignee: Xuan Gong > Labels: usability > Fix For: 2.0.4-beta > > Attachments: > job_1361915610508_0012-1361920250120-xuan-random-writer-136192043-9-0-FAILED-default.jhist, > MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch, MAPREDUCE-4693.3.patch, > MAPREDUCE-4693.4.patch > > > Currently the historyserver is not providing counters for failed tasks, even > though they are available via the AM as long as the job is still running. > Those counters are lost when the client needs to redirect to the > historyserver after the job completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13588767#comment-13588767 ] Hudson commented on MAPREDUCE-4693: --- Integrated in Hadoop-trunk-Commit #3392 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3392/]) MAPREDUCE-4693. Historyserver should provide counters for failed tasks. Contributed by Xuan Gong. (Revision 1450956) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1450956 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFailedEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/JobBuilder.java > Historyserver should provide counters for failed tasks > -- > > Key: MAPREDUCE-4693 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 2.0.3-alpha, 0.23.6 >Reporter: Jason Lowe >Assignee: Xuan Gong > Labels: usability > Fix For: 2.0.4-beta > > Attachments: > job_1361915610508_0012-1361920250120-xuan-random-writer-136192043-9-0-FAILED-default.jhist, > MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch, MAPREDUCE-4693.3.patch, > MAPREDUCE-4693.4.patch > > > Currently the historyserver is not providing counters for failed tasks, even > though they are available via the AM as long as the job is still running. > Those counters are lost when the client needs to redirect to the > historyserver after the job completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13588741#comment-13588741 ] Siddharth Seth commented on MAPREDUCE-4693: --- The latest patch (MAPREDUCE-4693.4.patch) looks good. +1 There's a couple of follow up tasks which may be required though. Job level counters in case of job failure. Task.selectBestAttempt needs a fix. > Historyserver should provide counters for failed tasks > -- > > Key: MAPREDUCE-4693 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 2.0.3-alpha, 0.23.6 >Reporter: Jason Lowe >Assignee: Xuan Gong > Labels: usability > Attachments: > job_1361915610508_0012-1361920250120-xuan-random-writer-136192043-9-0-FAILED-default.jhist, > MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch, MAPREDUCE-4693.3.patch, > MAPREDUCE-4693.4.patch > > > Currently the historyserver is not providing counters for failed tasks, even > though they are available via the AM as long as the job is still running. > Those counters are lost when the client needs to redirect to the > historyserver after the job completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13588609#comment-13588609
]
Hadoop QA commented on MAPREDUCE-4693:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12571240/job_1361915610508_0012-1361920250120-xuan-random-writer-136192043-9-0-FAILED-default.jhist
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3372//console
This message is automatically generated.
> Historyserver should provide counters for failed tasks
> --
>
> Key: MAPREDUCE-4693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver, mrv2
>Affects Versions: 2.0.3-alpha, 0.23.6
>Reporter: Jason Lowe
>Assignee: Xuan Gong
> Labels: usability
> Attachments:
> job_1361915610508_0012-1361920250120-xuan-random-writer-136192043-9-0-FAILED-default.jhist,
> MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch, MAPREDUCE-4693.3.patch,
> MAPREDUCE-4693.4.patch
>
>
> Currently the historyserver is not providing counters for failed tasks, even
> though they are available via the AM as long as the job is still running.
> Those counters are lost when the client needs to redirect to the
> historyserver after the job completes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13588605#comment-13588605 ] Xuan Gong commented on MAPREDUCE-4693: -- 1.Running the randomwriter example 2.Using hadoop job -fail-task to fail the task > Historyserver should provide counters for failed tasks > -- > > Key: MAPREDUCE-4693 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 2.0.3-alpha, 0.23.6 >Reporter: Jason Lowe >Assignee: Xuan Gong > Labels: usability > Attachments: > job_1361915610508_0012-1361920250120-xuan-random-writer-136192043-9-0-FAILED-default.jhist, > MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch, MAPREDUCE-4693.3.patch, > MAPREDUCE-4693.4.patch > > > Currently the historyserver is not providing counters for failed tasks, even > though they are available via the AM as long as the job is still running. > Those counters are lost when the client needs to redirect to the > historyserver after the job completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13587509#comment-13587509
]
Hadoop QA commented on MAPREDUCE-4693:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12571043/MAPREDUCE-4693.4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
hadoop-tools/hadoop-rumen.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3362//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3362//console
This message is automatically generated.
> Historyserver should provide counters for failed tasks
> --
>
> Key: MAPREDUCE-4693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver, mrv2
>Affects Versions: 2.0.3-alpha, 0.23.6
>Reporter: Jason Lowe
>Assignee: Xuan Gong
> Labels: usability
> Attachments: MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch,
> MAPREDUCE-4693.3.patch, MAPREDUCE-4693.4.patch
>
>
> Currently the historyserver is not providing counters for failed tasks, even
> though they are available via the AM as long as the job is still running.
> Those counters are lost when the client needs to redirect to the
> historyserver after the job completes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584571#comment-13584571
]
Hadoop QA commented on MAPREDUCE-4693:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12570511/MAPREDUCE-4693.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
hadoop-tools/hadoop-rumen.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3356//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3356//console
This message is automatically generated.
> Historyserver should provide counters for failed tasks
> --
>
> Key: MAPREDUCE-4693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver, mrv2
>Affects Versions: 2.0.3-alpha, 0.23.6
>Reporter: Jason Lowe
>Assignee: Xuan Gong
> Labels: usability
> Attachments: MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch,
> MAPREDUCE-4693.3.patch
>
>
> Currently the historyserver is not providing counters for failed tasks, even
> though they are available via the AM as long as the job is still running.
> Those counters are lost when the client needs to redirect to the
> historyserver after the job completes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583450#comment-13583450 ] Siddharth Seth commented on MAPREDUCE-4693: --- bq. ... and the counters is converted to jobhistory.JhCounters while serializing. Storing the counters as org.apache.hadoop.mapreduce.Counters is to prevent a duplicate copy of the counters till they're actually serialized that's the getDatum() method. (MAPREDUCE-3511) Other than this one change and a couple of minor formatting fixes, the patch looks good. > Historyserver should provide counters for failed tasks > -- > > Key: MAPREDUCE-4693 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 2.0.3-alpha, 0.23.6 >Reporter: Jason Lowe >Assignee: Xuan Gong > Labels: usability > Attachments: MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch > > > Currently the historyserver is not providing counters for failed tasks, even > though they are available via the AM as long as the job is still running. > Those counters are lost when the client needs to redirect to the > historyserver after the job completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582689#comment-13582689
]
Hadoop QA commented on MAPREDUCE-4693:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12570208/MAPREDUCE-4693.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
hadoop-tools/hadoop-rumen.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3347//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3347//console
This message is automatically generated.
> Historyserver should provide counters for failed tasks
> --
>
> Key: MAPREDUCE-4693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver, mrv2
>Affects Versions: 2.0.3-alpha, 0.23.6
>Reporter: Jason Lowe
>Assignee: Xuan Gong
> Labels: usability
> Attachments: MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch
>
>
> Currently the historyserver is not providing counters for failed tasks, even
> though they are available via the AM as long as the job is still running.
> Those counters are lost when the client needs to redirect to the
> historyserver after the job completes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582675#comment-13582675 ] Xuan Gong commented on MAPREDUCE-4693: -- bq:TaskAttempt20LineEventEmitter, Task20LineHistoryEventEmitter don't need to be changed - unless this change is being made in branch-1 as well. I did some changes on TaskFailedEvent and TaskAttemptUnsuccessfulCompletionEvent, add new constructer without counter parameter. Otherwise, there will be error on these two TaskAttempt20LineEventEmitter, Task20LineHistoryEventEmitter since we have added counter as new parameter at the old patch. bq:JobBuilder should be able to handle null counters. At the new patch, handle the null counters, use EMPTY_COUNTERS when the counters is null. bq:TaskFailedEvent and TaskAttemptUnsuccessfulCompletionEvent should store counters as org.apache.hadoop.mapreduce.Counters, and convert to jobhistory.JhCounters only while serializing. (See MapAttemptFinishedEvent). That's to lower the AM memory overhead in case the history events processor falls behind. I think this has already been handled. Already made changes on Event.avro, and the TaskFailed and TaskAttemptUnsuccessfulCompletion will be automatically generated by avro, and the counters is converted to jobhistory.JhCounters while serializing. bq:Needs a unit test. We already have a testcase to test it, I made a simple change just make sure the counters we got back will neither be null nor be empty. > Historyserver should provide counters for failed tasks > -- > > Key: MAPREDUCE-4693 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 2.0.3-alpha, 0.23.6 >Reporter: Jason Lowe >Assignee: Xuan Gong > Labels: usability > Attachments: MAPREDUCE-4693.1.patch, MAPREDUCE-4693.2.patch > > > Currently the historyserver is not providing counters for failed tasks, even > though they are available via the AM as long as the job is still running. > Those counters are lost when the client needs to redirect to the > historyserver after the job completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4693) Historyserver should provide counters for failed tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13581980#comment-13581980
]
Hadoop QA commented on MAPREDUCE-4693:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12570083/MAPREDUCE-4693.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-tools/hadoop-rumen.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3345//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3345//console
This message is automatically generated.
> Historyserver should provide counters for failed tasks
> --
>
> Key: MAPREDUCE-4693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver, mrv2
>Reporter: Jason Lowe
>Assignee: Xuan Gong
> Labels: usability
> Attachments: MAPREDUCE-4693.1.patch
>
>
> Currently the historyserver is not providing counters for failed tasks, even
> though they are available via the AM as long as the job is still running.
> Those counters are lost when the client needs to redirect to the
> historyserver after the job completes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
