[jira] [Commented] (MAPREDUCE-5570) Map task attempt with fetch failure has incorrect attempt finish time
[ https://issues.apache.org/jira/browse/MAPREDUCE-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13936220#comment-13936220 ] Hudson commented on MAPREDUCE-5570: --- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1727 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1727/]) MAPREDUCE-5570. Map task attempt with fetch failure has incorrect attempt finish time. Contributed by Rushabh S Shah (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1577692) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java > Map task attempt with fetch failure has incorrect attempt finish time > - > > Key: MAPREDUCE-5570 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5570 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am, mrv2 >Affects Versions: 0.23.9, 2.1.1-beta >Reporter: Jason Lowe >Assignee: Rushabh S Shah > Fix For: 2.4.0 > > Attachments: patch-MapReduce-5570-v2.patch, patch-MapReduce-5570.patch > > > If a map task attempt is retroactively failed due to excessive fetch failures > reported by reducers then the attempt's finish time is set to the time the > task was retroactively failed rather than when the task attempt completed. > This causes the map task attempt to appear to have run for much longer than > it actually did. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5570) Map task attempt with fetch failure has incorrect attempt finish time
[ https://issues.apache.org/jira/browse/MAPREDUCE-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13936197#comment-13936197 ] Hudson commented on MAPREDUCE-5570: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1702 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1702/]) MAPREDUCE-5570. Map task attempt with fetch failure has incorrect attempt finish time. Contributed by Rushabh S Shah (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1577692) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java > Map task attempt with fetch failure has incorrect attempt finish time > - > > Key: MAPREDUCE-5570 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5570 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am, mrv2 >Affects Versions: 0.23.9, 2.1.1-beta >Reporter: Jason Lowe >Assignee: Rushabh S Shah > Fix For: 2.4.0 > > Attachments: patch-MapReduce-5570-v2.patch, patch-MapReduce-5570.patch > > > If a map task attempt is retroactively failed due to excessive fetch failures > reported by reducers then the attempt's finish time is set to the time the > task was retroactively failed rather than when the task attempt completed. > This causes the map task attempt to appear to have run for much longer than > it actually did. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5570) Map task attempt with fetch failure has incorrect attempt finish time
[ https://issues.apache.org/jira/browse/MAPREDUCE-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13936148#comment-13936148 ] Hudson commented on MAPREDUCE-5570: --- FAILURE: Integrated in Hadoop-Yarn-trunk #510 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/510/]) MAPREDUCE-5570. Map task attempt with fetch failure has incorrect attempt finish time. Contributed by Rushabh S Shah (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1577692) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java > Map task attempt with fetch failure has incorrect attempt finish time > - > > Key: MAPREDUCE-5570 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5570 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am, mrv2 >Affects Versions: 0.23.9, 2.1.1-beta >Reporter: Jason Lowe >Assignee: Rushabh S Shah > Fix For: 2.4.0 > > Attachments: patch-MapReduce-5570-v2.patch, patch-MapReduce-5570.patch > > > If a map task attempt is retroactively failed due to excessive fetch failures > reported by reducers then the attempt's finish time is set to the time the > task was retroactively failed rather than when the task attempt completed. > This causes the map task attempt to appear to have run for much longer than > it actually did. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5570) Map task attempt with fetch failure has incorrect attempt finish time
[ https://issues.apache.org/jira/browse/MAPREDUCE-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935619#comment-13935619 ] Hudson commented on MAPREDUCE-5570: --- SUCCESS: Integrated in Hadoop-trunk-Commit #5331 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5331/]) MAPREDUCE-5570. Map task attempt with fetch failure has incorrect attempt finish time. Contributed by Rushabh S Shah (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1577692) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java > Map task attempt with fetch failure has incorrect attempt finish time > - > > Key: MAPREDUCE-5570 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5570 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am, mrv2 >Affects Versions: 0.23.9, 2.1.1-beta >Reporter: Jason Lowe >Assignee: Rushabh S Shah > Fix For: 2.4.0 > > Attachments: patch-MapReduce-5570-v2.patch, patch-MapReduce-5570.patch > > > If a map task attempt is retroactively failed due to excessive fetch failures > reported by reducers then the attempt's finish time is set to the time the > task was retroactively failed rather than when the task attempt completed. > This causes the map task attempt to appear to have run for much longer than > it actually did. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5570) Map task attempt with fetch failure has incorrect attempt finish time
[ https://issues.apache.org/jira/browse/MAPREDUCE-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935557#comment-13935557 ] Jason Lowe commented on MAPREDUCE-5570: --- +1 lgtm, committing this. > Map task attempt with fetch failure has incorrect attempt finish time > - > > Key: MAPREDUCE-5570 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5570 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am, mrv2 >Affects Versions: 0.23.9, 2.1.1-beta >Reporter: Jason Lowe >Assignee: Rushabh S Shah > Attachments: patch-MapReduce-5570-v2.patch, patch-MapReduce-5570.patch > > > If a map task attempt is retroactively failed due to excessive fetch failures > reported by reducers then the attempt's finish time is set to the time the > task was retroactively failed rather than when the task attempt completed. > This causes the map task attempt to appear to have run for much longer than > it actually did. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5570) Map task attempt with fetch failure has incorrect attempt finish time
[
https://issues.apache.org/jira/browse/MAPREDUCE-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1393#comment-1393
]
Hadoop QA commented on MAPREDUCE-5570:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12634804/patch-MapReduce-5570-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4427//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4427//console
This message is automatically generated.
> Map task attempt with fetch failure has incorrect attempt finish time
> -
>
> Key: MAPREDUCE-5570
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5570
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
>Affects Versions: 0.23.9, 2.1.1-beta
>Reporter: Jason Lowe
>Assignee: Rushabh S Shah
> Attachments: patch-MapReduce-5570-v2.patch, patch-MapReduce-5570.patch
>
>
> If a map task attempt is retroactively failed due to excessive fetch failures
> reported by reducers then the attempt's finish time is set to the time the
> task was retroactively failed rather than when the task attempt completed.
> This causes the map task attempt to appear to have run for much longer than
> it actually did.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-5570) Map task attempt with fetch failure has incorrect attempt finish time
[ https://issues.apache.org/jira/browse/MAPREDUCE-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13933477#comment-13933477 ] Jason Lowe commented on MAPREDUCE-5570: --- Looks good overall, just some cleanup needed on the test: - The test should verify the task is in the expected state before and after sending the fetch failure event and the task has a non-zero finish time, otherwise we could be passing the test for the wrong reasons (e.g.: finish time is 0 because task does not think it's finished or finish time doesn't change because we ignored the fetch failure event completely because task is in the wrong state). - Nit: The test doesn't need to sleep for a full second, probably only 1 to 10 msec. Just enough time to make sure the finish timestamp would expect to be different if it was updated. A second doesn't seem like much, but it adds up across all the tests if they sleep longer than necessary. - Nit: There's some unnecessary whitespace changes in the patch at the top of the TestTaskAttempt class > Map task attempt with fetch failure has incorrect attempt finish time > - > > Key: MAPREDUCE-5570 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5570 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am, mrv2 >Affects Versions: 0.23.9, 2.1.1-beta >Reporter: Jason Lowe >Assignee: Rushabh S Shah > Attachments: patch-MapReduce-5570.patch > > > If a map task attempt is retroactively failed due to excessive fetch failures > reported by reducers then the attempt's finish time is set to the time the > task was retroactively failed rather than when the task attempt completed. > This causes the map task attempt to appear to have run for much longer than > it actually did. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5570) Map task attempt with fetch failure has incorrect attempt finish time
[
https://issues.apache.org/jira/browse/MAPREDUCE-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932570#comment-13932570
]
Hadoop QA commented on MAPREDUCE-5570:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12634244/patch-MapReduce-5570.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4416//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4416//console
This message is automatically generated.
> Map task attempt with fetch failure has incorrect attempt finish time
> -
>
> Key: MAPREDUCE-5570
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5570
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
>Affects Versions: 0.23.9, 2.1.1-beta
>Reporter: Jason Lowe
>Assignee: Rushabh S Shah
> Attachments: patch-MapReduce-5570.patch
>
>
> If a map task attempt is retroactively failed due to excessive fetch failures
> reported by reducers then the attempt's finish time is set to the time the
> task was retroactively failed rather than when the task attempt completed.
> This causes the map task attempt to appear to have run for much longer than
> it actually did.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
