Failed: TEZ-3807 PreCommit Build #2595
Jira: https://issues.apache.org/jira/browse/TEZ-3807
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2595/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 330.39 KB...]
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] [Help 2]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tez-runtime-library
[INFO] Build failures were ignored.
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12879733/TEZ-3807.002.patch
against master revision 573ac29.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.runtime.library.common.sort.impl.dflt.TestDefaultSorter
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2595//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2595//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
6319609983ab1a61e62cf17a898dcf6a3170e70d logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
1 tests failed.
FAILED:
org.apache.tez.runtime.library.common.sort.impl.dflt.TestDefaultSorter.testEmptyCaseFileLengths
Error Message:
Number of empty partitions did not match!
Stack Trace:
java.lang.AssertionError: Number of empty partitions did not match!
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.tez.runtime.library.common.sort.impl.dflt.TestDefaultSorter.testEmptyCaseFileLengthsHelper(TestDefaultSorter.java:312)
at
org.apache.tez.runtime.library.common.sort.impl.dflt.TestDefaultSorter.testEmptyCaseFileLengths(TestDefaultSorter.java:279)
[jira] [Commented] (TEZ-3807) InMemoryWriter is not tested with RLE enabled
[
https://issues.apache.org/jira/browse/TEZ-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16108166#comment-16108166
]
TezQA commented on TEZ-3807:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12879733/TEZ-3807.002.patch
against master revision 573ac29.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.runtime.library.common.sort.impl.dflt.TestDefaultSorter
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2595//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2595//console
This message is automatically generated.
> InMemoryWriter is not tested with RLE enabled
> -
>
> Key: TEZ-3807
> URL: https://issues.apache.org/jira/browse/TEZ-3807
> Project: Apache Tez
> Issue Type: Test
>Reporter: Muhammad Samir Khan
>Assignee: Muhammad Samir Khan
> Attachments: TEZ-3807.001.patch, TEZ-3807.002.patch
>
>
> In TestIFile, A couple of test cases are supposed to test InMemoryWriter with
> RLE enabled but the RLE flag is turned off.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (TEZ-3807) InMemoryWriter is not tested with RLE enabled
[ https://issues.apache.org/jira/browse/TEZ-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16108089#comment-16108089 ] Muhammad Samir Khan commented on TEZ-3807: -- Fixed both. > InMemoryWriter is not tested with RLE enabled > - > > Key: TEZ-3807 > URL: https://issues.apache.org/jira/browse/TEZ-3807 > Project: Apache Tez > Issue Type: Test >Reporter: Muhammad Samir Khan >Assignee: Muhammad Samir Khan > Attachments: TEZ-3807.001.patch, TEZ-3807.002.patch > > > In TestIFile, A couple of test cases are supposed to test InMemoryWriter with > RLE enabled but the RLE flag is turned off. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (TEZ-3807) InMemoryWriter is not tested with RLE enabled
[ https://issues.apache.org/jira/browse/TEZ-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Samir Khan updated TEZ-3807: - Attachment: TEZ-3807.002.patch > InMemoryWriter is not tested with RLE enabled > - > > Key: TEZ-3807 > URL: https://issues.apache.org/jira/browse/TEZ-3807 > Project: Apache Tez > Issue Type: Test >Reporter: Muhammad Samir Khan >Assignee: Muhammad Samir Khan > Attachments: TEZ-3807.001.patch, TEZ-3807.002.patch > > > In TestIFile, A couple of test cases are supposed to test InMemoryWriter with > RLE enabled but the RLE flag is turned off. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TEZ-3803) Tasks can get killed due to insufficient progress while waiting for shuffle inputs to complete
[ https://issues.apache.org/jira/browse/TEZ-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107962#comment-16107962 ] Jason Lowe commented on TEZ-3803: - Thanks for updating the patch! I'm still confused why we need a background thread for reporting progress, because a thread seems like overkill for this. Why is it not sufficient to ping progress in the thread that is already waiting for fetchers to complete or new inputs rather than creating a new thread so the old one can wait forever? > Tasks can get killed due to insufficient progress while waiting for shuffle > inputs to complete > -- > > Key: TEZ-3803 > URL: https://issues.apache.org/jira/browse/TEZ-3803 > Project: Apache Tez > Issue Type: Bug >Reporter: Kuhu Shukla >Assignee: Kuhu Shukla >Priority: Critical > Attachments: TEZ-3803.001.patch, TEZ-3803.002.patch, > TEZ-3803.003.patch > > > In a scenario where a downstream task has no slow start and gets started > before all its shuffle inputs are done, the task can timeout as the wait does > not notify progress( set the "progress is being made bit") like it does in > MapReduce. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TEZ-3807) InMemoryWriter is not tested with RLE enabled
[ https://issues.apache.org/jira/browse/TEZ-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107916#comment-16107916 ] Jason Lowe commented on TEZ-3807: - Thanks for the report and the patch! The updated IFile Writer's constructor is always setting rle to true instead of honoring the constructor parameter. TestIFile#writeTestFile now ignores its codec parameter. > InMemoryWriter is not tested with RLE enabled > - > > Key: TEZ-3807 > URL: https://issues.apache.org/jira/browse/TEZ-3807 > Project: Apache Tez > Issue Type: Test >Reporter: Muhammad Samir Khan >Assignee: Muhammad Samir Khan > Attachments: TEZ-3807.001.patch > > > In TestIFile, A couple of test cases are supposed to test InMemoryWriter with > RLE enabled but the RLE flag is turned off. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Failed: TEZ-3803 PreCommit Build #2594
Jira: https://issues.apache.org/jira/browse/TEZ-3803
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2594/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 332.32 KB...]
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tez-yarn-timeline-history-with-acls
[INFO] Build failures were ignored.
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12879654/TEZ-3803.003.patch
against master revision 573ac29.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.history.TestHistoryParser
org.apache.tez.dag.history.ats.acls.TestATSHistoryWithACLs
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2594//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2594//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
6ba125edf6a312c1255a0b19d17fff19934d7908 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
[Fast Archiver] Compressed 3.50 MB of artifacts by 33.9% relative to #2593
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
5 tests failed.
FAILED: org.apache.tez.history.TestHistoryParser.testParserWithFailedJob
Error Message:
null
Stack Trace:
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.tez.history.TestHistoryParser.testParserWithFailedJob(TestHistoryParser.java:383)
FAILED: org.apache.tez.history.TestHistoryParser.testParserWithSuccessfulJob
Error Message:
null
Stack Trace:
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.tez.history.TestHistoryParser.testParserWithSuccessfulJob(TestHistoryParser.java:207)
FAILED:
org.apache.tez.dag.history.ats.acls.TestATSHistoryWithACLs.testDagLoggingEnabled
Error Message:
expected:<200> but was:<404>
Stack Trace:
java.lang.AssertionError: expected:<200> but was:<404>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.tez.dag.history.ats.acls.TestATSHistoryWithACLs.testDagLoggingEnabled(TestATSHistoryWithACLs.java:460)
FAILED:
org.apache.tez.dag.history.ats.acls.TestATSHistoryWithACLs.testSimpleAMACls
Error Message:
expected:<200> but was:<404>
Stack Trace:
java.lang.AssertionError: expected:<200> but was:<404>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.tez.dag.history.ats.acls.TestATSHistoryWithACLs.getTimelineData(TestATSHistoryWithACLs.java:149)
at
org.apache.tez.dag.his
[jira] [Commented] (TEZ-3803) Tasks can get killed due to insufficient progress while waiting for shuffle inputs to complete
[
https://issues.apache.org/jira/browse/TEZ-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107595#comment-16107595
]
TezQA commented on TEZ-3803:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12879654/TEZ-3803.003.patch
against master revision 573ac29.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.history.TestHistoryParser
org.apache.tez.dag.history.ats.acls.TestATSHistoryWithACLs
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2594//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2594//console
This message is automatically generated.
> Tasks can get killed due to insufficient progress while waiting for shuffle
> inputs to complete
> --
>
> Key: TEZ-3803
> URL: https://issues.apache.org/jira/browse/TEZ-3803
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Kuhu Shukla
>Assignee: Kuhu Shukla
>Priority: Critical
> Attachments: TEZ-3803.001.patch, TEZ-3803.002.patch,
> TEZ-3803.003.patch
>
>
> In a scenario where a downstream task has no slow start and gets started
> before all its shuffle inputs are done, the task can timeout as the wait does
> not notify progress( set the "progress is being made bit") like it does in
> MapReduce.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (TEZ-3803) Tasks can get killed due to insufficient progress while waiting for shuffle inputs to complete
[ https://issues.apache.org/jira/browse/TEZ-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated TEZ-3803: - Attachment: TEZ-3803.003.patch Revamped the patch per Jason's comments. I have not added a new config and am basically deriving the interval from progress timeout config. the wait interval is always 1/3 of the progress interval and the thread waits only if the value is > 0. > Tasks can get killed due to insufficient progress while waiting for shuffle > inputs to complete > -- > > Key: TEZ-3803 > URL: https://issues.apache.org/jira/browse/TEZ-3803 > Project: Apache Tez > Issue Type: Bug >Reporter: Kuhu Shukla >Assignee: Kuhu Shukla >Priority: Critical > Attachments: TEZ-3803.001.patch, TEZ-3803.002.patch, > TEZ-3803.003.patch > > > In a scenario where a downstream task has no slow start and gets started > before all its shuffle inputs are done, the task can timeout as the wait does > not notify progress( set the "progress is being made bit") like it does in > MapReduce. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
