[jira] [Created] (TEZ-2778) Improvements to handle read errors - part 2
Bikas Saha created TEZ-2778: --- Summary: Improvements to handle read errors - part 2 Key: TEZ-2778 URL: https://issues.apache.org/jira/browse/TEZ-2778 Project: Apache Tez Issue Type: Sub-task Reporter: Bikas Saha -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TEZ-2778) Improvements to handle read errors - part 2
[ https://issues.apache.org/jira/browse/TEZ-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha reassigned TEZ-2778: --- Assignee: Bikas Saha > Improvements to handle read errors - part 2 > --- > > Key: TEZ-2778 > URL: https://issues.apache.org/jira/browse/TEZ-2778 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Bikas Saha >Assignee: Bikas Saha > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2745) ClassNotFoundException of user code should fail dag
[
https://issues.apache.org/jira/browse/TEZ-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730245#comment-14730245
]
TezQA commented on TEZ-2745:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754123/TEZ-2745-5.patch
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 14 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.dag.history.logging.ats.TestATSHistoryWithMiniCluster
org.apache.tez.dag.history.logging.ats.TestATSHistoryLoggingService
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1073//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1073//console
This message is automatically generated.
> ClassNotFoundException of user code should fail dag
> ---
>
> Key: TEZ-2745
> URL: https://issues.apache.org/jira/browse/TEZ-2745
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.7.0, 0.5.4, 0.6.2, 0.8.0-alpha
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Attachments: TEZ-2745-1.patch, TEZ-2745-2.patch, TEZ-2745-3.patch,
> TEZ-2745-4.patch, TEZ-2745-5.patch
>
>
> This ClassNotFoundException is not captured now. The current behavior is AM
> crashed and relaunched again until max app attempt is reached.
> Here's user code used in AM:
> * EdgeManager
> * VertexManager
> * InputInitializer
> * OutputCommitter
> * Other user pluggable components (like DAGScheduler, HistoryServiceLogging
> etc.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-2745 PreCommit Build #1073
Jira: https://issues.apache.org/jira/browse/TEZ-2745
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1073/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3159 lines...]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754123/TEZ-2745-5.patch
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 14 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.dag.history.logging.ats.TestATSHistoryWithMiniCluster
org.apache.tez.dag.history.logging.ats.TestATSHistoryLoggingService
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1073//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1073//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
df716207071bcd96213f1e517214dfb89e0890fe logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #1070
Archived 53 artifacts
Archive block size is 32768
Received 2 blocks and 3158928 bytes
Compression is 2.0%
Took 1.3 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
###
## FAILED TESTS (if any)
##
4 tests failed.
FAILED:
org.apache.tez.dag.history.logging.ats.TestATSHistoryLoggingService.testTimelineServiceDisable
Error Message:
org.apache.tez.dag.api.TezReflectionException: Unable to load class:
org.apache.tez.dag.history.ats.acls.ATSHistoryACLPolicyManager
Stack Trace:
org.apache.hadoop.service.ServiceStateException:
org.apache.tez.dag.api.TezReflectionException: Unable to load class:
org.apache.tez.dag.history.ats.acls.ATSHistoryACLPolicyManager
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at
org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:44)
at
org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:78)
at
org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.serviceInit(ATSHistoryLoggingService.java:128)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.tez.dag.history.logging.ats.TestATSHistoryLoggingService.setup(TestATSHistoryLoggingService.java:68)
FAILED:
org.apache.tez.dag.history.logging.ats.TestATSHistoryLoggingService.testATSEventBatching
Error Message:
org.apache.tez.dag.api.TezReflectionException: Unable to load class:
org.apache.tez.dag.history.ats.acls.ATSHistoryACLPolicyManager
Stack Trace:
org.apache.hadoop.service.ServiceStateException:
org.apache.tez.dag.api.TezReflectionException: Unable to load class:
org.apache.tez.dag.history.ats.acls.ATSHistoryACLPolicyManager
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
[jira] [Updated] (TEZ-2745) ClassNotFoundException of user code should fail dag
[ https://issues.apache.org/jira/browse/TEZ-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2745: Attachment: TEZ-2745-5.patch Minor change to fix the TestMRRJobsDAGApi failure. Commit it after the prebuild pass. > ClassNotFoundException of user code should fail dag > --- > > Key: TEZ-2745 > URL: https://issues.apache.org/jira/browse/TEZ-2745 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0, 0.5.4, 0.6.2, 0.8.0-alpha >Reporter: Jeff Zhang >Assignee: Jeff Zhang > Attachments: TEZ-2745-1.patch, TEZ-2745-2.patch, TEZ-2745-3.patch, > TEZ-2745-4.patch, TEZ-2745-5.patch > > > This ClassNotFoundException is not captured now. The current behavior is AM > crashed and relaunched again until max app attempt is reached. > Here's user code used in AM: > * EdgeManager > * VertexManager > * InputInitializer > * OutputCommitter > * Other user pluggable components (like DAGScheduler, HistoryServiceLogging > etc.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-2777) DAG progress calculation in DAGImpl is incorrect
Sreenath Somarajapuram created TEZ-2777: --- Summary: DAG progress calculation in DAGImpl is incorrect Key: TEZ-2777 URL: https://issues.apache.org/jira/browse/TEZ-2777 Project: Apache Tez Issue Type: Bug Reporter: Sreenath Somarajapuram - getProgress function in org.apache.tez.dag.app.dag.impl.DAGImpl calculates the progress by taking average of all vertex progresses, and vertexProgress is average of task progresses. - We are calculating average of averages, which wouldn't give the expected results when total number of tasks in each vertices vary drastically. - Guess it should be average progress of all tasks under a DAG, and not average of vertex progresses. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2774) Reduce logging in the AM
[ https://issues.apache.org/jira/browse/TEZ-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730057#comment-14730057 ] Rajesh Balamohan edited comment on TEZ-2774 at 9/3/15 11:48 PM: YarnTaskSchedulerService - Changes related to "Releasing container", "Container's idle timeout expired. Releasing container", "Releasing held container as either there are pending" could be used for debugging. Can we retain them in INFO logging? Also need to check if changes to tez-container-log4j.properties would have any impact on the tez-tools (e.g swimlane tool). was (Author: rajesh.balamohan): YarnTaskSchedulerService - Changes related to "Releasing container", "Container's idle timeout expired. Releasing container", "Releasing held container as either there are pending" could be used for debugging. Can we retain them in INFO logging? > Reduce logging in the AM > > > Key: TEZ-2774 > URL: https://issues.apache.org/jira/browse/TEZ-2774 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2774.1.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2774) Reduce logging in the AM
[ https://issues.apache.org/jira/browse/TEZ-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730057#comment-14730057 ] Rajesh Balamohan commented on TEZ-2774: --- YarnTaskSchedulerService - Changes related to "Releasing container", "Container's idle timeout expired. Releasing container", "Releasing held container as either there are pending" could be used for debugging. Can we retain them in INFO logging? > Reduce logging in the AM > > > Key: TEZ-2774 > URL: https://issues.apache.org/jira/browse/TEZ-2774 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2774.1.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: TEZ-2774 PreCommit Build #1072
Jira: https://issues.apache.org/jira/browse/TEZ-2774
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1072/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3129 lines...]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754090/TEZ-2774.1.txt
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.dag.app.rm.TestContainerReuse
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1072//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1072//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
13b64c3665c30e7ea3abdb206d5047728758b461 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #1070
Archived 53 artifacts
Archive block size is 32768
Received 2 blocks and 3147824 bytes
Compression is 2.0%
Took 0.79 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
###
## FAILED TESTS (if any)
##
1 tests failed.
REGRESSION:
org.apache.tez.dag.app.rm.TestContainerReuse.testReuseWithTaskSpecificLaunchCmdOption
Error Message:
Wanted but not invoked:
taskSchedulerManagerForTest.taskAllocated(
0,
Mock for TaskAttempt, hashCode: 728200107,
,
Container: [ContainerId: container_1_0001_01_02, NodeId: host3:0,
NodeHttpAddress: host3:0, Resource: , Priority: 1,
Token: null, ]
);
-> at
org.apache.tez.dag.app.rm.TestContainerReuse.testReuseWithTaskSpecificLaunchCmdOption(TestContainerReuse.java:686)
However, there were other interactions with this mock:
taskSchedulerManagerForTest.init(
Configuration: core-default.xml, core-site.xml, yarn-default.xml,
yarn-site.xml
);
-> at
org.apache.tez.dag.app.rm.TestContainerReuse.testReuseWithTaskSpecificLaunchCmdOption(TestContainerReuse.java:537)
taskSchedulerManagerForTest.setConfig(
Configuration: core-default.xml, core-site.xml, yarn-default.xml,
yarn-site.xml
);
-> at
org.apache.tez.dag.app.rm.TestContainerReuse.testReuseWithTaskSpecificLaunchCmdOption(TestContainerReuse.java:537)
taskSchedulerManagerForTest.serviceInit(
Configuration: core-default.xml, core-site.xml, yarn-default.xml,
yarn-site.xml
);
-> at
org.apache.tez.dag.app.rm.TestContainerReuse.testReuseWithTaskSpecificLaunchCmdOption(TestContainerReuse.java:537)
taskSchedulerManagerForTest.start();
-> at
org.apache.tez.dag.app.rm.TestContainerReuse.testReuseWithTaskSpecificLaunchCmdOption(TestContainerReuse.java:538)
taskSchedulerManagerForTest.serviceStart();
-> at
org.apache.tez.dag.app.rm.TestContainerReuse.testReuseWithTaskSpecificLaunchCmdOption(TestContainerReuse.java:538)
taskSchedulerManagerForTest.instantiateSchedulers(
"host",
0,
"",
Mock for AppContext, hashCode: 1643034236
);
-> at
org.apache.tez.dag.app.rm.TestContainerReuse.testReuseWithTaskSpecificLaunchCmdOption(TestContainerReuse.java:538)
taskSchedulerManagerForTest.getContainerSignatureMatcher();
-> at
org.apache.tez.dag.app.rm.TestContainerReuse.testReuseWithTaskSpecificLaunchCmdOption(TestContai
[jira] [Commented] (TEZ-2774) Reduce logging in the AM
[
https://issues.apache.org/jira/browse/TEZ-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730044#comment-14730044
]
TezQA commented on TEZ-2774:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754090/TEZ-2774.1.txt
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.dag.app.rm.TestContainerReuse
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1072//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1072//console
This message is automatically generated.
> Reduce logging in the AM
>
>
> Key: TEZ-2774
> URL: https://issues.apache.org/jira/browse/TEZ-2774
> Project: Apache Tez
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: TEZ-2774.1.txt
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2766) Tez UI: Add vertex in-progress info in DAG details.
[
https://issues.apache.org/jira/browse/TEZ-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730028#comment-14730028
]
TezQA commented on TEZ-2766:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754055/TEZ-2766.1.patch
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.test.TestFaultTolerance
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1071//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1071//console
This message is automatically generated.
> Tez UI: Add vertex in-progress info in DAG details.
> ---
>
> Key: TEZ-2766
> URL: https://issues.apache.org/jira/browse/TEZ-2766
> Project: Apache Tez
> Issue Type: Sub-task
> Components: UI
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-2766.1.patch, TEZ-2766.wip.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-2766 PreCommit Build #1071
Jira: https://issues.apache.org/jira/browse/TEZ-2766
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1071/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 2951 lines...]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754055/TEZ-2766.1.patch
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.test.TestFaultTolerance
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1071//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1071//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
d550f9d9902d8d656a03887aeaeea372836db761 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #1070
Archived 53 artifacts
Archive block size is 32768
Received 6 blocks and 3019430 bytes
Compression is 6.1%
Took 0.96 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
###
## FAILED TESTS (if any)
##
1 tests failed.
REGRESSION:
org.apache.tez.test.TestFaultTolerance.testBasicInputFailureWithExit
Error Message:
expected: but was:
Stack Trace:
java.lang.AssertionError: expected: but was:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:135)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:114)
at
org.apache.tez.test.TestFaultTolerance.testBasicInputFailureWithExit(TestFaultTolerance.java:248)
[jira] [Commented] (TEZ-2774) Reduce logging in the AM
[
https://issues.apache.org/jira/browse/TEZ-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730019#comment-14730019
]
Hitesh Shah commented on TEZ-2774:
--
Comments:
{code}
LOG.info("Comparing client version with AM version"
+ ", clientVersion=" + clientVersion
+ ", AMVersion=" + dagVersionInfo.getVersion());
{code}
- this has been useful at times when debugging rolling upgrade goof-ups.
One-time line per AM instance so would be useful to retain.
{code}
if (LOG.isDebugEnabled()) {
1089 LOG.debug("remoteTaskSpec:" + remoteTaskSpec);
1090}
{code}
- Does this need any additional context on what is being logged? i.e.
taskAttemptId or taskId?
{code}
} else if (vertex.completedTaskCount %50 == 0 || vertex.completedTaskCount
== vertex.tasks.size()) {
2082 LOG.info("Checking tasks for vertex completion for "
2083 + vertex.logIdentifier
{code}
- should this be "modulo max(50, numTasks/50)" i.e log this at max 50 times?
{code}
for (ContainerStatus status : statuses) {
412 LOG.debug("Container " + status.getContainerId() + " is
completed");
413 }
{code}
- should enhance to log container exit code, diagnostics etc.
- maybe log failed containers at info ( not only in this case but others that
were converted to debug )?
- not required if AM is shutting down - which is the case if stopStarted is
true
{code}
LOG.debug("No taskRequests. Container's idle timeout delay expired or is new. "
+
640 "Releasing container"
641 + ", containerId=" + heldContainer.getContainer().getId()
642 + ", containerExpiryTime="
643 + heldContainer.getContainerExpiryTime()
644 + ", idleTimeout=" + idleContainerTimeoutMin
645 + ", taskRequestsCount=" + taskRequests.size()
646 + ", heldContainers=" + heldContainers.size()
647 + ", delayedContainers=" +
delayedContainerManager.delayedContainers.size()
648 + ", isNew=" + isNew);
649 }
639 releaseUnassignedContainers( 650
{code}
- might be worth retaining if this is not a very frequent log. If frequent,
then yes we can remove.
{code}
if (LOG.isDebugEnabled()) {
810 LOG.debug("Releasing held container as either there are
pending but "
811 + " unmatched requests or this is not a session"
812 + ", containerId=" + heldContainer.container.getId()
813 + ", pendingTasks=" + taskRequests.size()
814 + ", isSession=" + getContext().isSession()
815 + ". isNew=" + isNew);
816 }
{code}
- might be worth retaining with a less heavy message.
typo: timelineAclsClas - change to TimelineACLManagerClass
{code}
if (isEnabled) {
82allocatorClassName =
conf.get(TezConfiguration.TEZ_TASK_SCALE_MEMORY_ALLOCATOR_CLASS,
83
TezConfiguration.TEZ_TASK_SCALE_MEMORY_ALLOCATOR_CLASS_DEFAULT);
84 } else {
85allocatorClassName = null;
86 }
{code}
- non logging change?
{code}
private final long LOG_INTERVAL = 3l;
{code}
- from 2 seconds to 30 seconds?
Tez runtime log changes are present. Maybe change jira title?
> Reduce logging in the AM
>
>
> Key: TEZ-2774
> URL: https://issues.apache.org/jira/browse/TEZ-2774
> Project: Apache Tez
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: TEZ-2774.1.txt
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2745) ClassNotFoundException of user code should fail dag
[ https://issues.apache.org/jira/browse/TEZ-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730003#comment-14730003 ] Hitesh Shah commented on TEZ-2745: -- TestMRRJobsDAGApi timeout failure seems to be reproducible locally. > ClassNotFoundException of user code should fail dag > --- > > Key: TEZ-2745 > URL: https://issues.apache.org/jira/browse/TEZ-2745 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0, 0.5.4, 0.6.2, 0.8.0-alpha >Reporter: Jeff Zhang >Assignee: Jeff Zhang > Attachments: TEZ-2745-1.patch, TEZ-2745-2.patch, TEZ-2745-3.patch, > TEZ-2745-4.patch > > > This ClassNotFoundException is not captured now. The current behavior is AM > crashed and relaunched again until max app attempt is reached. > Here's user code used in AM: > * EdgeManager > * VertexManager > * InputInitializer > * OutputCommitter > * Other user pluggable components (like DAGScheduler, HistoryServiceLogging > etc.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2745) ClassNotFoundException of user code should fail dag
[ https://issues.apache.org/jira/browse/TEZ-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729942#comment-14729942 ] Hitesh Shah commented on TEZ-2745: -- +1 for the core changes. May need to check why TestMRRJobsDAGApi is failing. > ClassNotFoundException of user code should fail dag > --- > > Key: TEZ-2745 > URL: https://issues.apache.org/jira/browse/TEZ-2745 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0, 0.5.4, 0.6.2, 0.8.0-alpha >Reporter: Jeff Zhang >Assignee: Jeff Zhang > Attachments: TEZ-2745-1.patch, TEZ-2745-2.patch, TEZ-2745-3.patch, > TEZ-2745-4.patch > > > This ClassNotFoundException is not captured now. The current behavior is AM > crashed and relaunched again until max app attempt is reached. > Here's user code used in AM: > * EdgeManager > * VertexManager > * InputInitializer > * OutputCommitter > * Other user pluggable components (like DAGScheduler, HistoryServiceLogging > etc.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2097) TEZ-UI Add dag logs
[ https://issues.apache.org/jira/browse/TEZ-2097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729895#comment-14729895 ] Hitesh Shah commented on TEZ-2097: -- More comments: - DAGEventStartDag and DAGStartedEvent probably do not need to change. - DAGSubmittedEvent might be the best one to change along with DAGRecovered as they are the 2 main initial events for a DAG within an AM. > TEZ-UI Add dag logs > --- > > Key: TEZ-2097 > URL: https://issues.apache.org/jira/browse/TEZ-2097 > Project: Apache Tez > Issue Type: Bug > Components: UI >Reporter: Jeff Zhang >Priority: Critical > Attachments: TEZ-2097.1.patch > > > If dag fails due to AM error, there's no way to check the dag logs on tez-ui. > Users have to grab the app logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2097) TEZ-UI Add dag logs
[ https://issues.apache.org/jira/browse/TEZ-2097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729891#comment-14729891 ] Hitesh Shah commented on TEZ-2097: -- This needs to account for AM crashes and dag recovery. The DAGRecoveredEvent may need changing too. > TEZ-UI Add dag logs > --- > > Key: TEZ-2097 > URL: https://issues.apache.org/jira/browse/TEZ-2097 > Project: Apache Tez > Issue Type: Bug > Components: UI >Reporter: Jeff Zhang >Priority: Critical > Attachments: TEZ-2097.1.patch > > > If dag fails due to AM error, there's no way to check the dag logs on tez-ui. > Users have to grab the app logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2097) TEZ-UI Add dag logs
[ https://issues.apache.org/jira/browse/TEZ-2097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-2097: - Attachment: TEZ-2097.1.patch [~hitesh], [~pramachandran]. Attached an alternative to the RMWebServices approach. Instead, as early as feasible log the dag containerLogs to the ATS. This prevents us from hitting the RM WebServices and possibly the Generic History Server as the primary source of DAG containerLogs. > TEZ-UI Add dag logs > --- > > Key: TEZ-2097 > URL: https://issues.apache.org/jira/browse/TEZ-2097 > Project: Apache Tez > Issue Type: Bug > Components: UI >Reporter: Jeff Zhang >Priority: Critical > Attachments: TEZ-2097.1.patch > > > If dag fails due to AM error, there's no way to check the dag logs on tez-ui. > Users have to grab the app logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-2776) Add a way to trigger a log dump from the AM at runtime
Siddharth Seth created TEZ-2776: --- Summary: Add a way to trigger a log dump from the AM at runtime Key: TEZ-2776 URL: https://issues.apache.org/jira/browse/TEZ-2776 Project: Apache Tez Issue Type: Sub-task Reporter: Siddharth Seth Assignee: Siddharth Seth Likely a webservice which dumps out state like - # tasks running - # tasks completed per vertex - # events sent to each task per vertex - state of the scheduler - state of other components such as the NodeTracker - #nodes, etc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2774) Reduce logging in the AM
[ https://issues.apache.org/jira/browse/TEZ-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-2774: Attachment: TEZ-2774.1.txt The patch moves quite a bit of logging to DEBUG level. Changes some of the log messages to contain more context, fixes some typos. Contains some changes in the runtime as well. Please take a look and provide feedback, since logging is fairly important towards debugging jobs - especially if a job were to hang. Will wait for feedback before backporting this to older branches. > Reduce logging in the AM > > > Key: TEZ-2774 > URL: https://issues.apache.org/jira/browse/TEZ-2774 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2774.1.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-2775) Reduce logging in runtime components
Siddharth Seth created TEZ-2775: --- Summary: Reduce logging in runtime components Key: TEZ-2775 URL: https://issues.apache.org/jira/browse/TEZ-2775 Project: Apache Tez Issue Type: Sub-task Reporter: Siddharth Seth Specifically Shuffle, which logs a lot for each event being processed and data being fetched. Also PipelinedShuffle is fairly noisy - some of the information from here could be consolidated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2775) Reduce logging in runtime components
[ https://issues.apache.org/jira/browse/TEZ-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-2775: Target Version/s: 0.5.5 > Reduce logging in runtime components > > > Key: TEZ-2775 > URL: https://issues.apache.org/jira/browse/TEZ-2775 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Siddharth Seth > > Specifically Shuffle, which logs a lot for each event being processed and > data being fetched. > Also PipelinedShuffle is fairly noisy - some of the information from here > could be consolidated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2744) Logging cleanup and enhancements
[ https://issues.apache.org/jira/browse/TEZ-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729834#comment-14729834 ] Siddharth Seth commented on TEZ-2744: - Converting this into sub-tasks. Another bit that could be done is to set log levels for certain chatty classes like RackResolver to WARN by default. tez.am.log.level INFO would internally set everything to INFO, except RackResolver would be set to WARN. Setting it back to INFO would required setting this string explicitly. > Logging cleanup and enhancements > > > Key: TEZ-2744 > URL: https://issues.apache.org/jira/browse/TEZ-2744 > Project: Apache Tez > Issue Type: Improvement >Affects Versions: 0.5.0 >Reporter: Siddharth Seth > > There's far too much logging, some of which is redundant information - which > leads to large log files for jobs. > Also, some logging can be enhanced to include additional context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-2774) Reduce logging in the AM
Siddharth Seth created TEZ-2774: --- Summary: Reduce logging in the AM Key: TEZ-2774 URL: https://issues.apache.org/jira/browse/TEZ-2774 Project: Apache Tez Issue Type: Sub-task Reporter: Siddharth Seth Assignee: Siddharth Seth -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2472) Change slf4j version to 1.7.10
[ https://issues.apache.org/jira/browse/TEZ-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-2472: Attachment: TEZ-2472.3.txt Fixed tab on the com.ning line. Will commit after jenkins gets back. > Change slf4j version to 1.7.10 > -- > > Key: TEZ-2472 > URL: https://issues.apache.org/jira/browse/TEZ-2472 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2472.1.txt, TEZ-2472.2.txt, TEZ-2472.3.txt > > > To match the updated version in Hadoop. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2761) Tez UI: update the progress on the various pages on the UI
[ https://issues.apache.org/jira/browse/TEZ-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729818#comment-14729818 ] Hitesh Shah commented on TEZ-2761: -- webservice changes seem good. Is the task/task attempt related webservices being planned for in a separate patch? > Tez UI: update the progress on the various pages on the UI > -- > > Key: TEZ-2761 > URL: https://issues.apache.org/jira/browse/TEZ-2761 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Prakash Ramachandran >Assignee: Prakash Ramachandran > Attachments: TEZ-2761.1.patch, TEZ-2761.2.patch, TEZ-2761.3.patch, > TEZ-2761.wip.1.patch > > > The data displayed on the UI for running tasks should be updated with the > latest status fetched from AM. > This includes the status, progress, task count etc. The api should also allow > querying the values for the tasks (running, failed, succeeded...). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2472) Change slf4j version to 1.7.10
[ https://issues.apache.org/jira/browse/TEZ-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729794#comment-14729794 ] Hitesh Shah commented on TEZ-2472: -- +1 - mind fixing the tabs on the lines using "com.ning" showing up in the diff as part of this commit? > Change slf4j version to 1.7.10 > -- > > Key: TEZ-2472 > URL: https://issues.apache.org/jira/browse/TEZ-2472 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2472.1.txt, TEZ-2472.2.txt > > > To match the updated version in Hadoop. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2472) Change slf4j version to 1.7.10
[ https://issues.apache.org/jira/browse/TEZ-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-2472: Attachment: TEZ-2472.2.txt Updated the patch to allow the version to be set during the build process. > Change slf4j version to 1.7.10 > -- > > Key: TEZ-2472 > URL: https://issues.apache.org/jira/browse/TEZ-2472 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2472.1.txt, TEZ-2472.2.txt > > > To match the updated version in Hadoop. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2472) Change slf4j version to 1.7.10
[ https://issues.apache.org/jira/browse/TEZ-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729777#comment-14729777 ] Siddharth Seth commented on TEZ-2472: - A warning. Which is what we get now running against newer versions of hadoop. > Change slf4j version to 1.7.10 > -- > > Key: TEZ-2472 > URL: https://issues.apache.org/jira/browse/TEZ-2472 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2472.1.txt > > > To match the updated version in Hadoop. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2472) Change slf4j version to 1.7.10
[ https://issues.apache.org/jira/browse/TEZ-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729756#comment-14729756 ] Hitesh Shah commented on TEZ-2472: -- What happens if this is run against older versions of hadoop? > Change slf4j version to 1.7.10 > -- > > Key: TEZ-2472 > URL: https://issues.apache.org/jira/browse/TEZ-2472 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2472.1.txt > > > To match the updated version in Hadoop. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2472) Change slf4j version to 1.7.10
[ https://issues.apache.org/jira/browse/TEZ-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-2472: Attachment: TEZ-2472.1.txt Simple patch to upgrade the version used. [~hitesh] - please review. > Change slf4j version to 1.7.10 > -- > > Key: TEZ-2472 > URL: https://issues.apache.org/jira/browse/TEZ-2472 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2472.1.txt > > > To match the updated version in Hadoop. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2628) History logging plugin to write ATS events to HDFS
[ https://issues.apache.org/jira/browse/TEZ-2628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated TEZ-2628: Attachment: TEZ-2628.003.patch We ran into a problem with the domain files and HiveServer2. The domain files were relying on the client JVM shutting down to close the files, but HiveServer2 never exits and the files were never closing. This caused a pileup of open files over time. Here's an updated patch against master that changes the domain file handling so each domain put writes the data and closes the file. This will not be ideal if there are many domain puts occurring for an application, but in practice we see only one domain put per app. > History logging plugin to write ATS events to HDFS > -- > > Key: TEZ-2628 > URL: https://issues.apache.org/jira/browse/TEZ-2628 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: TEZ-2628.001.patch, TEZ-2628.002.patch, > TEZ-2628.003.patch, hive-timeline.json > > > This provides another history logging alternative that conceptually the same > as the timeline logging service but logs the entities to a file rather than > posting the events to the timeline server directly. When coupled with the > timeline store plugin from YARN-3942 it allows the Tez job to be decoupled > from the timeline server yet the Tez UI can still function properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2766) Tez UI: Add vertex in-progress info in DAG details.
[ https://issues.apache.org/jira/browse/TEZ-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2766: Attachment: TEZ-2766.1.patch - Added realtime vertex progress display to DAG details page - Displays Vertex Name, Progress, Status, Total, Succeeded, Running, Pending Tasks, and Failed, Killed task attempts. - Created an animated progress bar component with percentage display. - Added a new column type to display progress bar in the tables. UTs are depended on TEZ-2725 (UT framework), can be added only after the same is committed. [~pramachandran] Please review. > Tez UI: Add vertex in-progress info in DAG details. > --- > > Key: TEZ-2766 > URL: https://issues.apache.org/jira/browse/TEZ-2766 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2766.1.patch, TEZ-2766.wip.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2761) Tez UI: update the progress on the various pages on the UI
[
https://issues.apache.org/jira/browse/TEZ-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729586#comment-14729586
]
Sreenath Somarajapuram commented on TEZ-2761:
-
Just caught my eye.
Incorrect Indentation in AMWebController.java. 4 spaces used instead of 2.
{code}
+renderJSON(ImmutableMap.of(
+"dag", dagInfo
+));
&
+renderJSON(ImmutableMap.of(
+"vertices", verticesInfo
+));
{code}
But kind of a trivial thing. So +1 LGTM.
> Tez UI: update the progress on the various pages on the UI
> --
>
> Key: TEZ-2761
> URL: https://issues.apache.org/jira/browse/TEZ-2761
> Project: Apache Tez
> Issue Type: Sub-task
> Components: UI
>Reporter: Prakash Ramachandran
>Assignee: Prakash Ramachandran
> Attachments: TEZ-2761.1.patch, TEZ-2761.2.patch, TEZ-2761.3.patch,
> TEZ-2761.wip.1.patch
>
>
> The data displayed on the UI for running tasks should be updated with the
> latest status fetched from AM.
> This includes the status, progress, task count etc. The api should also allow
> querying the values for the tasks (running, failed, succeeded...).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2761) Tez UI: update the progress on the various pages on the UI
[
https://issues.apache.org/jira/browse/TEZ-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729516#comment-14729516
]
TezQA commented on TEZ-2761:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754035/TEZ-2761.3.patch
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1070//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1070//console
This message is automatically generated.
> Tez UI: update the progress on the various pages on the UI
> --
>
> Key: TEZ-2761
> URL: https://issues.apache.org/jira/browse/TEZ-2761
> Project: Apache Tez
> Issue Type: Sub-task
> Components: UI
>Reporter: Prakash Ramachandran
>Assignee: Prakash Ramachandran
> Attachments: TEZ-2761.1.patch, TEZ-2761.2.patch, TEZ-2761.3.patch,
> TEZ-2761.wip.1.patch
>
>
> The data displayed on the UI for running tasks should be updated with the
> latest status fetched from AM.
> This includes the status, progress, task count etc. The api should also allow
> querying the values for the tasks (running, failed, succeeded...).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Success: TEZ-2761 PreCommit Build #1070
Jira: https://issues.apache.org/jira/browse/TEZ-2761
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1070/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3459 lines...]
[INFO] Final Memory: 82M/986M
[INFO]
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754035/TEZ-2761.3.patch
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1070//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1070//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
2871b24ea505f4a23b85cad817e1b45d02c87415 logged out
==
==
Finished build.
==
==
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #1068
Archived 53 artifacts
Archive block size is 32768
Received 28 blocks and 2322784 bytes
Compression is 28.3%
Took 2.1 sec
Description set: TEZ-2761
Recording test results
Email was triggered for: Success
Sending email for trigger: Success
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Updated] (TEZ-2761) Tez UI: update the progress on the various pages on the UI
[ https://issues.apache.org/jira/browse/TEZ-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakash Ramachandran updated TEZ-2761: -- Attachment: TEZ-2761.3.patch addressing comments from patch 2 * stop the polling when route changes or model changes * the am exit/poll race condition will be handled as part of TEZ-2761 * fixed one issue with polling class * fixed findbugs warning [~Sreenath]/[~hitesh] can you review? > Tez UI: update the progress on the various pages on the UI > -- > > Key: TEZ-2761 > URL: https://issues.apache.org/jira/browse/TEZ-2761 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Prakash Ramachandran >Assignee: Prakash Ramachandran > Attachments: TEZ-2761.1.patch, TEZ-2761.2.patch, TEZ-2761.3.patch, > TEZ-2761.wip.1.patch > > > The data displayed on the UI for running tasks should be updated with the > latest status fetched from AM. > This includes the status, progress, task count etc. The api should also allow > querying the values for the tasks (running, failed, succeeded...). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-2773) Tez UI: in progress update handle the last update from AM
Prakash Ramachandran created TEZ-2773: - Summary: Tez UI: in progress update handle the last update from AM Key: TEZ-2773 URL: https://issues.apache.org/jira/browse/TEZ-2773 Project: Apache Tez Issue Type: Sub-task Components: UI Reporter: Prakash Ramachandran Assignee: Prakash Ramachandran TEZ-2761 handles the am in progress update. however if the am exits in between a poll. the ui progress update does not recieve the status that dag has finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2761) Tez UI: update the progress on the various pages on the UI
[
https://issues.apache.org/jira/browse/TEZ-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729175#comment-14729175
]
Sreenath Somarajapuram commented on TEZ-2761:
-
Just observed two more issues:
1. If we move out of DAG details page (Like come back to All Dags), polling is
not stopped and causes console error.
- UI starts hitting RM with an invalid url -
http://192.168.56.101:8088/proxy/undefined/ws/v2/tez/verticesInfo?dagID=&_=1441290684755
2. At times AM stops before the UI can get the final status & commit does not
reach the client.
- DAG details just displays status as running and progress 100%. Also the
polling is not stopped and causes console error.
- Final data returned by AM is
{"dag":{"progress":"1.0","id":"dag_1441285134705_0025_1","status":"RUNNING"}},
may be if AM is not reachable and progress is 1, we must set progress to
SUCCEEDED.
> Tez UI: update the progress on the various pages on the UI
> --
>
> Key: TEZ-2761
> URL: https://issues.apache.org/jira/browse/TEZ-2761
> Project: Apache Tez
> Issue Type: Sub-task
> Components: UI
>Reporter: Prakash Ramachandran
>Assignee: Prakash Ramachandran
> Attachments: TEZ-2761.1.patch, TEZ-2761.2.patch, TEZ-2761.wip.1.patch
>
>
> The data displayed on the UI for running tasks should be updated with the
> latest status fetched from AM.
> This includes the status, progress, task count etc. The api should also allow
> querying the values for the tasks (running, failed, succeeded...).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2761) Tez UI: update the progress on the various pages on the UI
[
https://issues.apache.org/jira/browse/TEZ-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729175#comment-14729175
]
Sreenath Somarajapuram edited comment on TEZ-2761 at 9/3/15 2:43 PM:
-
Just observed two more issues:
1. If we move out of DAG details page while its running(Like come back to All
Dags), polling is not stopped and causes console error.
- UI starts hitting RM with an invalid url -
http://192.168.56.101:8088/proxy/undefined/ws/v2/tez/verticesInfo?dagID=&_=1441290684755
2. At times AM stops before the UI can get the final status & commit does not
reach the client.
- DAG details just displays status as running and progress 100%. Also the
polling is not stopped and causes console error.
- Final data returned by AM is
{"dag":{"progress":"1.0","id":"dag_1441285134705_0025_1","status":"RUNNING"}},
may be if AM is not reachable and progress is 1, we must set progress to
SUCCEEDED.
was (Author: sreenath):
Just observed two more issues:
1. If we move out of DAG details page (Like come back to All Dags), polling is
not stopped and causes console error.
- UI starts hitting RM with an invalid url -
http://192.168.56.101:8088/proxy/undefined/ws/v2/tez/verticesInfo?dagID=&_=1441290684755
2. At times AM stops before the UI can get the final status & commit does not
reach the client.
- DAG details just displays status as running and progress 100%. Also the
polling is not stopped and causes console error.
- Final data returned by AM is
{"dag":{"progress":"1.0","id":"dag_1441285134705_0025_1","status":"RUNNING"}},
may be if AM is not reachable and progress is 1, we must set progress to
SUCCEEDED.
> Tez UI: update the progress on the various pages on the UI
> --
>
> Key: TEZ-2761
> URL: https://issues.apache.org/jira/browse/TEZ-2761
> Project: Apache Tez
> Issue Type: Sub-task
> Components: UI
>Reporter: Prakash Ramachandran
>Assignee: Prakash Ramachandran
> Attachments: TEZ-2761.1.patch, TEZ-2761.2.patch, TEZ-2761.wip.1.patch
>
>
> The data displayed on the UI for running tasks should be updated with the
> latest status fetched from AM.
> This includes the status, progress, task count etc. The api should also allow
> querying the values for the tasks (running, failed, succeeded...).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2761) Tez UI: update the progress on the various pages on the UI
[
https://issues.apache.org/jira/browse/TEZ-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729175#comment-14729175
]
Sreenath Somarajapuram edited comment on TEZ-2761 at 9/3/15 2:44 PM:
-
Just observed two more issues:
1. If we move out of DAG details page while its running(Like come back to All
Dags), polling is not stopped and causes console error.
- UI starts hitting RM with an invalid url -
http://192.168.56.101:8088/proxy/undefined/ws/v2/tez/verticesInfo?dagID=&_=1441290684755
- May be if appId is undefined we can stop polling.
2. At times AM stops before the UI can get the final status & commit does not
reach the client.
- DAG details just displays status as running and progress 100%. Also the
polling is not stopped and causes console error.
- Final data returned by AM is
{"dag":{"progress":"1.0","id":"dag_1441285134705_0025_1","status":"RUNNING"}},
may be if AM is not reachable and progress is 1, we must set progress to
SUCCEEDED.
was (Author: sreenath):
Just observed two more issues:
1. If we move out of DAG details page while its running(Like come back to All
Dags), polling is not stopped and causes console error.
- UI starts hitting RM with an invalid url -
http://192.168.56.101:8088/proxy/undefined/ws/v2/tez/verticesInfo?dagID=&_=1441290684755
2. At times AM stops before the UI can get the final status & commit does not
reach the client.
- DAG details just displays status as running and progress 100%. Also the
polling is not stopped and causes console error.
- Final data returned by AM is
{"dag":{"progress":"1.0","id":"dag_1441285134705_0025_1","status":"RUNNING"}},
may be if AM is not reachable and progress is 1, we must set progress to
SUCCEEDED.
> Tez UI: update the progress on the various pages on the UI
> --
>
> Key: TEZ-2761
> URL: https://issues.apache.org/jira/browse/TEZ-2761
> Project: Apache Tez
> Issue Type: Sub-task
> Components: UI
>Reporter: Prakash Ramachandran
>Assignee: Prakash Ramachandran
> Attachments: TEZ-2761.1.patch, TEZ-2761.2.patch, TEZ-2761.wip.1.patch
>
>
> The data displayed on the UI for running tasks should be updated with the
> latest status fetched from AM.
> This includes the status, progress, task count etc. The api should also allow
> querying the values for the tasks (running, failed, succeeded...).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2761) Tez UI: update the progress on the various pages on the UI
[ https://issues.apache.org/jira/browse/TEZ-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729046#comment-14729046 ] Sreenath Somarajapuram commented on TEZ-2761: - +1 looks good to me. > Tez UI: update the progress on the various pages on the UI > -- > > Key: TEZ-2761 > URL: https://issues.apache.org/jira/browse/TEZ-2761 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Prakash Ramachandran >Assignee: Prakash Ramachandran > Attachments: TEZ-2761.1.patch, TEZ-2761.2.patch, TEZ-2761.wip.1.patch > > > The data displayed on the UI for running tasks should be updated with the > latest status fetched from AM. > This includes the status, progress, task count etc. The api should also allow > querying the values for the tasks (running, failed, succeeded...). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2761) Tez UI: update the progress on the various pages on the UI
[
https://issues.apache.org/jira/browse/TEZ-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729035#comment-14729035
]
TezQA commented on TEZ-2761:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12753977/TEZ-2761.2.patch
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1069//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-TEZ-Build/1069//artifact/patchprocess/newPatchFindbugsWarningstez-dag.html
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1069//console
This message is automatically generated.
> Tez UI: update the progress on the various pages on the UI
> --
>
> Key: TEZ-2761
> URL: https://issues.apache.org/jira/browse/TEZ-2761
> Project: Apache Tez
> Issue Type: Sub-task
> Components: UI
>Reporter: Prakash Ramachandran
>Assignee: Prakash Ramachandran
> Attachments: TEZ-2761.1.patch, TEZ-2761.2.patch, TEZ-2761.wip.1.patch
>
>
> The data displayed on the UI for running tasks should be updated with the
> latest status fetched from AM.
> This includes the status, progress, task count etc. The api should also allow
> querying the values for the tasks (running, failed, succeeded...).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-2761 PreCommit Build #1069
Jira: https://issues.apache.org/jira/browse/TEZ-2761
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1069/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3463 lines...]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12753977/TEZ-2761.2.patch
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1069//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-TEZ-Build/1069//artifact/patchprocess/newPatchFindbugsWarningstez-dag.html
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1069//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
dfb5c15b5e24b0ef4263c27e0442ed66364d429d logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #1068
Archived 53 artifacts
Archive block size is 32768
Received 8 blocks and 2984495 bytes
Compression is 8.1%
Took 0.82 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Updated] (TEZ-2761) Tez UI: update the progress on the various pages on the UI
[ https://issues.apache.org/jira/browse/TEZ-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakash Ramachandran updated TEZ-2761: -- Attachment: TEZ-2761.2.patch addressed the comments from the review. * for the polling code will check if changes suggested (start next poll only after current one is over) separately. it does that already for the dag info (it checks if the record is being loaded, if so does not start a new request) * changing the dagID to dagIndex - will take this separately in separate jira. [~Sreenath]/[~hitesh] can you review? > Tez UI: update the progress on the various pages on the UI > -- > > Key: TEZ-2761 > URL: https://issues.apache.org/jira/browse/TEZ-2761 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Prakash Ramachandran >Assignee: Prakash Ramachandran > Attachments: TEZ-2761.1.patch, TEZ-2761.2.patch, TEZ-2761.wip.1.patch > > > The data displayed on the UI for running tasks should be updated with the > latest status fetched from AM. > This includes the status, progress, task count etc. The api should also allow > querying the values for the tasks (running, failed, succeeded...). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2725) Tez UI: Unit tests framework integration
[
https://issues.apache.org/jira/browse/TEZ-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728806#comment-14728806
]
TezQA commented on TEZ-2725:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12753945/TEZ-2725.2.patch
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1068//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1068//console
This message is automatically generated.
> Tez UI: Unit tests framework integration
>
>
> Key: TEZ-2725
> URL: https://issues.apache.org/jira/browse/TEZ-2725
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-2725.1.patch, TEZ-2725.2.patch
>
>
> - Investigate for the best UT framework for Tez UI, and integrate the same
> into the codebase.
> - UTs for each modules would be added as part of the respective patch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Success: TEZ-2725 PreCommit Build #1068
Jira: https://issues.apache.org/jira/browse/TEZ-2725
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1068/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3488 lines...]
[INFO] Final Memory: 82M/916M
[INFO]
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12753945/TEZ-2725.2.patch
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1068//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1068//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
3e2a9e01ec8f0a848f6a61745e049874627fe92c logged out
==
==
Finished build.
==
==
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #1040
Archived 53 artifacts
Archive block size is 32768
Received 19 blocks and 2659476 bytes
Compression is 19.0%
Took 0.63 sec
Description set: TEZ-2725
Recording test results
Email was triggered for: Success
Sending email for trigger: Success
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Commented] (TEZ-2761) Tez UI: update the progress on the various pages on the UI
[ https://issues.apache.org/jira/browse/TEZ-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728749#comment-14728749 ] Sreenath Somarajapuram commented on TEZ-2761: - Also, as we are sending indexes and not ids, guess it might be better to correct the query param name ~ Something like dagIndex & vertexIndex instead of dagId and vertexId. > Tez UI: update the progress on the various pages on the UI > -- > > Key: TEZ-2761 > URL: https://issues.apache.org/jira/browse/TEZ-2761 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Prakash Ramachandran >Assignee: Prakash Ramachandran > Attachments: TEZ-2761.1.patch, TEZ-2761.wip.1.patch > > > The data displayed on the UI for running tasks should be updated with the > latest status fetched from AM. > This includes the status, progress, task count etc. The api should also allow > querying the values for the tasks (running, failed, succeeded...). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2725) Tez UI: Unit tests framework integration
[ https://issues.apache.org/jira/browse/TEZ-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2725: Attachment: TEZ-2725.2.patch Moved tests to src/test folder for standardization & compatibility with pre commit build. > Tez UI: Unit tests framework integration > > > Key: TEZ-2725 > URL: https://issues.apache.org/jira/browse/TEZ-2725 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2725.1.patch, TEZ-2725.2.patch > > > - Investigate for the best UT framework for Tez UI, and integrate the same > into the codebase. > - UTs for each modules would be added as part of the respective patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2754) Tez UI: StartTime & EndTime is not displayed with right format in Graphical View
[ https://issues.apache.org/jira/browse/TEZ-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728682#comment-14728682 ] Sreenath Somarajapuram commented on TEZ-2754: - Thanks [~hitesh] Committed to both master and branch-0.7. > Tez UI: StartTime & EndTime is not displayed with right format in Graphical > View > > > Key: TEZ-2754 > URL: https://issues.apache.org/jira/browse/TEZ-2754 > Project: Apache Tez > Issue Type: Improvement >Affects Versions: 0.6.0, 0.7.0 >Reporter: Jeff Zhang >Assignee: Sreenath Somarajapuram > Attachments: Screen Shot 2015-08-31 at 8.51.05 PM.png, > TEZ-2754.1.patch, TEZ-2754.2.patch, TEZ-2754.3.patch > > > Attach the screenshot, currently the starttime and endtime is displayed with > millsecs, should use human readable format. > \cc [~Sreenath] [~pramachandran] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: TEZ-2745 PreCommit Build #1067
Jira: https://issues.apache.org/jira/browse/TEZ-2745
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1067/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 2953 lines...]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12753926/TEZ-2745-4.patch
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The following test timeouts occurred in :
org.apache.tez.mapreduce.TestMRRJobsDAGApi
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1067//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1067//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
360b1c2953881be5e5048e7ab8577bca225001fb logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #1040
Archived 53 artifacts
Archive block size is 32768
Received 15 blocks and 2725108 bytes
Compression is 15.3%
Took 0.67 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Commented] (TEZ-2745) ClassNotFoundException of user code should fail dag
[
https://issues.apache.org/jira/browse/TEZ-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728654#comment-14728654
]
TezQA commented on TEZ-2745:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12753926/TEZ-2745-4.patch
against master revision a0d59c3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The following test timeouts occurred in :
org.apache.tez.mapreduce.TestMRRJobsDAGApi
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1067//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1067//console
This message is automatically generated.
> ClassNotFoundException of user code should fail dag
> ---
>
> Key: TEZ-2745
> URL: https://issues.apache.org/jira/browse/TEZ-2745
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.7.0, 0.5.4, 0.6.2, 0.8.0-alpha
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Attachments: TEZ-2745-1.patch, TEZ-2745-2.patch, TEZ-2745-3.patch,
> TEZ-2745-4.patch
>
>
> This ClassNotFoundException is not captured now. The current behavior is AM
> crashed and relaunched again until max app attempt is reached.
> Here's user code used in AM:
> * EdgeManager
> * VertexManager
> * InputInitializer
> * OutputCommitter
> * Other user pluggable components (like DAGScheduler, HistoryServiceLogging
> etc.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
