[jira] [Commented] (TEZ-2855) NPE while routing events
[
https://issues.apache.org/jira/browse/TEZ-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936486#comment-14936486
]
TezQA commented on TEZ-2855:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12764386/TEZ-2855.2.txt
against master revision b153035.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1186//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1186//console
This message is automatically generated.
> NPE while routing events
>
>
> Key: TEZ-2855
> URL: https://issues.apache.org/jira/browse/TEZ-2855
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
>Priority: Critical
> Attachments: 2855log.gz, TEZ-2855.1.txt, TEZ-2855.2.txt
>
>
> Observed while running against 0.8.0-alpha. This will likely affect 0.7 as
> well - that'll be known after debugging.
> {code}
> 2015-09-24T12:13:42,675 ERROR [Dispatcher thread: Central]
> common.AsyncDispatcher: Error in dispatcher thread
> java.lang.NullPointerException
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handleRoutedTezEvents(VertexImpl.java:4429)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.access$4000(VertexImpl.java:203)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4175)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4167)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1906)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:202)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2069)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2055)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
> 2015-09-24T12:13:42,681 INFO [HistoryEventHandlingThread]
> impl.SimpleHistoryLoggingService: Writing event TASK_ATTEMPT_FINISHED to
> history file
> {code}
> Looks like the VertexManager was null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2855) NPE while routing events
[
https://issues.apache.org/jira/browse/TEZ-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936454#comment-14936454
]
Bikas Saha commented on TEZ-2855:
-
lgtm. would be good if the test was enhanced to check that after init, regular
event handling continues to occur without the intervention of the buffer.
> NPE while routing events
>
>
> Key: TEZ-2855
> URL: https://issues.apache.org/jira/browse/TEZ-2855
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
>Priority: Critical
> Attachments: 2855log.gz, TEZ-2855.1.txt, TEZ-2855.2.txt
>
>
> Observed while running against 0.8.0-alpha. This will likely affect 0.7 as
> well - that'll be known after debugging.
> {code}
> 2015-09-24T12:13:42,675 ERROR [Dispatcher thread: Central]
> common.AsyncDispatcher: Error in dispatcher thread
> java.lang.NullPointerException
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handleRoutedTezEvents(VertexImpl.java:4429)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.access$4000(VertexImpl.java:203)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4175)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4167)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1906)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:202)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2069)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2055)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
> 2015-09-24T12:13:42,681 INFO [HistoryEventHandlingThread]
> impl.SimpleHistoryLoggingService: Writing event TASK_ATTEMPT_FINISHED to
> history file
> {code}
> Looks like the VertexManager was null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2855) NPE while routing events
[
https://issues.apache.org/jira/browse/TEZ-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936452#comment-14936452
]
Jeff Zhang commented on TEZ-2855:
-
+1
> NPE while routing events
>
>
> Key: TEZ-2855
> URL: https://issues.apache.org/jira/browse/TEZ-2855
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
>Priority: Critical
> Attachments: 2855log.gz, TEZ-2855.1.txt, TEZ-2855.2.txt
>
>
> Observed while running against 0.8.0-alpha. This will likely affect 0.7 as
> well - that'll be known after debugging.
> {code}
> 2015-09-24T12:13:42,675 ERROR [Dispatcher thread: Central]
> common.AsyncDispatcher: Error in dispatcher thread
> java.lang.NullPointerException
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handleRoutedTezEvents(VertexImpl.java:4429)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.access$4000(VertexImpl.java:203)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4175)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4167)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1906)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:202)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2069)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2055)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
> 2015-09-24T12:13:42,681 INFO [HistoryEventHandlingThread]
> impl.SimpleHistoryLoggingService: Writing event TASK_ATTEMPT_FINISHED to
> history file
> {code}
> Looks like the VertexManager was null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2855) NPE while routing events
[
https://issues.apache.org/jira/browse/TEZ-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936394#comment-14936394
]
Siddharth Seth commented on TEZ-2855:
-
Thanks for the review. I don't think it matters that events are processed in
the middle of or at the end of initialization. All that matters is that the
VMPlugin is setup and has been initialized. VertexManagers will typically
subscribe for CONFIGURED notifications, before they try to obtain information
like num tasks. There's also additional notifications like parallelism updated
which they can subscribe to. Moving it to the end of Initialized doesn't
necessarily work - since initialization can be deferred till edge managers etc
are setup.
Updating the patch with the unnecessary v3 INIT removed. Please take another
look.
> NPE while routing events
>
>
> Key: TEZ-2855
> URL: https://issues.apache.org/jira/browse/TEZ-2855
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
>Priority: Critical
> Attachments: 2855log.gz, TEZ-2855.1.txt
>
>
> Observed while running against 0.8.0-alpha. This will likely affect 0.7 as
> well - that'll be known after debugging.
> {code}
> 2015-09-24T12:13:42,675 ERROR [Dispatcher thread: Central]
> common.AsyncDispatcher: Error in dispatcher thread
> java.lang.NullPointerException
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handleRoutedTezEvents(VertexImpl.java:4429)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.access$4000(VertexImpl.java:203)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4175)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4167)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1906)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:202)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2069)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2055)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
> 2015-09-24T12:13:42,681 INFO [HistoryEventHandlingThread]
> impl.SimpleHistoryLoggingService: Writing event TASK_ATTEMPT_FINISHED to
> history file
> {code}
> Looks like the VertexManager was null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2855) NPE while routing events
[
https://issues.apache.org/jira/browse/TEZ-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936160#comment-14936160
]
Jeff Zhang commented on TEZ-2855:
-
Comments:
* Should we put the VertexManager initialization code at the end of vertex
initialization ? Because due to the code change of this patch, vertex may
handle VMEvent when it is still uninitialized. That means
VM#onVertexManagerEvent may not behave correctly as some info of the
VertexManagerContext may not set correctly yet (such as numTasks)
* TestVertexImpl, it is not necessary to send init to V3 explicitly. V3's init
should be triggered by the init of V1 & V2
{code}
dispatcher.getEventHandler().handle(new VertexEvent(v3.getVertexId(),
VertexEventType.V_INIT)); // remove this
{code}
> NPE while routing events
>
>
> Key: TEZ-2855
> URL: https://issues.apache.org/jira/browse/TEZ-2855
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
>Priority: Critical
> Attachments: 2855log.gz, TEZ-2855.1.txt
>
>
> Observed while running against 0.8.0-alpha. This will likely affect 0.7 as
> well - that'll be known after debugging.
> {code}
> 2015-09-24T12:13:42,675 ERROR [Dispatcher thread: Central]
> common.AsyncDispatcher: Error in dispatcher thread
> java.lang.NullPointerException
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handleRoutedTezEvents(VertexImpl.java:4429)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.access$4000(VertexImpl.java:203)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4175)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4167)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1906)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:202)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2069)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2055)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
> 2015-09-24T12:13:42,681 INFO [HistoryEventHandlingThread]
> impl.SimpleHistoryLoggingService: Writing event TASK_ATTEMPT_FINISHED to
> history file
> {code}
> Looks like the VertexManager was null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2855) NPE while routing events
[
https://issues.apache.org/jira/browse/TEZ-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936074#comment-14936074
]
TezQA commented on TEZ-2855:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12764327/TEZ-2855.1.txt
against master revision 773.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1184//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1184//console
This message is automatically generated.
> NPE while routing events
>
>
> Key: TEZ-2855
> URL: https://issues.apache.org/jira/browse/TEZ-2855
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
>Priority: Critical
> Attachments: 2855log.gz, TEZ-2855.1.txt
>
>
> Observed while running against 0.8.0-alpha. This will likely affect 0.7 as
> well - that'll be known after debugging.
> {code}
> 2015-09-24T12:13:42,675 ERROR [Dispatcher thread: Central]
> common.AsyncDispatcher: Error in dispatcher thread
> java.lang.NullPointerException
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handleRoutedTezEvents(VertexImpl.java:4429)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.access$4000(VertexImpl.java:203)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4175)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4167)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1906)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:202)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2069)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2055)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
> 2015-09-24T12:13:42,681 INFO [HistoryEventHandlingThread]
> impl.SimpleHistoryLoggingService: Writing event TASK_ATTEMPT_FINISHED to
> history file
> {code}
> Looks like the VertexManager was null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2855) NPE while routing events
[
https://issues.apache.org/jira/browse/TEZ-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14907412#comment-14907412
]
Bikas Saha commented on TEZ-2855:
-
Orthogonal observation - this was greatly simplified by having the
remoteTaskSpec log which showed output vertices for all started attempts.
Without that log, we would have to manually figure out which vertex was the
output of the started attempts and that would have been cumbersome. This log
has been removed now and rightly so because in general its voluminous and not
useful. Perhaps the patch for this jira could add some debug info under
exception handling that would dump the entity and event. Some events may need
better toString() methods to show the info.
> NPE while routing events
>
>
> Key: TEZ-2855
> URL: https://issues.apache.org/jira/browse/TEZ-2855
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.8.0-alpha
>Reporter: Siddharth Seth
>Priority: Critical
> Attachments: 2855log.gz
>
>
> Observed while running against 0.8.0-alpha. This will likely affect 0.7 as
> well - that'll be known after debugging.
> {code}
> 2015-09-24T12:13:42,675 ERROR [Dispatcher thread: Central]
> common.AsyncDispatcher: Error in dispatcher thread
> java.lang.NullPointerException
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handleRoutedTezEvents(VertexImpl.java:4429)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.access$4000(VertexImpl.java:203)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4175)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4167)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1906)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:202)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2069)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2055)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
> 2015-09-24T12:13:42,681 INFO [HistoryEventHandlingThread]
> impl.SimpleHistoryLoggingService: Writing event TASK_ATTEMPT_FINISHED to
> history file
> {code}
> Looks like the VertexManager was null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2855) NPE while routing events
[
https://issues.apache.org/jira/browse/TEZ-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14907400#comment-14907400
]
Bikas Saha commented on TEZ-2855:
-
line 4429 is handling vm event and so vm is null -> vertex is still initializing
exception message does not show which vertex was handling the event. something
we should add under logging improvements
until the exception the following vertices had not assigned vm's but tasks that
write outputs to them had started. so it could be that some of tasks sent vm
events to these vertices before there was a vm to handle them. same would apply
to input initializer events also.
Reducer 29
Reducer 31
Reducer 33
Reducer 35
Reducer 37
Reducer 39
Reducer 41
Reducer 43
Reducer 45
Reducer 47
Reducer 49
> NPE while routing events
>
>
> Key: TEZ-2855
> URL: https://issues.apache.org/jira/browse/TEZ-2855
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.8.0-alpha
>Reporter: Siddharth Seth
>Priority: Critical
> Attachments: 2855log.gz
>
>
> Observed while running against 0.8.0-alpha. This will likely affect 0.7 as
> well - that'll be known after debugging.
> {code}
> 2015-09-24T12:13:42,675 ERROR [Dispatcher thread: Central]
> common.AsyncDispatcher: Error in dispatcher thread
> java.lang.NullPointerException
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handleRoutedTezEvents(VertexImpl.java:4429)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.access$4000(VertexImpl.java:203)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4175)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:4167)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> ~[hadoop-yarn-common-2.6.0.jar:?]
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1906)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:202)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2069)
> ~[TezAppJar.jar:0.8.0-alpha]
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2055)
> ~[TezAppJar.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
> [tez-common-0.8.0-alpha.jar:0.8.0-alpha]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
> 2015-09-24T12:13:42,681 INFO [HistoryEventHandlingThread]
> impl.SimpleHistoryLoggingService: Writing event TASK_ATTEMPT_FINISHED to
> history file
> {code}
> Looks like the VertexManager was null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
