[ https://issues.apache.org/jira/browse/TEZ-1560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516395#comment-14516395 ]
Jeff Zhang edited comment on TEZ-1560 at 4/28/15 5:22 AM: ---------------------------------------------------------- Upload a patch to fix this issue. The issue is that we miss one valid state machine transition from NEW to NEW due to V_SOURCE_VERTEX_RECOVERED. Also add one unit test for this in the patch. [~hitesh] Please help review was (Author: zjffdu): Upload a patch to fix this issue. The issue is that we miss one valid state machine transition from NEW to NEW due to V_SOURCE_VERTEX_RECOVERED. Also add one unit test for this in the patch. > Invalid state machine transition in recovery > -------------------------------------------- > > Key: TEZ-1560 > URL: https://issues.apache.org/jira/browse/TEZ-1560 > Project: Apache Tez > Issue Type: Bug > Reporter: Jeff Zhang > Assignee: Jeff Zhang > Priority: Critical > Attachments: TEZ-1560-1.patch, failed_tez_job.txt.gz > > > {code} > 2014-09-04 16:08:25,504 INFO [main] org.apache.tez.dag.app.dag.impl.DAGImpl: > dag_1409818083015_0001_1 transitioned from NEW to RUNNING > 2014-09-04 16:08:25,504 INFO [AsyncDispatcher event handler] > org.apache.tez.dag.app.dag.impl.VertexImpl: Recovered Vertex State, > vertexId=vertex_1409818083015_0001_1_00 [v1], state=NEW, > numInitedSourceVertices=0, numStartedSourceVertices=0, > numRecoveredSourceVertices=0, recoveredEvents=0, tasksIsNull=false, numTasks=0 > 2014-09-04 16:08:25,505 INFO [AsyncDispatcher event handler] > org.apache.tez.dag.app.dag.impl.VertexImpl: Root Inputs exist for Vertex: v1 > : {Input={InputName=Input}, > {Descriptor=ClassName=org.apache.tez.test.dag.MultiAttemptDAG$NoOpInput, > hasPayload=false}, > {ControllerDescriptor=ClassName=org.apache.tez.test.dag.MultiAttemptDAG$TestRootInputInitializer, > hasPayload=false}} > 2014-09-04 16:08:25,505 INFO [AsyncDispatcher event handler] > org.apache.tez.dag.app.dag.impl.VertexImpl: Starting root input initializer > for input: Input, with class: > [org.apache.tez.test.dag.MultiAttemptDAG$TestRootInputInitializer] > 2014-09-04 16:08:25,506 INFO [AsyncDispatcher event handler] > org.apache.tez.dag.app.dag.impl.VertexImpl: Setting user vertex manager > plugin: > org.apache.tez.test.dag.MultiAttemptDAG$FailOnAttemptVertexManagerPlugin on > vertex: v1 > 2014-09-04 16:08:25,508 INFO [AsyncDispatcher event handler] > org.apache.tez.dag.app.dag.impl.VertexImpl: Creating 2 for vertex: > vertex_1409818083015_0001_1_00 [v1] > 2014-09-04 16:08:25,518 INFO [AsyncDispatcher event handler] > org.apache.tez.dag.app.dag.impl.VertexImpl: Starting root input initializers: > 1 > 2014-09-04 16:08:25,520 INFO [InputInitializer [v1] #0] > org.apache.tez.dag.app.dag.RootInputInitializerManager: Starting > InputInitializer for Input: Input on vertex vertex_1409818083015_0001_1_00 > [v1] > 2014-09-04 16:08:25,522 INFO [AsyncDispatcher event handler] > org.apache.tez.dag.app.dag.RootInputInitializerManager: Succeeded > InputInitializer for Input: Input on vertex vertex_1409818083015_0001_1_00 > [v1] > 2014-09-04 16:08:25,523 INFO [AsyncDispatcher event handler] > org.apache.tez.dag.app.dag.impl.VertexImpl: vertex_1409818083015_0001_1_00 > [v1] transitioned from NEW to INITIALIZING due to event V_INIT > 2014-09-04 16:08:25,523 INFO [AsyncDispatcher event handler] > org.apache.tez.dag.app.dag.impl.VertexImpl: Recovered Vertex State, > vertexId=vertex_1409818083015_0001_1_01 [v2], state=NEW, > numInitedSourceVertices0, numStartedSourceVertices=0, > numRecoveredSourceVertices=1, tasksIsNull=false, numTasks=0 > 2014-09-04 16:08:25,523 ERROR [AsyncDispatcher event handler] > org.apache.tez.dag.app.dag.impl.VertexImpl: Can't handle Invalid event > V_SOURCE_VERTEX_RECOVERED on vertex v2 with vertexId > vertex_1409818083015_0001_1_01 at current state NEW > org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: > V_SOURCE_VERTEX_RECOVERED at NEW > at > org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:388) > at > org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302) > at > org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46) > at > org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448) > at > org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1344) > at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1) > at > org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1641) > at > org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1) > at > org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173) > at > org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106) > at java.lang.Thread.run(Thread.java:745) > 2014-09-04 16:08:25,524 FATAL [AsyncDispatcher event handler] > org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)