[
https://issues.apache.org/jira/browse/TEZ-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Zhang updated TEZ-1885:
----------------------------
Description:
Here's logs which demonstrate the scenario: Shutdown AM -> submit new DAG
But the error message is not correct (" App master already running a DAG" ), it
should be "AM unable to accept new DAG submissions, In the process of shutting
down"
{code}
2014-12-16 06:26:54,372 INFO [Timer-0] org.apache.tez.dag.app.DAGAppMaster: No
current running DAG, shutting down the AM
2014-12-16 06:26:54,372 INFO [Timer-0] org.apache.tez.dag.app.DAGAppMaster:
Handling DAGAppMaster shutdown
2014-12-16 06:26:56,465 INFO [IPC Server handler 0 on 37746]
org.apache.hadoop.ipc.Server: IPC Server handler 0 on 37746, call
org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.submitDAG from
10.66.7.123:35917 Call#7994 Retry#0: error:
org.apache.tez.dag.api.TezException: App master already running a DAG
org.apache.tez.dag.api.TezException: App master already running a DAG
at
org.apache.tez.dag.app.DAGAppMaster.submitDAGToAppMaster(DAGAppMaster.java:931)
at
org.apache.tez.dag.app.DAGAppMaster$DAGClientHandler.submitDAG(DAGAppMaster.java:1065)
at
org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.submitDAG(DAGClientAMProtocolBlockingPBServerImpl.java:131)
at
org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:8292)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:599)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
2014-12-16 06:26:56,491 INFO [IPC Server handler 0 on 37746]
org.apache.tez.dag.app.DAGAppMaster: Received message to shutdown AM
{code}
was:
{code}
2014-12-16 06:26:54,372 INFO [Timer-0] org.apache.tez.dag.app.DAGAppMaster: No
current running DAG, shutting down the AM
2014-12-16 06:26:54,372 INFO [Timer-0] org.apache.tez.dag.app.DAGAppMaster:
Handling DAGAppMaster shutdown
2014-12-16 06:26:56,465 INFO [IPC Server handler 0 on 37746]
org.apache.hadoop.ipc.Server: IPC Server handler 0 on 37746, call
org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.submitDAG from
10.66.7.123:35917 Call#7994 Retry#0: error:
org.apache.tez.dag.api.TezException: App master already running a DAG
org.apache.tez.dag.api.TezException: App master already running a DAG
at
org.apache.tez.dag.app.DAGAppMaster.submitDAGToAppMaster(DAGAppMaster.java:931)
at
org.apache.tez.dag.app.DAGAppMaster$DAGClientHandler.submitDAG(DAGAppMaster.java:1065)
at
org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.submitDAG(DAGClientAMProtocolBlockingPBServerImpl.java:131)
at
org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:8292)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:599)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
2014-12-16 06:26:56,491 INFO [IPC Server handler 0 on 37746]
org.apache.tez.dag.app.DAGAppMaster: Received message to shutdown AM
{code}
> Should check whether AM is shutting down first in submitDAGToAppMaster
> ----------------------------------------------------------------------
>
> Key: TEZ-1885
> URL: https://issues.apache.org/jira/browse/TEZ-1885
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
>
> Here's logs which demonstrate the scenario: Shutdown AM -> submit new DAG
> But the error message is not correct (" App master already running a DAG" ),
> it should be "AM unable to accept new DAG submissions, In the process of
> shutting down"
> {code}
> 2014-12-16 06:26:54,372 INFO [Timer-0] org.apache.tez.dag.app.DAGAppMaster:
> No current running DAG, shutting down the AM
> 2014-12-16 06:26:54,372 INFO [Timer-0] org.apache.tez.dag.app.DAGAppMaster:
> Handling DAGAppMaster shutdown
> 2014-12-16 06:26:56,465 INFO [IPC Server handler 0 on 37746]
> org.apache.hadoop.ipc.Server: IPC Server handler 0 on 37746, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.submitDAG
> from 10.66.7.123:35917 Call#7994 Retry#0: error:
> org.apache.tez.dag.api.TezException: App master already running a DAG
> org.apache.tez.dag.api.TezException: App master already running a DAG
> at
> org.apache.tez.dag.app.DAGAppMaster.submitDAGToAppMaster(DAGAppMaster.java:931)
> at
> org.apache.tez.dag.app.DAGAppMaster$DAGClientHandler.submitDAG(DAGAppMaster.java:1065)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.submitDAG(DAGClientAMProtocolBlockingPBServerImpl.java:131)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:8292)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:599)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> 2014-12-16 06:26:56,491 INFO [IPC Server handler 0 on 37746]
> org.apache.tez.dag.app.DAGAppMaster: Received message to shutdown AM
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
