[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581320#comment-14581320
]
Jeff Zhang commented on TEZ-1961:
-
Thanks [~sseth] [~bikassaha], committed to branch-0.7 & master
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Fix For: 0.8.0, 0.7.1
>
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch,
> TEZ-1961-4.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581222#comment-14581222
]
Bikas Saha commented on TEZ-1961:
-
Sure.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch,
> TEZ-1961-4.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580426#comment-14580426
]
Jeff Zhang commented on TEZ-1961:
-
[~bikassaha] If you have no further comment, I will commit it.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch,
> TEZ-1961-4.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569901#comment-14569901
]
Siddharth Seth commented on TEZ-1961:
-
Looks good as far as I'm concerned.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch,
> TEZ-1961-4.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569880#comment-14569880
]
Hitesh Shah commented on TEZ-1961:
--
ping [~bikassaha] [~sseth] for review
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch,
> TEZ-1961-4.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14547524#comment-14547524
]
TezQA commented on TEZ-1961:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12733435/TEZ-1961-4.patch
against master revision cea05e3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/686//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/686//console
This message is automatically generated.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch,
> TEZ-1961-4.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14547500#comment-14547500
]
Jeff Zhang commented on TEZ-1961:
-
bq. I believe the main change here is that submitDag() in non session mode will
wait for the AM to come up and move into RUNNING state (all on the client),
before returning a DAGClient to the user ?
The patch mostly looks good to me.
Yes. that's correct.
[~bikassaha] This patch won't affect the session mode, just make
getAppMasterStatus() available for non-session mode so that TezClient will wait
until the AM move into RUNNING state for non-session mode.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535285#comment-14535285
]
Siddharth Seth commented on TEZ-1961:
-
[~zjffdu] - I believe the main change here is that submitDag() in non session
mode will wait for the AM to come up and move into RUNNING state (all on the
client), before returning a DAGClient to the user ?
The patch mostly looks good to me.
[~bikassaha] - the patch doesn't change waitTillReady(). getAppMasterStatus(),
which seems to be public, is changed to get the AM state for non-session mode
as well. We're already exposing AppMasterStatus.INITIALIZING, SHUTDOWN etc in
non-session mode. READY seems to be the only state which is not exposed in
non-session mode. As long as READY is never sent back in non-session mode, I
think this should be OK to go.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533653#comment-14533653
]
Jeff Zhang commented on TEZ-1961:
-
I also rename DAGClientHandler#getSessionState to DAGClientHandler#getAMState
in this patch to avoid misleading. If it is valid, I can do it in this patch,
because the it is a very simple change.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533647#comment-14533647
]
Bikas Saha commented on TEZ-1961:
-
Its actually session state. Perhaps we should do that in a separate jira.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533607#comment-14533607
]
Jeff Zhang commented on TEZ-1961:
-
bq. That was because it returns session status, which makes no sense in
non-session mode.
I check the DAGClientHandler#getTezAppMasterStatus, it just return the
DAGAppMaster's state, The state should be also valid in non-session mode, right
?
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533202#comment-14533202
]
Bikas Saha commented on TEZ-1961:
-
bq. Previously DAGClientAMProtocol#getAMStatus is not supported for non-session
mode
That was because it returns session status, which makes no sense in non-session
mode. Please dont change that.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532244#comment-14532244
]
Jeff Zhang commented on TEZ-1961:
-
[~sseth] Please help review it.
* Wait for DAGAppMaster go to RUNNING/SHUTDOWN, then return DAGClient in
non-session mode. This can ensure that dag has started to run. [~bikassaha]
Previously DAGClientAMProtocol#getAMStatus is not supported for non-session
mode, is there any considering for that ? I make it supported under non-session
mode in this patch.
* The patch cause a little difference on the tracking URL of application. This
is one bug of YARN which has been been solved in YARN-2246 (solved in
hadoop-2.7)
The bug is there may be some suffix at the end of trackingURL when app move
from SUBMITTED to RUNNING. So after this patch, the trackingURL will change
from http://localhost:53419/proxy/application_1430963524753_0005 to
http://localhost:53419/proxy/application_1430963524753_0005/ui/
* Still keep the null currentDAG check in DAGClientHandler as sanity check.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532138#comment-14532138
]
TezQA commented on TEZ-1961:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12731085/TEZ-1961-3.patch
against master revision 02870f0.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/649//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/649//console
This message is automatically generated.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch, TEZ-1961-3.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532034#comment-14532034
]
TezQA commented on TEZ-1961:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12731077/TEZ-1961-2.patch
against master revision 02870f0.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.dag.app.TestPreemption
Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/647//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-TEZ-Build/647//artifact/patchprocess/newPatchFindbugsWarningstez-dag.html
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/647//console
This message is automatically generated.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Jeff Zhang
>Priority: Critical
> Attachments: TEZ-1961-1.patch, TEZ-1961-2.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531129#comment-14531129
]
Siddharth Seth commented on TEZ-1961:
-
Sounds good.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Priority: Critical
> Attachments: TEZ-1961-1.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14530410#comment-14530410
]
Jeff Zhang commented on TEZ-1961:
-
[~sseth] indicating NoRunningDag via a valid RPC response looks like a little
complicated. I have to add one flag to RPC response of getVertexStatus &
getDAGStatus and check that flag in these 2 methods.
After more deep dive, I found that this "No running dag" issue only happens for
non-session mode. In session mode, each dag submission will return rpc response
after the dag is set in DAGAppMaster. But in non-session mode, we generate
dagId in client side and don't wait for dag been set in DAGAppMaster. I am
working on one patch to return DAGClientImpl to client after dag been set in
DAGAppMaster. Almost done, still need to fix some test failure.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Priority: Critical
> Attachments: TEZ-1961-1.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14530064#comment-14530064
]
Jeff Zhang commented on TEZ-1961:
-
Right, thanks [~sseth], will upload a new patch to fix it.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Priority: Critical
> Attachments: TEZ-1961-1.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14530052#comment-14530052
]
Siddharth Seth commented on TEZ-1961:
-
[~zjffdu] - this may be a slightly more involved fix. Just logging is not
sufficient since that can lead to NPEs and other errors. The findbugs warning
is very relevant in this case.
The RPC layer is logging this message - which will likely happen for any
Exception. We could think of indicating NoRunningDag via a valid RPC response
(instead of throwing an exception).
getDAGStatusViaAM in DAGClientImpl or getDAGStatusViaAM in DAGClientRPCImpl
handle TezExceptions and suppress them, which is why this works (the
documentation mentions "// can be either due to a n/w issue of due to AM
completed." which is not the case here so may need to be updated).
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Priority: Critical
> Attachments: TEZ-1961-1.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1961) Remove misleading exception "No running dag" from AM logs
[
https://issues.apache.org/jira/browse/TEZ-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529978#comment-14529978
]
TezQA commented on TEZ-1961:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12730715/TEZ-1961-1.patch
against master revision 5679b28.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/634//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-TEZ-Build/634//artifact/patchprocess/newPatchFindbugsWarningstez-dag.html
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/634//console
This message is automatically generated.
> Remove misleading exception "No running dag" from AM logs
> -
>
> Key: TEZ-1961
> URL: https://issues.apache.org/jira/browse/TEZ-1961
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Priority: Critical
> Attachments: TEZ-1961-1.patch
>
>
> {code}
> 15/01/14 16:45:06 INFO ipc.Server: IPC Server handler 0 on 51000, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getDAGStatus
> from Call#0 Retry#0
> org.apache.tez.dag.api.TezException: No running dag at present
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getDAG(DAGClientHandler.java:84)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getACLManager(DAGClientHandler.java:151)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:94)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2041)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2037)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2035)
> 15/01/14 16:45:06 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> {code}
> This exception shows up fairly often and isn't very relevant - queries before
> a DAG is submitted to the AM.
> This is very misleading, especially for folks new to Tez, and should be
> removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
