[
https://issues.apache.org/jira/browse/TEZ-4638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18001373#comment-18001373
]
Butao Zhang commented on TEZ-4638:
----------------------------------
[~Li0829] Can https://issues.apache.org/jira/browse/TEZ-4032 fix your issue?
By setting configuration* tez.job.fs-server*s , to allow Tez am communicate
with multiple namenode in case of kerberos auth.
> Client authenticate failure when using Kerberos if there is big DAG plan
> needed HDFS
> ------------------------------------------------------------------------------------
>
> Key: TEZ-4638
> URL: https://issues.apache.org/jira/browse/TEZ-4638
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.10.2
> Reporter: Dong0829
> Priority: Major
> Attachments: TEZ-4638.patch
>
>
> Whenever the DAG plan is big and exceed the limit, the DAG plan will be
> uploaded to HDFS. After TEZ AM gets this request, it will need to go to HDFS
> to get the data, but in kerberos cluster, it will face below error:
> {quote}{{10.239.88.12:0. Failed on local exception: java.io.IOException:
> org.apache.hadoop.security.AccessControlException: Client cannot authenticate
> via:[TOKEN, KERBEROS]
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> ....
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.submitDAG(DAGClientAMProtocolBlockingPBServerImpl.java:172)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:8519)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server.processCall(ProtobufRpcEngine.java:484)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:595)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:573)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1227)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1226)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1145)
> at
> java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
> at java.base/javax.security.auth.Subject.doAs(Subject.java:439)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:3388)}}
> {quote}
> For the RCA, its because the submitDAG request is handled by the RPC Sever,
> and the hadoop server will use remote RPC client user as the current UGI
> using doAs (as above stack)
> For the remote UGI, it has no context for the Tez AM which has the tokens
> including KMS, HDFS and so on, so when it talking to the HDFS, it will fail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
