Hi, Please be aware that ConnectionRefused exception is has nothing to do w/ authentication. See the description from Hadoop wiki: "You get a ConnectionRefused <https://cwiki.apache.org/confluence/display/HADOOP2/ConnectionRefused> Exception when there is a machine at the address specified, but there is no program listening on the specific TCP port the client is using -and there is no firewall in the way silently dropping TCP connection requests. If you do not know what a TCP connection request is, please consult the specification <http://www.ietf.org/rfc/rfc793.txt>."
This means the namenode on host:port is not reachable in the TCP layer. Maybe there are multiple issues but I'm pretty sure that something is wrong in the K8S net config. BR, G On Fri, Apr 29, 2022 at 6:23 PM Pralabh Kumar <pralabhku...@gmail.com> wrote: > Hi dev Team > > Spark-25355 added the functionality of the proxy user on K8s . However > proxy user on K8s with Kerberized HDFS is not working . It is throwing > exception and > > 22/04/21 17:50:30 WARN Client: Exception encountered while connecting to > the server : org.apache.hadoop.security.AccessControlException: Client > cannot authenticate via:[TOKEN, KERBEROS] > > > Exception in thread "main" java.net.ConnectException: Call From > <driverpod> to <namenode> failed on connection exception: > java.net.ConnectException: Connection refused; For more details see: http: > //wiki.apache.org/hadoop/ConnectionRefused > > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown > Source) > > at > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown > Source) > > at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source) > > at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:831) > > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:755) > > at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1501) > > at org.apache.hadoop.ipc.Client.call(Client.java:1443) > > at org.apache.hadoop.ipc.Client.call(Client.java:1353) > > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228) > > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) > > at com.sun.proxy.$Proxy14.getFileInfo(Unknown Source) > > at > > > > On debugging deep , we found the proxy user doesn't have access to > delegation tokens in case of K8s .SparkSubmit.submit explicitly creating > the proxy user and this user doesn't have delegation token. > > > Please help me with the same. > > > Regards > > Pralabh Kumar > > > >
