Stanislav Matytsin created HADOOP-17447:
-------------------------------------------
Summary: Exception "Server has invalid Kerberos principal" when
try to connect using reverseDNS hostname by IP resolving
Key: HADOOP-17447
URL: https://issues.apache.org/jira/browse/HADOOP-17447
Project: Hadoop Common
Issue Type: Bug
Components: common
Affects Versions: 3.1.3
Reporter: Stanislav Matytsin
I try to create filesystem object to operate with remote kerberised cluster but
there is an exception because of fail verification NameNode server principal
with principal from config
The reason of exception:
In org.apache.hadoop.security.SaslRpcClient#getServerPrincipal there is
verifying process of NameNode server principal and principal which gets from
config. In config principal keeps in format nn/[email protected] where _HOST is
a placeholder for real NameNode host
Then there is replacing of _HOST placeholder with real host name of name node.
And real host name gets as result of InetAddress.getCanonicalHostName() method
(look
/Users/a16689075/.m2/repository/org/apache/hadoop/hadoop-common/3.1.3/hadoop-common-3.1.3-sources.jar!/org/apache/hadoop/security/SecurityUtil.java:211)
But if there is some reverse DNS in infrastructure where I run this code,
reverse DNS resolves host by IP and return host with dot at the end. So
InetAddress.getCanonicalHostName() returns real host of NameNode but with dot
at the end: nn/ex1.example.com. (the same is actual when you execute for
example nslookup ip-addr in command line)
So principal from config after placeholder replacement looks like
nn/[email protected] (where hostname keeps dot at the end)
Then there is checking if nameNode server pricipal is equals with principal
from config. And of course they are different because of dot exists at the end
of hostname in config principal
At the same time we can look at constructor
sun.security.krb5.PrincipalName#PrincipalName(java.lang.String, int,
java.lang.String where we can see similar logic for processing
InetAddress.getCanonicalHostName();
If InetAddress.getCanonicalHostName() received host with dot at the end, there
it cuts this dot and keeps only hostname without dot
Cluster has Kerberos Auth and configs with HA mode for HDFS with 2 namenodes
My code to reproduce this issue is:
...
HdfsConfiguration conf = new HdfsConfiguration(false);
conf.addResource(hdfsSiteXmlInputStream);
conf.addResource(coreSiteXmlInputStream)));
Path hdfsKeytabPath = Paths.get("./hdfs.keytab");
PrincipalName hdfsPrincipalName =
KerberosUtils.getPrincipalFromKeytab(hdfsKeytabPath.toFile());
UserGroupInformation.setConfiguration(conf);
UserGroupInformation.loginUserFromKeytab(hdfsPrincipalName.getName(),
hdfsKeytabPath.toAbsolutePath().toString());
fileSystem = FileSystem.get(conf);
Exception:
java.lang.IllegalArgumentException: Server has invalid Kerberos principal:
nn/[email protected], expecting: nn/[email protected]
at
org.apache.hadoop.security.SaslRpcClient.getServerPrincipal(SaslRpcClient.java:337)
at
org.apache.hadoop.security.SaslRpcClient.createSaslClient(SaslRpcClient.java:234)
at
org.apache.hadoop.security.SaslRpcClient.selectSaslClient(SaslRpcClient.java:160)
at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:390)
at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:627)
at org.apache.hadoop.ipc.Client$Connection.access$2300(Client.java:421)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:814)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:810)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:810)
... 73 common frames omitted
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
