thanks, but my issue is name node could *Login successful,but second namenode couldn't. and name node got a **HttpServer.start() threw a non Bind IOException:*
hdfs-site.xml: *<property>* * <name>dfs.webhdfs.enabled</name>* * <value>true</value>* *</property>* *<property>* * <name>dfs.block.access.token.enable</name>* * <value>true</value>* *</property>* *<!-- NameNode security config -->* *<property>* * <name>dfs.namenode.kerberos.principal</name>* * <value>hadoop/_h...@example.com <h...@example.com></value>* *</property>* *<property>* * <name>dfs.namenode.keytab.file</name>* * <value>/etc/hadoop/conf/hdfs.keytab</value>* *</property>* *<property>* * <name>dfs.https.port</name>* * <value>50470</value>* *</property>* *<property>* * <name>dfs.namenode.https-address</name>* * <value>dmp1.example.com:50470 <http://dmp1.example.com:50470></value>* *</property>* *<property>* * <name>dfs.namenode.kerberos.internal.spnego.principa</name>* * <value>HTTP/_h...@example.com <h...@example.com></value>* *</property>* *<property>* * <name>dfs.web.authentication.kerberos.keytab</name>* * <value>/etc/hadoop/conf/hdfs.keytab</value>* *</property>* *<property>* * <name>dfs.http.policy</name>* * <value>HTTPS_ONLY</value>* *</property>* *<property>* * <name>dfs.https.enable</name>* * <value>true</value>* *</property>* *<!-- secondary NameNode security config -->* *<property>* * <name>dfs.namenode.secondary.http-address</name>* * <value>dmp1.example.com:50090 <http://dmp1.example.com:50090></value>* *</property>* *<property>* * <name>dfs.secondary.namenode.keytab.file</name>* * <value>/etc/hadoop/conf/hdfs.keytab</value>* *</property>* *<property>* * <name>dfs.secondary.namenode.kerberos.principa</name>* * <value>hadoop/_h...@example.com <h...@example.com></value>* *</property> * *<property>* * <name>dfs.secondary.namenode.kerberos.internal.spnego.principal</name>* * <value>HTTP/_h...@example.com <h...@example.com></value>* *</property>* *<property>* * <name>dfs.namenode.secondary.https-port</name>* * <value>50470</value>* *</property>* *<!-- JournalNode security config -->* *<property>* * <name>dfs.journalnode.keytab.file</name>* * <value>/etc/hadoop/conf/hdfs.keytab</value>* *</property>* *<property>* * <name>dfs.journalnode.kerberos.principa</name>* * <value>hadoop/_h...@example.com <h...@example.com></value>* *</property> * *<property>* * <name>dfs.journalnode.kerberos.internal.spnego.principa</name>* * <value>HTTP/_h...@example.com <h...@example.com></value>* *</property>* *<property>* * <name>dfs.web.authentication.kerberos.keytab</name>* * <value>/etc/hadoop/conf/hdfs.keytab</value>* *</property>* *<!-- DataNode security config -->* *<property>* * <name>dfs.datanode.kerberos.principal</name>* * <value>hadoop/_h...@example.com <h...@example.com></value>* *</property>* *<property>* * <name>dfs.datanode.keytab.file</name>* * <value>/etc/hadoop/conf/hdfs.keytab</value>* *</property>* *<property>* * <name>dfs.datanode.data.dir.perm</name>* * <value>700</value>* *</property>* *<!-- datanode SASL-->* *<property>* * <name>dfs.datanode.address</name>* * <value>0.0.0.0:61004 <http://0.0.0.0:61004></value>* *</property>* *<property>* * <name>dfs.datanode.http.address</name>* * <value>0.0.0.0:61006 <http://0.0.0.0:61006></value>* *</property>* *<property>* * <name>dfs.datanode.https.address</name>* * <value>0.0.0.0:50470 <http://0.0.0.0:50470></value>* *</property>* *<property>* * <name>dfs.data.transfer.protection</name>* * <value>integrity</value>* *</property>* *<property>* * <name>dfs.web.authentication.kerberos.principal</name>* * <value>HTTP/_h...@example.com <h...@example.com></value>* *</property>* *<property>* * <name>dfs.web.authentication.kerberos.keytab</name>* * <value>/etc/hadoop/conf/hdfs.keytab</value>* *</property>* *and [hadoop@dmp1 hadoop-2.7.3]$ klist -ket /etc/hadoop/conf/hdfs.keytab* *Keytab name: FILE:/etc/hadoop/conf/hdfs.keytabKVNO Timestamp Principal---- ------------------- ------------------------------------------------------ 2 09/19/2016 16:00:41 hdfs/dmp1.example....@example.com <dmp1.example....@example.com> (aes256-cts-hmac-sha1-96) 2 09/19/2016 16:00:41 hdfs/dmp1.example....@example.com <dmp1.example....@example.com> (aes128-cts-hmac-sha1-96) 2 09/19/2016 16:00:41 hdfs/dmp1.example....@example.com <dmp1.example....@example.com> (des3-cbc-sha1) 2 09/19/2016 16:00:41 hdfs/dmp1.example....@example.com <dmp1.example....@example.com> (arcfour-hmac) 2 09/19/2016 16:00:41 hdfs/dmp2.example....@example.com <dmp2.example....@example.com> (aes256-cts-hmac-sha1-96) 2 09/19/2016 16:00:41 hdfs/dmp2.example....@example.com <dmp2.example....@example.com> (aes128-cts-hmac-sha1-96) 2 09/19/2016 16:00:41 hdfs/dmp2.example....@example.com <dmp2.example....@example.com> (des3-cbc-sha1) 2 09/19/2016 16:00:41 hdfs/dmp2.example....@example.com <dmp2.example....@example.com> (arcfour-hmac) 2 09/19/2016 16:00:41 hdfs/dmp3.example....@example.com <dmp3.example....@example.com> (aes256-cts-hmac-sha1-96) 2 09/19/2016 16:00:41 hdfs/dmp3.example....@example.com <dmp3.example....@example.com> (aes128-cts-hmac-sha1-96) 2 09/19/2016 16:00:41 hdfs/dmp3.example....@example.com <dmp3.example....@example.com> (des3-cbc-sha1) 2 09/19/2016 16:00:41 hdfs/dmp3.example....@example.com <dmp3.example....@example.com> (arcfour-hmac) 2 09/19/2016 16:00:41 HTTP/dmp1.example....@example.com <dmp1.example....@example.com> (aes256-cts-hmac-sha1-96) 2 09/19/2016 16:00:41 HTTP/dmp1.example....@example.com <dmp1.example....@example.com> (aes128-cts-hmac-sha1-96) 2 09/19/2016 16:00:41 HTTP/dmp1.example....@example.com <dmp1.example....@example.com> (des3-cbc-sha1) 2 09/19/2016 16:00:41 HTTP/dmp1.example....@example.com <dmp1.example....@example.com> (arcfour-hmac) 2 09/19/2016 16:00:41 HTTP/dmp2.example....@example.com <dmp2.example....@example.com> (aes256-cts-hmac-sha1-96) 2 09/19/2016 16:00:41 HTTP/dmp2.example....@example.com <dmp2.example....@example.com> (aes128-cts-hmac-sha1-96) 2 09/19/2016 16:00:41 HTTP/dmp2.example....@example.com <dmp2.example....@example.com> (des3-cbc-sha1) 2 09/19/2016 16:00:41 HTTP/dmp2.example....@example.com <dmp2.example....@example.com> (arcfour-hmac) 2 09/19/2016 16:00:41 HTTP/dmp3.example....@example.com <dmp3.example....@example.com> (aes256-cts-hmac-sha1-96) 2 09/19/2016 16:00:41 HTTP/dmp3.example....@example.com <dmp3.example....@example.com> (aes128-cts-hmac-sha1-96) 2 09/19/2016 16:00:41 HTTP/dmp3.example....@example.com <dmp3.example....@example.com> (des3-cbc-sha1) 2 09/19/2016 16:00:41 HTTP/dmp3.example....@example.com <dmp3.example....@example.com> (arcfour-hmac) 2 09/19/2016 20:21:03 hadoop/dmp1.example....@example.com <dmp1.example....@example.com> (aes256-cts-hmac-sha1-96) 2 09/19/2016 20:21:03 hadoop/dmp1.example....@example.com <dmp1.example....@example.com> (aes128-cts-hmac-sha1-96) 2 09/19/2016 20:21:03 hadoop/dmp1.example....@example.com <dmp1.example....@example.com> (des3-cbc-sha1) 2 09/19/2016 20:21:03 hadoop/dmp1.example....@example.com <dmp1.example....@example.com> (arcfour-hmac) 2 09/19/2016 20:21:03 hadoop/dmp2.example....@example.com <dmp2.example....@example.com> (aes256-cts-hmac-sha1-96) 2 09/19/2016 20:21:03 hadoop/dmp2.example....@example.com <dmp2.example....@example.com> (aes128-cts-hmac-sha1-96) 2 09/19/2016 20:21:03 hadoop/dmp2.example....@example.com <dmp2.example....@example.com> (des3-cbc-sha1) 2 09/19/2016 20:21:03 hadoop/dmp2.example....@example.com <dmp2.example....@example.com> (arcfour-hmac) 2 09/19/2016 20:21:03 hadoop/dmp3.example....@example.com <dmp3.example....@example.com> (aes256-cts-hmac-sha1-96) 2 09/19/2016 20:21:03 hadoop/dmp3.example....@example.com <dmp3.example....@example.com> (aes128-cts-hmac-sha1-96) 2 09/19/2016 20:21:03 hadoop/dmp3.example....@example.com <dmp3.example....@example.com> (des3-cbc-sha1) 2 09/19/2016 20:21:03 hadoop/dmp3.example....@example.com <dmp3.example....@example.com> (arcfour-hmac) * 2016-09-20 15:52 GMT+08:00 Rakesh Radhakrishnan <rake...@apache.org>: > >>>>>>Caused by: javax.security.auth.login.LoginException: Unable to > obtain password from user > > Could you please check kerberos principal name is specified correctly in > "hdfs-site.xml", which is used to authenticate against Kerberos. > > If keytab file defined in "hdfs-site.xml" and doesn't exists or wrong > path, you will see > this error. So, please verify the path and the keytab filename correctly > configured. > > I hope hadoop discussion thread, https://goo.gl/M6l3vv may help you. > > > >>>>>>>2016-09-20 00:54:06,665 INFO org.apache.hadoop.http.HttpServer2: > HttpServer.start() threw a non Bind IOException > java.io.IOException: !JsseListener: java.lang.NullPointerException > > This is probably due to some missing configuration. > Could you please re-check the ssl-server.xml, keystore and truststore > properties: > > ssl.server.keystore.location > ssl.server.keystore.keypassword > ssl.client.truststore.location > ssl.client.truststore.password > > Rakesh > > On Tue, Sep 20, 2016 at 10:53 AM, kevin <kiss.kevin...@gmail.com> wrote: > >> *hi,all:* >> *My environment : Centos7.2 hadoop2.7.3 jdk1.8* >> *after I config hdfs with kerberos ,I can't start up with >> sbin/start-dfs.sh* >> >> *::namenode log as below * >> >> *STARTUP_MSG: build = Unknown -r Unknown; compiled by 'root' on >> 2016-09-18T09:05Z* >> *STARTUP_MSG: java = 1.8.0_102* >> *************************************************************/* >> *2016-09-20 00:54:05,822 INFO >> org.apache.hadoop.hdfs.server.namenode.NameNode: registered UNIX signal >> handlers for [TERM, HUP, INT]* >> *2016-09-20 00:54:05,825 INFO >> org.apache.hadoop.hdfs.server.namenode.NameNode: createNameNode []* >> *2016-09-20 00:54:06,078 INFO >> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from >> hadoop-metrics2.properties* >> *2016-09-20 00:54:06,149 INFO >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot >> period at 10 second(s).* >> *2016-09-20 00:54:06,149 INFO >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system >> started* >> *2016-09-20 00:54:06,151 INFO >> org.apache.hadoop.hdfs.server.namenode.NameNode: fs.defaultFS is >> hdfs://dmp1.example.com:9000 <http://dmp1.example.com:9000>* >> *2016-09-20 00:54:06,152 INFO >> org.apache.hadoop.hdfs.server.namenode.NameNode: Clients are to use >> dmp1.example.com:9000 <http://dmp1.example.com:9000> to access this >> namenode/service.* >> *2016-09-20 00:54:06,446 INFO >> org.apache.hadoop.security.UserGroupInformation: Login successful for user >> hadoop/dmp1.example....@example.com <dmp1.example....@example.com> using >> keytab file /etc/hadoop/conf/hdfs.keytab* >> *2016-09-20 00:54:06,472 INFO org.apache.hadoop.hdfs.DFSUtil: Starting >> web server as: HTTP/dmp1.example....@example.com >> <dmp1.example....@example.com>* >> *2016-09-20 00:54:06,475 INFO org.apache.hadoop.hdfs.DFSUtil: Starting >> Web-server for hdfs at: https://dmp1.example.com:50470 >> <https://dmp1.example.com:50470>* >> *2016-09-20 00:54:06,517 INFO org.mortbay.log: Logging to >> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via >> org.mortbay.log.Slf4jLog* >> *2016-09-20 00:54:06,533 INFO >> org.apache.hadoop.security.authentication.server.AuthenticationFilter: >> Unable to initialize FileSignerSecretProvider, falling back to use random >> secrets.* >> *2016-09-20 00:54:06,542 INFO org.apache.hadoop.http.HttpRequestLog: Http >> request log for http.requests.namenode is not defined* >> *2016-09-20 00:54:06,546 INFO org.apache.hadoop.http.HttpServer2: Added >> global filter 'safety' >> (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)* >> *2016-09-20 00:54:06,548 INFO org.apache.hadoop.http.HttpServer2: Added >> filter static_user_filter >> (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to >> context hdfs* >> *2016-09-20 00:54:06,548 INFO org.apache.hadoop.http.HttpServer2: Added >> filter static_user_filter >> (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to >> context static* >> *2016-09-20 00:54:06,548 INFO org.apache.hadoop.http.HttpServer2: Added >> filter static_user_filter >> (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to >> context logs* >> *2016-09-20 00:54:06,653 INFO org.apache.hadoop.http.HttpServer2: Added >> filter 'org.apache.hadoop.hdfs.web.Au >> <http://org.apache.hadoop.hdfs.web.Au>thFilter' >> (class=org.apache.hadoop.hdfs.web.AuthFilter)* >> *2016-09-20 00:54:06,654 INFO org.apache.hadoop.http.HttpServer2: >> addJerseyResourcePackage: >> packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, >> pathSpec=/webhdfs/v1/** >> *2016-09-20 00:54:06,657 INFO org.apache.hadoop.http.HttpServer2: Adding >> Kerberos (SPNEGO) filter to getDelegationToken* >> *2016-09-20 00:54:06,658 INFO org.apache.hadoop.http.HttpServer2: Adding >> Kerberos (SPNEGO) filter to renewDelegationToken* >> *2016-09-20 00:54:06,658 INFO org.apache.hadoop.http.HttpServer2: Adding >> Kerberos (SPNEGO) filter to cancelDelegationToken* >> *2016-09-20 00:54:06,659 INFO org.apache.hadoop.http.HttpServer2: Adding >> Kerberos (SPNEGO) filter to fsck* >> *2016-09-20 00:54:06,659 INFO org.apache.hadoop.http.HttpServer2: Adding >> Kerberos (SPNEGO) filter to imagetransfer* >> *2016-09-20 00:54:06,665 WARN org.mortbay.log: >> java.lang.NullPointerException* >> *2016-09-20 00:54:06,665 INFO org.apache.hadoop.http.HttpServer2: >> HttpServer.start() threw a non Bind IOException* >> *java.io.IOException: !JsseListener: java.lang.NullPointerException* >> * at >> org.mortbay.jetty.security.SslSocketConnector.newServerSocket(SslSocketConnector.java:516)* >> * at >> org.apache.hadoop.security.ssl.SslSocketConnectorSecure.newServerSocket(SslSocketConnectorSecure.java:47)* >> * at org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)* >> * at >> org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:914)* >> * at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:856)* >> * at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.st >> <http://namenode.NameNodeHttpServer.st>art(NameNodeHttpServer.java:142)* >> * at >> org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:753)* >> * at >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:639)* >> * at >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:812)* >> * at >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:796)* >> * at >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1493)* >> * at >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559)* >> >> >> *::second namenode log as below * >> >> >> >> *STARTUP_MSG: build = Unknown -r Unknown; compiled by 'root' on >> 2016-09-18T09:05ZSTARTUP_MSG: java = >> 1.8.0_102************************************************************/2016-09-20 >> 00:54:14,885 INFO org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: >> registered UNIX signal handlers for [TERM, HUP, INT]2016-09-20 00:54:15,263 >> FATAL org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Failed to >> start secondary namenodejava.io.IOException: Login failure for hadoop from >> keytab /etc/hadoop/conf/hdfs.keytab: >> javax.security.auth.login.LoginException: Unable to obtain password from >> user at >> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:963) >> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:246) at >> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:217) >> at >> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:192) >> at >> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:671)Caused >> by: javax.security.auth.login.LoginException: Unable to obtain password >> from user at >> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:897) >> at >> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760) >> at >> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) at >> javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) at >> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) at >> javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) at >> javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) at >> java.security.AccessController.doPrivileged(Native Method) at >> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) at >> javax.security.auth.login.LoginContext.login(LoginContext.java:587) at >> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:954) >> ... 4 more* >> > >
