[jira] [Commented] (HDFS-3904) QJM: journalnode does not die/log ERROR when keytab is not found in secure mode
[ https://issues.apache.org/jira/browse/HDFS-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452609#comment-13452609 ] Todd Lipcon commented on HDFS-3904: --- Filed HADOOP-8786 with a patch for this. Will resolve this as dup since it is not specific to this branch. > QJM: journalnode does not die/log ERROR when keytab is not found in secure > mode > --- > > Key: HDFS-3904 > URL: https://issues.apache.org/jira/browse/HDFS-3904 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: QuorumJournalManager (HDFS-3077) >Reporter: Eli Collins >Priority: Minor > > Credit to Stephen Chu for finding this. The journalnode was incorrectly > configured (misplaced the keytab) with security enabled, when started the > JournalNode didn't die. It stayed running and logged a WARN message: > {noformat} > 2012-08-23 15:44:15,497 WARN org.mortbay.log: Failed startup of context > org.mortbay.jetty.webapp.WebAppContext@58c16b18{/,file:/usr/lib/hadoop-hdfs/webapps/journal} > javax.servlet.ServletException: javax.servlet.ServletException: Keytab does > not exist: /etc/hadoop/conf/hdfs.keytab > at > org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:185) > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:146) > at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) > at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at > org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713) > at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) > at > org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282) > at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518) > at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499) > at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at > org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) > at > org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156) > at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) > at org.mortbay.jetty.Server.doStart(Server.java:224) > at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at org.apache.hadoop.http.HttpServer.start(HttpServer.java:657) > at > org.apache.hadoop.hdfs.qjournal.server.JournalNodeHttpServer.start(JournalNodeHttpServer.java:83) > at > org.apache.hadoop.hdfs.qjournal.server.JournalNode.start(JournalNode.java:138) > at > org.apache.hadoop.hdfs.qjournal.server.JournalNode.run(JournalNode.java:120) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.hadoop.hdfs.qjournal.server.JournalNode.main(JournalNode.java:228) > Caused by: javax.servlet.ServletException: Keytab does not exist: > /etc/hadoop/conf/hdfs.keytab > at > org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:153) > ... 22 more > {noformat} > The other HDFS daemons, if I remember correctly, would die if they can't > authenticate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3904) QJM: journalnode does not die/log ERROR when keytab is not found in secure mode
[ https://issues.apache.org/jira/browse/HDFS-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450992#comment-13450992 ] Aaron T. Myers commented on HDFS-3904: -- I don't think this is true anymore, post HDFS-3893, at least when the keytab isn't set. This was previously a problem since the JN wasn't logging in as the actual JN principal, but rather only as the SPNEGO principal. Has this been tried on a recent build? > QJM: journalnode does not die/log ERROR when keytab is not found in secure > mode > --- > > Key: HDFS-3904 > URL: https://issues.apache.org/jira/browse/HDFS-3904 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: QuorumJournalManager (HDFS-3077) >Reporter: Eli Collins >Priority: Minor > > Credit to Stephen Chu for finding this. The journalnode was incorrectly > configured (misplaced the keytab) with security enabled, when started the > JournalNode didn't die. It stayed running and logged a WARN message: > {noformat} > 2012-08-23 15:44:15,497 WARN org.mortbay.log: Failed startup of context > org.mortbay.jetty.webapp.WebAppContext@58c16b18{/,file:/usr/lib/hadoop-hdfs/webapps/journal} > javax.servlet.ServletException: javax.servlet.ServletException: Keytab does > not exist: /etc/hadoop/conf/hdfs.keytab > at > org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:185) > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:146) > at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) > at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at > org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713) > at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) > at > org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282) > at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518) > at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499) > at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at > org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) > at > org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156) > at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) > at org.mortbay.jetty.Server.doStart(Server.java:224) > at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at org.apache.hadoop.http.HttpServer.start(HttpServer.java:657) > at > org.apache.hadoop.hdfs.qjournal.server.JournalNodeHttpServer.start(JournalNodeHttpServer.java:83) > at > org.apache.hadoop.hdfs.qjournal.server.JournalNode.start(JournalNode.java:138) > at > org.apache.hadoop.hdfs.qjournal.server.JournalNode.run(JournalNode.java:120) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.hadoop.hdfs.qjournal.server.JournalNode.main(JournalNode.java:228) > Caused by: javax.servlet.ServletException: Keytab does not exist: > /etc/hadoop/conf/hdfs.keytab > at > org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:153) > ... 22 more > {noformat} > The other HDFS daemons, if I remember correctly, would die if they can't > authenticate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3904) QJM: journalnode does not die/log ERROR when keytab is not found in secure mode
[ https://issues.apache.org/jira/browse/HDFS-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450958#comment-13450958 ] Todd Lipcon commented on HDFS-3904: --- I don't think this is unique to QJM. It looks like the exception is caught and swallowed inside jetty. See http://stackoverflow.com/questions/4329341/webapp-startup-fails-but-jetty-lifecycle-claims-started We could add a workaround for all HttpServers by calling {{getUnavailableException}} I believe. > QJM: journalnode does not die/log ERROR when keytab is not found in secure > mode > --- > > Key: HDFS-3904 > URL: https://issues.apache.org/jira/browse/HDFS-3904 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: QuorumJournalManager (HDFS-3077) >Reporter: Eli Collins >Priority: Minor > > Credit to Stephen Chu for finding this. The journalnode was incorrectly > configured (misplaced the keytab) with security enabled, when started the > JournalNode didn't die. It stayed running and logged a WARN message: > {noformat} > 2012-08-23 15:44:15,497 WARN org.mortbay.log: Failed startup of context > org.mortbay.jetty.webapp.WebAppContext@58c16b18{/,file:/usr/lib/hadoop-hdfs/webapps/journal} > javax.servlet.ServletException: javax.servlet.ServletException: Keytab does > not exist: /etc/hadoop/conf/hdfs.keytab > at > org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:185) > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:146) > at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) > at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at > org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713) > at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) > at > org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282) > at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518) > at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499) > at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at > org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) > at > org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156) > at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) > at org.mortbay.jetty.Server.doStart(Server.java:224) > at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at org.apache.hadoop.http.HttpServer.start(HttpServer.java:657) > at > org.apache.hadoop.hdfs.qjournal.server.JournalNodeHttpServer.start(JournalNodeHttpServer.java:83) > at > org.apache.hadoop.hdfs.qjournal.server.JournalNode.start(JournalNode.java:138) > at > org.apache.hadoop.hdfs.qjournal.server.JournalNode.run(JournalNode.java:120) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.hadoop.hdfs.qjournal.server.JournalNode.main(JournalNode.java:228) > Caused by: javax.servlet.ServletException: Keytab does not exist: > /etc/hadoop/conf/hdfs.keytab > at > org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:153) > ... 22 more > {noformat} > The other HDFS daemons, if I remember correctly, would die if they can't > authenticate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira