[ https://issues.apache.org/jira/browse/HDFS-15860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17640583#comment-17640583 ]
Zbigniew Kostrzewa commented on HDFS-15860: ------------------------------------------- [~elliot007] I've found a workaround/solution for my case by making my custom authentication handler inherit from {{AltKerberosAuthenticationHandler}} which effectively enabled SPNEGO for image transfer between NNs and kept my authentication scheme for Hadoop UIs. (details [here|https://hadoop.apache.org/docs/stable/hadoop-auth/Configuration.html] in section "AltKerberos Configuration") [~prabhujoseph] [~eyang] I found this was changed in https://issues.apache.org/jira/browse/HADOOP-16314 and though I understand the rationale I could not find a clear information what should developers of custom authentication handlers do in order to not hit this problem. Do you have any recommendations? > Standby Namenode bootstrap fails due to custom authentication handler being > run for /imagetransfer endpoint > ----------------------------------------------------------------------------------------------------------- > > Key: HDFS-15860 > URL: https://issues.apache.org/jira/browse/HDFS-15860 > Project: Hadoop HDFS > Issue Type: Bug > Components: ha > Affects Versions: 3.3.0 > Environment: CentOS Linux release 7.7.1908 (Core) > Reporter: Zbigniew Kostrzewa > Priority: Major > > I have extended Hadoop (YARN, HDFS, ...) web UIs with basic authentication by > implementing a custom authentication handler class and configuring it via > `hadoop.http.authentication.type` in core-site.xml. > Everything was working fine with Hadoop 2.x and 3.2.2. However, when I tried > switching to 3.3.0 standby Namenode failed during bootstrap when trying to > get fsimage: > {noformat} > 2021-02-25 21:18:52,356 ERROR [main] > org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. > java.io.IOException: java.lang.RuntimeException: > org.apache.hadoop.hdfs.server.common.HttpGetFailedException: Image transfer > servlet at http://XXXXX/imagetransfer?ge > timage=1&txid=0&storageInfo=-65:271209174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true > failed with status code 401 > Response message: > Authentication required > at > org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:549) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1728) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1821) > Caused by: java.lang.RuntimeException: > org.apache.hadoop.hdfs.server.common.HttpGetFailedException: Image transfer > servlet at http://XXXXX/imagetransfer?getimage=1&t > xid=0&storageInfo=-65:271209174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true > failed with status code 401 > Response message: > Authentication required > at > org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby$1.run(BootstrapStandby.java:127) > at > org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby$1.run(BootstrapStandby.java:121) > at > org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:485) > at > org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:121) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90) > at > org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:544) > ... 2 more > Caused by: org.apache.hadoop.hdfs.server.common.HttpGetFailedException: Image > transfer servlet at > http://XXXXXX/imagetransfer?getimage=1&txid=0&storageInfo=-65:271209 > 174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true > failed with status code 401 > Response message: > Authentication required > at org.apache.hadoop.hdfs.server.common.Util.doGetUrl(Util.java:168) > at > org.apache.hadoop.hdfs.server.namenode.TransferFsImage.doGetUrl(TransferFsImage.java:441) > at > org.apache.hadoop.hdfs.server.namenode.TransferFsImage.getFileClient(TransferFsImage.java:436) > at > org.apache.hadoop.hdfs.server.namenode.TransferFsImage.downloadImageToStorage(TransferFsImage.java:123) > at > org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.downloadImage(BootstrapStandby.java:357) > at > org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.doRun(BootstrapStandby.java:239) > at > org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.access$000(BootstrapStandby.java:82) > at > org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby$1.run(BootstrapStandby.java:125) > ... 8 more > {noformat} > It seems that in 3.3.0 my custom handler was being invoked for > `/imagetransfer` endpoint, from active NN: > {noformat} > 2021-02-25 21:18:52,328 DEBUG [qtp2033968586-22] > org.apache.hadoop.security.authentication.server.AuthenticationFilter: > Request > [http://XXXXX/imagetransfer?getimage=1&txid=0&storageInfo=-65:271209174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true] > triggering authentication. handler: class > hadoop_auth_handler.BasicAuthenticationHandler > {noformat} > which was not the case for 3.2.2 and earlier. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org