[ https://issues.apache.org/jira/browse/HDFS-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bangtao Zhou updated HDFS-6436: ------------------------------- Attachment: HDFS-6436.patch > WebHdfsFileSystem execute get, renew and cancel delegationtoken operation > should use spnego to authenticate > ----------------------------------------------------------------------------------------------------------- > > Key: HDFS-6436 > URL: https://issues.apache.org/jira/browse/HDFS-6436 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs > Affects Versions: 3.0.0, 2.4.0 > Environment: Kerberos > Reporter: Bangtao Zhou > Fix For: 3.0.0 > > Attachments: HDFS-6436.patch > > > while in kerberos secure mode, when using WebHdfsFileSystem to access HDFS, > it allways get an > *org.apache.hadoop.security.authentication.client.AuthenticationException: > Unauthorized*, for example, when call WebHdfsFileSystem.listStatus it will > execute a LISTSTATUS Op, and this Op should authenticate via *delegation > token*, so it will execute a GETDELEGATIONTOKEN Op to get a delegation > token(actually GETDELEGATIONTOKEN authenticates via *SPNEGO*), but it still > use delegation token to authenticate, so it allways get an Unauthorized > Exception. > Exception is like this: > {code:java} > 19:05:11.758 [main] DEBUG o.a.h.hdfs.web.URLConnectionFactory - open URL > connection > java.io.IOException: > org.apache.hadoop.security.authentication.client.AuthenticationException: > Unauthorized > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:287) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:82) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:538) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:406) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:434) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:430) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:1058) > 19:05:11.766 [main] DEBUG o.a.h.security.UserGroupInformation - > PrivilegedActionException as:bang...@cyhadoop.com (auth:KERBEROS) > cause:java.io.IOException: > org.apache.hadoop.security.authentication.client.AuthenticationException: > Unauthorized > at > org.apache.hadoop.hdfs.web.TokenAspect.ensureTokenInitialized(TokenAspect.java:134) > 19:05:11.767 [main] DEBUG o.a.h.security.UserGroupInformation - > PrivilegedActionException as:bang...@cyhadoop.com (auth:KERBEROS) > cause:java.io.IOException: > org.apache.hadoop.security.authentication.client.AuthenticationException: > Unauthorized > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:213) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getAuthParameters(WebHdfsFileSystem.java:371) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem.toUrl(WebHdfsFileSystem.java:392) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractFsPathRunner.getUrl(WebHdfsFileSystem.java:602) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:533) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:406) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:434) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:430) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem.listStatus(WebHdfsFileSystem.java:1037) > at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1483) > at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1523) > at org.apache.hadoop.fs.FileSystem$4.<init>(FileSystem.java:1679) > at > org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1678) > at > org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1661) > at org.apache.hadoop.fs.FileSystem$5.<init>(FileSystem.java:1723) > at org.apache.hadoop.fs.FileSystem.listFiles(FileSystem.java:1720) > at com.cyou.marketing.hop.filesystem.App$1.run(App.java:34) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:356) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) > at com.cyou.marketing.hop.filesystem.App.main(App.java:24) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120) > Caused by: > org.apache.hadoop.security.authentication.client.AuthenticationException: > Unauthorized > ... 40 more > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)