[jira] [Commented] (HDFS-15474) HttpFS: WebHdfsFileSystem cannot renew an expired delegation token from HttpFS response
[ https://issues.apache.org/jira/browse/HDFS-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159955#comment-17159955 ] Kihwal Lee commented on HDFS-15474: --- I believe the token ops are not implemented by HttpFS. They are handled at the filter level, which is a common component. One possibility is to have HttpFS relay token ops to the namenode. That way, you don't need to have a shared secret or a zookeeper instance for token ops when there are multiple servers. > HttpFS: WebHdfsFileSystem cannot renew an expired delegation token from > HttpFS response > --- > > Key: HDFS-15474 > URL: https://issues.apache.org/jira/browse/HDFS-15474 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > > When clients use WebHdfsFileSystem for HttpFS, they cannot renew expired > delegation tokens with the following error. > {noformat} > org.apache.hadoop.ipc.RemoteException: > org.apache.hadoop.security.token.SecretManager$InvalidToken: token > (owner=..., renewer=..., realUser=..., issueDate=..., maxDate=..., > sequenceNumber=..., masterKeyId=...) is expired > at > org.apache.hadoop.hdfs.web.JsonUtilClient.toRemoteException(JsonUtilClient.java:89) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:509) > ... > {noformat} > When using WebHdfsFileSystem for NameNode, it succeeds. This is because the > response of HttpFS is different from its of NameNode. We should fix the > response of HttpFS. > This issue is reported by Masayuki Yatagawa. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15474) HttpFS: WebHdfsFileSystem cannot renew an expired delegation token from HttpFS response
[ https://issues.apache.org/jira/browse/HDFS-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160890#comment-17160890 ] Takanobu Asanuma commented on HDFS-15474: - It seems an interesting idea. Actually, our cluster has multiple HttpFS servers, and they share delegation tokens using a ZooKeeper cluster. > HttpFS: WebHdfsFileSystem cannot renew an expired delegation token from > HttpFS response > --- > > Key: HDFS-15474 > URL: https://issues.apache.org/jira/browse/HDFS-15474 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > > When clients use WebHdfsFileSystem for HttpFS, they cannot renew expired > delegation tokens with the following error. > {noformat} > org.apache.hadoop.ipc.RemoteException: > org.apache.hadoop.security.token.SecretManager$InvalidToken: token > (owner=..., renewer=..., realUser=..., issueDate=..., maxDate=..., > sequenceNumber=..., masterKeyId=...) is expired > at > org.apache.hadoop.hdfs.web.JsonUtilClient.toRemoteException(JsonUtilClient.java:89) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:509) > ... > {noformat} > When using WebHdfsFileSystem for NameNode, it succeeds. This is because the > response of HttpFS is different from its of NameNode. We should fix the > response of HttpFS. > This issue is reported by Masayuki Yatagawa. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15474) HttpFS: WebHdfsFileSystem cannot renew an expired delegation token from HttpFS response
[ https://issues.apache.org/jira/browse/HDFS-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423065#comment-17423065 ] Wei-Chiu Chuang commented on HDFS-15474: Is this fixed by HDFS-15545? > HttpFS: WebHdfsFileSystem cannot renew an expired delegation token from > HttpFS response > --- > > Key: HDFS-15474 > URL: https://issues.apache.org/jira/browse/HDFS-15474 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > > When clients use WebHdfsFileSystem for HttpFS, they cannot renew expired > delegation tokens with the following error. > {noformat} > org.apache.hadoop.ipc.RemoteException: > org.apache.hadoop.security.token.SecretManager$InvalidToken: token > (owner=..., renewer=..., realUser=..., issueDate=..., maxDate=..., > sequenceNumber=..., masterKeyId=...) is expired > at > org.apache.hadoop.hdfs.web.JsonUtilClient.toRemoteException(JsonUtilClient.java:89) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:509) > ... > {noformat} > When using WebHdfsFileSystem for NameNode, it succeeds. This is because the > response of HttpFS is different from its of NameNode. We should fix the > response of HttpFS. > This issue is reported by Masayuki Yatagawa. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15474) HttpFS: WebHdfsFileSystem cannot renew an expired delegation token from HttpFS response
[ https://issues.apache.org/jira/browse/HDFS-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423194#comment-17423194 ] Takanobu Asanuma commented on HDFS-15474: - [~weichiu] Thanks for sharing the JIRA. But it seems to be a different problem becuase The problem of this JIRA occurs before calling {{WebHdfsFileSystem#replaceExpiredDelegationToken()}}. > HttpFS: WebHdfsFileSystem cannot renew an expired delegation token from > HttpFS response > --- > > Key: HDFS-15474 > URL: https://issues.apache.org/jira/browse/HDFS-15474 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > > When clients use WebHdfsFileSystem for HttpFS, they cannot renew expired > delegation tokens with the following error. > {noformat} > org.apache.hadoop.ipc.RemoteException: > org.apache.hadoop.security.token.SecretManager$InvalidToken: token > (owner=..., renewer=..., realUser=..., issueDate=..., maxDate=..., > sequenceNumber=..., masterKeyId=...) is expired > at > org.apache.hadoop.hdfs.web.JsonUtilClient.toRemoteException(JsonUtilClient.java:89) > at > org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:509) > ... > {noformat} > When using WebHdfsFileSystem for NameNode, it succeeds. This is because the > response of HttpFS is different from its of NameNode. We should fix the > response of HttpFS. > This issue is reported by Masayuki Yatagawa. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org