[jira] [Updated] (HDFS-4951) FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer
[
https://issues.apache.org/jira/browse/HDFS-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated HDFS-4951:
Attachment: HDFS-4951.patch
That's a good idea; it will prevent anyone from accidentally using it. The new
patch also removes the HttpFSKerberosAuthenticator.TOKEN_KIND.
FsShell commands using secure httpfs throw exceptions due to missing
TokenRenewer
-
Key: HDFS-4951
URL: https://issues.apache.org/jira/browse/HDFS-4951
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 3.0.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Attachments: HDFS-4951.patch, HDFS-4951.patch
It looks like there isn't a {{TokenRenewer}} for HttpFS delegation tokens
({{HTTPFS_DELEGATION_TOKENS}} tokens, so when it goes to cancel the token, it
throws an exception:
{noformat}
$ hadoop fs -ls webhdfs://host:14000
// File listing omitted
13/06/21 13:09:04 WARN token.Token: No TokenRenewer defined for token kind
HTTPFS_DELEGATION_TOKEN
13/06/21 13:09:04 WARN util.ShutdownHookManager: ShutdownHook
'ClientFinalizer' failed, java.lang.UnsupportedOperationException: Token
cancel is not supported for HTTPFS_DELEGATION_TOKEN tokens
java.lang.UnsupportedOperationException: Token cancel is not supported for
HTTPFS_DELEGATION_TOKEN tokens
at
org.apache.hadoop.security.token.Token$TrivialRenewer.cancel(Token.java:417)
at org.apache.hadoop.security.token.Token.cancel(Token.java:382)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.cancel(DelegationTokenRenewer.java:146)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.access$200(DelegationTokenRenewer.java:58)
at
org.apache.hadoop.fs.DelegationTokenRenewer.removeRenewAction(DelegationTokenRenewer.java:233)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.close(WebHdfsFileSystem.java:790)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2398)
at
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2414)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
{noformat}
WebHDFS doesn't have this problem because it has a {{TokenRenewer}} for its
delegation tokens ({{WEBHDFS delegation}} tokens).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4951) FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer
[
https://issues.apache.org/jira/browse/HDFS-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alejandro Abdelnur updated HDFS-4951:
-
Target Version/s: 2.1.0-beta
Affects Version/s: (was: 3.0.0)
2.1.0-beta
FsShell commands using secure httpfs throw exceptions due to missing
TokenRenewer
-
Key: HDFS-4951
URL: https://issues.apache.org/jira/browse/HDFS-4951
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 2.1.0-beta
Reporter: Robert Kanter
Assignee: Robert Kanter
Attachments: HDFS-4951.patch, HDFS-4951.patch
It looks like there isn't a {{TokenRenewer}} for HttpFS delegation tokens
({{HTTPFS_DELEGATION_TOKENS}} tokens, so when it goes to cancel the token, it
throws an exception:
{noformat}
$ hadoop fs -ls webhdfs://host:14000
// File listing omitted
13/06/21 13:09:04 WARN token.Token: No TokenRenewer defined for token kind
HTTPFS_DELEGATION_TOKEN
13/06/21 13:09:04 WARN util.ShutdownHookManager: ShutdownHook
'ClientFinalizer' failed, java.lang.UnsupportedOperationException: Token
cancel is not supported for HTTPFS_DELEGATION_TOKEN tokens
java.lang.UnsupportedOperationException: Token cancel is not supported for
HTTPFS_DELEGATION_TOKEN tokens
at
org.apache.hadoop.security.token.Token$TrivialRenewer.cancel(Token.java:417)
at org.apache.hadoop.security.token.Token.cancel(Token.java:382)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.cancel(DelegationTokenRenewer.java:146)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.access$200(DelegationTokenRenewer.java:58)
at
org.apache.hadoop.fs.DelegationTokenRenewer.removeRenewAction(DelegationTokenRenewer.java:233)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.close(WebHdfsFileSystem.java:790)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2398)
at
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2414)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
{noformat}
WebHDFS doesn't have this problem because it has a {{TokenRenewer}} for its
delegation tokens ({{WEBHDFS delegation}} tokens).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4951) FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer
[
https://issues.apache.org/jira/browse/HDFS-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alejandro Abdelnur updated HDFS-4951:
-
Resolution: Fixed
Fix Version/s: 2.1.0-beta
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Robert. Committed to trunk, branch-2, branch-2.1.
FsShell commands using secure httpfs throw exceptions due to missing
TokenRenewer
-
Key: HDFS-4951
URL: https://issues.apache.org/jira/browse/HDFS-4951
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 2.1.0-beta
Reporter: Robert Kanter
Assignee: Robert Kanter
Fix For: 2.1.0-beta
Attachments: HDFS-4951.patch, HDFS-4951.patch
It looks like there isn't a {{TokenRenewer}} for HttpFS delegation tokens
({{HTTPFS_DELEGATION_TOKENS}} tokens, so when it goes to cancel the token, it
throws an exception:
{noformat}
$ hadoop fs -ls webhdfs://host:14000
// File listing omitted
13/06/21 13:09:04 WARN token.Token: No TokenRenewer defined for token kind
HTTPFS_DELEGATION_TOKEN
13/06/21 13:09:04 WARN util.ShutdownHookManager: ShutdownHook
'ClientFinalizer' failed, java.lang.UnsupportedOperationException: Token
cancel is not supported for HTTPFS_DELEGATION_TOKEN tokens
java.lang.UnsupportedOperationException: Token cancel is not supported for
HTTPFS_DELEGATION_TOKEN tokens
at
org.apache.hadoop.security.token.Token$TrivialRenewer.cancel(Token.java:417)
at org.apache.hadoop.security.token.Token.cancel(Token.java:382)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.cancel(DelegationTokenRenewer.java:146)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.access$200(DelegationTokenRenewer.java:58)
at
org.apache.hadoop.fs.DelegationTokenRenewer.removeRenewAction(DelegationTokenRenewer.java:233)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.close(WebHdfsFileSystem.java:790)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2398)
at
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2414)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
{noformat}
WebHDFS doesn't have this problem because it has a {{TokenRenewer}} for its
delegation tokens ({{WEBHDFS delegation}} tokens).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4951) FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer
[
https://issues.apache.org/jira/browse/HDFS-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated HDFS-4951:
Attachment: HDFS-4951.patch
Instead of essentially redoing the delegation token renewer (and related) code
from WebHDFS for HttpFS, and because the server is using WebHDFSFileSystem
anyway, I think the best and simplest solution is to make HttpFS use the same
delegation token kind as WebHDFS is using.
The patch simply changes the token kind that HttpFS's DelegationTokenIdentifier
is using from HTTPFS_DELEGATION_TOKEN to WEBHDFS delegation. I manually
verified that using the FsShell with HttpFS now works properly.
FsShell commands using secure httpfs throw exceptions due to missing
TokenRenewer
-
Key: HDFS-4951
URL: https://issues.apache.org/jira/browse/HDFS-4951
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 3.0.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Attachments: HDFS-4951.patch
It looks like there isn't a {{TokenRenewer}} for HttpFS delegation tokens
({{HTTPFS_DELEGATION_TOKENS}} tokens, so when it goes to cancel the token, it
throws an exception:
{noformat}
$ hadoop fs -ls webhdfs://host:14000
// File listing omitted
13/06/21 13:09:04 WARN token.Token: No TokenRenewer defined for token kind
HTTPFS_DELEGATION_TOKEN
13/06/21 13:09:04 WARN util.ShutdownHookManager: ShutdownHook
'ClientFinalizer' failed, java.lang.UnsupportedOperationException: Token
cancel is not supported for HTTPFS_DELEGATION_TOKEN tokens
java.lang.UnsupportedOperationException: Token cancel is not supported for
HTTPFS_DELEGATION_TOKEN tokens
at
org.apache.hadoop.security.token.Token$TrivialRenewer.cancel(Token.java:417)
at org.apache.hadoop.security.token.Token.cancel(Token.java:382)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.cancel(DelegationTokenRenewer.java:146)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.access$200(DelegationTokenRenewer.java:58)
at
org.apache.hadoop.fs.DelegationTokenRenewer.removeRenewAction(DelegationTokenRenewer.java:233)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.close(WebHdfsFileSystem.java:790)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2398)
at
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2414)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
{noformat}
WebHDFS doesn't have this problem because it has a {{TokenRenewer}} for its
delegation tokens ({{WEBHDFS delegation}} tokens).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4951) FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer
[
https://issues.apache.org/jira/browse/HDFS-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated HDFS-4951:
Status: Patch Available (was: Open)
FsShell commands using secure httpfs throw exceptions due to missing
TokenRenewer
-
Key: HDFS-4951
URL: https://issues.apache.org/jira/browse/HDFS-4951
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 3.0.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Attachments: HDFS-4951.patch
It looks like there isn't a {{TokenRenewer}} for HttpFS delegation tokens
({{HTTPFS_DELEGATION_TOKENS}} tokens, so when it goes to cancel the token, it
throws an exception:
{noformat}
$ hadoop fs -ls webhdfs://host:14000
// File listing omitted
13/06/21 13:09:04 WARN token.Token: No TokenRenewer defined for token kind
HTTPFS_DELEGATION_TOKEN
13/06/21 13:09:04 WARN util.ShutdownHookManager: ShutdownHook
'ClientFinalizer' failed, java.lang.UnsupportedOperationException: Token
cancel is not supported for HTTPFS_DELEGATION_TOKEN tokens
java.lang.UnsupportedOperationException: Token cancel is not supported for
HTTPFS_DELEGATION_TOKEN tokens
at
org.apache.hadoop.security.token.Token$TrivialRenewer.cancel(Token.java:417)
at org.apache.hadoop.security.token.Token.cancel(Token.java:382)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.cancel(DelegationTokenRenewer.java:146)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.access$200(DelegationTokenRenewer.java:58)
at
org.apache.hadoop.fs.DelegationTokenRenewer.removeRenewAction(DelegationTokenRenewer.java:233)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.close(WebHdfsFileSystem.java:790)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2398)
at
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2414)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
{noformat}
WebHDFS doesn't have this problem because it has a {{TokenRenewer}} for its
delegation tokens ({{WEBHDFS delegation}} tokens).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4951) FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer
[
https://issues.apache.org/jira/browse/HDFS-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated HDFS-4951:
Description:
It looks like there isn't a {{TokenRenewer}} for HttpFS delegation tokens
({{HTTPFS_DELEGATION_TOKENS}} tokens, so when it goes to cancel the token, it
throws an exception:
{noformat}
$ hadoop fs -ls webhdfs://host:14000
// File listing omitted
13/06/21 13:09:04 WARN token.Token: No TokenRenewer defined for token kind
HTTPFS_DELEGATION_TOKEN
13/06/21 13:09:04 WARN util.ShutdownHookManager: ShutdownHook 'ClientFinalizer'
failed, java.lang.UnsupportedOperationException: Token cancel is not supported
for HTTPFS_DELEGATION_TOKEN tokens
java.lang.UnsupportedOperationException: Token cancel is not supported for
HTTPFS_DELEGATION_TOKEN tokens
at
org.apache.hadoop.security.token.Token$TrivialRenewer.cancel(Token.java:417)
at org.apache.hadoop.security.token.Token.cancel(Token.java:382)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.cancel(DelegationTokenRenewer.java:146)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.access$200(DelegationTokenRenewer.java:58)
at
org.apache.hadoop.fs.DelegationTokenRenewer.removeRenewAction(DelegationTokenRenewer.java:233)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.close(WebHdfsFileSystem.java:790)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2398)
at
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2414)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
{noformat}
WebHDFS doesn't have this problem because it has a {{TokenRenewer}} for its
delegation tokens ({{WEBHDFS delegation}} tokens).
was:
It looks like there isn't a {{TokenRenewer}} for HttpFS delegation tokens
({{HTTPFS_DELEGATION_TOKENS}} tokens, so when it goes to cancel the token, it
throws an exception:
{noformat}
$ hadoop fs -ls webhdfs://host:14000
// File listing omitted
13/06/21 13:09:04 WARN token.Token: No TokenRenewer defined for token kind
HTTPFS_DELEGATION_TOKEN
13/06/21 13:09:04 WARN util.ShutdownHookManager: ShutdownHook 'ClientFinalizer'
failed, java.lang.UnsupportedOperationException: Token cancel is not supported
for HTTPFS_DELEGATION_TOKEN tokens
java.lang.UnsupportedOperationException: Token cancel is not supported for
HTTPFS_DELEGATION_TOKEN tokens
at
org.apache.hadoop.security.token.Token$TrivialRenewer.cancel(Token.java:417)
at org.apache.hadoop.security.token.Token.cancel(Token.java:382)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.cancel(DelegationTokenRenewer.java:146)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.access$200(DelegationTokenRenewer.java:58)
at
org.apache.hadoop.fs.DelegationTokenRenewer.removeRenewAction(DelegationTokenRenewer.java:233)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.close(WebHdfsFileSystem.java:790)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2398)
at
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2414)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
{noformat}
WebHDFS doesn't have this problem because it has a {{TokenRenewer}}.
FsShell commands using secure httpfs throw exceptions due to missing
TokenRenewer
-
Key: HDFS-4951
URL: https://issues.apache.org/jira/browse/HDFS-4951
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 3.0.0
Reporter: Robert Kanter
Assignee: Robert Kanter
It looks like there isn't a {{TokenRenewer}} for HttpFS delegation tokens
({{HTTPFS_DELEGATION_TOKENS}} tokens, so when it goes to cancel the token, it
throws an exception:
{noformat}
$ hadoop fs -ls webhdfs://host:14000
// File listing omitted
13/06/21 13:09:04 WARN token.Token: No TokenRenewer defined for token kind
HTTPFS_DELEGATION_TOKEN
13/06/21 13:09:04 WARN util.ShutdownHookManager: ShutdownHook
'ClientFinalizer' failed, java.lang.UnsupportedOperationException: Token
cancel is not supported for HTTPFS_DELEGATION_TOKEN tokens
java.lang.UnsupportedOperationException: Token cancel is not supported for
HTTPFS_DELEGATION_TOKEN tokens
at
org.apache.hadoop.security.token.Token$TrivialRenewer.cancel(Token.java:417)
at org.apache.hadoop.security.token.Token.cancel(Token.java:382)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.cancel(DelegationTokenRenewer.java:146)
at
org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.access$200(DelegationTokenRenewer.java:58)
at
