[jira] [Created] (HDFS-3308) hftp/webhdfs can't get tokens if authority has no port
hftp/webhdfs can't get tokens if authority has no port -- Key: HDFS-3308 URL: https://issues.apache.org/jira/browse/HDFS-3308 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client Affects Versions: 0.23.0, 0.24.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Critical Token acquisition fails if a hftp or webhdfs filesystem is obtained with no port in the authority. Building a token service requires a port, and the renewer needs the port. The default port is not being used when there is no port in the uri. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3289) Commonize token selectors with fallback behavior
Commonize token selectors with fallback behavior
Key: HDFS-3289
URL: https://issues.apache.org/jira/browse/HDFS-3289
Project: Hadoop HDFS
Issue Type: Improvement
Components: hdfs client, security
Affects Versions: 0.24.0
Reporter: Daryn Sharp
Priority: Minor
{{WebHdfsDelegationTokenSelector}} and {{HftpDelegationTokenSelector}} are
essentially the same. They could be refactored to a new class in
org.apache.hadoop.hdfs.security.token.delegation (or an inner class in
DelegationTokenSelector) and then pass TOKEN_KIND in the constructor. The
selector should look for the given kind, and if not found, fallback to looking
for the hdfs/rpc token.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3268) Hdfs mishandles token service incompatible with HA
Hdfs mishandles token service incompatible with HA
Key: HDFS-3268
URL: https://issues.apache.org/jira/browse/HDFS-3268
Project: Hadoop HDFS
Issue Type: Bug
Components: ha, hdfs client
Affects Versions: 0.24.0, 2.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
The {{Hdfs AbstractFileSystem}} is overwriting the token service set by the
{{DFSClient}}. The service is not necessarily the correct one since
{{DFSClient}} is responsible for the service. Most importantly, this improper
behavior is overwriting the HA logical service which indirectly renders
{{FileContext}} incompatible with HA.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3255) HA DFS returns wrong token service
HA DFS returns wrong token service
--
Key: HDFS-3255
URL: https://issues.apache.org/jira/browse/HDFS-3255
Project: Hadoop HDFS
Issue Type: Bug
Components: ha, hdfs client
Affects Versions: 2.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
{{fs.getCanonicalService()}} must be equal to
{{fs.getDelegationToken(renewer).getService()}}. When HA is enabled, the DFS
token's service is a logical uri, but {{dfs.getCanonicalService()}} is only
returning the hostname of the logical uri.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3180) Add socket timeouts to webhdfs
Add socket timeouts to webhdfs -- Key: HDFS-3180 URL: https://issues.apache.org/jira/browse/HDFS-3180 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client Affects Versions: 0.23.0, 0.24.0 Reporter: Daryn Sharp WebHDFS connections may indefinitely hang due to no timeouts on the connection. WebHDFS should be adapted in a similar fashion to HDFS-3166 for hftp. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3098) Update FsShell tests for quoted metachars
Update FsShell tests for quoted metachars - Key: HDFS-3098 URL: https://issues.apache.org/jira/browse/HDFS-3098 Project: Hadoop HDFS Issue Type: Test Components: test Affects Versions: 0.24.0, 0.23.2 Reporter: Daryn Sharp Assignee: Daryn Sharp Need to add tests to TestDFSShell for quoted metachars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3104) Add tests for mkdir -p
Add tests for mkdir -p -- Key: HDFS-3104 URL: https://issues.apache.org/jira/browse/HDFS-3104 Project: Hadoop HDFS Issue Type: Test Components: test Affects Versions: 0.24.0, 0.23.2 Reporter: Daryn Sharp Assignee: Daryn Sharp Add tests for HADOOP-8175. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2784) Update hftp and hdfs for host-based token support
Update hftp and hdfs for host-based token support - Key: HDFS-2784 URL: https://issues.apache.org/jira/browse/HDFS-2784 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs client, name-node, security Affects Versions: 0.24.0, 0.23.1 Reporter: Daryn Sharp Assignee: Kihwal Lee Need to port 205 token changes and update any new related code dealing with tokens in these filesystems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2785) Update webhdfs and httpfs for host-based token support
Update webhdfs and httpfs for host-based token support -- Key: HDFS-2785 URL: https://issues.apache.org/jira/browse/HDFS-2785 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node, security Affects Versions: 0.24.0, 0.23.1 Reporter: Daryn Sharp Assignee: Robert Joseph Evans Need to port 205 tokens into these filesystems. Will mainly involve ensuring code duplicated from hftp is updated accordingly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2786) Fix host-based token incompatibilities in DFSUtil
Fix host-based token incompatibilities in DFSUtil - Key: HDFS-2786 URL: https://issues.apache.org/jira/browse/HDFS-2786 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node, security Affects Versions: 0.24.0, 0.23.1 Reporter: Daryn Sharp DFSUtil introduces new static methods that duplicate functionality in NetUtils. These new methods lack the logic necessary for host-based tokens to work. After speaking with Suresh, the approach being taken is: * DFSUtil.getSocketAddress will be removed. Callers will be reverted to using the NetUtils version. * DFSUtil.getDFSClient will changed to take accept a uri/host:port string instead of an InetSocketAddress. The method will internal call NetUtils.createSocketAddr. This alleviates the callers from being required to call NetUtils.createSocketAddr and reduce the opportunity for error that will break host-based tokens. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2652) Port token service changes from 205
Port token service changes from 205 --- Key: HDFS-2652 URL: https://issues.apache.org/jira/browse/HDFS-2652 Project: Hadoop HDFS Issue Type: New Feature Affects Versions: 0.24.0, 0.23.1 Reporter: Daryn Sharp Assignee: Daryn Sharp Need to merge the 205 token bug fixes and the feature to enable hostname-based tokens. See jiras linked to HADOOP-7808 for more details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2589) unnecessary hftp token fetch and renewal thread
unnecessary hftp token fetch and renewal thread
---
Key: HDFS-2589
URL: https://issues.apache.org/jira/browse/HDFS-2589
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 0.20.205.1
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Instantiation of the hftp filesystem is causing a token to be implicitly
created and added to a custom token renewal thread. With the new token renewal
feature in the JT, this causes the mapreduce {{obtainTokensForNamenodes}} to
fetch two tokens (an implicit and uncancelled token, and an explicit token) and
leave a spurious renewal thread running. This thread should not be running in
the JT.
After speaking with Owen, the quick solution is to lazy fetch the token, and to
lazy start the renewer thread.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2516) Tests for recursive copy/move commands
Tests for recursive copy/move commands -- Key: HDFS-2516 URL: https://issues.apache.org/jira/browse/HDFS-2516 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.20.205.0 Reporter: Daryn Sharp Add more tests for copy/move commands to ensure recursive behavior is correct and no NPEs occur. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2380) Security downgrade of token validation
Security downgrade of token validation
--
Key: HDFS-2380
URL: https://issues.apache.org/jira/browse/HDFS-2380
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 0.20.205.0, 0.23.0, 0.24.0
Reporter: Daryn Sharp
HADOOP-7119 introduced the {{KerberosAuthenticationHandler}} for web services.
It appears to have been merged into 205 to support webhdfs.
Prior to HADOOP-7119, the web service used by hftp/hsftp would validate tokens
using long kerberos user names. Now the realm is truncated from the user name
which caused hftp/hsftp to break. The {{JspHelper}} in the namenode rejected
the token validation due to the mismatched comparison between a now short user
(from the web service) and a long user (in the token). Subsequently, HDFS-2361
changed {{JspHelper}} to use the token's short user when comparing against the
now short web user.
The security ramification is it now appears to be easier to spoof other users
and access their files. Based on commentary in HDFS-2361, the case can be made
that other parts of hadoop are insecure with respect to user names, so it
doesn't matter that security has been further downgraded. I don't have know
knowledge to know if this true, or whether higher layers effectively guard
against lower level insecurities. In any case, this logic makes me uneasy,
especially when it comes to changing the security of a front door to hadoop.
Is there a technical reason why {{KerberosAuthenticationHandler}} should not be
changed (1-liner) to return the long user name? This would allow HDFS-2361 to
be reverted and return the former level of security to token validation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
