[
https://issues.apache.org/jira/browse/HDFS-9525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107148#comment-15107148
]
Daryn Sharp commented on HDFS-9525:
-----------------------------------
I think I understand better what you are trying to do, and I think you might be
able to accomplish your goals without much if any code change. I think the
main source of frustration is trying to access a secure cluster with security
disabled?
If you are trying to access any secure cluster: enable security in the config.
If you will also access an insecure cluster: also set
ipc.client.fallback-to-simple-auth-allowed=true. Now you should be able to
access any mixture of (in)secure clusters using hdfs or webhdfs.
There's also an existing config "mapreduce.job.credentials.binary" that can be
used to read in a token cache.
Aside: If using webhdfs for both source and target, I'd advise against it.
Webhdfs generates a much higher load on a cluster and is much less
fault-tolerant than normal hdfs. Our rule of thumb is always pull data (run
distcp on the target), read source with webhdfs (but only when RPC is acl-ed
off), always write to local target with hdfs.
If a code change is necessary, UGI should use
{{Configuration#getTrimmedStrings}} and unconditionally call
{{Credentials.readTokenStorageFile}} instead of allowing the user to specify an
invalid setting. Only webhdfs related change is
{{WebHdfsFileSystem.canRefreshDelegationToken}} should default to true.
> hadoop utilities need to support provided delegation tokens
> -----------------------------------------------------------
>
> Key: HDFS-9525
> URL: https://issues.apache.org/jira/browse/HDFS-9525
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: security
> Affects Versions: 3.0.0
> Reporter: Allen Wittenauer
> Assignee: HeeSoo Kim
> Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HDFS-7984.001.patch, HDFS-7984.002.patch,
> HDFS-7984.003.patch, HDFS-7984.004.patch, HDFS-7984.005.patch,
> HDFS-7984.006.patch, HDFS-7984.007.patch, HDFS-7984.patch,
> HDFS-9525.008.patch, HDFS-9525.branch-2.008.patch
>
>
> When using the webhdfs:// filesystem (especially from distcp), we need the
> ability to inject a delegation token rather than webhdfs initialize its own.
> This would allow for cross-authentication-zone file system accesses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
