[
https://issues.apache.org/jira/browse/HADOOP-4453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12641990#action_12641990
]
Chris Douglas commented on HADOOP-4453:
---------------------------------------
bq. SSL properties are system properties that affect all SSL connections that
the client make, not just those used in HsftpFileSystem.
This is a good point, but it also highlights the extent of this change. Since
the purpose of this JIRA is to improve the existing ssl support for
HsftpFileSystem, its side-effects shouldn't extend to all map/reduce tasks.
Ideally, it should also maintain distcp's FileSystem agnosticism.
The server code looks good. The client code in Child and DistCp can probably be
moved into HsftpFileSystem::initialize, which should also warn if it's called
more than once in the same JVM (assuming a second HsftpFileSystem handle can
invalidate the first; FileSystem cache hits should be benign). Unfortunately,
this creates a dilemma for distcp: its client and task resources may be at
different locations, but HsftpFileSystem will use the same property. This is
going to be true for many users of HsftpFileSystem.
Lacking a general need for similar, asymmetric configuration, I'd propose
adding a static config method to HsftpFileSystem:
{code}
public static void setSslConfigLocation(String loc, Configuration conf) {
conf.set("dfs.https.client.keystore.resource", loc);
}
{code}
And calling this before submitting the job from DistCp, if {{\-mapredSslLoc
<loc>}} is specified. If there is a general need, we can look into extending
GenericOptionsParser to support something more clever.
> Improve ssl handling for distcp
> -------------------------------
>
> Key: HADOOP-4453
> URL: https://issues.apache.org/jira/browse/HADOOP-4453
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.20.0
> Reporter: Chris Douglas
> Assignee: Kan Zhang
> Fix For: 0.20.0
>
> Attachments: 4453-0.patch, 4453-2.patch
>
>
> HsftpFileSystem is an ad hoc way to read from HDFS over ssl, targeting
> distcp. Its organization can be improved and its support of ssl features
> expanded.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
