[
https://issues.apache.org/jira/browse/MESOS-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006053#comment-14006053
]
Tom Arnfeld commented on MESOS-1405:
------------------------------------
> I like the hdfs client solution. That seems like a trivial addition to the
> fetcher.
Yeah, it should simply be an addition to the if statement to check whether it's
hdfs://. I'll take this one, then.
> Mesos fetcher does not support S3(n)
> ------------------------------------
>
> Key: MESOS-1405
> URL: https://issues.apache.org/jira/browse/MESOS-1405
> Project: Mesos
> Issue Type: Improvement
> Affects Versions: 0.18.2
> Reporter: Tom Arnfeld
> Priority: Minor
>
> The HDFS client is able to support both S3 and S3N. Details for the
> difference between the two can be found here:
> http://wiki.apache.org/hadoop/AmazonS3.
> Examples:
> s3://bucket/path.tar.gz <- S3 Block Store
> s3n://bucket/path.tar.gz <- S3 K/V Store
> Either we can simply pass these URIs through to the HDFS client (hdfs.cpp)
> and let hadoop do the work, or we can integrate with S3 directly. The latter
> then requires we have a way of managing S3 credentials, whereas using the
> HDFS client will just pull credentials from HADOOP_HOME.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
