[
https://issues.apache.org/jira/browse/HADOOP-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12696585#action_12696585
]
Craig Macdonald commented on HADOOP-5635:
-----------------------------------------
Wouldn't it be generally better if Hadoop was configured with a list of shared
file systems. Then, when the administrator permitted, users could use shared
NFS filesystems as sources and targets for map reduce jobs. E.g., in our setup,
/local/ and /users/ are shared to all nodes. If we wanted to do a quick map
reduce test on stuff storied in /local/ we would have to copy to the DFS, when
it would be OK to run as is.
{noformat}
<name>fs.shared.filesystems</name>
<value>hdfs://,file://users/,file://local/ </value>
{noformat}
> distributed cache doesn't work with other distributed file systems
> ------------------------------------------------------------------
>
> Key: HADOOP-5635
> URL: https://issues.apache.org/jira/browse/HADOOP-5635
> Project: Hadoop Core
> Issue Type: Bug
> Components: filecache
> Reporter: Andrew Hitchcock
> Priority: Minor
> Attachments: fix-distributed-cache.patch
>
>
> Currently the DistributedCache does a check to see if the file to be included
> is an HDFS URI. If the URI isn't in HDFS, it returns the default filesystem.
> This prevents using other distributed file systems -- such as s3, s3n, or kfs
> -- with distributed cache. When a user tries to use one of those
> filesystems, it reports an error that it can't find the path in HDFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
