[
https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14259323#comment-14259323
]
Sean Owen commented on SPARK-1529:
----------------------------------
This may be a dumb question on an old issue, but: you can already access local
filesystems through HDFS's FileSystem. Of course you can always access local
filesystems directly in general.
The FileSystem abstraction won't provide everything that java.nio does because
it is not a local filesystem.
But why would you want to put shuffle and temp files in HDFS? The MapR comment
confuses me more since its main trick is exposing HDFS as more like an NFS
mount point. But if anything that makes it already usable as is for this
purpose.
> Support setting spark.local.dirs to a hadoop FileSystem
> --------------------------------------------------------
>
> Key: SPARK-1529
> URL: https://issues.apache.org/jira/browse/SPARK-1529
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: Patrick Wendell
> Assignee: Cheng Lian
>
> In some environments, like with MapR, local volumes are accessed through the
> Hadoop filesystem interface. We should allow setting spark.local.dir to a
> Hadoop filesystem location.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
