[
https://issues.apache.org/jira/browse/SPARK-16817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401432#comment-15401432
]
Saisai Shao commented on SPARK-16817:
-------------------------------------
What's difference compared to use ramdisk to store shuffle data? If you want to
store the shuffle data on memory, ramdisk is the simplest way to achieve.
Also from my understanding, Alluxio may not be faster than ramdisk because of
several unnecessary distributed communication overhead.
> Enable storing of shuffle data in Alluxio
> -----------------------------------------
>
> Key: SPARK-16817
> URL: https://issues.apache.org/jira/browse/SPARK-16817
> Project: Spark
> Issue Type: New Feature
> Reporter: Tim Bisson
>
> If one is using Alluxio for storage, it would also be useful if Spark can
> store shuffle spill data in Alluxio. For example:
> spark.local.dir="alluxio://host:port/path"
> Several users on the Alluxio mailing list have asked for this feature:
> https://groups.google.com/forum/?fromgroups#!searchin/alluxio-users/shuffle$20spark|sort:relevance/alluxio-users/90pRZWRVi0s/mgLWLS5aAgAJ
> https://groups.google.com/forum/?fromgroups#!searchin/alluxio-users/shuffle$20spark|sort:relevance/alluxio-users/s9H93PnDebw/v_1_FMjR7vEJ
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
