[
https://issues.apache.org/jira/browse/SPARK-24137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460554#comment-16460554
]
Anirudh Ramanathan commented on SPARK-24137:
--------------------------------------------
SGTM. Let's try and share as much code as possible with
https://issues.apache.org/jira/browse/SPARK-23529.
> [K8s] Mount temporary directories in emptydir volumes
> -----------------------------------------------------
>
> Key: SPARK-24137
> URL: https://issues.apache.org/jira/browse/SPARK-24137
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 2.3.0
> Reporter: Matt Cheah
> Priority: Major
>
> Currently the Spark local directories do not get any volumes and volume
> mounts, which means we're writing Spark shuffle and cache contents to the
> file system mounted by Docker. This can be terribly inefficient. We should
> use emptydir volumes for these directories instead for significant
> performance improvements.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
