[ https://issues.apache.org/jira/browse/SPARK-32642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17182160#comment-17182160 ]
Apache Spark commented on SPARK-32642: -------------------------------------- User 'holdenk' has created a pull request for this issue: https://github.com/apache/spark/pull/29511 > Add support for ESS in Spark sidecar > ------------------------------------ > > Key: SPARK-32642 > URL: https://issues.apache.org/jira/browse/SPARK-32642 > Project: Spark > Issue Type: Improvement > Components: Kubernetes > Affects Versions: 3.1.0 > Reporter: Holden Karau > Assignee: Holden Karau > Priority: Major > > For Spark on Kubernetes there are multiple ways to achieve dynamic scaling. > Unlike in YARN mode ESS on Spark on K8s as a sidecar would not help with > dynamic scaling. Instead this helps with the situations where the executor > JVM may be overloaded or crash. If we do this as a sidecar we can share the > same mounts between the executor and the external shuffle storage. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org