[
https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541239#comment-16541239
]
Anirudh Ramanathan commented on SPARK-24432:
--------------------------------------------
Hi Mark, we did it once before in our fork in a certain way, but it seems like
some major changes are afoot in the general structure of dynamic allocation and
the external shuffle service (at the Spark level, and not just k8s). Given
that, we're holding off on this for 2.4. cc/ [~mcheah]
The weekly meetings we have on SIG Big Data might be a good place to engage if
you want to meet the community
(https://github.com/kubernetes/community/tree/master/sig-big-data).
We have a weekly Zoom meeting. All technical discussions and decisions happen
on the Apache mailing list.
> Add support for dynamic resource allocation
> -------------------------------------------
>
> Key: SPARK-24432
> URL: https://issues.apache.org/jira/browse/SPARK-24432
> Project: Spark
> Issue Type: New Feature
> Components: Kubernetes
> Affects Versions: 2.4.0
> Reporter: Yinan Li
> Priority: Major
>
> This is an umbrella ticket for work on adding support for dynamic resource
> allocation into the Kubernetes mode. This requires a Kubernetes-specific
> external shuffle service. The feature is available in our fork at
> github.com/apache-spark-on-k8s/spark.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
