Re: Volcano in spark distro

2023-08-23 Thread Santosh Pingale
uniKorn doesn't force us to add a new dependency at all while Volcano did. > > > It would useful to support volcano in spark distro itself just like > yunikorn. > In these day, I prefer and invest more Apache YuniKorn and, for Apache > Spark 4, we are going to add and focus more on K

Re: Volcano in spark distro

2023-08-22 Thread Dongjoon Hyun
dependency at all while Volcano did. > It would useful to support volcano in spark distro itself just like yunikorn. In these day, I prefer and invest more Apache YuniKorn and, for Apache Spark 4, we are going to add and focus more on K8s native features like `Scheduling Gate` additionally. Dongjoon.

Re: Volcano in spark distro

2023-08-22 Thread Yikun Jiang
22, 2023 at 8:14 PM Santosh Pingale wrote: > Hey all > > It would useful to support volcano in spark distro itself just like > yunikorn. So I am wondering what is the reason behind this decision of not > packaging it already. > Running Spark on Kubernetes - Spark 3.4.1 Docu

Re: Volcano in spark distro

2023-08-22 Thread Mich Talebzadeh
sh Pingale wrote: > Hey all > > It would useful to support volcano in spark distro itself just like > yunikorn. So I am wondering what is the reason behind this decision of not > packaging it already. > Running Spark on Kubernetes - Spark 3.4.1 Documentation > <https://sp

Volcano in spark distro

2023-08-22 Thread Santosh Pingale
Hey all It would useful to support volcano in spark distro itself just like yunikorn. So I am wondering what is the reason behind this decision of not packaging it already. https://spark.apache.org/docs/latest/running-on-kubernetes.html#using-volcano-as-customized-scheduler-for-spark