+1 (non-binding)
We are evaluating Kubernetes for a variety of data processing workloads.
Spark is the natural choice for some of these workloads. Native Spark on
Kubernetes is of interest to us as it brings in dynamic allocation, resource
isolation and improved notions of security.
--
View
The problem with maintaining this scheduler separately right now is that the
scheduler backend is dependent upon the CoarseGrainedSchedulerBackend class,
which is not as much a stable API as it is an internal class with components
that currently need to be shared by all of the scheduler
+1 (non-binding)
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/SPIP-Spark-on-Kubernetes-tp22147p22195.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
There are a fair number of people (myself included) who have interest in
making scheduler back-ends fully pluggable. That will represent a
significant impact to core spark architecture, with corresponding risk.
Adding the kubernetes back-end in a manner similar to the other three
back-ends has
