RE: Spark Kubernetes Operator

2023-04-26 Thread Aldo Culquicondor
We are welcoming contributors, as announced in the Kubernetes WG Batch https://docs.google.com/document/d/1XOeUN-K0aKmJJNq7H07r74n-mGgSFyiEDQ3ecwsGhec/edit#bookmark=id.gfgjt0nmbgjl If you are interested, you can find us in slack.k8s.io #wg-batch or ping @mwielgus on github/slack. Thanks On 2023/

Re: Spark Kubernetes Operator

2023-04-14 Thread Yuval Itzchakov
I'm not running on GKE. I am wondering what's the long term strategy around a Spark operator. Operators are the de-facto way to run complex deployments. The Flink community now has an official community led operator, and I was wondering if there are any similar plans for Spark. On Fri, Apr 14, 202

Re: Spark Kubernetes Operator

2023-04-14 Thread Mich Talebzadeh
Hi, What exactly are you trying to achieve? Spark on GKE works fine and you can run Datapoc now on GKE https://www.linkedin.com/pulse/running-google-dataproc-kubernetes-engine-gke-spark-mich/?trackingId=lz12GC5dRFasLiaJm5qDSw%3D%3D Unless I misunderstood your point. HTH Mich Talebzadeh, Lead S

Spark Kubernetes Operator

2023-04-14 Thread Yuval Itzchakov
Hi, ATM I see the most used option for a Spark operator is the one provided by Google: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator Unfortunately, it doesn't seem actively maintained. Are there any plans to support an official Apache Spark community driven operator?