RE: Spark Kubernetes Operator

[Aldo Culquicondor]

We are welcoming contributors, as announced in the Kubernetes WG Batch
https://docs.google.com/document/d/1XOeUN-K0aKmJJNq7H07r74n-mGgSFyiEDQ3ecwsGhec/edit#bookmark=id.gfgjt0nmbgjl

If you are interested, you can find us in slack.k8s.io #wg-batch or
ping @mwielgus on github/slack.

Thanks

On 2023/04/14 16:41:36 Yuval Itzchakov wrote:
> Hi,
>
> ATM I see the most used option for a Spark operator is the one provided by
> Google: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator
>
> Unfortunately, it doesn't seem actively maintained. Are there any plans to
> support an official Apache Spark community driven operator?
>

Re: Spark Kubernetes Operator

[Yuval Itzchakov]

I'm not running on GKE. I am wondering what's the long term strategy around
a Spark operator. Operators are the de-facto way to run complex
deployments. The Flink community now has an official community led
operator, and I was wondering if there are any similar plans for Spark.

On Fri, Apr 14, 2023, 19:51 Mich Talebzadeh 
wrote:

> Hi,
>
> What exactly are you trying to achieve? Spark on GKE works fine and you
> can run Datapoc now on GKE
> https://www.linkedin.com/pulse/running-google-dataproc-kubernetes-engine-gke-spark-mich/?trackingId=lz12GC5dRFasLiaJm5qDSw%3D%3D
>
> Unless I misunderstood your point.
>
> HTH
>
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
> London
> United Kingdom
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Fri, 14 Apr 2023 at 17:42, Yuval Itzchakov  wrote:
>
>> Hi,
>>
>> ATM I see the most used option for a Spark operator is the one provided
>> by Google: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator
>>
>> Unfortunately, it doesn't seem actively maintained. Are there any plans
>> to support an official Apache Spark community driven operator?
>>
>

Re: Spark Kubernetes Operator

[Mich Talebzadeh]

Hi,

What exactly are you trying to achieve? Spark on GKE works fine and you can
run Datapoc now on GKE
https://www.linkedin.com/pulse/running-google-dataproc-kubernetes-engine-gke-spark-mich/?trackingId=lz12GC5dRFasLiaJm5qDSw%3D%3D

Unless I misunderstood your point.

HTH


Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited
London
United Kingdom


   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Fri, 14 Apr 2023 at 17:42, Yuval Itzchakov  wrote:

> Hi,
>
> ATM I see the most used option for a Spark operator is the one provided by
> Google: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator
>
> Unfortunately, it doesn't seem actively maintained. Are there any plans to
> support an official Apache Spark community driven operator?
>