RE: Apache Spark Operator for Kubernetes?

2022-10-28 Thread Jim Halfpenny
Hi Clayton, I’m not aware of an official Apache operator, but I can recommend taking a look a the one we’re created at Stackable. https://github.com/stackabletech/spark-k8s-operator It’s actively maintained and we’d be happy to receive feedback if you have feature requests. Kind regards, Jim

Re: Apache Spark Operator for Kubernetes?

2022-10-14 Thread Artemis User
If you have the hardware resources, it isn't difficult to set up Spark in a kubernetes cluster.  The online doc describes everything you would need (https://spark.apache.org/docs/latest/running-on-kubernetes.html). You're right, both AWS EMR and Google's environment aren't flexible and not