Re: [Spark K8s] Seeking Advice on Scaling Spark Cluster in Kubernetes

2022-02-02 Thread Han Lin
claimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Wed, 2 Feb 2022 at 13:17, Han Lin wrote: > >> Hello, >> >> Before I ask my questions, let me say what I am trying to d

[Spark K8s] Seeking Advice on Scaling Spark Cluster in Kubernetes

2022-02-02 Thread Han Lin
Hello, Before I ask my questions, let me say what I am trying to do and briefly describe the setup I have so far. I am basically building an API service that serves a ML model which uses Spark ML. I have Spark deployed in Kubernetes in standalone mode (the default Spark manager) with 2 worker node