Hi Attila, Ah that makes sense. Thanks for the clarification!
Best, Shiqi On Mon, Apr 26, 2021 at 8:09 PM Attila Zsolt Piros < piros.attila.zs...@gmail.com> wrote: > Hi Shiqi, > > In case of client mode the driver runs locally: in the same machine, even > in the same process, of the spark submit. > > So if the application was submitted in a running POD then the driver will > be running in a POD and when outside of K8s then it will be running > outside. > This is why there is no config mentioned for this. > > From the deploy mode in general you can read here: > https://spark.apache.org/docs/latest/submitting-applications.html > > Best Regards, > Attila > > On Tue, Apr 27, 2021 at 12:03 AM Shiqi Sun <jack.sun...@gmail.com> wrote: > >> Hi Spark User group, >> >> I have a couple of quick questions about running Spark in Kubernetes >> between different deploy modes. >> >> As specified in >> https://spark.apache.org/docs/latest/running-on-kubernetes.html#client-mode, >> since Spark 2.4, client mode support is available when running in >> Kubernetes, and it says "when your application runs in client mode, the >> driver can run inside a pod or on a physical host". Then here come the >> questions. >> >> 1. If I understand correctly, in cluster mode, the driver is also running >> inside a k8s pod. Then, what's the difference between running it in cluster >> mode, versus running it in client mode when I choose to run my driver in a >> pod? >> >> 2. What does it mean by "running driver on a physical host"? Does it mean >> that it runs outside of the k8s cluster? What config should I pass to spark >> submit so that it runs this way, instead of running my driver into a k8s >> pod? >> >> Thanks! >> >> Best, >> Shiqi >> >
