Re: docker image distribution in Kubernetes cluster

Thanks Khalid for your notes I have not come across a use case where the docker version on the driver and executors need to be different. My thinking is that spark.kubernetes.executor.container.image is the correct reference as in the Kubernetes where container is the correct terminology and

Re: docker image distribution in Kubernetes cluster

Hi Mitch IMO, it's done to provide most flexibility. So, some users can have limited/restricted version of the image or with some additional software that they use on the executors that is used during processing. So, in your case you only need to provide the first one since the other two configs

Re: docker image distribution in Kubernetes cluster

Just a correction that in Spark 3.2 documentation it states that Property NameDefaultMeaning spark.kubernetes.container.image (none) Container image to use for the Spark application. This is usually of the form

docker image distribution in Kubernetes cluster

Hi, We have three conf parameters to distribute the docker image with spark-sumit in Kubernetes cluster. These are spark-submit --verbose \ --conf spark.kubernetes.driver.docker.image=${IMAGEGCP} \ --conf spark.kubernetes.executor.docker.image=${IMAGEGCP} \