Re: K8s-Spark client mode : Executor image not able to download application jar from driver

Thanks for explaining in such detail and pointing to the source code. Yes, its helpful and cleared lot of confusions. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail:

Re: K8s-Spark client mode : Executor image not able to download application jar from driver

Yes here is why the initial effort didnt work, explained a bit better. As I mentioned earlier SparkContext will add your jars/files (declared with the related conf properties) to the FileServer. If it is a local to the container's fs jar (has schema local:) it will just be resolved to: file +

Re: K8s-Spark client mode : Executor image not able to download application jar from driver

Hi Stavros, Thanks a lot for pointing in right direction. I got stuck in some release, so didn’t got time earlier. The mistake was “LINUX_APP_RESOURCE” : I was using “local” instead it should be “file”. I reached above due to your email only. What I understood: Driver image : $SPARK_HOME/bin

Re: K8s-Spark client mode : Executor image not able to download application jar from driver

Hi Nikhil, Application jar by default is added to spark.jars so it is fetched by executors when tasks are launched (behind the scenes SparkContext

K8s-Spark client mode : Executor image not able to download application jar from driver

Environment: Spark: 2.4.0 Kubernetes:1.14 Query: Does application jar needs to be part of both Driver and Executor image? Invocation point (from Java code): sparkLaunch = new SparkLauncher() .setMaster(LINUX_MASTER)