Im trying to run spark-submit to kubernetes cluster with spark 2.3 docker container image
The challenge im facing is application have a mainapplication.jar and other dependency files & jars which are located in Remote location like AWS s3 ,but as per spark 2.3 documentation there is something called kubernetes init-container to download remote dependencies but in this case im not creating any Podspec to include init-containers in kubernetes, as per documentation Spark 2.3 spark/kubernetes internally creates Pods (driver,executor) So not sure how can i use init-container for spark-submit when there are remote dependencies. https://spark.apache.org/docs/latest/running-on-kubernetes.html#using-remote-dependencies Please suggest
