Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20669 Hm, noted that we're making this tradeoff. We have an internal use case where we're pushing a custom logging properties file into the container using `spark.files`. Logging properties files need to be in the container before the JVM starts to configure the appenders from the get-go, but logging properties are more dynamic and probably don't belong in a statically built Docker image. We use YARN cluster mode primarily and rely on its file distribution, and we migrated to the fork's implementation of Kubernetes without having to change our internal setup. I think we can adapt to this change, but I don't think the use case I've described is as uncommon as one may think. There's plenty of lower-level tooling out there that requires the JVM to load files in static initializations. > Oh, btw, if you think that is a really, really important feature, you still don't need an init container for that. You can just run the dependency download tool before you run spark-submit in the driver container. Problem solved. Agreed. Init-containers are but one option to support this. The question was more if running spark-submit in client mode is completely sufficient, which it seems like it isn't in this specific case.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org