I have a config file that exploits type safe config library located on the local file system, and want to submit that file through spark-submit so that spark program can read customized parameters. For instance,
my.app { db { host = domain.cc port = 1234 db = dbname user = myuser passwd = mypass } } Spark submit code looks like spark-submit --class "my.app.Sample" --master local[*] --conf "spark.executor.extraJavaOptions=-Dconfig.file=/path/to/conf/myapp.conf" /path/to/my-app.jar But the program can not read the parameters such as db, user, host, and so on in my conf file. Passing with --files /path/to/myapp.conf doesn't work either. What is the correct way to submit that kind of conf file so that my spark job can read customized parameters from there? Thanks