You need to pass config before creating a session
val conf = new SparkConf()
// All three methods below are equivalent
conf.set("spark.executor.extraJavaOptions", "-Dbasicauth=myuser:mypassword")
conf.set("spark.executorEnv.basicauth", "myuser:mypassword")
conf.setExecutorEnv("basicauth",
Hi all,
I wrote an application that needs an environment variable. I can set this
variable with
--conf 'spark.executor.extraJavaOptions=-Dbasicauth=myuser:mypwd'
in spark-submit and it works well in standalone cluster mode.
But, I want set it inside the application code, because the variable
