Andrew Korzhuev created SPARK-23133: ---------------------------------------
Summary: Spark options are not passed to the Executor in Docker context Key: SPARK-23133 URL: https://issues.apache.org/jira/browse/SPARK-23133 Project: Spark Issue Type: Bug Components: Kubernetes Affects Versions: 2.3.0 Environment: Running Spark on K8s using supplied Docker image. Reporter: Andrew Korzhuev Reproduce: # Build image with `bin/docker-image-tool.sh`. # Submit application to k8s. Set executor options, e.g. ` --conf "spark.executor. extraJavaOptions=-Djava.security.auth.login.config=./jaas.conf"` # Visit Spark UI on executor and notice that option is not set. Expected behavior: options from spark-submit should be correctly passed to executor. Cause: `SPARK_EXECUTOR_JAVA_OPTS` is not defined in `entrypoint.sh` https://github.com/apache/spark/blob/branch-2.3/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh#L70 [https://github.com/apache/spark/blob/branch-2.3/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh#L44-L45] -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org