[ https://issues.apache.org/jira/browse/SPARK-23449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-23449: ---------------------------------- Fix Version/s: (was: 2.3.0) > Extra java options lose order in Docker context > ----------------------------------------------- > > Key: SPARK-23449 > URL: https://issues.apache.org/jira/browse/SPARK-23449 > Project: Spark > Issue Type: Bug > Components: Kubernetes > Affects Versions: 2.3.0 > Environment: Running Spark on K8S with supplied Docker image. Passing > along extra java options. > Reporter: Andrew Korzhuev > Priority: Minor > Original Estimate: 1h > Remaining Estimate: 1h > > `spark.driver.extraJavaOptions` and `spark.executor.extraJavaOptions` when > processed in `entrypoint.sh` does not preserve its ordering, which makes > `-XX:+UnlockExperimentalVMOptions` unusable, as you have to pass it before > any other experimental options. > > Steps to reproduce: > # Set `spark.driver.extraJavaOptions`, e.g. > `-XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:+CMSClassUnloadingEnabled > -XX:+UseCGroupMemoryLimitForHeap` > # Submit application to k8s cluster. > # Fetch logs and observe that on each run order of options is different and > when `-XX:+UnlockExperimentalVMOptions` is not the first startup will fail. > > Expected behaviour: > # Order of `extraJavaOptions` should be preserved. > > Cause: > `entrypoint.sh` fetches environment options with `env`, which doesn't > guarantee ordering. > {code:java} > env | grep SPARK_JAVA_OPT_ | sed 's/[^=]*=\(.*\)/\1/g' > > /tmp/java_opts.txt{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org