[ https://issues.apache.org/jira/browse/SPARK-23472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcelo Vanzin reassigned SPARK-23472: -------------------------------------- Assignee: (was: Marcelo Vanzin) > Add config properties for administrator JVM options > --------------------------------------------------- > > Key: SPARK-23472 > URL: https://issues.apache.org/jira/browse/SPARK-23472 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Affects Versions: 2.3.0 > Reporter: Ryan Blue > Priority: Major > > In our environment, users may need to add JVM options to their Spark > applications (e.g. to override log configuration). They typically use > {{--driver-java-options}} or {{spark.executor.extraJavaOptions}}. Both set > extraJavaOptions properties. We also have a set of administrator JVM options > to apply that set the garbage collector (G1GC) and kill the driver JVM on OOM. > These two use cases both need to set extraJavaOptions properties, but will > clobber one another. In the past we've maintained wrapper scripts, but this > causes our default properties to be maintained in scripts rather than our > spark-defaults.properties. > I think we should add defaultJavaOptions properties that are added along with > extraJavaOptions. Administrators could set defaultJavaOptions and these would > always get added to the JVM command line, along with any user options instead > of getting overwritten by user options. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org