[ https://issues.apache.org/jira/browse/SPARK-31530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091184#comment-17091184 ]
Hyukjin Kwon commented on SPARK-31530: -------------------------------------- Oh, gotya. > Spark submit fails if we provide extraJavaOption which contains Xmx as > substring > --------------------------------------------------------------------------------- > > Key: SPARK-31530 > URL: https://issues.apache.org/jira/browse/SPARK-31530 > Project: Spark > Issue Type: Bug > Components: Spark Submit > Affects Versions: 2.4.0 > Reporter: Mayank > Priority: Major > Labels: 2.4.0, Spark, Submit > > Spark submit doesn't allow Xmx anywhere in the spark.driver.extraJavaOptions > For eg: > {code:java} > bin\spark-submit --class org.apache.spark.examples.SparkPi --master local[*] > --conf "spark.driver.extraJavaOptions=-DmyKey=MyValueContainsXmx" > examples\jars\spark-examples_2.11-2.4.4.jar > Error: Not allowed to specify max heap(Xmx) memory settings through java > options (was -DmyKey=MyValueContainsXmx). Use the corresponding > --driver-memory or spark.driver.memory configuration instead.{code} > https://github.com/apache/spark/blob/v2.4.4/launcher/src/main/java/org/apache/spark/launcher/SparkClassCommandBuilder.java#L102 > [https://github.com/apache/spark/blob/master/launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java#L302] > Can we update the above condition to check more specific for eg -Xmx > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org