[
https://issues.apache.org/jira/browse/SPARK-44242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mridul Muralidharan resolved SPARK-44242.
-----------------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 41806
[https://github.com/apache/spark/pull/41806]
> Spark job submission failed because Xmx string is available on one parameter
> provided into spark.driver.extraJavaOptions
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-44242
> URL: https://issues.apache.org/jira/browse/SPARK-44242
> Project: Spark
> Issue Type: Bug
> Components: Spark Submit
> Affects Versions: 3.3.2, 3.4.1
> Reporter: Nicolas Fraison
> Assignee: Nicolas Fraison
> Priority: Major
> Fix For: 4.0.0
>
>
> The spark-submit command failed if Xmx string is found on any parameters
> provided to spark.driver.extraJavaOptions.
> For ex. running this spark-submit command line
> {code:java}
> ./bin/spark-submit --class org.apache.spark.examples.SparkPi --conf
> "spark.driver.extraJavaOptions=-Dtest=Xmx"
> examples/jars/spark-examples_2.12-3.4.1.jar 100{code}
> failed due to
> {code:java}
> Error: Not allowed to specify max heap(Xmx) memory settings through java
> options (was -Dtest=Xmx). Use the corresponding --driver-memory or
> spark.driver.memory configuration instead.{code}
> The check performed in
> [https://github.com/apache/spark/blob/master/launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java#L314]
> seems to broad
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
