[ https://issues.apache.org/jira/browse/SPARK-38960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean R. Owen resolved SPARK-38960. ---------------------------------- Fix Version/s: (was: 3.4.0) Resolution: Won't Fix > Spark should fail fast if initial memory too large(set by > "spark.executor.extraJavaOptions") for executor to start > ------------------------------------------------------------------------------------------------------------------ > > Key: SPARK-38960 > URL: https://issues.apache.org/jira/browse/SPARK-38960 > Project: Spark > Issue Type: Improvement > Components: Spark Core, Spark Submit, YARN > Affects Versions: 3.4.0 > Reporter: panbingkun > Priority: Minor > > if you set initial memory(set by > "spark.executor.extraJavaOptions=-Xms\{XXX}G" ) larger than maximum > memory(set by "spark.executor.memory") > Eg. > *spark.executor.memory=1G* > *spark.executor.extraJavaOptions=-Xms2G* > > from the driver process you just see executor failures with no warning, since > the more meaningful errors are buried in the executor logs. > Eg., on Yarn, you see: > {noformat} > Error occurred during initialization of VM > Initial heap size set to a larger value than the maximum heap size{noformat} > Instead we should just fail fast with a clear error message in the driver > logs. -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org