Alexandru Tofan created SPARK-39617: ---------------------------------------
Summary: Driver cores must be a positive number Key: SPARK-39617 URL: https://issues.apache.org/jira/browse/SPARK-39617 Project: Spark Issue Type: Bug Components: Mesos Affects Versions: 3.2.1 Environment: Spark 3.2.1 Mesos Java 8 OpenJDK Reporter: Alexandru Tofan When updating Spark 3.0.2 to Spark 3.2.1 I get the following error: {code:java} Exception in thread "main" org.apache.spark.SparkException: Driver cores must be a positive number at org.apache.spark.deploy.SparkSubmitArguments.error(SparkSubmitArguments.scala:634) at org.apache.spark.deploy.SparkSubmitArguments.validateSubmitArguments(SparkSubmitArguments.scala:257) at org.apache.spark.deploy.SparkSubmitArguments.validateArguments(SparkSubmitArguments.scala:234) at org.apache.spark.deploy.SparkSubmitArguments.<init>(SparkSubmitArguments.scala:119) at org.apache.spark.deploy.SparkSubmit$$anon$2$$anon$3.<init>(SparkSubmit.scala:1026) at org.apache.spark.deploy.SparkSubmit$$anon$2.parseArguments(SparkSubmit.scala:1026) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:85) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) {code} Obviously I set the driver-cores property but it doesn't matter, it is not used. I'm suspecting that it is something about the Mesos Dispatcher, it receives the property because it is being displayed correctly in the Dispatcher's UI, but it is not forwarded correctly to Spark-Submit, and therefore fails. -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org