[ https://issues.apache.org/jira/browse/SPARK-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198945#comment-14198945 ]
Greg Hill commented on SPARK-3884: ---------------------------------- For example: {noformat} spark-submit --master yarn-cluster --driver-memory 12424m --class org.apache.spark.examples.SparkPi /usr/lib/spark-yarn/lib/spark-examples*.jar 1000 OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000006fd280000, 4342677504, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (malloc) failed to allocate 4342677504 bytes for committing reserved memory {noformat} The client machine didn't have that much memory available, but the YARN nodes where the driver will be loaded do. It shouldn't be trying to allocate the memory locally. > If deploy mode is cluster, --driver-memory shouldn't apply to client JVM > ------------------------------------------------------------------------ > > Key: SPARK-3884 > URL: https://issues.apache.org/jira/browse/SPARK-3884 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 1.1.0 > Reporter: Sandy Ryza > Assignee: Sandy Ryza > -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org