[
https://issues.apache.org/jira/browse/SPARK-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198945#comment-14198945
]
Greg Hill commented on SPARK-3884:
--
For example:
{noformat}
spark-submit --master yarn-cluster --driver-memory 12424m --class
org.apache.spark.examples.SparkPi /usr/lib/spark-yarn/lib/spark-examples*.jar
1000
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0006fd28,
4342677504, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 4342677504 bytes for
committing reserved memory
{noformat}
The client machine didn't have that much memory available, but the YARN nodes
where the driver will be loaded do. It shouldn't be trying to allocate the
memory locally.
> If deploy mode is cluster, --driver-memory shouldn't apply to client JVM
>
>
> Key: SPARK-3884
> URL: https://issues.apache.org/jira/browse/SPARK-3884
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.1.0
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
