[
https://issues.apache.org/jira/browse/SPARK-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14234212#comment-14234212
]
Apache Spark commented on SPARK-1953:
-------------------------------------
User 'WangTaoTheTonic' has created a pull request for this issue:
https://github.com/apache/spark/pull/3607
> yarn client mode Application Master memory size is same as driver memory size
> -----------------------------------------------------------------------------
>
> Key: SPARK-1953
> URL: https://issues.apache.org/jira/browse/SPARK-1953
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.0.0
> Reporter: Thomas Graves
>
> With Spark on yarn in client mode, the application master that gets created
> to allocated containers gets the same amount of memory as the driver running
> on the client. (--driver-memory option through spark-submit) This could
> definitely be more then what is really needed, thus wasting resources. The
> Application Master should be very small and require very little memory since
> all its doing is allocating and starting containers.
> We should allow the memory for the application master to be configurable
> separate from the driver in client mode.
> We probably need to be careful about how we do this as to not cause
> confusion about what the options do in the various modes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
