[
https://issues.apache.org/jira/browse/SPARK-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095721#comment-14095721
]
Thomas Graves commented on SPARK-2140:
--------------------------------------
ah it seems things have changed. Its now actually the opposite problem now
where yarn alpha is getting less then it should be.
line 87 is what its actually asking from YARN. the calculateAMMemory is what
its using for heap. So it appears yarn stable is correct right now. It
appears yarn alpha in calculateAMmemory subtracts out the memory overhead when
it shouldn't.
> yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM
> -----------------------------------------------------------------
>
> Key: SPARK-2140
> URL: https://issues.apache.org/jira/browse/SPARK-2140
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.0.0
> Reporter: Thomas Graves
> Fix For: 1.0.1, 1.1.0
>
>
> The yarn stable client doesn't properly remove the MEMORY_OVERHEAD amount
> from the java heap size, the code to handle that is commented out (see
> function calculateAMMemory). We should fix this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
