[ https://issues.apache.org/jira/browse/SPARK-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Or resolved SPARK-12081. ------------------------------- Resolution: Fixed Fix Version/s: 1.6.0 > Make unified memory management work with small heaps > ---------------------------------------------------- > > Key: SPARK-12081 > URL: https://issues.apache.org/jira/browse/SPARK-12081 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 1.6.0 > Reporter: Andrew Or > Assignee: Andrew Or > Priority: Critical > Fix For: 1.6.0 > > > By default, Spark drivers and executors are 1GB. With the recent unified > memory mode, only 250MB is set aside for non-storage non-execution purposes > (spark.memory.fraction is 75%). However, especially in local mode, the driver > needs at least ~300MB. Some local jobs started to OOM because of this. > Two mutually exclusive proposals: > (1) First, cut out 300 MB, then take 75% of what remains > (2) Use min(75% of JVM heap size, JVM heap size - 300MB) -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org