Re: Spark Yarn executor container memory

Both are part of the heap. > On 16 Aug 2016, at 04:26, Lan Jiang wrote: > > Hello, > > My understanding is that YARN executor container memory is based on > "spark.executor.memory" + “spark.yarn.executor.memoryOverhead”. The first one > is for heap memory and second one is

Spark Yarn executor container memory

Hello, My understanding is that YARN executor container memory is based on "spark.executor.memory" + “spark.yarn.executor.memoryOverhead”. The first one is for heap memory and second one is for offheap memory. The spark.executor.memory is used by -Xmx to set the max heap size. Now my question