Thanks that explains a lot.
--
Jan Sterba
https://twitter.com/honzasterba | http://flickr.com/honzasterba |
http://500px.com/honzasterba
On Fri, Mar 11, 2016 at 2:36 PM, Silvio Fiorito
wrote:
> Hi Jan,
>
>
>
> Yes what you’re seeing is due to YARN container memory overhead. Also,
> typically the
Hi Jan,
Yes what you’re seeing is due to YARN container memory overhead. Also,
typically the memory increments for YARN containers is 1GB.
This gives a good overview:
http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/
Thanks,
Silvio
From: Jan Štěrba<