Hi Ibrahim,
If your worker machines only have 8GB of memory, then launching executors
with all the memory will leave no room for system processes. There is no
guideline, but I usually leave around 1GB just to be safe, so
conf.set("spark.executor.memory", "7g")
Andrew
2014-05-22 7:23 GMT-07:00 İbrahim Rıza HALLAÇ <ibrahimrizahal...@live.com>:
> In my situation each slave has 8 GB memory. I want to use the maximum
> memory that I can: .set("spark.executor.memory", "?g")
> How can I determine the amount of memory I should set ? It fails when I
> set it to 8GB.
>
