henry
Date: Tuesday, February 14, 2017 at 1:04 AM
To: Conversant
Cc: Jon Gregg , "user @spark"
Subject: Re: Lost executor 4 Container killed by YARN for exceeding memory
limits.
Hi,
How to set this parameters while launching spark shell
spark.shuffle.memoryFractio
GB.
>
> You may want to consult with your DevOps/Operations/Spark Admin team first.
>
>
>
> *From: *Jon Gregg
> *Date: *Monday, February 13, 2017 at 8:58 AM
> *To: *nancy henry
> *Cc: *"user @spark"
> *Subject: *Re: Lost executor 4 Container killed by YARN for
Date: Monday, February 13, 2017 at 8:58 AM
To: nancy henry
Cc: "user @spark"
Subject: Re: Lost executor 4 Container killed by YARN for exceeding memory
limits.
Setting Spark's memoryOverhead configuration variable is recommended in your
logs, and has helped me with these iss
Setting Spark's memoryOverhead configuration variable is recommended in
your logs, and has helped me with these issues in the past. Search for
"memoryOverhead" here:
http://spark.apache.org/docs/latest/running-on-yarn.html
That said, you're running on a huge cluster as it is. If it's possible to
Hi All,,
I am getting below error while I am trying to join 3 tables which are in
ORC format in hive from 5 10gb tables through hive context in spark
Container killed by YARN for exceeding memory limits. 11.1 GB of 11 GB
physical memory used. Consider boosting spark.yarn.executor.memoryOverhead.