How do you increase the spark block manager timeout?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Akka-disassociation-on-Java-SE-Embedded-tp6266p16176.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
.nabble.com/Akka-disassociation-on-Java-SE-Embedded-tp6266p16176.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
(http://Nabble.com).
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
Hi all,
This is what I found:
1. Like Aaron suggested, an executor will be killed silently when the
OS's memory is running out.
I've found this many times to conclude this it's real. Adding swap and
increasing the JVM heap solved the problem, but you will encounter OS
paging out and full GC.
2.
Thanks for the update! I've also run into the block manager timeout issue,
it might be a good idea to increase the default significantly (it would
probably timeout immediately if the TCP connection itself dropped anyway).
On Sun, Jun 1, 2014 at 9:48 AM, Chanwit Kaewkasi chan...@gmail.com wrote:
Sorry, to clarify: Spark *does* effectively turn Akka's failure detector
off.
On Tue, May 27, 2014 at 10:47 AM, Aaron Davidson ilike...@gmail.com wrote:
Spark should effectively turn Akka's failure detector off, because we
historically had problems with GCs and other issues causing
Spark should effectively turn Akka's failure detector off, because we
historically had problems with GCs and other issues causing
disassociations. The only thing that should cause these messages nowadays
is if the TCP connection (which Akka sustains between Actor Systems on
different machines)
May be that's explaining mine too.
Thank you very much, Aaron !!
Best regards,
-chanwit
--
Chanwit Kaewkasi
linkedin.com/in/chanwit
On Wed, May 28, 2014 at 12:47 AM, Aaron Davidson ilike...@gmail.com wrote:
Spark should effectively turn Akka's failure detector off, because we
historically
Hi all,
On an ARM cluster, I have been testing a wordcount program with JRE 7
and everything is OK. But when changing to the embedded version of
Java SE (Oracle's eJRE), the same program cannot complete all
computing stages.
It is failed by many Akka's disassociation.
- I've been trying to
