To add to the pile of information we're asking you to provide, what version of Spark are you running?
2014-08-13 11:11 GMT-07:00 Shivaram Venkataraman <shiva...@eecs.berkeley.edu >: > If the JVM heap size is close to the memory limit the OS sometimes kills > the process under memory pressure. I've usually found that lowering the > executor memory size helps. > > Shivaram > > > On Wed, Aug 13, 2014 at 11:01 AM, Matei Zaharia <matei.zaha...@gmail.com> > wrote: > >> What is your Spark executor memory set to? (You can see it in Spark's web >> UI at http://<driver>:4040 under the executors tab). One thing to be >> aware of is that the JVM never really releases memory back to the OS, so it >> will keep filling up to the maximum heap size you set. Maybe 4 executors >> with that much heap are taking a lot of the memory. >> >> Persist as DISK_ONLY should indeed stream data from disk, so I don't >> think that will be a problem. >> >> Matei >> >> On August 13, 2014 at 6:49:11 AM, rpandya (r...@iecommerce.com) wrote: >> >> After a lot of grovelling through logs, I found out that the Nagios >> monitor >> process detected that the machine was almost out of memory, and killed >> the >> SNAP executor process. >> >> So why is the machine running out of memory? Each node has 128GB of RAM, >> 4 >> executors, about 40GB of data. It did run out of memory if I tried to >> cache() the RDD, but I would hope that persist() is implemented so that >> it >> would stream to disk without trying to materialize too much data in RAM. >> >> Ravi >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Lost-executors-tp11722p12032.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> >
