Spark Job Doesn't End on Mesos

2016-08-09 Thread Todd Leo
-4372-0034' However, the process doesn’t quit after all. This is critical, because I’d like to use SparkLauncher to submit such jobs. If my job doesn’t end, jobs will pile up and fill up the memory. Pls help. :-| — BR, Todd Leo ​

OutOfMemoryError when using DataFrame created by Spark SQL

2015-03-25 Thread Todd Leo
Hi, I am using *Spark SQL* to query on my *Hive cluster*, following Spark SQL and DataFrame Guide https://spark.apache.org/docs/latest/sql-programming-guide.html step by step. However, my HiveQL via sqlContext.sql() fails and java.lang.OutOfMemoryError was raised. The expected result of such