Use of SPARK_DAEMON_JAVA_OPTS

2014-07-23 Thread MEETHU MATHEW
 Hi all, Sorry for taking this topic again,still I am confused on this. I set SPARK_DAEMON_JAVA_OPTS=-XX:+UseCompressedOops -Xmx8g              when I run my application,I  got the following line in logs. Spark Command: java -cp

Re: Use of SPARK_DAEMON_JAVA_OPTS

2014-07-23 Thread Andrew Or
Hi Meethu, SPARK_DAEMON_JAVA_OPTS is not intended for setting memory. Please use SPARK_DAEMON_MEMORY instead. It turns out that java respects only the last -Xms and -Xmx values, and in spark-class we put SPARK_DAEMON_JAVA_OPTS before the SPARK_DAEMON_MEMORY. In general, memory configuration in