Thanks! since my worker is on the same node, -Xss JVM option is for setting thread maximum stack size, my worker does show this option now. now I realized I accidently run the the app run in local mode as I didn't give the master URL when initializing the spark context, for local mode, how to pass jvm option to the app?
hadoop 17315 1 0 14:56 ? 00:02:12 /home/hadoop/ibm-java-x86_64-60/bin/java -cp :/home/hadoop/spark-0.9.0-incubating/conf:/home/hadoop/spark-0.9.0-incubating/assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-hadoop1.2.1.jar -Dspark.akka.logLifecycleEvents=true -Djava.library.path= -Xms512m -Xmx512m -Xss1024k org.apache.spark.deploy.worker.Worker spark://hdtest021.svl.ibm.com:7077 -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/SPARK-JAVA-OPTS-not-picked-up-by-the-application-tp2483p2510.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
