Re: Spark Job running on localhost on yarn cluster

2015-02-05 Thread kundan kumar
@spark.apache.org Subject: Re: Spark Job running on localhost on yarn cluster Have you set master in SparkConf/SparkContext in your code? Driver logs show in which mode the spark job is running. Double check if the logs mention local or yarn-cluster. Also, what's the error that you are getting

Re: Spark Job running on localhost on yarn cluster

2015-02-05 Thread Kostas Sakellis
Bhatnagar aniket.bhatna...@gmail.com Sent: February 4, 2015 6:16 AM To: kundan kumar iitr.kun...@gmail.com, spark users user@spark.apache.org Subject: Re: Spark Job running on localhost on yarn cluster Have you set master in SparkConf/SparkContext in your code? Driver logs show in which mode

Re: Spark Job running on localhost on yarn cluster

2015-02-04 Thread Michael Albert
if the parameters disagree with the code). Good luck! -Mike From: kundan kumar iitr.kun...@gmail.com To: spark users user@spark.apache.org Sent: Wednesday, February 4, 2015 7:41 AM Subject: Spark Job running on localhost on yarn cluster Hi,  I am trying to execute my code on a yarn cluster

Re: Spark Job running on localhost on yarn cluster

2015-02-04 Thread Felix C
Is YARN_CONF_DIR set? --- Original Message --- From: Aniket Bhatnagar aniket.bhatna...@gmail.com Sent: February 4, 2015 6:16 AM To: kundan kumar iitr.kun...@gmail.com, spark users user@spark.apache.org Subject: Re: Spark Job running on localhost on yarn cluster Have you set master in SparkConf

Spark Job running on localhost on yarn cluster

2015-02-04 Thread kundan kumar
Hi, I am trying to execute my code on a yarn cluster The command which I am using is $SPARK_HOME/bin/spark-submit --class EDDApp target/scala-2.10/edd-application_2.10-1.0.jar --master yarn-cluster --num-executors 3 --driver-memory 6g --executor-memory 7g outpuPath But, I can see that this