Re: Run Spark on Yarn Remotely

2013-12-16 Thread Tom Graves
The hadoop conf dir is what controls which YARN cluster it goes to so its a matter of putting in the correct configs for the cluster you want it to go to.  You have to execute the org.apache.spark.deploy.yarn.Client or your application will not run on yarn in standalone mode.   The client is wha

Run Spark on Yarn Remotely

2013-12-16 Thread Karavany, Ido
Hi All, We've started with deploying spark on Hadoop 2 and Yarn. Our previous configuration (still not a production cluster) was Spark on Mesos. We're running a java application (which runs from tomcat server). The application builds a singleton java spark context when it is first lunch and