Not sure if it will work, but you can try creating a dummy hadoop conf directory and put those files (*-site.xml) files inside it and hopefully spark will pick it up and submit it on that remote cluster. (If there isn't any network/firewall issues).
Thanks Best Regards On Mon, Nov 24, 2014 at 4:16 PM, Naveen Kumar Pokala < npok...@spcapitaliq.com> wrote: > Hi Akhil, > > > > But driver and yarn both are in different networks, How to specify (export > HADOOP_CONF_DIR=XXX) path. > > > > Like driver is from my windows machine and yarn is some unix machine on > different network. > > > > -Naveen. > > > > *From:* Akhil Das [mailto:ak...@sigmoidanalytics.com > <ak...@sigmoidanalytics.com>] > *Sent:* Monday, November 24, 2014 4:08 PM > *To:* Naveen Kumar Pokala > *Cc:* user@spark.apache.org > *Subject:* Re: Submit Spark driver on Yarn Cluster in client mode > > > > You can export the hadoop configurations dir (export > HADOOP_CONF_DIR=XXX) in the environment and then submit it like: > > > > ./bin/spark-submit \ > > --class org.apache.spark.examples.SparkPi \ > > --master yarn-cluster \ # can also be `yarn-client` for client mode > > --executor-memory 20G \ > > --num-executors 50 \ > > /path/to/examples.jar \ > > 1000 > > > > More details over here > https://spark.apache.org/docs/1.1.0/submitting-applications.html#launching-applications-with-spark-submit > > > Thanks > > Best Regards > > > > On Mon, Nov 24, 2014 at 4:01 PM, Naveen Kumar Pokala < > npok...@spcapitaliq.com> wrote: > > Hi, > > > > I want to submit my spark program from my machine on a YARN Cluster in > yarn client mode. > > > > How to specify al l the required details through SPARK submitter. > > > > Please provide me some details. > > > > -Naveen. > > >
