Amit, You can simply set “MASTER” as “yarn-client” before calling sparkR.init(). Sys.setenv("MASTER"="yarn-client")
I assume that you have set “YARN_CONF_DIR” env variable required for running Spark on YARN. If you want to set more YARN specific configurations, you can for example Sys.setenv ("SPARKR_SUBMIT_ARGS", " --master yarn-client --num-executors 4 sparkr-shell" Before calling sparkR.init(). From: Amit Behera [mailto:amit.bd...@gmail.com] Sent: Monday, November 9, 2015 2:36 AM To: user@spark.apache.org Subject: Connecting SparkR through Yarn Hi All, Spark Version = 1.5.1 Hadoop Version = 2.6.0 I set up the cluster in Amazon EC2 machines (1+5) I am able create a SparkContext object using init method from RStudio. But I do not know how can I create a SparkContext object in yarn mode. I got the below link to run on yarn. but in this document its given for Spark version >= 0.9.0 and <= 1.2. https://github.com/amplab-extras/SparkR-pkg/blob/master/README.md#running-on-yarn Please help me how can I connect SparkR on Yarn. Thanks, Amit.