I guess this is not related to SparkR. It seems that Spark can’t pick hostname/IP address of RM.
Make sure you have correctly set YARN_CONF_DIR env var and have configured address of RM in yarn-site.xml. From: Amit Behera [mailto:amit.bd...@gmail.com] Sent: Friday, November 13, 2015 9:38 PM To: Sun, Rui; user@spark.apache.org Subject: Re: Connecting SparkR through Yarn Hi Sun, Thank you for reply. I did the same, but now I am getting another issue. i.e: Not able to connect to ResourceManager after submitting the job the Error message showing something like this Connecting to ResourceManager at /0.0.0.0:8032<http://0.0.0.0:8032> ------------- INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032<http://0.0.0.0/0.0.0.0:8032>. Already tried 0 time(s); -------------INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032<http://0.0.0.0/0.0.0.0:8032>. Already tried 1 time(s) I tried resolving this , but not able to do. On Wed, Nov 11, 2015 at 12:02 PM, Sun, Rui <rui....@intel.com<mailto:rui....@intel.com>> wrote: Amit, You can simply set “MASTER” as “yarn-client” before calling sparkR.init(). Sys.setenv("MASTER"="yarn-client") I assume that you have set “YARN_CONF_DIR” env variable required for running Spark on YARN. If you want to set more YARN specific configurations, you can for example Sys.setenv ("SPARKR_SUBMIT_ARGS", " --master yarn-client --num-executors 4 sparkr-shell" Before calling sparkR.init(). From: Amit Behera [mailto:amit.bd...@gmail.com<mailto:amit.bd...@gmail.com>] Sent: Monday, November 9, 2015 2:36 AM To: user@spark.apache.org<mailto:user@spark.apache.org> Subject: Connecting SparkR through Yarn Hi All, Spark Version = 1.5.1 Hadoop Version = 2.6.0 I set up the cluster in Amazon EC2 machines (1+5) I am able create a SparkContext object using init method from RStudio. But I do not know how can I create a SparkContext object in yarn mode. I got the below link to run on yarn. but in this document its given for Spark version >= 0.9.0 and <= 1.2. https://github.com/amplab-extras/SparkR-pkg/blob/master/README.md#running-on-yarn Please help me how can I connect SparkR on Yarn. Thanks, Amit.
