Hi,
I have CDH 5.4 installed on a linux server. It has 1 cluster in which spark
is deployed as a history server.
I am trying to connect my laptop to the spark history server.
When I run spark-shell master ip: port number I get the following output
How can I verify that the worker is connected to
Hi Ashish,
Are you running Spark-on-YARN on the cluster with an instance of Spark History
server?
Also if you are using Cloudera Manager and using Spark on YARN, spark on yarn
service has a link for the history server web UI.
Can you paste the command and the output you are seeing in the
Thank you Ayan for your response.. But I have just realised that the Spark
is configured to be a history server.
Please, can somebody suggest to me how can I convert Spark history server
to be a Master server?
Thank you
Sincerely,
Ashish Dutt
On Wed, Jul 8, 2015 at 12:28 PM, ayan guha
Hello Guru,
Thank you for your quick response.
This is what i get when I try executing spark-shell master ip:port number
C:\spark-1.4.0\binspark-shell master IP:18088
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please
Hi Ashish,
If you are not using Spark on YARN and instead using Spark Standalone, you
don’t need Spark history server. More on the Web Interfaces is provided in the
following link. Since are using standalone mode, you should be able to access
the web UI for the master and workers at ports that
On UI?
Master: http://masterip:8080
Worker: http://workerIp:8081
On Wed, Jul 8, 2015 at 1:42 PM, Ashish Dutt ashish.du...@gmail.com wrote:
Hi,
I have CDH 5.4 installed on a linux server. It has 1 cluster in which
spark is deployed as a history server.
I am trying to connect my laptop to the
Hello Guru,
Many thanks for your reply.
I am new to this who thing. So pardon me for my naiivety at times.
I am not sure if I am using Spark standalone or Spark on Yarn because when
I check the port number of Spark it shows it as 18088 and like you have
mentioned maybe it is then Spark on Yarn.
