Re: pyspark - spark history server

2016-02-05 Thread cs user
Hi Folks, So the fix for me was to copy this file on the nodes built with Ambari: /usr/hdp/2.3.4.0-3485/spark/lib/spark-assembly-1.5.2.2.3.4.0-3485-hadoop2.7.1.2.3.4.0-3485.jar To this file on the client machine, external to the cluster: /opt/spark/lib/spark-assembly-1.5.2-hadoop2.6.0.jar I

pyspark - spark history server

2016-02-05 Thread cs user
Hi All, I'm having trouble getting a job to use the spark history server. We have a cluster configured with Ambari, if I run the job from one of the nodes within the Ambari configured cluster, everything works fine, the job appears in the spark history server. If I configure a client external to