Re: Spark 1.5 on CDH 5.4.0

HI guys I have documented the steps involved in getting Spark 1.5.1 run on CDH 5.4.0 here, let me know if it works for you as well https://www.linkedin.com/pulse/running-spark-151-cdh-deenar-toraskar-cfa?trk=hp-feed-article-title-publish Looking forward to CDH 5.5 which supports Spark 1.5.x out

Re: Spark 1.5 on CDH 5.4.0

Sandy The assembly jar does contain org.apache.spark.deploy.yarn.ExecutorLauncher. I am trying to find out how i can increase the logging level, so I know the exact classpath used by Yarn ContainerLaunch. Deenar On 23 October 2015 at 03:30, Sandy Ryza wrote: > Hi

Re: Spark 1.5 on CDH 5.4.0

I got this working. For others trying this It turns out in Spark 1.3/CDH5.4 spark.yarn.jar=local:/opt/cloudera/parcels/ I had changed this to reflect the 1.5.1 version of spark assembly jar spark.yarn.jar=/opt/spark-1.5.1-bin/... and this didn't work, I had to drop the "local:" prefix

Re: Spark 1.5 on CDH 5.4.0

Hi Deenar, The version of Spark you have may not be compiled with YARN support. If you inspect the contents of the assembly jar, does org.apache.spark.deploy.yarn.ExecutorLauncher exist? If not, you'll need to find a version that does have the YARN classes. You can also build your own using

Spark 1.5 on CDH 5.4.0

Hi I have got the prebuilt version of Spark 1.5 for Hadoop 2.6 ( http://www.apache.org/dyn/closer.lua/spark/spark-1.5.1/spark-1.5.1-bin-hadoop2.6.tgz) working with CDH 5.4.0 in local mode on a cluster with Kerberos. It works well including connecting to the Hive metastore. I am facing an issue