Please make sure the spark shell script you're running is pointed to <dir-to-spark-1.4.1>/bin/spark-shell
Just follow the instructions to correctly configure your spark 1.4.1 and execute correct script are enough. On Wed, Dec 9, 2015 at 11:28 AM, Divya Gehlot <divya.htco...@gmail.com> wrote: > Hi, > As per requirement I need to use Spark 1.4.1.But HDP doesnt comes with > Spark 1.4.1 version. > As instructed in this hortonworks page > <http://hortonworks.com/hadoop-tutorial/apache-spark-1-4-1-technical-preview-with-hdp/> > I am able to set up Spark 1.4 in HDP ,but when I run the spark shell It > shows Spark 1.3.1 REPL instead of spark 1.4.1 . > Do I need to make any configuration changes apart from instructions given > in above mentioned page. > How do I set spark 1.4.1 as the default spark engine. > Options : > 1.Do I need to remove the current spark 1.3.1 dir. > 2. I named my spark installation dir names spark 1.4.1,Do I need to rename > as Spark. > 3.Is there configuration needs to change in HDP to set spark 1.4.1 as > default spark engine . > > Would really appreciate your help. > > Thanks, > Regards, >
