Re: spark-submit failing but job running from scala ide

2016-09-26 Thread Marco Mistroni
Hi Vr your code works fine for me, running on Windows 10 vs Spark 1.6.1 i m guessing your Spark installation could be busted? That would explain why it works on your IDE, as you are just importing jars in your project. The java.io.IOException: Failed to connect to error is misleading, i have

Re: spark-submit failing but job running from scala ide

2016-09-26 Thread vr spark
Hi Jacek/All, I restarted my terminal and then i try spark-submit and again getting those errors. How do i see how many "runtimes" are running and how to have only one? some how my spark 1.6 and spark 2.0 are conflicting. how to fix it? i installed spark 1.6 earlier using this steps

Re: spark-submit failing but job running from scala ide

2016-09-25 Thread Jacek Laskowski
Hi, How did you install Spark 1.6? It's usually as simple as rm -rf $SPARK_1.6_HOME, but it really depends on how you installed it in the first place. Pozdrawiam, Jacek Laskowski https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark Follow me at

Re: spark-submit failing but job running from scala ide

2016-09-25 Thread vr spark
yes, i have both spark 1.6 and spark 2.0. I unset the spark home environment variable and pointed spark submit to 2.0. Its working now. How do i uninstall/remove spark 1.6 from mac? Thanks On Sun, Sep 25, 2016 at 4:28 AM, Jacek Laskowski wrote: > Hi, > > Can you execute

Re: spark-submit failing but job running from scala ide

2016-09-25 Thread Jacek Laskowski
Hi, Can you execute run-example SparkPi with your Spark installation? Also, see the logs: 16/09/24 23:15:15 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041. 16/09/24 23:15:15 INFO Utils: Successfully started service 'SparkUI' on port 4041. You've got two Spark

spark-submit failing but job running from scala ide

2016-09-25 Thread vr spark
Hi, I have this simple scala app which works fine when i run it as scala application from the scala IDE for eclipse. But when i export is as jar and run it from spark-submit i am getting below error. Please suggest *bin/spark-submit --class com.x.y.vr.spark.first.SimpleApp test.jar* 16/09/24