RE: Submitting Jobs Programmatically

2016-02-21 Thread Patrick Mi
Mukherjee [mailto:arkoprovomukher...@gmail.com] Sent: Saturday, 20 February 2016 4:27 p.m. To: Ted Yu Cc: Holden Karau; user Subject: Re: Submitting Jobs Programmatically Hello, Thanks much. I could start the service. When I run my program, the launcher is not being able to find the app class

Re: Submitting Jobs Programmatically

2016-02-19 Thread Arko Provo Mukherjee
Hello, Thanks much. I could start the service. When I run my program, the launcher is not being able to find the app class: java.lang.ClassNotFoundException: SparkSubmitter at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at

Re: Submitting Jobs Programmatically

2016-02-19 Thread Ted Yu
Cycling old bits: http://search-hadoop.com/m/q3RTtHrxMj2abwOk2 On Fri, Feb 19, 2016 at 6:40 PM, Arko Provo Mukherjee < arkoprovomukher...@gmail.com> wrote: > Hi, > > Thanks for your response. Is there a similar link for Windows? I am > not sure the .sh scripts would run on windows. > > My

Re: Submitting Jobs Programmatically

2016-02-19 Thread Arko Provo Mukherjee
Hi, Thanks for your response. Is there a similar link for Windows? I am not sure the .sh scripts would run on windows. My default the start-all.sh doesn't work and I don't see anything in localhos:8080 I will do some more investigation and come back. Thanks again for all your help! Thanks &

Re: Submitting Jobs Programmatically

2016-02-19 Thread Ted Yu
Please see https://spark.apache.org/docs/latest/spark-standalone.html On Fri, Feb 19, 2016 at 6:27 PM, Arko Provo Mukherjee < arkoprovomukher...@gmail.com> wrote: > Hi, > > Thanks for your response, that really helped. > > However, I don't believe the job is being submitted. When I run spark >

Re: Submitting Jobs Programmatically

2016-02-19 Thread Arko Provo Mukherjee
Hi, Thanks for your response, that really helped. However, I don't believe the job is being submitted. When I run spark from the shell, I don't need to start it up explicitly. Do I need to start up Spark on my machine before running this program? I see the following in the SPARK_HOME\bin

Re: Submitting Jobs Programmatically

2016-02-19 Thread Holden Karau
How are you trying to launch your application? Do you have the Spark jars on your class path? On Friday, February 19, 2016, Arko Provo Mukherjee < arkoprovomukher...@gmail.com> wrote: > Hello, > > I am trying to submit a spark job via a program. > > When I run it, I receive the following error:

Submitting Jobs Programmatically

2016-02-19 Thread Arko Provo Mukherjee
Hello, I am trying to submit a spark job via a program. When I run it, I receive the following error: Exception in thread "Thread-1" java.lang.NoClassDefFoundError: org/apache/spark/launcher/SparkLauncher at Spark.SparkConnector.run(MySpark.scala:33) at