Hi TD, Basically, I see two issues. With <scope>provided</scope> the job does not start localy. It does start in Cluster but seems no data is getting processed.
Thanks, Swetha On Tue, Nov 17, 2015 at 7:04 PM, Tim Barthram <tim.barth...@iag.com.au> wrote: > If you are running a local context, could it be that you should use: > > > > <scope>provided</scope> > > > > ? > > > > Thanks, > > Tim > > > > *From:* swetha kasireddy [mailto:swethakasire...@gmail.com] > *Sent:* Wednesday, 18 November 2015 2:01 PM > *To:* Tathagata Das > *Cc:* user > *Subject:* Re: Streaming Job gives error after changing to version 1.5.2 > > > > This error I see locally. > > > > On Tue, Nov 17, 2015 at 5:44 PM, Tathagata Das <t...@databricks.com> > wrote: > > Are you running 1.5.2-compiled jar on a Spark 1.5.2 cluster? > > > > On Tue, Nov 17, 2015 at 5:34 PM, swetha <swethakasire...@gmail.com> wrote: > > > > Hi, > > I see java.lang.NoClassDefFoundError after changing the Streaming job > version to 1.5.2. Any idea as to why this is happening? Following are my > dependencies and the error that I get. > > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-core_2.10</artifactId> > <version>${sparkVersion}</version> > <scope>provided</scope> > </dependency> > > > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-streaming_2.10</artifactId> > <version>${sparkVersion}</version> > <scope>provided</scope> > </dependency> > > > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-sql_2.10</artifactId> > <version>${sparkVersion}</version> > <scope>provided</scope> > </dependency> > > > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-hive_2.10</artifactId> > <version>${sparkVersion}</version> > <scope>provided</scope> > </dependency> > > > > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-streaming-kafka_2.10</artifactId> > <version>${sparkVersion}</version> > </dependency> > > > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/spark/streaming/StreamingContext > at java.lang.Class.getDeclaredMethods0(Native Method) > at java.lang.Class.privateGetDeclaredMethods(Class.java:2693) > at java.lang.Class.privateGetMethodRecursive(Class.java:3040) > at java.lang.Class.getMethod0(Class.java:3010) > at java.lang.Class.getMethod(Class.java:1776) > at > com.intellij.rt.execution.application.AppMain.main(AppMain.java:125) > Caused by: java.lang.ClassNotFoundException: > org.apache.spark.streaming.StreamingContext > at java.net.URLClassLoader$1.run(URLClassLoader.java:372) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:360) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Streaming-Job-gives-error-after-changing-to-version-1-5-2-tp25406.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > > > > > > _____________________________________________________________________ > > The information transmitted in this message and its attachments (if any) > is intended > only for the person or entity to which it is addressed. > The message may contain confidential and/or privileged material. Any > review, > retransmission, dissemination or other use of, or taking of any action in > reliance > upon this information, by persons or entities other than the intended > recipient is > prohibited. > > If you have received this in error, please contact the sender and delete > this e-mail > and associated material from any computer. > > The intended recipient of this e-mail may only use, reproduce, disclose or > distribute > the information contained in this e-mail and any attached files, with the > permission > of the sender. > > This message has been scanned for viruses. > _____________________________________________________________________ >
