Re: java.lang.ClassNotFoundException: org.apache.spark.streaming.twitter.TwitterReceiver
gt;>> > http://maven.apache.org/xsd/maven-4.0.0.xsd;> >>>>> > 4.0.0 >>>>> > SparkFirstTry >>>>> > SparkFirstTry >>>>> > 0.0.1-SNAPSHOT >>>>> > >>>>> > >>>>> > >>>>> > org.apache.spark >>>>> > spark-core_2.10 >>>>> > 1.5.1 >>>>> > provided >>>>> > >>>>> > >>>>> > >>>>> > org.apache.spark >>>>> > spark-streaming_2.10 >>>>> > 1.5.1 >>>>> > provided >>>>> > >>>>> > >>>>> > >>>>> > org.twitter4j >>>>> > twitter4j-stream >>>>> > 3.0.3 >>>>> > >>>>> > >>>>> > org.apache.spark >>>>> > spark-streaming-twitter_2.10 >>>>> > 1.0.0 >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > src >>>>> > >>>>> > >>>>> > maven-compiler-plugin >>>>> > 3.3 >>>>> > >>>>> > 1.8 >>>>> > 1.8 >>>>> > >>>>> > >>>>> > >>>>> > maven-assembly-plugin >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > com.test.sparkTest.SimpleApp >>>>> > >>>>> > >>>>> > >>>>> > >>>>> jar-with-dependencies >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > The application starts successfully but no tweets comes and this >>>>> exception >>>>> > is thrown >>>>> > >>>>> > 15/11/08 15:55:46 WARN TaskSetManager: Lost task 0.0 in stage 4.0 >>>>> (TID 78, >>>>> > 192.168.122.39): java.io.IOException: >>>>> java.lang.ClassNotFoundException: >>>>> > org.apache.spark.streaming.twitter.TwitterReceiver >>>>> > at >>>>> org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1163) >>>>> > at >>>>> > >>>>> org.apache.spark.rdd.ParallelCollectionPartition.readObject(ParallelCollectionRDD.scala:70) >>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>> > at >>>>> > >>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>>> > at >>>>> > >>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>> > at java.lang.reflect.Method.invoke(Method.java:497) >>>>> > at >>>>> > >>>>> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) >>>>> > at >>>>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900) >>>>> > at >>>>> > >>>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >>>>> > at >>>>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >>>>> > at >>>>> > >>>>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >>>>> > at >>>>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >>>>> > at >>>>> > >>>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >>>>> > at >>>>> java.io.Objec
Re: java.lang.ClassNotFoundException: org.apache.spark.streaming.twitter.TwitterReceiver
>>>>>> > >>>>>> > } >>>>>> > >>>>>> > >>>>>> > here is the pom file >>>>>> > >>>>>> > http://maven.apache.org/POM/4.0.0; >>>>>> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >>>>>> > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 >>>>>> > http://maven.apache.org/xsd/maven-4.0.0.xsd;> >>>>>> > 4.0.0 >>>>>> > SparkFirstTry >>>>>> > SparkFirstTry >>>>>> > 0.0.1-SNAPSHOT >>>>>> > >>>>>> > >>>>>> > >>>>>> > org.apache.spark >>>>>> > spark-core_2.10 >>>>>> > 1.5.1 >>>>>> > provided >>>>>> > >>>>>> > >>>>>> > >>>>>> > org.apache.spark >>>>>> > spark-streaming_2.10 >>>>>> > 1.5.1 >>>>>> > provided >>>>>> > >>>>>> > >>>>>> > >>>>>> > org.twitter4j >>>>>> > twitter4j-stream >>>>>> > 3.0.3 >>>>>> > >>>>>> > >>>>>> > org.apache.spark >>>>>> > spark-streaming-twitter_2.10 >>>>>> > 1.0.0 >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > src >>>>>> > >>>>>> > >>>>>> > maven-compiler-plugin >>>>>> > 3.3 >>>>>> > >>>>>> > 1.8 >>>>>> > 1.8 >>>>>> > >>>>>> > >>>>>> > >>>>>> > maven-assembly-plugin >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > com.test.sparkTest.SimpleApp >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > jar-with-dependencies >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > The application starts successfully but no tweets comes and this >>>>>> > exception >>>>>> > is thrown >>>>>> > >>>>>> > 15/11/08 15:55:46 WARN TaskSetManager: Lost task 0.0 in stage 4.0 (TID >>>>>> > 78, >>>>>> > 192.168.122.39): java.io.IOException: java.lang.ClassNotFoundException: >>>>>> > org.apache.spark.streaming.twitter.TwitterReceiver >>>>>> > at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1163) >>>>>> > at >>>>>> > org.apache.spark.rdd.ParallelCollectionPartition.readObject(ParallelCollectionRDD.scala:70) >>>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>> > at >>>>>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>>>> > at >>>>>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>> > at java.lang.reflect.Method.invoke(Method.java:497) >>>>>> > at >>>>>> > java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) >>>>>> > at >>>>>> > java.io.ObjectInputStream.readSerialData
Re: java.lang.ClassNotFoundException: org.apache.spark.streaming.twitter.TwitterReceiver
If I packaged the application and submit it, it works fine but I need to
run it from eclipse.
Is there any problem running the application from eclipse ?
On 9 November 2015 at 12:27, Tathagata Das <t...@databricks.com> wrote:
> How are you submitting the spark application?
> You are supposed to submit the fat-jar of the application that include the
> spark-streaming-twitter dependency (and its subdeps) but not
> spark-streaming and spark-core.
>
> On Mon, Nov 9, 2015 at 1:02 AM, أنس الليثي <dev.fano...@gmail.com> wrote:
>
>> I tried to remove maven and adding the dependencies manually using build
>> path > configure build path > add external jars, then adding the jars
>> manually but it did not work.
>>
>> I tried to create another project and copied the code from the first app
>> but the problem still the same.
>>
>> I event tried to change eclipse with another version, but the same
>> problem exist.
>>
>> :( :( :( :(
>>
>> On 9 November 2015 at 10:47, أنس الليثي <dev.fano...@gmail.com> wrote:
>>
>>> I tried both, the same exception still thrown
>>>
>>> On 9 November 2015 at 10:45, Sean Owen <so...@cloudera.com> wrote:
>>>
>>>> You included a very old version of the Twitter jar - 1.0.0. Did you
>>>> mean 1.5.1?
>>>>
>>>> On Mon, Nov 9, 2015 at 7:36 AM, fanooos <dev.fano...@gmail.com> wrote:
>>>> > This is my first Spark Stream application. The setup is as following
>>>> >
>>>> > 3 nodes running a spark cluster. One master node and two slaves.
>>>> >
>>>> > The application is a simple java application streaming from twitter
>>>> and
>>>> > dependencies managed by maven.
>>>> >
>>>> > Here is the code of the application
>>>> >
>>>> > public class SimpleApp {
>>>> >
>>>> > public static void main(String[] args) {
>>>> >
>>>> > SparkConf conf = new SparkConf().setAppName("Simple
>>>> > Application").setMaster("spark://rethink-node01:7077");
>>>> >
>>>> > JavaStreamingContext sc = new JavaStreamingContext(conf, new
>>>> > Duration(1000));
>>>> >
>>>> > ConfigurationBuilder cb = new ConfigurationBuilder();
>>>> >
>>>> > cb.setDebugEnabled(true).setOAuthConsumerKey("ConsumerKey")
>>>> > .setOAuthConsumerSecret("ConsumerSecret")
>>>> > .setOAuthAccessToken("AccessToken")
>>>> > .setOAuthAccessTokenSecret("TokenSecret");
>>>> >
>>>> > OAuthAuthorization auth = new OAuthAuthorization(cb.build());
>>>> >
>>>> > JavaDStream tweets = TwitterUtils.createStream(sc,
>>>> auth);
>>>> >
>>>> > JavaDStream statuses = tweets.map(new
>>>> Function<Status,
>>>> > String>() {
>>>> > public String call(Status status) throws Exception {
>>>> > return status.getText();
>>>> > }
>>>> > });
>>>> >
>>>> > statuses.print();;
>>>> >
>>>> > sc.start();
>>>> >
>>>> > sc.awaitTermination();
>>>> >
>>>> > }
>>>> >
>>>> > }
>>>> >
>>>> >
>>>> > here is the pom file
>>>> >
>>>> > http://maven.apache.org/POM/4.0.0;
>>>> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>>>> > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
>>>> > http://maven.apache.org/xsd/maven-4.0.0.xsd;>
>>>> > 4.0.0
>>>> > SparkFirstTry
>>>> > SparkFirstTry
>>>> > 0.0.1-SNAPSHOT
>>>> >
>>>> >
>>>> >
>>>> > org.apache.spark
>>>> > spark-core_2.10
>>>> > 1.5.1
>>>> > provided
>>>> >
>>>> >
>>>> >
>>>> > org.apache.spark
>>>> > spark-streaming_2.10
>>>> >
Re: java.lang.ClassNotFoundException: org.apache.spark.streaming.twitter.TwitterReceiver
How are you submitting the spark application?
You are supposed to submit the fat-jar of the application that include the
spark-streaming-twitter dependency (and its subdeps) but not
spark-streaming and spark-core.
On Mon, Nov 9, 2015 at 1:02 AM, أنس الليثي <dev.fano...@gmail.com> wrote:
> I tried to remove maven and adding the dependencies manually using build
> path > configure build path > add external jars, then adding the jars
> manually but it did not work.
>
> I tried to create another project and copied the code from the first app
> but the problem still the same.
>
> I event tried to change eclipse with another version, but the same problem
> exist.
>
> :( :( :( :(
>
> On 9 November 2015 at 10:47, أنس الليثي <dev.fano...@gmail.com> wrote:
>
>> I tried both, the same exception still thrown
>>
>> On 9 November 2015 at 10:45, Sean Owen <so...@cloudera.com> wrote:
>>
>>> You included a very old version of the Twitter jar - 1.0.0. Did you mean
>>> 1.5.1?
>>>
>>> On Mon, Nov 9, 2015 at 7:36 AM, fanooos <dev.fano...@gmail.com> wrote:
>>> > This is my first Spark Stream application. The setup is as following
>>> >
>>> > 3 nodes running a spark cluster. One master node and two slaves.
>>> >
>>> > The application is a simple java application streaming from twitter and
>>> > dependencies managed by maven.
>>> >
>>> > Here is the code of the application
>>> >
>>> > public class SimpleApp {
>>> >
>>> > public static void main(String[] args) {
>>> >
>>> > SparkConf conf = new SparkConf().setAppName("Simple
>>> > Application").setMaster("spark://rethink-node01:7077");
>>> >
>>> > JavaStreamingContext sc = new JavaStreamingContext(conf, new
>>> > Duration(1000));
>>> >
>>> > ConfigurationBuilder cb = new ConfigurationBuilder();
>>> >
>>> > cb.setDebugEnabled(true).setOAuthConsumerKey("ConsumerKey")
>>> > .setOAuthConsumerSecret("ConsumerSecret")
>>> > .setOAuthAccessToken("AccessToken")
>>> > .setOAuthAccessTokenSecret("TokenSecret");
>>> >
>>> > OAuthAuthorization auth = new OAuthAuthorization(cb.build());
>>> >
>>> > JavaDStream tweets = TwitterUtils.createStream(sc,
>>> auth);
>>> >
>>> > JavaDStream statuses = tweets.map(new Function<Status,
>>> > String>() {
>>> > public String call(Status status) throws Exception {
>>> > return status.getText();
>>> > }
>>> > });
>>> >
>>> > statuses.print();;
>>> >
>>> > sc.start();
>>> >
>>> > sc.awaitTermination();
>>> >
>>> > }
>>> >
>>> > }
>>> >
>>> >
>>> > here is the pom file
>>> >
>>> > http://maven.apache.org/POM/4.0.0;
>>> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>>> > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
>>> > http://maven.apache.org/xsd/maven-4.0.0.xsd;>
>>> > 4.0.0
>>> > SparkFirstTry
>>> > SparkFirstTry
>>> > 0.0.1-SNAPSHOT
>>> >
>>> >
>>> >
>>> > org.apache.spark
>>> > spark-core_2.10
>>> > 1.5.1
>>> > provided
>>> >
>>> >
>>> >
>>> > org.apache.spark
>>> > spark-streaming_2.10
>>> > 1.5.1
>>> > provided
>>> >
>>> >
>>> >
>>> > org.twitter4j
>>> > twitter4j-stream
>>> > 3.0.3
>>> >
>>> >
>>> > org.apache.spark
>>> > spark-streaming-twitter_2.10
>>> > 1.0.0
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > src
>>> >
>>> >
Re: java.lang.ClassNotFoundException: org.apache.spark.streaming.twitter.TwitterReceiver
I tried to remove maven and adding the dependencies manually using build
path > configure build path > add external jars, then adding the jars
manually but it did not work.
I tried to create another project and copied the code from the first app
but the problem still the same.
I event tried to change eclipse with another version, but the same problem
exist.
:( :( :( :(
On 9 November 2015 at 10:47, أنس الليثي <dev.fano...@gmail.com> wrote:
> I tried both, the same exception still thrown
>
> On 9 November 2015 at 10:45, Sean Owen <so...@cloudera.com> wrote:
>
>> You included a very old version of the Twitter jar - 1.0.0. Did you mean
>> 1.5.1?
>>
>> On Mon, Nov 9, 2015 at 7:36 AM, fanooos <dev.fano...@gmail.com> wrote:
>> > This is my first Spark Stream application. The setup is as following
>> >
>> > 3 nodes running a spark cluster. One master node and two slaves.
>> >
>> > The application is a simple java application streaming from twitter and
>> > dependencies managed by maven.
>> >
>> > Here is the code of the application
>> >
>> > public class SimpleApp {
>> >
>> > public static void main(String[] args) {
>> >
>> > SparkConf conf = new SparkConf().setAppName("Simple
>> > Application").setMaster("spark://rethink-node01:7077");
>> >
>> > JavaStreamingContext sc = new JavaStreamingContext(conf, new
>> > Duration(1000));
>> >
>> > ConfigurationBuilder cb = new ConfigurationBuilder();
>> >
>> > cb.setDebugEnabled(true).setOAuthConsumerKey("ConsumerKey")
>> > .setOAuthConsumerSecret("ConsumerSecret")
>> > .setOAuthAccessToken("AccessToken")
>> > .setOAuthAccessTokenSecret("TokenSecret");
>> >
>> > OAuthAuthorization auth = new OAuthAuthorization(cb.build());
>> >
>> > JavaDStream tweets = TwitterUtils.createStream(sc,
>> auth);
>> >
>> > JavaDStream statuses = tweets.map(new Function<Status,
>> > String>() {
>> > public String call(Status status) throws Exception {
>> > return status.getText();
>> > }
>> > });
>> >
>> > statuses.print();;
>> >
>> > sc.start();
>> >
>> > sc.awaitTermination();
>> >
>> > }
>> >
>> > }
>> >
>> >
>> > here is the pom file
>> >
>> > http://maven.apache.org/POM/4.0.0;
>> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>> > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
>> > http://maven.apache.org/xsd/maven-4.0.0.xsd;>
>> > 4.0.0
>> > SparkFirstTry
>> > SparkFirstTry
>> > 0.0.1-SNAPSHOT
>> >
>> >
>> >
>> > org.apache.spark
>> > spark-core_2.10
>> > 1.5.1
>> > provided
>> >
>> >
>> >
>> > org.apache.spark
>> > spark-streaming_2.10
>> > 1.5.1
>> > provided
>> >
>> >
>> >
>> > org.twitter4j
>> > twitter4j-stream
>> > 3.0.3
>> >
>> >
>> > org.apache.spark
>> > spark-streaming-twitter_2.10
>> > 1.0.0
>> >
>> >
>> >
>> >
>> >
>> > src
>> >
>> >
>> > maven-compiler-plugin
>> > 3.3
>> >
>> > 1.8
>> > 1.8
>> >
>> >
>> >
>> > maven-assembly-plugin
>> >
>> >
>> >
>> >
>> > com.test.sparkTest.SimpleApp
>> >
>> >
>> >
>> >
>> jar-with-dependencies
>> >
>> >
>> >
>> >
>> >
>> >
java.lang.ClassNotFoundException: org.apache.spark.streaming.twitter.TwitterReceiver
This is my first Spark Stream application. The setup is as following
3 nodes running a spark cluster. One master node and two slaves.
The application is a simple java application streaming from twitter and
dependencies managed by maven.
Here is the code of the application
public class SimpleApp {
public static void main(String[] args) {
SparkConf conf = new SparkConf().setAppName("Simple
Application").setMaster("spark://rethink-node01:7077");
JavaStreamingContext sc = new JavaStreamingContext(conf, new
Duration(1000));
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(true).setOAuthConsumerKey("ConsumerKey")
.setOAuthConsumerSecret("ConsumerSecret")
.setOAuthAccessToken("AccessToken")
.setOAuthAccessTokenSecret("TokenSecret");
OAuthAuthorization auth = new OAuthAuthorization(cb.build());
JavaDStream tweets = TwitterUtils.createStream(sc, auth);
JavaDStream statuses = tweets.map(new Function<Status,
String>() {
public String call(Status status) throws Exception {
return status.getText();
}
});
statuses.print();;
sc.start();
sc.awaitTermination();
}
}
here is the pom file
http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
4.0.0
SparkFirstTry
SparkFirstTry
0.0.1-SNAPSHOT
org.apache.spark
spark-core_2.10
1.5.1
provided
org.apache.spark
spark-streaming_2.10
1.5.1
provided
org.twitter4j
twitter4j-stream
3.0.3
org.apache.spark
spark-streaming-twitter_2.10
1.0.0
src
maven-compiler-plugin
3.3
1.8
1.8
maven-assembly-plugin
com.test.sparkTest.SimpleApp
jar-with-dependencies
The application starts successfully but no tweets comes and this exception
is thrown
15/11/08 15:55:46 WARN TaskSetManager: Lost task 0.0 in stage 4.0 (TID 78,
192.168.122.39): java.io.IOException: java.lang.ClassNotFoundException:
org.apache.spark.streaming.twitter.TwitterReceiver
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1163)
at
org.apache.spark.rdd.ParallelCollectionPartition.readObject(ParallelCollectionRDD.scala:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
at
org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:72)
at
org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:98)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:194)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException:
org.apache.spark.streaming.twitter.TwitterReceiver
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at
org.apache.spark.se
Re: java.lang.ClassNotFoundException: org.apache.spark.streaming.twitter.TwitterReceiver
You included a very old version of the Twitter jar - 1.0.0. Did you mean 1.5.1?
On Mon, Nov 9, 2015 at 7:36 AM, fanooos <dev.fano...@gmail.com> wrote:
> This is my first Spark Stream application. The setup is as following
>
> 3 nodes running a spark cluster. One master node and two slaves.
>
> The application is a simple java application streaming from twitter and
> dependencies managed by maven.
>
> Here is the code of the application
>
> public class SimpleApp {
>
> public static void main(String[] args) {
>
> SparkConf conf = new SparkConf().setAppName("Simple
> Application").setMaster("spark://rethink-node01:7077");
>
> JavaStreamingContext sc = new JavaStreamingContext(conf, new
> Duration(1000));
>
> ConfigurationBuilder cb = new ConfigurationBuilder();
>
> cb.setDebugEnabled(true).setOAuthConsumerKey("ConsumerKey")
> .setOAuthConsumerSecret("ConsumerSecret")
> .setOAuthAccessToken("AccessToken")
> .setOAuthAccessTokenSecret("TokenSecret");
>
> OAuthAuthorization auth = new OAuthAuthorization(cb.build());
>
> JavaDStream tweets = TwitterUtils.createStream(sc, auth);
>
> JavaDStream statuses = tweets.map(new Function<Status,
> String>() {
> public String call(Status status) throws Exception {
> return status.getText();
> }
> });
>
> statuses.print();;
>
> sc.start();
>
> sc.awaitTermination();
>
> }
>
> }
>
>
> here is the pom file
>
> http://maven.apache.org/POM/4.0.0;
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> http://maven.apache.org/xsd/maven-4.0.0.xsd;>
> 4.0.0
> SparkFirstTry
> SparkFirstTry
> 0.0.1-SNAPSHOT
>
>
>
> org.apache.spark
> spark-core_2.10
> 1.5.1
> provided
>
>
>
> org.apache.spark
> spark-streaming_2.10
> 1.5.1
> provided
>
>
>
> org.twitter4j
> twitter4j-stream
> 3.0.3
>
>
> org.apache.spark
> spark-streaming-twitter_2.10
> 1.0.0
>
>
>
>
>
> src
>
>
> maven-compiler-plugin
> 3.3
>
> 1.8
> 1.8
>
>
>
> maven-assembly-plugin
>
>
>
>
> com.test.sparkTest.SimpleApp
>
>
>
> jar-with-dependencies
>
>
>
>
>
>
>
>
>
> The application starts successfully but no tweets comes and this exception
> is thrown
>
> 15/11/08 15:55:46 WARN TaskSetManager: Lost task 0.0 in stage 4.0 (TID 78,
> 192.168.122.39): java.io.IOException: java.lang.ClassNotFoundException:
> org.apache.spark.streaming.twitter.TwitterReceiver
> at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1163)
> at
> org.apache.spark.rdd.ParallelCollectionPartition.readObject(ParallelCollectionRDD.scala:70)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
> at
> org.apache.sp
