Hi!
I'm beginning with the development in Spark Streaming.. And I'm learning
with the examples available in the spark directory. There are several
applications and I want to make modifications.
I can execute the TwitterPopularTags normally with command:
./bin/run-example TwitterPopularTags <auth>
So,
I moved the source code to a separate folder with the structure:
./src/main/scala/
With the files:
-TwitterPopularTags
-TwitterUtils
-StreamingExamples
-TwitterInputDStream
But when I run the command:
./bin/spark-submit --class "TwitterPopularTags" --master local[4]
/<MY_DIR>/TwitterTest/target/scala-2.10/simple-project_2.10-1.0.jar <auth>
I receive the following error:
Exception in thread "main" java.lang.NoClassDefFoundError:
twitter4j/auth/Authorization
at TwitterUtils$.createStream(TwitterUtils.scala:42)
at TwitterPopularTags$.main(TwitterPopularTags.scala:65)
at TwitterPopularTags.main(TwitterPopularTags.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:303)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:55)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: twitter4j.auth.Authorization
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 10 more
This is my sbt build file:
name := "Simple Project"
version := "1.0"
scalaVersion := "2.10.4"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.0.2"
libraryDependencies += "org.apache.spark" %% "spark-streaming" % "1.0.2"
libraryDependencies += "org.twitter4j" % "twitter4j-core" % "3.0.3"
libraryDependencies += "org.twitter4j" % "twitter4j-stream" % "3.0.3"
resolvers += "Akka Repository" at "http://repo.akka.io/releases/";
Can anybody help me?
Thanks a lot!
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Twitter-Example-Error-tp12600.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
