Hi Tathagata, I actually have a separate question. What's the usage of lib_managed folder inside spark source folder ? Are those the library required for spark streaming to run ? Do they needed to be added to spark classpath when starting sparking cluster?
Weide On Sat, May 3, 2014 at 7:08 PM, Weide Zhang <[email protected]> wrote: > Hi Tathagata, > > I figured out the reason. I was adding a wrong kafka lib along side with > the version spark uses. Sorry for spamming. > > Weide > > > On Sat, May 3, 2014 at 7:04 PM, Tathagata Das <[email protected] > > wrote: > >> I am a little confused about the version of Spark you are using. Are you >> using Spark 0.9.1 that uses scala 2.10.3 ? >> >> TD >> >> >> On Sat, May 3, 2014 at 6:16 PM, Weide Zhang <[email protected]> wrote: >> >>> Hi I'm trying to run the kafka-word-count example in spark2.9.1. I >>> encountered some exception when initialize kafka consumer/producer config. >>> I'm using scala 2.10.3 and used maven build inside spark streaming kafka >>> library comes with spark2.9.1. Any one see this exception before? >>> >>> Thanks, >>> >>> producer: >>> [java] The args attribute is deprecated. Please use nested arg >>> elements. >>> [java] Exception in thread "main" java.lang.NoClassDefFoundError: >>> scala/Tuple2$mcLL$sp >>> [java] at >>> kafka.producer.ProducerConfig.<init>(ProducerConfig.scala:56) >>> [java] at >>> com.turn.apache.KafkaWordCountProducer$.main(HelloWorld.scala:89) >>> [java] at >>> com.turn.apache.KafkaWordCountProducer.main(HelloWorld.scala) >>> [java] Caused by: java.lang.ClassNotFoundException: >>> scala.Tuple2$mcLL$sp >>> [java] at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>> [java] at java.security.AccessController.doPrivileged(Native >>> Method) >>> [java] at >>> java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>> [java] at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>> [java] at >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>> [java] at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>> [java] ... 3 more >>> [java] Java Result: 1 >>> >>> >> >
