I using kafka_2.10-1.1.0.jar on spark 1.1.0 — Sent from Mailbox
On Wed, Oct 29, 2014 at 12:31 AM, null <har...@nexgate.com> wrote: > Thanks! How do I find out which Kafka jar to use for scala 2.10.4? > — > Sent from Mailbox > On Wed, Oct 29, 2014 at 12:26 AM, Akhil Das <ak...@sigmoidanalytics.com> > wrote: >> Looks like the kafka jar that you are using isn't compatible with your >> scala version. >> Thanks >> Best Regards >> On Wed, Oct 29, 2014 at 11:50 AM, Harold Nguyen <har...@nexgate.com> wrote: >>> Hi, >>> >>> Just wondering if you've seen the following error when reading from Kafka: >>> >>> ERROR ReceiverTracker: Deregistered receiver for stream 0: Error starting >>> receiver 0 - java.lang.NoClassDefFoundError: scala/reflect/ClassManifest >>> at kafka.utils.Log4jController$.<init>(Log4jController.scala:29) >>> at kafka.utils.Log4jController$.<clinit>(Log4jController.scala) >>> at kafka.utils.Logging$class.$init$(Logging.scala:29) >>> at kafka.utils.VerifiableProperties.<init>(VerifiableProperties.scala:24) >>> at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:78) >>> at >>> org.apache.spark.streaming.kafka.KafkaReceiver.onStart(KafkaInputDStream.scala:96) >>> at >>> org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:121) >>> at >>> org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:106) >>> at >>> org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverLauncher$$anonfun$9.apply(ReceiverTracker.scala:264) >>> at >>> org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverLauncher$$anonfun$9.apply(ReceiverTracker.scala:257) >>> at >>> org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1121) >>> at >>> org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1121) >>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:62) >>> at org.apache.spark.scheduler.Task.run(Task.scala:54) >>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> at java.lang.Thread.run(Thread.java:744) >>> Caused by: java.lang.ClassNotFoundException: scala.reflect.ClassManifest >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) >>> ... 18 more >>> >>> Thanks, >>> >>> Harold >>>
