Hi Kevin Just a heads up at the recent spark summit in S.F. There was a presentation about streaming in 2.0. They said that streaming was not going to production ready in 2.0.
I am not sure if the older 1.6.x version will be supported. My project will not be able to upgrade with streaming support. We also use kafka Andy From: Marco Mistroni <mmistr...@gmail.com> Date: Monday, July 25, 2016 at 2:33 AM To: kevin <kiss.kevin...@gmail.com> Cc: "user @spark" <user@spark.apache.org>, "dev.spark" <d...@spark.apache.org> Subject: Re: where I can find spark-streaming-kafka for spark2.0 > Hi Kevin > you should not need to rebuild everything. > Instead, i believe you should launch spark-submit by specifying the kafka jar > file in your --packages... i had to follow same when integrating spark > streaming with flume > > have you checked this link ? > https://spark.apache.org/docs/latest/streaming-kafka-integration.html > > > hth > > > > On Mon, Jul 25, 2016 at 10:20 AM, kevin <kiss.kevin...@gmail.com> wrote: >> I have compile it from source code >> >> 2016-07-25 12:05 GMT+08:00 kevin <kiss.kevin...@gmail.com>: >>> hi,all : >>> I try to run example org.apache.spark.examples.streaming.KafkaWordCount , I >>> got error : >>> Exception in thread "main" java.lang.NoClassDefFoundError: >>> org/apache/spark/streaming/kafka/KafkaUtils$ >>> at >>> org.apache.spark.examples.streaming.KafkaWordCount$.main(KafkaWordCount.scal >>> a:57) >>> at >>> org.apache.spark.examples.streaming.KafkaWordCount.main(KafkaWordCount.scala>>> ) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62>>> ) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl >>> .java:43) >>> at java.lang.reflect.Method.invoke(Method.java:498) >>> at >>> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$ru >>> nMain(SparkSubmit.scala:724) >>> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) >>> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) >>> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119) >>> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) >>> Caused by: java.lang.ClassNotFoundException: >>> org.apache.spark.streaming.kafka.KafkaUtils$ >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> ... 11 more >>> >>> so where I can find spark-streaming-kafka for spark2.0 >> >
