Re: Error while Spark 1.6.1 streaming from Kafka-2.11_0.10.0.1 cluster

2016-09-23 Thread sagarcasual .
Hi, Thanks for the response, The issue I am facing is only for the clustered Kafka 2.11 based version 0.10.0.1 and Spark 1.6.1 with following dependencies. org.apache.spark:spark-core_2.10:1.6.1 compile group: 'org.apache.spark', name: 'spark-streaming_2.10', version:'1.6.1' compile group: 'org.apa

Re: Error while Spark 1.6.1 streaming from Kafka-2.11_0.10.0.1 cluster

2016-09-23 Thread Cody Koeninger
For Spark 2.0 there are two kafka artifacts, spark-streaming-kafka-0-10 (0.10 and later brokers only) and spark-streaming-kafka-0-8 (should work with 0.8 and later brokers). The docs explaining this were merged to master just after 2.0 released, so they haven't been published yet. There are usage

Re: Error while Spark 1.6.1 streaming from Kafka-2.11_0.10.0.1 cluster

2016-09-22 Thread Cody Koeninger
Do you have the ability to try using Spark 2.0 with the streaming-kafka-0-10 connector? I'd expect the 1.6.1 version to be compatible with kafka 0.10, but it would be good to rule that out. On Thu, Sep 22, 2016 at 1:37 PM, sagarcasual . wrote: > Hello, > > I am trying to stream data out of kafka