Re: Error while Spark 1.6.1 streaming from Kafka-2.11_0.10.0.1 cluster

2016-09-23 Thread sagarcasual .
Hi, Thanks for the response, The issue I am facing is only for the clustered Kafka 2.11 based version 0.10.0.1 and Spark 1.6.1 with following dependencies. org.apache.spark:spark-core_2.10:1.6.1 compile group: 'org.apache.spark', name: 'spark-streaming_2.10', version:'1.6.1' compile group:

Re: Error while Spark 1.6.1 streaming from Kafka-2.11_0.10.0.1 cluster

2016-09-23 Thread Cody Koeninger
For Spark 2.0 there are two kafka artifacts, spark-streaming-kafka-0-10 (0.10 and later brokers only) and spark-streaming-kafka-0-8 (should work with 0.8 and later brokers). The docs explaining this were merged to master just after 2.0 released, so they haven't been published yet. There are usage

Fwd: Error while Spark 1.6.1 streaming from Kafka-2.11_0.10.0.1 cluster

2016-09-22 Thread sagarcasual .
Also you mentioned about streaming-kafka-0-10 connector, what connector is this, do you know the dependency ? I did not see mention of it in the documents For current Spark 1.6.1 to Kafka 0.10.0.1 standalone, the only dependencies I have are org.apache.spark:spark-core_2.10:1.6.1 compile group:

Re: Error while Spark 1.6.1 streaming from Kafka-2.11_0.10.0.1 cluster

2016-09-22 Thread Cody Koeninger
Do you have the ability to try using Spark 2.0 with the streaming-kafka-0-10 connector? I'd expect the 1.6.1 version to be compatible with kafka 0.10, but it would be good to rule that out. On Thu, Sep 22, 2016 at 1:37 PM, sagarcasual . wrote: > Hello, > > I am trying to

Error while Spark 1.6.1 streaming from Kafka-2.11_0.10.0.1 cluster

2016-09-22 Thread sagarcasual .
Hello, I am trying to stream data out of kafka cluster (2.11_0.10.0.1) using Spark 1.6.1 I am receiving following error, and I confirmed that Topic to which I am trying to connect exists with the data . Any idea what could be the case? kafka.common.UnknownTopicOrPartitionException at