subject:"Which kafka client to use with spark streaming"

RE: Which kafka client to use with spark streaming

2017-12-26 Thread Serkan TAS

lt;serkan@enerjisa.com>; user <user@spark.apache.org> Subject: Re: Which kafka client to use with spark streaming Do not add a dependency on kafka-clients, the spark-streaming-kafka library has appropriate transitive dependencies. Either version of the spark-streaming-kafka library sh

Re: Which kafka client to use with spark streaming

2017-12-26 Thread Cody Koeninger

Do not add a dependency on kafka-clients, the spark-streaming-kafka library has appropriate transitive dependencies. Either version of the spark-streaming-kafka library should work with 1.0 brokers; what problems were you having? On Mon, Dec 25, 2017 at 7:58 PM, Diogo Munaro Vieira

Re: Which kafka client to use with spark streaming

2017-12-25 Thread Diogo Munaro Vieira

Hey Serkan, it depends of your Kafka version... Is it 0.8.2? Em 25 de dez de 2017 06:17, "Serkan TAS" escreveu: > Hi, > > > > Working on spark 2.2.0 cluster and 1.0 kafka brokers. > > > > I was using the library > > "org.apache.spark" % "spark-streaming-kafka-0-10_2.11"

Which kafka client to use with spark streaming

2017-12-25 Thread Serkan TAS

Hi, Working on spark 2.2.0 cluster and 1.0 kafka brokers. I was using the library "org.apache.spark" % "spark-streaming-kafka-0-10_2.11" % "2.2.0" and had lots of problems during streaming process then downgraded to "org.apache.spark" % "spark-streaming-kafka-0-8_2.11" % "2.2.0"