0.9 brokers should be backwards compatible with 0.8 simple consumer, which is what the direct stream uses. If you're finding any problems, please report them.
There's work underway to add 0.9 consumer support to spark, at https://issues.apache.org/jira/browse/SPARK-12177 On Wed, Feb 3, 2016 at 9:56 AM, Pavel Sýkora <pav...@seznam.cz> wrote: > Hi, > > According to the Spark docs, Spark Streaming 1.5 (and 1.6) is compatible > with Kafka 0.8.2.1 (Direct Kafka API). Nevertheless, I need to use Kafka > 0.9.0 with Spark 1.5.x streaming. > > I tried to use Kafka 0.9.0 as both source and output of Spark 1.5 > Streaming, but it seems it works well. > > Does anybody have the same experience? Or does anybody have some problems > integrating Spark Streaming 1.5 or 1.6 with Kafka 0.9.0? > > Thanks in advance, > > Pavel >