0.9 brokers should be backwards compatible with 0.8 simple consumer, which
is what the direct stream uses. If you're finding any problems, please
report them.
There's work underway to add 0.9 consumer support to spark, at
https://issues.apache.org/jira/browse/SPARK-12177
On Wed, Feb 3, 2016 at 9:56 AM, Pavel Sýkora wrote:
> Hi,
>
> According to the Spark docs, Spark Streaming 1.5 (and 1.6) is compatible
> with Kafka 0.8.2.1 (Direct Kafka API). Nevertheless, I need to use Kafka
> 0.9.0 with Spark 1.5.x streaming.
>
> I tried to use Kafka 0.9.0 as both source and output of Spark 1.5
> Streaming, but it seems it works well.
>
> Does anybody have the same experience? Or does anybody have some problems
> integrating Spark Streaming 1.5 or 1.6 with Kafka 0.9.0?
>
> Thanks in advance,
>
> Pavel
>