I haven't done Structured Streaming in Spark 2.1 with Kafka 0.9.x, but I did do stream processing with Spark 2.0.1 and Kafka 0.10.
Here's the official documenation that I used for Spark Streaming with Kafka 0.10: https://spark.apache.org/docs/2.1.0/streaming-kafka-integration.html. It looks like you might need to use the 0.8.x connector, but I found that connector to be inadequate for some use cases, such as using a regex to subscribe to multiple topics. Your results may vary, and I wouldn't be surprised if there was another source of official documentation on the topic. On Mon, May 15, 2017 at 5:25 AM Swapnil Chougule <the.swapni...@gmail.com> wrote: > Hello > > I am new to structured streaming. Wanted to learn if there is support for > Kafka 0.8.x or Kafka 0.9.x in structured streaming ? > My Kafka source is of version 0.9.x & want get have structured streaming > solution on top of it. > I checked documentation for Spark release 2.1.0 but didn't get exact info. > Any help is most welcome. Thanks in advance. > > --Swapnil >
