0.10 consumers won't work on an earlier broker. Earlier consumers will (should?) work on a 0.10 broker.
The main things earlier consumers lack from a user perspective is support for SSL, and pre-fetching messages. The implementation is totally and completely different however, in ways that leak to the end user. On Fri, Oct 7, 2016 at 3:15 PM, Reynold Xin <r...@databricks.com> wrote: > Does Kafka 0.10 work on a Kafka 0.8/0.9 cluster? > > > On Fri, Oct 7, 2016 at 1:14 PM, Jeremy Smith <jeremy.sm...@acorns.com> > wrote: >> >> +1 >> >> We're on CDH, and it will probably be a while before they support Kafka >> 0.10. At the same time, we don't use their Spark and we're looking forward >> to upgrading to 2.0.x and using structured streaming. >> >> I was just going to write our own Kafka Source implementation which uses >> the existing KafkaRDD but it would be much easier to get buy-in for an >> official Spark module. >> >> Jeremy >> >> On Fri, Oct 7, 2016 at 12:41 PM, Michael Armbrust <mich...@databricks.com> >> wrote: >>> >>> We recently merged support for Kafak 0.10.0 in Structured Streaming, but >>> I've been hearing a few people tell me that they are stuck on an older >>> version of Kafka and cannot upgrade. I'm considering revisiting >>> SPARK-17344, but it would be good to have more information. >>> >>> Could people please vote or comment on the above ticket if a lack of >>> support for older versions of kafka would block you from trying out >>> structured streaming? >>> >>> Thanks! >>> >>> Michael >> >> > --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
