Re: Spark Kafka adapter questions

2018-08-20 Thread Ted Yu
93-driver-0 > > 18/08/20 22:29:34 INFO AbstractCoordinator: Discovered coordinator FQDN>:9093 (id: 2147483647 rack: null) for group > spark-kafka-source-1aa50598-99d1-4c53-a73c-fa6637a219b2--1338794993-driver-0. > > > > > > Also, I’m not sure if it’s relevant but I am running on Datab

RE: Spark Kafka adapter questions

2018-08-20 Thread Basil Hariri
github master on 8/8/18 and Kafka v2.0. Thanks so much for your help, let me know if there’s anything else I can provide Sincerely, Basil From: Ted Yu Sent: Friday, August 17, 2018 4:20 PM To: basil.har...@microsoft.com.invalid Cc: dev Subject: Re: Spark Kafka adapter questions If you have

Re: Spark Kafka adapter questions

2018-08-17 Thread Ted Yu
If you have picked up all the changes for SPARK-18057, the Kafka “broker” supporting v1.0+ should be compatible with Spark's Kafka adapter. Can you post more details about the “failed to send SSL close message” errors ? (The default Kafka version is 2.0.0 in Spark Kafka adapter after SPARK-18057

Spark Kafka adapter questions

2018-08-17 Thread Basil Hariri
Hi all, I work on Azure Event Hubs (Microsoft's PaaS offering similar to Apache Kafka) and am trying to get our new Kafka head to play nice with Spark's Kafka adapter. The goal is for our Kafka