Re: Connectors using new Kafka consumer API

Ok, I understand your point, thanks. Let me see what I can be done there. I may come back if it doesn't work out there:-) On Wed, Nov 9, 2016 at 9:25 AM, Cody Koeninger wrote: > Ok... in general it seems to me like effort would be better spent > trying to help upstream, as

Re: Connectors using new Kafka consumer API

Ok... in general it seems to me like effort would be better spent trying to help upstream, as opposed to us making a 5th slightly different interface to kafka (currently have 0.8 receiver, 0.8 dstream, 0.10 dstream, 0.10 structured stream) On Tue, Nov 8, 2016 at 10:05 PM, Mark Grover

Re: Connectors using new Kafka consumer API

Have you asked the assignee on the Kafka jira whether they'd be willing to accept help on it? On Tue, Nov 8, 2016 at 5:26 PM, Mark Grover wrote: > Hi all, > We currently have a new direct stream connector, thanks to work by Cody and > others on SPARK-12177. > > However, that

Connectors using new Kafka consumer API

Hi all, We currently have a new direct stream connector, thanks to work by Cody and others on SPARK-12177. However, that can't be used in secure clusters that require Kerberos authentication. That's because Kafka currently doesn't support delegation tokens (KAFKA-1696