Re: SPIP: Support Kafka delegation token in Structured Streaming

Hi Saisai, The reasons why I've originally set the goal only structured streaming is the following: * Haven't seen big interest in the DStream area for new features * Separate the concerns even if there is a need All in all happy to port the feature to DStream if you think it worth and you can

Re: SPIP: Support Kafka delegation token in Structured Streaming

Hi Jungtaek, Thanks for your comments, just reacted on them. BR, G On Sat, Sep 29, 2018 at 2:50 PM Jungtaek Lim wrote: > Hi Gabor, > > Thanks for proposing the feature. I'm definitely interested to see this > feature, but honestly I'm not familiar with how Spark deals with delegation > token

Re: SPIP: Support Kafka delegation token in Structured Streaming

I like this proposal. Since Kafka already provides delegation token mechanism, we can also leverage Spark's delegation token framework to add Kafka as a built-in support. BTW I think there's no much difference in support structured streaming and DStream, maybe we can set both as goal. Thanks

Re: SPIP: Support Kafka delegation token in Structured Streaming

Hi Gabor, Thanks for proposing the feature. I'm definitely interested to see this feature, but honestly I'm not familiar with how Spark deals with delegation token for HDFS and HBase. I'll try to review the doc in general, and try to learn it, and review again based on understanding. Thanks,

SPIP: Support Kafka delegation token in Structured Streaming

Hi all, I am writing this e-mail in order to discuss the delegation token support for kafka feature which is reported in SPARK-25501 . I've prepared a SPIP