Re: Structured Streaming with Kafka source/sink

[Ted Yu]

Please see this thread:

http://search-hadoop.com/m/q3RTt9XAz651PiG/Adhoc+queries+spark+streaming=Re+Adhoc+queries+on+Spark+2+0+with+Structured+Streaming

> On May 11, 2016, at 1:47 AM, Ofir Manor  wrote:
> 
> Hi,
> I'm trying out Structured Streaming from current 2.0 branch.
> Does the branch currently support Kafka as either source or sink? I couldn't 
> find a specific JIRA or design doc for that in SPARK-8360 or in the 
> examples... Is it still targeted for 2.0?
> 
> Also, I naively assume it will look similar to hdfs or JDBC stream("path"), 
> where the path will be some sort of Kafka URI (maybe protocol + broker list + 
> topic list). Is that the current thinking?
> 
> Thanks,
> Ofir Manor
> 
> Co-Founder & CTO | Equalum
> 
> Mobile: +972-54-7801286 | Email: ofir.ma...@equalum.io

Structured Streaming with Kafka source/sink

[Ofir Manor]

Hi,
I'm trying out Structured Streaming from current 2.0 branch.
Does the branch currently support Kafka as either source or sink? I
couldn't find a specific JIRA or design doc for that in SPARK-8360 or in
the examples... Is it still targeted for 2.0?

Also, I naively assume it will look similar to hdfs or JDBC *stream("path")*,
where the path will be some sort of Kafka URI (maybe protocol + broker list
+ topic list). Is that the current thinking?

Thanks,

Ofir Manor

Co-Founder & CTO | Equalum

Mobile: +972-54-7801286 | Email: ofir.ma...@equalum.io