Re: Structured Streaming and Microbatches

2016-07-13 Thread Jacek Laskowski
Hi, It's still microbatching architecture with triggers as batchIntervals. It's just faster by default and the API is more pleasant, i.e. Dataset-driven. Jacek On 13 Jul 2016 10:35 p.m., "Matthias Niehoff" < matthias.nieh...@codecentric.de> wrote: Hi everybody, as far as I understand with new

Structured Streaming and Microbatches

2016-07-13 Thread Matthias Niehoff
Hi everybody, as far as I understand with new the structured Streaming API the output will not get processed every x seconds anymore. Instead the data will be processed as soon as is arrived. But there might be a delay due to processing time for the data. A small example: Data comes in and the