Re: How does Spark Structured Streaming determine an event has arrived late?

for not making this clear in the picture. In hindsight, the picture can be >> made much better. :) >> >> Hope this explanation helps! >> >> TD >> >> On Tue, Feb 27, 2018 at 2:26 AM, kant kodali <kanth...@gmail.com> wrote: >> >>> I re

Re: How does Spark Structured Streaming determine an event has arrived late?

ot making this clear in the picture. In hindsight, the picture can be > made much better. :) > > Hope this explanation helps! > > TD > > On Tue, Feb 27, 2018 at 2:26 AM, kant kodali <kanth...@gmail.com> wrote: > >> I read through the spark structured stre

Re: How does Spark Structured Streaming determine an event has arrived late?

"time" is actually "event-time". My apologies for not making this clear in the picture. In hindsight, the picture can be made much better. :) Hope this explanation helps! TD On Tue, Feb 27, 2018 at 2:26 AM, kant kodali <kanth...@gmail.com> wrote: > I read through the sp

How does Spark Structured Streaming determine an event has arrived late?

I read through the spark structured streaming documentation and I wonder how does spark structured streaming determine an event has arrived late? Does it compare the event-time with the processing time? [image: enter image description here] <https://i.stack.imgur.com/CXH4i.png> Taking the