Re: Late arriving events

2016-07-06 Thread Chen Qin
Jamie, Sorry for late reply, some of my thoughts inline. -Chen > > Another way to do this is to kick off a parallel job to do the backfill > from the previous savepoint without stopping the current "realtime" job. > This way you would not have to have a "blackout". This assumes your final >

Re: Late arriving events

2016-07-05 Thread Jamie Grier
I put a few comments in-line below... On Tue, Jul 5, 2016 at 4:06 PM, Chen Qin <qinnc...@gmail.com> wrote: > Hi there, > > I understand Flink currently doesn't support handling late arriving > events. In reality, a exact-once link job needs to deal data missing or > backf

Late arriving events

2016-07-05 Thread Chen Qin
Hi there, I understand Flink currently doesn't support handling late arriving events. In reality, a exact-once link job needs to deal data missing or backfill from time to time without rewind to previous save point, which implies restored job suffers blackout while it tried to catch up