Re: Stream Join With Early firings

2018-06-23 Thread Johannes Schulte
Thanks Fabian! This seems to be the way to go On Tue, Jun 19, 2018 at 12:18 PM Fabian Hueske wrote: > Hi Johannes, > > You are right. You should approach the problem with the semantics that you > need before thinking about optimizations such as state size. > > The Table API / SQL offers (in

Re: Stream Join With Early firings

2018-06-19 Thread Fabian Hueske
Hi Johannes, You are right. You should approach the problem with the semantics that you need before thinking about optimizations such as state size. The Table API / SQL offers (in v1.5.0) two types of joins: 1) Windowed joins where each record joins with records in a time-range of the other

Re: Stream Join With Early firings

2018-06-18 Thread Johannes Schulte
Hi Fabian, thanks for the hints, though I somehow got the feeling that I am on the wrong track given how much code I would need to write for implementing a "blueprint" usecase. Would a join be more simple using the Table API? In the end it's the classical Order & OrderPosition example, where the

Re: Stream Join With Early firings

2018-06-18 Thread Fabian Hueske
Hi Johannes, EventTimeSessionWindows [1] use the EventTimeTrigger [2] as default trigger (see EventTimeSessionWindows.getDefaultTrigger()). I would take the EventTimeTrigger and extend it with early firing functionality. However, there are a few things to consider * you need to be aware that

Stream Join With Early firings

2018-06-13 Thread Johannes Schulte
Hi, I am joining two streams with a session window and want to emit a joined (early) result for every element arriving on one of the streams. Currently the code looks like this: s1.join(s2) .where(s1.id).equalTo(s2.id) .window(EventTimeSessionWindows.withGap(Time.minutes(15))) // trigger(?)