Re: time-windowed joins and tumbling windows

2020-03-25 Thread Vinod Mehra
Thanks Timo for the suggestion! Also apologies for missing your response last week. I will try to come up with a reproducible test case. On Wed, Mar 18, 2020 at 9:27 AM Timo Walther wrote: > Hi Vinod, > > thanks for answering my questions. The == Optimized Logical Plan == > looks as expected.

Re: time-windowed joins and tumbling windows

2020-03-18 Thread Timo Walther
Hi Vinod, thanks for answering my questions. The == Optimized Logical Plan == looks as expected. However, the == Physical Execution Plan == seems to be quite complex. Are you sure that watermarks don't get lost in some of those custom operators before entering the SQL part of the pipeline?

Re: time-windowed joins and tumbling windows

2020-03-13 Thread Vinod Mehra
I wanted to add that when I used the following the watermark was delayed by 3 hours instead of 2 hours that I would have expected: AND o.rowtime BETWEEN c.rowtime - INTERVAL '2' hour AND c.rowtime (time window constraint between o and c: 1st and 3rd table) Thanks, Vinod On Fri, Mar 13, 2020 at

Re: time-windowed joins and tumbling windows

2020-03-13 Thread Vinod Mehra
Thanks Timo for responding back! Answers below: > 1) Which planner are you using? We are using Flink 1.8 and using the default planner (org.apache.flink.table.calcite.FlinkPlannerImpl) from: org.apache.flink:flink-table-planner_2.11:1.8 > 2) How do you create your watermarks? We are using

Re: time-windowed joins and tumbling windows

2020-03-13 Thread Timo Walther
Hi Vinod, I cannot spot any problems in your SQL query. Some questions for clarification: 1) Which planner are you using? 2) How do you create your watermarks? 3) Did you unit test with only parallelism of 1 or higher? 4) Can you share the output of TableEnvironment.explain() with us?

time-windowed joins and tumbling windows

2020-03-10 Thread Vinod Mehra
Hi! We are testing the following 3 way time windowed join to keep the retained state size small. Using joins for the first time here. It works in unit tests but we are not able to get expected results in production. We are still troubleshooting this issue. Can you please help us review this in