Re: Watermark generation in Temporal Table Join

2020-03-22 Thread Dominik Wosiński
Hey, generally, that's what I thought more or less. I think I understand the behavior itself, thanks for explaining it to me. But what actually concerns me is the fact that this *assignTimestampsAndWatermarks* is required if You will select this Long field, which basically means that the type of s

Re: Watermark generation in Temporal Table Join

2020-03-22 Thread Kurt Young
Hi, after looking at your code, i think i might find the root cause. The reason is the additional `AssignerWithPeriodicWatermarks` [1] you added in the long version. Since the temporal table join could only get joined result of [3000, 6500, 8500], so the watermark this operator would generate will

Re: Watermark generation in Temporal Table Join

2020-03-19 Thread Dominik Wosiński
Hey, Sure, I have create something that can be called a minimal reproducible example. It's not the prettiest since it uses a lot of *Thread.sleep* but it allows to be sure that the input is exactly what you want. https://github.com/DomWos/FlinkTTF/tree/long-vs-timestamp In the long-vs-timestamp br

Re: Watermark generation in Temporal Table Join

2020-03-18 Thread Kurt Young
Hi, AFAIK there is no special watermark generation logic for temporal table join operator. Could you share your example's codes then I can help to analyze and debug? Best, Kurt On Tue, Mar 17, 2020 at 9:53 PM Dominik Wosiński wrote: > Hey Guys, > I have observed a weird behavior on using the

Watermark generation in Temporal Table Join

2020-03-17 Thread Dominik Wosiński
Hey Guys, I have observed a weird behavior on using the Temporal Table Join and the way it pushes the Watermark forward. Generally, I think the question is *When is the Watermark pushed forward by the Temporal Table Join?* The issue I have noticed is that Watermark seems to be pushed forward even