Hello,
I'm in the process of updating from Flink 1.11.3 to 1.14.3, and it seems the
interval join in my pipeline is no longer working. More specifically, I have a
sliding window after the interval join, and the window isn't firing. After many
tests, I ended up creating a custom operator that extends IntervalJoinOperator
and I overrode processWatermark1() and processWatermark2() to add logs and
check when they are called. I can see that processWatermark1() isn't called.
For completeness, this is how I use my custom operator:
joinOperator = new CustomIntervalJoinOperator(...);
stream1.connect(stream2)
.keyBy(selector1, selector2)
.transform("Interval Join", TypeInformation.of(Pojo.class), joinOperator);
---
Some more information in case it's relevant:
- stream2 is obtained from a side output.
- both stream1 and stream2 have watermarks assigned by custom strategies. I
also log watermark creation, and I can see that watermarks are indeed emitted
as expected in both streams.
Strangely, my watermark strategies mark themselves idle if they don't receive
new events after 10 minutes, and if I send some events and wait 10 minutes,
processWatermark1() is called! On the other hand, if I continuously send
events, it is never called.
Is this a known issue?
Regards,
Alexis.
