Hello,
I have 2 Kafka sources that are configured with a watermark strategy
instantiated like this:
WatermarkStrategy.<T>forBoundedOutOfOrderness(maxAllowedWatermarkDrift)
.withIdleness(idleTimeout) // 5 seconds currently
.withWatermarkAlignment(alignmentGroup,
maxAllowedWatermarkDrift, Duration.ofSeconds(1L))
The alignment group is the same for both, but each one consumes from a
different topic. During a test, I ensured that one of the topics didn't
receive any messages, but when I check the logs I see multiple entries like
this:
Distributing maxAllowedWatermark=1707149933770 of group=dispatcher to
subTaskIds=[0] for source Source: GenericChangeMessageDeserializer.
where maxAllowedWatermark grows all the time.
Maybe my understanding is wrong, but I think this means the source is never
marked as idle even though it didn't receive any new messages in the Kafka
topic?
Regards,
Alexis.
