Hi Matthias,
I think I understand the implications of idleness. In my case I really do
need it since even in the production environment one of the Kafka topics
will receive messages only sporadically.
With regards to the code, I have very limited understanding of Flink
internals, but that part I
Hi Alexis,
Yes, I guess so, while not utterly acquainted with that part of the code.
Apparently the SourceCoordinator cannot come up with a proper watermark time,
if watermarking is turned off (idle mode of stream), and then it deducts
watermark time from the remaining non-idle sources.
It’s
Hi Matthias,
thanks for looking at this. Would you then say this comment in the source
code is not really valid?
https://github.com/apache/flink/blob/release-1.18/flink-runtime/src/main/java/org/apache/flink/runtime/source/coordinator/SourceCoordinator.java#L181
That's where the log I was
Good morning Alexis,
withIdleness(…) is easily misunderstood, it actually means that the thus
configured stream is exempt from watermark processing after 5 seconds (in your
case).
Hence also watermark alignment is turned off for the stream until a new event
arrives.
.withIdleness(…) is good
Ah and I forgot to mention, this is with Flink 1.18.1
Am Mo., 5. Feb. 2024 um 18:00 Uhr schrieb Alexis Sarda-Espinosa <
sarda.espin...@gmail.com>:
> Hello,
>
> I have 2 Kafka sources that are configured with a watermark strategy
> instantiated like this:
>
>
