[
https://issues.apache.org/jira/browse/FLINK-32496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739276#comment-17739276
]
Rui Fan commented on FLINK-32496:
---------------------------------
{quote}A source will be marked idle if the source waits for alignment for a
long time. Is this a bug?
{quote}
Hi [~haishui] , from the current code, this's expected. Idle or Not Idle is a
general mechanism: when a source does not update the watermark for more than
idleTimeout due to any reason, it will be considered as idle. Of course, the
case of watermark alignment is also included.
However watermark alignment is a special case. If we find that watermark
alignment causes source idle to cause some behavioral bugs. Then we may need to
rethink whether it should not be marked as idle during watermark alignment? Of
course , we need to see if there are other solutions for this kind of bug.
WDYT?
> Sources with idleness and alignment always wait for alignment when part of
> multiple sources is idle
> ---------------------------------------------------------------------------------------------------
>
> Key: FLINK-32496
> URL: https://issues.apache.org/jira/browse/FLINK-32496
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
> Affects Versions: 1.16.2, 1.17.1
> Reporter: haishui
> Assignee: Rui Fan
> Priority: Major
> Labels: pull-request-available
>
> Sources with idleness and alignment always wait for alignment when part of
> multiple sources is idle.
> *Root cause:*
> In
> [SourceOperator|https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/SourceOperator.java],
> `lastEmittedWatermark` is Long.MAX_VALUE if a source is idle.
> When other source is active, the `currentMaxDesiredWatermark` is less then
> Long.MAX_VALUE.
> So the `shouldWaitForAlignment` method is always true for idle sources.
>
> What's more, the source will become idle if a source wait for alignment for a
> long time, which also should be considered.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
