Hi! This seems to be a real bug with the local-global aggregation and how it handles idle source subtasks. I've created FLINK-41254 [1] to track this, I will open a PR soon. This is not a new bug, it's been present since the introduction of this operator. In short, the local aggregation operator will never flush its buffer if its source subtask goes idle.
Best regards, Mate [1] https://issues.apache.org/jira/browse/FLINK-41254 On Wed, Oct 28, 2026 at 12:47 PM 陈雨铨(靖愉) via user <[email protected]> wrote: > > Hi Flink community, > We observed a potential correctness issue with two-phase event-time window > aggregation in a managed deployment reporting Flink version 2.2.1. > Configuration: > - Kafka source with 2 partitions; source and aggregation parallelism both > set to 2. > - 10-second TUMBLE windows. > - Watermark: event_time - INTERVAL '3' SECOND. > - table.exec.source.idle-timeout = '15 s'. > - table.optimizer.agg-phase-strategy = 'TWO_PHASE'. > - The execution plan contains LocalWindowAggregate and > GlobalWindowAggregate. > Reproduction: > 1. Send four records with the same grouping key into the same window: > three records to partition P0, each with amount 10, and one record to P1 > with amount 101. > 2. Stop sending to P1. Continue sending records with advancing event > timestamps to P0, using a separate grouping key. > 3. Allow P1 to become idle and observe the target window result. > Expected: COUNT = 4, SUM = 131. > Actual: COUNT = 3, SUM = 30. > The missing contribution is already absent before P1 resumes. Switching to > ONE_PHASE produces the correct result. A TWO_PHASE control run also > produced the correct result when a checkpoint occurred after the P1 record > arrived but before P1 became idle. > This suggests an interaction between idleness, watermark propagation, and > two-phase aggregation, but we have not confirmed the exact root cause. > Is this a known issue, and is there an existing fix we should evaluate? I > can share the SQL, input sequence, and additional test results. > I have been unable to complete Jira account registration, so I am > reporting this here first. > Thanks, > Yuquan Chen > >
