MartijnVisser opened a new pull request, #28572:
URL: https://github.com/apache/flink/pull/28572
## What is the purpose of the change
Fixes a flaky failure in
`CommonExecSinkITCase#testStreamRecordTimestampInserterSinkRuntimeProvider`.
The test's `TestSource` reader returned `END_OF_INPUT` as soon as it had
emitted its rows. `SingleSplitEnumerator` assigns the split and then, via the
coordinator thread, delivers a `NoMoreSplitsEvent` as a separate event. When
the split-holding subtask finished before that event was delivered, the
coordinator sent it to an already-FINISHED task; the framework reports the
undeliverable event as a lost OperatorEvent and triggers a failover, which the
job's `NoRestartBackoffTimeStrategy` turns into a hard failure.
This is the first observed occurrence (build 76431, commit `cb17c87788a`);
the fix is justified on the structural race rather than on frequency. The
failing subtask is `(1/4)`, the split holder, matching the diagnosis. No data
is lost (all rows are emitted) — this is a benign test-config race, not a
coordinator bug. Proving stacktrace from the CI watchdog:
```
org.apache.flink.table.api.TableException: Failed to wait job finish
at
CommonExecSinkITCase.testStreamRecordTimestampInserterSinkRuntimeProvider(CommonExecSinkITCase.java:127)
Caused by: JobExecutionException: Job execution failed.
Caused by: org.apache.flink.runtime.JobException: Recovery is suppressed by
NoRestartBackoffTimeStrategy
Caused by: org.apache.flink.util.FlinkException: An OperatorEvent from an
OperatorCoordinator to a task was lost. Triggering task failover to ensure
consistency. Event: '[NoMoreSplitEvent]', targetTask: Source: T1[154] ->
WatermarkAssigner[155] -> StreamRecordTimestampInserter[156] -> T1[156]: Writer
(1/4) - execution #0
Caused by:
org.apache.flink.runtime.operators.coordination.TaskNotRunningException: Task
is not running, but in state FINISHED
```
## Brief change log
- The split-holding test reader now defers `END_OF_INPUT` until
`notifyNoMoreSplits()` has been received, removing the window in which it could
finish before the `NoMoreSplitsEvent` is delivered (mirroring how
`SourceReaderBase.finishedOrAvailableLater()` defers completion until
no-more-splits is set). No emitted data or assertion changed.
- After emitting, the reader busy-polls (returns `NOTHING_AVAILABLE` on an
already-complete `isAvailable()`) until the event lands; this is bounded and
not a regression, since the no-split readers in this test already behaved that
way.
## Verifying this change
This change is already covered by existing tests. Verified locally by
running `CommonExecSinkITCase` repeatedly (all 8 tests green).
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (Claude Opus 4.8 (1M context))
Generated-by: Claude Opus 4.8 (1M context)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
