MartijnVisser opened a new pull request, #29553: URL: https://github.com/apache/flink/pull/29553
Unchanged backport of https://github.com/apache/flink/pull/27157 (9b21a9716cc), which went to master, release-1.19, release-2.0 and release-2.1 but not to this branch. It is the release-1.19 commit bd916bd89e7 (RocksDB only, ForSt on 1.20 has no snapshot strategy) plus the hotfix 1e367509833 that moves the "Will re-use" log line to TRACE. When the completion notification of checkpoint N arrives after N+1 has started, N+1 re-uploads the sst files of N. Once N+1 completes the JobManager drops the handles of N, N+2 still references them, and the `SharedStateRegistry` rejects that acknowledgement with "Attempt to reference unknown state". The acknowledgement is lost and the checkpoint waits for its timeout. The fix removes every re-uploaded file from the base set. On release-1.20 this is what fails `WindowDistinctAggregateStressITCase` (FLINK-40663), 37 of 243 table jobs on the GitHub Actions nightlies and pushes since 2026-09-14. Last seen in https://github.com/apache/flink/actions/runs/39531020653/job/119010155405 (nightly 2026-10-21, `Java 17 / Test (module: table)`). FLINK-35499 recorded the same exception on 1.20 in July 2024. `testCheckpointIsIncrementalWithLateNotification` fails on this branch without the production change and passes with it, on JDK 8 and 17. The ITCase under CI's flags in a two-CPU container hit the exception in 1 of 30 runs on the base and in 0 of 89 with this change. The fix's prune trace is in 83 of those 89 logs. The rocksdb module tests, `EventTimeWindowCheckpointingITCase`, spotless, checkstyle and the planner ArchUnit tests pass. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Claude Fable 5.1) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
