1996fanrui opened a new pull request, #29065:
URL: https://github.com/apache/flink/pull/29065
## What is the purpose of the change
Spilled channel-state files produced during checkpointing during recovery
were released only when `drain()`
ran. Several fetch→drain abort paths never reach `drain` (`readInputData`
throws after a file was spilled; the
`requestPartitions` or trigger-install mail is dropped/throws;
`thenRunAsync(drain)` is rejected after the
`channelIOExecutor` shut down), so the produced `FetchedChannelState` was
never closed and its spill files
survived until TaskManager shutdown — accumulating across recovery-failure
loops on a long-lived pooled TM.
This gives the produced state a single owner that force-closes it on any
such abort.
## Brief change log
- [FLINK-40520] Force-close the produced `FetchedChannelState` on abort:
close it in a `finally` in `SequentialChannelStateReaderImpl.readInputData` for
a fetch-time throw, and register it with the task `cancelables` for the
fetch→drain cancel/fail window. Idempotent with the happy path.
## Verifying this change
This change added a test:
- `SequentialChannelStateReaderImplTest` asserts the spill files are
deleted when `readInputData` aborts after spilling at least one file (fails
without the fix).
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: yes (channel-state
recovery)
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [ ] Yes (please specify the tool below)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
