minxhe opened a new pull request, #29574:
URL: https://github.com/apache/flink/pull/29574
## What is the purpose of the change
In RocksDB/ForSt, an invalid `RocksIterator` (`isValid() == false`)
indicates either normal end-of-data or a read/I/O error; only
`RocksIterator#status()` distinguishes the two. In `flink-statebackend-forst`,
async `MapState` operations had two related iterator bugs:
1. **Silent read-error truncation:** `ForStDBIterRequest` (map iteration),
`ForStDBMapCheckRequest` (`asyncIsEmpty`), and `ForStDBBunchPutRequest`
(`clear`) treated every `!isValid()` as normal exhaustion without checking
`status()`, so a read error silently produced a truncated page, a false "empty"
result, or a partial clear.
2. **Leaked or cross-thread-closed iterators on failure:**
- `ForStIterateOperation` completed failed or skipped requests
exceptionally without closing their `ForStDBIterRequest`, leaking native
`RocksIterator` handles.
- `ForStStateExecutor#executeBatchRequests` closed every
`ForStDBIterRequest` in `iterRequests` on whichever thread failed the batch,
racing sibling worker threads still reading from their iterators or closing
iterators already handed off to completed pages.
This PR checks `status()` whenever an iterator becomes invalid, closes
failed/skipped iterate requests on their owning worker thread in
`ForStIterateOperation`, and removes the cross-thread close loop from
`ForStStateExecutor`.
Jira: https://issues.apache.org/jira/browse/FLINK-41094
## Brief change log
- Check `rocksIterator.status()` when `!rocksIterator.isValid()` in
`ForStDBIterRequest`, `ForStDBMapCheckRequest`, and `ForStDBBunchPutRequest`,
failing the request on `RocksDBException`.
- Close `ForStDBIterRequest` on its owning worker thread in
`ForStIterateOperation#failRequest` (attaching any close exception as
suppressed) on both the catch path and the skipped-continuation path.
- Remove the cross-thread `iterRequest.close()` loop in
`ForStStateExecutor#executeBatchRequests`.
- Add regression tests in `ForStDBIterateOperationTest` and
`ForStStateExecutorTest`.
## Verifying this change
This change added tests and can be verified as follows:
- On unmodified `master` (`8d01380ff5bf9892462ef9808b1b56fd9723f54d`), all
8 new test invocations across `ForStDBIterateOperationTest` and
`ForStStateExecutorTest` fail (leaked native iterator handles on worker
failure/skip, silent success on `RocksIterator#status()` errors, and
cross-thread iterator closure).
- With the fix, all 15 tests across `ForStDBIterateOperationTest` and
`ForStStateExecutorTest` pass.
```sh
./mvnw -B -ntp -pl flink-state-backends/flink-statebackend-forst
-Dflink.forkCountUnitTest=1 \
-Dtest=ForStDBIterateOperationTest,ForStStateExecutorTest test
# 15 run, 0 failures
./mvnw -B -ntp -pl flink-state-backends/flink-statebackend-forst \
spotless:check checkstyle:check
# 0 violations
```
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): yes (async
ForSt `MapState` iterator status check and failure cleanup)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: GitHub Copilot CLI 1.0.99-2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
