cshuo opened a new pull request, #19973:
URL: https://github.com/apache/hudi/pull/19973
### Describe the issue this Pull Request addresses
`IndexWriteFunction` ignores the result of retrying a buffer write after
flushing. An index record that exceeds the entire buffer capacity can therefore
be silently dropped. Its cleanup also assumes initialization completed and
stops at the first exception, potentially skipping the memory pool or write
client.
Related to #19664.
### Summary and Changelog
- Check the retry result and fail explicitly when a single index record
cannot fit in the buffer.
- Remove the `MemoryPagesExhaustedException` catch from record writes; this
exception is raised when creating a buffer.
- Handle partially initialized resources and continue closing remaining
resources after a cleanup failure, preserving subsequent exceptions as
suppressed exceptions.
- Add regression coverage for oversized index records, partial
initialization, cleanup failures, and managed memory release.
### Impact
Oversized index records now fail the task instead of being silently dropped.
Resource cleanup completes as far as possible when initialization or closing
fails. No public API or configuration changes.
### Risk Level
Low. The changes affect buffer-write failure handling and resource cleanup;
normal writes and checkpoint behavior are preserved.
Validated with JDK 17 and Flink 2.2: 7 tests passed (0 failures/errors):
```bash
mvn -Pflink2.2 -pl hudi-flink-datasource/hudi-flink -am \
'-Dtest=TestIndexWriteFunction,TestWriteCopyOnWrite#testIndexWriteFunctionWithSmallBuffer+testIndexWriteFunctionWithCheckpoint'
\
-Dsurefire.failIfNoSpecifiedTests=false \
-DskipITs -DskipSparkTests -DskipScalaTests test
```
### Documentation Update
None. This fixes error handling without introducing configuration or API
changes.
### Contributor's checklist
- [ ] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
