cshuo opened a new pull request, #20242: URL: https://github.com/apache/hudi/pull/20242
### Describe the issue this Pull Request addresses When a Flink bulk insert task is cancelled or fails before `endInput()`, `BulkInsertWriteFunction.close()` only closes the write client. The writer helper owns separate file handles, so their writers and associated resources can remain open. ### Summary and Changelog - Close the writer helper during function cleanup, including cancellation and failure paths. - Close the write client in a `finally` block so helper cleanup failures do not skip client cleanup. - Add lifecycle regression tests for partial initialization, cancellation, write failures, checked and runtime cleanup exceptions, and cleanup after normal `endInput()` without closing file handles twice. ### Impact Releases bulk insert writer resources when tasks exit before completing their input. Cleanup does not send a completion event to the coordinator. No configuration or storage format changes. The function's `close()` override now declares `IOException`, consistent with the inherited lifecycle contract. ### Risk Level Low. The change uses the existing helper cleanup method, which clears successfully closed handles and makes subsequent cleanup after `endInput()` a no-op for those handles. Validation with Java 11 and Flink 1.20: - Before the fix, 5 of the 8 new lifecycle test cases failed. - After the fix, all 26 cases in `TestBulkInsertWriteFunction` and `TestBulkInsertWriteHelper` passed. - Maven Checkstyle and `git diff --check` passed. ```bash mvn -pl hudi-flink-datasource/hudi-flink -am -Pflink1.20 -Dtest=TestBulkInsertWriteFunction,TestBulkInsertWriteHelper -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false -DskipITs test ``` ### Documentation Update None. ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
