fanruan-tuco opened a new pull request, #55629: URL: https://github.com/apache/arrow/pull/55629
### Rationale for this change When a dictionary-encoded column write fails — for example because the dictionary hash table cannot grow and the memory pool returns OOM — the writer's encoder can be left pointing at freed memory. The file writer destructor still calls `ColumnWriterImpl::Close()`, which attempts to write the dictionary page, dereferences the freed allocation, and crashes with SIGSEGV instead of surfacing the original error. Expected behavior: a failed write must not cause subsequent close/destruction to access freed memory or crash the process. Fixes #55628. ### What changes are included in this PR? - Track a `failed_` flag on `ColumnWriterImpl`. - Mark the writer failed around all three write entry points (`WriteBatch`, `WriteBatchSpaced`, `WriteArrow`). Both thrown exceptions and non-OK `Status` returns leave the flag set. - `Close()` refuses to finalize the writer (dictionaries, pages, metadata) once a write has failed. - Writes to an already closed or failed writer are rejected, so a caller cannot recover a writer whose encoder state is no longer trustworthy. ### Are these changes tested? Yes. The new regression tests inject allocation failures with a `ProxyMemoryPool` that fails allocations at or above a threshold when the dictionary hash table grows. Coverage includes: - `WriteBatch`, `WriteBatchSpaced`, and `WriteArrow`; - explicit `Close()` versus destruction without closing; - memory pressure persisting versus recovering after the failure; - a non-OK `Status` return (rather than an exception) from `WriteArrow`. That is 13 new cases. The column writer suite passes on a Debug build (316 tests; one Snappy test skipped, two pre-existing tests disabled). ### Are there any user-facing changes? No public API changes. After a failed write, the writer now reports the failure on `Close()` and during destruction instead of crashing. **This PR contains a "Critical Fix".** It fixes a crash (SIGSEGV) during writer destruction after an out-of-memory dictionary write. The crash is a use-after-free: `HashTable::Upsize()` releases the old entries buffer before the replacement allocation succeeds, while `entries_` keeps pointing at it, and `Close()` then reads through that dangling pointer. ### Was AI used for this PR? In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR. **PR code and description written by:** - [ ] Human - [x] AI **Reviewed before submission by:** - [x] Human - [ ] AI - [ ] Not reviewed 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
