fanruan-tuco opened a new pull request, #55629:
URL: https://github.com/apache/arrow/pull/55629

   ### Rationale for this change
   
   When a dictionary-encoded column write fails — for example because the 
dictionary hash table cannot grow and the memory pool returns OOM — the 
writer's encoder can be left pointing at freed memory. The file writer 
destructor still calls `ColumnWriterImpl::Close()`, which attempts to write the 
dictionary page, dereferences the freed allocation, and crashes with SIGSEGV 
instead of surfacing the original error.
   
   Expected behavior: a failed write must not cause subsequent 
close/destruction to access freed memory or crash the process.
   
   Fixes #55628.
   
   ### What changes are included in this PR?
   
   - Track a `failed_` flag on `ColumnWriterImpl`.
   - Mark the writer failed around all three write entry points (`WriteBatch`, 
`WriteBatchSpaced`, `WriteArrow`). Both thrown exceptions and non-OK `Status` 
returns leave the flag set.
   - `Close()` refuses to finalize the writer (dictionaries, pages, metadata) 
once a write has failed.
   - Writes to an already closed or failed writer are rejected, so a caller 
cannot recover a writer whose encoder state is no longer trustworthy.
   
   ### Are these changes tested?
   
   Yes. The new regression tests inject allocation failures with a 
`ProxyMemoryPool` that fails allocations at or above a threshold when the 
dictionary hash table grows. Coverage includes:
   
   - `WriteBatch`, `WriteBatchSpaced`, and `WriteArrow`;
   - explicit `Close()` versus destruction without closing;
   - memory pressure persisting versus recovering after the failure;
   - a non-OK `Status` return (rather than an exception) from `WriteArrow`.
   
   That is 13 new cases. The column writer suite passes on a Debug build (316 
tests; one Snappy test skipped, two pre-existing tests disabled).
   
   ### Are there any user-facing changes?
   
   No public API changes. After a failed write, the writer now reports the 
failure on `Close()` and during destruction instead of crashing.
   
   **This PR contains a "Critical Fix".** It fixes a crash (SIGSEGV) during 
writer destruction after an out-of-memory dictionary write. The crash is a 
use-after-free: `HashTable::Upsize()` releases the old entries buffer before 
the replacement allocation succeeds, while `entries_` keeps pointing at it, and 
`Close()` then reads through that dangling pointer.
   
   ### Was AI used for this PR?
   
   In accordance to the [AI generation 
guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code),
 please disclose below whether and how AI was used in this PR.
   
   **PR code and description written by:**
   
   - [ ] Human
   - [x] AI
   
   **Reviewed before submission by:**
   
   - [x] Human
   - [ ] AI
   - [ ] Not reviewed
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to