M-Tesla opened a new pull request, #11003: URL: https://github.com/apache/arrow-rs/pull/11003
# Which issue does this PR close? - Closes #10929. # Rationale for this change Content-defined chunking forces a data page break after every chunk except the last. Writing the chunk can already have flushed that page, when the chunk hits `data_page_size_limit` or `data_page_row_count_limit` exactly at the boundary. The forced `add_data_page` then flushes with nothing buffered. For BOOLEAN under `PARQUET_2_0` (RLE), that panics: `RLE value encoder is not initialized`. Other encodings do not panic, but they write a data page with zero values. `should_add_data_page`, `dict_fallback`, and `flush_data_pages` already skip the empty case. `add_data_page` did not. # What changes are included in this PR? `GenericColumnWriter::add_data_page` returns `Ok(())` when `num_buffered_values == 0`. No public API change. # Are these changes tested? - BOOLEAN, 500k rows, `data_page_size_limit(1024)` and CDC 8KiB/16KiB: write succeeds, no empty data pages, roundtrip row count matches - INT32, 500k rows, `data_page_row_count_limit(128)` and the same CDC options: no empty data pages `cargo test -p parquet --lib -- cdc_boolean_small_pages_does_not_panic` and `cdc_int32_row_count_limit_does_not_emit_empty_pages`. `cargo clippy -p parquet --lib -- -D warnings`. # Are there any user-facing changes? BOOLEAN CDC writes that previously panicked now succeed. Empty data pages are no longer emitted for the same forced-break case on other encodings. # AI Disclosure Assisted draft of the empty-buffer guard and regression tests. The skip matches the existing guards on `should_add_data_page`, `dict_fallback`, and `flush_data_pages`. Reviewed and verified with the checks above. Made with [Cursor](https://cursor.com) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
