Bardakor opened a new pull request, #10821: URL: https://github.com/apache/arrow-rs/pull/10821
# Which issue does this PR close? - Closes #10734. # Rationale for this change `SerializedPageReader` validates a page's compressed size against the bytes remaining in its column chunk, but previously only checked that `uncompressed_page_size` was non-negative. `decode_page` then used that untrusted value as a `Vec` capacity before decompression could detect a mismatch. Parquet's column metadata already supplies a structural bound: one page's uncompressed payload cannot exceed the column chunk's total uncompressed size, which includes every page and its header. Checking that invariant before decode rejects an inconsistent file without introducing a configurable limit for valid large pages. # What changes are included in this PR? - Retain the column chunk's declared uncompressed size in `SerializedPageReader`. - Validate each page's uncompressed size before decode in both the sequential and offset-index paths. - Populate required total-uncompressed-size metadata in two page-writer test fixtures that construct column metadata directly. # Are these changes tested? Yes. The regression constructs a tiny invalid Snappy page and verifies that both reader paths return `Invalid page header` before decompression. The compressed payload is intentionally invalid, so reaching the codec would produce a different error. Commands run: - `cargo test -p parquet` - `cargo clippy -p parquet --all-targets --all-features -- -D warnings` - `cargo +stable fmt --all -- --check` - `git diff --check` # Are there any user-facing changes? Malformed Parquet files whose page uncompressed size exceeds their column chunk total now return an error before allocating a decompression buffer. There is no public API change. # AI usage Codex assisted with tracing the reader paths, constructing the regression test, and drafting the change. I reviewed the Parquet metadata invariant, inspected every changed line, ran the checks above, and can debug and own the final patch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
