nikhilred2619 opened a new pull request, #55391:
URL: https://github.com/apache/arrow/pull/55391

   ### Rationale for this change
   
   This PR addresses #55325 by allowing existing Parquet `FileMetaData` to be 
supplied when constructing a `ParquetFileFragment`.
   
   Currently, even when metadata is already available, fragments may need to 
access individual Parquet file footers. The new construction path allows 
callers to reuse that metadata directly, avoiding unnecessary file reads.
   
   ### What changes are included in this PR?
   
   - Extended `ParquetFileFormat::MakeFragment` to accept preloaded metadata.
   - Reused the existing `SetMetadata` implementation to initialize the 
fragment's metadata and schema information.
   - Added an optional `metadata` parameter to Python's `make_fragment()` API.
   - Implemented `FileMetaData.subset()` to select row groups from existing 
metadata.
   - Added bounds validation for invalid row-group indices.
   - Added C++ and Python regression tests covering metadata reuse, row-group 
selection, object lifetimes, and fragment construction without additional file 
reads.
   
   The implementation follows the construction-time approach discussed in the 
issue and preserves existing behavior when metadata is not provided.
   
   ### Are these changes tested?
   
   Development verification results:
   
   - C++ dataset tests: 64 passed.
   - C++ Parquet internal tests: 355 passed.
   - Python tests: 682 passed, 94 skipped.
   - Valgrind: 0 errors.
   - Formatting and lint checks passed.
   
   The patch was also verified to apply cleanly against the current upstream 
`main` branch. CI validation on the submitted branch is pending.
   
   ### Are there any user-facing changes?
   
   Yes. Python users can provide existing metadata during fragment creation and 
use `FileMetaData.subset()`.
   
   Existing API usage remains supported.
   
   Fixes #55325.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to