dejankrak-db opened a new pull request, #11877: URL: https://github.com/apache/arrow-rs/pull/11877
# Which issue does this PR close? - Closes #11876. # Rationale for this change #10109 added the Parquet `FILE` logical type, but not its Arrow mapping. A `FILE` group reads as a plain struct, and `ArrowWriter` cannot write the annotation, so Arrow-based writers cannot produce conforming `FILE` columns. # What changes are included in this PR? Following the Variant precedent: - A new `parquet::arrow::file_type::FileType` extension type, named `arrow.parquet.file`, with no metadata. Its storage is a struct whose fields are any subset of `uri`, `offset`, `size`, `content_type`, `checksum`, `inline`, each nullable, matching the Parquet-side validation from #10109 (string fields accept `Utf8`/`LargeUtf8`/`Utf8View`, `inline` accepts any binary type). - Read: `try_add_extension_type` / `has_extension_type` tag a `FILE` group with `FileType`. - Write: `logical_type_for_struct` returns `LogicalType::File` for a struct field with `FileType`; its gating now covers either `variant_experimental` or `file_experimental`. Sub-field ids come from the existing `PARQUET:field_id` metadata. - A `file_experimental` feature (also part of `experimental`), documented in the README and crate docs, and a CI compile check with `--no-default-features --features file_experimental`. # Are these changes tested? Yes, in `parquet/src/arrow/file_type.rs`: accepted and rejected struct shapes, an Arrow to Parquet to Arrow schema round trip (annotation and field ids), a struct without the extension staying unannotated, and `ArrowWriter` write and read round trips at the top level and as a list element. Locally: `cargo test -p parquet --all-features --lib` passes, `cargo clippy -p parquet --all-targets --all-features -- -D warnings` is clean, docs build with `-D warnings`, and `cargo check -p parquet` passes with default features, `variant_experimental` only, and `file_experimental` only. # Are there any user-facing changes? A new, experimental, opt-in `file_experimental` feature and `FileType` extension type. No change without the feature. The code, tests, and this description were drafted with an AI assistant and reviewed by me before submitting. This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
