sdf-jkl opened a new issue, #11938: URL: https://github.com/apache/arrow-rs/issues/11938
### Describe the bug `VariantType::try_new` and `VariantType::validate` accept any `DataType::Struct`, including structs with none of the required Variant fields. I ran into this while working on the Parquet Variant implementation for [DataFusion's canonical extension types issue, apache/datafusion#21144](https://github.com/apache/datafusion/issues/21144). The DataFusion wrapper should be able to delegate storage-schema validation to Arrow's extension type, as the other canonical extensions do. Currently, it needs additional validation through `VariantArray`. The [current implementation](https://github.com/apache/arrow-rs/blob/dc59495122884e4aced624e6744f268131472094/parquet-variant-compute/src/variant_array.rs#L151-L168) only checks `matches!(data_type, DataType::Struct(_))`. By comparison, [TimestampWithOffset](https://github.com/apache/arrow-rs/blob/dc59495122884e4aced624e6744f268131472094/arrow-schema/src/extension/canonical/timestamp_with_offset.rs#L90-L137) validates its child fields and their types and nullability. ### To Reproduce With `arrow-schema = "60.0.0"` and `parquet-variant-compute = "60.0.0"`: ```rust use arrow_schema::{DataType, Field}; use arrow_schema::extension::ExtensionType; use parquet_variant_compute::VariantType; let invalid = DataType::Struct( vec![Field::new("unrelated", DataType::Int32, false)].into(), ); // Both currently succeed, although this is not a Variant storage schema. assert!(VariantType::try_new(&invalid, "").is_ok()); assert!(VariantType::validate(&invalid, "").is_ok()); ``` ### Expected behavior Reject storage schemas that violate the [canonical Variant field/layout rules](https://arrow.apache.org/docs/format/CanonicalExtensions.html#parquet-variant), while accepting the permitted shredded and encoded layouts. Schema validation should be shared with the array constructor where applicable; validating the encoded values remains a separate concern. ### Additional context This was raised in the original extension PR: [#8392 review comment](https://github.com/apache/arrow-rs/pull/8392#discussion_r2367584252). The [reply](https://github.com/apache/arrow-rs/pull/8392#discussion_r2369544536) agreed and pointed to [#8387](https://github.com/apache/arrow-rs/issues/8387), which is now closed after adding binary storage-type support. The extension-type validation gap remains on `main` at `dc594951`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
