scovich opened a new pull request, #8438: URL: https://github.com/apache/arrow-rs/pull/8438
# Which issue does this PR close? Closes - https://github.com/apache/arrow-rs/issues/8435 - https://github.com/apache/arrow-rs/issues/8420 # Rationale for this change It turns out we were too permissive in our handling of `typed_value` columns and certain other exceptional cases that parquet's variant integration tests specifically expect readers to reject. # What changes are included in this PR? * Simplify `VariantArray::value` to work directly with (optional) `value` and `typed_value` columns instead of the `ShreddingState` enum * Hijack `rewrite_to_view_types` to also reject illegal column types * Fix several broken tests * Remove several illegal tests (that were exercising invalid shredding scenarios) # Are these changes tested? Yes. # Are there any user-facing changes? Behavior change: We no longer tolerate invalid-type `typed_value` columns when reading shredded variant data. At least, not in code paths that go through `VariantArray::value`. There may still be some leakage in the shredded path step handling of `variant_get`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
