sdf-jkl commented on code in PR #9791:
URL: https://github.com/apache/arrow-rs/pull/9791#discussion_r3182714893
##########
parquet-variant-compute/src/unshred_variant.rs:
##########
@@ -175,12 +174,17 @@ impl<'a> UnshredVariantRowBuilder<'a> {
}
}
- /// Creates a new UnshredVariantRowBuilder from shredding state
- /// Returns None for None/None case - caller decides how to handle based
on context
- fn try_new_opt(shredding_state: BorrowedShreddingState<'a>) ->
Result<Option<Self>> {
- let value = shredding_state.value_field();
- let typed_value = shredding_state.typed_value_field();
- let Some(typed_value) = typed_value else {
+ /// Creates a new UnshredVariantRowBuilder from the `(value, typed_value)`
pair of a shredded
+ /// variant struct. Returns None for the None/None case - caller decides
how to handle based on
+ /// context.
+ fn try_new_opt(inner_struct: &'a StructArray) -> Result<Option<Self>> {
Review Comment:
1. Nothing serious. I thought the `BorrowedShreddingState` redundant and
wanted to remove it as dead code.
2. We tried this in #8354. There we follow the `variant_get` path and
extract the shredding state. For Struct arrays we can use
`BorrowedShreddingState`, but for List arrays, we write a new array using
`take` kernel. We can't use `BorrowedShreddingState` here as reference outlives
the value.
3. `PrimitiveArray` is only cloned in `ShreddingState` and `unshred_variant`
builder. `StructArray` is cloned in: `VariantArray::try_new`,
`ShreddedVariantFieldArray::try_new` and `canonicalize_shredded_types`. We make
the situation a little worse for `ShreddingState` by introducing one extra
`ArrayRef` Arc bump per present field. Same for `PrimitiveArray`s for unshred
builders.
4. I think a good alternative is Arc'ing the inner `StructArray`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
