mbutrovich opened a new issue, #11686: URL: https://github.com/apache/arrow-rs/issues/11686
**Describe the bug** `MutableArrayData::extend` on sparse union `ArrayData` with a nonzero offset copies the children's values from the rows before the offset. The type ids come from the right rows, so the output pairs each type id with the wrong value. `make_array` of the same `ArrayData` reads the right rows since #11649. [`build_extend_sparse`](https://github.com/apache/arrow-rs/blob/3658fc01e8fed4179d476aa3efc34b2723f89dd3/arrow-data/src/transform/union.rs#L36-L52) reads the type ids through `array.buffer::<i8>(0)`, which applies the offset. It then extends each child over `start..start + len` without adding `array.offset()`, but a sparse union's children are not sliced by the parent's offset. This is the sparse union form of #11588. **To Reproduce** ```rust use std::sync::Arc; use arrow_array::{Int32Array, UnionArray, make_array}; use arrow_data::transform::MutableArrayData; use arrow_schema::{DataType, Field, UnionFields}; fn main() { let fields = UnionFields::try_new([0], [Field::new("a", DataType::Int32, true)]).unwrap(); let union = UnionArray::try_new( fields, vec![0i8, 0, 0].into(), None, vec![Arc::new(Int32Array::from(vec![10, 20, 30]))], ) .unwrap(); // Offset 1 on the parent, children left whole, as a C Data Interface // producer such as pyarrow exports a slice let data = union.to_data().into_builder().offset(1).len(2).build().unwrap(); println!("{:?}", make_array(data.clone())); // child values [20, 30] let mut m = MutableArrayData::new(vec![&data], false, 2); m.try_extend(0, 0, 2).unwrap(); println!("{:?}", make_array(m.freeze())); // child values [10, 20] } ``` **Expected behavior** The output holds the rows `make_array` reads from the same `ArrayData`: `[20, 30]`. **Additional context** This is not a duplicate of #11582. That bug was in `From<ArrayData> for UnionArray`, which `make_array` calls, and its fix in #11649 changed only `arrow-array/src/array/union_array.rs`. `MutableArrayData` copies from `ArrayData` without building a `UnionArray`, so that fix does not reach it. Both bugs need the same input: sparse union `ArrayData` with a nonzero offset, which `ArrayData::slice` and C Data Interface producers create. Found by property-based tests on `main` at 3658fc01e8fed4179d476aa3efc34b2723f89dd3. Dense unions are not affected, because their offsets buffer gives each row's child index directly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
