mbutrovich opened a new issue, #11582:
URL: https://github.com/apache/arrow-rs/issues/11582
### Describe the bug
Converting sliced `ArrayData` of a sparse union back to an array returns
values from the wrong rows. `UnionArray::slice` is correct, and so are dense
unions.
### To Reproduce
```rust
use std::sync::Arc;
use arrow_array::{make_array, Array, ArrayRef, Int32Array, UnionArray};
use arrow_schema::{DataType, Field, UnionFields};
fn main() {
let fields = UnionFields::try_new([0], [Field::new("a", DataType::Int32,
true)]).unwrap();
let child: ArrayRef = Arc::new(Int32Array::from(vec![10, 20, 30]));
let union = UnionArray::try_new(fields, vec![0i8, 0, 0].into(), None,
vec![child]).unwrap();
let sliced = make_array(union.to_data().slice(1, 1));
let sliced = sliced.as_any().downcast_ref::<UnionArray>().unwrap();
println!("{:?}", sliced.value(0)); // [10], expected [20]
}
```
### Expected behavior
`make_array(union.to_data().slice(1, 1))` holds the union's second row,
`[20]`, as `union.slice(1, 1)` does.
### Additional context
`From<ArrayData> for UnionArray` slices the type ids by the data offset but
builds each child with `make_array(cd)`, unsliced
([union_array.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-array/src/array/union_array.rs#L689-L709)).
For a dense union the offsets buffer locates the child rows, so the offset is
accounted for. A sparse union's children are indexed by row
([Columnar.rst](https://github.com/apache/arrow/blob/fcf8f3014227cafe24deeb044c2d51f5fc9902e5/docs/source/format/Columnar.rst?plain=1#L942-L944)),
so they need the same slice.
Sliced `ArrayData` of a sparse union also arrives this way from
`ArrayData::slice` callers and from FFI producers that export an offset
([CDataInterface.rst](https://github.com/apache/arrow/blob/6b4c5248127881c506d4e21e6c95a5c4ced5d0d4/docs/source/format/CDataInterface.rst?plain=1#L456-L459)).
An IPC round trip of a sliced `List` of a sparse union fails, which may have
the same cause (not verified; see the IPC issue).
The unsliced children also reach `logical_nulls`. If the child above is
`[Some(10), None, Some(30)]`, `is_null` on the sliced array returns 3 rows for
a 1-row array, because the one-field path returns the child's nulls as they are
([union_array.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-array/src/array/union_array.rs#L797-L809)).
Found with property-based tests that build arrays in every physical layout
the Arrow format allows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
