neilconway opened a new issue, #11008:
URL: https://github.com/apache/arrow-rs/issues/11008
### Describe the bug
The Arrow spec says that "unselected" child values in sparse unions should
be ignored:
> Only the slot in the array corresponding to the type index is considered.
All "unselected" values are ignored and could be any semantically correct array
value.
However, arrow-rs's array equality for sparse unions (`ArrayData::eq`, and
therefore `PartialEq for dyn Array`) compares every child array over the full
slot range, regardless of which child each slot's type id selects. This is
inconsistent with the spec, as well as the Arrow C++ implementation.
### To Reproduce
```rust
use std::sync::Arc;
use arrow_array::{Array, ArrayRef, Int32Array, UnionArray};
use arrow_buffer::ScalarBuffer;
use arrow_schema::{DataType, Field, UnionFields};
fn sparse(b: Int32Array) -> UnionArray {
let fields = UnionFields::try_new(
[0, 1],
[
Field::new("a", DataType::Int32, true),
Field::new("b", DataType::Int32, true),
],
)
.unwrap();
// One slot, selecting child `a`, whose value is 7 on both sides.
UnionArray::try_new(
fields,
ScalarBuffer::from(vec![0i8]),
None,
vec![Arc::new(Int32Array::from(vec![7])) as ArrayRef, Arc::new(b)],
)
.unwrap()
}
let lhs = sparse(Int32Array::from(vec![Some(100)]));
let rhs = sparse(Int32Array::from(vec![Some(200)]));
assert_eq!(lhs.value(0).as_ref(), rhs.value(0).as_ref()); // logical values
agree
assert!(&lhs as &dyn Array == &rhs as &dyn Array); // fails
let rhs = sparse(Int32Array::from(vec![None]));
assert!(!lhs.is_null(0) && !rhs.is_null(0));
assert!(&lhs as &dyn Array == &rhs as &dyn Array); // also fails
```
### Expected behavior
Both assertions should pass.
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
