mbutrovich opened a new issue, #11595:
URL: https://github.com/apache/arrow-rs/issues/11595
### Describe the bug
For a dense union with a single field whose type id is not 0,
`logical_nulls` reports no nulls, even where the child is null. `is_null`,
`is_not_null` and `logical_null_count` give wrong results as a consequence.
With type id 0 the same array is correct.
### To Reproduce
```rust
use std::sync::Arc;
use arrow_arith::boolean::is_null;
use arrow_array::{Array, ArrayRef, Int32Array, UnionArray};
use arrow_schema::{DataType, Field, UnionFields};
fn main() {
let fields = UnionFields::try_new([5], [Field::new("a", DataType::Int32,
true)]).unwrap();
let child: ArrayRef = Arc::new(Int32Array::from(vec![None, Some(1)]));
let union = UnionArray::try_new(fields, vec![5i8, 5].into(),
Some(vec![0, 1].into()), vec![child]).unwrap();
println!("logical_null_count = {}", union.logical_null_count()); // 0,
expected 1
println!("{:?}", is_null(&union).unwrap()); // [false, false], expected
[true, false]
}
```
### Expected behavior
`logical_null_count` is 1 and `is_null` is `[true, false]`.
### Additional context
The single-field path passes the child's nulls to `gather_nulls` under type
id 0
([union_array.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-array/src/array/union_array.rs#L797-L809)).
`gather_nulls` looks nulls up by each row's type id
([union_array.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-array/src/array/union_array.rs#L617-L626)),
so rows with type id 5 find nothing and read as valid. Using the field's own
type id would fix it.
Found with property-based tests that build arrays in every physical layout
the Arrow format allows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
