yongster opened a new issue, #10990:
URL: https://github.com/apache/arrow-rs/issues/10990
### Describe the bug
`nullif(left, mask)` should return `null` at positions where `mask` is
`true`.
The current generic implementation adds a top-level null bitmap to the input
`ArrayData`. This works for ordinary arrays, but it does not work for
`RunEndEncoded` or `Union` arrays:
- Run-end encoded arrays derive logical nulls from their `values` child
array.
- Union arrays derive logical nulls from the selected child value.
- Neither representation uses a top-level null bitmap to represent logical
nulls.
As a result, the null bitmap written by `nullif` is ignored when the result
is converted back to `RunArray` or `UnionArray`. The operation succeeds but
silently returns non-null values where nulls were requested.
### To Reproduce
```rust
use arrow_array::{BooleanArray, Int16Array, Int32Array, RunArray};
use arrow_array::types::Int16Type;
use arrow_select::nullif::nullif;
let ree = RunArray::<Int16Type>::try_new(
&Int16Array::from(vec![1, 2]),
&Int32Array::from(vec![10, 20]),
)?;
let mask = BooleanArray::from(vec![Some(false), Some(true)]);
let result = nullif(&ree, &mask)?;
### Expected behavior
[10, null]
### Additional context
[10, 20]
The same issue occurs for UnionArray: a top-level null bitmap is added, but
it is ignored by the Union logical-null representation.
Possible direction
nullif should dispatch explicitly for these types instead of using the
generic top-level null bitmap path.
- For RunEndEncoded, one possible implementation is to convert mask == true
positions into nullable take indices and reuse the Run-End Encoded take
implementation.
- For Union, nulls must be represented by selecting a nullable child and
writing a null value into that child.
- If a Union has no nullable child field, we need to define whether the
kernel should return an error or widen the output field nullability.
Related
- #10909
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
