mbutrovich opened a new issue, #11593:
URL: https://github.com/apache/arrow-rs/issues/11593
### Describe the bug
`zip` with two `StringViewArray` or `BinaryViewArray` scalars and a mask
that has both true and false rows can return a wrong falsy value. This happens
when the falsy value is short enough to be inlined in its view but its scalar
array still has data buffers, as a slice of a larger array does.
### To Reproduce
```rust
use arrow_array::{Array, BooleanArray, Scalar, StringViewArray};
use arrow_select::zip::zip;
fn main() {
// Both scalars are short, so inline, but each slice keeps its array's
data buffer
let t = StringViewArray::from(vec!["x", "a string longer than twelve
bytes"]).slice(0, 1);
let f = StringViewArray::from(vec!["hello", "another string longer than
twelve"]).slice(0, 1);
let out = zip(&BooleanArray::from(vec![true, false]), &Scalar::new(&t),
&Scalar::new(&f)).unwrap();
let out = out.as_any().downcast_ref::<StringViewArray>().unwrap();
println!("{:?}", out.iter().collect::<Vec<_>>());
// [Some("x"), Some("hellp")], expected [Some("x"), Some("hello")]
}
```
### Expected behavior
`[Some("x"), Some("hello")]`.
### Additional context
`ByteViewScalarImpl::get_views_for_non_nullable` decides whether the falsy
view is inlined by checking whether the falsy scalar has data buffers
([zip.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-select/src/zip.rs#L820-L830)).
If it has buffers, the view is read as a buffer reference and its buffer index
is rewritten. For an inlined value, view bytes 8 to 11 hold value bytes 4 to 7,
so a value of 5 or more bytes changes ("hello" becomes "hellp" here). For a
shorter value they hold [padding the format requires to be
zero](https://github.com/apache/arrow/blob/fcf8f3014227cafe24deeb044c2d51f5fc9902e5/docs/source/format/Columnar.rst?plain=1#L510-L512),
and `validate_full` then rejects the output. Checking the view's length (`<=
12`) instead of the buffers would avoid both.
The bug came in with #8963. Found with property-based tests that build
arrays in every physical layout the Arrow format allows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
