mbutrovich opened a new issue, #11608: URL: https://github.com/apache/arrow-rs/issues/11608
### Describe the bug `take` with a null index on a `StructArray` or `FixedSizeListArray` returns "Cannot take null indices from a RunEndEncoded array with a non-nullable values field" when a child is a run-end encoded array whose values field is not nullable. The same happens for a union child with no field that can hold a null. The parent's own validity can represent the null row, and the child's slot under it is unspecified, so the take can succeed. This is the follow-up agreed in #10994: https://github.com/apache/arrow-rs/pull/10994#issuecomment-5548430052 > for structs with non-nullable ree/union, currently we reject these because we try to take on them, even though we can mask them with the structs own null buffer (overly strict) ### To Reproduce ```rust use std::sync::Arc; use arrow_array::types::Int32Type; use arrow_array::{make_array, Array, ArrayRef, FixedSizeListArray, Int32Array, RunArray, StructArray}; use arrow_buffer::NullBuffer; use arrow_schema::{DataType, Field, Fields}; fn main() { // A run-end encoded array whose values field is not nullable let run = RunArray::<Int32Type>::try_new(&Int32Array::from(vec![1, 2]), &Int32Array::from(vec![7, 8])).unwrap(); let ree_type = DataType::RunEndEncoded( Arc::new(Field::new("run_ends", DataType::Int32, false)), Arc::new(Field::new("values", DataType::Int32, false)), ); let ree: ArrayRef = make_array(run.to_data().into_builder().data_type(ree_type.clone()).build().unwrap()); let indices = Int32Array::from(vec![Some(0), None]); let s = StructArray::new(Fields::from(vec![Field::new("f0", ree_type.clone(), false)]), vec![ree.clone()], Some(NullBuffer::new_valid(2))); println!("{:?}", arrow_select::take::take(&s, &indices, None).map(|a| a.null_count())); // Err(ComputeError("Cannot take null indices from a RunEndEncoded array with a non-nullable values field")) let l = FixedSizeListArray::new(Arc::new(Field::new("item", ree_type, false)), 1, ree, None); println!("{:?}", arrow_select::take::take(&l, &indices, None).map(|a| a.null_count())); // Err(ComputeError("Cannot take null indices from a RunEndEncoded array with a non-nullable values field")) } ``` ### Expected behavior `Ok(1)` for both: one null row, held in the parent's validity. ### Additional context [`field_can_represent_take_null`](https://github.com/apache/arrow-rs/blob/dd240116c75a15a2a048a041c808d67f3f32a375/arrow-select/src/take.rs#L480-L495) decides whether a run-end encoded or union array can hold a null. The struct and fixed-size list paths pass the null indices to their children unchanged, so the child's check fails even though the parent masks the row. The harness also saw the struct case with a union child. Found with property-based tests that build arrays in every physical layout the Arrow format allows, on `main` at `72887ed9`. The code is unchanged on `main` at `dd240116`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
