connortsui20 opened a new issue, #8623: URL: https://github.com/apache/arrow-rs/issues/8623
Right now, there is no way to represent a non-nullable, "degenerate" `FixedSizeListArray` with arbitrary length. By degenerate, I mean that the `size` of each list scalar is equal to 0, meaning the `FixedSizeList` does not actually store any data. The `try_new` constructor will use the length of the null buffer to determine the length of the array in the degenerate case, but if the null buffer is `None` (indicating a non-nullable array), then the code incorrectly defaults to 0. https://github.com/apache/arrow-rs/blob/751b0822a7f0b2647c1c662131131b35f268bfef/arrow-array/src/array/fixed_size_list_array.rs#L155-L156 # Possible Solutions You cannot correctly handle this case by relying on the length of the null buffer, so I believe there are only 2 sane solutions. The first is to fix the `try_new` constructor and require it to take in a `len: usize`. Since a breaking change like this might not be acceptable, then a `try_new_with_length` constructor can be made that does take in a `len` parameter, and the existing `try_new` can call into `try_new_with_length` while documenting that if users want this specific degenerate and non-nullable case, they must use `try_new_with_length`. ### Related Issues - https://github.com/apache/arrow-rs/issues/5614 - https://github.com/apache/arrow-rs/issues/4623 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
