tustvold commented on code in PR #3276:
URL: https://github.com/apache/arrow-rs/pull/3276#discussion_r1042720804
##########
arrow/src/ffi.rs:
##########
@@ -578,7 +578,7 @@ unsafe fn create_buffer(
index: usize,
len: usize,
) -> Option<Buffer> {
- if array.buffers.is_null() {
+ if array.buffers.is_null() || array.n_buffers == 0 {
Review Comment:
If I understand the motivation for this it is to accommodate arrays where no
null buffer is present.
Looking at the logic in `ArrowArrayRef::to_data` it does the following
* Treats buffer at offset 0 as the null buffer and blindly loads it
* Treats all other buffers as values buffers
I think this logic is just wrong, in particular I think it will end up
treating the first buffer of a UnionArray as a null buffer. I think what it
should do is lookup the layout like `FFI_ArrowArray::new` and use
`can_contain_null_mask` to determine if there should be a null mask present.
##########
arrow/src/ffi.rs:
##########
@@ -657,13 +657,20 @@ pub trait ArrowArrayRef {
let len = self.buffer_len(index)?;
- unsafe { create_buffer(self.owner().clone(), self.array(),
index, len) }
- .ok_or_else(|| {
- ArrowError::CDataInterface(format!(
- "The external buffer at position {} is null.",
- index - 1
- ))
- })
+ match unsafe {
+ create_buffer(self.owner().clone(), self.array(), index,
len)
+ } {
+ Some(buf) => Ok(buf),
+ None if len == 0 => {
+ // Null data buffer, which Rust doesn't allow. So
create
Review Comment:
:+1:
FWIW we do actually just use a null-like sentinel but it isn't null. This is
a bit questionable, if I have time I might revisit the use of `NonNull` in the
low-level alloc interfaces.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
