vntoinekaio opened a new issue, #11943:
URL: https://github.com/apache/arrow-rs/issues/11943
**Describe the bug**
`ArrayData::typed_buffer` checks that a buffer holds at least the bytes the
array needs, then calls `Buffer::typed_data::<T>()` on the whole buffer.
`typed_data` asserts that the buffer length is a multiple of `size_of::<T>()`,
so a buffer that is long enough but carries a few trailing bytes panics during
validation. `validate` itself only requires a minimum size for these buffers.
This is reachable from the IPC reader with validation enabled: changing the
length of a `Utf8` offsets buffer in a record batch's metadata from 24 to 66
bytes makes `FileReader` panic with `assertion failed: prefix.is_empty() &&
suffix.is_empty()` (`arrow-buffer/src/buffer/immutable.rs`). Arrow C++ (pyarrow
23.0.1) reads the same file, and `validate(full=True)` accepts it.
**To Reproduce**
```rust
let mut offsets = Buffer::from_slice_ref([0_i32, 2, 5]).as_slice().to_vec();
offsets.extend_from_slice(&[0, 0]); // 14 bytes
let data = ArrayData::builder(DataType::Utf8)
.len(2)
.add_buffer(Buffer::from_slice_ref(&offsets))
.add_buffer(Buffer::from_slice_ref(b"hello"))
.build(); // panics in Buffer::typed_data
```
**Expected behavior**
The array validates (only the first 12 bytes of the offsets buffer are
used), as in Arrow C++. A misaligned buffer is reported as an error, not a
panic.
**Additional context**
Present from 55.2.0 to 60.0.0 and on `main`. Related but distinct: #7124 /
#11044 / #11658 (short validity buffers) and #11857 / #11881 (out-of-bounds
buffer slices). Found by flipping single bytes of a small IPC file; this was
the only panic left that those do not cover.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
