etseidl opened a new issue, #11221:
URL: https://github.com/apache/arrow-rs/issues/11221
### Describe the bug
`ByteArrayDecoderPlain::read` can read fewer values than requested if its
input buffer is exhausted early on an exact record boundary. The relevant
(edited for brevity) code is
```rust
let mut read = 0;
let buf = self.buf.as_ref();
while self.offset < self.buf.len() && read != to_read {
if self.offset + 4 > buf.len() {
return Err(ParquetError::EOF("eof decoding byte
array".into()));
}
...
read += 1;
}
self.max_remaining_values -= to_read;
Ok(to_read)
```
Within the loop a check is made for a premature EOF, but if the buffer ends
on an exact boundary, the loop will terminate, but we still decrement by and
return `to_read` rather than `read`.
### To Reproduce
A unit test which fails on main:
```rust
#[test]
fn test_plain_decoder_reports_values_actually_read() {
// The page claims to contain two values, but its buffer contains
only
// one complete PLAIN-encoded BYTE_ARRAY value.
let buffer = Bytes::from_static(&[3, 0, 0, 0, b'f', b'o', b'o']);
let mut decoder = ByteArrayDecoderPlain::new(buffer, 2, Some(2),
false);
let mut output = OffsetBuffer::<i32>::with_capacity(2);
assert_eq!(decoder.read(&mut output, 2).unwrap(), 1);
assert_eq!(output.values.as_slice(), b"foo");
assert_eq!(output.offsets.as_slice(), &[0, 3]);
assert_eq!(decoder.max_remaining_values, 1);
}
```
### Expected behavior
`read` should either report the actual number of values read, or return the
same EOF error in this scenario.
### Additional context
This was discovered by Codex during a review of #10420.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
