pchintar opened a new pull request, #9778:
URL: https://github.com/apache/arrow-rs/pull/9778
# Which issue does this PR close?
- Closes #9777 .
# Rationale for this change
In `arrow-ipc/src/reader.rs`, buffers are currently allocated with
`MutableBuffer::from_len_zeroed(len)` and then passed to `read_exact`, which
fully overwrites the entire buffer contents.
This results in an unnecessary full memory pass for all data bytes:
* first zero-initialization
* then complete overwrite by the incoming data
Since `read_exact` guarantees that the provided slice is fully written on
success, the initial zeroing step is redundant and can be safely avoided by
allocating with capacity and setting the length before the read.
# What changes are included in this PR?
### Affected locations
* `read_block` (file reader path)
* `MessageReader::maybe_next` (stream reader path, message body buffer)
### Before/Current Approach
```rust
let mut buf = MutableBuffer::from_len_zeroed(len);
reader.read_exact(&mut buf)?;
```
### After Approach
```rust
let mut buf = MutableBuffer::with_capacity(len);
unsafe { buf.set_len(len) };
reader.read_exact(buf.as_slice_mut())?;
```
### Rationale
`read_exact` fully overwrites the provided slice on success, so the initial
zero-fill is redundant.
This change removes one full memory write pass per read without altering
behavior.
## Safety
The `unsafe set_len` usage is sound because:
* the buffer is not accessed between `set_len` and `read_exact`
* `read_exact` either fully initializes the buffer or returns an error
* on error, the buffer is not used and is dropped immediately
* no partially initialized data is exposed to safe Rust
Crash or interruption scenarios (panic, early return, process termination)
do not introduce unsoundness, as the buffer is never observed unless
`read_exact` completes successfully.
# Are these changes tested?
Yes the changes were tested successfully by running:
```bash
cargo test -p arrow-ipc
cargo fmt --all
cargo clippy --all-targets --all-features -- -D warnings
```
# Are there any user-facing changes?
No, there are no changes made to any public api code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
