pchintar commented on code in PR #9778:
URL: https://github.com/apache/arrow-rs/pull/9778#discussion_r3117710120
##########
arrow-ipc/src/reader.rs:
##########
@@ -875,8 +875,12 @@ fn read_block<R: Read + Seek>(mut reader: R, block:
&Block) -> Result<Buffer, Ar
let metadata_len = block.metaDataLength().to_usize().unwrap();
let total_len = body_len.checked_add(metadata_len).unwrap();
- let mut buf = MutableBuffer::from_len_zeroed(total_len);
- reader.read_exact(&mut buf)?;
+ let mut buf = MutableBuffer::with_capacity(total_len);
+ // Buffer is immediately fully initialized by `read_exact` before any read
occurs
Review Comment:
Thanks for the reply @jhorstmann , so I dug into this a bit more and agree
the `set_len + read_exact` approach isn’t suitable for generic `Read`.
While looking into alternatives, I came across the newer `read_buf` API
using `BorrowedBuf`, which is designed specifically for reading into
uninitialized memory safely. It seems like it would let us avoid the zero-fill
while still preserving alignment via `MutableBuffer`.
Something like:
```rust
use std::io::{Read, BorrowedBuf};
let mut buf = MutableBuffer::with_capacity(total_len);
{
let spare = buf.spare_capacity_mut();
let mut borrowed = BorrowedBuf::from(spare);
reader.read_buf_exact(borrowed)?;
}
unsafe { buf.set_len(total_len) };
```
Would you be okay with taking this approach here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
