alamb commented on code in PR #9971:
URL: https://github.com/apache/arrow-rs/pull/9971#discussion_r3248937925
##########
arrow-buffer/src/buffer/immutable.rs:
##########
@@ -84,6 +86,89 @@ pub struct Buffer {
length: usize,
}
+/// An aligned byte buffer that can be filled through `Read::read_exact` and
+/// converted into [`Buffer`] without copying.
+///
+/// This is useful for readers that need Arrow buffer alignment without
+/// first zero-initializing the allocation.
+pub struct AlignedVec {
+ ptr: NonNull<u8>,
+ len: usize,
+ layout: Layout,
+}
+
+impl AlignedVec {
+ /// Allocates `len` bytes with the requested alignment.
+ pub fn new(len: usize, align: usize) -> Self {
+ let layout =
+ Layout::from_size_align(len, align).expect("failed to create
layout for AlignedVec");
+
+ let ptr = match layout.size() {
+ 0 => dangling_ptr(),
+ _ => {
+ // Safety: `layout` has non-zero size and was constructed
above.
+ let raw_ptr = unsafe { std::alloc::alloc(layout) };
+ NonNull::new(raw_ptr).unwrap_or_else(||
handle_alloc_error(layout))
+ }
+ };
+
+ Self { ptr, len, layout }
+ }
+}
+
+// Allows callers such as `Read::read_exact` to view the allocated region as
+// bytes after it has been filled.
+impl Deref for AlignedVec {
+ type Target = [u8];
+
+ fn deref(&self) -> &[u8] {
+ // Safety: `ptr` points to `len` bytes owned by this AlignedVec.
+ unsafe { std::slice::from_raw_parts(self.ptr.as_ptr(), self.len) }
Review Comment:
🤔
Taking a step back I think what we would have to do is to change the
structure of the reader from "read a Page and then convert it to the right kind
of buffer" so that any time the code needs to read a page we pass in the
resulting type<T>.
I think you partly did this in this PR, but for some reasons I don't fully
understand the `read_page` is still in terms of Vec<u8> rather than `Vec<T>`
One thing I did notice was that when the pages are compressed, they need to
be read as Vec<u8> (or at least the alignment isn't critiical as they get
decompressed again immediately)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
