Hi,
I agree with Jorge's assessment, we should allocate buffers with a certain
preferred alignment, but not actually require it when creating buffers from
raw pointers or in any computation kernels. Kernels currently do not seem
to rely on alignment, also simd implementations use unaligned loads.
Hi,
Thanks for the quick answers, and for the clarification wrt to the specs,
Antoine.
I believe that no-one had the goal of making an architecture-depent
requirement of aligned memory. A plausible explanation is that no-one tried
to create a mis-aligned buffer, since, in Rust, they are
Hi;
I have implemented Rust's specific memory alignments. If we want to enable
fixed alignment on all platforms, we can do it by a feature gate. By default,
it can come with the fixed alignment of 64; other users can enable
platform-specific alignments with feature gate.
Best,
Mahmut
On Sep
I don’t think we were ever explicit about the alignment being a requirement vs
recommendation with Rust. I think that to this point all the Rust contributors
were working exclusively in Rust. We likely added checks on alignment based on
this perspective.
This is all to say that I don’t think
By the way, half-relatedly, you may also be interested in the C data
interface at some point:
https://arrow.apache.org/docs/format/CDataInterface.html
We're also planning an experimental C stream interface above the C data
interface:
https://github.com/apache/arrow/pull/8052
Le 18/09/2020 à 18:03, Jorge Cardoso Leitão a écrit :
> // panics with "memory not aligned"
> Buffer::from_raw_parts(address as *const u8, size, size)
>
> I get an address such as `4624199872` (i64), but, when converted to `*const
> u8`, our rust implementation does not consider it to be memory
Hi,
I am trying to convert pyarrow buffers into Rust buffers and vice-versa, to
perform zero-copy from and to pyarrow, to and from Rust's library.
I was able to perform the operation rust -> pyarrow, using something along
the lines of
// 64 bits system
let pointer = buffer.raw_data() as i64;
