Hi; I have implemented Rust's specific memory alignments. If we want to enable fixed alignment on all platforms, we can do it by a feature gate. By default, it can come with the fixed alignment of 64; other users can enable platform-specific alignments with feature gate.
Best, Mahmut On Sep 18, 2020, 19:28, at 19:28, Patrick Horan <paddy.ho...@hey.com.invalid> wrote: >I don’t think we were ever explicit about the alignment being a >requirement vs recommendation with Rust. I think that to this point >all the Rust contributors were working exclusively in Rust. We likely >added checks on alignment based on this perspective. > >This is all to say that I don’t think there are any objections to >removing these checks. We certainly want to enable these use cases. > >Paddy >Antoine Pitrou <anto...@python.org> wrote: >“By the way, half-relatedly, you may also be interested in the C data >interface at some point: >https://arrow.apache.org/docs/format/CDataInterface.html > >We're also planning an experimental C stream interface above the C >data >interface: >https://github.com/apache/arrow/pull/8052 >https://github.com/apache/arrow/blob/b71d9652c8f41764ea5c0036c720cea786ee564a/docs/source/format/CStreamInterface.rst > >Regards > >Antoine. > > >Le 18/09/2020 à 18:03, Jorge Cardoso Leitão a écrit : >> Hi, >> >> I am trying to convert pyarrow buffers into Rust buffers and >vice-versa, to >> perform zero-copy from and to pyarrow, to and from Rust's library. >> >> I was able to perform the operation rust -> pyarrow, using something >along >> the lines of >> >> // 64 bits system >> let pointer = buffer.raw_data() as i64; >> pyarrow.call1("foreign_buffer >> ><https://arrow.apache.org/docs/python/generated/pyarrow.foreign_buffer.html>", >> (pointer, buffer.len())) >> >> However, I am not being able to perform the opposite transformation, >due to >> a memory misalignment check that is failing. I.e. >> >> let address = buffer.getattr("address")?.extract::<i64>()?; >> let size = buffer.getattr("size")?.extract::<i64>()? as usize; >> >> // panics with "memory not aligned" >> Buffer::from_raw_parts(address as *const u8, size, size) >> >> I get an address such as `4624199872` (i64), but, when converted to >`*const >> u8`, our rust implementation does not consider it to be memory >aligned (at >> least in x86_64) and panics. Has anyone worked in this problem? Any >hints >> on what I am doing wrong? >> >> AFAIK, our rust implementation requires different alignments, >depending on >> the architecture (see memory.rs >> ><https://github.com/apache/arrow/blob/master/rust/arrow/src/memory.rs>). >> Wasn't this supposed to be a single alignment, consistent throughout >the >> arrow? I am confused. >> >> Any help would be greatly appreciated! >> >> Best, >> Jorges >>”
