viirya commented on PR #2883: URL: https://github.com/apache/arrow-rs/pull/2883#issuecomment-1279889842
> Because the allocation containing all the buffers is aligned to at least 8 bytes, and all the contained buffers are padded to a multiple of 8 bytes in length, each buffer starts and ends at an 8 byte boundary. Currently padding only guarantees 8 bytes alignment. Any larger alignment requirement can just fail the check. I'm not sure if the official doc explicitly asks for a 8 bytes alignment. For example, > Implementations are recommended to allocate memory on aligned addresses (multiple of 8- or 64-bytes) and pad (overallocate) to a length that is a multiple of 8 or 64 bytes. and for IPC, > The body, a flat sequence of memory buffers written end-to-end with appropriate padding to ensure a minimum of 8-byte alignment It sounds like it can be any alignment larger than 8 bytes. Maybe we can change to 16 bytes alignment? > As mentioned on the ticket, the issue appears to be that arm requires 16-byte alignment for i128 types, which isn't guaranteed by the standard which only mandates padding up to 8 bytes. As such we will need to copy the buffer to a new correctly aligned allocation in such a case. We could/should probably do this in general where the allocation is not sufficiently aligned. Hmm, for now this sounds like a special case (DecimalArray + arm). I'm okay for the copying approach. I will modify this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
