If the data buffers are still aligned, then I don't think we should add a breaking change just for avoiding the copy on the metadata? I'd expect said metadata to be small enough that zero-copy doesn't really affect performance.
François On Sun, Jun 30, 2019 at 4:01 AM Micah Kornfield <emkornfi...@gmail.com> wrote: > > While working on trying to fix undefined behavior for unaligned memory > accesses [1], I ran into an issue with the IPC specification [2] which > prevents us from ever achieving zero-copy memory mapping and having aligned > accesses (i.e. clean UBSan runs). > > Flatbuffer metadata needs 8-byte alignment to guarantee aligned accesses. > > In the IPC format we align each message to 8-byte boundaries. We then > write a int32_t integer to to denote the size of flat buffer metadata, > followed immediately by the flatbuffer metadata. This means the > flatbuffer metadata will never be 8 byte aligned. > > Do people care? A simple fix would be to use int64_t instead of int32_t > for length. However, any fix essentially breaks all previous client > library versions or incurs a memory copy. > > [1] https://github.com/apache/arrow/pull/4757 > [2] https://arrow.apache.org/docs/ipc.html
