Did a quick bench of accessing long buffer not 8 bytes aligned. Giving
enough conditions, looks it does shows unaligned access has some penalty
over aligned access. But I don't think this is an issue in practice.
Please be very skeptical to this benchmark. It's hard to get it right
given the
Hi Jorge,
> Wrt to the extension type: I am not sure we can make it fast, though: the
> interpretation of the bytes would need to be done dynamically (instead of
> statically) because we can't compile the struct prior to receiving it (via
> IPC or FFI). This interpretation would be part of hot
>
> My own impression is that the emphasis may be slightly exagerated. But
> perhaps some other benchmarks would prove differently.
This is probably true. [1] is the original mailing list discussion. I
think lack of measurable differences and high overhead for 64 byte
alignment was the reason
Le 06/09/2021 à 23:20, Jorge Cardoso Leitão a écrit :
Thanks a lot Antoine for the pointers. Much appreciated!
Generally, it should not hurt to align allocations to 64 bytes anyway,
since you are generally dealing with large enough data that the
(small) memory overhead doesn't matter.
Not
To add to Antoine's points, besides data alignment being beneficial for
reducing cache line reads/write and overall using the cache more
effectively, another key point is when using vector (SIMD) registers.
Although recent CPUs can load unaligned data to vector registers at similar
speeds as
Thanks a lot Antoine for the pointers. Much appreciated!
Generally, it should not hurt to align allocations to 64 bytes anyway,
> since you are generally dealing with large enough data that the
> (small) memory overhead doesn't matter.
>
Not for performance. However, 64 byte alignment in Rust
Le 06/09/2021 à 19:45, Antoine Pitrou a écrit :
Specifically, I performed two types of tests, a "random sum" where we
compute the sum of the values taken at random indices, and "sum", where we
sum all values of the array (buffer[1] of the primitive array), both for
array ranging from 2^10 to
On Mon, 6 Sep 2021 18:09:31 +0100
Jorge Cardoso Leitão wrote:
> Hi,
>
> We have a whole section related to byte alignment (
> https://arrow.apache.org/docs/format/Columnar.html#buffer-alignment-and-padding)
> recommending 64 byte alignment and referring to intel's manual.
>
> Do we have
Hi,
We have a whole section related to byte alignment (
https://arrow.apache.org/docs/format/Columnar.html#buffer-alignment-and-padding)
recommending 64 byte alignment and referring to intel's manual.
Do we have evidence that this alignment helps (besides intel claims)?
I am asking because
