Le 06/09/2021 à 23:20, Jorge Cardoso Leitão a écrit :
Thanks a lot Antoine for the pointers. Much appreciated!
Generally, it should not hurt to align allocations to 64 bytes anyway,
since you are generally dealing with large enough data that the
(small) memory overhead doesn't matter.
Not for performance. However, 64 byte alignment in Rust requires
maintaining a custom container, a custom allocator, and the inability to
interoperate with `std::Vec` and the ecosystem that is based on it, since
std::Vec allocates with alignment T (.e.g int32), not 64 bytes. For anyone
interested, the background for this is this old PR [1] in this in arrow2
[2].
I see. In the C++ implementation, we are not compatible with the default
allocator either (but C++ allocators as defined by the standard library
don't support resizing, which doesn't make them terribly useful for
Arrow anyway).
Neither myself in micro benches nor Ritchie from polars (query engine) in
large scale benches observe any difference in the archs we have available.
This is not consistent with the emphasis we put on the memory alignments
discussion [3], and I am trying to understand the root cause for this
inconsistency.
My own impression is that the emphasis may be slightly exagerated. But
perhaps some other benchmarks would prove differently.
By prefetching I mean implicit; no intrinsics involved.
Well, I'm not aware that implicit prefetching depends on alignment.
Regards
Antoine.
