Le 04/06/2020 à 18:11, Rémi Dettai a écrit : > > Ideally, we should be able to presize the array to a good enough > estimate. > You should be able to get away with a correct estimation because parquet > column metadata contains the uncompressed size. But is their anything wrong > with this idea of mmaping huge "runways" for our larger allocations ?
I don't know, but I suspect that if common allocators aren't doing it, there must be some downsides. I think trying to avoid most (re)allocations should be our preferred course of action here. Regards Antoine.
