Andres Freund wrote: > On 2018-01-24 14:25:37 -0500, Robert Haas wrote: > > On Wed, Jan 24, 2018 at 1:43 PM, Andres Freund <and...@anarazel.de> wrote: > > > Indeed. Don't think RAW_BUF_SIZE is quite big enough for that on most > > > platforms though. From man mallopt: > > > Balancing these factors leads to a default setting of 128*1024 for > > > the M_MMAP_THRESHOLD parameter. > > > Additionally, even when malloc() chooses to use mmap() to back an > > > allocation, it'll still needs a header to know the size of the > > > allocation and such. So exactly using a size of a multiple of 4KB will > > > still leave you with wasted space. Due to the latter I can't see it > > > mattering whether or not we add +1 to a power-of-two size. > > > > Well, it depends on how it works. dsa_allocate, for example, never > > adds a header to the size of the allocation. > > glibc's malloc does add a header. My half-informed suspicion is that > most newer malloc backing allocators will have a header, because > maintaining a shared lookup-by-address table is pretty expensive to > maintain. A bit of metadata indicating size and/or source of the > allocation makes using thread-local information a lot easier.
Sounds like it'd be smart to allocate something closer to M_MMAP_THRESHOLD (which with typical values would be about double the amount of memory the current RAW_BUF_SIZE value), minus a few dozen bytes to allow for palloc's and malloc's respective headers. That way we bet for a mmap()ed allocation with minimum space wastage across all layers. Not sure whether we want to try to minimize wastage through clever use of malloc_usable_size() on each backend's first allocation -- that'd probably just be overengineering. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
