Hi,Back in the bug #14231 thread [1], dealing with performance issues in reorderbuffer due to excessive number of expensive free() calls, I've proposed to resolve that by a custom slab-like memory allocator, suitable for fixed-size allocations. I'd like to put this into the next CF, and it's probably too invasive change to count as a bugfix anyway.
[1] https://www.postgresql.org/message-id/flat/20160706185502.1426.28143%40wrigleys.postgresql.org
This patch actually includes two new memory allocators (not one). Very brief summary (for more detailed explanation of the ideas, see comments at the beginning of slab.c and genslab.c):
Slab ---- * suitable for fixed-length allocations (other pallocs fail) * much simpler than AllocSet (no global freelist management etc.) * free space is tracked per block (using a simple bitmap)* which allows freeing the block once all chunks are freed (AllocSet will hold the memory forever, in the hope of reusing it)
GenSlab -------* suitable for non-fixed-length allocations, but with chunks of mostly the same size (initially unknown, the context will tune itself)
* a combination AllocSet and Slab (or a sequence of Slab allocators) * the goal is to do most allocations in Slab context* there's always a single 'current' Slab context, and every now and and then it's replaced with a new generation (with the chunk size computed from recent requests)
* the AllocSet context is used for chunks too large for current SlabSo none of this is meant as a universal replacement of AllocSet, but in the suitable cases the results seem really promising. For example for the simple test query in [1], the performance improvement is this:
N | master | patched
-----------------------------
10000 | 100ms | 100ms
50000 | 15000ms | 350ms
100000 | 146000ms | 700ms
200000 | ? | 1400ms
That's a fairly significant improvement, and the submitted version of
the patches should perform even better (~2x, IIRC).
There's a bunch of TODOs - e.g. handling of realloc() calls in the GenSlab, and probably things I haven't thought about.
regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
0001-simple-slab-allocator-fixed-size-allocations.patch
Description: binary/octet-stream
0002-generational-slab-auto-tuning-allocator.patch
Description: binary/octet-stream
-- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
