On Mon, Jul 26, 2010 at 12:51 PM, Markus Wanner <mar...@bluegap.ch> wrote: >> Dynamically allocating out of a 2MB >> segment gives up most of that flexibility. > > Absolutely, that's why I'd like to see other modules that use the dynamic > allocator. The more the better.
Right, I agree. The problem is that I don't think they can. The elephant in the room is shared_buffers, which I believe to be typically BY FAR the largest consumer of shared memory. It would be absolutely fantastic if we had a shared_buffers implementation that could free up unused buffers when they're not needed, or add more when required. But there are several reasons why I don't believe that will ever happen. One, much of the code that uses shared_buffers relies on shared_buffers being located at a fixed memory address on a contiguous chunk, and it's hard to see how we could change that assumption without sacrificing performance. Two, the overall size of the shared memory arena is largely dependent on the size of shared_buffers, so unless you also have the ability to resize the arena on the fly (which is well-nigh to impossible with our current architecture, and maybe with any architecture), resizing shared_buffers doesn't actually add that much flexibility. Three, the need for shared buffers is elastic rather than absolute: stealing a few shared buffers for a defined purpose (like sending imessages) is perfectly reasonable, but it's rarely going to be a good idea for the buffer manager to proactively free up memory just in case some other part of the system might need some. If you have a system that normally has 4GB of shared buffers and some other module borrows 100MB and then returns it, the system will just cache less data while that memory is in use and then start right back up caching more again once it's returned. That's very nice, and it's hard to see how else to achieve that result. Of course, there are other parts of the system (a whole bunch of them) that used shared memory also, and perhaps some of those could be modified to use the dynamic allocator as well. But they're getting by without it now, so maybe they don't really need it. The SLRU stuff, I think, works more or less like shared buffers (so you have the same set of issues) and I think most of the other users are allocating small, fixed-size chunks. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers