On Sat, 18 Feb 2023 at 06:52, Andres Freund <and...@anarazel.de> wrote: > And did you increase ALLOCSET_DEFAULT_INITSIZE everywhere, or just passed a > larger block size in CreateExecutorState()? If the latter,the context > freelist wouldn't even come into play.
I think this piece of information is critical to confirm what the issue is. > A 8MB max block size is pretty darn small if you have a workload that ends up > with a gigabytes worth of blocks. We should probably review that separately. These kinds of definitions don't age well. The current ones appear about 23 years old now, so we might be overdue to reconsider what they're set to. 2002-12-15 21:01:34 +0000 150) #define ALLOCSET_DEFAULT_MINSIZE 0 2000-06-28 03:33:33 +0000 151) #define ALLOCSET_DEFAULT_INITSIZE (8 * 1024) 2000-06-28 03:33:33 +0000 152) #define ALLOCSET_DEFAULT_MAXSIZE (8 * 1024 * 1024) ... I recall having a desktop with 256MBs of RAM back then... Let's get to the bottom of where the problem is here before we consider adjusting those. If the problem is unrelated to that then we shouldn't be discussing that here. > And the problem also could just be that the default initial blocks size takes > too long to ramp up to a reasonable block size. I think it's 20 blocks to get > from ALLOCSET_DEFAULT_INITSIZE to ALLOCSET_DEFAULT_MAXSIZE. Even if you > allocate a good bit more than 8MB, having to additionally go through 20 > smaller chunks is going to be noticable until you reach a good bit higher > number of blocks. Well, let's try to help Johan get the information to us. I've attached a quickly put together patch which adds some debug stuff to aset.c. Johan, if you have a suitable test instance to try this on, can you send us the filtered DEBUG output from the log messages starting with "AllocSet" with and without your change? Just the output for just the 2nd execution of the query in question is fine. The first execution is not useful as the cache of MemoryContexts may not be populated by that time. It sounds like it's the foreign server that would need to be patched with this to test it. If you can send that in two files we should be able to easily see what has changed in terms of malloc() calls between the two runs. David
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c index 026f545676..590595bf25 100644 --- a/src/backend/utils/mmgr/aset.c +++ b/src/backend/utils/mmgr/aset.c @@ -427,6 +427,8 @@ AllocSetContextCreateInternal(MemoryContext parent, ((MemoryContext) set)->mem_allocated = set->keeper->endptr - ((char *) set); + elog(DEBUG1, "AllocSetContextCreateInternal: recycling context for %s", ((MemoryContext) set)->name); + return (MemoryContext) set; } } @@ -522,6 +524,8 @@ AllocSetContextCreateInternal(MemoryContext parent, ((MemoryContext) set)->mem_allocated = firstBlockSize; + elog(DEBUG1, "AllocSetContextCreateInternal: allocating new context for %s (%zu bytes)", ((MemoryContext)set)->name, firstBlockSize); + return (MemoryContext) set; } @@ -733,6 +737,8 @@ AllocSetAlloc(MemoryContext context, Size size) if (block == NULL) return NULL; + elog(DEBUG1, "AllocSetAlloc: malloc(%zu) dedicated block: %s", blksize, context->name); + context->mem_allocated += blksize; block->aset = set; @@ -943,6 +949,8 @@ AllocSetAlloc(MemoryContext context, Size size) if (block == NULL) return NULL; + elog(DEBUG1, "AllocSetAlloc: malloc(%zu) new block: %s", blksize, context->name); + context->mem_allocated += blksize; block->aset = set;