On Wed, 14 Oct 2026 at 09:28, David Rowley <[email protected]> wrote: > > On Tue, 13 Oct 2026 at 07:22, Matthias van de Meent > <[email protected]> wrote: > > 0004/0005: Adjust for Proxy -> System /SystemCxt rename > > Here's a delta patch with various fixups for v4-0003
Thanks, attached is the adjusted patch series as v5, now again including "rel.h: const-ify opclass-related fields of RelationData", which I'd accidentally left out in v4. This should make CFBot happy again. [lots of review changes] > * Various comment adjustments. Applied, thanks! > I spent some time running this under Valgrind, accessing the memory of > a chunk made by SystemContextAlloc() and another after doing > SystemContextRealloc() on a chunk. Valgrind is correctly giving me > warnings when I access the byte before the pointer and when I access > the sentinel byte manually. I don't see any warnings when I access > bytes that are valid to access. > > I'm not aware of any other issues. Thanks for the review! Kind regards, Matthias van de Meent Databricks (https://www.databricks.com)
From 4b1df087882c45cb79187270f5342b5df603df97 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Sat, 31 Jan 2026 13:58:34 +0100 Subject: [PATCH v5 1/6] rel.h: const-ify opclass-related fields of RelationData This enables future sharing of these fields across indexes that all use the same opclasses across all key columns. Whilst rd_supportinfo matches this same description, its contents are dynamically filled with data, and so can't be marked const. --- src/backend/access/index/indexam.c | 4 ++-- src/backend/utils/cache/relcache.c | 17 +++++++++++------ src/include/utils/rel.h | 6 +++--- 3 files changed, 16 insertions(+), 11 deletions(-) diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c index db811056991..f092e8f7b71 100644 --- a/src/backend/access/index/indexam.c +++ b/src/backend/access/index/indexam.c @@ -747,7 +747,7 @@ index_getprocid(Relation irel, AttrNumber attnum, uint16 procnum) { - RegProcedure *loc; + const RegProcedure *loc; int nproc; int procindex; @@ -802,7 +802,7 @@ index_getprocinfo(Relation irel, /* Initialize the lookup info if first time through */ if (locinfo->fn_oid == InvalidOid) { - RegProcedure *loc = irel->rd_support; + const RegProcedure *loc = irel->rd_support; RegProcedure procId; Assert(loc != NULL); diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index d8f04a05309..b710b577be7 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -1454,6 +1454,9 @@ RelationInitIndexAccessInfo(Relation relation) int indnatts; int indnkeyatts; uint16 amsupport; + Oid *opfamily; + Oid *opcintype; + RegProcedure *support; /* * Make a copy of the pg_index entry for the index. Since pg_index @@ -1510,9 +1513,9 @@ RelationInitIndexAccessInfo(Relation relation) * Allocate arrays to hold data. Opclasses are not used for included * columns, so allocate them for indnkeyatts only. */ - relation->rd_opfamily = (Oid *) + opfamily = (Oid *) MemoryContextAllocZero(indexcxt, indnkeyatts * sizeof(Oid)); - relation->rd_opcintype = (Oid *) + opcintype = (Oid *) MemoryContextAllocZero(indexcxt, indnkeyatts * sizeof(Oid)); amsupport = relation->rd_indam->amsupport; @@ -1520,14 +1523,14 @@ RelationInitIndexAccessInfo(Relation relation) { int nsupport = indnatts * amsupport; - relation->rd_support = (RegProcedure *) + support = (RegProcedure *) MemoryContextAllocZero(indexcxt, nsupport * sizeof(RegProcedure)); relation->rd_supportinfo = (FmgrInfo *) MemoryContextAllocZero(indexcxt, nsupport * sizeof(FmgrInfo)); } else { - relation->rd_support = NULL; + support = NULL; relation->rd_supportinfo = NULL; } @@ -1567,10 +1570,12 @@ RelationInitIndexAccessInfo(Relation relation) * opfamilies and opclass input types. (aminfo and supportinfo are left * as zeroes, and are filled on-the-fly when used) */ - IndexSupportInitialize(indclass, relation->rd_support, - relation->rd_opfamily, relation->rd_opcintype, + IndexSupportInitialize(indclass, support, opfamily, opcintype, amsupport, indnkeyatts); + relation->rd_opfamily = opfamily; + relation->rd_opcintype = opcintype; + relation->rd_support = support; /* * Similarly extract indoption and copy it to the cache entry */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 7b8c2b1e362..2fdb88fa28c 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -204,9 +204,9 @@ typedef struct RelationData MemoryContext rd_indexcxt; /* private memory cxt for this stuff */ /* use "struct" here to avoid needing to include amapi.h: */ const struct IndexAmRoutine *rd_indam; /* index AM's API struct */ - Oid *rd_opfamily; /* OIDs of op families for each index col */ - Oid *rd_opcintype; /* OIDs of opclass declared input data types */ - RegProcedure *rd_support; /* OIDs of support procedures */ + const Oid *rd_opfamily; /* OIDs of op families for each index col */ + const Oid *rd_opcintype; /* OIDs of opclass declared input data types */ + const RegProcedure *rd_support; /* OIDs of support procedures */ struct FmgrInfo *rd_supportinfo; /* lookup info for support procedures */ int16 *rd_indoption; /* per-column AM-specific flags */ List *rd_indexprs; /* index expression trees, if any */ -- 2.54.0 (Apple Git-157)
From 6a23e9e835011d71a98f871ecd61e794fed66ba7 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Wed, 25 Feb 2026 02:16:16 +0100 Subject: [PATCH v5 3/6] Deduplicate some index attributes in the relcache Many indexes have the same opclasses; so this deduplicates all fields that are directly dependent on the selected opclasses. This can save a significant amount of memory when partitioned indexes are involved, as those always have the same definition. I have a few ideas on how to implement this for other fields as well, but that's not needed in this patch. This does come at a cost of a slight increase in the amount of work required to load index relations into catalog caches; but I'd consider that a worthy trade-off. --- src/backend/access/index/indexam.c | 29 +- src/backend/utils/cache/relcache.c | 911 ++++++++++++++++++++++------- src/backend/utils/fmgr/fmgr.c | 107 +++- src/include/c.h | 4 + src/include/fmgr.h | 6 +- src/include/nodes/pathnodes.h | 2 +- src/include/utils/rel.h | 2 +- src/include/utils/relcache.h | 2 +- 8 files changed, 847 insertions(+), 216 deletions(-) diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c index f092e8f7b71..42cff0144f9 100644 --- a/src/backend/access/index/indexam.c +++ b/src/backend/access/index/indexam.c @@ -49,6 +49,7 @@ #include "catalog/pg_type.h" #include "nodes/execnodes.h" #include "pgstat.h" +#include "nodes/memnodes.h" #include "storage/lmgr.h" #include "storage/lock.h" #include "storage/predicate.h" @@ -804,6 +805,7 @@ index_getprocinfo(Relation irel, { const RegProcedure *loc = irel->rd_support; RegProcedure procId; + Node *opcoptions; Assert(loc != NULL); @@ -819,17 +821,28 @@ index_getprocinfo(Relation irel, elog(ERROR, "missing support function %d for attribute %d of index \"%s\"", procnum, attnum, RelationGetRelationName(irel)); - fmgr_info_cxt(procId, locinfo, irel->rd_indexcxt); + /* + * irel->rd_supportinfo[].fn_expr and .fn_mcxt are pre-populated + * during cache initialization; though only after optsproc has been + * used. + */ + opcoptions = locinfo->fn_expr; + Assert(MemoryContextIsValid(locinfo->fn_mcxt)); + Assert(locinfo->fn_mcxt); + fmgr_info_cxt(procId, locinfo, locinfo->fn_mcxt); if (procnum != optsproc) { - /* Initialize locinfo->fn_expr with opclass options Const */ - bytea **attoptions = RelationGetIndexAttOptions(irel, false); - MemoryContext oldcxt = MemoryContextSwitchTo(irel->rd_indexcxt); - - set_fn_opclass_options(locinfo, attoptions[attnum - 1]); - - MemoryContextSwitchTo(oldcxt); + /* + * Even without attoptions, opcoptions here would not be NULL, + * but instead would have ((Const *) opcoptions)->constisnull. + */ + if (opcoptions == NULL) + elog(PANIC, "Invalid opcoptions for valid Relation: NULL instead of Const<null>"); + if (nodeTag(opcoptions) != T_Const) + elog(PANIC, "Unexpected node tag for opcoptions: %d, expected %d", + nodeTag(opcoptions), T_Const); + locinfo->fn_expr = opcoptions; } } diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 93e06206db4..99c8aec14d1 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -68,6 +68,7 @@ #include "commands/policy.h" #include "commands/publicationcmds.h" #include "commands/trigger.h" +#include "common/hashfn.h" #include "common/int.h" #include "miscadmin.h" #include "nodes/makefuncs.h" @@ -93,7 +94,7 @@ #include "utils/snapmgr.h" #include "utils/syscache.h" -#define RELCACHE_INIT_FILEMAGIC 0x573266 /* version ID value */ +#define RELCACHE_INIT_FILEMAGIC 0x573267 /* version ID value */ /* * Whether to bother checking if relation cache memory needs to be freed @@ -137,6 +138,31 @@ typedef struct relidcacheent static HTAB *RelationIdCache; +typedef struct RelShapeCacheKey +{ + AttrNumber nkeyatts; + uint16 nsupport; + /* opclass-specific data */ + const Oid *opclass; + const bytea *const *opcoptions; /* Treated as an array of NULL + * elements when NULL */ +} RelShapeCacheKey; + +typedef struct RelShapeCacheEnt +{ + RelShapeCacheKey key; + int rsc_refcount; /* refcount */ + MemoryContext rsc_mctx; /* memory context for this entry */ + /* Allocations in rsc_mctx */ + const Oid *rsc_opfamily; /* opfamilies of key.opclass */ + const Oid *rsc_opcintype; + const RegProcedure *rsc_support; /* support functions */ + FmgrInfo *rsc_supportinfo; /* can be updated */ +} RelShapeCacheEnt; + +static HTAB *RelShapeCache; +static MemoryContext RelShapeContext; + /* * This flag is false until we have prepared the critical relcache entries * that are needed to do indexscans on the tables read by relcache building. @@ -294,6 +320,7 @@ static void AtEOXact_cleanup(Relation relation, bool isCommit); static void AtEOSubXact_cleanup(Relation relation, bool isCommit, SubTransactionId mySubid, SubTransactionId parentSubid); static bool load_relcache_init_file(bool shared); +static bool load_relcache_init_index(Relation relation, FILE *fp); static void write_relcache_init_file(bool shared); static void write_item(const void *data, Size len, FILE *fp); @@ -315,13 +342,21 @@ static void CheckNNConstraintFetch(Relation relation); static int CheckConstraintCmp(const void *a, const void *b); static void InitIndexAmRoutine(Relation relation); static void IndexSupportInitialize(oidvector *indclass, AttrNumber nKeyAtts, StrategyNumber maxSupportNumber, - MemoryContext indexcxt, const Oid **rd_opfamily, const Oid **rd_opcintype, - const RegProcedure **rd_support, FmgrInfo **rd_supportinfo); + const Oid **rd_opfamily, const Oid **rd_opcintype, + const RegProcedure **rd_support, FmgrInfo **rd_supportinfo, + const bytea *const **rd_opcoptions); +static void IndexSupportDeregister(oidvector *indclass, AttrNumber nKeyAtts, + StrategyNumber maxSupportNumber, + const bytea * const*rd_opcoptions); +static bytea **IndexRelationGetAttOptions(Relation relation); static OpClassCacheEnt *LookupOpclassInfo(Oid operatorClassOid, StrategyNumber numSupport); static void RelationCacheInitFileRemoveInDir(const char *tblspcpath); static void unlink_initfile(const char *initfilename, int elevel); +static uint32 HashRelShapeKey(const void *key, Size keysize); +static int CmpRelShapeKey(const void *key1, const void *key2, Size keysize); +static void *CopyRelShapeKey(void *dest, const void *src, Size keysize); /* * ScanPgRelation @@ -1451,6 +1486,8 @@ RelationInitIndexAccessInfo(Relation relation) int indnatts; int indnkeyatts; uint16 amsupport; + const bytea *const *opcoptions; + bool hasopcoptions = false; /* * Make a copy of the pg_index entry for the index. Since pg_index @@ -1545,14 +1582,58 @@ RelationInitIndexAccessInfo(Relation relation) /* * Fill the support procedure OID array, as well as the info about - * opfamilies and opclass input types. (aminfo and supportinfo are left - * as zeroes, and are filled on-the-fly when used) + * opfamilies and opclass input types. + * + * rd_opcoptions and rd_supportinfo are only minimally initialized here: + * rd_opcoptions is updated with the real values down below if we have + * any non-NULL options, supportinfo's function data is loaded only when + * accessed by a caller. */ - IndexSupportInitialize(indclass, indnkeyatts, amsupport, indexcxt, + IndexSupportInitialize(indclass, indnkeyatts, amsupport, &relation->rd_opfamily, &relation->rd_opcintype, &relation->rd_support, - &relation->rd_supportinfo); + &relation->rd_supportinfo, + &relation->rd_opcoptions); + + opcoptions = (const bytea *const *) IndexRelationGetAttOptions(relation); + + for (int i = 0; i < indnkeyatts; i++) + { + if (opcoptions[i] != NULL) + hasopcoptions = true; + } + + if (hasopcoptions) + { + IndexSupportDeregister(indclass, indnkeyatts, amsupport, + relation->rd_opcoptions); + + relation->rd_opcoptions = opcoptions; + + /* + * This fills in all fields, again; now whilst taking the new opcoptions + * into account. + */ + IndexSupportInitialize(indclass, indnkeyatts, amsupport, + &relation->rd_opfamily, + &relation->rd_opcintype, + &relation->rd_support, + &relation->rd_supportinfo, + &relation->rd_opcoptions); + } + + /* + * Opcoptions was either copied into the cache context, or wasn't used; + * in either case it's now unused and we should free it. + */ + Assert(opcoptions != relation->rd_opcoptions); + for (int i = 0; i < indnkeyatts; i++) + { + if (opcoptions[i] != NULL) + pfree(unconstify(bytea *, opcoptions[i])); + } + pfree(unconstify_array(const bytea *, opcoptions)); /* * Similarly extract indoption and copy it to the cache entry @@ -1565,8 +1646,6 @@ RelationInitIndexAccessInfo(Relation relation) indoption = (int2vector *) DatumGetPointer(indoptionDatum); memcpy(relation->rd_indoption, indoption->values, indnkeyatts * sizeof(int16)); - (void) RelationGetIndexAttOptions(relation, false); - /* * expressions, predicate, exclusion caches will be filled later */ @@ -1594,58 +1673,162 @@ RelationInitIndexAccessInfo(Relation relation) static void IndexSupportInitialize(oidvector *indclass, AttrNumber nKeyAtts, StrategyNumber maxSupportNumber, - MemoryContext indexcxt, const Oid **rd_opfamily, const Oid **rd_opcintype, const RegProcedure **rd_support, - FmgrInfo **rd_supportinfo) + FmgrInfo **rd_supportinfo, + const bytea *const **rd_opcoptions) { - Oid *opFamily; - Oid *opcInType; - RegProcedure *indexSupport; + RelShapeCacheKey key = {0}; + RelShapeCacheEnt *entry; + bool found; - if (maxSupportNumber > 0) - { - int nprocs = maxSupportNumber * nKeyAtts; - *rd_supportinfo = (FmgrInfo *) - MemoryContextAllocZero(indexcxt, nprocs * sizeof(FmgrInfo)); - indexSupport = (RegProcedure *) - MemoryContextAllocZero(indexcxt, nprocs * sizeof(RegProcedure)); - } - else - { - *rd_supportinfo = NULL; - indexSupport = NULL; - } + key.nkeyatts = nKeyAtts; + key.nsupport = maxSupportNumber; + key.opclass = indclass->values; + key.opcoptions = *rd_opcoptions; + + entry = hash_search(RelShapeCache, &key, HASH_ENTER, &found); - opFamily = (Oid *) - MemoryContextAllocZero(indexcxt, nKeyAtts * sizeof(Oid)); - opcInType = (Oid *) - MemoryContextAllocZero(indexcxt, nKeyAtts * sizeof(Oid)); + Assert(entry != NULL); - for (int attIndex = 0; attIndex < nKeyAtts; attIndex++) + if (!found) { - OpClassCacheEnt *opcentry; + Oid *opFamily; + Oid *opcInType; + RegProcedure *support; + FmgrInfo *supportinfo; + MemoryContext shapectx; - if (!OidIsValid(indclass->values[attIndex])) - elog(ERROR, "bogus pg_index tuple"); + shapectx = AllocSetContextCreate(RelShapeContext, "index shape", + ALLOCSET_SMALL_SIZES); - /* look up the info for this opclass, using a cache */ - opcentry = LookupOpclassInfo(indclass->values[attIndex], - maxSupportNumber); + if (maxSupportNumber > 0) + { + int nprocs = maxSupportNumber * nKeyAtts; + supportinfo = (FmgrInfo *) + MemoryContextAllocZero(shapectx, nprocs * sizeof(FmgrInfo)); + support = (RegProcedure *) + MemoryContextAllocZero(shapectx, nprocs * sizeof(RegProcedure)); + } + else + { + supportinfo = NULL; + support = NULL; + } + + opFamily = (Oid *) + MemoryContextAllocZero(shapectx, nKeyAtts * sizeof(Oid)); + opcInType = (Oid *) + MemoryContextAllocZero(shapectx, nKeyAtts * sizeof(Oid)); + + for (int attIndex = 0; attIndex < nKeyAtts; attIndex++) + { + OpClassCacheEnt *opcentry; + + if (!OidIsValid(indclass->values[attIndex])) + elog(ERROR, "bogus pg_index tuple"); + + /* look up the info for this opclass, using a cache */ + opcentry = LookupOpclassInfo(indclass->values[attIndex], + maxSupportNumber); + + /* copy cached data into relcache entry */ + opFamily[attIndex] = opcentry->opcfamily; + opcInType[attIndex] = opcentry->opcintype; + + if (maxSupportNumber > 0) + { + /* copy support proc IDs into the support proc array */ + memcpy(&support[attIndex * maxSupportNumber], + opcentry->supportProcs, + maxSupportNumber * sizeof(RegProcedure)); + } + } - /* copy cached data into relcache entry */ - opFamily[attIndex] = opcentry->opcfamily; - opcInType[attIndex] = opcentry->opcintype; if (maxSupportNumber > 0) - memcpy(&indexSupport[attIndex * maxSupportNumber], - opcentry->supportProcs, - maxSupportNumber * sizeof(RegProcedure)); + { + set_fn_opclass_options_bulk(supportinfo, + entry->key.opcoptions, + maxSupportNumber, + nKeyAtts, shapectx); + } + + entry->rsc_opfamily = opFamily; + entry->rsc_opcintype = opcInType; + entry->rsc_support = support; + entry->rsc_supportinfo = supportinfo; + entry->rsc_refcount = 1; + entry->rsc_mctx = shapectx; + + Assert(maxSupportNumber == 0 || entry->rsc_supportinfo); + Assert(maxSupportNumber == 0 || entry->rsc_support); + Assert(maxSupportNumber == 0 || entry->key.opcoptions); } - *rd_support = indexSupport; - *rd_opcintype = opcInType; - *rd_opfamily = opFamily; + entry->rsc_refcount++; + + *rd_opcoptions = entry->key.opcoptions; + + *rd_opfamily = entry->rsc_opfamily; + *rd_opcintype = entry->rsc_opcintype; + *rd_support = entry->rsc_support; + *rd_supportinfo = entry->rsc_supportinfo; +} + +static void +IndexSupportDeregister(oidvector *indclass, AttrNumber nKeyAtts, + StrategyNumber maxSupportNumber, + const bytea *const *rd_opcoptions) +{ + RelShapeCacheKey key = {0}; + RelShapeCacheEnt *entry; + bool found; + + key.nkeyatts = nKeyAtts; + key.nsupport = maxSupportNumber; + key.opclass = indclass->values; + key.opcoptions = rd_opcoptions; + + entry = hash_search(RelShapeCache, &key, HASH_FIND, &found); + + Assert(found); + + entry->rsc_refcount--; + + if (entry->rsc_refcount > 1) + return; + + Assert(entry->rsc_refcount == 1); + /* + * Apart from key.opclass[], which is taken from an oidvector, all key + * fields are stored in RelationData and are expected to be passed + * directly by the caller. + */ + Assert(entry->key.opcoptions == key.opcoptions); + Assert(entry->key.opclass != key.opclass); + + /* + * Store the opclass reference, so we can use it after removing the hash + * entry. + */ + key.opclass = entry->key.opclass; + + /* free memory associated with this shape, in bulk */ + MemoryContextDelete(entry->rsc_mctx); + + /* Remove the hash entry */ + entry = hash_search(RelShapeCache, &key, HASH_REMOVE, &found); + Assert(found); + + /* free the key's allocated elements */ + pfree(unconstify(Oid *, key.opclass)); + for (int i = 0; i < key.nkeyatts; i++) + { + if (key.opcoptions[i] != NULL) + pfree(unconstify(bytea *, key.opcoptions[i])); + } + pfree(unconstify_array(const bytea *, key.opcoptions)); } /* @@ -2454,6 +2637,29 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc) /* break mutual link with stats entry */ pgstat_unlink_relation(relation); + /* if it's an index relation, deregister it from the shape cache */ + if (relation->rd_indam && relation->rd_opcoptions) + { + Datum indclassDatum; + bool isnull; + oidvector *indclass; + + Assert(relation->rd_indextuple != NULL); + + indclassDatum = fastgetattr(relation->rd_indextuple, + Anum_pg_index_indclass, + GetPgIndexDescriptor(), + &isnull); + Assert(!isnull); + + indclass = (oidvector *) DatumGetPointer(indclassDatum); + + IndexSupportDeregister(indclass, + IndexRelationGetNumberOfKeyAttributes(relation), + relation->rd_indam->amsupport, + relation->rd_opcoptions); + } + /* * Free all the subsidiary data structures of the relcache entry, then the * entry itself. @@ -4023,6 +4229,25 @@ RelationCacheInitialize(void) RelationIdCache = hash_create("Relcache by OID", INITRELCACHESIZE, &ctl, HASH_ELEM | HASH_BLOBS); + RelShapeContext = + AllocSetContextCreate(CacheMemoryContext, "Relation shape cache", + ALLOCSET_DEFAULT_SIZES); + + /* + * create hashtable that indexes and caches relation shapes + */ + ctl.keysize = sizeof(RelShapeCacheKey); + ctl.entrysize = sizeof(RelShapeCacheEnt); + ctl.keycopy = CopyRelShapeKey; + ctl.hash = HashRelShapeKey; + ctl.match = CmpRelShapeKey; + ctl.hcxt = RelShapeContext; + + RelShapeCache = hash_create("Relation shape hash", INITRELCACHESIZE, + &ctl, HASH_ELEM | + HASH_FUNCTION | HASH_COMPARE | HASH_KEYCOPY | + HASH_CONTEXT); + /* * reserve enough in_progress_list slots for many cases */ @@ -4394,6 +4619,192 @@ RelationCacheInitializePhase3(void) } } +static uint32 +HashRelShapeKey(const void *key, Size keysize) +{ + const RelShapeCacheKey *rskey = key; + uint32 hdr_hash, opc_hash, opcopt_hash; + + hdr_hash = hash_combine(hash_uint32(rskey->nkeyatts), + hash_uint32(rskey->nsupport)); + opc_hash = hash_any((void *) rskey->opclass, + sizeof(Oid) * rskey->nkeyatts); + + /* + * Opcoptions hashing is special; as it may be NULL in search keys. + * Additionally, the array (if present) can contain nulls. We must still + * hash these NULLs, because positionality matters: An index with + * (opts, NULL) must hash differently from one with (NULL, opts) or even + * (opts). If we ignored NULLs in the array, they'd be guaranteed to have + * an equal hash value, causing hash conflicts with performance issues + * as a result. + */ + opcopt_hash = 0; + if (rskey->opcoptions == NULL) + { + /* + * Only present for search operations, but regardless, we'll have to + * combine the hashes of every (missing) element of the NULL array. + */ + for (int i = 0; i < rskey->nkeyatts; i++) + opcopt_hash = hash_combine(opcopt_hash, 0); + } + else + { + for (int i = 0; i < rskey->nkeyatts; i++) + { + const bytea *opt = rskey->opcoptions[i]; + uint32 hash; + + if (opt == NULL) + hash = 0; + else + { + hash = hash_any((const unsigned char *) VARDATA(opt), + (int) VARSIZE_ANY_EXHDR(opt)); + } + + opcopt_hash = hash_combine(opcopt_hash, hash); + } + } + + return hash_combine(opcopt_hash, hash_combine(hdr_hash, opc_hash)); +} + +static int +CmpRelShapeKey(const void *key1, const void *key2, Size keysize) +{ + const RelShapeCacheKey *rskey1 = key1; + const RelShapeCacheKey *rskey2 = key2; + + if (rskey1->nkeyatts != rskey2->nkeyatts || + rskey1->nsupport != rskey2->nsupport) + return 1; + + if (rskey1->opclass != rskey2->opclass && + memcmp(rskey1->opclass, rskey2->opclass, + sizeof(Oid) * rskey1->nkeyatts) != 0) + return 1; + + /* both NULL, or point to the same memory */ + if (rskey1->opcoptions == rskey2->opcoptions) + return 0; + + /* + * One of the two arrays is null; they only match when the allocated + * array's elements are all NULL too + */ + if (rskey1->opcoptions == NULL || rskey2->opcoptions == NULL) + { + const bytea *const *opts; + + if (rskey1->opcoptions != NULL) + { + Assert(rskey2->opcoptions == NULL); + opts = rskey1->opcoptions; + } + else + { + Assert(rskey2->opcoptions != NULL); + opts = rskey2->opcoptions; + } + + for (int i = 0; i < rskey1->nkeyatts; i++) + { + if (opts[i] != NULL) + return 1; + } + + return 0; + } + + /* + * Compare the options elements; now that we've established that + * both arrays are non-null. + */ + for (int i = 0; i < rskey1->nkeyatts; i++) + { + const bytea *a = rskey1->opcoptions[i]; + const bytea *b = rskey2->opcoptions[i]; + + /* both elements are NULL, or point to the same value */ + if (a == b) + continue; + + /* not the same: only one of the elements is NULL */ + if (a == NULL || b == NULL) + return 1; + + /* different; if the sizes are different */ + if (VARSIZE_ANY_EXHDR(a) != VARSIZE_ANY_EXHDR(b)) + return 1; + + /* different; if the data in the options is different */ + if (memcmp(VARDATA(a), VARDATA(b), VARSIZE_ANY_EXHDR(a)) != 0) + return 1; + } + + /* no differences detected */ + return 0; +} + +static void * +CopyRelShapeKey(void *dest, const void *src, Size keysize) +{ + RelShapeCacheKey *destrskey = dest; + const RelShapeCacheKey *srcrskey = src; + Oid *opclass; + const bytea **opcoptions; + + opclass = MemoryContextAlloc(RelShapeContext, + sizeof(Oid) * srcrskey->nkeyatts); + opcoptions = MemoryContextAlloc(RelShapeContext, + sizeof(bytea *) * srcrskey->nkeyatts); + + memcpy(opclass, srcrskey->opclass, sizeof(Oid) * srcrskey->nkeyatts); + + if (srcrskey->opcoptions != NULL) + { + for (int i = 0; i < srcrskey->nkeyatts; i++) + { + const bytea *srcopt = srcrskey->opcoptions[i]; + Size srcsize; + + if (srcopt != NULL) + { + bytea *newopt; + + srcsize = VARSIZE_ANY_EXHDR(srcopt); + newopt = MemoryContextAlloc(RelShapeContext, srcsize + VARHDRSZ); + SET_VARSIZE(newopt, srcsize + VARHDRSZ); + memcpy(VARDATA(newopt), VARDATA(srcopt), srcsize); + opcoptions[i] = newopt; + } + else + { + opcoptions[i] = NULL; + } + } + } + else + { + for (int i = 0; i < srcrskey->nkeyatts; i++) + opcoptions[i] = NULL; + } + + destrskey->nkeyatts = srcrskey->nkeyatts; + destrskey->nsupport = srcrskey->nsupport; + + destrskey->opclass = opclass; + destrskey->opcoptions = opcoptions; + + Assert(HashRelShapeKey(src, keysize) == HashRelShapeKey(dest, keysize)); + Assert(CmpRelShapeKey(src, dest, keysize) == 0); + + return destrskey; +} + + /* * Load one critical system index into the relcache * @@ -6047,13 +6458,13 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc) } static bytea ** -CopyIndexAttOptions(bytea **srcopts, int natts) +CopyIndexAttOptions(const bytea *const *srcopts, int natts) { bytea **opts = palloc_array(bytea *, natts); for (int i = 0; i < natts; i++) { - bytea *opt = srcopts[i]; + const bytea *opt = srcopts[i]; opts[i] = !opt ? NULL : (bytea *) DatumGetPointer(datumCopy(PointerGetDatum(opt), false, -1)); @@ -6062,29 +6473,19 @@ CopyIndexAttOptions(bytea **srcopts, int natts) return opts; } -/* - * RelationGetIndexAttOptions - * get AM/opclass-specific options for an index parsed into a binary form - */ -bytea ** -RelationGetIndexAttOptions(Relation relation, bool copy) +static bytea ** +IndexRelationGetAttOptions(Relation relation) { - MemoryContext oldcxt; - bytea **opts = relation->rd_opcoptions; - Oid relid = RelationGetRelid(relation); - int natts = IndexRelationGetNumberOfKeyAttributes(relation); - int i; - - /* Try to copy cached options. */ - if (opts) - return copy ? CopyIndexAttOptions(opts, natts) : opts; + bytea **opts; + int natts = IndexRelationGetNumberOfKeyAttributes(relation); + Oid relid = relation->rd_id; /* Get and parse opclass options. */ opts = palloc0_array(bytea *, natts); - for (i = 0; i < natts; i++) + if (criticalRelcachesBuilt && relid != AttributeRelidNumIndexId) { - if (criticalRelcachesBuilt && relid != AttributeRelidNumIndexId) + for (int i = 0; i < natts; i++) { Datum attoptions = get_attoptions(relid, i + 1); @@ -6095,23 +6496,30 @@ RelationGetIndexAttOptions(Relation relation, bool copy) } } - /* Copy parsed options to the cache. */ - oldcxt = MemoryContextSwitchTo(relation->rd_indexcxt); - relation->rd_opcoptions = CopyIndexAttOptions(opts, natts); - MemoryContextSwitchTo(oldcxt); + return opts; +} - if (copy) - return opts; +/* + * RelationGetIndexAttOptions + * get AM/opclass-specific options for an index parsed into a binary form + */ +const bytea *const * +RelationGetIndexAttOptions(Relation relation, bool copy) +{ + const bytea *const *opts = relation->rd_opcoptions; + int natts = IndexRelationGetNumberOfKeyAttributes(relation); - for (i = 0; i < natts; i++) - { - if (opts[i]) - pfree(opts[i]); - } + /* + * Every index Relation in the relcache should've gone through either + * load_relcache_init_index or RelationInitIndexAccessInfo, both of + * which initialize rd_opcoptions. + */ + Assert(opts); - pfree(opts); + if (copy) + return (const bytea *const *) CopyIndexAttOptions(opts, natts); - return relation->rd_opcoptions; + return opts; } /* @@ -6387,119 +6795,12 @@ load_relcache_init_file(bool shared) */ if (rel->rd_rel->relkind == RELKIND_INDEX) { - MemoryContext indexcxt; - Oid *opfamily; - Oid *opcintype; - RegProcedure *support; - int nsupport; - int16 *indoption; - Oid *indcollation; - /* Count nailed indexes to ensure we have 'em all */ if (rel->rd_isnailed) nailed_indexes++; - /* read the pg_index tuple */ - if (fread(&len, 1, sizeof(len), fp) != sizeof(len)) - goto read_failed; - - rel->rd_indextuple = (HeapTuple) palloc(len); - if (fread(rel->rd_indextuple, 1, len, fp) != len) - goto read_failed; - - /* Fix up internal pointers in the tuple -- see heap_copytuple */ - rel->rd_indextuple->t_data = (HeapTupleHeader) ((char *) rel->rd_indextuple + HEAPTUPLESIZE); - rel->rd_index = (Form_pg_index) GETSTRUCT(rel->rd_indextuple); - - /* - * prepare index info context --- parameters should match - * RelationInitIndexAccessInfo - */ - indexcxt = AllocSetContextCreate(CacheMemoryContext, - "index info", - ALLOCSET_SMALL_SIZES); - rel->rd_indexcxt = indexcxt; - MemoryContextCopyAndSetIdentifier(indexcxt, - RelationGetRelationName(rel)); - - /* - * Now we can fetch the index AM's API struct. (We can't store - * that in the init file, since it contains function pointers that - * might vary across server executions. Fortunately, it should be - * safe to call the amhandler even while bootstrapping indexes.) - */ - InitIndexAmRoutine(rel); - - /* read the vector of opfamily OIDs */ - if (fread(&len, 1, sizeof(len), fp) != sizeof(len)) - goto read_failed; - - opfamily = (Oid *) MemoryContextAlloc(indexcxt, len); - if (fread(opfamily, 1, len, fp) != len) - goto read_failed; - - rel->rd_opfamily = opfamily; - - /* read the vector of opcintype OIDs */ - if (fread(&len, 1, sizeof(len), fp) != sizeof(len)) - goto read_failed; - - opcintype = (Oid *) MemoryContextAlloc(indexcxt, len); - if (fread(opcintype, 1, len, fp) != len) - goto read_failed; - - rel->rd_opcintype = opcintype; - - /* read the vector of support procedure OIDs */ - if (fread(&len, 1, sizeof(len), fp) != sizeof(len)) - goto read_failed; - support = (RegProcedure *) MemoryContextAlloc(indexcxt, len); - if (fread(support, 1, len, fp) != len) - goto read_failed; - - rel->rd_support = support; - - /* read the vector of collation OIDs */ - if (fread(&len, 1, sizeof(len), fp) != sizeof(len)) - goto read_failed; - - indcollation = (Oid *) MemoryContextAlloc(indexcxt, len); - if (fread(indcollation, 1, len, fp) != len) - goto read_failed; - - rel->rd_indcollation = indcollation; - - /* read the vector of indoption values */ - if (fread(&len, 1, sizeof(len), fp) != sizeof(len)) + if (!load_relcache_init_index(rel, fp)) goto read_failed; - - indoption = (int16 *) MemoryContextAlloc(indexcxt, len); - if (fread(indoption, 1, len, fp) != len) - goto read_failed; - - rel->rd_indoption = indoption; - - /* read the vector of opcoptions values */ - rel->rd_opcoptions = (bytea **) - MemoryContextAllocZero(indexcxt, sizeof(*rel->rd_opcoptions) * relform->relnatts); - - for (i = 0; i < relform->relnatts; i++) - { - if (fread(&len, 1, sizeof(len), fp) != sizeof(len)) - goto read_failed; - - if (len > 0) - { - rel->rd_opcoptions[i] = (bytea *) MemoryContextAlloc(indexcxt, len); - if (fread(rel->rd_opcoptions[i], 1, len, fp) != len) - goto read_failed; - } - } - - /* set up zeroed fmgr-info vector */ - nsupport = relform->relnatts * rel->rd_indam->amsupport; - rel->rd_supportinfo = (FmgrInfo *) - MemoryContextAllocZero(indexcxt, nsupport * sizeof(FmgrInfo)); } else { @@ -6661,6 +6962,202 @@ read_failed: return false; } +static bool +load_relcache_init_index(Relation rel, FILE *fp) +{ + MemoryContext indexcxt; + RelShapeCacheKey key; + const bytea **opcoptions; + RelShapeCacheEnt *entry; + Oid *opfamily; + Oid *opcintype; + RegProcedure *support; + int16 *indoption; + Oid *indcollation; + Oid indclass[INDEX_MAX_KEYS]; + Size len; + Datum indclassDatum; + bool isnull, found; + + /* read the pg_index tuple */ + if (fread(&len, 1, sizeof(len), fp) != sizeof(len)) + return false; + + Assert(len >= HEAPTUPLESIZE + SizeofHeapTupleHeader + sizeof(FormData_pg_index)); + + rel->rd_indextuple = (HeapTuple) palloc(len); + Assert(len != 0); + + if (fread(rel->rd_indextuple, 1, len, fp) != len) + return false; + Assert(rel->rd_indextuple->t_len >= 0); + + /* Fix up internal pointers in the tuple -- see heap_copytuple */ + rel->rd_indextuple->t_data = (HeapTupleHeader) (((char *) rel->rd_indextuple) + HEAPTUPLESIZE); + rel->rd_index = (Form_pg_index) GETSTRUCT(rel->rd_indextuple); + + /* + * prepare index info context --- parameters should match + * RelationInitIndexAccessInfo + */ + indexcxt = AllocSetContextCreate(CacheMemoryContext, "index info", + ALLOCSET_SMALL_SIZES); + rel->rd_indexcxt = indexcxt; + MemoryContextCopyAndSetIdentifier(indexcxt, + RelationGetRelationName(rel)); + + /* + * Now we can fetch the index AM's API struct. (We can't store + * that in the init file, since it contains function pointers that + * might vary across server executions. Fortunately, it should be + * safe to call the amhandler even while bootstrapping indexes.) + */ + InitIndexAmRoutine(rel); + + key.nkeyatts = IndexRelationGetNumberOfKeyAttributes(rel); + key.nsupport = rel->rd_indam->amsupport; + + indclassDatum = fastgetattr(rel->rd_indextuple, + Anum_pg_index_indclass, + GetPgIndexDescriptor(), + &isnull); + Assert(!isnull); + memcpy(&indclass[0], + ((oidvector *) DatumGetPointer(indclassDatum))->values, + sizeof(Oid) * key.nkeyatts); + key.opclass = indclass; + + /* read the vector of opcoptions values */ + opcoptions = (const bytea **) + MemoryContextAlloc(RelShapeContext, sizeof(const bytea *) * key.nkeyatts); + + /* read the values */ + for (int i = 0; i < key.nkeyatts; i++) + { + if (fread(&len, 1, sizeof(len), fp) != sizeof(len)) + return false; + + if (len > 0) + { + bytea *read = (bytea *) + MemoryContextAlloc(RelShapeContext, len); + + Assert(len >= 4); + + if (fread(read, 1, len, fp) != len) + return false; + + opcoptions[i] = read; + } + else + opcoptions[i] = NULL; + } + + Assert(opcoptions); + Assert(key.nkeyatts > 0 && key.nkeyatts <= INDEX_MAX_KEYS); + Assert(key.nsupport); + + key.opcoptions = opcoptions; + entry = hash_search(RelShapeCache, &key, HASH_ENTER, &found); + +#define READ_ITEM(type, name, neachkey, context) \ + do { \ + if (fread(&len, 1, sizeof(len), fp) != sizeof(len)) \ + return false; \ + \ + if (len != sizeof(type) * (neachkey) * key.nkeyatts) \ + return false; \ + \ + name = (type *) \ + MemoryContextAlloc(context, len); \ + \ + if (fread(name, 1, len, fp) != len) \ + return false; \ + } while (0) + + READ_ITEM(Oid, opfamily, 1, RelShapeContext); + READ_ITEM(Oid, opcintype, 1, RelShapeContext); + READ_ITEM(RegProcedure, support, key.nsupport, RelShapeContext); + READ_ITEM(Oid, indcollation, 1, indexcxt); + READ_ITEM(int16, indoption, 1, indexcxt); + + if (!found) + { + MemoryContext idxshapectx; + char *bulkalloc; + char *bulkalloc_end PG_USED_FOR_ASSERTS_ONLY; + Size total_alloc = 0; + + entry->rsc_refcount = 1; + + idxshapectx = AllocSetContextCreate(RelShapeContext, "index shape", + ALLOCSET_SMALL_SIZES); + + total_alloc = sizeof(FmgrInfo) * key.nsupport * key.nkeyatts; + total_alloc += sizeof(Oid) * 2 * key.nkeyatts; + total_alloc += sizeof(RegProcedure) * key.nsupport * key.nkeyatts; + + + bulkalloc = MemoryContextAllocZero(idxshapectx, total_alloc); + bulkalloc_end = bulkalloc + total_alloc; + + entry->rsc_mctx = idxshapectx; + + if (key.nsupport > 0) + { + entry->rsc_supportinfo = (FmgrInfo *) bulkalloc; + bulkalloc += sizeof(FmgrInfo) * key.nkeyatts * key.nsupport; + + memcpy(bulkalloc, support, + sizeof(RegProcedure) * key.nkeyatts * key.nsupport); + entry->rsc_support = (Oid *) bulkalloc; + bulkalloc += sizeof(RegProcedure) * key.nkeyatts * key.nsupport; + } + else + { + entry->rsc_supportinfo = NULL; + entry->rsc_support = NULL; + } + + memcpy(bulkalloc, opfamily, sizeof(Oid) * key.nkeyatts); + entry->rsc_opfamily = (Oid *) bulkalloc; + bulkalloc += sizeof(Oid) * key.nkeyatts; + + memcpy(bulkalloc, opcintype, sizeof(Oid) * key.nkeyatts); + entry->rsc_opcintype = (Oid *) bulkalloc; + bulkalloc += sizeof(Oid) * key.nkeyatts; + + Assert(bulkalloc == bulkalloc_end); + + Assert(entry->key.opcoptions); + Assert(entry->key.opclass); + Assert(entry->rsc_opcintype); + Assert(entry->rsc_opfamily); + Assert(entry->rsc_supportinfo || entry->key.nsupport == 0); + Assert(entry->rsc_support || entry->key.nsupport == 0); + + set_fn_opclass_options_bulk(entry->rsc_supportinfo, + entry->key.opcoptions, + key.nsupport, + key.nkeyatts, idxshapectx); + } + pfree(opfamily); + pfree(opcintype); + pfree(support); + + entry->rsc_refcount++; + + rel->rd_opfamily = entry->rsc_opfamily; + rel->rd_opcintype = entry->rsc_opcintype; + rel->rd_support = entry->rsc_support; + rel->rd_indcollation = indcollation; + rel->rd_indoption = indoption; + rel->rd_opcoptions = entry->key.opcoptions; + rel->rd_supportinfo = entry->rsc_supportinfo; + + return true; +} + /* * Write out a new initialization file with the current contents * of the relcache (either shared rels or local rels, as indicated). @@ -6785,46 +7282,56 @@ write_relcache_init_file(bool shared) */ if (rel->rd_rel->relkind == RELKIND_INDEX) { + int nkeyatts = IndexRelationGetNumberOfKeyAttributes(rel); +#ifdef USE_ASSERT_CHECKING + bool isnull; + Datum indclassDatum = + fastgetattr(rel->rd_indextuple, + Anum_pg_index_indclass, + GetPgIndexDescriptor(), + &isnull); + Assert(!isnull); + Assert(((oidvector *) DatumGetPointer(indclassDatum))->values[0] != InvalidOid); +#endif + /* write the pg_index tuple */ /* we assume this was created by heap_copytuple! */ write_item(rel->rd_indextuple, HEAPTUPLESIZE + rel->rd_indextuple->t_len, fp); + /* write the vector of opcoptions values */ + for (i = 0; i < nkeyatts; i++) + { + const bytea *opt = rel->rd_opcoptions[i]; + + write_item(opt, opt ? VARSIZE(opt) : 0, fp); + } + /* write the vector of opfamily OIDs */ write_item(rel->rd_opfamily, - relform->relnatts * sizeof(Oid), + nkeyatts * sizeof(Oid), fp); /* write the vector of opcintype OIDs */ write_item(rel->rd_opcintype, - relform->relnatts * sizeof(Oid), + nkeyatts * sizeof(Oid), fp); /* write the vector of support procedure OIDs */ write_item(rel->rd_support, - relform->relnatts * (rel->rd_indam->amsupport * sizeof(RegProcedure)), + nkeyatts * (rel->rd_indam->amsupport * sizeof(RegProcedure)), fp); /* write the vector of collation OIDs */ write_item(rel->rd_indcollation, - relform->relnatts * sizeof(Oid), + nkeyatts * sizeof(Oid), fp); /* write the vector of indoption values */ write_item(rel->rd_indoption, - relform->relnatts * sizeof(int16), + nkeyatts * sizeof(int16), fp); - - Assert(rel->rd_opcoptions); - - /* write the vector of opcoptions values */ - for (i = 0; i < relform->relnatts; i++) - { - bytea *opt = rel->rd_opcoptions[i]; - - write_item(opt, opt ? VARSIZE(opt) : 0, fp); - } } } diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c index b2fd6dbb73d..fe9603690c2 100644 --- a/src/backend/utils/fmgr/fmgr.c +++ b/src/backend/utils/fmgr/fmgr.c @@ -2027,19 +2027,122 @@ get_fn_expr_variadic(FmgrInfo *flinfo) } /* - * Set options to FmgrInfo of opclass support function. + * Set options of an uncached FmgrInfo of opclass support function. * * Opclass support functions are called outside of expressions. Thanks to that * we can use fn_expr to store opclass options as bytea constant. + * + * The relcache code will populate a relation's rd_supportinfo through + * set_fn_opclass_options_bulk(). This function is therefore mostly useful + * for non-default (e.g. cross-type) support procedures. */ void -set_fn_opclass_options(FmgrInfo *flinfo, bytea *options) +set_fn_opclass_options(FmgrInfo *flinfo, const bytea *options) { + Assert(flinfo->fn_expr == NULL); flinfo->fn_expr = (Node *) makeConst(BYTEAOID, -1, InvalidOid, -1, PointerGetDatum(options), options == NULL, false); } +/* used only in set/clear_fn_opclass_options_bulk */ +static const Const OPCOPTION_NULL_CONST = { + .xpr = { .type = T_Const }, + .consttype = BYTEAOID, + .consttypmod = -1, + .constcollid = InvalidOid, + .constlen = -1, + .constvalue = (Datum) (uintptr_t) NULL, + .constisnull = true, + .constbyval = false, + .location = -1 +}; + +/* + * Set options to FmgrInfos of opclass support functions. + * + * Opclass support functions are called outside of expressions, and so don't + * use the fn_expr field. As such, we use the field to store the index' + * opclass options inside a bytea-typed Const node. + * + * Note: For NULL opcopts, a sentinel OPCOPTION_NULL_CONST is used, all other + * options get their own Const allocation in the caller's memory context. + */ +void +set_fn_opclass_options_bulk(FmgrInfo *flinfo, const bytea *const *opcopts, + int nsupport, int nkeycols, + MemoryContext shapectx) +{ + MemoryContext old_ctx; + Const *bulk = NULL; + int nonnulls = 0; + + if (nsupport == 0) + return; + + old_ctx = MemoryContextSwitchTo(shapectx); + + Assert(flinfo != NULL); + Assert(opcopts != NULL); + + for (int attno = 0; attno < nkeycols; attno++) + { + if (opcopts[attno] != NULL) + nonnulls++; + } + + if (nonnulls) + bulk = palloc0(sizeof(Const) * nonnulls); + nonnulls = 0; + + for (int attno = 0; attno < nkeycols; attno++) + { + Node *option; + const bytea *opcoption = opcopts[attno]; + + /* Avoid allocating memory in the common case of no opcoptions */ + if (opcoption == NULL) + option = (Node *) &OPCOPTION_NULL_CONST; + else + { + Const *coption = &bulk[nonnulls++]; + coption->xpr.type = T_Const; + + coption->consttype = BYTEAOID; + coption->consttypmod = -1; + coption->constcollid = InvalidOid; + coption->constlen = -1; + coption->constvalue = PointerGetDatum(opcoption); + coption->constisnull = false; + coption->constbyval = false; + + option = (Node *) coption; + } + Assert(option != NULL); + + /* + * Assign the const container and shape memory context. + * + * We push the same allocation into all support functions of the same + * attribute, so we can avoid what would be duplicate allocations, and + * minimize duplicated allocations. All infos also use the same + * MemoryContext associated with this shape, because assigning a + * Relation's rd_indexcxt would cause stale pointers when one index is + * dropped but others of the same shape survive. + */ + for (int support = 0; support < nsupport; support++) + { + Assert(flinfo->fn_expr == NULL); + + flinfo->fn_expr = option; + flinfo->fn_mcxt = shapectx; + flinfo++; + } + } + + MemoryContextSwitchTo(old_ctx); +} + /* * Check if options are defined for opclass support function. */ diff --git a/src/include/c.h b/src/include/c.h index 367b32c5b6f..5c6baedde71 100644 --- a/src/include/c.h +++ b/src/include/c.h @@ -1429,6 +1429,10 @@ typedef struct PGAlignedXLogBlock PGAlignedXLogBlock; #define unconstify(underlying_type, expr) \ (StaticAssertVariableIsOfTypeMacro(expr, const underlying_type), \ (underlying_type) (expr)) +#define unconstify_array(underlying_type, expr) \ + (StaticAssertVariableIsOfTypeMacro(expr, underlying_type const*), \ + (underlying_type*) (expr)) + #define unvolatize(underlying_type, expr) \ (StaticAssertVariableIsOfTypeMacro(expr, volatile underlying_type), \ (underlying_type) (expr)) diff --git a/src/include/fmgr.h b/src/include/fmgr.h index 04b7914095f..5d55301d707 100644 --- a/src/include/fmgr.h +++ b/src/include/fmgr.h @@ -779,7 +779,11 @@ extern bool get_call_expr_arg_stable(Node *expr, int argnum); extern bool get_fn_expr_variadic(FmgrInfo *flinfo); extern bytea *get_fn_opclass_options(FmgrInfo *flinfo); extern bool has_fn_opclass_options(FmgrInfo *flinfo); -extern void set_fn_opclass_options(FmgrInfo *flinfo, bytea *options); +extern void set_fn_opclass_options(FmgrInfo *flinfo, const bytea *options); +extern void set_fn_opclass_options_bulk(FmgrInfo *flinfo, + const bytea * const*opcopts, + int ninfos, int nkeycols, + MemoryContext shapectx); extern bool CheckFunctionValidatorAccess(Oid validatorOid, Oid functionOid); /* diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h index 8615db98a55..6f4b16ab5c8 100644 --- a/src/include/nodes/pathnodes.h +++ b/src/include/nodes/pathnodes.h @@ -1392,7 +1392,7 @@ typedef struct IndexOptInfo /* do NULLs come first in the sort order? or NULL if partitioned index */ bool *nulls_first pg_node_attr(array_size(nkeycolumns)); /* opclass-specific options for columns */ - bytea **opclassoptions pg_node_attr(read_write_ignore); + const bytea *const *opclassoptions pg_node_attr(read_write_ignore); /* which index cols can be returned in an index-only scan? */ bool *canreturn pg_node_attr(array_size(ncolumns)); /* OID of the access method (in pg_am) */ diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 2fdb88fa28c..c38bc900087 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -215,7 +215,7 @@ typedef struct RelationData Oid *rd_exclprocs; /* OIDs of exclusion ops' procs, if any */ uint16 *rd_exclstrats; /* exclusion ops' strategy numbers, if any */ Oid *rd_indcollation; /* OIDs of index collations */ - bytea **rd_opcoptions; /* parsed opclass-specific options */ + const bytea *const *rd_opcoptions; /* parsed opclass-specific options */ /* * rd_amcache is available for index and table AMs to cache private data diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h index 3e97da6a538..e1f51c4261e 100644 --- a/src/include/utils/relcache.h +++ b/src/include/utils/relcache.h @@ -60,7 +60,7 @@ extern Oid RelationGetToastChunkIdType(Relation relation); extern List *RelationGetIndexExpressions(Relation relation); extern List *RelationGetDummyIndexExpressions(Relation relation); extern List *RelationGetIndexPredicate(Relation relation); -extern bytea **RelationGetIndexAttOptions(Relation relation, bool copy); +extern const bytea *const *RelationGetIndexAttOptions(Relation relation, bool copy); /* * Which set of columns to return by RelationGetIndexAttrBitmap. -- 2.54.0 (Apple Git-157)
From 303a819e7f6a880127424f36d3d3bed736dae79b Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 31 Aug 2026 22:22:45 +0200 Subject: [PATCH v5 5/6] Relcache: Use System contexts for "index info" and "index shape" This avoids aset's block-level overheads and enables the use of the system allocator, which frequently has slightly better memory performance/efficiency for longer-lived allocations. --- src/backend/utils/cache/relcache.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 99c8aec14d1..c9cdf566e64 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -1528,9 +1528,8 @@ RelationInitIndexAccessInfo(Relation relation) * a context, and not just a couple of pallocs, is so that we won't leak * any subsidiary info attached to fmgr lookup records. */ - indexcxt = AllocSetContextCreate(CacheMemoryContext, - "index info", - ALLOCSET_SMALL_SIZES); + indexcxt = SystemContextCreate(CacheMemoryContext, "index info"); + relation->rd_indexcxt = indexcxt; MemoryContextCopyAndSetIdentifier(indexcxt, RelationGetRelationName(relation)); @@ -1700,8 +1699,7 @@ IndexSupportInitialize(oidvector *indclass, AttrNumber nKeyAtts, FmgrInfo *supportinfo; MemoryContext shapectx; - shapectx = AllocSetContextCreate(RelShapeContext, "index shape", - ALLOCSET_SMALL_SIZES); + shapectx = SystemContextCreate(RelShapeContext, "index shape"); if (maxSupportNumber > 0) { @@ -7000,8 +6998,7 @@ load_relcache_init_index(Relation rel, FILE *fp) * prepare index info context --- parameters should match * RelationInitIndexAccessInfo */ - indexcxt = AllocSetContextCreate(CacheMemoryContext, "index info", - ALLOCSET_SMALL_SIZES); + indexcxt = SystemContextCreate(CacheMemoryContext, "index info"); rel->rd_indexcxt = indexcxt; MemoryContextCopyAndSetIdentifier(indexcxt, RelationGetRelationName(rel)); @@ -7090,8 +7087,7 @@ load_relcache_init_index(Relation rel, FILE *fp) entry->rsc_refcount = 1; - idxshapectx = AllocSetContextCreate(RelShapeContext, "index shape", - ALLOCSET_SMALL_SIZES); + idxshapectx = SystemContextCreate(RelShapeContext, "index shape"); total_alloc = sizeof(FmgrInfo) * key.nsupport * key.nkeyatts; total_alloc += sizeof(Oid) * 2 * key.nkeyatts; -- 2.54.0 (Apple Git-157)
From a5ba6cb3c608d531199d5368307e50d54a9a42c2 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Tue, 6 Oct 2026 16:29:30 +0200 Subject: [PATCH v5 6/6] Relcache: Reduce number of allocations into shape cache System contexts have low per-context overhead, but significant overhead per allocation. To reduce the total overhead, we allocate most shape arrays in one go, so that we avoid the overhead of multiple allocations. --- src/backend/utils/cache/relcache.c | 40 +++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 9 deletions(-) diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index c9cdf566e64..ef23417ce8c 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -1697,28 +1697,50 @@ IndexSupportInitialize(oidvector *indclass, AttrNumber nKeyAtts, Oid *opcInType; RegProcedure *support; FmgrInfo *supportinfo; + size_t bulkallocSize = 0; + char *bulkalloc; MemoryContext shapectx; + int nprocs = maxSupportNumber * nKeyAtts; + + /* supportinfo */ + bulkallocSize += sizeof(FmgrInfo) * nprocs; + /* support */ + bulkallocSize += sizeof(RegProcedure) * nprocs; + /* opcInType */ + bulkallocSize += sizeof(Oid) * nKeyAtts; + /* opFamily */ + bulkallocSize += sizeof(Oid) * nKeyAtts; shapectx = SystemContextCreate(RelShapeContext, "index shape"); + bulkalloc = MemoryContextAllocZero(shapectx, bulkallocSize); + if (maxSupportNumber > 0) { - int nprocs = maxSupportNumber * nKeyAtts; - supportinfo = (FmgrInfo *) - MemoryContextAllocZero(shapectx, nprocs * sizeof(FmgrInfo)); - support = (RegProcedure *) - MemoryContextAllocZero(shapectx, nprocs * sizeof(RegProcedure)); + supportinfo = (FmgrInfo *) bulkalloc; + bulkalloc += nprocs * sizeof(FmgrInfo); + bulkallocSize -= nprocs * sizeof(FmgrInfo); + + support = (RegProcedure *) bulkalloc; + bulkalloc += nprocs * sizeof(RegProcedure); + bulkallocSize -= nprocs * sizeof(RegProcedure); } else { + Assert(nprocs == 0); supportinfo = NULL; support = NULL; } - opFamily = (Oid *) - MemoryContextAllocZero(shapectx, nKeyAtts * sizeof(Oid)); - opcInType = (Oid *) - MemoryContextAllocZero(shapectx, nKeyAtts * sizeof(Oid)); + opFamily = (Oid *) bulkalloc; + bulkalloc += nKeyAtts * sizeof(Oid); + bulkallocSize -= nKeyAtts * sizeof(Oid); + + opcInType = (Oid *) bulkalloc; + bulkalloc += nKeyAtts * sizeof(Oid); + bulkallocSize -= nKeyAtts * sizeof(Oid); + + Assert(bulkallocSize == 0); for (int attIndex = 0; attIndex < nKeyAtts; attIndex++) { -- 2.54.0 (Apple Git-157)
From b323b413f7d75162077352f3eca2ad9bf836b502 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Thu, 26 Feb 2026 15:24:03 -0500 Subject: [PATCH v5 4/6] MemCTX: Add minimal system context type This context just defers allocations to the system malloc. This makes individual allocations more expensive to do, but the overhead of maintaining the allocations over long periods of time is considered to be lower. Co-authored-by: Matthias van de Meent <[email protected]> Co-authored-by: Andres Freund <[email protected]> Reviewed-by: David Rowley <[email protected]> --- src/backend/nodes/gen_node_support.pl | 2 +- src/backend/utils/adt/mcxtfuncs.c | 3 + src/backend/utils/mmgr/Makefile | 3 +- src/backend/utils/mmgr/README | 15 +- src/backend/utils/mmgr/mcxt.c | 14 +- src/backend/utils/mmgr/meson.build | 1 + src/backend/utils/mmgr/systemctx.c | 523 ++++++++++++++++++++++++++ src/include/nodes/memnodes.h | 3 +- src/include/utils/memutils.h | 4 + src/include/utils/memutils_internal.h | 18 +- src/tools/pgindent/typedefs.list | 2 + 11 files changed, 581 insertions(+), 7 deletions(-) create mode 100644 src/backend/utils/mmgr/systemctx.c diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl index 0b766272018..6432a908731 100644 --- a/src/backend/nodes/gen_node_support.pl +++ b/src/backend/nodes/gen_node_support.pl @@ -136,7 +136,7 @@ my @abstract_types = qw(Node); # they otherwise don't participate in node support. my @extra_tags = qw( IntList OidList XidList - AllocSetContext GenerationContext SlabContext BumpContext + AllocSetContext GenerationContext SlabContext BumpContext SystemContext TIDBitmap WindowObjectData ); diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c index 1a4dbbeb8db..ffc26fbdcd0 100644 --- a/src/backend/utils/adt/mcxtfuncs.c +++ b/src/backend/utils/adt/mcxtfuncs.c @@ -160,6 +160,9 @@ PutMemoryContextsStatsTupleStore(Tuplestorestate *tupstore, case T_BumpContext: type = "Bump"; break; + case T_SystemContext: + type = "SystemContext"; + break; default: type = "???"; break; diff --git a/src/backend/utils/mmgr/Makefile b/src/backend/utils/mmgr/Makefile index 01a1fb85270..44a719ff279 100644 --- a/src/backend/utils/mmgr/Makefile +++ b/src/backend/utils/mmgr/Makefile @@ -22,6 +22,7 @@ OBJS = \ mcxt.o \ memdebug.o \ portalmem.o \ - slab.o + slab.o \ + systemctx.o include $(top_srcdir)/src/backend/common.mk diff --git a/src/backend/utils/mmgr/README b/src/backend/utils/mmgr/README index 695088bb66d..59105fe0966 100644 --- a/src/backend/utils/mmgr/README +++ b/src/backend/utils/mmgr/README @@ -471,8 +471,8 @@ thrashing. Alternative Memory Context Implementations ------------------------------------------ -aset.c (AllocSetContext) is our default general-purpose allocator. Three other -allocator types also exist which are special-purpose: +aset.c (AllocSetContext) is our default general-purpose allocator. Several +other allocator types also exist which are special-purpose: * slab.c (SlabContext) is designed for allocations of fixed-sized chunks. The fixed chunk size must be specified when creating the context. @@ -495,6 +495,17 @@ allocator types also exist which are special-purpose: allocations. Blocks are only free'd back to the operating system when the context is reset or deleted. +* systemctx.c (SystemContext) is best suited for cases that require long-lived + allocations, but have minimal allocation traffic that doesn't require the + lowest latencies. Because headers for each allocation are much larger than + with other context types, this context type is not well suited when many + small allocations are required. Every allocation results in a malloc() + call, and every pfree results in a free(). The most suitable use cases are + for contexts that require very few and long-lived allocations. The memory + overhead for the context itself is much smaller than with other context + types, so this implementation can be suitable when a large number of + contexts are required. + For further details, please read the header comment in the corresponding .c file. diff --git a/src/backend/utils/mmgr/mcxt.c b/src/backend/utils/mmgr/mcxt.c index 917cc0ac771..8f73e72b8d8 100644 --- a/src/backend/utils/mmgr/mcxt.c +++ b/src/backend/utils/mmgr/mcxt.c @@ -132,6 +132,19 @@ static const MemoryContextMethods mcxt_methods[] = { [MCTX_BUMP_ID].check = BumpCheck, #endif + /* systemctx.c */ + [MCTX_SYSTEM_ID].alloc = SystemContextAlloc, + [MCTX_SYSTEM_ID].free_p = SystemContextFree, + [MCTX_SYSTEM_ID].realloc = SystemContextRealloc, + [MCTX_SYSTEM_ID].reset = SystemContextReset, + [MCTX_SYSTEM_ID].delete_context = SystemContextDelete, + [MCTX_SYSTEM_ID].get_chunk_context = SystemContextGetChunkContext, + [MCTX_SYSTEM_ID].get_chunk_space = SystemContextGetChunkSpace, + [MCTX_SYSTEM_ID].is_empty = SystemContextIsEmpty, + [MCTX_SYSTEM_ID].stats = SystemContextStats, +#ifdef MEMORY_CONTEXT_CHECKING + [MCTX_SYSTEM_ID].check = SystemContextCheck, +#endif /* * Reserved and unused IDs should have dummy entries here. This allows us @@ -141,7 +154,6 @@ static const MemoryContextMethods mcxt_methods[] = { */ BOGUS_MCTX(MCTX_1_RESERVED_GLIBC_ID), BOGUS_MCTX(MCTX_2_RESERVED_GLIBC_ID), - BOGUS_MCTX(MCTX_8_UNUSED_ID), BOGUS_MCTX(MCTX_9_UNUSED_ID), BOGUS_MCTX(MCTX_10_UNUSED_ID), BOGUS_MCTX(MCTX_11_UNUSED_ID), diff --git a/src/backend/utils/mmgr/meson.build b/src/backend/utils/mmgr/meson.build index 61837e9f91b..65e55f430ea 100644 --- a/src/backend/utils/mmgr/meson.build +++ b/src/backend/utils/mmgr/meson.build @@ -11,4 +11,5 @@ backend_sources += files( 'memdebug.c', 'portalmem.c', 'slab.c', + 'systemctx.c', ) diff --git a/src/backend/utils/mmgr/systemctx.c b/src/backend/utils/mmgr/systemctx.c new file mode 100644 index 00000000000..1de718130f4 --- /dev/null +++ b/src/backend/utils/mmgr/systemctx.c @@ -0,0 +1,523 @@ +/*------------------------------------------------------------------------- + * + * systemctx.c + * System context allocator definitions. + * + * SystemContext has no concept of grouping several allocations into blocks + * like many of the other allocators do. Each palloc() results in a malloc(), + * and each pfree() results in a free(), therefore, this isn't suitable for + * cases that require many allocations. The most suitable use cases are for + * contexts that require very few and long-lived allocations. + * + * The memory overhead for the context itself is much smaller than with other + * context types, so this implementation can be suitable when a large number + * of contexts are required. + * + * Portions Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/backend/utils/mmgr/systemctx.c + * + * NOTE: Allocations are MAXALIGNed. + * + *------------------------------------------------------------------------- + */ +#include "postgres.h" + +#include "lib/ilist.h" +#include "utils/memdebug.h" +#include "utils/memutils.h" +#include "utils/memutils_memorychunk.h" +#include "utils/memutils_internal.h" + +/* Struct definition for the System memory context. */ +typedef struct SystemContext +{ + MemoryContextData header; /* Standard memory-context fields */ + + dlist_head allocations; /* list of all allocated SystemBlocks */ +} SystemContext; + +/* Struct definition for SystemContext's malloc()'d header. */ +typedef struct SystemBlock +{ + dlist_node node; /* node in SystemContext->allocations */ + size_t sz; /* malloc'd size of this allocation */ + SystemContext *context; /* direct pointer to the owning context */ +} SystemBlock; + +#define SYSTEM_CONTEXTSIZE (MAXALIGN(sizeof(SystemContext))) +#define SYSTEM_BLOCKHDRSZ (MAXALIGN(sizeof(SystemBlock))) +#define SYSTEM_CHUNKHDRSZ (sizeof(MemoryChunk)) + +/* + * SystemContextIsValid + * True iff ctx is a valid SystemContext. + */ +#define SystemContextIsValid(ctx) ((ctx) && IsA(ctx, SystemContext)) + +/* + * ChunkGetBlock + * Return address of SystemBlock for given MemoryChunk. + */ +#define ChunkGetBlock(chunk) \ + ((SystemBlock *) (((char *) chunk) - SYSTEM_BLOCKHDRSZ)) + +/* + * BlockGetChunk + * Return address of MemoryChunk for given SystemBlock. + */ +#define BlockGetChunk(blk) \ + ((MemoryChunk *) (((char *) blk) + SYSTEM_BLOCKHDRSZ)) + +/* + * SystemContextCreate + * Create a SystemContext memory context + */ +MemoryContext +SystemContextCreate(MemoryContext parent, const char *name) +{ + SystemContext *ctx; + + /* + * Allocate the context struct. Unlike other memory contexts, we don't + * add additional space here for the initial block. + */ + ctx = (SystemContext *) malloc(SYSTEM_CONTEXTSIZE); + if (ctx == NULL) + { + MemoryContextStats(TopMemoryContext); + ereport(ERROR, + (errcode(ERRCODE_OUT_OF_MEMORY), + errmsg("out of memory"), + errdetail("Failed while creating memory context \"%s\".", + name))); + } + + VALGRIND_CREATE_MEMPOOL(ctx, 0, false); + VALGRIND_MEMPOOL_ALLOC(ctx, ctx, SYSTEM_CONTEXTSIZE); + + dlist_init(&ctx->allocations); + + /* Finally, do the type-independent part of context creation */ + MemoryContextCreate((MemoryContext) ctx, T_SystemContext, MCTX_SYSTEM_ID, + parent, name); + + ((MemoryContext) ctx)->mem_allocated = SYSTEM_CONTEXTSIZE; + + return (MemoryContext) ctx; +} + +/* + * SystemContextAlloc + * Alloc memory into the SystemContext memory context. + */ +void * +SystemContextAlloc(MemoryContext context, size_t size, int flags) +{ + SystemContext *ctx = (SystemContext *) context; + size_t blksize; + SystemBlock *block; + MemoryChunk *chunk; + + Assert(SystemContextIsValid(ctx)); + + /* validate 'size' is within the limits for the given 'flags' */ + MemoryContextCheckSize(context, size, flags); + + /* adjust size for sentinel byte */ +#ifdef MEMORY_CONTEXT_CHECKING + blksize = MAXALIGN(size + 1); +#else + blksize = MAXALIGN(size); +#endif + + blksize += SYSTEM_BLOCKHDRSZ + SYSTEM_CHUNKHDRSZ; + + block = (SystemBlock *) malloc(blksize); + if (block == NULL) + return MemoryContextAllocationFailure(context, size, flags); + + VALGRIND_MEMPOOL_ALLOC(ctx, block, SYSTEM_BLOCKHDRSZ); + + context->mem_allocated += blksize; + + dlist_node_init(&block->node); + + block->context = ctx; + block->sz = blksize; + + dlist_push_tail(&ctx->allocations, &block->node); + + chunk = BlockGetChunk(block); + + MemoryChunkSetHdrMaskExternal(chunk, MCTX_SYSTEM_ID); + +#ifdef MEMORY_CONTEXT_CHECKING + chunk->requested_size = size; + /* set mark to catch clobber of "unused" space */ + set_sentinel(MemoryChunkGetPointer(chunk), size); +#endif + +#ifdef RANDOMIZE_ALLOCATED_MEMORY + /* fill the allocated space with junk */ + randomize_mem((char *) MemoryChunkGetPointer(chunk), size); +#endif + + VALGRIND_MAKE_MEM_NOACCESS(chunk, SYSTEM_CHUNKHDRSZ); + + return MemoryChunkGetPointer(chunk); +} + +/* + * SystemContextFree + * Free memory from the SystemContext memory context. + */ +void +SystemContextFree(void *pointer) +{ + SystemContext *ctx; + MemoryChunk *chunk = PointerGetMemoryChunk(pointer); + SystemBlock *block; + + VALGRIND_MAKE_MEM_DEFINED(chunk, SYSTEM_CHUNKHDRSZ); + + Assert(MemoryChunkIsExternal(chunk)); + + block = ChunkGetBlock(chunk); + + ctx = block->context; + + Assert(SystemContextIsValid(ctx)); + +#ifdef MEMORY_CONTEXT_CHECKING + /* Test for someone scribbling on unused space in chunk */ + if (!sentinel_ok(pointer, chunk->requested_size)) + elog(WARNING, "detected write past chunk end in %s %p", + ctx->header.name, chunk); +#endif + + /* ok, remove block from the list, and free it */ + dlist_delete_from(&ctx->allocations, &block->node); + + ctx->header.mem_allocated -= block->sz; + +#ifdef CLOBBER_FREED_MEMORY + wipe_mem(block, block->sz); +#endif + + VALGRIND_MEMPOOL_FREE(ctx, block); + + free(block); +} + +/* + * SystemContextRealloc + * Realloc memory in the SystemContext memory context. + */ +void * +SystemContextRealloc(void *pointer, size_t size, int flags) +{ + SystemBlock *block, + *new_block; + SystemContext *ctx; + MemoryChunk *chunk = PointerGetMemoryChunk(pointer); + size_t new_blksz; + + VALGRIND_MAKE_MEM_DEFINED(chunk, SYSTEM_CHUNKHDRSZ); + + Assert(MemoryChunkIsExternal(chunk)); + + block = ChunkGetBlock(chunk); + + ctx = block->context; + + Assert(SystemContextIsValid(ctx)); + + /* only check size in paths where the limits could be hit */ + MemoryContextCheckSize((MemoryContext) ctx, size, flags); + +#ifdef MEMORY_CONTEXT_CHECKING + /* Test for someone scribbling on unused space in chunk */ + if (!sentinel_ok(pointer, chunk->requested_size)) + elog(WARNING, "detected write past chunk end in %s %p", + ctx->header.name, chunk); +#endif + +#ifdef MEMORY_CONTEXT_CHECKING + /* adjust for sentinel byte, and align */ + new_blksz = MAXALIGN(size + 1); +#else + new_blksz = MAXALIGN(size); +#endif + new_blksz += SYSTEM_BLOCKHDRSZ + SYSTEM_CHUNKHDRSZ; + + new_block = realloc(block, new_blksz); + if (new_block == NULL) + { + VALGRIND_MAKE_MEM_NOACCESS(chunk, SYSTEM_CHUNKHDRSZ); + return MemoryContextAllocationFailure((MemoryContext) ctx, size, flags); + } + + /* + * Temporarily unlink the allocation, because the address of the + * allocation may change. + */ + dlist_delete_from_thoroughly(&ctx->allocations, &new_block->node); + + /* Move the block-header vchunk */ + VALGRIND_MEMPOOL_CHANGE(ctx, block, new_block, SYSTEM_BLOCKHDRSZ); + block = new_block; + + dlist_push_tail(&ctx->allocations, &block->node); + + chunk = BlockGetChunk(block); + + /* when enlarging, randomize the enlarged portion */ +#ifdef RANDOMIZE_ALLOCATED_MEMORY + if (new_blksz > block->sz) + randomize_mem(((char *) block) + block->sz, new_blksz - block->sz); +#endif + + /* update chunk's size data */ +#ifdef MEMORY_CONTEXT_CHECKING + chunk->requested_size = size; + set_sentinel(MemoryChunkGetPointer(chunk), size); +#endif + + /* adjust memory accounting for the context */ + ctx->header.mem_allocated -= block->sz; + ctx->header.mem_allocated += new_blksz; + + /* and adjust the size indicator of the block header */ + block->sz = new_blksz; + + /* Disallow access to the chunk header. */ + VALGRIND_MAKE_MEM_NOACCESS(chunk, SYSTEM_CHUNKHDRSZ); + + return MemoryChunkGetPointer(chunk); +} + +/* + * SystemContextReset + * Reset the SystemContext memory context. + */ +void +SystemContextReset(MemoryContext context) +{ + SystemContext *ctx = (SystemContext *) context; + + Assert(SystemContextIsValid(ctx)); + + while (!dlist_is_empty(&ctx->allocations)) + { + SystemBlock *block = + dlist_container(SystemBlock, node, + dlist_pop_head_node(&ctx->allocations)); + size_t blksize = block->sz; + +#ifdef CLOBBER_FREED_MEMORY + wipe_mem(block, blksize); +#endif + + ctx->header.mem_allocated -= blksize; + + /* + * We need to free the block header's vchunk explicitly, although the + * user-data vchunks within will go away in the TRIM below. Otherwise + * Valgrind complains about leaked allocations. + */ + VALGRIND_MEMPOOL_FREE(ctx, block); + + free(block); + } + + Assert(ctx->header.mem_allocated == SYSTEM_CONTEXTSIZE); + Assert(SystemContextIsEmpty(context)); + + /* + * Instruct Valgrind to throw away all the vchunks associated with this + * context, except for the one covering the SystemContext. This gets rid + * of the vchunks for whatever user data is getting discarded by the + * context reset. + */ + VALGRIND_MEMPOOL_TRIM(ctx, ctx, SYSTEM_CONTEXTSIZE); +} + +/* + * SystemContextDelete + * Delete this SystemContext memory context. + */ +void +SystemContextDelete(MemoryContext context) +{ + SystemContext *ctx = (SystemContext *) context; + + Assert(SystemContextIsValid(ctx)); + +#ifdef MEMORY_CONTEXT_CHECKING + SystemContextCheck(context); +#endif + + SystemContextReset(context); + + VALGRIND_DESTROY_MEMPOOL(context); + + free(ctx); +} + +/* + * SystemContextGetChunkContext + * Return the MemoryContext that 'pointer' belongs to. + */ +MemoryContext +SystemContextGetChunkContext(void *pointer) +{ + MemoryChunk *chunk = PointerGetMemoryChunk(pointer); + SystemBlock *block; + SystemContext *ctx; + + VALGRIND_MAKE_MEM_DEFINED(chunk, SYSTEM_CHUNKHDRSZ); + + Assert(MemoryChunkIsExternal(chunk)); + block = ChunkGetBlock(chunk); + + VALGRIND_MAKE_MEM_NOACCESS(chunk, SYSTEM_CHUNKHDRSZ); + + ctx = block->context; + + Assert(SystemContextIsValid(ctx)); + + return (MemoryContext) ctx; +} + +/* + * SystemContextGetChunkSpace + * Given a palloc'd chunk, determine the total space + * it occupies (including all memory-allocation overhead). + */ +size_t +SystemContextGetChunkSpace(void *pointer) +{ + MemoryChunk *chunk = PointerGetMemoryChunk(pointer); + SystemBlock *block; + + VALGRIND_MAKE_MEM_DEFINED(chunk, SYSTEM_CHUNKHDRSZ); + + Assert(MemoryChunkIsExternal(chunk)); + block = ChunkGetBlock(chunk); + + VALGRIND_MAKE_MEM_NOACCESS(chunk, SYSTEM_CHUNKHDRSZ); + + return block->sz; +} + +/* + * SystemContextIsEmpty + * Is the SystemContext empty of any allocated space? + */ +bool +SystemContextIsEmpty(MemoryContext context) +{ + SystemContext *ctx = (SystemContext *) context; + + Assert(SystemContextIsValid(ctx)); + + return dlist_is_empty(&ctx->allocations); +} + +/* + * SystemContextStats + * Compute stats about memory consumption of a SystemContext context. + * + * printfunc: if not NULL, pass a human-readable stats string to this. + * passthru: pass this pointer through to printfunc. + * totals: if not NULL, add stats about this context into *totals. + * print_to_stderr: print stats to stderr if true, elog otherwise. + */ +void +SystemContextStats(MemoryContext context, MemoryStatsPrintFunc printfunc, + void *passthru, MemoryContextCounters *totals, + bool print_to_stderr) +{ + SystemContext *ctx = (SystemContext *) context; + dlist_iter iter; + size_t totalspace = SYSTEM_CONTEXTSIZE; + size_t nchunks = 0; + + dlist_foreach(iter, &ctx->allocations) + { + SystemBlock *block = dlist_container(SystemBlock, node, iter.cur); + + nchunks++; + totalspace += block->sz; + } + + + if (printfunc) + { + char stats_string[200]; + + snprintf(stats_string, sizeof(stats_string), + "%zu total in %zu chunks", + totalspace, nchunks); + printfunc(context, passthru, stats_string, print_to_stderr); + } + + if (totals) + { + totals->nblocks += nchunks; + totals->totalspace += totalspace; + } +} + +#ifdef MEMORY_CONTEXT_CHECKING +/* + * SystemContextCheck + * Walk through chunks and check consistency of memory. + * + * NOTE: report errors as WARNING, *not* ERROR or FATAL. Otherwise you'll + * find yourself in an infinite loop when trouble occurs, because this + * routine will be entered again when elog cleanup tries to release memory! + */ +void +SystemContextCheck(MemoryContext context) +{ + SystemContext *ctx = (SystemContext *) context; + const char *name = context->name; + dlist_iter iter; + size_t total_allocated = SYSTEM_CONTEXTSIZE; + + /* walk all blocks in this context */ + dlist_foreach(iter, &ctx->allocations) + { + SystemBlock *block = dlist_container(SystemBlock, node, iter.cur); + MemoryChunk *chunk = BlockGetChunk(block); + void *pointer = MemoryChunkGetPointer(chunk); + size_t expected_sz; + + /* Allow access to the chunk header. */ + VALGRIND_MAKE_MEM_DEFINED(chunk, SYSTEM_CHUNKHDRSZ); + + total_allocated += block->sz; + expected_sz = MAXALIGN(chunk->requested_size + 1) + SYSTEM_BLOCKHDRSZ + + SYSTEM_CHUNKHDRSZ; + + if (block->sz != expected_sz) + elog(WARNING, "problem in SystemContext %s: chunk %p has an unexpected size; block->sz is %zu but expected size is %zu bytes", + name, chunk, block->sz, expected_sz); + + if (!sentinel_ok(pointer, chunk->requested_size)) + elog(WARNING, "problem in SystemContext %s: detected write past chunk end in block %p, chunk %p", + name, block, chunk); + + VALGRIND_MAKE_MEM_NOACCESS(chunk, SYSTEM_CHUNKHDRSZ); + } + + Assert(total_allocated == context->mem_allocated); + + if (total_allocated != ctx->header.mem_allocated) + elog(WARNING, "problem in SystemContext %s: sum of memory %zu does not match header's %zu", + name, total_allocated, ctx->header.mem_allocated); +} +#endif diff --git a/src/include/nodes/memnodes.h b/src/include/nodes/memnodes.h index 4b24778fe1e..87a7b3b3e57 100644 --- a/src/include/nodes/memnodes.h +++ b/src/include/nodes/memnodes.h @@ -147,6 +147,7 @@ typedef struct MemoryContextData (IsA((context), AllocSetContext) || \ IsA((context), SlabContext) || \ IsA((context), GenerationContext) || \ - IsA((context), BumpContext))) + IsA((context), BumpContext) || \ + IsA((context), SystemContext))) #endif /* MEMNODES_H */ diff --git a/src/include/utils/memutils.h b/src/include/utils/memutils.h index 38036c7c703..25078ae39c8 100644 --- a/src/include/utils/memutils.h +++ b/src/include/utils/memutils.h @@ -150,6 +150,10 @@ extern MemoryContext BumpContextCreate(MemoryContext parent, Size initBlockSize, Size maxBlockSize); +/* systemctx.c */ +extern MemoryContext SystemContextCreate(MemoryContext parent, + const char *name); + /* * Recommended default alloc parameters, suitable for "ordinary" contexts * that might hold quite a lot of data. diff --git a/src/include/utils/memutils_internal.h b/src/include/utils/memutils_internal.h index 475e91b336b..97ebec7a493 100644 --- a/src/include/utils/memutils_internal.h +++ b/src/include/utils/memutils_internal.h @@ -95,6 +95,22 @@ extern void BumpStats(MemoryContext context, MemoryStatsPrintFunc printfunc, extern void BumpCheck(MemoryContext context); #endif +/* These functions implement the MemoryContext API for the System context. */ +extern void *SystemContextAlloc(MemoryContext context, size_t size, int flags); +extern void SystemContextFree(void *pointer); +extern void *SystemContextRealloc(void *pointer, size_t size, int flags); +extern void SystemContextReset(MemoryContext context); +extern void SystemContextDelete(MemoryContext context); +extern MemoryContext SystemContextGetChunkContext(void *pointer); +extern size_t SystemContextGetChunkSpace(void *pointer); +extern bool SystemContextIsEmpty(MemoryContext context); +extern void SystemContextStats(MemoryContext context, MemoryStatsPrintFunc printfunc, + void *passthru, MemoryContextCounters *totals, + bool print_to_stderr); +#ifdef MEMORY_CONTEXT_CHECKING +extern void SystemContextCheck(MemoryContext context); +#endif + /* * How many extra bytes do we need to request in order to ensure that we can * align a pointer to 'alignto'. Since palloc'd pointers are already aligned @@ -128,7 +144,7 @@ typedef enum MemoryContextMethodID MCTX_SLAB_ID, MCTX_ALIGNED_REDIRECT_ID, MCTX_BUMP_ID, - MCTX_8_UNUSED_ID, + MCTX_SYSTEM_ID, MCTX_9_UNUSED_ID, MCTX_10_UNUSED_ID, MCTX_11_UNUSED_ID, diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index a7a8dbdd23f..e512b3e522c 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -3059,6 +3059,8 @@ SysScanDesc SysScanDescData SyscacheCallbackFunction SysloggerStartupData +SystemBlock +SystemContext SystemRowsSamplerData SystemSamplerData SystemTimeSamplerData -- 2.54.0 (Apple Git-157)
From 04fa9a2992114aa1ce67fdf19a4bcfdcd7b1ef3d Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Sat, 31 Jan 2026 15:03:26 +0100 Subject: [PATCH v5 2/6] relcache: refactor RelationInitIndexAccessInfo This prepares RelationInitIndexAccessInfo and IndexSupportInitialize for future optimizations that enable deduplication of various key attributes. --- src/backend/utils/cache/relcache.c | 87 +++++++++++++++--------------- 1 file changed, 45 insertions(+), 42 deletions(-) diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index b710b577be7..93e06206db4 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -314,12 +314,9 @@ static int AttrDefaultCmp(const void *a, const void *b); static void CheckNNConstraintFetch(Relation relation); static int CheckConstraintCmp(const void *a, const void *b); static void InitIndexAmRoutine(Relation relation); -static void IndexSupportInitialize(oidvector *indclass, - RegProcedure *indexSupport, - Oid *opFamily, - Oid *opcInType, - StrategyNumber maxSupportNumber, - AttrNumber maxAttributeNumber); +static void IndexSupportInitialize(oidvector *indclass, AttrNumber nKeyAtts, StrategyNumber maxSupportNumber, + MemoryContext indexcxt, const Oid **rd_opfamily, const Oid **rd_opcintype, + const RegProcedure **rd_support, FmgrInfo **rd_supportinfo); static OpClassCacheEnt *LookupOpclassInfo(Oid operatorClassOid, StrategyNumber numSupport); static void RelationCacheInitFileRemoveInDir(const char *tblspcpath); @@ -1454,9 +1451,6 @@ RelationInitIndexAccessInfo(Relation relation) int indnatts; int indnkeyatts; uint16 amsupport; - Oid *opfamily; - Oid *opcintype; - RegProcedure *support; /* * Make a copy of the pg_index entry for the index. Since pg_index @@ -1512,27 +1506,11 @@ RelationInitIndexAccessInfo(Relation relation) /* * Allocate arrays to hold data. Opclasses are not used for included * columns, so allocate them for indnkeyatts only. + * + * Note that rd_opcintype/_opfamily/_support/_supportinfo are filled + * in by IndexSupportInitialize. */ - opfamily = (Oid *) - MemoryContextAllocZero(indexcxt, indnkeyatts * sizeof(Oid)); - opcintype = (Oid *) - MemoryContextAllocZero(indexcxt, indnkeyatts * sizeof(Oid)); - amsupport = relation->rd_indam->amsupport; - if (amsupport > 0) - { - int nsupport = indnatts * amsupport; - - support = (RegProcedure *) - MemoryContextAllocZero(indexcxt, nsupport * sizeof(RegProcedure)); - relation->rd_supportinfo = (FmgrInfo *) - MemoryContextAllocZero(indexcxt, nsupport * sizeof(FmgrInfo)); - } - else - { - support = NULL; - relation->rd_supportinfo = NULL; - } relation->rd_indcollation = (Oid *) MemoryContextAllocZero(indexcxt, indnkeyatts * sizeof(Oid)); @@ -1570,12 +1548,12 @@ RelationInitIndexAccessInfo(Relation relation) * opfamilies and opclass input types. (aminfo and supportinfo are left * as zeroes, and are filled on-the-fly when used) */ - IndexSupportInitialize(indclass, support, opfamily, opcintype, - amsupport, indnkeyatts); + IndexSupportInitialize(indclass, indnkeyatts, amsupport, indexcxt, + &relation->rd_opfamily, + &relation->rd_opcintype, + &relation->rd_support, + &relation->rd_supportinfo); - relation->rd_opfamily = opfamily; - relation->rd_opcintype = opcintype; - relation->rd_support = support; /* * Similarly extract indoption and copy it to the cache entry */ @@ -1614,16 +1592,38 @@ RelationInitIndexAccessInfo(Relation relation) * for the index and access method. */ static void -IndexSupportInitialize(oidvector *indclass, - RegProcedure *indexSupport, - Oid *opFamily, - Oid *opcInType, +IndexSupportInitialize(oidvector *indclass, AttrNumber nKeyAtts, StrategyNumber maxSupportNumber, - AttrNumber maxAttributeNumber) + MemoryContext indexcxt, + const Oid **rd_opfamily, + const Oid **rd_opcintype, + const RegProcedure **rd_support, + FmgrInfo **rd_supportinfo) { - int attIndex; + Oid *opFamily; + Oid *opcInType; + RegProcedure *indexSupport; - for (attIndex = 0; attIndex < maxAttributeNumber; attIndex++) + if (maxSupportNumber > 0) + { + int nprocs = maxSupportNumber * nKeyAtts; + *rd_supportinfo = (FmgrInfo *) + MemoryContextAllocZero(indexcxt, nprocs * sizeof(FmgrInfo)); + indexSupport = (RegProcedure *) + MemoryContextAllocZero(indexcxt, nprocs * sizeof(RegProcedure)); + } + else + { + *rd_supportinfo = NULL; + indexSupport = NULL; + } + + opFamily = (Oid *) + MemoryContextAllocZero(indexcxt, nKeyAtts * sizeof(Oid)); + opcInType = (Oid *) + MemoryContextAllocZero(indexcxt, nKeyAtts * sizeof(Oid)); + + for (int attIndex = 0; attIndex < nKeyAtts; attIndex++) { OpClassCacheEnt *opcentry; @@ -1642,6 +1642,10 @@ IndexSupportInitialize(oidvector *indclass, opcentry->supportProcs, maxSupportNumber * sizeof(RegProcedure)); } + + *rd_support = indexSupport; + *rd_opcintype = opcInType; + *rd_opfamily = opFamily; } /* @@ -6068,8 +6072,7 @@ RelationGetIndexAttOptions(Relation relation, bool copy) MemoryContext oldcxt; bytea **opts = relation->rd_opcoptions; Oid relid = RelationGetRelid(relation); - int natts = RelationGetNumberOfAttributes(relation); /* XXX - * IndexRelationGetNumberOfKeyAttributes */ + int natts = IndexRelationGetNumberOfKeyAttributes(relation); int i; /* Try to copy cached options. */ -- 2.54.0 (Apple Git-157)
