pgsql: Optimize GenerationAlloc() and SlabAlloc()
Optimize GenerationAlloc() and SlabAlloc() In a similar effort to 413c18401, separate out the hot and cold paths in GenerationAlloc() and SlabAlloc() to avoid having to setup the stack frame for the hot path. This additionally adjusts how we use the GenerationContext's freeblock. Freeblock, when set, is now always empty and we only switch to using it when the current allocation request finds the current block does not have enough space and the freeblock is large enough to accomodate the allocation. This commit also adjusts GenerationFree() so that if we pfree the final allocation in the current generation block, we now mark that block as empty and keep it as the current block. Previously we free'd that block and set the current block to NULL. Doing that meant we needed a special case in GenerationAlloc to check if GenerationContext.block was NULL. So this both reduces free/malloc calls and reduces the work done in GenerationAlloc(). In passing, improve some comments in aset.c Discussion: https://postgr.es/m/CAApHDvpHVSJqqb4B4OZLixr=cotkq-ekkbwzqvzvo_biyvu...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/a0cd95448067273a5cf92ad578a1e2de3b62aa2f Modified Files -- src/backend/utils/mmgr/aset.c | 16 +- src/backend/utils/mmgr/generation.c | 415 src/backend/utils/mmgr/slab.c | 232 3 files changed, 386 insertions(+), 277 deletions(-)
pgsql: Support partition pruning on boolcol IS [NOT] UNKNOWN
Support partition pruning on boolcol IS [NOT] UNKNOWN While working on 4c2369ac5, I noticed we went out of our way not to support clauses on boolean partitioned tables in the form of "IS UNKNOWN" and "IS NOT UNKNOWN". It's almost as much code to disallow this as it is to allow it, so let's allow it. Discussion: https://postgr.es/m/CAApHDvobKtcN6+xOuOfcutfp6T7jP=JPA9y3=maeqnukdds...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/07c36c1333e00df5a4baf6b04ba2f3f300f57d69 Modified Files -- src/backend/partitioning/partprune.c | 71 +++ src/test/regress/expected/partition_prune.out | 56 + src/test/regress/sql/partition_prune.sql | 7 +++ 3 files changed, 93 insertions(+), 41 deletions(-)
pgsql: Add PostgreSQL::Test::Cluster::wait_for_event()
Add PostgreSQL::Test::Cluster::wait_for_event() Per a demand from the author and the reviewer of this commit, this adds to Cluster.pm a helper routine that can be used to monitor when a process reaches a wanted wait event. This can be used in combination with the module injection_points for the "wait" callback, though it is not limited to it as this monitors pg_stat_activity for a wait_event and a backend_type. Author: Bertrand Drouvot Reviewed-by: Andrey Borodin Discussion: https://postgr.es/m/zebb4rmpez06t...@ip-10-97-1-34.eu-west-3.compute.internal Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/eca2c1ea85eb22716f93339f4e7cec3145c25723 Modified Files -- src/test/perl/PostgreSQL/Test/Cluster.pm | 23 +++ src/test/recovery/t/041_checkpoint_at_promote.pl | 8 +--- 2 files changed, 24 insertions(+), 7 deletions(-)
pgsql: Add regression test for restart points during promotion
Add regression test for restart points during promotion This test serves as a way to demonstrate how to use the features introduced in 37b369dc67bc, while providing coverage for 7863ee4def65 that caused the startup process to throw "PANIC: could not locate a valid checkpoint record" when starting recovery. The test checks that a node is able to properly restart following a crash when a restart point was finishing across a promotion, with an injection point added in the middle of CreateRestartPoint() to stop the restartpoint in flight. Note that this test fails when 7863ee4def65 is reverted. Kyotaro Horiguchi is the original author of this test, that has been originally posted on the thread where 7863ee4def65 was discussed. I have just upgraded and polished it to rely on injection points, making it much cheaper to reproduce the failure. This test requires injection points to be enabled in the builds, hence meson and ./configure need an update to pass this knowledge down to the test. The name of the new injection point follows the same naming convention as 6a1ea02c491d. The Makefile's EXTRA_INSTALL of recovery TAP tests is updated to include modules/injection_points. Author: Kyotaro Horiguchi, Michael Paquier Reviewed-by: Andrey Borodin, Bertrand Drouvot Discussion: https://postgr.es/m/zdluxbk5hgpol...@paquier.xyz Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/6782709df81f6c68450172605907fa9ff2b7f2b1 Modified Files -- src/backend/access/transam/xlog.c| 7 + src/test/recovery/Makefile | 7 +- src/test/recovery/meson.build| 4 + src/test/recovery/t/041_checkpoint_at_promote.pl | 170 +++ 4 files changed, 187 insertions(+), 1 deletion(-)
pgsql: injection_points: Add wait and wakeup of processes
injection_points: Add wait and wakeup of processes This commit adds two features to the in-core module for injection points: - A new callback called "wait" that can be attached to an injection point to make it wait. - A new SQL function to update the shared state and broadcast the update using a condition variable. This function uses an input an injection point name. This offers the possibility to stop a process in flight and wake it up in a controlled manner, which is useful when implementing tests that aim to trigger scenarios for race conditions (some tests are planned for integration). The logic uses a set of counters with a condition variable to monitor and broadcast the changes. Up to 8 waits can be registered in a single run, which should be plenty enough. Waits can be monitored in pg_stat_activity, based on the injection point name which is registered in a custom wait event under the "Extension" category. The shared memory state used by the module is registered using the DSM registry, and is optional, so there is no need to load the module with shared_preload_libraries to be able to use these features. Author: Michael Paquier Reviewed-by: Andrey Borodin, Bertrand Drouvot Discussion: https://postgr.es/m/zdluxbk5hgpol...@paquier.xyz Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/37b369dc67bc44a3aab5c07e2c0012475efd00f3 Modified Files -- .../injection_points/injection_points--1.0.sql | 10 ++ .../modules/injection_points/injection_points.c| 155 + src/tools/pgindent/typedefs.list | 1 + 3 files changed, 166 insertions(+)
pgsql: Replace BackendIds with 0-based ProcNumbers
Replace BackendIds with 0-based ProcNumbers Now that BackendId was just another index into the proc array, it was redundant with the 0-based proc numbers used in other places. Replace all usage of backend IDs with proc numbers. The only place where the term "backend id" remains is in a few pgstat functions that expose backend IDs at the SQL level. Those IDs are now in fact 0-based ProcNumbers too, but the documentation still calls them "backend ids". That term still seems appropriate to describe what the numbers are, so I let it be. One user-visible effect is that pg_temp_0 is now a valid temp schema name, for backend with ProcNumber 0. Reviewed-by: Andres Freund Discussion: https://www.postgresql.org/message-id/8171f1aa-496f-46a6-afc3-c46fe7a9b...@iki.fi Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/024c521117579a6d356050ad3d78fdc95e44eefa Modified Files -- doc/src/sgml/config.sgml | 2 +- doc/src/sgml/monitoring.sgml | 18 ++--- doc/src/sgml/storage.sgml | 2 +- doc/src/sgml/xact.sgml | 4 +- src/backend/access/transam/README | 10 +-- src/backend/access/transam/clog.c | 18 ++--- src/backend/access/transam/multixact.c | 64 src/backend/access/transam/parallel.c | 10 +-- src/backend/access/transam/twophase.c | 43 ++- src/backend/access/transam/xact.c | 9 ++- src/backend/access/transam/xlogprefetcher.c| 2 +- src/backend/access/transam/xlogutils.c | 6 +- src/backend/backup/basebackup_incremental.c| 2 +- src/backend/catalog/catalog.c | 14 ++-- src/backend/catalog/namespace.c| 30 src/backend/catalog/storage.c | 30 src/backend/commands/async.c | 89 ++ src/backend/commands/dbcommands.c | 2 +- src/backend/commands/indexcmds.c | 2 +- src/backend/commands/sequence.c| 2 +- src/backend/libpq/pqmq.c | 10 +-- src/backend/postmaster/pgarch.c| 12 +-- src/backend/postmaster/walsummarizer.c | 16 ++-- .../replication/logical/applyparallelworker.c | 4 +- src/backend/replication/slot.c | 2 +- src/backend/replication/walsender.c| 2 +- src/backend/storage/buffer/buf_init.c | 2 +- src/backend/storage/buffer/bufmgr.c| 30 src/backend/storage/buffer/localbuf.c | 6 +- src/backend/storage/ipc/procarray.c| 44 +-- src/backend/storage/ipc/procsignal.c | 36 - src/backend/storage/ipc/sinvaladt.c| 45 +-- src/backend/storage/ipc/standby.c | 6 +- src/backend/storage/lmgr/lmgr.c| 2 +- src/backend/storage/lmgr/lock.c| 26 +++ src/backend/storage/lmgr/predicate.c | 10 +-- src/backend/storage/lmgr/proc.c| 39 +- src/backend/storage/smgr/md.c | 4 +- src/backend/storage/smgr/smgr.c| 2 +- src/backend/utils/activity/backend_status.c| 50 ++-- src/backend/utils/adt/dbsize.c | 14 ++-- src/backend/utils/adt/lockfuncs.c | 13 ++-- src/backend/utils/adt/mcxtfuncs.c | 7 +- src/backend/utils/adt/pgstatfuncs.c| 50 ++-- src/backend/utils/cache/inval.c| 4 +- src/backend/utils/cache/relcache.c | 30 src/backend/utils/error/csvlog.c | 4 +- src/backend/utils/error/elog.c | 6 +- src/backend/utils/error/jsonlog.c | 4 +- src/backend/utils/init/globals.c | 6 +- src/backend/utils/time/snapmgr.c | 6 +- src/common/relpath.c | 24 +++--- src/include/access/twophase.h | 2 +- src/include/catalog/namespace.h| 3 +- src/include/common/relpath.h | 4 +- src/include/libpq/pqmq.h | 2 +- src/include/miscadmin.h| 5 -- src/include/postmaster/postmaster.h| 2 +- src/include/storage/backendid.h| 41 -- src/include/storage/lock.h | 18 ++--- src/include/storage/proc.h | 36 - src/include/storage/procarray.h| 8 +- src/include/storage/proclist.h | 38 - src/include/storage/proclist_types.h | 18 +++-- src/include/storage/procnumber.h | 43 +++ src/include/storage/procsignal.h |
pgsql: Redefine backend ID to be an index into the proc array
Redefine backend ID to be an index into the proc array Previously, backend ID was an index into the ProcState array, in the shared cache invalidation manager (sinvaladt.c). The entry in the ProcState array was reserved at backend startup by scanning the array for a free entry, and that was also when the backend got its backend ID. Things become slightly simpler if we redefine backend ID to be the index into the PGPROC array, and directly use it also as an index to the ProcState array. This uses a little more memory, as we reserve a few extra slots in the ProcState array for aux processes that don't need them, but the simplicity is worth it. Aux processes now also have a backend ID. This simplifies the reservation of BackendStatusArray and ProcSignal slots. You can now convert a backend ID into an index into the PGPROC array simply by subtracting 1. We still use 0-based "pgprocnos" in various places, for indexes into the PGPROC array, but the only difference now is that backend IDs start at 1 while pgprocnos start at 0. (The next commmit will get rid of the term "backend ID" altogether and make everything 0-based.) There is still a 'backendId' field in PGPROC, now part of 'vxid' which encapsulates the backend ID and local transaction ID together. It's needed for prepared xacts. For regular backends, the backendId is always equal to pgprocno + 1, but for prepared xact PGPROC entries, it's the ID of the original backend that processed the transaction. Reviewed-by: Andres Freund, Reid Thompson Discussion: https://www.postgresql.org/message-id/8171f1aa-496f-46a6-afc3-c46fe7a9b...@iki.fi Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ab355e3a88de745607f6dd4c21f0119b5c68f2ad Modified Files -- src/backend/access/transam/twophase.c | 37 ++--- src/backend/access/transam/xact.c | 12 +- src/backend/catalog/namespace.c | 2 +- src/backend/commands/sequence.c | 2 +- src/backend/executor/functions.c| 4 +- src/backend/postmaster/auxprocess.c | 12 +- src/backend/storage/ipc/procarray.c | 73 +- src/backend/storage/ipc/procsignal.c| 27 ++-- src/backend/storage/ipc/sinvaladt.c | 200 +--- src/backend/storage/ipc/standby.c | 1 + src/backend/storage/lmgr/lock.c | 28 ++-- src/backend/storage/lmgr/proc.c | 32 +++-- src/backend/utils/activity/backend_status.c | 52 +++- src/backend/utils/adt/lockfuncs.c | 2 +- src/backend/utils/adt/mcxtfuncs.c | 14 +- src/backend/utils/error/csvlog.c| 4 +- src/backend/utils/error/elog.c | 6 +- src/backend/utils/error/jsonlog.c | 6 +- src/backend/utils/init/postinit.c | 10 +- src/backend/utils/time/snapmgr.c| 5 +- src/include/miscadmin.h | 2 - src/include/storage/backendid.h | 12 +- src/include/storage/lock.h | 9 +- src/include/storage/proc.h | 32 - src/include/storage/procarray.h | 4 + src/include/storage/procsignal.h| 2 +- src/include/storage/sinvaladt.h | 4 - src/pl/plpgsql/src/pl_exec.c| 10 +- 28 files changed, 282 insertions(+), 322 deletions(-)
pgsql: GUC table: Add description to computed variables
GUC table: Add description to computed variables Per suggestion from Kyotaro Horiguchi Discussion: https://postgr.es/m/20240229.130404.1411153273308142188.horikyota@gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/30b8d6e4ce1112168ddfe8cdbba76fbefd304b34 Modified Files -- src/backend/utils/misc/guc_tables.c | 8 1 file changed, 4 insertions(+), 4 deletions(-)
