Hi, On Wed, Feb 04, 2026 at 10:26:35AM -0600, Sami Imseih wrote: > > Will address below in the comments. > > Sorry there was one more comment I forgot about. > > in v5-0004, instead of `flush_anytime_cb`, can't we > just pass the anytime flag to the `flush_pending_cb` and > take care of what needs to be flushed there? > > It will be up to the author of the kind if they want to further > split work in multiple functions, but we could just have a > single callback that is ANYTIME aware.
Thank you Sami and Zsolt for the reviews! PFA attached v6, addressing the reviews comments. Main changes are: 0001: - Adds a pgstat_schedule_anytime_update() macro to schedule the next anytime flush - The flush_pending_cb and flush_static_cb callbacks now receive an anytime_only boolean parameter. Most of the time it's not used (except for assertions), but it's preparatory work for moving the relations stats to anytime (without introducing a new callback) in 0005. 0002: Add some tests for anytime flush on custom stats. It adds a few functions: we could/may want to avoid some duplicate code and update the version to 1.1 though. For the former, I don't have a strong opinion and for the later I'm not sure (for example 07ff701dbd53 did not bump the version). 0005: Get rid of the FLUSH_MIXED mode, the new anytime callback and the pgstat_report_mixed_anytime variable introduced in v5. Instead v6: - change RELATION and DATABASE kinds from FLUSH_AT_TXN_BOUNDARY to FLUSH_ANYTIME - Modify pgstat_relation_flush_cb() to handle anytime_only parameter: when true, then flush only non-transactional stats and when false, then flush all the stats Remark: I did not add any additional checks in pgstat_report_anytime_stat(), since we enter it for good reason: at least one the flush has to be triggered. I'm not sure it's worth to add more check to try to discard one of them. The flush functions return early if they don't have anything to flush anyway. So the gain would be to try to avoid one function call at pgstat_flush_interval interval, but we would need to maitain the variable(s) and avoid possible race conditions with pgstat_report_stat(). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com
>From b2e59b407ca3fd60f5f917c9ec5f93047d0ac649 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot <[email protected]> Date: Mon, 5 Jan 2026 09:41:39 +0000 Subject: [PATCH v6 1/5] Add pgstat_report_anytime_stat() for periodic stats flushing Long running transactions can accumulate significant statistics (WAL, IO, ...) that remain unflushed until the transaction ends. This delays visibility of resource usage in monitoring views like pg_stat_io and pg_stat_wal and produces spikes when flushed. This commit introduces pgstat_report_anytime_stat(), which flushes non transactional statistics even inside active transactions. A new timeout handler fires every second (if enabled while adding pending stats) to call this function, ensuring timely stats visibility without waiting for transaction completion. Implementation details: - Add PgStat_FlushMode enum to classify stats kinds: * FLUSH_ANYTIME: Stats that can always be flushed (WAL, IO, ...) * FLUSH_AT_TXN_BOUNDARY: Stats requiring transaction boundaries - Modify pgstat_flush_pending_entries() and pgstat_flush_fixed_stats() to accept a boolean anytime_only parameter: * When false: flushes all stats (existing behavior) * When true: flushes only FLUSH_ANYTIME stats and skips FLUSH_AT_TXN_BOUNDARY stats - The flush_pending_cb and flush_static_cb callbacks now receive an anytime_only boolean parameter. Most of the time it's not used (except for assertions), but it's preparatory work for moving the relations stats to anytime (without introducin a new callback). - Add pgstat_schedule_anytime_update() macro to schedule the next anytime flush, relying on PGSTAT_MIN_INTERVAL The force parameter in pgstat_report_anytime_stat() is currently unused (always called with force=false) but reserved for future use cases requiring immediate flushing. --- src/backend/access/transam/xlog.c | 6 + src/backend/postmaster/bgwriter.c | 9 +- src/backend/postmaster/checkpointer.c | 10 +- src/backend/postmaster/startup.c | 2 + src/backend/postmaster/walsummarizer.c | 9 +- src/backend/postmaster/walwriter.c | 9 +- src/backend/replication/walreceiver.c | 9 +- src/backend/replication/walsender.c | 8 +- src/backend/tcop/postgres.c | 12 ++ src/backend/utils/activity/pgstat.c | 112 ++++++++++++++---- src/backend/utils/activity/pgstat_backend.c | 13 +- src/backend/utils/activity/pgstat_bgwriter.c | 2 +- .../utils/activity/pgstat_checkpointer.c | 2 +- src/backend/utils/activity/pgstat_database.c | 2 +- src/backend/utils/activity/pgstat_function.c | 4 +- src/backend/utils/activity/pgstat_io.c | 10 +- src/backend/utils/activity/pgstat_relation.c | 12 +- src/backend/utils/activity/pgstat_slru.c | 6 +- .../utils/activity/pgstat_subscription.c | 4 +- src/backend/utils/activity/pgstat_wal.c | 10 +- src/backend/utils/init/globals.c | 1 + src/backend/utils/init/postinit.c | 3 + src/include/miscadmin.h | 1 + src/include/pgstat.h | 16 +++ src/include/utils/pgstat_internal.h | 52 ++++++-- src/include/utils/timeout.h | 1 + .../test_custom_stats/test_custom_var_stats.c | 4 +- src/tools/pgindent/typedefs.list | 1 + 28 files changed, 264 insertions(+), 66 deletions(-) 10.8% src/backend/postmaster/ 6.0% src/backend/replication/ 50.7% src/backend/utils/activity/ 6.0% src/backend/ 19.3% src/include/utils/ 5.6% src/include/ diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 13ec6225b85..d01b11c7470 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -1085,6 +1085,9 @@ XLogInsertRecord(XLogRecData *rdata, pgWalUsage.wal_fpi += num_fpi; pgWalUsage.wal_fpi_bytes += fpi_bytes; + /* Schedule next anytime stats update timeout */ + pgstat_schedule_anytime_update(); + /* Required for the flush of pending stats WAL data */ pgstat_report_fixed = true; } @@ -2066,6 +2069,9 @@ AdvanceXLInsertBuffer(XLogRecPtr upto, TimeLineID tli, bool opportunistic) pgWalUsage.wal_buffers_full++; TRACE_POSTGRESQL_WAL_BUFFER_WRITE_DIRTY_DONE(); + /* Schedule next anytime stats update timeout */ + pgstat_schedule_anytime_update(); + /* * Required for the flush of pending stats WAL data, per * update of pgWalUsage. diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c index 0956bd39a85..059c601c3b8 100644 --- a/src/backend/postmaster/bgwriter.c +++ b/src/backend/postmaster/bgwriter.c @@ -49,7 +49,9 @@ #include "storage/smgr.h" #include "storage/standby.h" #include "utils/memutils.h" +#include "utils/pgstat_internal.h" #include "utils/resowner.h" +#include "utils/timeout.h" #include "utils/timestamp.h" /* @@ -103,7 +105,7 @@ BackgroundWriterMain(const void *startup_data, size_t startup_data_len) pqsignal(SIGINT, SIG_IGN); pqsignal(SIGTERM, SignalHandlerForShutdownRequest); /* SIGQUIT handler was already set up by InitPostmasterChild */ - pqsignal(SIGALRM, SIG_IGN); + InitializeTimeouts(); /* establishes SIGALRM handler */ pqsignal(SIGPIPE, SIG_IGN); pqsignal(SIGUSR1, procsignal_sigusr1_handler); pqsignal(SIGUSR2, SIG_IGN); @@ -113,6 +115,11 @@ BackgroundWriterMain(const void *startup_data, size_t startup_data_len) */ pqsignal(SIGCHLD, SIG_DFL); + /* + * Register timeouts needed + */ + RegisterTimeout(ANYTIME_STATS_UPDATE_TIMEOUT, AnytimeStatsUpdateTimeoutHandler); + /* * We just started, assume there has been either a shutdown or * end-of-recovery snapshot. diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c index e03c19123bc..e11c4b099c8 100644 --- a/src/backend/postmaster/checkpointer.c +++ b/src/backend/postmaster/checkpointer.c @@ -66,8 +66,9 @@ #include "utils/acl.h" #include "utils/guc.h" #include "utils/memutils.h" +#include "utils/pgstat_internal.h" #include "utils/resowner.h" - +#include "utils/timeout.h" /*---------- * Shared memory area for communication between checkpointer and backends @@ -215,7 +216,7 @@ CheckpointerMain(const void *startup_data, size_t startup_data_len) pqsignal(SIGINT, ReqShutdownXLOG); pqsignal(SIGTERM, SIG_IGN); /* ignore SIGTERM */ /* SIGQUIT handler was already set up by InitPostmasterChild */ - pqsignal(SIGALRM, SIG_IGN); + InitializeTimeouts(); /* establishes SIGALRM handler */ pqsignal(SIGPIPE, SIG_IGN); pqsignal(SIGUSR1, procsignal_sigusr1_handler); pqsignal(SIGUSR2, SignalHandlerForShutdownRequest); @@ -225,6 +226,11 @@ CheckpointerMain(const void *startup_data, size_t startup_data_len) */ pqsignal(SIGCHLD, SIG_DFL); + /* + * Register timeouts needed + */ + RegisterTimeout(ANYTIME_STATS_UPDATE_TIMEOUT, AnytimeStatsUpdateTimeoutHandler); + /* * Initialize so that first time-driven event happens at the correct time. */ diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c index cdbe53dd262..4954fe425b7 100644 --- a/src/backend/postmaster/startup.c +++ b/src/backend/postmaster/startup.c @@ -32,6 +32,7 @@ #include "storage/standby.h" #include "utils/guc.h" #include "utils/memutils.h" +#include "utils/pgstat_internal.h" #include "utils/timeout.h" @@ -245,6 +246,7 @@ StartupProcessMain(const void *startup_data, size_t startup_data_len) RegisterTimeout(STANDBY_DEADLOCK_TIMEOUT, StandbyDeadLockHandler); RegisterTimeout(STANDBY_TIMEOUT, StandbyTimeoutHandler); RegisterTimeout(STANDBY_LOCK_TIMEOUT, StandbyLockTimeoutHandler); + RegisterTimeout(ANYTIME_STATS_UPDATE_TIMEOUT, AnytimeStatsUpdateTimeoutHandler); /* * Unblock signals (they were blocked when the postmaster forked us) diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c index 2d8f57099fd..9f8ef8159d1 100644 --- a/src/backend/postmaster/walsummarizer.c +++ b/src/backend/postmaster/walsummarizer.c @@ -48,6 +48,8 @@ #include "storage/shmem.h" #include "utils/guc.h" #include "utils/memutils.h" +#include "utils/pgstat_internal.h" +#include "utils/timeout.h" #include "utils/wait_event.h" /* @@ -249,7 +251,7 @@ WalSummarizerMain(const void *startup_data, size_t startup_data_len) pqsignal(SIGINT, SignalHandlerForShutdownRequest); pqsignal(SIGTERM, SignalHandlerForShutdownRequest); /* SIGQUIT handler was already set up by InitPostmasterChild */ - pqsignal(SIGALRM, SIG_IGN); + InitializeTimeouts(); /* establishes SIGALRM handler */ pqsignal(SIGPIPE, SIG_IGN); pqsignal(SIGUSR1, procsignal_sigusr1_handler); pqsignal(SIGUSR2, SIG_IGN); /* not used */ @@ -271,6 +273,11 @@ WalSummarizerMain(const void *startup_data, size_t startup_data_len) */ pqsignal(SIGCHLD, SIG_DFL); + /* + * Register timeouts needed + */ + RegisterTimeout(ANYTIME_STATS_UPDATE_TIMEOUT, AnytimeStatsUpdateTimeoutHandler); + /* * If an exception is encountered, processing resumes here. */ diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c index 23e79a32345..ded0f250288 100644 --- a/src/backend/postmaster/walwriter.c +++ b/src/backend/postmaster/walwriter.c @@ -61,7 +61,9 @@ #include "storage/smgr.h" #include "utils/hsearch.h" #include "utils/memutils.h" +#include "utils/pgstat_internal.h" #include "utils/resowner.h" +#include "utils/timeout.h" /* @@ -106,7 +108,7 @@ WalWriterMain(const void *startup_data, size_t startup_data_len) pqsignal(SIGINT, SignalHandlerForShutdownRequest); pqsignal(SIGTERM, SignalHandlerForShutdownRequest); /* SIGQUIT handler was already set up by InitPostmasterChild */ - pqsignal(SIGALRM, SIG_IGN); + InitializeTimeouts(); /* establishes SIGALRM handler */ pqsignal(SIGPIPE, SIG_IGN); pqsignal(SIGUSR1, procsignal_sigusr1_handler); pqsignal(SIGUSR2, SIG_IGN); /* not used */ @@ -116,6 +118,11 @@ WalWriterMain(const void *startup_data, size_t startup_data_len) */ pqsignal(SIGCHLD, SIG_DFL); + /* + * Register timeouts needed + */ + RegisterTimeout(ANYTIME_STATS_UPDATE_TIMEOUT, AnytimeStatsUpdateTimeoutHandler); + /* * Create a memory context that we will do all our work in. We do this so * that we can reset the context during error recovery and thereby avoid diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index 10e64a7d1f4..11b7c114d3b 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -77,7 +77,9 @@ #include "utils/builtins.h" #include "utils/guc.h" #include "utils/pg_lsn.h" +#include "utils/pgstat_internal.h" #include "utils/ps_status.h" +#include "utils/timeout.h" #include "utils/timestamp.h" @@ -252,7 +254,7 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) pqsignal(SIGINT, SIG_IGN); pqsignal(SIGTERM, die); /* request shutdown */ /* SIGQUIT handler was already set up by InitPostmasterChild */ - pqsignal(SIGALRM, SIG_IGN); + InitializeTimeouts(); /* establishes SIGALRM handler */ pqsignal(SIGPIPE, SIG_IGN); pqsignal(SIGUSR1, procsignal_sigusr1_handler); pqsignal(SIGUSR2, SIG_IGN); @@ -260,6 +262,11 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) /* Reset some signals that are accepted by postmaster but not here */ pqsignal(SIGCHLD, SIG_DFL); + /* + * Register timeouts needed + */ + RegisterTimeout(ANYTIME_STATS_UPDATE_TIMEOUT, AnytimeStatsUpdateTimeoutHandler); + /* Load the libpq-specific functions */ load_file("libpqwalreceiver", false); if (WalReceiverFunctions == NULL) diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 2cde8ebc729..a7214d0dc6f 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1987,8 +1987,8 @@ WalSndWaitForWal(XLogRecPtr loc) if (TimestampDifferenceExceeds(last_flush, now, WALSENDER_STATS_FLUSH_INTERVAL)) { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); + pgstat_flush_io(false, true); + (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); last_flush = now; } @@ -3016,8 +3016,8 @@ WalSndLoop(WalSndSendDataCallback send_data) if (TimestampDifferenceExceeds(last_flush, now, WALSENDER_STATS_FLUSH_INTERVAL)) { - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); + pgstat_flush_io(false, true); + (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); last_flush = now; } diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 02e9aaa6bca..c7bc409b06f 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -3509,6 +3509,18 @@ ProcessInterrupts(void) pgstat_report_stat(true); } + /* + * Flush stats outside of transaction boundary if the timeout fired. + * Unlike transactional stats, these can be flushed even inside a running + * transaction. + */ + if (AnytimeStatsUpdateTimeoutPending) + { + AnytimeStatsUpdateTimeoutPending = false; + + pgstat_report_anytime_stat(false); + } + if (ProcSignalBarrierPending) ProcessProcSignalBarrier(); diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c index 11bb71cad5a..411b65aae3e 100644 --- a/src/backend/utils/activity/pgstat.c +++ b/src/backend/utils/activity/pgstat.c @@ -112,6 +112,7 @@ #include "utils/guc_hooks.h" #include "utils/memutils.h" #include "utils/pgstat_internal.h" +#include "utils/timeout.h" #include "utils/timestamp.h" @@ -122,8 +123,6 @@ * ---------- */ -/* minimum interval non-forced stats flushes.*/ -#define PGSTAT_MIN_INTERVAL 1000 /* how long until to block flushing pending stats updates */ #define PGSTAT_MAX_INTERVAL 60000 /* when to call pgstat_report_stat() again, even when idle */ @@ -187,7 +186,8 @@ static void pgstat_init_snapshot_fixed(void); static void pgstat_reset_after_failure(void); -static bool pgstat_flush_pending_entries(bool nowait); +static bool pgstat_flush_pending_entries(bool nowait, bool anytime_only); +static bool pgstat_flush_fixed_stats(bool nowait, bool anytime_only); static void pgstat_prep_snapshot(void); static void pgstat_build_snapshot(void); @@ -288,6 +288,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .fixed_amount = false, .write_to_file = true, + .flush_mode = FLUSH_AT_TXN_BOUNDARY, /* so pg_stat_database entries can be seen in all databases */ .accessed_across_databases = true, @@ -305,6 +306,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .fixed_amount = false, .write_to_file = true, + .flush_mode = FLUSH_AT_TXN_BOUNDARY, .shared_size = sizeof(PgStatShared_Relation), .shared_data_off = offsetof(PgStatShared_Relation, stats), @@ -321,6 +323,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .fixed_amount = false, .write_to_file = true, + .flush_mode = FLUSH_AT_TXN_BOUNDARY, .shared_size = sizeof(PgStatShared_Function), .shared_data_off = offsetof(PgStatShared_Function, stats), @@ -336,6 +339,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .fixed_amount = false, .write_to_file = true, + .flush_mode = FLUSH_AT_TXN_BOUNDARY, .accessed_across_databases = true, @@ -353,6 +357,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .fixed_amount = false, .write_to_file = true, + .flush_mode = FLUSH_AT_TXN_BOUNDARY, /* so pg_stat_subscription_stats entries can be seen in all databases */ .accessed_across_databases = true, @@ -370,6 +375,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .fixed_amount = false, .write_to_file = false, + .flush_mode = FLUSH_ANYTIME, .accessed_across_databases = true, @@ -436,6 +442,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .fixed_amount = true, .write_to_file = true, + .flush_mode = FLUSH_ANYTIME, .snapshot_ctl_off = offsetof(PgStat_Snapshot, io), .shared_ctl_off = offsetof(PgStat_ShmemControl, io), @@ -453,6 +460,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .fixed_amount = true, .write_to_file = true, + .flush_mode = FLUSH_ANYTIME, .snapshot_ctl_off = offsetof(PgStat_Snapshot, slru), .shared_ctl_off = offsetof(PgStat_ShmemControl, slru), @@ -470,6 +478,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .fixed_amount = true, .write_to_file = true, + .flush_mode = FLUSH_ANYTIME, .snapshot_ctl_off = offsetof(PgStat_Snapshot, wal), .shared_ctl_off = offsetof(PgStat_ShmemControl, wal), @@ -775,23 +784,11 @@ pgstat_report_stat(bool force) partial_flush = false; /* flush of variable-numbered stats tracked in pending entries list */ - partial_flush |= pgstat_flush_pending_entries(nowait); + partial_flush |= pgstat_flush_pending_entries(nowait, false); /* flush of other stats kinds */ if (pgstat_report_fixed) - { - for (PgStat_Kind kind = PGSTAT_KIND_MIN; kind <= PGSTAT_KIND_MAX; kind++) - { - const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind); - - if (!kind_info) - continue; - if (!kind_info->flush_static_cb) - continue; - - partial_flush |= kind_info->flush_static_cb(nowait); - } - } + partial_flush |= pgstat_flush_fixed_stats(nowait, false); last_flush = now; @@ -1293,7 +1290,8 @@ pgstat_prep_pending_entry(PgStat_Kind kind, Oid dboid, uint64 objid, bool *creat if (entry_ref->pending == NULL) { - size_t entrysize = pgstat_get_kind_info(kind)->pending_size; + const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind); + size_t entrysize = kind_info->pending_size; Assert(entrysize != (size_t) -1); @@ -1345,9 +1343,14 @@ pgstat_delete_pending_entry(PgStat_EntryRef *entry_ref) /* * Flush out pending variable-numbered stats. + * + * If anytime_only is true, only flushes FLUSH_ANYTIME entries. + * This is safe to call inside transactions. + * + * If anytime_only is false, flushes all entries. */ static bool -pgstat_flush_pending_entries(bool nowait) +pgstat_flush_pending_entries(bool nowait, bool anytime_only) { bool have_pending = false; dlist_node *cur = NULL; @@ -1377,8 +1380,22 @@ pgstat_flush_pending_entries(bool nowait) Assert(!kind_info->fixed_amount); Assert(kind_info->flush_pending_cb != NULL); + /* Skip transactional stats if we're in anytime_only mode */ + if (anytime_only && kind_info->flush_mode == FLUSH_AT_TXN_BOUNDARY) + { + have_pending = true; + + if (dlist_has_next(&pgStatPending, cur)) + next = dlist_next_node(&pgStatPending, cur); + else + next = NULL; + + cur = next; + continue; + } + /* flush the stats, if possible */ - did_flush = kind_info->flush_pending_cb(entry_ref, nowait); + did_flush = kind_info->flush_pending_cb(entry_ref, nowait, anytime_only); Assert(did_flush || nowait); @@ -1402,6 +1419,33 @@ pgstat_flush_pending_entries(bool nowait) return have_pending; } +/* + * Flush fixed-amount stats. + * + * If anytime_only is true, only flushes FLUSH_ANYTIME stats (safe inside transactions). + * If anytime_only is false, flushes all stats with flush_static_cb. + */ +static bool +pgstat_flush_fixed_stats(bool nowait, bool anytime_only) +{ + bool partial_flush = false; + + for (PgStat_Kind kind = PGSTAT_KIND_MIN; kind <= PGSTAT_KIND_MAX; kind++) + { + const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind); + + if (!kind_info || !kind_info->flush_static_cb) + continue; + + /* Skip transactional stats if we're in anytime_only mode */ + if (anytime_only && kind_info->flush_mode == FLUSH_AT_TXN_BOUNDARY) + continue; + + partial_flush |= kind_info->flush_static_cb(nowait, anytime_only); + } + + return partial_flush; +} /* ------------------------------------------------------------ * Helper / infrastructure functions @@ -2119,3 +2163,31 @@ assign_stats_fetch_consistency(int newval, void *extra) if (pgstat_fetch_consistency != newval) force_stats_snapshot_clear = true; } + +/* + * Flushes only FLUSH_ANYTIME stats using non-blocking locks. Transactional + * stats (FLUSH_AT_TXN_BOUNDARY) remain pending until transaction boundary. + * Safe to call inside transactions. + */ +void +pgstat_report_anytime_stat(bool force) +{ + bool nowait = !force; + + pgstat_assert_is_up(); + + /* Flush stats outside of transaction boundary */ + pgstat_flush_pending_entries(nowait, true); + pgstat_flush_fixed_stats(nowait, true); +} + +/* + * Timeout handler for flushing non-transactional stats. + */ +void +AnytimeStatsUpdateTimeoutHandler(void) +{ + AnytimeStatsUpdateTimeoutPending = true; + InterruptPending = true; + SetLatch(MyLatch); +} diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c index 1350f5f62f1..b0bd220882d 100644 --- a/src/backend/utils/activity/pgstat_backend.c +++ b/src/backend/utils/activity/pgstat_backend.c @@ -31,6 +31,7 @@ #include "storage/procarray.h" #include "utils/memutils.h" #include "utils/pgstat_internal.h" +#include "utils/timeout.h" /* * Backend statistics counts waiting to be flushed out. These counters may be @@ -66,6 +67,9 @@ pgstat_count_backend_io_op_time(IOObject io_object, IOContext io_context, INSTR_TIME_ADD(PendingBackendStats.pending_io.pending_times[io_object][io_context][io_op], io_time); + /* Schedule next anytime stats update timeout */ + pgstat_schedule_anytime_update(); + backend_has_iostats = true; pgstat_report_fixed = true; } @@ -82,6 +86,9 @@ pgstat_count_backend_io_op(IOObject io_object, IOContext io_context, PendingBackendStats.pending_io.counts[io_object][io_context][io_op] += cnt; PendingBackendStats.pending_io.bytes[io_object][io_context][io_op] += bytes; + /* Schedule next anytime stats update timeout */ + pgstat_schedule_anytime_update(); + backend_has_iostats = true; pgstat_report_fixed = true; } @@ -268,7 +275,7 @@ pgstat_flush_backend_entry_wal(PgStat_EntryRef *entry_ref) * if some statistics could not be flushed due to lock contention. */ bool -pgstat_flush_backend(bool nowait, bits32 flags) +pgstat_flush_backend(bool nowait, bits32 flags, bool anytime_only) { PgStat_EntryRef *entry_ref; bool has_pending_data = false; @@ -311,9 +318,9 @@ pgstat_flush_backend(bool nowait, bits32 flags) * If some stats could not be flushed due to lock contention, return true. */ bool -pgstat_backend_flush_cb(bool nowait) +pgstat_backend_flush_cb(bool nowait, bool anytime_only) { - return pgstat_flush_backend(nowait, PGSTAT_BACKEND_FLUSH_ALL); + return pgstat_flush_backend(nowait, PGSTAT_BACKEND_FLUSH_ALL, anytime_only); } /* diff --git a/src/backend/utils/activity/pgstat_bgwriter.c b/src/backend/utils/activity/pgstat_bgwriter.c index ed2fd801189..1c5f0c3ec40 100644 --- a/src/backend/utils/activity/pgstat_bgwriter.c +++ b/src/backend/utils/activity/pgstat_bgwriter.c @@ -61,7 +61,7 @@ pgstat_report_bgwriter(void) /* * Report IO statistics */ - pgstat_flush_io(false); + pgstat_flush_io(false, true); } /* diff --git a/src/backend/utils/activity/pgstat_checkpointer.c b/src/backend/utils/activity/pgstat_checkpointer.c index 1f70194b7a7..2d89a082464 100644 --- a/src/backend/utils/activity/pgstat_checkpointer.c +++ b/src/backend/utils/activity/pgstat_checkpointer.c @@ -68,7 +68,7 @@ pgstat_report_checkpointer(void) /* * Report IO statistics */ - pgstat_flush_io(false); + pgstat_flush_io(false, true); } /* diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c index d7f6d4c5ee6..61094f96a6c 100644 --- a/src/backend/utils/activity/pgstat_database.c +++ b/src/backend/utils/activity/pgstat_database.c @@ -425,7 +425,7 @@ pgstat_reset_database_timestamp(Oid dboid, TimestampTz ts) * false without flushing the entry. Otherwise returns true. */ bool -pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) +pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_only) { PgStatShared_Database *sharedent; PgStat_StatDBEntry *pendingent; diff --git a/src/backend/utils/activity/pgstat_function.c b/src/backend/utils/activity/pgstat_function.c index e6b84283c6c..5ba4958382f 100644 --- a/src/backend/utils/activity/pgstat_function.c +++ b/src/backend/utils/activity/pgstat_function.c @@ -190,11 +190,13 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize) * false without flushing the entry. Otherwise returns true. */ bool -pgstat_function_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) +pgstat_function_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_only) { PgStat_FunctionCounts *localent; PgStatShared_Function *shfuncent; + Assert(!anytime_only); + localent = (PgStat_FunctionCounts *) entry_ref->pending; shfuncent = (PgStatShared_Function *) entry_ref->shared_stats; diff --git a/src/backend/utils/activity/pgstat_io.c b/src/backend/utils/activity/pgstat_io.c index 28de24538dc..7cd32900236 100644 --- a/src/backend/utils/activity/pgstat_io.c +++ b/src/backend/utils/activity/pgstat_io.c @@ -19,6 +19,7 @@ #include "executor/instrument.h" #include "storage/bufmgr.h" #include "utils/pgstat_internal.h" +#include "utils/timeout.h" static PgStat_PendingIO PendingIOStats; static bool have_iostats = false; @@ -79,6 +80,9 @@ pgstat_count_io_op(IOObject io_object, IOContext io_context, IOOp io_op, /* Add the per-backend counts */ pgstat_count_backend_io_op(io_object, io_context, io_op, cnt, bytes); + /* Schedule next anytime stats update timeout */ + pgstat_schedule_anytime_update(); + have_iostats = true; pgstat_report_fixed = true; } @@ -172,9 +176,9 @@ pgstat_fetch_stat_io(void) * Simpler wrapper of pgstat_io_flush_cb() */ void -pgstat_flush_io(bool nowait) +pgstat_flush_io(bool nowait, bool anytime_only) { - (void) pgstat_io_flush_cb(nowait); + (void) pgstat_io_flush_cb(nowait, anytime_only); } /* @@ -186,7 +190,7 @@ pgstat_flush_io(bool nowait) * acquired. Otherwise, return false. */ bool -pgstat_io_flush_cb(bool nowait) +pgstat_io_flush_cb(bool nowait, bool anytime_only) { LWLock *bktype_lock; PgStat_BktypeIO *bktype_shstats; diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index bc8c43b96aa..04d21483d93 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -267,8 +267,8 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, * is done -- which will likely vacuum many relations -- or until the * VACUUM command has processed all tables and committed. */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); + pgstat_flush_io(false, true); + (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* @@ -362,8 +362,8 @@ pgstat_report_analyze(Relation rel, pgstat_unlock_entry(entry_ref); /* see pgstat_report_vacuum() */ - pgstat_flush_io(false); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); + pgstat_flush_io(false, true); + (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* @@ -812,7 +812,7 @@ pgstat_twophase_postabort(FullTransactionId fxid, uint16 info, * entry when successfully flushing. */ bool -pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) +pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_only) { Oid dboid; PgStat_TableStatus *lstats; /* pending stats entry */ @@ -820,6 +820,8 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) PgStat_StatTabEntry *tabentry; /* table entry of shared stats */ PgStat_StatDBEntry *dbentry; /* pending database entry */ + Assert(!anytime_only); + dboid = entry_ref->shared_entry->key.dboid; lstats = (PgStat_TableStatus *) entry_ref->pending; shtabstats = (PgStatShared_Relation *) entry_ref->shared_stats; diff --git a/src/backend/utils/activity/pgstat_slru.c b/src/backend/utils/activity/pgstat_slru.c index 2190f388eae..bf8a4d58673 100644 --- a/src/backend/utils/activity/pgstat_slru.c +++ b/src/backend/utils/activity/pgstat_slru.c @@ -19,6 +19,7 @@ #include "utils/pgstat_internal.h" #include "utils/timestamp.h" +#include "utils/timeout.h" static inline PgStat_SLRUStats *get_slru_entry(int slru_idx); @@ -139,7 +140,7 @@ pgstat_get_slru_index(const char *name) * acquired. Otherwise return false. */ bool -pgstat_slru_flush_cb(bool nowait) +pgstat_slru_flush_cb(bool nowait, bool anytime_only) { PgStatShared_SLRU *stats_shmem = &pgStatLocal.shmem->slru; int i; @@ -223,6 +224,9 @@ get_slru_entry(int slru_idx) Assert((slru_idx >= 0) && (slru_idx < SLRU_NUM_ELEMENTS)); + /* Schedule next anytime stats update timeout */ + pgstat_schedule_anytime_update(); + have_slrustats = true; pgstat_report_fixed = true; diff --git a/src/backend/utils/activity/pgstat_subscription.c b/src/backend/utils/activity/pgstat_subscription.c index 500b1899188..c4614817966 100644 --- a/src/backend/utils/activity/pgstat_subscription.c +++ b/src/backend/utils/activity/pgstat_subscription.c @@ -116,11 +116,13 @@ pgstat_fetch_stat_subscription(Oid subid) * false without flushing the entry. Otherwise returns true. */ bool -pgstat_subscription_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) +pgstat_subscription_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_only) { PgStat_BackendSubEntry *localent; PgStatShared_Subscription *shsubent; + Assert(!anytime_only); + localent = (PgStat_BackendSubEntry *) entry_ref->pending; shsubent = (PgStatShared_Subscription *) entry_ref->shared_stats; diff --git a/src/backend/utils/activity/pgstat_wal.c b/src/backend/utils/activity/pgstat_wal.c index 183e0a7a97b..eb5f8f46925 100644 --- a/src/backend/utils/activity/pgstat_wal.c +++ b/src/backend/utils/activity/pgstat_wal.c @@ -51,12 +51,12 @@ pgstat_report_wal(bool force) nowait = !force; /* flush wal stats */ - (void) pgstat_wal_flush_cb(nowait); - pgstat_flush_backend(nowait, PGSTAT_BACKEND_FLUSH_WAL); + (void) pgstat_wal_flush_cb(nowait, true); + pgstat_flush_backend(nowait, PGSTAT_BACKEND_FLUSH_WAL, true); /* flush IO stats */ - pgstat_flush_io(nowait); - (void) pgstat_flush_backend(nowait, PGSTAT_BACKEND_FLUSH_IO); + pgstat_flush_io(nowait, true); + (void) pgstat_flush_backend(nowait, PGSTAT_BACKEND_FLUSH_IO, true); } /* @@ -88,7 +88,7 @@ pgstat_wal_have_pending(void) * acquired. Otherwise return false. */ bool -pgstat_wal_flush_cb(bool nowait) +pgstat_wal_flush_cb(bool nowait, bool anytime_only) { PgStatShared_Wal *stats_shmem = &pgStatLocal.shmem->wal; WalUsage wal_usage_diff = {0}; diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c index 36ad708b360..ad44826c39e 100644 --- a/src/backend/utils/init/globals.c +++ b/src/backend/utils/init/globals.c @@ -40,6 +40,7 @@ volatile sig_atomic_t IdleSessionTimeoutPending = false; volatile sig_atomic_t ProcSignalBarrierPending = false; volatile sig_atomic_t LogMemoryContextPending = false; volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false; +volatile sig_atomic_t AnytimeStatsUpdateTimeoutPending = false; volatile uint32 InterruptHoldoffCount = 0; volatile uint32 QueryCancelHoldoffCount = 0; volatile uint32 CritSectionCount = 0; diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c index 3f401faf3de..f45365f47f7 100644 --- a/src/backend/utils/init/postinit.c +++ b/src/backend/utils/init/postinit.c @@ -64,6 +64,7 @@ #include "utils/injection_point.h" #include "utils/memutils.h" #include "utils/pg_locale.h" +#include "utils/pgstat_internal.h" #include "utils/portal.h" #include "utils/ps_status.h" #include "utils/snapmgr.h" @@ -765,6 +766,8 @@ InitPostgres(const char *in_dbname, Oid dboid, RegisterTimeout(CLIENT_CONNECTION_CHECK_TIMEOUT, ClientCheckTimeoutHandler); RegisterTimeout(IDLE_STATS_UPDATE_TIMEOUT, IdleStatsUpdateTimeoutHandler); + RegisterTimeout(ANYTIME_STATS_UPDATE_TIMEOUT, + AnytimeStatsUpdateTimeoutHandler); } /* diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h index db559b39c4d..8aeb9628871 100644 --- a/src/include/miscadmin.h +++ b/src/include/miscadmin.h @@ -96,6 +96,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending; extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending; extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending; extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending; +extern PGDLLIMPORT volatile sig_atomic_t AnytimeStatsUpdateTimeoutPending; extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending; extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost; diff --git a/src/include/pgstat.h b/src/include/pgstat.h index fff7ecc2533..b340a680614 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -35,6 +35,9 @@ /* Default directory to store temporary statistics data in */ #define PG_STAT_TMP_DIR "pg_stat_tmp" +/* Minimum interval non-forced stats flushes */ +#define PGSTAT_MIN_INTERVAL 1000 + /* Values for track_functions GUC variable --- order is significant! */ typedef enum TrackFunctionsLevel { @@ -533,8 +536,21 @@ extern void pgstat_initialize(void); /* Functions called from backends */ extern long pgstat_report_stat(bool force); +extern void pgstat_report_anytime_stat(bool force); extern void pgstat_force_next_flush(void); +/* + * Schedule the next anytime stats update timeout. + * + * This should be called whenever accumulating statistics that support + * FLUSH_ANYTIME flushing mode. + */ +#define pgstat_schedule_anytime_update() \ + do { \ + if (IsUnderPostmaster && !get_timeout_active(ANYTIME_STATS_UPDATE_TIMEOUT)) \ + enable_timeout_after(ANYTIME_STATS_UPDATE_TIMEOUT, PGSTAT_MIN_INTERVAL); \ + } while (0) + extern void pgstat_reset_counters(void); extern void pgstat_reset(PgStat_Kind kind, Oid dboid, uint64 objid); extern void pgstat_reset_of_kind(PgStat_Kind kind); diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h index 9b8fbae00ed..607f4255268 100644 --- a/src/include/utils/pgstat_internal.h +++ b/src/include/utils/pgstat_internal.h @@ -224,6 +224,19 @@ typedef struct PgStat_SubXactStatus PgStat_TableXactStatus *first; /* head of list for this subxact */ } PgStat_SubXactStatus; +/* + * Flush mode for statistics kinds. + * + * FLUSH_AT_TXN_BOUNDARY has to be the first because we want it to be the + * default value. + */ +typedef enum PgStat_FlushMode +{ + FLUSH_AT_TXN_BOUNDARY, /* All fields can only be flushed at + * transaction boundary */ + FLUSH_ANYTIME, /* All fields can be flushed anytime, + * including within transactions */ +} PgStat_FlushMode; /* * Metadata for a specific kind of statistics. @@ -251,6 +264,16 @@ typedef struct PgStat_KindInfo */ bool track_entry_count:1; + /* + * The mode of when to flush stats. See PgStat_FlushMode for more details. + * + * This member only has meaning for statistics kinds that accumulate + * pending stats and use flush callbacks. For kinds that write directly to + * shared memory (e.g., archiver, bgwriter, checkpointer), this member has + * no effect. + */ + PgStat_FlushMode flush_mode; + /* * The size of an entry in the shared stats hash table (pointed to by * PgStatShared_HashEntry->body). For fixed-numbered statistics, this is @@ -297,8 +320,10 @@ typedef struct PgStat_KindInfo * For variable-numbered stats: flush pending stats. Required if pending * data is used. See flush_static_cb when dealing with stats data that * that cannot use PgStat_EntryRef->pending. + * + * The anytime_only parameter indicates whether this is an anytime flush. */ - bool (*flush_pending_cb) (PgStat_EntryRef *sr, bool nowait); + bool (*flush_pending_cb) (PgStat_EntryRef *sr, bool nowait, bool anytime_only); /* * For variable-numbered stats: delete pending stats. Optional. @@ -366,8 +391,10 @@ typedef struct PgStat_KindInfo * * "pgstat_report_fixed" needs to be set to trigger the flush of pending * stats. + * + * The anytime_only parameter indicates whether this is an anytime flush. */ - bool (*flush_static_cb) (bool nowait); + bool (*flush_static_cb) (bool nowait, bool anytime_only); /* * For fixed-numbered statistics: Reset All. @@ -677,6 +704,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind, extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid); extern void pgstat_snapshot_fixed(PgStat_Kind kind); +extern void AnytimeStatsUpdateTimeoutHandler(void); /* @@ -696,8 +724,8 @@ extern void pgstat_archiver_snapshot_cb(void); #define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */ #define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL) -extern bool pgstat_flush_backend(bool nowait, bits32 flags); -extern bool pgstat_backend_flush_cb(bool nowait); +extern bool pgstat_flush_backend(bool nowait, bits32 flags, bool anytime_only); +extern bool pgstat_backend_flush_cb(bool nowait, bool anytime_only); extern void pgstat_backend_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); @@ -729,7 +757,7 @@ extern void AtEOXact_PgStat_Database(bool isCommit, bool parallel); extern PgStat_StatDBEntry *pgstat_prep_database_pending(Oid dboid); extern void pgstat_reset_database_timestamp(Oid dboid, TimestampTz ts); -extern bool pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait); +extern bool pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_only); extern void pgstat_database_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); @@ -737,7 +765,7 @@ extern void pgstat_database_reset_timestamp_cb(PgStatShared_Common *header, Time * Functions in pgstat_function.c */ -extern bool pgstat_function_flush_cb(PgStat_EntryRef *entry_ref, bool nowait); +extern bool pgstat_function_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_only); extern void pgstat_function_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); @@ -745,9 +773,9 @@ extern void pgstat_function_reset_timestamp_cb(PgStatShared_Common *header, Time * Functions in pgstat_io.c */ -extern void pgstat_flush_io(bool nowait); +extern void pgstat_flush_io(bool nowait, bool anytime_only); -extern bool pgstat_io_flush_cb(bool nowait); +extern bool pgstat_io_flush_cb(bool nowait, bool anytime_only); extern void pgstat_io_init_shmem_cb(void *stats); extern void pgstat_io_reset_all_cb(TimestampTz ts); extern void pgstat_io_snapshot_cb(void); @@ -762,7 +790,7 @@ extern void AtEOSubXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool extern void AtPrepare_PgStat_Relations(PgStat_SubXactStatus *xact_state); extern void PostPrepare_PgStat_Relations(PgStat_SubXactStatus *xact_state); -extern bool pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait); +extern bool pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_only); extern void pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref); extern void pgstat_relation_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); @@ -809,7 +837,7 @@ extern PgStatShared_Common *pgstat_init_entry(PgStat_Kind kind, * Functions in pgstat_slru.c */ -extern bool pgstat_slru_flush_cb(bool nowait); +extern bool pgstat_slru_flush_cb(bool nowait, bool anytime_only); extern void pgstat_slru_init_shmem_cb(void *stats); extern void pgstat_slru_reset_all_cb(TimestampTz ts); extern void pgstat_slru_snapshot_cb(void); @@ -820,7 +848,7 @@ extern void pgstat_slru_snapshot_cb(void); */ extern void pgstat_wal_init_backend_cb(void); -extern bool pgstat_wal_flush_cb(bool nowait); +extern bool pgstat_wal_flush_cb(bool nowait, bool anytime_only); extern void pgstat_wal_init_shmem_cb(void *stats); extern void pgstat_wal_reset_all_cb(TimestampTz ts); extern void pgstat_wal_snapshot_cb(void); @@ -830,7 +858,7 @@ extern void pgstat_wal_snapshot_cb(void); * Functions in pgstat_subscription.c */ -extern bool pgstat_subscription_flush_cb(PgStat_EntryRef *entry_ref, bool nowait); +extern bool pgstat_subscription_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_only); extern void pgstat_subscription_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h index 0965b590b34..10723bb664c 100644 --- a/src/include/utils/timeout.h +++ b/src/include/utils/timeout.h @@ -35,6 +35,7 @@ typedef enum TimeoutId IDLE_SESSION_TIMEOUT, IDLE_STATS_UPDATE_TIMEOUT, CLIENT_CONNECTION_CHECK_TIMEOUT, + ANYTIME_STATS_UPDATE_TIMEOUT, STARTUP_PROGRESS_TIMEOUT, /* First user-definable timeout reason */ USER_TIMEOUT, diff --git a/src/test/modules/test_custom_stats/test_custom_var_stats.c b/src/test/modules/test_custom_stats/test_custom_var_stats.c index 64a8fe63cce..bc0b5d6e0eb 100644 --- a/src/test/modules/test_custom_stats/test_custom_var_stats.c +++ b/src/test/modules/test_custom_stats/test_custom_var_stats.c @@ -83,7 +83,7 @@ static dsa_area *custom_stats_description_dsa = NULL; /* Flush callback: merge pending stats into shared memory */ static bool test_custom_stats_var_flush_pending_cb(PgStat_EntryRef *entry_ref, - bool nowait); + bool nowait, bool anytime_only); /* Serialization callback: write auxiliary entry data */ static void test_custom_stats_var_to_serialized_data(const PgStat_HashKey *key, @@ -150,7 +150,7 @@ _PG_init(void) * Returns false only if nowait=true and lock acquisition fails. */ static bool -test_custom_stats_var_flush_pending_cb(PgStat_EntryRef *entry_ref, bool nowait) +test_custom_stats_var_flush_pending_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_only) { PgStat_StatCustomVarEntry *pending_entry; PgStatShared_CustomVarEntry *shared_entry; diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 9f5ee8fd482..860f835c088 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2268,6 +2268,7 @@ PgStat_Counter PgStat_EntryRef PgStat_EntryRefHashEntry PgStat_FetchConsistency +PgStat_FlushMode PgStat_FunctionCallUsage PgStat_FunctionCounts PgStat_HashKey -- 2.34.1
>From c381419eb6d5018c3fe5476ac6edae8673ac5343 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot <[email protected]> Date: Thu, 5 Feb 2026 05:54:34 +0000 Subject: [PATCH v6 2/5] Add anytime flush tests for custom stats --- .../test_custom_stats/t/001_custom_stats.pl | 41 ++++++++++++ .../test_custom_fixed_stats--1.0.sql | 10 +++ .../test_custom_fixed_stats.c | 66 +++++++++++++++++++ .../test_custom_var_stats--1.0.sql | 5 ++ .../test_custom_stats/test_custom_var_stats.c | 27 ++++++++ 5 files changed, 149 insertions(+) 31.4% src/test/modules/test_custom_stats/t/ 68.5% src/test/modules/test_custom_stats/ diff --git a/src/test/modules/test_custom_stats/t/001_custom_stats.pl b/src/test/modules/test_custom_stats/t/001_custom_stats.pl index 9e6a7a38577..36d9fc3fde1 100644 --- a/src/test/modules/test_custom_stats/t/001_custom_stats.pl +++ b/src/test/modules/test_custom_stats/t/001_custom_stats.pl @@ -156,5 +156,46 @@ $result = $node->safe_psql('postgres', ); is($result, "0", "report of fixed-sized after manual reset"); +# Test FLUSH_ANYTIME mechanism with custom fixed stats +# This verifies that custom stats can be flushed during a transaction + +# Reset stats first +$node->safe_psql('postgres', q(select test_custom_stats_fixed_reset())); +$node->safe_psql('postgres', q(select pg_stat_force_next_flush())); + +my $anytime_test = q[ + BEGIN; + -- Accumulate stats + select test_custom_stats_fixed_anytime_update() from generate_series(1, 2); + -- Force anytime flush (inside transaction!) + select pg_stat_force_anytime_flush(); + -- Check + select 'anytime:'||numcalls from test_custom_stats_fixed_report(); +]; + +$result = $node->safe_psql('postgres', $anytime_test); +like($result, qr/^anytime:2/m, + "anytime fixed stats flushed during transaction"); + +# Test FLUSH_ANYTIME mechanism with custom variable stats +# This verifies that custom stats can be flushed during a transaction + +$node->safe_psql('postgres', q(select pg_stat_force_next_flush())); + +$anytime_test = q[ + BEGIN; + -- Accumulate stats + select test_custom_stats_var_anytime_update('entry2'); + select test_custom_stats_var_anytime_update('entry2'); + -- Force anytime flush (inside transaction!) + select pg_stat_force_anytime_flush(); + -- Check + select * from test_custom_stats_var_report('entry2'); +]; + +$result = $node->safe_psql('postgres', $anytime_test); +like($result, qr/^entry2|2|/m, + "anytime var stats flushed during transaction"); + # Test completed successfully done_testing(); diff --git a/src/test/modules/test_custom_stats/test_custom_fixed_stats--1.0.sql b/src/test/modules/test_custom_stats/test_custom_fixed_stats--1.0.sql index 69a93b5241f..c0a418c3ae3 100644 --- a/src/test/modules/test_custom_stats/test_custom_fixed_stats--1.0.sql +++ b/src/test/modules/test_custom_stats/test_custom_fixed_stats--1.0.sql @@ -18,3 +18,13 @@ CREATE FUNCTION test_custom_stats_fixed_reset() RETURNS void AS 'MODULE_PATHNAME', 'test_custom_stats_fixed_reset' LANGUAGE C STRICT PARALLEL UNSAFE; + +CREATE FUNCTION test_custom_stats_fixed_anytime_update() +RETURNS void +AS 'MODULE_PATHNAME' +LANGUAGE C STRICT PARALLEL UNSAFE; + +CREATE FUNCTION pg_stat_force_anytime_flush() +RETURNS void +AS 'MODULE_PATHNAME' +LANGUAGE C STRICT PARALLEL UNSAFE; diff --git a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c index 908bd18a7c7..6b3bc3257ab 100644 --- a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c +++ b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c @@ -18,6 +18,7 @@ #include "pgstat.h" #include "utils/builtins.h" #include "utils/pgstat_internal.h" +#include "utils/timeout.h" PG_MODULE_MAGIC_EXT( .name = "test_custom_fixed_stats", @@ -43,11 +44,13 @@ typedef struct PgStatShared_CustomFixedEntry static void test_custom_stats_fixed_init_shmem_cb(void *stats); static void test_custom_stats_fixed_reset_all_cb(TimestampTz ts); static void test_custom_stats_fixed_snapshot_cb(void); +static bool test_custom_stats_fixed_flush_cb(bool nowait, bool anytime_only); static const PgStat_KindInfo custom_stats = { .name = "test_custom_fixed_stats", .fixed_amount = true, /* exactly one entry */ .write_to_file = true, /* persist to stats file */ + .flush_mode = FLUSH_ANYTIME, /* can be flushed anytime */ .shared_size = sizeof(PgStat_StatCustomFixedEntry), .shared_data_off = offsetof(PgStatShared_CustomFixedEntry, stats), @@ -56,8 +59,12 @@ static const PgStat_KindInfo custom_stats = { .init_shmem_cb = test_custom_stats_fixed_init_shmem_cb, .reset_all_cb = test_custom_stats_fixed_reset_all_cb, .snapshot_cb = test_custom_stats_fixed_snapshot_cb, + .flush_static_cb = test_custom_stats_fixed_flush_cb, }; +/* Pending statistics */ +static PgStat_StatCustomFixedEntry PendingCustomStats = {0}; + /* * Kind ID for test_custom_fixed_stats. */ @@ -141,6 +148,38 @@ test_custom_stats_fixed_snapshot_cb(void) #undef FIXED_COMP } +/* + * test_custom_stats_fixed_flush_cb + * Flush pending stats to shared memory + */ +static bool +test_custom_stats_fixed_flush_cb(bool nowait, bool anytime_only) +{ + PgStatShared_CustomFixedEntry *stats_shmem; + + /* Nothing to flush if no calls were made */ + if (PendingCustomStats.numcalls == 0) + return false; + + stats_shmem = pgstat_get_custom_shmem_data(PGSTAT_KIND_TEST_CUSTOM_FIXED_STATS); + + if (nowait && !LWLockConditionalAcquire(&stats_shmem->lock, LW_EXCLUSIVE)) + return true; /* failed to flush */ + + LWLockAcquire(&stats_shmem->lock, LW_EXCLUSIVE); + + pgstat_begin_changecount_write(&stats_shmem->changecount); + stats_shmem->stats.numcalls += PendingCustomStats.numcalls; + pgstat_end_changecount_write(&stats_shmem->changecount); + + LWLockRelease(&stats_shmem->lock); + + /* Reset pending stats */ + PendingCustomStats.numcalls = 0; + + return false; /* successfully flushed */ +} + /*-------------------------------------------------------------------------- * SQL-callable functions *-------------------------------------------------------------------------- @@ -222,3 +261,30 @@ test_custom_stats_fixed_report(PG_FUNCTION_ARGS) /* Return as tuple */ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls))); } + +/* + * test_custom_stats_fixed_anytime_update + * Increment call counter and schedule anytime flush + */ +PG_FUNCTION_INFO_V1(test_custom_stats_fixed_anytime_update); +Datum +test_custom_stats_fixed_anytime_update(PG_FUNCTION_ARGS) +{ + /* Accumulate in pending stats */ + PendingCustomStats.numcalls++; + + /* Schedule anytime stats update */ + pgstat_schedule_anytime_update(); + pgstat_report_fixed = true; + + PG_RETURN_VOID(); +} + +/* Helper function for testing ANYTIME flush */ +PG_FUNCTION_INFO_V1(pg_stat_force_anytime_flush); +Datum +pg_stat_force_anytime_flush(PG_FUNCTION_ARGS) +{ + pgstat_report_anytime_stat(true); + PG_RETURN_VOID(); +} diff --git a/src/test/modules/test_custom_stats/test_custom_var_stats--1.0.sql b/src/test/modules/test_custom_stats/test_custom_var_stats--1.0.sql index 5ed8cfc2dcf..ed66d38981e 100644 --- a/src/test/modules/test_custom_stats/test_custom_var_stats--1.0.sql +++ b/src/test/modules/test_custom_stats/test_custom_var_stats--1.0.sql @@ -24,3 +24,8 @@ CREATE FUNCTION test_custom_stats_var_report(INOUT name TEXT, RETURNS SETOF record AS 'MODULE_PATHNAME', 'test_custom_stats_var_report' LANGUAGE C STRICT PARALLEL UNSAFE; + +CREATE FUNCTION test_custom_stats_var_anytime_update(IN name TEXT) +RETURNS void +AS 'MODULE_PATHNAME', 'test_custom_stats_var_anytime_update' +LANGUAGE C STRICT PARALLEL UNSAFE; diff --git a/src/test/modules/test_custom_stats/test_custom_var_stats.c b/src/test/modules/test_custom_stats/test_custom_var_stats.c index bc0b5d6e0eb..207e841911b 100644 --- a/src/test/modules/test_custom_stats/test_custom_var_stats.c +++ b/src/test/modules/test_custom_stats/test_custom_var_stats.c @@ -17,6 +17,7 @@ #include "storage/dsm_registry.h" #include "utils/builtins.h" #include "utils/pgstat_internal.h" +#include "utils/timeout.h" PG_MODULE_MAGIC_EXT( .name = "test_custom_var_stats", @@ -107,6 +108,7 @@ static const PgStat_KindInfo custom_stats = { .name = "test_custom_var_stats", .fixed_amount = false, /* variable number of entries */ .write_to_file = true, /* persist across restarts */ + .flush_mode = FLUSH_ANYTIME, /* can be flushed anytime */ .track_entry_count = true, /* count active entries */ .accessed_across_databases = true, /* global statistics */ .shared_size = sizeof(PgStatShared_CustomVarEntry), @@ -689,3 +691,28 @@ test_custom_stats_var_report(PG_FUNCTION_ARGS) SRF_RETURN_DONE(funcctx); } + +/* + * test_custom_stats_var_anytime_update + * Increment custom statistic counter and schedule anytime flush + */ +PG_FUNCTION_INFO_V1(test_custom_stats_var_anytime_update); +Datum +test_custom_stats_var_anytime_update(PG_FUNCTION_ARGS) +{ + char *stat_name = text_to_cstring(PG_GETARG_TEXT_PP(0)); + PgStat_EntryRef *entry_ref; + PgStat_StatCustomVarEntry *pending_entry; + + /* Get pending entry in local memory */ + entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_TEST_CUSTOM_VAR_STATS, InvalidOid, + PGSTAT_CUSTOM_VAR_STATS_IDX(stat_name), NULL); + + pending_entry = (PgStat_StatCustomVarEntry *) entry_ref->pending; + pending_entry->numcalls++; + + /* Schedule anytime stats update */ + pgstat_schedule_anytime_update(); + + PG_RETURN_VOID(); +} -- 2.34.1
>From 69f3e4f063a956f429ee0dbdbd213dcad8242613 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot <[email protected]> Date: Wed, 28 Jan 2026 07:53:13 +0000 Subject: [PATCH v6 3/5] Add GUC to specify non-transactional statistics flush interval Adding pgstat_flush_interval, a new GUC to set the interval between flushes of non-transactional statistics. --- doc/src/sgml/config.sgml | 34 +++++++++++++++++++ src/backend/utils/activity/pgstat.c | 16 +++++++++ src/backend/utils/misc/guc_parameters.dat | 10 ++++++ src/backend/utils/misc/postgresql.conf.sample | 1 + src/include/pgstat.h | 6 ++-- src/include/utils/guc_hooks.h | 1 + 6 files changed, 64 insertions(+), 4 deletions(-) 59.1% doc/src/sgml/ 14.2% src/backend/utils/activity/ 14.5% src/backend/utils/misc/ 12.0% src/include/ diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index f1af1505cf3..20666679f90 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -8871,6 +8871,40 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; </listitem> </varlistentry> + <varlistentry id="guc-stats-flush-interval" xreflabel="stats_flush_interval"> + <term><varname>stats_flush_interval</varname> (<type>integer</type>) + <indexterm> + <primary><varname>stats_flush_interval</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Sets the interval at which statistics that can be updated while a + transaction is still running are made visible. These include, for example, + WAL activity and I/O operations. + Such statistics are refreshed at the specified interval and can be observed + during active transactions in monitoring views such as + <link linkend="monitoring-pg-stat-io-view"><structname>pg_stat_io</structname></link> + and + <link linkend="monitoring-pg-stat-wal-view"><structname>pg_stat_wal</structname></link>. + Other statistics are only made visible at transaction end and are not + affected by this setting. + If the value is specified without a unit, milliseconds are assumed. + The default is 10 seconds (<literal>10s</literal>), which is generally + the smallest practical value for long-running transactions. + </para> + <note> + <para> + This parameter does not affect statistics that are only reported at + transaction end, such as the columns of <structname>pg_stat_all_tables</structname> + (for example, <structfield>n_tup_ins</structfield>, <structfield>n_tup_upd</structfield>, + and <structfield>n_tup_del</structfield>). These statistics are always + flushed at the end of a transaction. + </para> + </note> + </listitem> + </varlistentry> + </variablelist> </sect2> diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c index 411b65aae3e..79eb59b5625 100644 --- a/src/backend/utils/activity/pgstat.c +++ b/src/backend/utils/activity/pgstat.c @@ -123,6 +123,8 @@ * ---------- */ +/* minimum interval non-forced stats flushes.*/ +#define PGSTAT_MIN_INTERVAL 1000 /* how long until to block flushing pending stats updates */ #define PGSTAT_MAX_INTERVAL 60000 /* when to call pgstat_report_stat() again, even when idle */ @@ -203,6 +205,7 @@ static inline bool pgstat_is_kind_valid(PgStat_Kind kind); bool pgstat_track_counts = false; int pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_CACHE; +int pgstat_flush_interval = 10000; /* ---------- @@ -2164,6 +2167,19 @@ assign_stats_fetch_consistency(int newval, void *extra) force_stats_snapshot_clear = true; } +/* + * GUC assign_hook for stats_flush_interval. + */ +void +assign_stats_flush_interval(int newval, void *extra) +{ + if (get_timeout_active(ANYTIME_STATS_UPDATE_TIMEOUT)) + { + disable_timeout(ANYTIME_STATS_UPDATE_TIMEOUT, false); + enable_timeout_after(ANYTIME_STATS_UPDATE_TIMEOUT, newval); + } +} + /* * Flushes only FLUSH_ANYTIME stats using non-blocking locks. Transactional * stats (FLUSH_AT_TXN_BOUNDARY) remain pending until transaction boundary. diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index c1f1603cd39..fc0e4259b36 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2789,6 +2789,16 @@ assign_hook => 'assign_stats_fetch_consistency', }, +{ name => 'stats_flush_interval', type => 'int', context => 'PGC_USERSET', group => 'STATS_CUMULATIVE', + short_desc => 'Sets the interval between flushes of non-transactional statistics.', + flags => 'GUC_UNIT_MS', + variable => 'pgstat_flush_interval', + boot_val => '10000', + min => '1000', + max => 'INT_MAX', + assign_hook => 'assign_stats_flush_interval' +}, + { name => 'subtransaction_buffers', type => 'int', context => 'PGC_POSTMASTER', group => 'RESOURCES_MEM', short_desc => 'Sets the size of the dedicated buffer pool used for the subtransaction cache.', long_desc => '0 means use a fraction of "shared_buffers".', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index 1ae594af843..3f998a0bea0 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -673,6 +673,7 @@ #track_wal_io_timing = off #track_functions = none # none, pl, all #stats_fetch_consistency = cache # cache, none, snapshot +#stats_flush_interval = 10s # in milliseconds # - Monitoring - diff --git a/src/include/pgstat.h b/src/include/pgstat.h index b340a680614..ef856dbf55b 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -35,9 +35,6 @@ /* Default directory to store temporary statistics data in */ #define PG_STAT_TMP_DIR "pg_stat_tmp" -/* Minimum interval non-forced stats flushes */ -#define PGSTAT_MIN_INTERVAL 1000 - /* Values for track_functions GUC variable --- order is significant! */ typedef enum TrackFunctionsLevel { @@ -548,7 +545,7 @@ extern void pgstat_force_next_flush(void); #define pgstat_schedule_anytime_update() \ do { \ if (IsUnderPostmaster && !get_timeout_active(ANYTIME_STATS_UPDATE_TIMEOUT)) \ - enable_timeout_after(ANYTIME_STATS_UPDATE_TIMEOUT, PGSTAT_MIN_INTERVAL); \ + enable_timeout_after(ANYTIME_STATS_UPDATE_TIMEOUT, pgstat_flush_interval); \ } while (0) extern void pgstat_reset_counters(void); @@ -828,6 +825,7 @@ extern PgStat_WalStats *pgstat_fetch_stat_wal(void); extern PGDLLIMPORT bool pgstat_track_counts; extern PGDLLIMPORT int pgstat_track_functions; extern PGDLLIMPORT int pgstat_fetch_consistency; +extern PGDLLIMPORT int pgstat_flush_interval; /* diff --git a/src/include/utils/guc_hooks.h b/src/include/utils/guc_hooks.h index b6ecb0e769f..3a2ae6c41cd 100644 --- a/src/include/utils/guc_hooks.h +++ b/src/include/utils/guc_hooks.h @@ -132,6 +132,7 @@ extern bool check_session_authorization(char **newval, void **extra, GucSource s extern void assign_session_authorization(const char *newval, void *extra); extern void assign_session_replication_role(int newval, void *extra); extern void assign_stats_fetch_consistency(int newval, void *extra); +extern void assign_stats_flush_interval(int newval, void *extra); extern bool check_ssl(bool *newval, void **extra, GucSource source); extern bool check_stage_log_stats(bool *newval, void **extra, GucSource source); extern bool check_standard_conforming_strings(bool *newval, void **extra, -- 2.34.1
>From de9afb385c171f3e0f8de2a0dbe4f72143216686 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot <[email protected]> Date: Tue, 6 Jan 2026 11:06:31 +0000 Subject: [PATCH v6 4/5] Remove useless calls to flush some stats Now that some stats can be flushed outside of transaction boundaries, remove useless calls to report/flush some stats. Those calls were in place because before commit <XXXX> stats were flushed only at transaction boundaries. Note that: - it reverts 039549d70f6 (it just keeps its tests) - it can't be done for checkpointer and bgworker for example because they don't have a flush callback to call - it can't be done for auxiliary process (walsummarizer for example) because they currently do not register the new timeout handler --- src/backend/replication/walreceiver.c | 10 ------ src/backend/replication/walsender.c | 36 ++------------------ src/backend/utils/activity/pgstat_relation.c | 13 ------- src/test/recovery/t/001_stream_rep.pl | 1 + src/test/subscription/t/001_rep_changes.pl | 1 + 5 files changed, 4 insertions(+), 57 deletions(-) 69.4% src/backend/replication/ 23.4% src/backend/utils/activity/ 3.4% src/test/recovery/t/ 3.5% src/test/subscription/t/ diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index 11b7c114d3b..953ba97ed00 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -571,16 +571,6 @@ WalReceiverMain(const void *startup_data, size_t startup_data_len) */ bool requestReply = false; - /* - * Report pending statistics to the cumulative stats - * system. This location is useful for the report as it - * is not within a tight loop in the WAL receiver, to - * avoid bloating pgstats with requests, while also making - * sure that the reports happen each time a status update - * is sent. - */ - pgstat_report_wal(false); - /* * Check if time since last receive from primary has * reached the configured limit. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index a7214d0dc6f..9a136e35b48 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -94,14 +94,10 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/pg_lsn.h" -#include "utils/pgstat_internal.h" #include "utils/ps_status.h" #include "utils/timeout.h" #include "utils/timestamp.h" -/* Minimum interval used by walsender for stats flushes, in ms */ -#define WALSENDER_STATS_FLUSH_INTERVAL 1000 - /* * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. * @@ -1846,7 +1842,6 @@ WalSndWaitForWal(XLogRecPtr loc) int wakeEvents; uint32 wait_event = 0; static XLogRecPtr RecentFlushPtr = InvalidXLogRecPtr; - TimestampTz last_flush = 0; /* * Fast path to avoid acquiring the spinlock in case we already know we @@ -1867,7 +1862,6 @@ WalSndWaitForWal(XLogRecPtr loc) { bool wait_for_standby_at_stop = false; long sleeptime; - TimestampTz now; /* Clear any already-pending wakeups */ ResetLatch(MyLatch); @@ -1973,8 +1967,7 @@ WalSndWaitForWal(XLogRecPtr loc) * new WAL to be generated. (But if we have nothing to send, we don't * want to wake on socket-writable.) */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); wakeEvents = WL_SOCKET_READABLE; @@ -1983,15 +1976,6 @@ WalSndWaitForWal(XLogRecPtr loc) Assert(wait_event != 0); - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - WalSndWait(wakeEvents, sleeptime, wait_event); } @@ -2894,8 +2878,6 @@ WalSndCheckTimeOut(void) static void WalSndLoop(WalSndSendDataCallback send_data) { - TimestampTz last_flush = 0; - /* * Initialize the last reply timestamp. That enables timeout processing * from hereon. @@ -2985,9 +2967,6 @@ WalSndLoop(WalSndSendDataCallback send_data) * WalSndWaitForWal() handle any other blocking; idle receivers need * its additional actions. For physical replication, also block if * caught up; its send_data does not block. - * - * The IO statistics are reported in WalSndWaitForWal() for the - * logical WAL senders. */ if ((WalSndCaughtUp && send_data != XLogSendLogical && !streamingDoneSending) || @@ -2995,7 +2974,6 @@ WalSndLoop(WalSndSendDataCallback send_data) { long sleeptime; int wakeEvents; - TimestampTz now; if (!streamingDoneReceiving) wakeEvents = WL_SOCKET_READABLE; @@ -3006,21 +2984,11 @@ WalSndLoop(WalSndSendDataCallback send_data) * Use fresh timestamp, not last_processing, to reduce the chance * of reaching wal_sender_timeout before sending a keepalive. */ - now = GetCurrentTimestamp(); - sleeptime = WalSndComputeSleeptime(now); + sleeptime = WalSndComputeSleeptime(GetCurrentTimestamp()); if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; - /* Report IO statistics, if needed */ - if (TimestampDifferenceExceeds(last_flush, now, - WALSENDER_STATS_FLUSH_INTERVAL)) - { - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); - last_flush = now; - } - /* Sleep until something happens or we time out */ WalSndWait(wakeEvents, sleeptime, WAIT_EVENT_WAL_SENDER_MAIN); } diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 04d21483d93..ae2952cae89 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -260,15 +260,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, } pgstat_unlock_entry(entry_ref); - - /* - * Flush IO statistics now. pgstat_report_stat() will flush IO stats, - * however this will not be called until after an entire autovacuum cycle - * is done -- which will likely vacuum many relations -- or until the - * VACUUM command has processed all tables and committed. - */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* @@ -360,10 +351,6 @@ pgstat_report_analyze(Relation rel, } pgstat_unlock_entry(entry_ref); - - /* see pgstat_report_vacuum() */ - pgstat_flush_io(false, true); - (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO, true); } /* diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl index e9ac67813c7..c058a5f9b1f 100644 --- a/src/test/recovery/t/001_stream_rep.pl +++ b/src/test/recovery/t/001_stream_rep.pl @@ -15,6 +15,7 @@ my $node_primary = PostgreSQL::Test::Cluster->new('primary'); $node_primary->init( allows_streaming => 1, auth_extra => [ '--create-role' => 'repl_role' ]); +$node_primary->append_conf('postgresql.conf', "stats_flush_interval= '1s'"); $node_primary->start; my $backup_name = 'my_backup'; diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl index 7d41715ed81..bceec2adede 100644 --- a/src/test/subscription/t/001_rep_changes.pl +++ b/src/test/subscription/t/001_rep_changes.pl @@ -11,6 +11,7 @@ use Test::More; # Initialize publisher node my $node_publisher = PostgreSQL::Test::Cluster->new('publisher'); $node_publisher->init(allows_streaming => 'logical'); +$node_publisher->append_conf('postgresql.conf', "stats_flush_interval= '1s'"); $node_publisher->start; # Create subscriber node -- 2.34.1
>From 5a57a7034acf37b11f9020e415c801963270bd33 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot <[email protected]> Date: Mon, 19 Jan 2026 06:27:55 +0000 Subject: [PATCH v6 5/5] Change RELATION and DATABASE stats to anytime flush This commit allows mixing fields with different transaction behavior within the same RELATION or DATABASE statistics kind: some fields are transactional (e.g., tuple inserts/updates/deletes) while others are non-transactional (e.g., sequential scans, blocks read). It modifies the relation flush callback to handle the anytime_only parameter introduced in commit <nnnn>. Implementation details: - Change RELATION from FLUSH_AT_TXN_BOUNDARY to FLUSH_ANYTIME - Change DATABASE from FLUSH_AT_TXN_BOUNDARY to FLUSH_ANYTIME - Modify pgstat_relation_flush_cb() to handle anytime_only parameter: when true, then flush only non-transactional stats and when false, then flush all the stats. When set to true, it clears flushed fields from pending stats to prevent double-counting at transaction boundary DATABASE stats inherit the anytime flush behavior so that relation-derived stats (tuples_returned, tuples_fetched, blocks_fetched, blocks_hit) are visible while transactions are in progress. Tests are added to verify the anytime flush behavior for mixed fields. --- doc/src/sgml/monitoring.sgml | 37 ++++++- src/backend/utils/activity/pgstat.c | 17 ++-- src/backend/utils/activity/pgstat_relation.c | 86 ++++++++++++---- src/include/pgstat.h | 27 ++++- src/test/isolation/expected/stats.out | 102 +++++++++++++++++++ src/test/isolation/expected/stats_1.out | 102 +++++++++++++++++++ src/test/isolation/specs/stats.spec | 27 ++++- 7 files changed, 368 insertions(+), 30 deletions(-) 13.0% doc/src/sgml/ 25.9% src/backend/utils/activity/ 6.0% src/include/ 49.8% src/test/isolation/expected/ 5.1% src/test/isolation/specs/ diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index b77d189a500..aa7bd2e2e2a 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -3767,6 +3767,19 @@ description | Waiting for a newly initialized WAL file to reach durable storage </tgroup> </table> + <note> + <para> + Some statistics are updated while a transaction is in progress (for example, + <structfield>blks_read</structfield>, <structfield>blks_hit</structfield>, + <structfield>tup_returned</structfield> and <structfield>tup_fetched</structfield>). + Statistics that either do not depend on transactions or require transactional + consistency are updated only when the transaction ends. Statistics that require + transactional consistency include <structfield>xact_commit</structfield>, + <structfield>xact_rollback</structfield>, <structfield>tup_inserted</structfield>, + <structfield>tup_updated</structfield> and <structfield>tup_deleted</structfield>. + </para> + </note> + </sect2> <sect2 id="monitoring-pg-stat-database-conflicts-view"> @@ -3956,8 +3969,8 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structfield>last_seq_scan</structfield> <type>timestamp with time zone</type> </para> <para> - The time of the last sequential scan on this table, based on the - most recent transaction stop time + The approximate time of the last sequential scan on this table, updated + at least every <varname>stats_flush_interval</varname> </para></entry> </row> @@ -3984,8 +3997,8 @@ description | Waiting for a newly initialized WAL file to reach durable storage <structfield>last_idx_scan</structfield> <type>timestamp with time zone</type> </para> <para> - The time of the last index scan on this table, based on the - most recent transaction stop time + The approximate time of the last index scan on this table, updated + at least every <varname>stats_flush_interval</varname> </para></entry> </row> @@ -4223,6 +4236,15 @@ description | Waiting for a newly initialized WAL file to reach durable storage </tgroup> </table> + <note> + <para> + The <structfield>seq_scan</structfield>, <structfield>last_seq_scan</structfield>, + <structfield>seq_tup_read</structfield>, <structfield>idx_scan</structfield>, + <structfield>last_idx_scan</structfield> and <structfield>idx_tup_fetch</structfield> + are updated while the transactions are in progress. + </para> + </note> + </sect2> <sect2 id="monitoring-pg-stat-all-indexes-view"> @@ -4404,6 +4426,13 @@ description | Waiting for a newly initialized WAL file to reach durable storage tuples (see <xref linkend="indexes-multicolumn"/>). </para> </note> + <note> + <para> + The <structfield>idx_scan</structfield>, <structfield>last_idx_scan</structfield>, + <structfield>idx_tup_read</structfield> and <structfield>idx_tup_fetch</structfield> + are updated while the transactions are in progress. + </para> + </note> <tip> <para> <command>EXPLAIN ANALYZE</command> outputs the total number of index diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c index 79eb59b5625..9234185de64 100644 --- a/src/backend/utils/activity/pgstat.c +++ b/src/backend/utils/activity/pgstat.c @@ -291,7 +291,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .fixed_amount = false, .write_to_file = true, - .flush_mode = FLUSH_AT_TXN_BOUNDARY, + .flush_mode = FLUSH_ANYTIME, /* so pg_stat_database entries can be seen in all databases */ .accessed_across_databases = true, @@ -309,7 +309,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .fixed_amount = false, .write_to_file = true, - .flush_mode = FLUSH_AT_TXN_BOUNDARY, + .flush_mode = FLUSH_ANYTIME, .shared_size = sizeof(PgStatShared_Relation), .shared_data_off = offsetof(PgStatShared_Relation, stats), @@ -1344,10 +1344,12 @@ pgstat_delete_pending_entry(PgStat_EntryRef *entry_ref) dlist_delete(&entry_ref->pending_node); } + /* * Flush out pending variable-numbered stats. * - * If anytime_only is true, only flushes FLUSH_ANYTIME entries. + * If anytime_only is true, only flushes FLUSH_ANYTIME entries. For entries + * that support it, the callback may flush only non-transactional fields. * This is safe to call inside transactions. * * If anytime_only is false, flushes all entries. @@ -1378,6 +1380,7 @@ pgstat_flush_pending_entries(bool nowait, bool anytime_only) PgStat_Kind kind = key.kind; const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind); bool did_flush; + bool is_partial_flush = false; dlist_node *next; Assert(!kind_info->fixed_amount); @@ -1397,19 +1400,21 @@ pgstat_flush_pending_entries(bool nowait, bool anytime_only) continue; } - /* flush the stats, if possible */ did_flush = kind_info->flush_pending_cb(entry_ref, nowait, anytime_only); Assert(did_flush || nowait); + /* Partial flush only happens in anytime mode for FLUSH_ANYTIME stats */ + is_partial_flush = (anytime_only && kind_info->flush_mode == FLUSH_ANYTIME); + /* determine next entry, before deleting the pending entry */ if (dlist_has_next(&pgStatPending, cur)) next = dlist_next_node(&pgStatPending, cur); else next = NULL; - /* if successfully flushed, remove entry */ - if (did_flush) + /* if successfull non-partial flush, remove entry */ + if (did_flush && !is_partial_flush) pgstat_delete_pending_entry(entry_ref); else have_pending = true; diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index ae2952cae89..10ab9e4cc76 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -47,7 +47,19 @@ static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_lev static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info); static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop); static void restore_truncdrop_counters(PgStat_TableXactStatus *trans); +static void flush_relation_anytime_stats(PgStat_StatTabEntry *tabentry, + PgStat_TableCounts *counts, bool anytime_only); +/* + * Update database statistics with non-transactional stats. + */ +#define UPDATE_DATABASE_ANYTIME_STATS(dbentry, counts) \ + do { \ + (dbentry)->tuples_returned += (counts)->tuples_returned; \ + (dbentry)->tuples_fetched += (counts)->tuples_fetched; \ + (dbentry)->blocks_fetched += (counts)->blocks_fetched; \ + (dbentry)->blocks_hit += (counts)->blocks_hit; \ + } while (0) /* * Copy stats between relations. This is used for things like REINDEX @@ -789,6 +801,29 @@ pgstat_twophase_postabort(FullTransactionId fxid, uint16 info, rec->tuples_inserted + rec->tuples_updated; } +/* + * Helper function to flush non-transactional statistics. + */ +static void +flush_relation_anytime_stats(PgStat_StatTabEntry *tabentry, PgStat_TableCounts *counts, + bool anytime_only) +{ + TimestampTz t; + + tabentry->numscans += counts->numscans; + if (counts->numscans) + { + t = anytime_only ? GetCurrentTimestamp() : GetCurrentTransactionStopTimestamp(); + if (t > tabentry->lastscan) + tabentry->lastscan = t; + } + + tabentry->tuples_returned += counts->tuples_returned; + tabentry->tuples_fetched += counts->tuples_fetched; + tabentry->blocks_fetched += counts->blocks_fetched; + tabentry->blocks_hit += counts->blocks_hit; +} + /* * Flush out pending stats for the entry * @@ -797,6 +832,13 @@ pgstat_twophase_postabort(FullTransactionId fxid, uint16 info, * * Some of the stats are copied to the corresponding pending database stats * entry when successfully flushing. + * + * If anytime_only is true, only non-transactional fields are flushed + * (numscans, tuples_returned, tuples_fetched, blocks_fetched, blocks_hit). + * Transactional fields remain pending until transaction boundary. + * + * Some of the stats are copied to the corresponding pending database stats + * entry when successfully flushing. */ bool pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_only) @@ -807,8 +849,6 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_o PgStat_StatTabEntry *tabentry; /* table entry of shared stats */ PgStat_StatDBEntry *dbentry; /* pending database entry */ - Assert(!anytime_only); - dboid = entry_ref->shared_entry->key.dboid; lstats = (PgStat_TableStatus *) entry_ref->pending; shtabstats = (PgStatShared_Relation *) entry_ref->shared_stats; @@ -824,19 +864,36 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_o if (!pgstat_lock_entry(entry_ref, nowait)) return false; - /* add the values to the shared entry. */ tabentry = &shtabstats->stats; - tabentry->numscans += lstats->counts.numscans; - if (lstats->counts.numscans) + if (anytime_only) { - TimestampTz t = GetCurrentTransactionStopTimestamp(); - if (t > tabentry->lastscan) - tabentry->lastscan = t; + /* Flush non-transactional statistics */ + flush_relation_anytime_stats(tabentry, &lstats->counts, true); + + pgstat_unlock_entry(entry_ref); + + /* Also update the corresponding fields in database stats */ + dbentry = pgstat_prep_database_pending(dboid); + UPDATE_DATABASE_ANYTIME_STATS(dbentry, &lstats->counts); + + /* + * Clear the flushed fields from pending stats to prevent + * double-counting when we flush all fields at transaction boundary. + */ + lstats->counts.numscans = 0; + lstats->counts.tuples_returned = 0; + lstats->counts.tuples_fetched = 0; + lstats->counts.blocks_fetched = 0; + lstats->counts.blocks_hit = 0; + + return true; } - tabentry->tuples_returned += lstats->counts.tuples_returned; - tabentry->tuples_fetched += lstats->counts.tuples_fetched; + + /* Flush non-transactional statistics */ + flush_relation_anytime_stats(tabentry, &lstats->counts, false); + tabentry->tuples_inserted += lstats->counts.tuples_inserted; tabentry->tuples_updated += lstats->counts.tuples_updated; tabentry->tuples_deleted += lstats->counts.tuples_deleted; @@ -866,9 +923,6 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_o */ tabentry->ins_since_vacuum += lstats->counts.tuples_inserted; - tabentry->blocks_fetched += lstats->counts.blocks_fetched; - tabentry->blocks_hit += lstats->counts.blocks_hit; - /* Clamp live_tuples in case of negative delta_live_tuples */ tabentry->live_tuples = Max(tabentry->live_tuples, 0); /* Likewise for dead_tuples */ @@ -878,13 +932,11 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait, bool anytime_o /* The entry was successfully flushed, add the same to database stats */ dbentry = pgstat_prep_database_pending(dboid); - dbentry->tuples_returned += lstats->counts.tuples_returned; - dbentry->tuples_fetched += lstats->counts.tuples_fetched; + UPDATE_DATABASE_ANYTIME_STATS(dbentry, &lstats->counts); + dbentry->tuples_inserted += lstats->counts.tuples_inserted; dbentry->tuples_updated += lstats->counts.tuples_updated; dbentry->tuples_deleted += lstats->counts.tuples_deleted; - dbentry->blocks_fetched += lstats->counts.blocks_fetched; - dbentry->blocks_hit += lstats->counts.blocks_hit; return true; } diff --git a/src/include/pgstat.h b/src/include/pgstat.h index ef856dbf55b..06639198f28 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -21,6 +21,7 @@ #include "utils/backend_status.h" /* for backward compatibility */ /* IWYU pragma: export */ #include "utils/pgstat_kind.h" #include "utils/relcache.h" +#include "utils/timeout.h" #include "utils/wait_event.h" /* for backward compatibility */ /* IWYU pragma: export */ @@ -537,10 +538,11 @@ extern void pgstat_report_anytime_stat(bool force); extern void pgstat_force_next_flush(void); /* - * Schedule the next anytime stats update timeout. + * Schedule the next anytime stats update timeout and mark that we have + * mixed anytime stats pending. * * This should be called whenever accumulating statistics that support - * FLUSH_ANYTIME flushing mode. + * FLUSH_ANYTIME or FLUSH_MIXED flushing modes. */ #define pgstat_schedule_anytime_update() \ do { \ @@ -703,37 +705,58 @@ extern void pgstat_report_analyze(Relation rel, #define pgstat_count_heap_scan(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ + { \ (rel)->pgstat_info->counts.numscans++; \ + pgstat_schedule_anytime_update(); \ + } \ } while (0) #define pgstat_count_heap_getnext(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ + { \ (rel)->pgstat_info->counts.tuples_returned++; \ + pgstat_schedule_anytime_update(); \ + } \ } while (0) #define pgstat_count_heap_fetch(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ + { \ (rel)->pgstat_info->counts.tuples_fetched++; \ + pgstat_schedule_anytime_update(); \ + } \ } while (0) #define pgstat_count_index_scan(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ + { \ (rel)->pgstat_info->counts.numscans++; \ + pgstat_schedule_anytime_update(); \ + } \ } while (0) #define pgstat_count_index_tuples(rel, n) \ do { \ if (pgstat_should_count_relation(rel)) \ + { \ (rel)->pgstat_info->counts.tuples_returned += (n); \ + pgstat_schedule_anytime_update(); \ + } \ } while (0) #define pgstat_count_buffer_read(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ + { \ (rel)->pgstat_info->counts.blocks_fetched++; \ + pgstat_schedule_anytime_update(); \ + } \ } while (0) #define pgstat_count_buffer_hit(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ + { \ (rel)->pgstat_info->counts.blocks_hit++; \ + pgstat_schedule_anytime_update(); \ + } \ } while (0) extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n); diff --git a/src/test/isolation/expected/stats.out b/src/test/isolation/expected/stats.out index cfad309ccf3..11e3e57806d 100644 --- a/src/test/isolation/expected/stats.out +++ b/src/test/isolation/expected/stats.out @@ -2245,6 +2245,108 @@ seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum (1 row) +starting permutation: s2_begin s2_table_select s1_sleep s1_table_stats s2_track_counts_off s2_table_select s1_sleep s1_table_stats s2_track_counts_on s2_table_select s1_sleep s1_table_stats s2_table_drop s2_commit +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_begin: BEGIN; +step s2_table_select: SELECT * FROM test_stat_tab ORDER BY key, value; +key|value +---+----- +k0 | 1 +(1 row) + +step s1_sleep: SELECT pg_sleep(1.5); +pg_sleep +-------- + +(1 row) + +step s1_table_stats: + SELECT + pg_stat_get_numscans(tso.oid) AS seq_scan, + pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, + pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, + pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, + pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, + pg_stat_get_live_tuples(tso.oid) AS n_live_tup, + pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_vacuum_count(tso.oid) AS vacuum_count + FROM test_stat_oid AS tso + WHERE tso.name = 'test_stat_tab' + +seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count +--------+------------+---------+---------+---------+----------+----------+------------ + 1| 1| 1| 0| 0| 1| 0| 0 +(1 row) + +step s2_track_counts_off: SET track_counts = off; +step s2_table_select: SELECT * FROM test_stat_tab ORDER BY key, value; +key|value +---+----- +k0 | 1 +(1 row) + +step s1_sleep: SELECT pg_sleep(1.5); +pg_sleep +-------- + +(1 row) + +step s1_table_stats: + SELECT + pg_stat_get_numscans(tso.oid) AS seq_scan, + pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, + pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, + pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, + pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, + pg_stat_get_live_tuples(tso.oid) AS n_live_tup, + pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_vacuum_count(tso.oid) AS vacuum_count + FROM test_stat_oid AS tso + WHERE tso.name = 'test_stat_tab' + +seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count +--------+------------+---------+---------+---------+----------+----------+------------ + 1| 1| 1| 0| 0| 1| 0| 0 +(1 row) + +step s2_track_counts_on: SET track_counts = on; +step s2_table_select: SELECT * FROM test_stat_tab ORDER BY key, value; +key|value +---+----- +k0 | 1 +(1 row) + +step s1_sleep: SELECT pg_sleep(1.5); +pg_sleep +-------- + +(1 row) + +step s1_table_stats: + SELECT + pg_stat_get_numscans(tso.oid) AS seq_scan, + pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, + pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, + pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, + pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, + pg_stat_get_live_tuples(tso.oid) AS n_live_tup, + pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_vacuum_count(tso.oid) AS vacuum_count + FROM test_stat_oid AS tso + WHERE tso.name = 'test_stat_tab' + +seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count +--------+------------+---------+---------+---------+----------+----------+------------ + 2| 2| 1| 0| 0| 1| 0| 0 +(1 row) + +step s2_table_drop: DROP TABLE test_stat_tab; +step s2_commit: COMMIT; + starting permutation: s1_track_counts_off s1_table_stats s1_track_counts_on pg_stat_force_next_flush ------------------------ diff --git a/src/test/isolation/expected/stats_1.out b/src/test/isolation/expected/stats_1.out index e1d937784cb..aef582e7582 100644 --- a/src/test/isolation/expected/stats_1.out +++ b/src/test/isolation/expected/stats_1.out @@ -2253,6 +2253,108 @@ seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum (1 row) +starting permutation: s2_begin s2_table_select s1_sleep s1_table_stats s2_track_counts_off s2_table_select s1_sleep s1_table_stats s2_track_counts_on s2_table_select s1_sleep s1_table_stats s2_table_drop s2_commit +pg_stat_force_next_flush +------------------------ + +(1 row) + +step s2_begin: BEGIN; +step s2_table_select: SELECT * FROM test_stat_tab ORDER BY key, value; +key|value +---+----- +k0 | 1 +(1 row) + +step s1_sleep: SELECT pg_sleep(1.5); +pg_sleep +-------- + +(1 row) + +step s1_table_stats: + SELECT + pg_stat_get_numscans(tso.oid) AS seq_scan, + pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, + pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, + pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, + pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, + pg_stat_get_live_tuples(tso.oid) AS n_live_tup, + pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_vacuum_count(tso.oid) AS vacuum_count + FROM test_stat_oid AS tso + WHERE tso.name = 'test_stat_tab' + +seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count +--------+------------+---------+---------+---------+----------+----------+------------ + 1| 1| 1| 0| 0| 1| 0| 0 +(1 row) + +step s2_track_counts_off: SET track_counts = off; +step s2_table_select: SELECT * FROM test_stat_tab ORDER BY key, value; +key|value +---+----- +k0 | 1 +(1 row) + +step s1_sleep: SELECT pg_sleep(1.5); +pg_sleep +-------- + +(1 row) + +step s1_table_stats: + SELECT + pg_stat_get_numscans(tso.oid) AS seq_scan, + pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, + pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, + pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, + pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, + pg_stat_get_live_tuples(tso.oid) AS n_live_tup, + pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_vacuum_count(tso.oid) AS vacuum_count + FROM test_stat_oid AS tso + WHERE tso.name = 'test_stat_tab' + +seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count +--------+------------+---------+---------+---------+----------+----------+------------ + 1| 1| 1| 0| 0| 1| 0| 0 +(1 row) + +step s2_track_counts_on: SET track_counts = on; +step s2_table_select: SELECT * FROM test_stat_tab ORDER BY key, value; +key|value +---+----- +k0 | 1 +(1 row) + +step s1_sleep: SELECT pg_sleep(1.5); +pg_sleep +-------- + +(1 row) + +step s1_table_stats: + SELECT + pg_stat_get_numscans(tso.oid) AS seq_scan, + pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, + pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, + pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, + pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, + pg_stat_get_live_tuples(tso.oid) AS n_live_tup, + pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_vacuum_count(tso.oid) AS vacuum_count + FROM test_stat_oid AS tso + WHERE tso.name = 'test_stat_tab' + +seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count +--------+------------+---------+---------+---------+----------+----------+------------ + 2| 2| 1| 0| 0| 1| 0| 0 +(1 row) + +step s2_table_drop: DROP TABLE test_stat_tab; +step s2_commit: COMMIT; + starting permutation: s1_track_counts_off s1_table_stats s1_track_counts_on pg_stat_force_next_flush ------------------------ diff --git a/src/test/isolation/specs/stats.spec b/src/test/isolation/specs/stats.spec index da16710da0f..47414eb6009 100644 --- a/src/test/isolation/specs/stats.spec +++ b/src/test/isolation/specs/stats.spec @@ -50,6 +50,8 @@ step s1_rollback { ROLLBACK; } step s1_prepare_a { PREPARE TRANSACTION 'a'; } step s1_commit_prepared_a { COMMIT PREPARED 'a'; } step s1_rollback_prepared_a { ROLLBACK PREPARED 'a'; } +# Has to be greater than session 2 stats_flush_interval +step s1_sleep { SELECT pg_sleep(1.5); } # Function stats steps step s1_ff { SELECT pg_stat_force_next_flush(); } @@ -132,12 +134,16 @@ step s1_slru_check_stats { session s2 -setup { SET stats_fetch_consistency = 'none'; } +setup { + SET stats_fetch_consistency = 'none'; + SET stats_flush_interval = '1s'; +} step s2_begin { BEGIN; } step s2_commit { COMMIT; } step s2_commit_prepared_a { COMMIT PREPARED 'a'; } step s2_rollback_prepared_a { ROLLBACK PREPARED 'a'; } step s2_ff { SELECT pg_stat_force_next_flush(); } +step s2_table_drop { DROP TABLE test_stat_tab; } # Function stats steps step s2_track_funcs_all { SET track_functions = 'all'; } @@ -156,6 +162,8 @@ step s2_func_stats { } # Relation stats steps +step s2_track_counts_on { SET track_counts = on; } +step s2_track_counts_off { SET track_counts = off; } step s2_table_select { SELECT * FROM test_stat_tab ORDER BY key, value; } step s2_table_update_k1 { UPDATE test_stat_tab SET value = value + 1 WHERE key = 'k1';} @@ -435,6 +443,23 @@ permutation s1_table_drop s1_table_stats +### Check that some stats are updated (seq_scan and seq_tup_read) +### while the transaction is still running +permutation + s2_begin + s2_table_select + s1_sleep + s1_table_stats + s2_track_counts_off + s2_table_select + s1_sleep + s1_table_stats + s2_track_counts_on + s2_table_select + s1_sleep + s1_table_stats + s2_table_drop + s2_commit ### Check that we don't count changes with track counts off, but allow access ### to prior stats -- 2.34.1
