On Thu, Dec 30, 2021 at 3:30 PM Melanie Plageman
<melanieplage...@gmail.com> wrote:
>
> On Tue, Dec 21, 2021 at 8:32 PM Melanie Plageman
> <melanieplage...@gmail.com> wrote:
> > On Thu, Dec 16, 2021 at 3:18 PM Andres Freund <and...@anarazel.de> wrote:
> > > > > > From 9f22da9041e1e1fbc0ef003f5f78f4e72274d438 Mon Sep 17 00:00:00
> > > > > > 2001
> > > > > > From: Melanie Plageman <melanieplage...@gmail.com>
> > > > > > Date: Wed, 24 Nov 2021 12:20:10 -0500
> > > > > > Subject: [PATCH v17 6/7] Remove superfluous bgwriter stats
> > > > > >
> > > > > > Remove stats from pg_stat_bgwriter which are now more clearly
> > > > > > expressed
> > > > > > in pg_stat_buffers.
> > > > > >
> > > > > > TODO:
> > > > > > - make pg_stat_checkpointer view and move relevant stats into it
> > > > > > - add additional stats to pg_stat_bgwriter
> > > > >
> > > > > When do you think it makes sense to tackle these wrt committing some
> > > > > of the
> > > > > patches?
> > > >
> > > > Well, the new stats are a superset of the old stats (no stats have been
> > > > removed that are not represented in the new or old views). So, I don't
> > > > see that as a blocker for committing these patches.
> > >
> > > > Since it is weird that pg_stat_bgwriter had mostly checkpointer stats,
> > > > I've edited this commit to rename that view to pg_stat_checkpointer.
> > >
> > > > I have not made a separate view just for maxwritten_clean (presumably
> > > > called pg_stat_bgwriter), but I would not be opposed to doing this if
> > > > you thought having a view with a single column isn't a problem (in the
> > > > event that we don't get around to adding more bgwriter stats right
> > > > away).
> > >
> > > How about keeping old bgwriter values in place in the view , but generated
> > > from the new stats stuff?
> >
> > I tried this, but I actually don't think it is the right way to go. In
> > order to maintain the old view with the new source code, I had to add
> > new code to maintain a separate resets array just for the bgwriter view.
> > It adds some fiddly code that will be annoying to maintain (the reset
> > logic is confusing enough as is).
> > And, besides the implementation complexity, if a user resets
> > pg_stat_bgwriter and not pg_stat_buffers (or vice versa), they will
> > see totally different numbers for "buffers_backend" in pg_stat_bgwriter
> > than shared buffers written by B_BACKEND in pg_stat_buffers. I would
> > find that confusing.
>
> In a quick chat off-list, Andres suggested it might be okay to have a
> single reset target for both the pg_stat_buffers view and legacy
> pg_stat_bgwriter view. So, I am planning to share a new patchset which
> has only the new "buffers" target which will also reset the legacy
> pg_stat_bgwriter view.
>
> I'll also remove the bgwriter stats I proposed and the
> pg_stat_checkpointer view to keep things simple for now.
>
I've done the above in v20, attached.
- Melanie
From 14e20657d6a6674cc286b5ce1a29560889cf7833 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplage...@gmail.com>
Date: Wed, 24 Nov 2021 12:07:37 -0500
Subject: [PATCH v20 6/8] Add system view tracking IO ops per backend type
Add pg_stat_buffers, a system view which tracks the number of IO
operations (allocs, writes, fsyncs, and extends) done through each IO
path (e.g. shared buffers, local buffers, unbuffered IO) by each type of
backend.
Some of these should always be zero. For example, checkpointer does not
use a BufferAccessStrategy (currently), so the "strategy" IO Path for
checkpointer will be 0 for all IO operations (alloc, write, fsync, and
extend). All possible combinations of IOPath and IOOp are enumerated in
the view but not all are populated or even possible at this point.
All backends increment a counter in an array of IO stat counters in
their PgBackendStatus when performing an IO operation. On exit, backends
send these stats to the stats collector to be persisted.
When the pg_stat_buffers view is queried, one backend will sum live
backends' stats with saved stats from exited backends and subtract saved
reset stats, returning the total.
Each row of the view is stats for a particular backend type for a
particular IO Path (e.g. shared buffer accesses by checkpointer) and
each column in the view is the total number of IO operations done (e.g.
writes).
So a cell in the view would be, for example, the number of shared
buffers written by checkpointer since the last stats reset.
Suggested by Andres Freund
Author: Melanie Plageman <melanieplage...@gmail.com>
Reviewed-by: Justin Pryzby <pry...@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/flat/20200124195226.lth52iydq2n2uilq%40alap3.anarazel.de
---
doc/src/sgml/monitoring.sgml | 119 +++++++++++++++-
src/backend/catalog/system_views.sql | 11 ++
src/backend/postmaster/pgstat.c | 13 ++
src/backend/utils/activity/backend_status.c | 19 ++-
src/backend/utils/adt/pgstatfuncs.c | 150 ++++++++++++++++++++
src/include/catalog/pg_proc.dat | 9 ++
src/include/pgstat.h | 1 +
src/include/utils/backend_status.h | 1 +
src/test/regress/expected/rules.out | 8 ++
9 files changed, 323 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index c94eb57a59..30df3d473e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -435,6 +435,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</entry>
</row>
+ <row>
+ <entry><structname>pg_stat_buffers</structname><indexterm><primary>pg_stat_buffers</primary></indexterm></entry>
+ <entry>A row for each IO path for each backend type showing
+ statistics about backend IO operations. See
+ <link linkend="monitoring-pg-stat-buffers-view">
+ <structname>pg_stat_buffers</structname></link> for details.
+ </entry>
+ </row>
+
<row>
<entry><structname>pg_stat_wal</structname><indexterm><primary>pg_stat_wal</primary></indexterm></entry>
<entry>One row only, showing statistics about WAL activity. See
@@ -3604,7 +3613,102 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
<structfield>stats_reset</structfield> <type>timestamp with time zone</type>
</para>
<para>
- Time at which these statistics were last reset
+ Time at which these statistics were last reset.
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ </sect2>
+
+ <sect2 id="monitoring-pg-stat-buffers-view">
+ <title><structname>pg_stat_buffers</structname></title>
+
+ <indexterm>
+ <primary>pg_stat_buffers</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_stat_buffers</structname> view has a row for each backend
+ type for each possible IO path containing global data for the cluster for
+ that backend and IO path.
+ </para>
+
+ <table id="pg-stat-buffers-view" xreflabel="pg_stat_buffers">
+ <title><structname>pg_stat_buffers</structname> View</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>backend_type</structfield> <type>text</type>
+ </para>
+ <para>
+ Type of backend (e.g. background worker, autovacuum worker).
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>io_path</structfield> <type>text</type>
+ </para>
+ <para>
+ IO path taken (e.g. shared buffers, direct).
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>alloc</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of buffers allocated.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>extend</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of buffers extended.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>fsync</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of buffers fsynced.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>write</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of buffers written.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>stats_reset</structfield> <type>timestamp with time zone</type>
+ </para>
+ <para>
+ Time at which these statistics were last reset.
</para></entry>
</row>
</tbody>
@@ -5209,12 +5313,13 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
</para>
<para>
Resets some cluster-wide statistics counters to zero, depending on the
- argument. The argument can be <literal>buffers</literal> to reset
- all the counters shown in
- the <structname>pg_stat_bgwriter</structname>
- view, <literal>archiver</literal> to reset all the counters shown in
- the <structname>pg_stat_archiver</structname> view or <literal>wal</literal>
- to reset all the counters shown in the <structname>pg_stat_wal</structname> view.
+ argument. The argument can be <literal>archiver</literal> to reset all
+ the counters shown in the <structname>pg_stat_archiver</structname>
+ view, <literal>buffers</literal> to reset all the counters shown in
+ both the <structname>pg_stat_bgwriter</structname> view and
+ <structname>pg_stat_buffers</structname> view, or
+ <literal>wal</literal> to reset all the counters shown in the
+ <structname>pg_stat_wal</structname> view.
</para>
<para>
This function is restricted to superusers by default, but other users
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 61b515cdb8..e214d23056 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1076,6 +1076,17 @@ CREATE VIEW pg_stat_bgwriter AS
pg_stat_get_buf_alloc() AS buffers_alloc,
pg_stat_get_bgwriter_stat_reset_time() AS stats_reset;
+CREATE VIEW pg_stat_buffers AS
+SELECT
+ b.backend_type,
+ b.io_path,
+ b.alloc,
+ b.extend,
+ b.fsync,
+ b.write,
+ b.stats_reset
+FROM pg_stat_get_buffers() b;
+
CREATE VIEW pg_stat_wal AS
SELECT
w.wal_records,
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 71db0b7b14..f4bff26630 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -2921,6 +2921,19 @@ pgstat_twophase_postabort(TransactionId xid, uint16 info,
rec->tuples_inserted + rec->tuples_updated;
}
+/*
+ * Support function for SQL-callable pgstat* functions. Returns a pointer to
+ * the PgStat_BackendIOPathOps structure tracking IO operations statistics for
+ * both exited backends and reset arithmetic.
+ */
+PgStat_BackendIOPathOps *
+pgstat_fetch_exited_backend_buffers(void)
+{
+ backend_read_statsfile();
+
+ return &globalStats.buffers;
+}
+
/* ----------
* pgstat_fetch_stat_dbentry() -
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 9fb888f2ca..08e3f8f167 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -50,7 +50,7 @@ int pgstat_track_activity_query_size = 1024;
PgBackendStatus *MyBEEntry = NULL;
-static PgBackendStatus *BackendStatusArray = NULL;
+PgBackendStatus *BackendStatusArray = NULL;
static char *BackendAppnameBuffer = NULL;
static char *BackendClientHostnameBuffer = NULL;
static char *BackendActivityBuffer = NULL;
@@ -236,6 +236,23 @@ CreateSharedBackendStatus(void)
#endif
}
+const char *
+GetIOPathDesc(IOPath io_path)
+{
+ switch (io_path)
+ {
+ case IOPATH_DIRECT:
+ return "direct";
+ case IOPATH_LOCAL:
+ return "local";
+ case IOPATH_SHARED:
+ return "shared";
+ case IOPATH_STRATEGY:
+ return "strategy";
+ }
+ return "unknown IO path";
+}
+
/*
* Initialize pgstats backend activity state, and set up our on-proc-exit
* hook. Called from InitPostgres and AuxiliaryProcessMain. For auxiliary
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index f529c1561a..c16dc3ba9a 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1796,6 +1796,156 @@ pg_stat_get_buf_alloc(PG_FUNCTION_ARGS)
PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->buf_alloc);
}
+/*
+* When adding a new column to the pg_stat_buffers view, add a new enum
+* value here above BUFFERS_NUM_COLUMNS.
+*/
+enum
+{
+ BUFFERS_COLUMN_BACKEND_TYPE,
+ BUFFERS_COLUMN_IO_PATH,
+ BUFFERS_COLUMN_ALLOCS,
+ BUFFERS_COLUMN_EXTENDS,
+ BUFFERS_COLUMN_FSYNCS,
+ BUFFERS_COLUMN_WRITES,
+ BUFFERS_COLUMN_RESET_TIME,
+ BUFFERS_NUM_COLUMNS,
+};
+
+/*
+ * Helper function to get the correct row in the pg_stat_buffers view.
+ */
+static inline Datum *
+get_pg_stat_buffers_row(Datum all_values[BACKEND_NUM_TYPES][IOPATH_NUM_TYPES][BUFFERS_NUM_COLUMNS],
+ BackendType backend_type, IOPath io_path)
+{
+ return all_values[backend_type_get_idx(backend_type)][io_path];
+}
+
+Datum
+pg_stat_get_buffers(PG_FUNCTION_ARGS)
+{
+ PgStat_BackendIOPathOps *backend_io_path_ops;
+ PgBackendStatus *beentry;
+ Datum reset_time;
+
+ ReturnSetInfo *rsinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+
+ Datum all_values[BACKEND_NUM_TYPES][IOPATH_NUM_TYPES][BUFFERS_NUM_COLUMNS];
+ bool all_nulls[BACKEND_NUM_TYPES][IOPATH_NUM_TYPES][BUFFERS_NUM_COLUMNS];
+
+ rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+
+ /* check to see if caller supports us returning a tuplestore */
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("materialize mode required, but it is not allowed in this context")));
+
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+ tupstore = tuplestore_begin_heap((bool) (rsinfo->allowedModes & SFRM_Materialize_Random),
+ false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+ MemoryContextSwitchTo(oldcontext);
+
+ memset(all_values, 0, sizeof(all_values));
+ memset(all_nulls, 0, sizeof(all_nulls));
+
+ /* Loop through all live backends and count their IO Ops for each IO Path */
+ beentry = BackendStatusArray;
+
+ for (int i = 0; i < MaxBackends + NUM_AUXPROCTYPES; i++, beentry++)
+ {
+ IOOpCounters *io_ops;
+
+ /*
+ * Don't count dead backends. They will be added below. There are no
+ * rows in the view for BackendType B_INVALID, so skip those as well.
+ */
+ if (beentry->st_procpid == 0 || beentry->st_backendType == B_INVALID)
+ continue;
+
+ io_ops = beentry->io_path_stats;
+
+ for (int i = 0; i < IOPATH_NUM_TYPES; i++)
+ {
+ Datum *row = get_pg_stat_buffers_row(all_values, beentry->st_backendType, i);
+
+ /*
+ * BUFFERS_COLUMN_RESET_TIME, BUFFERS_COLUMN_BACKEND_TYPE, and
+ * BUFFERS_COLUMN_IO_PATH will all be set when looping through
+ * exited backends array
+ */
+ row[BUFFERS_COLUMN_ALLOCS] += pg_atomic_read_u64(&io_ops->allocs);
+ row[BUFFERS_COLUMN_EXTENDS] += pg_atomic_read_u64(&io_ops->extends);
+ row[BUFFERS_COLUMN_FSYNCS] += pg_atomic_read_u64(&io_ops->fsyncs);
+ row[BUFFERS_COLUMN_WRITES] += pg_atomic_read_u64(&io_ops->writes);
+ io_ops++;
+ }
+ }
+
+ /* Add stats from all exited backends */
+ backend_io_path_ops = pgstat_fetch_exited_backend_buffers();
+
+ reset_time = TimestampTzGetDatum(backend_io_path_ops->stat_reset_timestamp);
+
+ for (int i = 0; i < BACKEND_NUM_TYPES; i++)
+ {
+ BackendType backend_type = idx_get_backend_type(i);
+
+ PgStatIOOpCounters *io_ops =
+ backend_io_path_ops->ops[i].io_path_ops;
+ PgStatIOOpCounters *resets =
+ backend_io_path_ops->resets[i].io_path_ops;
+
+ Datum backend_type_desc =
+ CStringGetTextDatum(GetBackendTypeDesc(backend_type));
+
+ for (int j = 0; j < IOPATH_NUM_TYPES; j++)
+ {
+ Datum *row = get_pg_stat_buffers_row(all_values, backend_type, j);
+
+ row[BUFFERS_COLUMN_BACKEND_TYPE] = backend_type_desc;
+ row[BUFFERS_COLUMN_IO_PATH] = CStringGetTextDatum(GetIOPathDesc(j));
+ row[BUFFERS_COLUMN_RESET_TIME] = reset_time;
+ row[BUFFERS_COLUMN_ALLOCS] += io_ops->allocs - resets->allocs;
+ row[BUFFERS_COLUMN_EXTENDS] += io_ops->extends - resets->extends;
+ row[BUFFERS_COLUMN_FSYNCS] += io_ops->fsyncs - resets->fsyncs;
+ row[BUFFERS_COLUMN_WRITES] += io_ops->writes - resets->writes;
+ io_ops++;
+ resets++;
+ }
+ }
+
+ for (int i = 0; i < BACKEND_NUM_TYPES; i++)
+ {
+ for (int j = 0; j < IOPATH_NUM_TYPES; j++)
+ {
+ Datum *values = all_values[i][j];
+ bool *nulls = all_nulls[i][j];
+
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+ }
+
+ return (Datum) 0;
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4d992dc224..24d1fc29a2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5636,6 +5636,15 @@
proname => 'pg_stat_get_buf_alloc', provolatile => 's', proparallel => 'r',
prorettype => 'int8', proargtypes => '', prosrc => 'pg_stat_get_buf_alloc' },
+{ oid => '8459', descr => 'statistics: counts of all IO operations done through all IO paths by each type of backend.',
+ proname => 'pg_stat_get_buffers', provolatile => 's', proisstrict => 'f',
+ prorows => '52', proretset => 't',
+ proparallel => 'r', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,text,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{o,o,o,o,o,o,o}',
+ proargnames => '{backend_type,io_path,alloc,extend,fsync,write,stats_reset}',
+ prosrc => 'pg_stat_get_buffers' },
+
{ oid => '1136', descr => 'statistics: information about WAL activity',
proname => 'pg_stat_get_wal', proisstrict => 'f', provolatile => 's',
proparallel => 'r', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 4505825c87..d0700e6efe 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1296,6 +1296,7 @@ extern void pgstat_sum_io_path_ops(PgStatIOOpCounters *dest, IOOpCounters *src);
* generate the pgstat* views.
* ----------
*/
+extern PgStat_BackendIOPathOps *pgstat_fetch_exited_backend_buffers(void);
extern PgStat_StatDBEntry *pgstat_fetch_stat_dbentry(Oid dbid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 92f00e1fce..ecd0668161 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -316,6 +316,7 @@ extern PGDLLIMPORT int pgstat_track_activity_query_size;
* ----------
*/
extern PGDLLIMPORT PgBackendStatus *MyBEEntry;
+extern PGDLLIMPORT PgBackendStatus *BackendStatusArray;
/* ----------
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index b58b062b10..5869ce442f 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1828,6 +1828,14 @@ pg_stat_bgwriter| SELECT pg_stat_get_bgwriter_timed_checkpoints() AS checkpoints
pg_stat_get_buf_fsync_backend() AS buffers_backend_fsync,
pg_stat_get_buf_alloc() AS buffers_alloc,
pg_stat_get_bgwriter_stat_reset_time() AS stats_reset;
+pg_stat_buffers| SELECT b.backend_type,
+ b.io_path,
+ b.alloc,
+ b.extend,
+ b.fsync,
+ b.write,
+ b.stats_reset
+ FROM pg_stat_get_buffers() b(backend_type, io_path, alloc, extend, fsync, write, stats_reset);
pg_stat_database| SELECT d.oid AS datid,
d.datname,
CASE
--
2.30.2
From 85234ca76f31103fe7a2106a6b99a7da2d5a991e Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplage...@gmail.com>
Date: Mon, 3 Jan 2022 18:26:45 -0500
Subject: [PATCH v20 5/8] Add "buffers" to pgstat_reset_shared_counters
Backends count IO operations for various IO paths in their
PgBackendStatus. Upon exit, they send these counts to the stats
collector.
Prior to this commit, the IO operations stats from exited backends
persisted by the stats collector would have been been reset when
pgstat_reset_shared_counters() was invoked with target "bgwriter".
However the IO operations stats in each live backend's PgBackendStatus
would remain the same. Thus the totals calculated from both live and
exited backends would be incorrect after a reset.
Backends' PgBackendStatuses cannot be written to by another backend;
therefore, in order to calculate correct totals after a reset has
occurred, the backend sending the reset message to the stats collector
now reads the IO operation stats totals from live backends and sends
them to the stats collector to be persisted in an array of "resets"
which can be used to calculate the correct totals after a reset.
Because the IO operations statistics are broader in scope than those in
pg_stat_bgwriter, rename the reset target to "buffers". The "buffers"
target will reset all IO operations statistics and all statistics for
the pg_stat_bgwriter view maintained by the stats collector.
Author: Melanie Plageman <melanieplage...@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/20200124195226.lth52iydq2n2uilq%40alap3.anarazel.de
---
doc/src/sgml/monitoring.sgml | 2 +-
src/backend/postmaster/pgstat.c | 87 ++++++++++++++++++---
src/backend/utils/activity/backend_status.c | 27 +++++++
src/include/pgstat.h | 29 +++++--
src/include/utils/backend_status.h | 2 +
5 files changed, 129 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 62f2a3332b..c94eb57a59 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5209,7 +5209,7 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
</para>
<para>
Resets some cluster-wide statistics counters to zero, depending on the
- argument. The argument can be <literal>bgwriter</literal> to reset
+ argument. The argument can be <literal>buffers</literal> to reset
all the counters shown in
the <structname>pg_stat_bgwriter</structname>
view, <literal>archiver</literal> to reset all the counters shown in
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 11db91f62b..71db0b7b14 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -1509,6 +1509,36 @@ pgstat_reset_counters(void)
pgstat_send(&msg, sizeof(msg));
}
+/*
+ * Helper function to collect and send live backends' current IO operations
+ * stats counters when a stats reset is initiated so that they may be deducted
+ * from future totals.
+ */
+static void
+pgstat_send_buffers_reset(PgStat_MsgResetsharedcounter *msg)
+{
+ PgStatIOPathOps ops[BACKEND_NUM_TYPES];
+
+ memset(ops, 0, sizeof(ops));
+ pgstat_report_live_backend_io_path_ops(ops);
+
+ /*
+ * Iterate through the array of all IOOps for all IOPaths for each
+ * BackendType.
+ *
+ * An individual message is sent for each backend type because sending all
+ * IO operations in one message would exceed the PGSTAT_MAX_MSG_SIZE of
+ * 1000.
+ */
+ for (int i = 0; i < BACKEND_NUM_TYPES; i++)
+ {
+ msg->m_backend_resets.backend_type = idx_get_backend_type(i);
+ memcpy(&msg->m_backend_resets.iop, &ops[i],
+ sizeof(msg->m_backend_resets.iop));
+ pgstat_send(msg, sizeof(PgStat_MsgResetsharedcounter));
+ }
+}
+
/* ----------
* pgstat_reset_shared_counters() -
*
@@ -1526,19 +1556,25 @@ pgstat_reset_shared_counters(const char *target)
if (pgStatSock == PGINVALID_SOCKET)
return;
+ pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
+ if (strcmp(target, "buffers") == 0)
+ {
+ msg.m_resettarget = RESET_BUFFERS;
+ pgstat_send_buffers_reset(&msg);
+ return;
+ }
+
if (strcmp(target, "archiver") == 0)
msg.m_resettarget = RESET_ARCHIVER;
- else if (strcmp(target, "bgwriter") == 0)
- msg.m_resettarget = RESET_BGWRITER;
else if (strcmp(target, "wal") == 0)
msg.m_resettarget = RESET_WAL;
else
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unrecognized reset target: \"%s\"", target),
- errhint("Target must be \"archiver\", \"bgwriter\", or \"wal\".")));
+ errhint(
+ "Target must be \"archiver\", \"buffers\", or \"wal\".")));
- pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_RESETSHAREDCOUNTER);
pgstat_send(&msg, sizeof(msg));
}
@@ -4425,6 +4461,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
*/
ts = GetCurrentTimestamp();
globalStats.bgwriter.stat_reset_timestamp = ts;
+ globalStats.buffers.stat_reset_timestamp = ts;
archiverStats.stat_reset_timestamp = ts;
walStats.stat_reset_timestamp = ts;
@@ -5588,18 +5625,46 @@ pgstat_recv_resetcounter(PgStat_MsgResetcounter *msg, int len)
static void
pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
{
- if (msg->m_resettarget == RESET_BGWRITER)
- {
- /* Reset the global, bgwriter and checkpointer statistics for the cluster. */
- memset(&globalStats, 0, sizeof(globalStats));
- globalStats.bgwriter.stat_reset_timestamp = GetCurrentTimestamp();
- }
- else if (msg->m_resettarget == RESET_ARCHIVER)
+ if (msg->m_resettarget == RESET_ARCHIVER)
{
/* Reset the archiver statistics for the cluster. */
memset(&archiverStats, 0, sizeof(archiverStats));
archiverStats.stat_reset_timestamp = GetCurrentTimestamp();
}
+ else if (msg->m_resettarget == RESET_BUFFERS)
+ {
+ /*
+ * Reset global stats for bgwriter, buffers, and checkpointer.
+ *
+ * Because the stats collector cannot write to live backends'
+ * PgBackendStatuses, it maintains an array of "resets". The reset
+ * message contains the current values of these counters for live
+ * backends. The stats collector saves these in its "resets" array,
+ * then zeroes out the exited backends' saved IO operations counters.
+ * This is required to calculate an accurate total for each IO
+ * operations counter post reset.
+ */
+ BackendType backend_type = msg->m_backend_resets.backend_type;
+
+ /*
+ * We reset each member individually (as opposed to resetting the
+ * entire globalStats struct) because we need to preserve the resets
+ * array (globalStats.buffers.resets).
+ *
+ * Though globalStats.buffers.ops, globalStats.bgwriter, and
+ * globalStats.checkpointer only need to be reset once, doing so for
+ * every message is less brittle and the extra cost is irrelevant given
+ * how often stats are reset.
+ */
+ memset(&globalStats.bgwriter, 0, sizeof(globalStats.bgwriter));
+ memset(&globalStats.checkpointer, 0, sizeof(globalStats.checkpointer));
+ memset(&globalStats.buffers.ops, 0, sizeof(globalStats.buffers.ops));
+ globalStats.bgwriter.stat_reset_timestamp = GetCurrentTimestamp();
+ globalStats.buffers.stat_reset_timestamp = GetCurrentTimestamp();
+ memcpy(&globalStats.buffers.resets[backend_type_get_idx(backend_type)],
+ &msg->m_backend_resets.iop.io_path_ops,
+ sizeof(msg->m_backend_resets.iop.io_path_ops));
+ }
else if (msg->m_resettarget == RESET_WAL)
{
/* Reset the WAL statistics for the cluster. */
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 44c9a6e1a6..9fb888f2ca 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -631,6 +631,33 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+/*
+ * Iterate through BackendStatusArray and capture live backends' stats on IOOps
+ * for all IOPaths, adding them to that backend type's member of the
+ * backend_io_path_ops structure.
+ */
+void
+pgstat_report_live_backend_io_path_ops(PgStatIOPathOps *backend_io_path_ops)
+{
+ PgBackendStatus *beentry = BackendStatusArray;
+
+ /*
+ * Loop through live backends and capture reset values
+ */
+ for (int i = 0; i < MaxBackends + NUM_AUXPROCTYPES; i++, beentry++)
+ {
+ int idx;
+
+ /* Don't count dead backends or those with type B_INVALID. */
+ if (beentry->st_procpid == 0 || beentry->st_backendType == B_INVALID)
+ continue;
+
+ idx = backend_type_get_idx(beentry->st_backendType);
+ pgstat_sum_io_path_ops(backend_io_path_ops[idx].io_path_ops,
+ (IOOpCounters *) beentry->io_path_stats);
+ }
+}
+
/* --------
* pgstat_report_query_id() -
*
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 23496afffc..4505825c87 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -141,7 +141,7 @@ typedef struct PgStat_TableCounts
typedef enum PgStat_Shared_Reset_Target
{
RESET_ARCHIVER,
- RESET_BGWRITER,
+ RESET_BUFFERS,
RESET_WAL
} PgStat_Shared_Reset_Target;
@@ -357,7 +357,8 @@ typedef struct PgStatIOPathOps
/*
* Sent by a backend to the stats collector to report all IOOps for all IOPaths
- * for a given type of a backend. This will happen when the backend exits.
+ * for a given type of a backend. This will happen when the backend exits or
+ * when stats are reset.
*/
typedef struct PgStat_MsgIOPathOps
{
@@ -375,9 +376,12 @@ typedef struct PgStat_MsgIOPathOps
*/
typedef struct PgStat_BackendIOPathOps
{
+ TimestampTz stat_reset_timestamp;
PgStatIOPathOps ops[BACKEND_NUM_TYPES];
+ PgStatIOPathOps resets[BACKEND_NUM_TYPES];
} PgStat_BackendIOPathOps;
+
/* ----------
* PgStat_MsgResetcounter Sent by the backend to tell the collector
* to reset counters
@@ -389,15 +393,28 @@ typedef struct PgStat_MsgResetcounter
Oid m_databaseid;
} PgStat_MsgResetcounter;
-/* ----------
- * PgStat_MsgResetsharedcounter Sent by the backend to tell the collector
- * to reset a shared counter
- * ----------
+/*
+ * Sent by the backend to tell the collector to reset a shared counter.
+ *
+ * In addition to the message header and reset target, the message also
+ * contains an array with all of the IO operations for all IO paths done by a
+ * particular backend type.
+ *
+ * This is needed because the IO operation stats for live backends cannot be
+ * safely modified by other processes. Therefore, to correctly calculate the
+ * total IO operations for a particular backend type after a reset, the balance
+ * of IO operations for live backends at the time of prior resets must be
+ * subtracted from the total IO operations.
+ *
+ * To satisfy this requirement, the process initiating the reset will read the
+ * IO operations counters from live backends and send them to the stats
+ * collector which maintains an array of reset values.
*/
typedef struct PgStat_MsgResetsharedcounter
{
PgStat_MsgHdr m_hdr;
PgStat_Shared_Reset_Target m_resettarget;
+ PgStat_MsgIOPathOps m_backend_resets;
} PgStat_MsgResetsharedcounter;
/* ----------
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index ae4597c5fe..92f00e1fce 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -375,6 +375,7 @@ extern void pgstat_bestart(void);
extern void pgstat_clear_backend_activity_snapshot(void);
/* Activity reporting functions */
+typedef struct PgStatIOPathOps PgStatIOPathOps;
static inline void
pgstat_inc_ioop(IOOp io_op, IOPath io_path)
@@ -402,6 +403,7 @@ pgstat_inc_ioop(IOOp io_op, IOPath io_path)
}
}
extern void pgstat_report_activity(BackendState state, const char *cmd_str);
+extern void pgstat_report_live_backend_io_path_ops(PgStatIOPathOps *backend_io_path_ops);
extern void pgstat_report_query_id(uint64 query_id, bool force);
extern void pgstat_report_tempfile(size_t filesize);
extern void pgstat_report_appname(const char *appname);
--
2.30.2
From 7c367848f63d099f8f95ecdab516e911b4e69b3a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplage...@gmail.com>
Date: Wed, 24 Nov 2021 12:21:08 -0500
Subject: [PATCH v20 8/8] small comment correction
Naming callers in function comment is brittle and unnecessary.
---
src/backend/utils/activity/backend_status.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 08e3f8f167..d508b819b1 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -254,11 +254,11 @@ GetIOPathDesc(IOPath io_path)
}
/*
- * Initialize pgstats backend activity state, and set up our on-proc-exit
- * hook. Called from InitPostgres and AuxiliaryProcessMain. For auxiliary
- * process, MyBackendId is invalid. Otherwise, MyBackendId must be set, but we
- * must not have started any transaction yet (since the exit hook must run
- * after the last transaction exit).
+ * Initialize pgstats backend activity state, and set up our on-proc-exit hook.
+ *
+ * For auxiliary process, MyBackendId is invalid. Otherwise, MyBackendId must
+ * be set, but we must not have started any transaction yet (since the exit
+ * hook must run after the last transaction exit).
*
* NOTE: MyDatabaseId isn't set yet; so the shutdown hook has to be careful.
*/
@@ -296,7 +296,6 @@ pgstat_beinit(void)
* pgstat_bestart() -
*
* Initialize this backend's entry in the PgBackendStatus array.
- * Called from InitPostgres.
*
* Apart from auxiliary processes, MyBackendId, MyDatabaseId,
* session userid, and application_name must be set for a
--
2.30.2
From 51c5118ae8b189204c7f594c57d816a428f70da3 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplage...@gmail.com>
Date: Wed, 24 Nov 2021 11:16:55 -0500
Subject: [PATCH v20 4/8] Send IO operations to stats collector
On exit, backends send the IO operations they have done on all IO Paths
to the stats collector. The stats collector adds these counts to its
existing counts stored in a global data structure it maintains and
persists.
PgStatIOOpCounters contains the same information as backend_status.h's
IOOpCounters, however IOOpCounters' members must be atomics and the
stats collector has no such requirement.
Suggested by Andres Freund
Author: Melanie Plageman <melanieplage...@gmail.com>
Reviewed-by: Justin Pryzby <pry...@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/flat/20200124195226.lth52iydq2n2uilq%40alap3.anarazel.de
---
src/backend/postmaster/pgstat.c | 100 ++++++++++++++++++++++++++++-
src/include/miscadmin.h | 2 +
src/include/pgstat.h | 56 ++++++++++++++++
src/include/utils/backend_status.h | 37 +++++++++++
4 files changed, 194 insertions(+), 1 deletion(-)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 7264d2c727..11db91f62b 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -126,7 +126,7 @@ char *pgstat_stat_filename = NULL;
char *pgstat_stat_tmpname = NULL;
/*
- * BgWriter and WAL global statistics counters.
+ * BgWriter, Checkpointer, and WAL global statistics counters.
* Stored directly in a stats message structure so they can be sent
* without needing to copy things around. We assume these init to zeroes.
*/
@@ -369,6 +369,7 @@ static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
static void pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len);
static void pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len);
static void pgstat_recv_checkpointer(PgStat_MsgCheckpointer *msg, int len);
+static void pgstat_recv_io_path_ops(PgStat_MsgIOPathOps *msg, int len);
static void pgstat_recv_wal(PgStat_MsgWal *msg, int len);
static void pgstat_recv_slru(PgStat_MsgSLRU *msg, int len);
static void pgstat_recv_funcstat(PgStat_MsgFuncstat *msg, int len);
@@ -3152,6 +3153,14 @@ pgstat_shutdown_hook(int code, Datum arg)
{
Assert(!pgstat_is_shutdown);
+ /*
+ * Only need to send stats on IOOps for IOPaths when a process exits. Users
+ * requiring IOOps for both live and exited backends can read from live
+ * backends' PgBackendStatuses and sum this with totals from exited
+ * backends persisted by the stats collector.
+ */
+ pgstat_send_buffers();
+
/*
* If we got as far as discovering our own database ID, we can report what
* we did to the collector. Otherwise, we'd be sending an invalid
@@ -3301,6 +3310,46 @@ pgstat_send_bgwriter(void)
MemSet(&PendingBgWriterStats, 0, sizeof(PendingBgWriterStats));
}
+/*
+ * Before exiting, a backend sends its IO operations statistics to the
+ * collector so that they may be persisted.
+ */
+void
+pgstat_send_buffers(void)
+{
+ PgStatIOOpCounters *io_path_ops;
+ PgStat_MsgIOPathOps msg;
+
+ PgBackendStatus *beentry = MyBEEntry;
+ PgStat_Counter sum = 0;
+
+ if (!beentry || beentry->st_backendType == B_INVALID)
+ return;
+
+ memset(&msg, 0, sizeof(msg));
+ msg.backend_type = beentry->st_backendType;
+
+ io_path_ops = msg.iop.io_path_ops;
+ pgstat_sum_io_path_ops(io_path_ops, (IOOpCounters *)
+ &beentry->io_path_stats);
+
+ /* If no IO was done, don't bother sending anything to the stats collector. */
+ for (int i = 0; i < IOPATH_NUM_TYPES; i++)
+ {
+ sum += io_path_ops[i].allocs;
+ sum += io_path_ops[i].extends;
+ sum += io_path_ops[i].fsyncs;
+ sum += io_path_ops[i].writes;
+ }
+
+ if (sum == 0)
+ return;
+
+ pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_IO_PATH_OPS);
+ pgstat_send(&msg, sizeof(msg));
+}
+
+
/* ----------
* pgstat_send_checkpointer() -
*
@@ -3483,6 +3532,29 @@ pgstat_send_subscription_purge(PgStat_MsgSubscriptionPurge *msg)
pgstat_send(msg, len);
}
+/*
+ * Helper function to sum all IO operations stats for all IOPaths (e.g. shared,
+ * local) from live backends with those in the equivalent stats structure for
+ * exited backends.
+ * Note that this adds and doesn't set, so the destination stats structure
+ * should be zeroed out by the caller initially.
+ * This would commonly be used to transfer all IOOp stats for all IOPaths for a
+ * particular backend type to the pgstats structure.
+ */
+void
+pgstat_sum_io_path_ops(PgStatIOOpCounters *dest, IOOpCounters *src)
+{
+ for (int i = 0; i < IOPATH_NUM_TYPES; i++)
+ {
+ dest->allocs += pg_atomic_read_u64(&src->allocs);
+ dest->extends += pg_atomic_read_u64(&src->extends);
+ dest->fsyncs += pg_atomic_read_u64(&src->fsyncs);
+ dest->writes += pg_atomic_read_u64(&src->writes);
+ dest++;
+ src++;
+ }
+}
+
/* ----------
* PgstatCollectorMain() -
*
@@ -3692,6 +3764,10 @@ PgstatCollectorMain(int argc, char *argv[])
pgstat_recv_checkpointer(&msg.msg_checkpointer, len);
break;
+ case PGSTAT_MTYPE_IO_PATH_OPS:
+ pgstat_recv_io_path_ops(&msg.msg_io_path_ops, len);
+ break;
+
case PGSTAT_MTYPE_WAL:
pgstat_recv_wal(&msg.msg_wal, len);
break;
@@ -5813,6 +5889,28 @@ pgstat_recv_checkpointer(PgStat_MsgCheckpointer *msg, int len)
globalStats.checkpointer.buf_fsync_backend += msg->m_buf_fsync_backend;
}
+static void
+pgstat_recv_io_path_ops(PgStat_MsgIOPathOps *msg, int len)
+{
+ PgStatIOOpCounters *src_io_path_ops;
+ PgStatIOOpCounters *dest_io_path_ops;
+
+ src_io_path_ops = msg->iop.io_path_ops;
+ dest_io_path_ops =
+ globalStats.buffers.ops[backend_type_get_idx(msg->backend_type)].io_path_ops;
+
+ for (int i = 0; i < IOPATH_NUM_TYPES; i++)
+ {
+ PgStatIOOpCounters *src = &src_io_path_ops[i];
+ PgStatIOOpCounters *dest = &dest_io_path_ops[i];
+
+ dest->allocs += src->allocs;
+ dest->extends += src->extends;
+ dest->fsyncs += src->fsyncs;
+ dest->writes += src->writes;
+ }
+}
+
/* ----------
* pgstat_recv_wal() -
*
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1d688f9e51..85fe522780 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -339,6 +339,8 @@ typedef enum BackendType
B_WAL_WRITER,
} BackendType;
+#define BACKEND_NUM_TYPES B_WAL_WRITER
+
extern BackendType MyBackendType;
extern const char *GetBackendTypeDesc(BackendType backendType);
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 5b51b58e5a..23496afffc 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -73,6 +73,7 @@ typedef enum StatMsgType
PGSTAT_MTYPE_ARCHIVER,
PGSTAT_MTYPE_BGWRITER,
PGSTAT_MTYPE_CHECKPOINTER,
+ PGSTAT_MTYPE_IO_PATH_OPS,
PGSTAT_MTYPE_WAL,
PGSTAT_MTYPE_SLRU,
PGSTAT_MTYPE_FUNCSTAT,
@@ -335,6 +336,48 @@ typedef struct PgStat_MsgDropdb
} PgStat_MsgDropdb;
+/*
+ * Structure for counting all types of IOOps in the stats collector
+ */
+typedef struct PgStatIOOpCounters
+{
+ PgStat_Counter allocs;
+ PgStat_Counter extends;
+ PgStat_Counter fsyncs;
+ PgStat_Counter writes;
+} PgStatIOOpCounters;
+
+/*
+ * Structure for counting all IOOps on all types of IOPaths.
+ */
+typedef struct PgStatIOPathOps
+{
+ PgStatIOOpCounters io_path_ops[IOPATH_NUM_TYPES];
+} PgStatIOPathOps;
+
+/*
+ * Sent by a backend to the stats collector to report all IOOps for all IOPaths
+ * for a given type of a backend. This will happen when the backend exits.
+ */
+typedef struct PgStat_MsgIOPathOps
+{
+ PgStat_MsgHdr m_hdr;
+
+ BackendType backend_type;
+ PgStatIOPathOps iop;
+} PgStat_MsgIOPathOps;
+
+/*
+ * Structure used by stats collector to keep track of all types of exited
+ * backends' IOOps for all IOPaths as well as all stats from live backends at
+ * the time of stats reset. resets is populated using a reset message sent to
+ * the stats collector.
+ */
+typedef struct PgStat_BackendIOPathOps
+{
+ PgStatIOPathOps ops[BACKEND_NUM_TYPES];
+} PgStat_BackendIOPathOps;
+
/* ----------
* PgStat_MsgResetcounter Sent by the backend to tell the collector
* to reset counters
@@ -756,6 +799,7 @@ typedef union PgStat_Msg
PgStat_MsgArchiver msg_archiver;
PgStat_MsgBgWriter msg_bgwriter;
PgStat_MsgCheckpointer msg_checkpointer;
+ PgStat_MsgIOPathOps msg_io_path_ops;
PgStat_MsgWal msg_wal;
PgStat_MsgSLRU msg_slru;
PgStat_MsgFuncstat msg_funcstat;
@@ -939,6 +983,7 @@ typedef struct PgStat_GlobalStats
PgStat_CheckpointerStats checkpointer;
PgStat_BgWriterStats bgwriter;
+ PgStat_BackendIOPathOps buffers;
} PgStat_GlobalStats;
/*
@@ -1215,8 +1260,19 @@ extern void pgstat_twophase_postabort(TransactionId xid, uint16 info,
extern void pgstat_send_archiver(const char *xlog, bool failed);
extern void pgstat_send_bgwriter(void);
+/*
+ * While some processes send some types of statistics to the collector at
+ * regular intervals (e.g. CheckpointerMain() calling
+ * pgstat_send_checkpointer()), IO operations stats are only sent by
+ * pgstat_send_buffers() when a process exits (in pgstat_shutdown_hook()). IO
+ * operations stats from live backends can be read from their PgBackendStatuses
+ * and, if desired, summed with totals from exited backends persisted by the
+ * stats collector.
+ */
+extern void pgstat_send_buffers(void);
extern void pgstat_send_checkpointer(void);
extern void pgstat_send_wal(bool force);
+extern void pgstat_sum_io_path_ops(PgStatIOOpCounters *dest, IOOpCounters *src);
/* ----------
* Support functions for the SQL-callable functions to
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 56a0f25296..ae4597c5fe 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -331,6 +331,43 @@ extern void CreateSharedBackendStatus(void);
* ----------
*/
+/* Utility functions */
+
+/*
+ * When maintaining an array of information about all valid BackendTypes, in
+ * order to avoid wasting the 0th spot, use this helper to convert a valid
+ * BackendType to a valid location in the array (given that no spot is
+ * maintained for B_INVALID BackendType).
+ */
+static inline int backend_type_get_idx(BackendType backend_type)
+{
+ /*
+ * backend_type must be one of the valid backend types. If caller is
+ * maintaining backend information in an array that includes B_INVALID,
+ * this function is unnecessary.
+ */
+ Assert(backend_type > B_INVALID && backend_type <= BACKEND_NUM_TYPES);
+ return backend_type - 1;
+}
+
+/*
+ * When using a value from an array of information about all valid
+ * BackendTypes, add 1 to the index before using it as a BackendType to adjust
+ * for not maintaining a spot for B_INVALID BackendType.
+ */
+static inline BackendType idx_get_backend_type(int idx)
+{
+ int backend_type = idx + 1;
+ /*
+ * If the array includes a spot for B_INVALID BackendType this function is
+ * not required.
+ */
+ Assert(backend_type > B_INVALID && backend_type <= BACKEND_NUM_TYPES);
+ return backend_type;
+}
+
+extern const char *GetIOPathDesc(IOPath io_path);
+
/* Initialization functions */
extern void pgstat_beinit(void);
extern void pgstat_bestart(void);
--
2.30.2
From 147d4f64a8488ac03b98b4ae7a8a4ded3072b2ed Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplage...@gmail.com>
Date: Mon, 3 Jan 2022 15:01:10 -0500
Subject: [PATCH v20 7/8] Remove superfluous bgwriter stats code
After adding io_path_stats to PgBackendStatus, all backends keep track
of all IO done on all types of IO paths. When backends exit, they send
their IO operations stats to the stats collector to be persisted.
These statistics are available in the pg_stat_buffers view, making the
buffers_checkpoint, buffers_clean, buffers_backend,
buffers_backend_fsync, and buffers_alloc columns in the pg_stat_bgwriter
view redundant.
In order to maintain backward compatability, these columns in
pg_stat_bgwriter remain and are derived from the pg_stat_buffers view.
The structs used to track the statistics for these columns in the
pg_stat_bgwriter view and the functions querying them have been removed.
Additionally, since the "buffers" stats reset target resets both the IO
operations stats structs and the bgwriter stats structs, this member of
the bgwriter stats structs is no longer needed. Instead derive the
stats_reset column in the pg_stat_bgwriter view from pg_stat_buffers as
well.
---
src/backend/catalog/system_views.sql | 28 ++++++++++-----------
src/backend/postmaster/checkpointer.c | 29 ++-------------------
src/backend/postmaster/pgstat.c | 7 ------
src/backend/storage/buffer/bufmgr.c | 6 -----
src/backend/utils/adt/pgstatfuncs.c | 36 ---------------------------
src/include/catalog/pg_proc.dat | 26 -------------------
src/include/pgstat.h | 11 --------
src/test/regress/expected/rules.out | 24 +++++++++++++-----
8 files changed, 34 insertions(+), 133 deletions(-)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index e214d23056..b992d9300a 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1062,20 +1062,6 @@ CREATE VIEW pg_stat_archiver AS
s.stats_reset
FROM pg_stat_get_archiver() s;
-CREATE VIEW pg_stat_bgwriter AS
- SELECT
- pg_stat_get_bgwriter_timed_checkpoints() AS checkpoints_timed,
- pg_stat_get_bgwriter_requested_checkpoints() AS checkpoints_req,
- pg_stat_get_checkpoint_write_time() AS checkpoint_write_time,
- pg_stat_get_checkpoint_sync_time() AS checkpoint_sync_time,
- pg_stat_get_bgwriter_buf_written_checkpoints() AS buffers_checkpoint,
- pg_stat_get_bgwriter_buf_written_clean() AS buffers_clean,
- pg_stat_get_bgwriter_maxwritten_clean() AS maxwritten_clean,
- pg_stat_get_buf_written_backend() AS buffers_backend,
- pg_stat_get_buf_fsync_backend() AS buffers_backend_fsync,
- pg_stat_get_buf_alloc() AS buffers_alloc,
- pg_stat_get_bgwriter_stat_reset_time() AS stats_reset;
-
CREATE VIEW pg_stat_buffers AS
SELECT
b.backend_type,
@@ -1087,6 +1073,20 @@ SELECT
b.stats_reset
FROM pg_stat_get_buffers() b;
+CREATE VIEW pg_stat_bgwriter AS
+ SELECT
+ pg_stat_get_bgwriter_timed_checkpoints() AS checkpoints_timed,
+ pg_stat_get_bgwriter_requested_checkpoints() AS checkpoints_req,
+ pg_stat_get_checkpoint_write_time() AS checkpoint_write_time,
+ pg_stat_get_checkpoint_sync_time() AS checkpoint_sync_time,
+ (SELECT write FROM pg_stat_buffers WHERE backend_type = 'checkpointer' AND io_path = 'shared') AS buffers_checkpoint,
+ (SELECT write FROM pg_stat_buffers WHERE backend_type = 'background writer' AND io_path = 'shared') AS buffers_clean,
+ pg_stat_get_bgwriter_maxwritten_clean() AS maxwritten_clean,
+ (SELECT write FROM pg_stat_buffers WHERE backend_type = 'client backend' AND io_path = 'shared') AS buffers_backend,
+ (SELECT fsync FROM pg_stat_buffers WHERE backend_type = 'client backend' AND io_path = 'shared') AS buffers_backend_fsync,
+ (SELECT alloc FROM pg_stat_buffers WHERE backend_type = 'client backend' AND io_path = 'shared') AS buffers_alloc,
+ (SELECT stats_reset FROM pg_stat_buffers LIMIT 1) AS stats_reset;
+
CREATE VIEW pg_stat_wal AS
SELECT
w.wal_records,
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 8440b2b802..b9c3745474 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -90,17 +90,9 @@
* requesting backends since the last checkpoint start. The flags are
* chosen so that OR'ing is the correct way to combine multiple requests.
*
- * num_backend_writes is used to count the number of buffer writes performed
- * by user backend processes. This counter should be wide enough that it
- * can't overflow during a single processing cycle. num_backend_fsync
- * counts the subset of those writes that also had to do their own fsync,
- * because the checkpointer failed to absorb their request.
- *
* The requests array holds fsync requests sent by backends and not yet
* absorbed by the checkpointer.
*
- * Unlike the checkpoint fields, num_backend_writes, num_backend_fsync, and
- * the requests fields are protected by CheckpointerCommLock.
*----------
*/
typedef struct
@@ -124,9 +116,6 @@ typedef struct
ConditionVariable start_cv; /* signaled when ckpt_started advances */
ConditionVariable done_cv; /* signaled when ckpt_done advances */
- uint32 num_backend_writes; /* counts user backend buffer writes */
- uint32 num_backend_fsync; /* counts user backend fsync calls */
-
int num_requests; /* current # of requests */
int max_requests; /* allocated array size */
CheckpointerRequest requests[FLEXIBLE_ARRAY_MEMBER];
@@ -1081,10 +1070,6 @@ ForwardSyncRequest(const FileTag *ftag, SyncRequestType type)
LWLockAcquire(CheckpointerCommLock, LW_EXCLUSIVE);
- /* Count all backend writes regardless of if they fit in the queue */
- if (!AmBackgroundWriterProcess())
- CheckpointerShmem->num_backend_writes++;
-
/*
* If the checkpointer isn't running or the request queue is full, the
* backend will have to perform its own fsync request. But before forcing
@@ -1094,13 +1079,12 @@ ForwardSyncRequest(const FileTag *ftag, SyncRequestType type)
(CheckpointerShmem->num_requests >= CheckpointerShmem->max_requests &&
!CompactCheckpointerRequestQueue()))
{
+ LWLockRelease(CheckpointerCommLock);
+
/*
* Count the subset of writes where backends have to do their own
* fsync
*/
- if (!AmBackgroundWriterProcess())
- CheckpointerShmem->num_backend_fsync++;
- LWLockRelease(CheckpointerCommLock);
pgstat_inc_ioop(IOOP_FSYNC, IOPATH_SHARED);
return false;
}
@@ -1257,15 +1241,6 @@ AbsorbSyncRequests(void)
LWLockAcquire(CheckpointerCommLock, LW_EXCLUSIVE);
- /* Transfer stats counts into pending pgstats message */
- PendingCheckpointerStats.m_buf_written_backend
- += CheckpointerShmem->num_backend_writes;
- PendingCheckpointerStats.m_buf_fsync_backend
- += CheckpointerShmem->num_backend_fsync;
-
- CheckpointerShmem->num_backend_writes = 0;
- CheckpointerShmem->num_backend_fsync = 0;
-
/*
* We try to avoid holding the lock for a long time by copying the request
* array, and processing the requests after releasing the lock.
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index f4bff26630..0b52533140 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4473,7 +4473,6 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
* existing statsfile).
*/
ts = GetCurrentTimestamp();
- globalStats.bgwriter.stat_reset_timestamp = ts;
globalStats.buffers.stat_reset_timestamp = ts;
archiverStats.stat_reset_timestamp = ts;
walStats.stat_reset_timestamp = ts;
@@ -5672,7 +5671,6 @@ pgstat_recv_resetsharedcounter(PgStat_MsgResetsharedcounter *msg, int len)
memset(&globalStats.bgwriter, 0, sizeof(globalStats.bgwriter));
memset(&globalStats.checkpointer, 0, sizeof(globalStats.checkpointer));
memset(&globalStats.buffers.ops, 0, sizeof(globalStats.buffers.ops));
- globalStats.bgwriter.stat_reset_timestamp = GetCurrentTimestamp();
globalStats.buffers.stat_reset_timestamp = GetCurrentTimestamp();
memcpy(&globalStats.buffers.resets[backend_type_get_idx(backend_type)],
&msg->m_backend_resets.iop.io_path_ops,
@@ -5944,9 +5942,7 @@ pgstat_recv_archiver(PgStat_MsgArchiver *msg, int len)
static void
pgstat_recv_bgwriter(PgStat_MsgBgWriter *msg, int len)
{
- globalStats.bgwriter.buf_written_clean += msg->m_buf_written_clean;
globalStats.bgwriter.maxwritten_clean += msg->m_maxwritten_clean;
- globalStats.bgwriter.buf_alloc += msg->m_buf_alloc;
}
/* ----------
@@ -5962,9 +5958,6 @@ pgstat_recv_checkpointer(PgStat_MsgCheckpointer *msg, int len)
globalStats.checkpointer.requested_checkpoints += msg->m_requested_checkpoints;
globalStats.checkpointer.checkpoint_write_time += msg->m_checkpoint_write_time;
globalStats.checkpointer.checkpoint_sync_time += msg->m_checkpoint_sync_time;
- globalStats.checkpointer.buf_written_checkpoints += msg->m_buf_written_checkpoints;
- globalStats.checkpointer.buf_written_backend += msg->m_buf_written_backend;
- globalStats.checkpointer.buf_fsync_backend += msg->m_buf_fsync_backend;
}
static void
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 528995e7fb..6ec15ea00e 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -2165,7 +2165,6 @@ BufferSync(int flags)
if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
- PendingCheckpointerStats.m_buf_written_checkpoints++;
num_written++;
}
}
@@ -2274,9 +2273,6 @@ BgBufferSync(WritebackContext *wb_context)
*/
strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
- /* Report buffer alloc counts to pgstat */
- PendingBgWriterStats.m_buf_alloc += recent_alloc;
-
/*
* If we're not running the LRU scan, just stop after doing the stats
* stuff. We mark the saved state invalid so that we can recover sanely
@@ -2473,8 +2469,6 @@ BgBufferSync(WritebackContext *wb_context)
reusable_buffers++;
}
- PendingBgWriterStats.m_buf_written_clean += num_written;
-
#ifdef BGW_DEBUG
elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
recent_alloc, smoothed_alloc, strategy_delta, bufs_ahead,
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c16dc3ba9a..b25a688e17 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1738,18 +1738,6 @@ pg_stat_get_bgwriter_requested_checkpoints(PG_FUNCTION_ARGS)
PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->requested_checkpoints);
}
-Datum
-pg_stat_get_bgwriter_buf_written_checkpoints(PG_FUNCTION_ARGS)
-{
- PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_written_checkpoints);
-}
-
-Datum
-pg_stat_get_bgwriter_buf_written_clean(PG_FUNCTION_ARGS)
-{
- PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->buf_written_clean);
-}
-
Datum
pg_stat_get_bgwriter_maxwritten_clean(PG_FUNCTION_ARGS)
{
@@ -1772,30 +1760,6 @@ pg_stat_get_checkpoint_sync_time(PG_FUNCTION_ARGS)
pgstat_fetch_stat_checkpointer()->checkpoint_sync_time);
}
-Datum
-pg_stat_get_bgwriter_stat_reset_time(PG_FUNCTION_ARGS)
-{
- PG_RETURN_TIMESTAMPTZ(pgstat_fetch_stat_bgwriter()->stat_reset_timestamp);
-}
-
-Datum
-pg_stat_get_buf_written_backend(PG_FUNCTION_ARGS)
-{
- PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_written_backend);
-}
-
-Datum
-pg_stat_get_buf_fsync_backend(PG_FUNCTION_ARGS)
-{
- PG_RETURN_INT64(pgstat_fetch_stat_checkpointer()->buf_fsync_backend);
-}
-
-Datum
-pg_stat_get_buf_alloc(PG_FUNCTION_ARGS)
-{
- PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->buf_alloc);
-}
-
/*
* When adding a new column to the pg_stat_buffers view, add a new enum
* value here above BUFFERS_NUM_COLUMNS.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 24d1fc29a2..bbc613eddf 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5594,25 +5594,11 @@
proname => 'pg_stat_get_bgwriter_requested_checkpoints', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => '',
prosrc => 'pg_stat_get_bgwriter_requested_checkpoints' },
-{ oid => '2771',
- descr => 'statistics: number of buffers written by the bgwriter during checkpoints',
- proname => 'pg_stat_get_bgwriter_buf_written_checkpoints', provolatile => 's',
- proparallel => 'r', prorettype => 'int8', proargtypes => '',
- prosrc => 'pg_stat_get_bgwriter_buf_written_checkpoints' },
-{ oid => '2772',
- descr => 'statistics: number of buffers written by the bgwriter for cleaning dirty buffers',
- proname => 'pg_stat_get_bgwriter_buf_written_clean', provolatile => 's',
- proparallel => 'r', prorettype => 'int8', proargtypes => '',
- prosrc => 'pg_stat_get_bgwriter_buf_written_clean' },
{ oid => '2773',
descr => 'statistics: number of times the bgwriter stopped processing when it had written too many buffers while cleaning',
proname => 'pg_stat_get_bgwriter_maxwritten_clean', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => '',
prosrc => 'pg_stat_get_bgwriter_maxwritten_clean' },
-{ oid => '3075', descr => 'statistics: last reset for the bgwriter',
- proname => 'pg_stat_get_bgwriter_stat_reset_time', provolatile => 's',
- proparallel => 'r', prorettype => 'timestamptz', proargtypes => '',
- prosrc => 'pg_stat_get_bgwriter_stat_reset_time' },
{ oid => '3160',
descr => 'statistics: checkpoint time spent writing buffers to disk, in milliseconds',
proname => 'pg_stat_get_checkpoint_write_time', provolatile => 's',
@@ -5623,18 +5609,6 @@
proname => 'pg_stat_get_checkpoint_sync_time', provolatile => 's',
proparallel => 'r', prorettype => 'float8', proargtypes => '',
prosrc => 'pg_stat_get_checkpoint_sync_time' },
-{ oid => '2775', descr => 'statistics: number of buffers written by backends',
- proname => 'pg_stat_get_buf_written_backend', provolatile => 's',
- proparallel => 'r', prorettype => 'int8', proargtypes => '',
- prosrc => 'pg_stat_get_buf_written_backend' },
-{ oid => '3063',
- descr => 'statistics: number of backend buffer writes that did their own fsync',
- proname => 'pg_stat_get_buf_fsync_backend', provolatile => 's',
- proparallel => 'r', prorettype => 'int8', proargtypes => '',
- prosrc => 'pg_stat_get_buf_fsync_backend' },
-{ oid => '2859', descr => 'statistics: number of buffer allocations',
- proname => 'pg_stat_get_buf_alloc', provolatile => 's', proparallel => 'r',
- prorettype => 'int8', proargtypes => '', prosrc => 'pg_stat_get_buf_alloc' },
{ oid => '8459', descr => 'statistics: counts of all IO operations done through all IO paths by each type of backend.',
proname => 'pg_stat_get_buffers', provolatile => 's', proisstrict => 'f',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index d0700e6efe..b17edd8679 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -522,9 +522,7 @@ typedef struct PgStat_MsgBgWriter
{
PgStat_MsgHdr m_hdr;
- PgStat_Counter m_buf_written_clean;
PgStat_Counter m_maxwritten_clean;
- PgStat_Counter m_buf_alloc;
} PgStat_MsgBgWriter;
/* ----------
@@ -537,9 +535,6 @@ typedef struct PgStat_MsgCheckpointer
PgStat_Counter m_timed_checkpoints;
PgStat_Counter m_requested_checkpoints;
- PgStat_Counter m_buf_written_checkpoints;
- PgStat_Counter m_buf_written_backend;
- PgStat_Counter m_buf_fsync_backend;
PgStat_Counter m_checkpoint_write_time; /* times in milliseconds */
PgStat_Counter m_checkpoint_sync_time;
} PgStat_MsgCheckpointer;
@@ -970,10 +965,7 @@ typedef struct PgStat_ArchiverStats
*/
typedef struct PgStat_BgWriterStats
{
- PgStat_Counter buf_written_clean;
PgStat_Counter maxwritten_clean;
- PgStat_Counter buf_alloc;
- TimestampTz stat_reset_timestamp;
} PgStat_BgWriterStats;
/*
@@ -986,9 +978,6 @@ typedef struct PgStat_CheckpointerStats
PgStat_Counter requested_checkpoints;
PgStat_Counter checkpoint_write_time; /* times in milliseconds */
PgStat_Counter checkpoint_sync_time;
- PgStat_Counter buf_written_checkpoints;
- PgStat_Counter buf_written_backend;
- PgStat_Counter buf_fsync_backend;
} PgStat_CheckpointerStats;
/*
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 5869ce442f..9736c1fe37 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1821,13 +1821,25 @@ pg_stat_bgwriter| SELECT pg_stat_get_bgwriter_timed_checkpoints() AS checkpoints
pg_stat_get_bgwriter_requested_checkpoints() AS checkpoints_req,
pg_stat_get_checkpoint_write_time() AS checkpoint_write_time,
pg_stat_get_checkpoint_sync_time() AS checkpoint_sync_time,
- pg_stat_get_bgwriter_buf_written_checkpoints() AS buffers_checkpoint,
- pg_stat_get_bgwriter_buf_written_clean() AS buffers_clean,
+ ( SELECT pg_stat_buffers.write
+ FROM pg_stat_buffers
+ WHERE ((pg_stat_buffers.backend_type = 'checkpointer'::text) AND (pg_stat_buffers.io_path = 'shared'::text))) AS buffers_checkpoint,
+ ( SELECT pg_stat_buffers.write
+ FROM pg_stat_buffers
+ WHERE ((pg_stat_buffers.backend_type = 'background writer'::text) AND (pg_stat_buffers.io_path = 'shared'::text))) AS buffers_clean,
pg_stat_get_bgwriter_maxwritten_clean() AS maxwritten_clean,
- pg_stat_get_buf_written_backend() AS buffers_backend,
- pg_stat_get_buf_fsync_backend() AS buffers_backend_fsync,
- pg_stat_get_buf_alloc() AS buffers_alloc,
- pg_stat_get_bgwriter_stat_reset_time() AS stats_reset;
+ ( SELECT pg_stat_buffers.write
+ FROM pg_stat_buffers
+ WHERE ((pg_stat_buffers.backend_type = 'client backend'::text) AND (pg_stat_buffers.io_path = 'shared'::text))) AS buffers_backend,
+ ( SELECT pg_stat_buffers.fsync
+ FROM pg_stat_buffers
+ WHERE ((pg_stat_buffers.backend_type = 'client backend'::text) AND (pg_stat_buffers.io_path = 'shared'::text))) AS buffers_backend_fsync,
+ ( SELECT pg_stat_buffers.alloc
+ FROM pg_stat_buffers
+ WHERE ((pg_stat_buffers.backend_type = 'client backend'::text) AND (pg_stat_buffers.io_path = 'shared'::text))) AS buffers_alloc,
+ ( SELECT pg_stat_buffers.stats_reset
+ FROM pg_stat_buffers
+ LIMIT 1) AS stats_reset;
pg_stat_buffers| SELECT b.backend_type,
b.io_path,
b.alloc,
--
2.30.2
From 37fa6f0503e6d959e70b70e8a936dee7a7fc4c0f Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplage...@gmail.com>
Date: Wed, 24 Nov 2021 10:32:56 -0500
Subject: [PATCH v20 3/8] Add IO operation counters to PgBackendStatus
Add an array of counters to PgBackendStatus which count the buffers
allocated, extended, fsynced, and written by a given backend.
Each "IO Op" (alloc, fsync, extend, write) is counted per "IO Path"
(direct, local, shared, or strategy).
"local" and "shared" IO Path counters count operations on local and
shared buffers.
The "strategy" IO Path counts buffers alloc'd/written/read/fsync'd as
part of a BufferAccessStrategy.
The "direct" IO Path counts blocks of IO which are read, written, or
fsync'd using smgrwrite/extend/immedsync directly (as opposed to through
[Local]BufferAlloc()).
With this commit, backends increment a counter in the array in their
PgBackendStatus when performing an IO operation.
Future patches will persist the IO stat counters from a backend's
PgBackendStatus upon backend exit and use the counters to provide
observability of database IO operations.
Note that this commit does not add code to increment the "direct" path.
A future patch adding wrappers for smgrwrite(), smgrextend(), and
smgrimmedsync() would provide a good location to call pgstat_inc_ioop()
for unbuffered IO and avoid regressions for future users of these
functions.
Author: Melanie Plageman <melanieplage...@gmail.com>
Reviewed-by: Justin Pryzby <pry...@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/flat/20200124195226.lth52iydq2n2uilq%40alap3.anarazel.de
---
src/backend/postmaster/checkpointer.c | 1 +
src/backend/storage/buffer/bufmgr.c | 47 +++++++++++---
src/backend/storage/buffer/freelist.c | 22 ++++++-
src/backend/storage/buffer/localbuf.c | 3 +
src/backend/storage/sync/sync.c | 1 +
src/backend/utils/activity/backend_status.c | 10 +++
src/include/storage/buf_internals.h | 4 +-
src/include/utils/backend_status.h | 68 +++++++++++++++++++++
8 files changed, 141 insertions(+), 15 deletions(-)
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 25a18b7a14..8440b2b802 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -1101,6 +1101,7 @@ ForwardSyncRequest(const FileTag *ftag, SyncRequestType type)
if (!AmBackgroundWriterProcess())
CheckpointerShmem->num_backend_fsync++;
LWLockRelease(CheckpointerCommLock);
+ pgstat_inc_ioop(IOOP_FSYNC, IOPATH_SHARED);
return false;
}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index b4532948d3..528995e7fb 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -480,7 +480,7 @@ static BufferDesc *BufferAlloc(SMgrRelation smgr,
BlockNumber blockNum,
BufferAccessStrategy strategy,
bool *foundPtr);
-static void FlushBuffer(BufferDesc *buf, SMgrRelation reln);
+static void FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOPath iopath);
static void FindAndDropRelFileNodeBuffers(RelFileNode rnode,
ForkNumber forkNum,
BlockNumber nForkBlock,
@@ -972,6 +972,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (isExtend)
{
+ pgstat_inc_ioop(IOOP_EXTEND, IOPATH_SHARED);
/* new buffers are zero-filled */
MemSet((char *) bufBlock, 0, BLCKSZ);
/* don't set checksum for all-zero page */
@@ -1172,6 +1173,8 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
/* Loop here in case we have to try another victim buffer */
for (;;)
{
+ bool from_ring;
+
/*
* Ensure, while the spinlock's not yet held, that there's a free
* refcount entry.
@@ -1182,7 +1185,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
* Select a victim buffer. The buffer is returned with its header
* spinlock still held!
*/
- buf = StrategyGetBuffer(strategy, &buf_state);
+ buf = StrategyGetBuffer(strategy, &buf_state, &from_ring);
Assert(BUF_STATE_GET_REFCOUNT(buf_state) == 0);
@@ -1219,6 +1222,8 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (LWLockConditionalAcquire(BufferDescriptorGetContentLock(buf),
LW_SHARED))
{
+ IOPath iopath;
+
/*
* If using a nondefault strategy, and writing the buffer
* would require a WAL flush, let the strategy decide whether
@@ -1236,7 +1241,7 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
UnlockBufHdr(buf, buf_state);
if (XLogNeedsFlush(lsn) &&
- StrategyRejectBuffer(strategy, buf))
+ StrategyRejectBuffer(strategy, buf, &from_ring))
{
/* Drop lock/pin and loop around for another buffer */
LWLockRelease(BufferDescriptorGetContentLock(buf));
@@ -1245,13 +1250,27 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
}
}
+ /*
+ * When a strategy is in use, if the dirty buffer was selected
+ * from the strategy ring and we did not bother checking the
+ * freelist or doing a clock sweep to look for a clean shared
+ * buffer to use, the write will be counted as a strategy
+ * write. However, if the dirty buffer was obtained from the
+ * freelist or a clock sweep, it is counted as a regular write.
+ *
+ * When a strategy is not in use, at this point the write can
+ * only be a "regular" write of a dirty buffer.
+ */
+
+ iopath = from_ring ? IOPATH_STRATEGY : IOPATH_SHARED;
+
/* OK, do the I/O */
TRACE_POSTGRESQL_BUFFER_WRITE_DIRTY_START(forkNum, blockNum,
smgr->smgr_rnode.node.spcNode,
smgr->smgr_rnode.node.dbNode,
smgr->smgr_rnode.node.relNode);
- FlushBuffer(buf, NULL);
+ FlushBuffer(buf, NULL, iopath);
LWLockRelease(BufferDescriptorGetContentLock(buf));
ScheduleBufferTagForWriteback(&BackendWritebackContext,
@@ -2552,10 +2571,11 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context)
* Pin it, share-lock it, write it. (FlushBuffer will do nothing if the
* buffer is clean by the time we've locked it.)
*/
+
PinBuffer_Locked(bufHdr);
LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_SHARED);
- FlushBuffer(bufHdr, NULL);
+ FlushBuffer(bufHdr, NULL, IOPATH_SHARED);
LWLockRelease(BufferDescriptorGetContentLock(bufHdr));
@@ -2803,9 +2823,12 @@ BufferGetTag(Buffer buffer, RelFileNode *rnode, ForkNumber *forknum,
*
* If the caller has an smgr reference for the buffer's relation, pass it
* as the second parameter. If not, pass NULL.
+ *
+ * IOPath will always be IOPATH_SHARED except when a buffer access strategy is
+ * used and the buffer being flushed is a buffer from the strategy ring.
*/
static void
-FlushBuffer(BufferDesc *buf, SMgrRelation reln)
+FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOPath iopath)
{
XLogRecPtr recptr;
ErrorContextCallback errcallback;
@@ -2897,6 +2920,8 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
bufToWrite,
false);
+ pgstat_inc_ioop(IOOP_WRITE, iopath);
+
if (track_io_timing)
{
INSTR_TIME_SET_CURRENT(io_time);
@@ -3533,6 +3558,8 @@ FlushRelationBuffers(Relation rel)
localpage,
false);
+ pgstat_inc_ioop(IOOP_WRITE, IOPATH_LOCAL);
+
buf_state &= ~(BM_DIRTY | BM_JUST_DIRTIED);
pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
@@ -3568,7 +3595,7 @@ FlushRelationBuffers(Relation rel)
{
PinBuffer_Locked(bufHdr);
LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_SHARED);
- FlushBuffer(bufHdr, RelationGetSmgr(rel));
+ FlushBuffer(bufHdr, RelationGetSmgr(rel), IOPATH_SHARED);
LWLockRelease(BufferDescriptorGetContentLock(bufHdr));
UnpinBuffer(bufHdr, true);
}
@@ -3664,7 +3691,7 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels)
{
PinBuffer_Locked(bufHdr);
LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_SHARED);
- FlushBuffer(bufHdr, srelent->srel);
+ FlushBuffer(bufHdr, srelent->srel, IOPATH_SHARED);
LWLockRelease(BufferDescriptorGetContentLock(bufHdr));
UnpinBuffer(bufHdr, true);
}
@@ -3720,7 +3747,7 @@ FlushDatabaseBuffers(Oid dbid)
{
PinBuffer_Locked(bufHdr);
LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_SHARED);
- FlushBuffer(bufHdr, NULL);
+ FlushBuffer(bufHdr, NULL, IOPATH_SHARED);
LWLockRelease(BufferDescriptorGetContentLock(bufHdr));
UnpinBuffer(bufHdr, true);
}
@@ -3747,7 +3774,7 @@ FlushOneBuffer(Buffer buffer)
Assert(LWLockHeldByMe(BufferDescriptorGetContentLock(bufHdr)));
- FlushBuffer(bufHdr, NULL);
+ FlushBuffer(bufHdr, NULL, IOPATH_SHARED);
}
/*
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 6be80476db..45d73995b2 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -19,6 +19,7 @@
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
#include "storage/proc.h"
+#include "utils/backend_status.h"
#define INT_ACCESS_ONCE(var) ((int)(*((volatile int *)&(var))))
@@ -198,7 +199,7 @@ have_free_buffer(void)
* return the buffer with the buffer header spinlock still held.
*/
BufferDesc *
-StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state)
+StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_ring)
{
BufferDesc *buf;
int bgwprocno;
@@ -212,7 +213,8 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state)
if (strategy != NULL)
{
buf = GetBufferFromRing(strategy, buf_state);
- if (buf != NULL)
+ *from_ring = buf != NULL;
+ if (*from_ring)
return buf;
}
@@ -247,6 +249,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state)
* the rate of buffer consumption. Note that buffers recycled by a
* strategy object are intentionally not counted here.
*/
+ pgstat_inc_ioop(IOOP_ALLOC, IOPATH_SHARED);
pg_atomic_fetch_add_u32(&StrategyControl->numBufferAllocs, 1);
/*
@@ -683,8 +686,14 @@ AddBufferToRing(BufferAccessStrategy strategy, BufferDesc *buf)
* if this buffer should be written and re-used.
*/
bool
-StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf)
+StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool *from_ring)
{
+ /*
+ * Start by assuming that we will use the dirty buffer selected by
+ * StrategyGetBuffer().
+ */
+ *from_ring = true;
+
/* We only do this in bulkread mode */
if (strategy->btype != BAS_BULKREAD)
return false;
@@ -700,5 +709,12 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf)
*/
strategy->buffers[strategy->current] = InvalidBuffer;
+ /*
+ * Since we will not be writing out a dirty buffer from the ring, set
+ * from_ring to false so that the caller does not count this write as a
+ * "strategy write" and can do proper bookkeeping.
+ */
+ *from_ring = false;
+
return true;
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index 04b3558ea3..f396a2b68d 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -20,6 +20,7 @@
#include "executor/instrument.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
+#include "utils/backend_status.h"
#include "utils/guc.h"
#include "utils/memutils.h"
#include "utils/resowner_private.h"
@@ -226,6 +227,8 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
localpage,
false);
+ pgstat_inc_ioop(IOOP_WRITE, IOPATH_LOCAL);
+
/* Mark not-dirty now in case we error out below */
buf_state &= ~BM_DIRTY;
pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
diff --git a/src/backend/storage/sync/sync.c b/src/backend/storage/sync/sync.c
index d4083e8a56..3be06d5d5a 100644
--- a/src/backend/storage/sync/sync.c
+++ b/src/backend/storage/sync/sync.c
@@ -420,6 +420,7 @@ ProcessSyncRequests(void)
total_elapsed += elapsed;
processed++;
+ pgstat_inc_ioop(IOOP_FSYNC, IOPATH_SHARED);
if (log_checkpoints)
elog(DEBUG1, "checkpoint sync: number=%d file=%s time=%.3f ms",
processed,
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 7229598822..44c9a6e1a6 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -400,6 +400,16 @@ pgstat_bestart(void)
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ for (int i = 0; i < IOPATH_NUM_TYPES; i++)
+ {
+ IOOpCounters *io_ops = &lbeentry.io_path_stats[i];
+
+ pg_atomic_init_u64(&io_ops->allocs, 0);
+ pg_atomic_init_u64(&io_ops->extends, 0);
+ pg_atomic_init_u64(&io_ops->fsyncs, 0);
+ pg_atomic_init_u64(&io_ops->writes, 0);
+ }
+
/*
* we don't zero st_progress_param here to save cycles; nobody should
* examine it until st_progress_command has been set to something other
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index 7c6653311a..15f5724fa1 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -310,10 +310,10 @@ extern void ScheduleBufferTagForWriteback(WritebackContext *context, BufferTag *
/* freelist.c */
extern BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy,
- uint32 *buf_state);
+ uint32 *buf_state, bool *from_ring);
extern void StrategyFreeBuffer(BufferDesc *buf);
extern bool StrategyRejectBuffer(BufferAccessStrategy strategy,
- BufferDesc *buf);
+ BufferDesc *buf, bool *from_ring);
extern int StrategySyncStart(uint32 *complete_passes, uint32 *num_buf_alloc);
extern void StrategyNotifyBgWriter(int bgwprocno);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 8042b817df..56a0f25296 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -13,6 +13,7 @@
#include "datatype/timestamp.h"
#include "libpq/pqcomm.h"
#include "miscadmin.h" /* for BackendType */
+#include "port/atomics.h"
#include "utils/backend_progress.h"
@@ -31,12 +32,47 @@ typedef enum BackendState
STATE_DISABLED
} BackendState;
+/* ----------
+ * IO Stats reporting utility types
+ * ----------
+ */
+
+typedef enum IOOp
+{
+ IOOP_ALLOC,
+ IOOP_EXTEND,
+ IOOP_FSYNC,
+ IOOP_WRITE,
+} IOOp;
+
+#define IOOP_NUM_TYPES (IOOP_WRITE + 1)
+
+typedef enum IOPath
+{
+ IOPATH_DIRECT,
+ IOPATH_LOCAL,
+ IOPATH_SHARED,
+ IOPATH_STRATEGY,
+} IOPath;
+
+#define IOPATH_NUM_TYPES (IOPATH_STRATEGY + 1)
/* ----------
* Shared-memory data structures
* ----------
*/
+/*
+ * Structure for counting all types of IOOp for a live backend.
+ */
+typedef struct IOOpCounters
+{
+ pg_atomic_uint64 allocs;
+ pg_atomic_uint64 extends;
+ pg_atomic_uint64 fsyncs;
+ pg_atomic_uint64 writes;
+} IOOpCounters;
+
/*
* PgBackendSSLStatus
*
@@ -168,6 +204,12 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /*
+ * Stats on all IOOps for all IOPaths for this backend. These should be
+ * incremented whenever an IO Operation is performed.
+ */
+ IOOpCounters io_path_stats[IOPATH_NUM_TYPES];
} PgBackendStatus;
@@ -296,6 +338,32 @@ extern void pgstat_bestart(void);
extern void pgstat_clear_backend_activity_snapshot(void);
/* Activity reporting functions */
+
+static inline void
+pgstat_inc_ioop(IOOp io_op, IOPath io_path)
+{
+ IOOpCounters *io_ops;
+ PgBackendStatus *beentry = MyBEEntry;
+
+ Assert(beentry);
+
+ io_ops = &beentry->io_path_stats[io_path];
+ switch (io_op)
+ {
+ case IOOP_ALLOC:
+ pg_atomic_unlocked_inc_counter(&io_ops->allocs);
+ break;
+ case IOOP_EXTEND:
+ pg_atomic_unlocked_inc_counter(&io_ops->extends);
+ break;
+ case IOOP_FSYNC:
+ pg_atomic_unlocked_inc_counter(&io_ops->fsyncs);
+ break;
+ case IOOP_WRITE:
+ pg_atomic_unlocked_inc_counter(&io_ops->writes);
+ break;
+ }
+}
extern void pgstat_report_activity(BackendState state, const char *cmd_str);
extern void pgstat_report_query_id(uint64 query_id, bool force);
extern void pgstat_report_tempfile(size_t filesize);
--
2.30.2
From b6ada2c3033b03dac6cbdddf0e29454f9d26d986 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplage...@gmail.com>
Date: Mon, 11 Oct 2021 16:15:06 -0400
Subject: [PATCH v20 1/8] Read-only atomic backend write function
For counters in shared memory which can be read by any backend but only
written to by one backend, an atomic is still needed to protect against
torn values; however, pg_atomic_fetch_add_u64() is overkill for
incrementing such counters. pg_atomic_unlocked_inc_counter() is a helper
function which can be used to increment these values safely without
unnecessary overhead.
Author: Thomas Munro <tmu...@postgresql.org>
Reviewed-by: Melanie Plageman <melanieplage...@gmail.com>
Discussion: https://www.postgresql.org/message-id/CA%2BhUKGJ06d3h5JeOtAv4h52n0vG1jOPZxqMCn5FySJQUVZA32w%40mail.gmail.com
Discussion: https://www.postgresql.org/message-id/flat/20200124195226.lth52iydq2n2uilq%40alap3.anarazel.de
---
src/include/port/atomics.h | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/src/include/port/atomics.h b/src/include/port/atomics.h
index 856338f161..af30da32e5 100644
--- a/src/include/port/atomics.h
+++ b/src/include/port/atomics.h
@@ -519,6 +519,17 @@ pg_atomic_sub_fetch_u64(volatile pg_atomic_uint64 *ptr, int64 sub_)
return pg_atomic_sub_fetch_u64_impl(ptr, sub_);
}
+/*
+ * On modern systems this is really just *counter++. On some older systems
+ * there might be more to it (due to an inability to read and write 64-bit
+ * values atomically).
+ */
+static inline void
+pg_atomic_unlocked_inc_counter(pg_atomic_uint64 *counter)
+{
+ pg_atomic_write_u64(counter, pg_atomic_read_u64(counter) + 1);
+}
+
#undef INSIDE_ATOMICS_H
#endif /* ATOMICS_H */
--
2.30.2
From 569bfe5bd2493d8873d8763c1767ccf34fa15505 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplage...@gmail.com>
Date: Tue, 14 Dec 2021 12:26:56 -0500
Subject: [PATCH v20 2/8] Move backend pgstat initialization earlier
Initialize the pgstats subsystem earlier during process initialization
so that more process types have a backend activity state
(PgBackendStatus).
Conditionally initializing backend activity state in some types of
processes and not in others necessitates surprising special cases in the
code.
This particular commit was motivated by single user mode missing a
backend activity state.
This commit also adds a new BackendType for standalone backends,
B_STANDALONE_BACKEND (and alphabetizes the BackendTypes). Both the
bootstrap backend and single user mode backends will have BackendType
B_STANDALONE_BACKEND.
Author: Melanie Plageman <melanieplage...@gmail.com>
Discussion: https://www.postgresql.org/message-id/CAAKRu_aaq33UnG4TXq3S-OSXGWj1QGf0sU%2BECH4tNwGFNERkZA%40mail.gmail.com
---
src/backend/utils/init/miscinit.c | 23 ++++++++++++++---------
src/backend/utils/init/postinit.c | 7 +++----
src/include/miscadmin.h | 7 ++++---
3 files changed, 21 insertions(+), 16 deletions(-)
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 88801374b5..41d0b023cd 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -176,6 +176,8 @@ InitStandaloneProcess(const char *argv0)
{
Assert(!IsPostmasterEnvironment);
+ MyBackendType = B_STANDALONE_BACKEND;
+
/*
* Start our win32 signal implementation
*/
@@ -255,6 +257,9 @@ GetBackendTypeDesc(BackendType backendType)
case B_INVALID:
backendDesc = "not initialized";
break;
+ case B_ARCHIVER:
+ backendDesc = "archiver";
+ break;
case B_AUTOVAC_LAUNCHER:
backendDesc = "autovacuum launcher";
break;
@@ -273,9 +278,18 @@ GetBackendTypeDesc(BackendType backendType)
case B_CHECKPOINTER:
backendDesc = "checkpointer";
break;
+ case B_LOGGER:
+ backendDesc = "logger";
+ break;
+ case B_STANDALONE_BACKEND:
+ backendDesc = "standalone backend";
+ break;
case B_STARTUP:
backendDesc = "startup";
break;
+ case B_STATS_COLLECTOR:
+ backendDesc = "stats collector";
+ break;
case B_WAL_RECEIVER:
backendDesc = "walreceiver";
break;
@@ -285,15 +299,6 @@ GetBackendTypeDesc(BackendType backendType)
case B_WAL_WRITER:
backendDesc = "walwriter";
break;
- case B_ARCHIVER:
- backendDesc = "archiver";
- break;
- case B_STATS_COLLECTOR:
- backendDesc = "stats collector";
- break;
- case B_LOGGER:
- backendDesc = "logger";
- break;
}
return backendDesc;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 7292e51f7d..11f1fec17e 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -623,6 +623,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(CLIENT_CONNECTION_CHECK_TIMEOUT, ClientCheckTimeoutHandler);
}
+ pgstat_beinit();
+
/*
* If this is either a bootstrap process nor a standalone backend, start
* up the XLOG machinery, and register to have it closed down at exit.
@@ -638,6 +640,7 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
*/
CreateAuxProcessResourceOwner();
+ pgstat_bestart();
StartupXLOG();
/* Release (and warn about) any buffer pins leaked in StartupXLOG */
ReleaseAuxProcessResources(true);
@@ -665,7 +668,6 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
EnablePortalManager();
/* Initialize status reporting */
- pgstat_beinit();
/*
* Load relcache entries for the shared system catalogs. This must create
@@ -903,10 +905,7 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
* transaction we started before returning.
*/
if (!bootstrap)
- {
- pgstat_bestart();
CommitTransactionCommand();
- }
return;
}
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 90a3016065..1d688f9e51 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -323,19 +323,20 @@ extern void SwitchBackToLocalLatch(void);
typedef enum BackendType
{
B_INVALID = 0,
+ B_ARCHIVER,
B_AUTOVAC_LAUNCHER,
B_AUTOVAC_WORKER,
B_BACKEND,
B_BG_WORKER,
B_BG_WRITER,
B_CHECKPOINTER,
+ B_LOGGER,
+ B_STANDALONE_BACKEND,
B_STARTUP,
+ B_STATS_COLLECTOR,
B_WAL_RECEIVER,
B_WAL_SENDER,
B_WAL_WRITER,
- B_ARCHIVER,
- B_STATS_COLLECTOR,
- B_LOGGER,
} BackendType;
extern BackendType MyBackendType;
--
2.30.2
