On Tue, Sep 8, 2026 at 6:33 PM David Geier <[email protected]> wrote:
> >>>> However, I'm wondering if the better approach wouldn't be to change > fd.c > >>>> to use a long-lived memory context. Then all bookkeeping would happen > >>>> automatically and the memory size could simply be reported via > existing > >>>> memory context stats infrastructure. > >>>> > >>>> Not entirely sure though if there's some roadblock when switching to a > >>>> memory context. > >>> I don't see any issue with this either. However, the only benefit we > >> would > >>> gain is using existing infrastructure but only for backend vfd cache > >> memory > >>> (i.e cache_bytes). > >>> Everything else stays the same (counters, cluster-wide memory); > >> therefore, > >>> if there's no other benefit to replacing with memory contexts, maybe > it's > >>> not worth it. > >> > >> The biggest benefit in my view is consistency with the rest of > PostgreSQL. > >> That is from a usage point of view as well as from a coding point of > view. > >> If you want, I can give that a try and share a patch with you if > >> successful. > >> > > Yes of course, I’d be happy to take a look. > Attached is the patch. It's pretty small and passes regress tests. > Nice one. Few comments on it: #1: + oldcontext = MemoryContextSwitchTo(VfdCxt); + newVfdCache = repalloc_array(VfdCache, Vfd, newCacheSize); + MemoryContextSwitchTo(oldcontext); Can't we just do this? + newVfdCache = repalloc_array(VfdCache, Vfd, newCacheSize); because repalloc doesn't need CurrentMemoryContext. #2: + newDescs = MemoryContextAllocExtended(VfdCxt, + newMax * sizeof(AllocateDesc), MCXT_ALLOC_NO_OOM); if (newDescs == NULL) return false; + memcpy(newDescs, allocatedDescs, maxAllocatedDescs * sizeof(AllocateDesc)); + pfree(allocatedDescs); We can also just replace it with: + newDescs = repalloc_array_extended(allocatedDescs, AllocateDesc, + newMax, MCXT_ALLOC_NO_OOM); Correct? Attached are v7 patches containing your patch for memory contexts with these small suggestions and pg_stat_vfdcache patches like before. Regards, Ayoub
From 59f823ed36790b0a3d1bb38b75910ff8a8824edc Mon Sep 17 00:00:00 2001 From: AyoubKAZ <[email protected]> Date: Tue, 25 Aug 2026 00:46:15 +0200 Subject: [PATCH v7 2/3] Add pg_stat_vfdcache view for VFD cache statistics PostgreSQL's virtual file descriptor (VFD) layer maintains a per-backend cache of open file descriptors bounded by max_files_per_process (default 1000). When the cache is full, the least-recently-used entry is evicted (its OS fd closed) so a new file can be opened. A subsequent access to an evicted file must call open() again. A trivial example is with partitioned tables: a table with 1500 partitions requires up to many file descriptors per full scan (main fork, vm ...), which is more than the default limit, causing potential evictions and reopens. This commit adds: pg_stat_vfdcache -- a single-row view exposing cluster-wide VFD cache statistics: hits number of VFD cache hits misses number of VFD cache misses max_open_fds maximum number of file descriptors available to each backend process hit_ratio hits / (hits + misses) stats_reset timestamp of last counter reset pg_stat_reset_vfdcache() -- resets shared VFD counters The implementation follows the same cumulative shared statistics infrastructure like pgstat_bgwriter and others do. Event counting remains cheap in backend-local pending storage and is flushed into shared fixed stats which requires locking. Hit and miss counters are placed in FileAccess(), which is the single gate through which all VFD-mediated file reads, writes, truncations, and size checks pass. --- doc/src/sgml/monitoring.sgml | 107 ++++++++++++++++++ src/backend/catalog/system_views.sql | 16 +++ src/backend/storage/file/fd.c | 5 +- src/backend/utils/activity/Makefile | 1 + src/backend/utils/activity/meson.build | 1 + src/backend/utils/activity/pgstat.c | 17 +++ src/backend/utils/activity/pgstat_vfdcache.c | 113 +++++++++++++++++++ src/backend/utils/adt/pgstatfuncs.c | 47 +++++++- src/include/catalog/pg_proc.dat | 30 +++++ src/include/pgstat.h | 26 +++++ src/include/utils/pgstat_internal.h | 22 +++- src/include/utils/pgstat_kind.h | 3 +- src/test/regress/expected/rules.out | 8 ++ src/test/regress/expected/stats.out | 36 +++++- src/test/regress/sql/stats.sql | 13 +++ 15 files changed, 439 insertions(+), 6 deletions(-) create mode 100644 src/backend/utils/activity/pgstat_vfdcache.c diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index b403fb990a7..f7c41c2d960 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -563,6 +563,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser </entry> </row> + <row> + <entry><structname>pg_stat_vfdcache</structname><indexterm><primary>pg_stat_vfdcache</primary></indexterm></entry> + <entry>One row only, showing cluster-wide statistics about virtual file + descriptor (VFD) cache activity. See + <link linkend="monitoring-pg-stat-vfdcache-view"> + <structname>pg_stat_vfdcache</structname></link> for details. + </entry> + </row> + <row> <entry><structname>pg_stat_wal</structname><indexterm><primary>pg_stat_wal</primary></indexterm></entry> <entry>One row only, showing statistics about WAL activity. See @@ -3905,6 +3914,89 @@ description | Waiting for a newly initialized WAL file to reach durable storage </para> </sect2> + <sect2 id="monitoring-pg-stat-vfdcache-view"> + <title><structname>pg_stat_vfdcache</structname></title> + + <indexterm zone="monitoring-pg-stat-vfdcache-view"> + <primary>pg_stat_vfdcache</primary> + </indexterm> + + <para> + The <structname>pg_stat_vfdcache</structname> view will always have a + single row, containing data about cluster-wide VFD (Virtual File + Descriptor) cache activity. + </para> + + <table id="pg-stat-vfdcache-view" xreflabel="pg_stat_vfdcache"> + <title><structname>pg_stat_vfdcache</structname> View</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + Column Type + </para> + <para> + Description + </para></entry> + </row> + </thead> + + <tbody> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>hits</structfield> <type>bigint</type> + </para> + <para> + Number of file accesses where the physical file descriptor was + already open in the cache, requiring no system call + </para></entry> + </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>misses</structfield> <type>bigint</type> + </para> + <para> + Number of file accesses where the physical file descriptor had + been evicted from the cache, requiring <function>open()</function> + to be called again before the access could proceed + </para></entry> + </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>max_open_fds</structfield> <type>integer</type> + </para> + <para> + Maximum number of file descriptors available to each backend process + </para></entry> + </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>hit_ratio</structfield> <type>float8</type> + </para> + <para> + Fraction of file accesses that were cache hits: + <literal>hits / (hits + misses)</literal> + </para></entry> + </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>stats_reset</structfield> <type>timestamp with time zone</type> + </para> + <para> + Time at which the counters were last reset by + <function>pg_stat_reset_vfdcache()</function> + </para></entry> + </row> + </tbody> + </tgroup> + </table> + + </sect2> + <sect2 id="monitoring-pg-stat-wal-view"> <title><structname>pg_stat_wal</structname></title> @@ -6153,6 +6245,21 @@ description | Waiting for a newly initialized WAL file to reach durable storage can be granted EXECUTE to run the function. </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_stat_reset_vfdcache</primary> + </indexterm> + <function>pg_stat_reset_vfdcache</function> () + <returnvalue>void</returnvalue> + </para> + <para> + Reset shared VFD cache statistics counters to zero. The reset + timestamp is recorded in + <structname>pg_stat_vfdcache</structname>.<structfield>stats_reset</structfield>. + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 8612d99a890..844fea1391f 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -1568,3 +1568,19 @@ CREATE VIEW pg_aios AS SELECT * FROM pg_get_aios(); REVOKE ALL ON pg_aios FROM PUBLIC; GRANT SELECT ON pg_aios TO pg_read_all_stats; + +CREATE VIEW pg_stat_vfdcache AS + SELECT + pg_stat_get_vfd_hits() AS hits, + pg_stat_get_vfd_misses() AS misses, + pg_stat_get_vfd_max_open_fds() AS max_open_fds, + CASE + WHEN pg_stat_get_vfd_hits() + pg_stat_get_vfd_misses() = 0 + THEN NULL::float8 + ELSE pg_stat_get_vfd_hits()::float8 + / (pg_stat_get_vfd_hits() + pg_stat_get_vfd_misses()) + END AS hit_ratio, + pg_stat_get_vfd_stat_reset_time() AS stats_reset; + +REVOKE ALL ON pg_stat_vfdcache FROM PUBLIC; +GRANT SELECT ON pg_stat_vfdcache TO PUBLIC; diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index f0fe48f60dc..d7ecaf5e6c5 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -1479,6 +1479,7 @@ static int FileAccess(File file) { int returnValue; + bool is_open; DO_DB(elog(LOG, "FileAccess %d (%s)", file, VfdCache[file].fileName)); @@ -1487,8 +1488,10 @@ FileAccess(File file) * Is the file open? If not, open it and put it at the head of the LRU * ring (possibly closing the least recently used file to get an FD). */ + is_open = !FileIsNotOpen(file); + pgstat_count_vfd_access(is_open); - if (FileIsNotOpen(file)) + if (!is_open) { returnValue = LruInsert(file); if (returnValue != 0) diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile index 2e32d1485d6..264e170bb95 100644 --- a/src/backend/utils/activity/Makefile +++ b/src/backend/utils/activity/Makefile @@ -34,6 +34,7 @@ OBJS = \ pgstat_shmem.o \ pgstat_slru.o \ pgstat_subscription.o \ + pgstat_vfdcache.o \ pgstat_wal.o \ pgstat_xact.o \ wait_event.o \ diff --git a/src/backend/utils/activity/meson.build b/src/backend/utils/activity/meson.build index e6dcb2e26fc..4ac9c5e93e1 100644 --- a/src/backend/utils/activity/meson.build +++ b/src/backend/utils/activity/meson.build @@ -19,6 +19,7 @@ backend_sources += files( 'pgstat_shmem.c', 'pgstat_slru.c', 'pgstat_subscription.c', + 'pgstat_vfdcache.c', 'pgstat_wal.c', 'pgstat_xact.c', ) diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c index 5177f880f70..ab90a01d9af 100644 --- a/src/backend/utils/activity/pgstat.c +++ b/src/backend/utils/activity/pgstat.c @@ -516,6 +516,23 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .reset_all_cb = pgstat_wal_reset_all_cb, .snapshot_cb = pgstat_wal_snapshot_cb, }, + + [PGSTAT_KIND_VFDCACHE] = { + .name = "vfdcache", + + .fixed_amount = true, + .write_to_file = true, + + .snapshot_ctl_off = offsetof(PgStat_Snapshot, vfdcache), + .shared_ctl_off = offsetof(PgStat_ShmemControl, vfdcache), + .shared_data_off = offsetof(PgStatShared_VfdCache, stats), + .shared_data_len = sizeof(((PgStatShared_VfdCache *) 0)->stats), + + .flush_static_cb = pgstat_vfdcache_flush_cb, + .init_shmem_cb = pgstat_vfdcache_init_shmem_cb, + .reset_all_cb = pgstat_vfdcache_reset_all_cb, + .snapshot_cb = pgstat_vfdcache_snapshot_cb, + }, }; /* diff --git a/src/backend/utils/activity/pgstat_vfdcache.c b/src/backend/utils/activity/pgstat_vfdcache.c new file mode 100644 index 00000000000..b036759031c --- /dev/null +++ b/src/backend/utils/activity/pgstat_vfdcache.c @@ -0,0 +1,113 @@ +/* ------------------------------------------------------------------------- + * + * pgstat_vfdcache.c + * Implementation of VFD cache statistics. + * + * VFD events are first counted in backend-local pending storage and then + * flushed into shared-memory cumulative stats, following the same model as + * other fixed stats kinds. + * + * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/backend/utils/activity/pgstat_vfdcache.c + * ------------------------------------------------------------------------- + */ + +#include "postgres.h" + +#include "pgstat.h" +#include "utils/memutils.h" +#include "utils/pgstat_internal.h" + +/* + * Backend-local VFD counters waiting to be flushed. + */ +PgStat_VfdCacheStats PendingVfdCacheStats = {0}; + +/* + * Count a VFD cache access as either a hit or miss. + */ +void +pgstat_count_vfd_access(bool hit) +{ + if (hit) + PendingVfdCacheStats.vfd_hits++; + else + PendingVfdCacheStats.vfd_misses++; + pgstat_report_fixed = true; +} + +/* + * Flush out backend-local pending VFD cache stats. + */ +bool +pgstat_vfdcache_flush_cb(bool nowait) +{ + PgStatShared_VfdCache *stats_shmem = &pgStatLocal.shmem->vfdcache; + + if (pg_memory_is_all_zeros(&PendingVfdCacheStats, + sizeof(struct PgStat_VfdCacheStats))) + return false; + + if (!nowait) + LWLockAcquire(&stats_shmem->lock, LW_EXCLUSIVE); + else if (!LWLockConditionalAcquire(&stats_shmem->lock, LW_EXCLUSIVE)) + return true; + + stats_shmem->stats.vfd_hits += PendingVfdCacheStats.vfd_hits; + stats_shmem->stats.vfd_misses += PendingVfdCacheStats.vfd_misses; + + LWLockRelease(&stats_shmem->lock); + + MemSet(&PendingVfdCacheStats, 0, sizeof(PendingVfdCacheStats)); + + return false; +} + +/* + * Support function for SQL-callable pg_stat_get_vfd_* functions. + */ +PgStat_VfdCacheStats * +pgstat_fetch_stat_vfdcache(void) +{ + pgstat_snapshot_fixed(PGSTAT_KIND_VFDCACHE); + + return &pgStatLocal.snapshot.vfdcache; +} + +void +pgstat_vfdcache_init_shmem_cb(void *stats) +{ + PgStatShared_VfdCache *stats_shmem = (PgStatShared_VfdCache *) stats; + + LWLockInitialize(&stats_shmem->lock, LWTRANCHE_PGSTATS_DATA); +} + +void +pgstat_vfdcache_reset_all_cb(TimestampTz ts) +{ + PgStatShared_VfdCache *stats_shmem = &pgStatLocal.shmem->vfdcache; + + LWLockAcquire(&stats_shmem->lock, LW_EXCLUSIVE); + MemSet(&stats_shmem->stats, 0, sizeof(stats_shmem->stats)); + stats_shmem->stats.stat_reset_timestamp = ts; + LWLockRelease(&stats_shmem->lock); +} + +void +pgstat_vfdcache_snapshot_cb(void) +{ + PgStatShared_VfdCache *stats_shmem = &pgStatLocal.shmem->vfdcache; + + LWLockAcquire(&stats_shmem->lock, LW_SHARED); + memcpy(&pgStatLocal.snapshot.vfdcache, &stats_shmem->stats, + sizeof(pgStatLocal.snapshot.vfdcache)); + LWLockRelease(&stats_shmem->lock); +} + +void +pgstat_reset_vfdcache(void) +{ + pgstat_reset_of_kind(PGSTAT_KIND_VFDCACHE); +} diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 0d47d745c18..3499dd40700 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -26,6 +26,7 @@ #include "pgstat.h" #include "postmaster/bgworker.h" #include "replication/logicallauncher.h" +#include "storage/fd.h" #include "storage/proc.h" #include "storage/procarray.h" #include "utils/acl.h" @@ -1398,6 +1399,47 @@ pg_stat_get_buf_alloc(PG_FUNCTION_ARGS) PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->buf_alloc); } +Datum +pg_stat_get_vfd_hits(PG_FUNCTION_ARGS) +{ + PG_RETURN_INT64(pgstat_fetch_stat_vfdcache()->vfd_hits); +} + +Datum +pg_stat_get_vfd_misses(PG_FUNCTION_ARGS) +{ + PG_RETURN_INT64(pgstat_fetch_stat_vfdcache()->vfd_misses); +} + +Datum +pg_stat_get_vfd_max_open_fds(PG_FUNCTION_ARGS) +{ + PG_RETURN_INT32(max_safe_fds); +} + +Datum +pg_stat_get_vfd_stat_reset_time(PG_FUNCTION_ARGS) +{ + TimestampTz ts = pgstat_fetch_stat_vfdcache()->stat_reset_timestamp; + + if (ts == 0) + PG_RETURN_NULL(); + + PG_RETURN_TIMESTAMPTZ(ts); +} + +/* + * pg_stat_reset_vfdcache + * Reset shared VFD cache counters. + */ +Datum +pg_stat_reset_vfdcache(PG_FUNCTION_ARGS) +{ + pgstat_reset_vfdcache(); + PG_RETURN_VOID(); +} + + /* * When adding a new column to the pg_stat_io view and the * pg_stat_get_backend_io() function, add a new enum value here above @@ -2099,6 +2141,7 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) XLogPrefetchResetStats(); pgstat_reset_of_kind(PGSTAT_KIND_SLRU); pgstat_reset_of_kind(PGSTAT_KIND_WAL); + pgstat_reset_of_kind(PGSTAT_KIND_VFDCACHE); PG_RETURN_VOID(); } @@ -2119,13 +2162,15 @@ pg_stat_reset_shared(PG_FUNCTION_ARGS) XLogPrefetchResetStats(); else if (strcmp(target, "slru") == 0) pgstat_reset_of_kind(PGSTAT_KIND_SLRU); + else if (strcmp(target, "vfdcache") == 0) + pgstat_reset_of_kind(PGSTAT_KIND_VFDCACHE); else if (strcmp(target, "wal") == 0) pgstat_reset_of_kind(PGSTAT_KIND_WAL); else ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("unrecognized reset target: \"%s\"", target), - errhint("Target must be \"archiver\", \"bgwriter\", \"checkpointer\", \"io\", \"lock\", \"recovery_prefetch\", \"slru\", or \"wal\"."))); + errhint("Target must be \"archiver\", \"bgwriter\", \"checkpointer\", \"io\", \"lock\", \"recovery_prefetch\", \"slru\", \"vfdcache\", or \"wal\"."))); PG_RETURN_VOID(); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 960763ee50b..d5b5f08bf5c 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -12772,5 +12772,35 @@ { oid => '6462', descr => 'hash', proname => 'hashoid8extended', prorettype => 'int8', proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' }, +{ oid => '9560', + descr => 'statistics: number of VFD cache hits', + proname => 'pg_stat_get_vfd_hits', + provolatile => 'v', proparallel => 'r', + prorettype => 'int8', proargtypes => '', + prosrc => 'pg_stat_get_vfd_hits' }, +{ oid => '9561', + descr => 'statistics: number of VFD cache misses', + proname => 'pg_stat_get_vfd_misses', + provolatile => 'v', proparallel => 'r', + prorettype => 'int8', proargtypes => '', + prosrc => 'pg_stat_get_vfd_misses' }, +{ oid => '9564', + descr => 'statistics: timestamp of last VFD cache stats reset', + proname => 'pg_stat_get_vfd_stat_reset_time', + provolatile => 'v', proparallel => 'r', + prorettype => 'timestamptz', proargtypes => '', + prosrc => 'pg_stat_get_vfd_stat_reset_time' }, +{ oid => '9565', + descr => 'statistics: reset shared VFD cache counters', + proname => 'pg_stat_reset_vfdcache', + provolatile => 'v', proparallel => 'r', + prorettype => 'void', proargtypes => '', + prosrc => 'pg_stat_reset_vfdcache' }, +{ oid => '9566', + descr => 'statistics: max number of file descriptors available to backend', + proname => 'pg_stat_get_vfd_max_open_fds', + provolatile => 'v', proparallel => 'r', + prorettype => 'int4', proargtypes => '', + prosrc => 'pg_stat_get_vfd_max_open_fds' }, ] diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 204782fd630..e8395664df2 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -305,6 +305,19 @@ typedef struct PgStat_CheckpointerStats TimestampTz stat_reset_timestamp; } PgStat_CheckpointerStats; +/* --------- + * PgStat_VfdCacheStats Virtual File Descriptor cache statistics + * + * Tracks hit/miss events in the VFD cache (fd.c). These counters + * are accumulated in shared fixed stats and exposed by pg_stat_vfdcache. + * --------- + */ +typedef struct PgStat_VfdCacheStats +{ + PgStat_Counter vfd_hits; /* fd was open, no open() was needed */ + PgStat_Counter vfd_misses; /* fd was VFD_CLOSED, open() was required */ + TimestampTz stat_reset_timestamp; +} PgStat_VfdCacheStats; /* * Types related to counting IO operations @@ -666,6 +679,13 @@ extern PgStat_BgWriterStats *pgstat_fetch_stat_bgwriter(void); extern void pgstat_report_checkpointer(void); extern PgStat_CheckpointerStats *pgstat_fetch_stat_checkpointer(void); +/* + * Functions in pgstat_vfdcache.c + */ + +extern PgStat_VfdCacheStats *pgstat_fetch_stat_vfdcache(void); +extern void pgstat_reset_vfdcache(void); +extern void pgstat_count_vfd_access(bool hit); /* * Functions in pgstat_io.c @@ -941,6 +961,12 @@ extern PGDLLIMPORT int pgstat_fetch_consistency; /* updated directly by bgwriter and bufmgr */ extern PGDLLIMPORT PgStat_BgWriterStats PendingBgWriterStats; +/* + * Variables in pgstat_vfdcache.c + */ + +/* updated by VFD counting functions called from fd.c */ +extern PGDLLIMPORT PgStat_VfdCacheStats PendingVfdCacheStats; /* * Variables in pgstat_checkpointer.c diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h index 14369e59a1c..e24a1c84c9f 100644 --- a/src/include/utils/pgstat_internal.h +++ b/src/include/utils/pgstat_internal.h @@ -296,7 +296,7 @@ typedef struct PgStat_KindInfo /* * For variable-numbered stats: flush pending stats. Required if pending * data is used. See flush_static_cb when dealing with stats data that - * that cannot use PgStat_EntryRef->pending. + * cannot use PgStat_EntryRef->pending. */ bool (*flush_pending_cb) (PgStat_EntryRef *sr, bool nowait); @@ -486,6 +486,13 @@ typedef struct PgStatShared_Wal PgStat_WalStats stats; } PgStatShared_Wal; +typedef struct PgStatShared_VfdCache +{ + /* lock protects ->stats */ + LWLock lock; + PgStat_VfdCacheStats stats; +} PgStatShared_VfdCache; + /* ---------- @@ -587,6 +594,7 @@ typedef struct PgStat_ShmemControl PgStatShared_Lock lock; PgStatShared_SLRU slru; PgStatShared_Wal wal; + PgStatShared_VfdCache vfdcache; /* * Custom stats data with fixed-numbered objects, indexed by (PgStat_Kind @@ -623,6 +631,8 @@ typedef struct PgStat_Snapshot PgStat_WalStats wal; + PgStat_VfdCacheStats vfdcache; + /* * Data in snapshot for custom fixed-numbered statistics, indexed by * (PgStat_Kind - PGSTAT_KIND_CUSTOM_MIN). Each entry is allocated in @@ -739,6 +749,16 @@ extern void pgstat_checkpointer_reset_all_cb(TimestampTz ts); extern void pgstat_checkpointer_snapshot_cb(void); +/* + * Functions in pgstat_vfdcache.c + */ + +extern bool pgstat_vfdcache_flush_cb(bool nowait); +extern void pgstat_vfdcache_init_shmem_cb(void *stats); +extern void pgstat_vfdcache_reset_all_cb(TimestampTz ts); +extern void pgstat_vfdcache_snapshot_cb(void); + + /* * Functions in pgstat_database.c */ diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h index 45ca599d0dd..d998a3fc091 100644 --- a/src/include/utils/pgstat_kind.h +++ b/src/include/utils/pgstat_kind.h @@ -40,9 +40,10 @@ #define PGSTAT_KIND_LOCK 12 #define PGSTAT_KIND_SLRU 13 #define PGSTAT_KIND_WAL 14 +#define PGSTAT_KIND_VFDCACHE 15 #define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE -#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL +#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_VFDCACHE #define PGSTAT_KIND_BUILTIN_SIZE (PGSTAT_KIND_BUILTIN_MAX + 1) /* Custom stats kinds */ diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 1a29d46213e..cdf21d82924 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -2426,6 +2426,14 @@ pg_stat_user_tables| SELECT relid, stats_reset FROM pg_stat_all_tables WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text)); +pg_stat_vfdcache| SELECT pg_stat_get_vfd_hits() AS hits, + pg_stat_get_vfd_misses() AS misses, + pg_stat_get_vfd_max_open_fds() AS max_open_fds, + CASE + WHEN ((pg_stat_get_vfd_hits() + pg_stat_get_vfd_misses()) = 0) THEN NULL::double precision + ELSE ((pg_stat_get_vfd_hits())::double precision / ((pg_stat_get_vfd_hits() + pg_stat_get_vfd_misses()))::double precision) + END AS hit_ratio, + pg_stat_get_vfd_stat_reset_time() AS stats_reset; pg_stat_wal| SELECT wal_records, wal_fpi, wal_bytes, diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index 8b15471248b..64f9cbc6e0c 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -128,7 +128,8 @@ SELECT id, name, fixed_amount, 12 | lock | t | f | t 13 | slru | t | f | t 14 | wal | t | f | t -(14 rows) + 15 | vfdcache | t | f | t +(15 rows) -- ensure that both seqscan and indexscan plans are allowed SET enable_seqscan TO on; @@ -1263,10 +1264,41 @@ SELECT stats_reset > :'wal_reset_ts'::timestamptz FROM pg_stat_wal; t (1 row) +-- Test that reset_shared with vfdcache specified as the stats type works +SELECT stats_reset AS vfdcache_reset_ts FROM pg_stat_vfdcache \gset +SELECT pg_stat_reset_shared('vfdcache'); + pg_stat_reset_shared +---------------------- + +(1 row) + +SELECT stats_reset > :'vfdcache_reset_ts'::timestamptz FROM pg_stat_vfdcache; + ?column? +---------- + t +(1 row) + +-- Test that VFD cache hits and misses are tracked after file access +SELECT hits + misses AS vfd_accesses_before FROM pg_stat_vfdcache \gset +CREATE TEMP TABLE test_vfd_activity (i int); +INSERT INTO test_vfd_activity SELECT generate_series(1, 100); +DROP TABLE test_vfd_activity; +SELECT pg_stat_force_next_flush(); + pg_stat_force_next_flush +-------------------------- + +(1 row) + +SELECT (hits + misses) > :vfd_accesses_before FROM pg_stat_vfdcache; + ?column? +---------- + t +(1 row) + -- Test error case for reset_shared with unknown stats type SELECT pg_stat_reset_shared('unknown'); ERROR: unrecognized reset target: "unknown" -HINT: Target must be "archiver", "bgwriter", "checkpointer", "io", "lock", "recovery_prefetch", "slru", or "wal". +HINT: Target must be "archiver", "bgwriter", "checkpointer", "io", "lock", "recovery_prefetch", "slru", "vfdcache", or "wal". -- Test that reset works for pg_stat_database and pg_stat_database_conflicts -- Since pg_stat_database stats_reset starts out as NULL, reset it once first so that we -- have a baseline for comparison. The same for pg_stat_database_conflicts as it shares diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index 674637e172b..f460b2125b3 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -581,6 +581,19 @@ SELECT stats_reset AS wal_reset_ts FROM pg_stat_wal \gset SELECT pg_stat_reset_shared('wal'); SELECT stats_reset > :'wal_reset_ts'::timestamptz FROM pg_stat_wal; +-- Test that reset_shared with vfdcache specified as the stats type works +SELECT stats_reset AS vfdcache_reset_ts FROM pg_stat_vfdcache \gset +SELECT pg_stat_reset_shared('vfdcache'); +SELECT stats_reset > :'vfdcache_reset_ts'::timestamptz FROM pg_stat_vfdcache; + +-- Test that VFD cache hits and misses are tracked after file access +SELECT hits + misses AS vfd_accesses_before FROM pg_stat_vfdcache \gset +CREATE TEMP TABLE test_vfd_activity (i int); +INSERT INTO test_vfd_activity SELECT generate_series(1, 100); +DROP TABLE test_vfd_activity; +SELECT pg_stat_force_next_flush(); +SELECT (hits + misses) > :vfd_accesses_before FROM pg_stat_vfdcache; + -- Test error case for reset_shared with unknown stats type SELECT pg_stat_reset_shared('unknown'); -- 2.34.1
From 5c91a9d7068b7186ff889ae914603d9ce259339a Mon Sep 17 00:00:00 2001 From: David Geier <[email protected]> Date: Tue, 8 Sep 2026 18:22:33 +0200 Subject: [PATCH v7 1/3] Vfd cache uses memory context instead of malloc --- src/backend/storage/file/fd.c | 54 +++++++++++++++-------------------- 1 file changed, 23 insertions(+), 31 deletions(-) diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index 190c9974494..f0fe48f60dc 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -99,6 +99,7 @@ #include "storage/ipc.h" #include "utils/guc.h" #include "utils/guc_hooks.h" +#include "utils/memutils.h" #include "utils/resowner.h" #include "utils/varlena.h" #include "utils/wait_event.h" @@ -207,7 +208,7 @@ typedef struct vfd File lruLessRecently; pgoff_t fileSize; /* current size of file (0 if not temporary) */ char *fileName; /* name of file, or NULL for unused VFD */ - /* NB: fileName is malloc'd, and must be free'd when closing the VFD */ + /* NB: fileName is allocated in VfdCxt, and must be pfree'd when closing the VFD */ int fileFlags; /* open(2) flags for (re)opening the file */ mode_t fileMode; /* mode to pass to open(2) */ } Vfd; @@ -217,6 +218,7 @@ typedef struct vfd * needed. 'File' values are indexes into this array. * Note that VfdCache[0] is not a usable VFD, just a list header. */ +static MemoryContext VfdCxt; static Vfd *VfdCache; static Size SizeVfdCache = 0; @@ -905,12 +907,13 @@ InitFileAccess(void) { Assert(SizeVfdCache == 0); /* call me only once */ + if (VfdCxt == NULL) + VfdCxt = AllocSetContextCreate(TopMemoryContext, + "Vfd cache context", + ALLOCSET_DEFAULT_SIZES); + /* initialize cache header entry */ - VfdCache = (Vfd *) malloc(sizeof(Vfd)); - if (VfdCache == NULL) - ereport(FATAL, - (errcode(ERRCODE_OUT_OF_MEMORY), - errmsg("out of memory"))); + VfdCache = MemoryContextAlloc(VfdCxt, sizeof(Vfd)); MemSet(&(VfdCache[0]), 0, sizeof(Vfd)); VfdCache->fd = VFD_CLOSED; @@ -1422,13 +1425,9 @@ AllocateVfd(void) newCacheSize = 32; /* - * Be careful not to clobber VfdCache ptr if realloc fails. + * Be careful not to clobber VfdCache ptr if allocation fails. */ - newVfdCache = (Vfd *) realloc(VfdCache, sizeof(Vfd) * newCacheSize); - if (newVfdCache == NULL) - ereport(ERROR, - (errcode(ERRCODE_OUT_OF_MEMORY), - errmsg("out of memory"))); + newVfdCache = repalloc_array(VfdCache, Vfd, newCacheSize); VfdCache = newVfdCache; /* @@ -1466,7 +1465,7 @@ FreeVfd(File file) if (vfdP->fileName != NULL) { - free(vfdP->fileName); + pfree(vfdP->fileName); vfdP->fileName = NULL; } vfdP->fdstate = 0x0; @@ -1582,14 +1581,7 @@ PathNameOpenFilePerm(const char *fileName, int fileFlags, mode_t fileMode) DO_DB(elog(LOG, "PathNameOpenFilePerm: %s %x %o", fileName, fileFlags, fileMode)); - /* - * We need a malloc'd copy of the file name; fail cleanly if no room. - */ - fnamecopy = strdup(fileName); - if (fnamecopy == NULL) - ereport(ERROR, - (errcode(ERRCODE_OUT_OF_MEMORY), - errmsg("out of memory"))); + fnamecopy = MemoryContextStrdup(VfdCxt, fileName); file = AllocateVfd(); vfdP = &VfdCache[file]; @@ -1612,7 +1604,7 @@ PathNameOpenFilePerm(const char *fileName, int fileFlags, mode_t fileMode) int save_errno = errno; FreeVfd(file); - free(fnamecopy); + pfree(fnamecopy); errno = save_errno; return -1; } @@ -2555,6 +2547,11 @@ reserveAllocatedDesc(void) AllocateDesc *newDescs; int newMax; + if (VfdCxt == NULL) + VfdCxt = AllocSetContextCreate(TopMemoryContext, + "Vfd cache context", + ALLOCSET_DEFAULT_SIZES); + /* Quick out if array already has a free slot. */ if (numAllocatedDescs < maxAllocatedDescs) return true; @@ -2568,12 +2565,8 @@ reserveAllocatedDesc(void) if (allocatedDescs == NULL) { newMax = FD_MINFREE / 3; - newDescs = (AllocateDesc *) malloc(newMax * sizeof(AllocateDesc)); - /* Out of memory already? Treat as fatal error. */ - if (newDescs == NULL) - ereport(ERROR, - (errcode(ERRCODE_OUT_OF_MEMORY), - errmsg("out of memory"))); + newDescs = MemoryContextAlloc(VfdCxt, + newMax * sizeof(AllocateDesc)); allocatedDescs = newDescs; maxAllocatedDescs = newMax; return true; @@ -2593,9 +2586,8 @@ reserveAllocatedDesc(void) newMax = max_safe_fds / 3; if (newMax > maxAllocatedDescs) { - newDescs = (AllocateDesc *) realloc(allocatedDescs, - newMax * sizeof(AllocateDesc)); - /* Treat out-of-memory as a non-fatal error. */ + newDescs = repalloc_array_extended(allocatedDescs, AllocateDesc, + newMax, MCXT_ALLOC_NO_OOM); if (newDescs == NULL) return false; allocatedDescs = newDescs; -- 2.34.1
From 64ffd5ba130ccbdb354527719e75768335bb3b2e Mon Sep 17 00:00:00 2001 From: AyoubKAZ <[email protected]> Date: Wed, 9 Sep 2026 03:54:52 +0200 Subject: [PATCH v7 3/3] Add VFD cache footprint metrics to pg_stat_vfdcache Extend pg_stat_vfdcache with three additional columns: open_entries number of currently open file descriptors inside VFD cache allocated_entries total number of allocated slots in the cache cache_bytes total memory allocated by the VFD cache memory context These are live gauges. Each backend publishes its current VFD cache footprint into backend shared stats during backend stats flush, and SQL accessors sum those backend shared stats entries to produce cluster-wide totals. The memory footprint is measured directly from the dedicated VFD memory context using MemoryContextMemAllocated(), avoiding manual bookkeeping. --- doc/src/sgml/monitoring.sgml | 32 +++++++ src/backend/catalog/system_views.sql | 3 + src/backend/storage/file/fd.c | 36 ++++++++ src/backend/utils/activity/pgstat_backend.c | 52 ++++++++++++ src/backend/utils/adt/pgstatfuncs.c | 93 +++++++++++++++++++++ src/include/catalog/pg_proc.dat | 18 ++++ src/include/pgstat.h | 15 ++++ src/include/storage/fd.h | 3 + src/include/utils/pgstat_internal.h | 6 +- src/test/regress/expected/rules.out | 3 + src/test/regress/expected/stats.out | 10 +++ src/test/regress/sql/stats.sql | 6 ++ 12 files changed, 276 insertions(+), 1 deletion(-) diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index f7c41c2d960..a2a618bd945 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -3963,6 +3963,38 @@ description | Waiting for a newly initialized WAL file to reach durable storage </para></entry> </row> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>open_entries</structfield> <type>bigint</type> + </para> + <para> + Sum of VFD slots currently holding an open kernel file descriptor + across all active backends (equivalent to summing each backend's + <literal>nfile</literal> counter) + </para></entry> + </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>allocated_entries</structfield> <type>bigint</type> + </para> + <para> + Sum of allocated VFD cache entries across all active backends, + including entries whose file descriptor has been closed by the LRU + eviction policy but whose slot has not yet been freed + </para></entry> + </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>cache_bytes</structfield> <type>bigint</type> + </para> + <para> + Sum of memory allocated by the VFD cache memory context across all active backends, + in bytes + </para></entry> + </row> + <row> <entry role="catalog_table_entry"><para role="column_definition"> <structfield>max_open_fds</structfield> <type>integer</type> diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 844fea1391f..2b5a92b1e70 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -1573,6 +1573,9 @@ CREATE VIEW pg_stat_vfdcache AS SELECT pg_stat_get_vfd_hits() AS hits, pg_stat_get_vfd_misses() AS misses, + pg_stat_get_vfd_cache_open_entries() AS open_entries, + pg_stat_get_vfd_cache_allocated_entries() AS allocated_entries, + pg_stat_get_vfd_cache_bytes() AS cache_bytes, pg_stat_get_vfd_max_open_fds() AS max_open_fds, CASE WHEN pg_stat_get_vfd_hits() + pg_stat_get_vfd_misses() = 0 diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c index d7ecaf5e6c5..e05197095cf 100644 --- a/src/backend/storage/file/fd.c +++ b/src/backend/storage/file/fd.c @@ -1511,6 +1511,42 @@ FileAccess(File file) return 0; } +/* + * Return the number of VFD slots currently holding an open file descriptor. + * This is the nfile counter, which tracks FDs actually open in the kernel. + */ +uint64 +GetVfdCacheOpenEntries(void) +{ + return (uint64) nfile; +} + +/* + * Return the number of currently allocated VFD entries for this backend, + * excluding slot 0 which is used as freelist/LRU header. + */ +uint64 +GetVfdCacheAllocatedEntries(void) +{ + if (SizeVfdCache == 0) + return 0; + + return (uint64) (SizeVfdCache - 1); +} + +/* + * Return the current memory footprint of the VFD cache for this backend, + * measured as the total memory allocated by its dedicated memory context. + */ +uint64 +GetVfdCacheBytes(void) +{ + if (VfdCxt == NULL) + return 0; + + return (uint64) MemoryContextMemAllocated(VfdCxt, false); +} + /* * Called whenever a temporary file is deleted to report its size. */ diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c index b736b2ccc6f..a493b559911 100644 --- a/src/backend/utils/activity/pgstat_backend.c +++ b/src/backend/utils/activity/pgstat_backend.c @@ -27,6 +27,7 @@ #include "access/xlog.h" #include "executor/instrument.h" #include "storage/bufmgr.h" +#include "storage/fd.h" #include "storage/proc.h" #include "storage/procarray.h" #include "utils/memutils.h" @@ -49,6 +50,12 @@ static bool backend_has_lockstats = false; */ static WalUsage prevBackendWalUsage; +/* + * Last per-backend VFD cache gauges published to backend stats. + * Used to detect whether anything changed since the last flush. + */ +static PgStat_BackendVfdCacheStats prevBackendVfdCacheStats; + /* * Utility routines to report I/O stats for backends, kept here to avoid * exposing PendingBackendStats to the outside world. @@ -253,6 +260,21 @@ pgstat_backend_wal_have_pending(void) return (pgWalUsage.wal_records != prevBackendWalUsage.wal_records); } +/* + * Determine whether VFD cache gauges have changed since the last flush. + */ +static inline bool +pgstat_backend_vfdcache_have_pending(void) +{ + PgStat_Counter cur_open = (PgStat_Counter) GetVfdCacheOpenEntries(); + PgStat_Counter cur_alloc = (PgStat_Counter) GetVfdCacheAllocatedEntries(); + PgStat_Counter cur_bytes = (PgStat_Counter) GetVfdCacheBytes(); + + return cur_open != prevBackendVfdCacheStats.open_entries || + cur_alloc != prevBackendVfdCacheStats.allocated_entries || + cur_bytes != prevBackendVfdCacheStats.cache_bytes; +} + /* * Flush out locally pending backend WAL statistics. Locking is managed * by the caller. @@ -326,6 +348,27 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref) backend_has_lockstats = false; } +/* + * Flush out locally pending backend VFD cache gauges. Locking is managed + * by the caller. No need to check for pending data here; the caller does + * that before acquiring the lock. + */ +static void +pgstat_flush_backend_entry_vfdcache(PgStat_EntryRef *entry_ref) +{ + PgStatShared_Backend *shbackendent; + PgStat_BackendVfdCacheStats *bktype_shstats; + + shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats; + bktype_shstats = &shbackendent->stats.vfdcache_stats; + + bktype_shstats->open_entries = (PgStat_Counter) GetVfdCacheOpenEntries(); + bktype_shstats->allocated_entries = (PgStat_Counter) GetVfdCacheAllocatedEntries(); + bktype_shstats->cache_bytes = (PgStat_Counter) GetVfdCacheBytes(); + + prevBackendVfdCacheStats = *bktype_shstats; +} + /* * Flush out locally pending backend statistics * @@ -354,6 +397,11 @@ pgstat_flush_backend(bool nowait, uint32 flags) if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats) has_pending_data = true; + /* Some VFD cache data pending? */ + if ((flags & PGSTAT_BACKEND_FLUSH_VFDCACHE) && + pgstat_backend_vfdcache_have_pending()) + has_pending_data = true; + if (!has_pending_data) return false; @@ -372,6 +420,9 @@ pgstat_flush_backend(bool nowait, uint32 flags) if (flags & PGSTAT_BACKEND_FLUSH_LOCK) pgstat_flush_backend_entry_lock(entry_ref); + if (flags & PGSTAT_BACKEND_FLUSH_VFDCACHE) + pgstat_flush_backend_entry_vfdcache(entry_ref); + pgstat_unlock_entry(entry_ref); return false; @@ -411,6 +462,7 @@ pgstat_create_backend(ProcNumber procnum) MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending)); backend_has_iostats = false; backend_has_lockstats = false; + MemSet(&prevBackendVfdCacheStats, 0, sizeof(prevBackendVfdCacheStats)); /* * Initialize prevBackendWalUsage with pgWalUsage so that diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 3499dd40700..aa6b16f38c4 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -15,6 +15,7 @@ #include "postgres.h" #include "access/htup_details.h" +#include "access/xact.h" #include "access/xlog.h" #include "access/xlogprefetcher.h" #include "catalog/catalog.h" @@ -1411,6 +1412,98 @@ pg_stat_get_vfd_misses(PG_FUNCTION_ARGS) PG_RETURN_INT64(pgstat_fetch_stat_vfdcache()->vfd_misses); } +/* + * Sum per-backend VFD cache gauges across all currently active backends. + * Results are cached for the duration of the current SQL statement to + * avoid redundant scans when the view calls multiple getter functions. + */ +static void +pgstat_get_vfd_backend_sums(PgStat_Counter *open_sum, + PgStat_Counter *alloc_sum, + PgStat_Counter *bytes_sum) +{ + static TimestampTz cached_stmt_start_ts = 0; + static PgStat_Counter cached_open_sum = 0; + static PgStat_Counter cached_alloc_sum = 0; + static PgStat_Counter cached_bytes_sum = 0; + TimestampTz stmt_start_ts = GetCurrentStatementStartTimestamp(); + int num_backends = pgstat_fetch_stat_numbackends(); + + if (cached_stmt_start_ts == stmt_start_ts) + { + *open_sum = cached_open_sum; + *alloc_sum = cached_alloc_sum; + *bytes_sum = cached_bytes_sum; + return; + } + + *open_sum = 0; + *alloc_sum = 0; + *bytes_sum = 0; + + for (int curr_backend = 1; curr_backend <= num_backends; curr_backend++) + { + LocalPgBackendStatus *local_beentry; + PgBackendStatus *beentry; + PgStat_Backend *backend_stats; + + local_beentry = pgstat_get_local_beentry_by_index(curr_backend); + beentry = &local_beentry->backendStatus; + + if (!pgstat_tracks_backend_bktype(beentry->st_backendType)) + continue; + + backend_stats = pgstat_fetch_stat_backend(local_beentry->proc_number); + if (!backend_stats) + continue; + + *open_sum += backend_stats->vfdcache_stats.open_entries; + *alloc_sum += backend_stats->vfdcache_stats.allocated_entries; + *bytes_sum += backend_stats->vfdcache_stats.cache_bytes; + } + + cached_open_sum = *open_sum; + cached_alloc_sum = *alloc_sum; + cached_bytes_sum = *bytes_sum; + cached_stmt_start_ts = stmt_start_ts; +} + +Datum +pg_stat_get_vfd_cache_open_entries(PG_FUNCTION_ARGS) +{ + PgStat_Counter open_sum; + PgStat_Counter alloc_sum; + PgStat_Counter bytes_sum; + + pgstat_get_vfd_backend_sums(&open_sum, &alloc_sum, &bytes_sum); + + PG_RETURN_INT64(open_sum); +} + +Datum +pg_stat_get_vfd_cache_allocated_entries(PG_FUNCTION_ARGS) +{ + PgStat_Counter open_sum; + PgStat_Counter alloc_sum; + PgStat_Counter bytes_sum; + + pgstat_get_vfd_backend_sums(&open_sum, &alloc_sum, &bytes_sum); + + PG_RETURN_INT64(alloc_sum); +} + +Datum +pg_stat_get_vfd_cache_bytes(PG_FUNCTION_ARGS) +{ + PgStat_Counter open_sum; + PgStat_Counter alloc_sum; + PgStat_Counter bytes_sum; + + pgstat_get_vfd_backend_sums(&open_sum, &alloc_sum, &bytes_sum); + + PG_RETURN_INT64(bytes_sum); +} + Datum pg_stat_get_vfd_max_open_fds(PG_FUNCTION_ARGS) { diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index d5b5f08bf5c..f7a4ef14d7a 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -12802,5 +12802,23 @@ provolatile => 'v', proparallel => 'r', prorettype => 'int4', proargtypes => '', prosrc => 'pg_stat_get_vfd_max_open_fds' }, +{ oid => '9562', + descr => 'statistics: number of VFD slots with an open file descriptor across backends', + proname => 'pg_stat_get_vfd_cache_open_entries', + provolatile => 'v', proparallel => 'r', + prorettype => 'int8', proargtypes => '', + prosrc => 'pg_stat_get_vfd_cache_open_entries' }, +{ oid => '9563', + descr => 'statistics: total number of allocated VFD cache entries across backends', + proname => 'pg_stat_get_vfd_cache_allocated_entries', + provolatile => 'v', proparallel => 'r', + prorettype => 'int8', proargtypes => '', + prosrc => 'pg_stat_get_vfd_cache_allocated_entries' }, +{ oid => '9567', + descr => 'statistics: total memory footprint of VFD cache across backends in bytes', + proname => 'pg_stat_get_vfd_cache_bytes', + provolatile => 'v', proparallel => 'r', + prorettype => 'int8', proargtypes => '', + prosrc => 'pg_stat_get_vfd_cache_bytes' }, ] diff --git a/src/include/pgstat.h b/src/include/pgstat.h index e8395664df2..73cd735de06 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -319,6 +319,20 @@ typedef struct PgStat_VfdCacheStats TimestampTz stat_reset_timestamp; } PgStat_VfdCacheStats; +/* ------- + * PgStat_BackendVfdCacheStats Per-backend VFD cache gauges + * + * These are live gauges (not cumulative counters) snapshotted from each + * backend's fd.c variables during stats flush. + * ------- + */ +typedef struct PgStat_BackendVfdCacheStats +{ + PgStat_Counter open_entries; /* FDs currently open (= nfile) */ + PgStat_Counter allocated_entries; /* total allocated VFD slots */ + PgStat_Counter cache_bytes; /* memory footprint in bytes */ +} PgStat_BackendVfdCacheStats; + /* * Types related to counting IO operations */ @@ -584,6 +598,7 @@ typedef struct PgStat_Backend PgStat_BktypeIO io_stats; PgStat_WalCounters wal_counters; PgStat_PendingLock lock_stats; + PgStat_BackendVfdCacheStats vfdcache_stats; } PgStat_Backend; /* --------- diff --git a/src/include/storage/fd.h b/src/include/storage/fd.h index 8ac466fd346..672e18ca19e 100644 --- a/src/include/storage/fd.h +++ b/src/include/storage/fd.h @@ -149,6 +149,9 @@ extern char *FilePathName(File file); extern int FileGetRawDesc(File file); extern int FileGetRawFlags(File file); extern mode_t FileGetRawMode(File file); +extern uint64 GetVfdCacheOpenEntries(void); +extern uint64 GetVfdCacheAllocatedEntries(void); +extern uint64 GetVfdCacheBytes(void); /* Operations used for sharing named temporary files */ extern File PathNameCreateTemporaryFile(const char *path, bool error_on_failure); diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h index e24a1c84c9f..8b08990cc1b 100644 --- a/src/include/utils/pgstat_internal.h +++ b/src/include/utils/pgstat_internal.h @@ -724,7 +724,11 @@ extern void pgstat_archiver_snapshot_cb(void); #define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */ #define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */ #define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */ -#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK) +#define PGSTAT_BACKEND_FLUSH_VFDCACHE (1 << 3) /* Flush VFD cache gauges */ +#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | \ + PGSTAT_BACKEND_FLUSH_WAL | \ + PGSTAT_BACKEND_FLUSH_LOCK | \ + PGSTAT_BACKEND_FLUSH_VFDCACHE) extern bool pgstat_flush_backend(bool nowait, uint32 flags); extern bool pgstat_backend_flush_cb(bool nowait); diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index cdf21d82924..3d38a1bf6b8 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -2428,6 +2428,9 @@ pg_stat_user_tables| SELECT relid, WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text)); pg_stat_vfdcache| SELECT pg_stat_get_vfd_hits() AS hits, pg_stat_get_vfd_misses() AS misses, + pg_stat_get_vfd_cache_open_entries() AS open_entries, + pg_stat_get_vfd_cache_allocated_entries() AS allocated_entries, + pg_stat_get_vfd_cache_bytes() AS cache_bytes, pg_stat_get_vfd_max_open_fds() AS max_open_fds, CASE WHEN ((pg_stat_get_vfd_hits() + pg_stat_get_vfd_misses()) = 0) THEN NULL::double precision diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index 64f9cbc6e0c..21deb48c0b0 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -1295,6 +1295,16 @@ SELECT (hits + misses) > :vfd_accesses_before FROM pg_stat_vfdcache; t (1 row) +-- Test that VFD cache footprint metrics are non-negative and consistent +SELECT open_entries <= allocated_entries, + allocated_entries > 0, + cache_bytes > 0 +FROM pg_stat_vfdcache; + ?column? | ?column? | ?column? +----------+----------+---------- + t | t | t +(1 row) + -- Test error case for reset_shared with unknown stats type SELECT pg_stat_reset_shared('unknown'); ERROR: unrecognized reset target: "unknown" diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index f460b2125b3..2f745ef78bb 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -594,6 +594,12 @@ DROP TABLE test_vfd_activity; SELECT pg_stat_force_next_flush(); SELECT (hits + misses) > :vfd_accesses_before FROM pg_stat_vfdcache; +-- Test that VFD cache footprint metrics are non-negative and consistent +SELECT open_entries <= allocated_entries, + allocated_entries > 0, + cache_bytes > 0 +FROM pg_stat_vfdcache; + -- Test error case for reset_shared with unknown stats type SELECT pg_stat_reset_shared('unknown'); -- 2.34.1
