On Tue, Mar 7, 2023 at 11:17 AM Julien Rouhaud <rjuju...@gmail.com> wrote:
>
> On Tue, Mar 07, 2023 at 01:56:24PM +0900, Michael Paquier wrote:
> > On Tue, Mar 07, 2023 at 12:42:20PM +0800, Julien Rouhaud wrote:
> > > ah right I should have checked. but the same ABI compatibility concern
> > > still exists for version 1.0 of the extension.
> >
> > Yes, we'd better make sure that the past code is able to run, at
> > least. Now I am not really convinced that we have the need to enforce
> > an error with the new code even if 1.0 is still installed,
>
> So keep this "deprecated" C function working, as it would only be a few lines
> of code?
>
> > so as it is
> > possible to remove all the traces of the code that triggers errors if
> > an end LSN is higher than the current insert LSN for primaries or
> > replayed LSN for standbys.
>
> +1 for that
I understand that we want to keep till_end_of_wal functions defined
around in .c file so that if someone does CREATE EXTENSION
pg_walinspect WITH VERSION '1.0'; on the latest extension shared
library (with 1.1 version), the till_end_of_wal functions should work
for them.
Also, I noticed that there's some improvement needed for the input
validations, especially for the end_lsn.
Here I'm with the v3 patch addressing the above comments. Please
review it further.
1. When start_lsn is NULL or invalid ('0/0'), emit an error. There was
a comment on the functions automatically determining start_lsn to be
the oldest WAL LSN. I'm not implementing this change now, as it
requires extra work to traverse the pg_wal directory. I'm planning to
do it in the next set of improvements where I'm planning to make the
functions timeline-aware, introduce functions for inspecting
wal_buffers and so on.
2. When end_lsn is NULL or invalid ('0/0') IOW end_lsn is not
specified, deduce end_lsn to be the current flush LSN when not in
recovery, current replayed LSN when in recovery. This is the main
change that avoids till_end_of_wal functions in version 1.1.
3. When end_lsn is specified but greater than or equal to the
start_lsn, return NULL. Given the above review comments on more errors
being reported, I chose to return NULL for better usability.
4. When end_lsn is specified but less than the start_lsn, get
info/stats up until end_lsn.
5. Retained pg_get_wal_records_info_till_end_of_wal and
pg_get_wal_stats_till_end_of_wal for backward compatibility.
6. Piggybacked these functions and behaviour under the new HEAD-only
extension version 1.1 introduced recently, instead of bumping to 1.2.
When PG16 is out, users will have 1.1 with all of these new
functionality.
7. Added tests to verify the extension update path in
oldextversions.sql similar to other extensions'. (suggested by Michael
Paquier).
8. Added a note in the pg_walinspect documentation about removal of
pg_get_wal_records_info_till_end_of_wal and
pg_get_wal_stats_till_end_of_wal in version 1.1 and how the other
functions can be used to achieve the same functionality and how these
till_end_of_wal functions can work if extension is installed
explicitly with version 1.0.
9. Refactored the tests according to the new behaviours.
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
From 9d63a01bc634bbb899ce9f4645bf677bcbaec998 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Wed, 8 Mar 2023 07:39:36 +0000
Subject: [PATCH v3] Rework pg_walinspect functions
---
contrib/pg_walinspect/Makefile | 2 +-
.../pg_walinspect/expected/oldextversions.out | 64 +++++
.../pg_walinspect/expected/pg_walinspect.out | 87 ++++++-
contrib/pg_walinspect/meson.build | 1 +
.../pg_walinspect/pg_walinspect--1.0--1.1.sql | 57 ++++-
contrib/pg_walinspect/pg_walinspect.c | 235 +++++++++++-------
contrib/pg_walinspect/sql/oldextversions.sql | 27 ++
contrib/pg_walinspect/sql/pg_walinspect.sql | 51 +++-
doc/src/sgml/pgwalinspect.sgml | 71 ++----
9 files changed, 442 insertions(+), 153 deletions(-)
create mode 100644 contrib/pg_walinspect/expected/oldextversions.out
create mode 100644 contrib/pg_walinspect/sql/oldextversions.sql
diff --git a/contrib/pg_walinspect/Makefile b/contrib/pg_walinspect/Makefile
index 7033878a79..22090f7716 100644
--- a/contrib/pg_walinspect/Makefile
+++ b/contrib/pg_walinspect/Makefile
@@ -9,7 +9,7 @@ PGFILEDESC = "pg_walinspect - functions to inspect contents of PostgreSQL Write-
EXTENSION = pg_walinspect
DATA = pg_walinspect--1.0.sql pg_walinspect--1.0--1.1.sql
-REGRESS = pg_walinspect
+REGRESS = pg_walinspect oldextversions
REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_walinspect/walinspect.conf
diff --git a/contrib/pg_walinspect/expected/oldextversions.out b/contrib/pg_walinspect/expected/oldextversions.out
new file mode 100644
index 0000000000..c3ebcc2b49
--- /dev/null
+++ b/contrib/pg_walinspect/expected/oldextversions.out
@@ -0,0 +1,64 @@
+-- test old extension version entry points
+DROP EXTENSION pg_walinspect;
+CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+-- Move to new version 1.1
+ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
+-- New function introduced in 1.1
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+ pg_get_functiondef
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT lsn pg_lsn, OUT reltablespace oid, OUT reldatabase oid, OUT relfilenode oid, OUT relblocknumber bigint, OUT forkname text, OUT fpi bytea)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_fpi_info$function$ +
+
+(1 row)
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_records_info'::regproc);
+ pg_get_functiondef
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT start_lsn pg_lsn, OUT end_lsn pg_lsn, OUT prev_lsn pg_lsn, OUT xid xid, OUT resource_manager text, OUT record_type text, OUT record_length integer, OUT main_data_length integer, OUT fpi_length integer, OUT description text, OUT block_ref text)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_records_info$function$ +
+
+(1 row)
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_of_wal'::regproc);
+ERROR: function "pg_get_wal_records_info_till_end_of_wal" does not exist
+LINE 1: SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_...
+ ^
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_stats'::regproc);
+ pg_get_functiondef
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, per_record boolean DEFAULT false, OUT "resource_manager/record_type" text, OUT count bigint, OUT count_percentage double precision, OUT record_size bigint, OUT record_size_percentage double precision, OUT fpi_size bigint, OUT fpi_size_percentage double precision, OUT combined_size bigint, OUT combined_size_percentage double precision)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_stats$function$ +
+
+(1 row)
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'::regproc);
+ERROR: function "pg_get_wal_stats_till_end_of_wal" does not exist
+LINE 1: SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'...
+ ^
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+ pg_get_functiondef
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ CREATE OR REPLACE FUNCTION public.pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL::pg_lsn, OUT lsn pg_lsn, OUT reltablespace oid, OUT reldatabase oid, OUT relfilenode oid, OUT relblocknumber bigint, OUT forkname text, OUT fpi bytea)+
+ RETURNS SETOF record +
+ LANGUAGE c +
+ PARALLEL SAFE +
+ AS '$libdir/pg_walinspect', $function$pg_get_wal_fpi_info$function$ +
+
+(1 row)
+
+DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index 9bcb05354e..71dcd8cfbb 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -14,38 +14,99 @@ INSERT INTO sample_tbl SELECT * FROM generate_series(3, 4);
-- ===================================================================
-- Tests for input validation
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
-ERROR: WAL start LSN must be less than end LSN
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
-ERROR: WAL start LSN must be less than end LSN
+-- Invalid input LSNs
+SELECT * FROM pg_get_wal_record_info('0/0'); -- ERROR
+ERROR: invalid input LSN
+SELECT * FROM pg_get_wal_records_info('0/0'); -- ERROR
+ERROR: invalid start LSN
+SELECT * FROM pg_get_wal_stats('0/0'); -- ERROR
+ERROR: invalid start LSN
+SELECT * FROM pg_get_wal_fpi_info('0/0'); -- ERROR
+ERROR: invalid start LSN
+-- Start LSN NULL
+SELECT * FROM pg_get_wal_records_info(NULL); -- ERROR
+ERROR: invalid start LSN
+SELECT * FROM pg_get_wal_stats(NULL); -- ERROR
+ERROR: invalid start LSN
+SELECT * FROM pg_get_wal_fpi_info(NULL); -- ERROR
+ERROR: invalid start LSN
+-- Start LSN > End LSN
+SELECT * FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- NULL
+ start_lsn | end_lsn | prev_lsn | xid | resource_manager | record_type | record_length | main_data_length | fpi_length | description | block_ref
+-----------+---------+----------+-----+------------------+-------------+---------------+------------------+------------+-------------+-----------
+ | | | | | | | | | |
+(1 row)
+
+SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- NULL
+ resource_manager/record_type | count | count_percentage | record_size | record_size_percentage | fpi_size | fpi_size_percentage | combined_size | combined_size_percentage
+------------------------------+-------+------------------+-------------+------------------------+----------+---------------------+---------------+--------------------------
+ | | | | | | | |
+(1 row)
+
+SELECT * FROM pg_get_wal_fpi_info(:'wal_lsn2', :'wal_lsn1'); -- NULL
+ lsn | reltablespace | reldatabase | relfilenode | relblocknumber | forkname | fpi
+-----+---------------+-------------+-------------+----------------+----------+-----
+ | | | | | |
+(1 row)
+
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+ ok
+----
+ t
+(1 row)
+
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', '0/0');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', NULL);
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
ok
----
t
(1 row)
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1');
+ ok
+----
+ t
+(1 row)
+
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', '0/0');
+ ok
+----
+ t
+(1 row)
+
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', NULL);
ok
----
t
@@ -90,6 +151,14 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3', :'wal_lsn4')
t
(1 row)
+-- Check till end of WAL if we get FPI from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3')
+ WHERE relfilenode = :'sample_tbl_oid';
+ ok
+----
+ t
+(1 row)
+
-- ===================================================================
-- Tests for permissions
-- ===================================================================
diff --git a/contrib/pg_walinspect/meson.build b/contrib/pg_walinspect/meson.build
index bf7b79b1b7..80059f6119 100644
--- a/contrib/pg_walinspect/meson.build
+++ b/contrib/pg_walinspect/meson.build
@@ -30,6 +30,7 @@ tests += {
'regress': {
'sql': [
'pg_walinspect',
+ 'oldextversions',
],
# Disabled because these tests require "wal_level=replica", which
# some runningcheck users do not have (e.g. buildfarm clients).
diff --git a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
index 1e9e1e6115..219a088eb9 100644
--- a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
+++ b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
@@ -3,11 +3,64 @@
-- complain if script is sourced in psql, rather than via ALTER EXTENSION
\echo Use "ALTER EXTENSION pg_walinspect UPDATE TO '1.1'" to load this file. \quit
+/* Drop the unneeded functions and redefine needed ones */
+DROP FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn);
+DROP FUNCTION pg_get_wal_records_info_till_end_of_wal(pg_lsn);
+DROP FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean);
+DROP FUNCTION pg_get_wal_stats_till_end_of_wal(pg_lsn, boolean);
+
+--
+-- pg_get_wal_records_info()
+--
+CREATE FUNCTION pg_get_wal_records_info(IN start_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
+ OUT start_lsn pg_lsn,
+ OUT end_lsn pg_lsn,
+ OUT prev_lsn pg_lsn,
+ OUT xid xid,
+ OUT resource_manager text,
+ OUT record_type text,
+ OUT record_length int4,
+ OUT main_data_length int4,
+ OUT fpi_length int4,
+ OUT description text,
+ OUT block_ref text
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_get_wal_records_info'
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn) TO pg_read_server_files;
+
+--
+-- pg_get_wal_stats()
+--
+CREATE FUNCTION pg_get_wal_stats(IN start_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
+ IN per_record boolean DEFAULT false,
+ OUT "resource_manager/record_type" text,
+ OUT count int8,
+ OUT count_percentage float8,
+ OUT record_size int8,
+ OUT record_size_percentage float8,
+ OUT fpi_size int8,
+ OUT fpi_size_percentage float8,
+ OUT combined_size int8,
+ OUT combined_size_percentage float8
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_get_wal_stats'
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
+
+REVOKE EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean) TO pg_read_server_files;
+
--
-- pg_get_wal_fpi_info()
--
CREATE FUNCTION pg_get_wal_fpi_info(IN start_lsn pg_lsn,
- IN end_lsn pg_lsn,
+ IN end_lsn pg_lsn DEFAULT NULL,
OUT lsn pg_lsn,
OUT reltablespace oid,
OUT reldatabase oid,
@@ -18,7 +71,7 @@ CREATE FUNCTION pg_get_wal_fpi_info(IN start_lsn pg_lsn,
)
RETURNS SETOF record
AS 'MODULE_PATHNAME', 'pg_get_wal_fpi_info'
-LANGUAGE C STRICT PARALLEL SAFE;
+LANGUAGE C CALLED ON NULL INPUT PARALLEL SAFE;
REVOKE EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn) FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c
index b7b0a805ee..4451e3e557 100644
--- a/contrib/pg_walinspect/pg_walinspect.c
+++ b/contrib/pg_walinspect/pg_walinspect.c
@@ -37,13 +37,13 @@ PG_FUNCTION_INFO_V1(pg_get_wal_records_info_till_end_of_wal);
PG_FUNCTION_INFO_V1(pg_get_wal_stats);
PG_FUNCTION_INFO_V1(pg_get_wal_stats_till_end_of_wal);
-static bool IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn);
+static XLogRecPtr GetCurrentLSN(void);
static XLogReaderState *InitXLogReaderState(XLogRecPtr lsn);
static XLogRecord *ReadNextXLogRecord(XLogReaderState *xlogreader);
static void GetWALRecordInfo(XLogReaderState *record, Datum *values,
bool *nulls, uint32 ncols);
-static XLogRecPtr ValidateInputLSNs(bool till_end_of_wal,
- XLogRecPtr start_lsn, XLogRecPtr end_lsn);
+static void GetInputLSNs(FunctionCallInfo fcinfo, XLogRecPtr *start_lsn,
+ XLogRecPtr *end_lsn);
static void GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
XLogRecPtr end_lsn);
static void GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo,
@@ -59,27 +59,25 @@ static void GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
static void GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
/*
- * Check if the given LSN is in future. Also, return the LSN up to which the
- * server has WAL.
+ * Return the LSN up to which the server has WAL.
*/
-static bool
-IsFutureLSN(XLogRecPtr lsn, XLogRecPtr *curr_lsn)
+static XLogRecPtr
+GetCurrentLSN(void)
{
+ XLogRecPtr curr_lsn;
+
/*
* We determine the current LSN of the server similar to how page_read
* callback read_local_xlog_page_no_wait does.
*/
if (!RecoveryInProgress())
- *curr_lsn = GetFlushRecPtr(NULL);
+ curr_lsn = GetFlushRecPtr(NULL);
else
- *curr_lsn = GetXLogReplayRecPtr(NULL);
+ curr_lsn = GetXLogReplayRecPtr(NULL);
- Assert(!XLogRecPtrIsInvalid(*curr_lsn));
+ Assert(!XLogRecPtrIsInvalid(curr_lsn));
- if (lsn >= *curr_lsn)
- return true;
-
- return false;
+ return curr_lsn;
}
/*
@@ -295,8 +293,14 @@ GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
* records between start and end LSNs. Decompression is applied to the
* blocks, if necessary.
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * This function determines end LSN if it is not specified or specified as NULL
+ * or invalid. In such cases, the end LSN is assigned with the last flushed LSN
+ * when not in recovery or the last replayed LSN when in recovery.
+ *
+ * This function emits an error if start LSN is invalid or in future i.e. LSN
+ * the database system doesn't know about.
+ *
+ * This function returns NULL, when start LSN is past the end LSN.
*/
Datum
pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
@@ -307,10 +311,14 @@ pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
MemoryContext old_cxt;
MemoryContext tmp_cxt;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn);
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
InitMaterializedSRF(fcinfo, 0);
@@ -345,8 +353,8 @@ pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
/*
* Get WAL record info.
*
- * This function emits an error if a future WAL LSN i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * This function emits an error if input LSN is invalid or in future i.e. LSN
+ * the database system doesn't know about.
*/
Datum
pg_get_wal_record_info(PG_FUNCTION_ARGS)
@@ -363,7 +371,14 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
lsn = PG_GETARG_LSN(0);
- if (IsFutureLSN(lsn, &curr_lsn))
+ if (XLogRecPtrIsInvalid(lsn))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid input LSN")));
+
+ curr_lsn = GetCurrentLSN();
+
+ if (lsn >= curr_lsn)
{
/*
* GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
@@ -402,44 +417,36 @@ pg_get_wal_record_info(PG_FUNCTION_ARGS)
}
/*
- * Validate the input LSNs and compute end LSN for till_end_of_wal versions.
+ * Get start LSN and get/deduce end LSN.
*/
-static XLogRecPtr
-ValidateInputLSNs(bool till_end_of_wal, XLogRecPtr start_lsn,
- XLogRecPtr end_lsn)
+static void
+GetInputLSNs(FunctionCallInfo fcinfo, XLogRecPtr *start_lsn,
+ XLogRecPtr *end_lsn)
{
XLogRecPtr curr_lsn;
- if (IsFutureLSN(start_lsn, &curr_lsn))
- {
- /*
- * GetFlushRecPtr or GetXLogReplayRecPtr gives "end+1" LSN of the last
- * record flushed or replayed respectively. But let's use the LSN up
- * to "end" in user facing message.
- */
+ if (PG_ARGISNULL(0) ||
+ ((*start_lsn = PG_GETARG_LSN(0)) == InvalidXLogRecPtr))
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future start LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
- }
-
- if (till_end_of_wal)
- end_lsn = curr_lsn;
+ errmsg("invalid start LSN")));
- if (end_lsn > curr_lsn)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("cannot accept future end LSN"),
- errdetail("Last known WAL LSN on the database system is at %X/%X.",
- LSN_FORMAT_ARGS(curr_lsn))));
+ curr_lsn = GetCurrentLSN();
- if (start_lsn >= end_lsn)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("WAL start LSN must be less than end LSN")));
+ if (PG_ARGISNULL(1))
+ *end_lsn = curr_lsn;
+ else
+ {
+ *end_lsn = PG_GETARG_LSN(1);
- return end_lsn;
+ /*
+ * Adjust end LSN to what the system knows at this point instead of
+ * raising errors for better usability of the functions.
+ */
+ if (XLogRecPtrIsInvalid(*end_lsn) ||
+ *end_lsn > curr_lsn)
+ *end_lsn = curr_lsn;
+ }
}
/*
@@ -494,8 +501,14 @@ GetWALRecordsInfo(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get info and data of all WAL records between start LSN and end LSN.
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * This function determines end LSN if it is not specified or specified as NULL
+ * or invalid. In such cases, the end LSN is assigned with the last flushed LSN
+ * when not in recovery or the last replayed LSN when in recovery.
+ *
+ * This function emits an error if start LSN is invalid or in future i.e. LSN
+ * the database system doesn't know about.
+ *
+ * This function returns NULL, when start LSN is past the end LSN.
*/
Datum
pg_get_wal_records_info(PG_FUNCTION_ARGS)
@@ -503,31 +516,14 @@ pg_get_wal_records_info(PG_FUNCTION_ARGS)
XLogRecPtr start_lsn;
XLogRecPtr end_lsn;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
-
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
-
- GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
-
- PG_RETURN_VOID();
-}
-
-/*
- * Get info and data of all WAL records from start LSN till end of WAL.
- *
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
- */
-Datum
-pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
-{
- XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
-
- start_lsn = PG_GETARG_LSN(0);
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn);
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
@@ -732,8 +728,14 @@ GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
/*
* Get stats of all WAL records between start LSN and end LSN.
*
- * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
- * the database system doesn't know about is specified.
+ * This function determines end LSN if it is not specified or specified as NULL
+ * or invalid. In such cases, the end LSN is assigned with the last flushed LSN
+ * when not in recovery or the last replayed LSN when in recovery.
+ *
+ * This function emits an error if start LSN is invalid or in future i.e. LSN
+ * the database system doesn't know about.
+ *
+ * This function returns NULL, when start LSN is past the end LSN.
*/
Datum
pg_get_wal_stats(PG_FUNCTION_ARGS)
@@ -742,11 +744,16 @@ pg_get_wal_stats(PG_FUNCTION_ARGS)
XLogRecPtr end_lsn;
bool stats_per_record;
- start_lsn = PG_GETARG_LSN(0);
- end_lsn = PG_GETARG_LSN(1);
- stats_per_record = PG_GETARG_BOOL(2);
+ GetInputLSNs(fcinfo, &start_lsn, &end_lsn);
+
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
- end_lsn = ValidateInputLSNs(false, start_lsn, end_lsn);
+ stats_per_record = PG_GETARG_BOOL(2);
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
@@ -754,22 +761,70 @@ pg_get_wal_stats(PG_FUNCTION_ARGS)
}
/*
- * Get stats of all WAL records from start LSN till end of WAL.
+ * NB: Following till_end_of_wal functions have been removed in newer versions
+ * of extension. However, we keep them around for backward compatibility. Which
+ * means, these functions work if someone explicitly installs the older
+ * extension version (using CREATE EXTENSION pg_walinspect WITH VERSION '1.0';)
+ * containing them.
*
- * This function emits an error if a future start i.e. WAL LSN the database
- * system doesn't know about is specified.
+ * If definitions of these functions are removed completely, the extension
+ * fails to install.
+ *
+ * In newer versions, one can use pg_get_wal_records_info()/pg_get_wal_stats()
+ * for the same till_end_of_wal functionality.
*/
+
+Datum
+pg_get_wal_records_info_till_end_of_wal(PG_FUNCTION_ARGS)
+{
+ XLogRecPtr start_lsn;
+ XLogRecPtr end_lsn;
+
+ if (PG_ARGISNULL(0) ||
+ ((start_lsn = PG_GETARG_LSN(0)) == InvalidXLogRecPtr))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid start LSN")));
+
+ /* Let's compute end LSN ourselves. */
+ end_lsn = GetCurrentLSN();
+
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
+
+ GetWALRecordsInfo(fcinfo, start_lsn, end_lsn);
+
+ PG_RETURN_VOID();
+}
+
Datum
pg_get_wal_stats_till_end_of_wal(PG_FUNCTION_ARGS)
{
XLogRecPtr start_lsn;
- XLogRecPtr end_lsn = InvalidXLogRecPtr;
+ XLogRecPtr end_lsn;
bool stats_per_record;
- start_lsn = PG_GETARG_LSN(0);
- stats_per_record = PG_GETARG_BOOL(1);
+ if (PG_ARGISNULL(0) ||
+ ((start_lsn = PG_GETARG_LSN(0)) == InvalidXLogRecPtr))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid start LSN")));
+
+ /* Let's compute end LSN ourselves. */
+ end_lsn = GetCurrentLSN();
+
+ /*
+ * When start LSN is past the end LSN, let's return NULL instead of raising
+ * an error for better usability of the functions.
+ */
+ if (start_lsn >= end_lsn)
+ PG_RETURN_NULL();
- end_lsn = ValidateInputLSNs(true, start_lsn, end_lsn);
+ stats_per_record = PG_GETARG_BOOL(2);
GetWalStats(fcinfo, start_lsn, end_lsn, stats_per_record);
diff --git a/contrib/pg_walinspect/sql/oldextversions.sql b/contrib/pg_walinspect/sql/oldextversions.sql
new file mode 100644
index 0000000000..2c4f484177
--- /dev/null
+++ b/contrib/pg_walinspect/sql/oldextversions.sql
@@ -0,0 +1,27 @@
+-- test old extension version entry points
+
+DROP EXTENSION pg_walinspect;
+CREATE EXTENSION pg_walinspect WITH VERSION '1.0';
+
+-- Move to new version 1.1
+ALTER EXTENSION pg_walinspect UPDATE TO '1.1';
+
+-- New function introduced in 1.1
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_records_info'::regproc);
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_records_info_till_end_of_wal'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_stats'::regproc);
+
+-- Removed function
+SELECT pg_get_functiondef('pg_get_wal_stats_till_end_of_wal'::regproc);
+
+-- Redefined function
+SELECT pg_get_functiondef('pg_get_wal_fpi_info'::regproc);
+
+DROP EXTENSION pg_walinspect;
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index 849201a1f8..1d73782388 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -17,23 +17,56 @@ INSERT INTO sample_tbl SELECT * FROM generate_series(3, 4);
-- Tests for input validation
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+-- Invalid input LSNs
+SELECT * FROM pg_get_wal_record_info('0/0'); -- ERROR
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- ERROR
+SELECT * FROM pg_get_wal_records_info('0/0'); -- ERROR
+
+SELECT * FROM pg_get_wal_stats('0/0'); -- ERROR
+
+SELECT * FROM pg_get_wal_fpi_info('0/0'); -- ERROR
+
+-- Start LSN NULL
+SELECT * FROM pg_get_wal_records_info(NULL); -- ERROR
+
+SELECT * FROM pg_get_wal_stats(NULL); -- ERROR
+
+SELECT * FROM pg_get_wal_fpi_info(NULL); -- ERROR
+
+-- Start LSN > End LSN
+SELECT * FROM pg_get_wal_records_info(:'wal_lsn2', :'wal_lsn1'); -- NULL
+
+SELECT * FROM pg_get_wal_stats(:'wal_lsn2', :'wal_lsn1'); -- NULL
+
+SELECT * FROM pg_get_wal_fpi_info(:'wal_lsn2', :'wal_lsn1'); -- NULL
-- ===================================================================
-- Tests for all function executions
-- ===================================================================
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_record_info(:'wal_lsn1');
+
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1');
+
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', '0/0');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2');
+-- Get info till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', NULL);
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_records_info_till_end_of_wal(:'wal_lsn1');
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats(:'wal_lsn1', :'wal_lsn2');
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1');
-SELECT COUNT(*) >= 0 AS ok FROM pg_get_wal_stats_till_end_of_wal(:'wal_lsn1');
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', '0/0');
+
+-- Get stats till end of WAL
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_stats(:'wal_lsn1', NULL);
-- ===================================================================
-- Test for filtering out WAL records of a particular table
@@ -68,6 +101,10 @@ SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3', :'wal_lsn4')
WHERE relfilenode = :'sample_tbl_oid';
+-- Check till end of WAL if we get FPI from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3')
+ WHERE relfilenode = :'sample_tbl_oid';
+
-- ===================================================================
-- Tests for permissions
-- ===================================================================
diff --git a/doc/src/sgml/pgwalinspect.sgml b/doc/src/sgml/pgwalinspect.sgml
index 3d7cdb95cc..86ce4381d5 100644
--- a/doc/src/sgml/pgwalinspect.sgml
+++ b/doc/src/sgml/pgwalinspect.sgml
@@ -61,6 +61,7 @@
Gets WAL record information of a given LSN. If the given LSN isn't
at the start of a WAL record, it gives the information of the next
available valid WAL record; or an error if no such record is found.
+ If given LSN is not yet available, the function will raise an error.
For example, usage of the function is as
follows:
<screen>
@@ -85,7 +86,7 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info">
<term>
<function>
- pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn)
+ pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL)
returns setof record
</function>
</term>
@@ -94,8 +95,9 @@ block_ref | blkref #0: rel 1663/5/60221 fork main blk 2
<para>
Gets information of all the valid WAL records between
<replaceable>start_lsn</replaceable> and <replaceable>end_lsn</replaceable>.
- Returns one row per WAL record. If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
+ Returns one row per WAL record. If <replaceable>end_lsn</replaceable>
+ isn't specified, it returns information till the end of WAL.
+ If <replaceable>start_lsn</replaceable> is not yet available, the
function will raise an error. For example:
<screen>
postgres=# SELECT * FROM pg_get_wal_records_info('0/1E913618', '0/1E913740') LIMIT 1;
@@ -116,27 +118,10 @@ block_ref |
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-records-info-till-end-of-wal">
- <term>
- <function>
- pg_get_wal_records_info_till_end_of_wal(start_lsn pg_lsn)
- returns setof record
- </function>
- </term>
-
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_records_info()</function>,
- except that it gets information of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till the end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="pgwalinspect-funcs-pg-get-wal-stats">
<term>
<function>
- pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn, per_record boolean DEFAULT false)
+ pg_get_wal_stats(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL, per_record boolean DEFAULT false)
returns setof record
</function>
</term>
@@ -149,9 +134,9 @@ block_ref |
<replaceable>resource_manager</replaceable> type. When
<replaceable>per_record</replaceable> is set to <literal>true</literal>,
it returns one row per <replaceable>record_type</replaceable>.
- If <replaceable>start_lsn</replaceable>
- or <replaceable>end_lsn</replaceable> are not yet available, the
- function will raise an error. For example:
+ If <replaceable>end_lsn</replaceable> isn't specified, it returns
+ information till the end of WAL. If <replaceable>start_lsn</replaceable>
+ is not yet available, the function will raise an error. For example:
<screen>
postgres=# SELECT * FROM pg_get_wal_stats('0/1E847D00', '0/1E84F500')
WHERE count > 0 LIMIT 1 AND
@@ -171,28 +156,14 @@ combined_size_percentage | 2.8634072910530795
</listitem>
</varlistentry>
- <varlistentry id="pgwalinspect-funcs-pg-get-wal-stats-till-end-of-wal">
+ <varlistentry>
<term>
<function>
- pg_get_wal_stats_till_end_of_wal(start_lsn pg_lsn, per_record boolean DEFAULT false)
+ pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn DEFAULT NULL)
returns setof record
</function>
</term>
- <listitem>
- <para>
- This function is the same as <function>pg_get_wal_stats()</function>,
- except that it gets statistics of all the valid WAL records from
- <replaceable>start_lsn</replaceable> till end of WAL.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>
- <function>pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
- </term>
-
<listitem>
<para>
Gets a copy of full page images as <type>bytea</type> values (after
@@ -200,9 +171,9 @@ combined_size_percentage | 2.8634072910530795
with all the valid WAL records between
<replaceable>start_lsn</replaceable> and
<replaceable>end_lsn</replaceable>. Returns one row per full page image.
- If <replaceable>start_lsn</replaceable> or
- <replaceable>end_lsn</replaceable> are not yet available, the function
- will raise an error. For example:
+ If <replaceable>end_lsn</replaceable> isn't specified, it returns
+ information till the end of WAL. If <replaceable>start_lsn</replaceable>
+ is not yet available, the function will raise an error. For example:
<screen>
postgres=# SELECT lsn, reltablespace, reldatabase, relfilenode, relblocknumber,
forkname, substring(fpi for 24) as fpi_trimmed
@@ -219,8 +190,20 @@ fpi_trimmed | \x00000000b89e660100000000a003c0030020042000000000
</para>
</listitem>
</varlistentry>
-
</variablelist>
+
+ <note>
+ <para>
+ Note that <function>pg_get_wal_records_info_till_end_of_wal</function> and
+ <function>pg_get_wal_stats_till_end_of_wal</function> functions have been
+ removed in the <filename>pg_walinspect</filename> version
+ <literal>1.1</literal>. The same functionality can be achieved with
+ <function>pg_get_wal_records_info</function> and
+ <function>pg_get_wal_stats</function> functions. However,
+ <function>till_end_of_wal</function> functions will still work if the
+ extension is installed explicitly with version <literal>1.0</literal>.
+ </para>
+ </note>
</sect2>
<sect2 id="pgwalinspect-author">
--
2.34.1
