pgsql: Remove error message hints mentioning configure options

2022-04-07 Thread Peter Eisentraut
Remove error message hints mentioning configure options These are usually not useful since users will use packaged distributions and won't be interested in rebuilding their installation from source. Also, we have only used these kinds of hints for some features and in some places, not

pgsql: pgstat: Update docs to match the shared memory stats reality.

2022-04-07 Thread Andres Freund
pgstat: Update docs to match the shared memory stats reality. This includes removing documentation for stats_temp_directory, adding documentation for stats_fetch_consistency, rephrasing references to the stats collector and documenting that starting a cleanly shut down standby will not remove

pgsql: pg_stat_statements: Track I/O timing for temporary file blocks

2022-04-07 Thread Michael Paquier
pg_stat_statements: Track I/O timing for temporary file blocks This commit adds two new columns to pg_stat_statements, called temp_blk_read_time and temp_blk_write_time. Those columns respectively show the time spent to read and write temporary file blocks on disk, whose tracking has been added

pgsql: Documentation for SQL/JSON features

2022-04-07 Thread Andrew Dunstan
Documentation for SQL/JSON features This documents the features added in commits f79b803dcc, f4fb45d15c, 33a377608f, 1a36bc9dba, 606948b058, 49082c2cc3, 4e34747c88, and fadb48b00e. I have cleaned up the aggregate section of the submitted docs, but there is still a deal of copy editing required.

pgsql: Track I/O timing for temporary file blocks in EXPLAIN (BUFFERS)

2022-04-07 Thread Michael Paquier
Track I/O timing for temporary file blocks in EXPLAIN (BUFFERS) Previously, the output of EXPLAIN (BUFFERS) option showed only the I/O timing spent reading and writing shared and local buffers. This commit adds on top of that the I/O timing for temporary buffers in the output of EXPLAIN (for

pgsql: Fix recovery_prefetch docs.

2022-04-07 Thread Thomas Munro
Fix recovery_prefetch docs. Correct a typo and a couple of sentences that weren't updated to reflect recent changes to the code. Reported-by: Justin Pryzby Discussion: https://postgr.es/m/2022040712.GC24419%40telsasoft.com Branch -- master Details ---

pgsql: pgstat: Hide instability in stats.spec with -DCATCACHE_FORCE_REL

2022-04-07 Thread Andres Freund
pgstat: Hide instability in stats.spec with -DCATCACHE_FORCE_RELEASE. With -DCATCACHE_FORCE_RELEASE a few tests failed. Those were trying to test behavior in the absence of invalidation processing and -DCATCACHE_FORCE_RELEASE obviously adds a lot of invalidation processing. The test already tried

pgsql: pgstat: add/extend tests for resetting various kinds of stats.

2022-04-07 Thread Andres Freund
pgstat: add/extend tests for resetting various kinds of stats. - subscriber stats reset path was untested - slot stat sreset path for all slots was untested - pg_stat_database.sessions etc was untested - pg_stat_reset_shared() was untested, for any kind of shared stats - pg_stat_reset() was

pgsql: Truncate line pointer array during heap pruning.

2022-04-07 Thread Peter Geoghegan
Truncate line pointer array during heap pruning. Reclaim space from the line pointer array when heap pruning leaves behind a contiguous group of LP_UNUSED items at the end of the array. This happens during subsequent page defragmentation. Certain kinds of heap line pointer bloat are ameliorated

pgsql: Teach planner and executor about monotonic window funcs

2022-04-07 Thread David Rowley
Teach planner and executor about monotonic window funcs Window functions such as row_number() always return a value higher than the previously returned value for tuples in any given window partition. Traditionally queries such as; SELECT * FROM ( SELECT *, row_number() over (order by c) rn

pgsql: Extend plsample example to include a trigger handler.

2022-04-07 Thread Tom Lane
Extend plsample example to include a trigger handler. Mark Wong and Konstantina Skovola, reviewed by Chapman Flack Discussion: https://postgr.es/m/Yd8Cz22eHi80XS30@workstation-mark-wong Branch -- master Details ---

Re: pgsql: Rewrite some RI code to avoid using SPI

2022-04-07 Thread Tom Lane
Alvaro Herrera writes: > On 2022-Apr-07, Tom Lane wrote: >> (1) We've added enough instability to the tree this week already. > Several animals failed already in ways that look obviously connected to > this commit, so I can't disagree: Yeah, I saw that after complaining. Looks like all and

pgsql: pgstat: test stats interactions with physical replication.

2022-04-07 Thread Andres Freund
pgstat: test stats interactions with physical replication. Tests that standbys: - drop stats for objects when the those records are replayed - persist stats across graceful restarts - discard stats after immediate / crash restarts Author: Melanie Plageman Author: Andres Freund Discussion:

pgsql: Add minimal tests for recovery conflict handling.

2022-04-07 Thread Andres Freund
Add minimal tests for recovery conflict handling. Previously none of our tests triggered recovery conflicts. The test is primarily motivated by needing tests for recovery conflict stats for shared memory based pgstats. But it's also a decent start for recovery conflict handling in general. The

pgsql: Revert "Rewrite some RI code to avoid using SPI"

2022-04-07 Thread Alvaro Herrera
Revert "Rewrite some RI code to avoid using SPI" This reverts commit 99392cdd78b788295e52b9f4942fa11992fd5ba9. We'd rather rewrite ri_triggers.c as a whole rather than piecemeal. Discussion: https://postgr.es/m/e1ncxx2-000mft...@gemulon.postgresql.org Branch -- master Details ---

Re: pgsql: Rewrite some RI code to avoid using SPI

2022-04-07 Thread Alvaro Herrera
On 2022-Apr-07, Tom Lane wrote: > Just for the record, I didn't approve of that patch, and I don't > think cramming it in a few hours before feature freeze is a good > way to proceed. > (1) We've added enough instability to the tree this week already. Several animals failed already in ways that

pgsql: psql: add \dconfig command to show server's configuration parame

2022-04-07 Thread Tom Lane
psql: add \dconfig command to show server's configuration parameters. Plain \dconfig is basically equivalent to SHOW except that you can give it a pattern with wildcards, either to match multiple GUCs or because you don't exactly remember the name you want. \dconfig+ adds type, context, and

Re: pgsql: Rewrite some RI code to avoid using SPI

2022-04-07 Thread Tom Lane
Alvaro Herrera writes: > Rewrite some RI code to avoid using SPI Just for the record, I didn't approve of that patch, and I don't think cramming it in a few hours before feature freeze is a good way to proceed. (1) We've added enough instability to the tree this week already. (2) I'm still

pgsql: pgstat: add tests for handling of restarts, including crashes.

2022-04-07 Thread Andres Freund
pgstat: add tests for handling of restarts, including crashes. Test that stats are restored during normal restarts, discarded after a crash / immediate restart, and that a corrupted stats file leads to stats being reset. Author: Melanie Plageman Author: Andres Freund Discussion:

pgsql: Rewrite some RI code to avoid using SPI

2022-04-07 Thread Alvaro Herrera
Rewrite some RI code to avoid using SPI Modify the subroutines called by RI trigger functions that want to check if a given referenced value exists in the referenced relation to simply scan the foreign key constraint's unique index, instead of using SPI to execute SELECT 1 FROM

pgsql: Fix test instability introduced in e349c95d3e9 due to async dedu

2022-04-07 Thread Andres Freund
Fix test instability introduced in e349c95d3e9 due to async deduplication. The statement emitting notifies tried to make sure page boundaries were crossed, but failed to do so reliably due to deduplication. Reported-By: c...@anastigmatix.net Discussion:

pgsql: Add isolation tests for snapshot behavior in ri_triggers.c

2022-04-07 Thread Alvaro Herrera
Add isolation tests for snapshot behavior in ri_triggers.c They are to check the behavior of RI_FKey_check() and ri_Check_Pk_Match(). A test case whereby RI_FKey_check() queries a partitioned PK table under REPEATABLE READ isolation produces wrong output due to a bug of the partition-descriptor

pgsql: Revert "Logical decoding of sequences"

2022-04-07 Thread Tomas Vondra
Revert "Logical decoding of sequences" This reverts a sequence of commits, implementing features related to logical decoding and replication of sequences: - 0da92dc530c9251735fc70b20cd004d9630a1266 - 80901b32913ffa59bf157a4d88284b2b3a7511d9 - b779d7d8fdae088d70da5ed9fcd8205035676df3 -

pgsql: doc: Fix man page whitespace issues

2022-04-07 Thread Peter Eisentraut
doc: Fix man page whitespace issues Whitespace between tags is significant, and in some cases it creates extra vertical space in man pages. The fix is to remove some newlines in the markup. Branch -- master Details ---

pgsql: Fix off-by-one error in pg_waldump, introduced in 5c279a6d350.

2022-04-07 Thread Jeff Davis
Fix off-by-one error in pg_waldump, introduced in 5c279a6d350. Per report by Bharath Rupireddy. Discussion: https://postgr.es/m/calj2acx+pwdk2myjdu8cb1ot7ouso6kd5-fkkegduestszj...@mail.gmail.com Branch -- master Details ---

pgsql: Fix another buildfarm issue from commit 5c279a6d350.

2022-04-07 Thread Jeff Davis
Fix another buildfarm issue from commit 5c279a6d350. Discussion: https://postgr.es/m/3280724.1649340...@sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/957aa4d87a419d18710283cf4f53ba7d3ead6bbe Modified Files --

pgsql: Unlogged sequences

2022-04-07 Thread Peter Eisentraut
Unlogged sequences Add support for unlogged sequences. Unlike for unlogged tables, this is not a performance feature. It allows sequences associated with unlogged tables to be excluded from replication. A new subcommand ALTER SEQUENCE ... SET LOGGED/UNLOGGED is added. An identity/serial

Re: pgsql: Prefetch data referenced by the WAL, take II.

2022-04-07 Thread Tom Lane
Andrew Dunstan writes: > On 4/7/22 04:36, Andres Freund wrote: >> Or maybe it's the scripts that are screwed up? Because I don't see any error >> checking in headerscheck/cpluspluscheck. And indeed, locally they show the >> errors, but exit with 0. > Yeah, you can't rely on the exit status, if

Re: pgsql: Prefetch data referenced by the WAL, take II.

2022-04-07 Thread Andrew Dunstan
On 4/7/22 04:36, Andres Freund wrote: > Hi, > > On 2022-04-07 07:44:20 +, Thomas Munro wrote: >> Prefetch data referenced by the WAL, take II. > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake=2022-04-07%2008%3A17%3A27 > thinks that xlogpretcher.h doesn't include enough... > >

pgsql: Fix typo in xlogrecovery.c code comment

2022-04-07 Thread Daniel Gustafsson
Fix typo in xlogrecovery.c code comment Author: Bharath Rupireddy Discussion: https://postgr.es/m/CALj2ACUoPtnReT=yaqmcwltccpk7p83xjea8tirx8q0_sjh...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/bab588cd5cbbeb43cda6e20c967b43000ea2aa80 Modified

pgsql: Avoid element in man pages

2022-04-07 Thread Peter Eisentraut
Avoid element in man pages The upstream DocBook manpages stylesheet apparently does not handle the element at all, and so the content comes out unformatted, which is not useful. As a workaround, replace with a nested , which ends up effectively the same in output. Branch -- master

pgsql: Include some missing headers.

2022-04-07 Thread Thomas Munro
Include some missing headers. Per headerscheck on BF animal crake, and Andres. Discussion: https://postgr.es/m/20220407083630.n62vgwqfy2v6wsrd%40alap3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/5b186308fbc41d0713fa91426d57a2afe37dd969 Modified

Re: pgsql: Prefetch data referenced by the WAL, take II.

2022-04-07 Thread Andres Freund
Hi, On 2022-04-07 07:44:20 +, Thomas Munro wrote: > Prefetch data referenced by the WAL, take II. https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake=2022-04-07%2008%3A17%3A27 thinks that xlogpretcher.h doesn't include enough... I had added those checks to CI, but apparently

pgsql: pgstat: add alternate output for stats.spec, for the 2PC disable

2022-04-07 Thread Andres Freund
pgstat: add alternate output for stats.spec, for the 2PC disabled case. It might be worth instead splitting the test up to produce a smaller alternative output file. But that's not trivial either, due to the number of steps defined. And more than I want to do tonight. Per buildfarm. Branch

pgsql: Try to silence "-Wmissing-braces" complaints in rmgrdesc.c.

2022-04-07 Thread Andres Freund
Try to silence "-Wmissing-braces" complaints in rmgrdesc.c. Per buildfarm member lapwing. https://postgr.es/m/20220407065640.xljttqcs46k4l...@alap3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/6392f2a0968c20ecde4d27b6652703ad931fce92 Modified Files

pgsql: Prefetch data referenced by the WAL, take II.

2022-04-07 Thread Thomas Munro
Prefetch data referenced by the WAL, take II. Introduce a new GUC recovery_prefetch. When enabled, look ahead in the WAL and try to initiate asynchronous reading of referenced data blocks that are not yet cached in our buffer pool. For now, this is done with posix_fadvise(), which has several

Re: pgsql: pgstat: store statistics in shared memory.

2022-04-07 Thread Michael Paquier
On Thu, Apr 07, 2022 at 12:31:49AM -0700, Andres Freund wrote: > And rorqual seems happy again. Thanks! -- Michael signature.asc Description: PGP signature

pgsql: Fix warning introduced in 5c279a6d350.

2022-04-07 Thread Jeff Davis
Fix warning introduced in 5c279a6d350. Change two macros to be static inline functions instead to keep the data type consistent. This avoids a "comparison is always true" warning that was occurring with -Wtype-limits. In the process, change the names to look less like macros. Discussion:

Re: pgsql: pgstat: store statistics in shared memory.

2022-04-07 Thread Andres Freund
Hi, On 2022-04-06 23:35:27 -0700, Andres Freund wrote: > On 2022-04-07 15:09:27 +0900, Michael Paquier wrote: > > On Thu, Apr 07, 2022 at 04:37:02AM +, Andres Freund wrote: > > > pgstat: store statistics in shared memory. > > > > > > Previously the statistics collector received statistics

pgsql: pgstat: add pg_stat_have_stats() test helper.

2022-04-07 Thread Andres Freund
pgstat: add pg_stat_have_stats() test helper. Will be used by tests committed subsequently. Bumps catversion (this time for real, the one in 0f96965c658 got lost when rebasing over 5c279a6d350). Author: Melanie Plageman Discussion:

pgsql: pgstat: add tests for transaction behaviour, 2PC, function stats

2022-04-07 Thread Andres Freund
pgstat: add tests for transaction behaviour, 2PC, function stats. Author: Andres Freund Author: Melanie Plageman Discussion: https://postgr.es/m/20220303021600.hs34ghqcw6zco...@alap3.anarazel.de Branch -- master Details ---

Re: pgsql: pgstat: add pg_stat_force_next_flush(), use it to simplify tests

2022-04-07 Thread Andres Freund
Hi, On 2022-04-07 14:01:14 +0700, John Naylor wrote: > On Thu, Apr 7, 2022 at 1:40 PM Andres Freund wrote: > > Bumps catversion. > > Looks like this part didn't make it. Damn. I think it was automatically resolved away by git when rebasing, because 5c279a6d350 contained the same change. git

Re: pgsql: pgstat: add pg_stat_force_next_flush(), use it to simplify tests

2022-04-07 Thread John Naylor
On Thu, Apr 7, 2022 at 1:40 PM Andres Freund wrote: > Bumps catversion. Looks like this part didn't make it. -- John Naylor EDB: http://www.enterprisedb.com

pgsql: pgstat: prevent fix pgstat_reinit_entry() from zeroing out lwloc

2022-04-07 Thread Andres Freund
pgstat: prevent fix pgstat_reinit_entry() from zeroing out lwlock. Zeroing out an lwlock in a normal build turns out to not trigger any alarms, if nobody can use the lwlock at that moment (as the case here). But with --disable-spinlocks --disable-atomics, the sema field needs to be initialized.

pgsql: pgstat: add pg_stat_force_next_flush(), use it to simplify tests

2022-04-07 Thread Andres Freund
pgstat: add pg_stat_force_next_flush(), use it to simplify tests. In the stats collector days it was hard to write tests for the stats system, because fundamentally delivery of stats messages over UDP was not synchronous (nor guaranteed). Now we easily can force pending stats updates to be

pgsql: pgstat: fix small bug in pgstat_drop_relation().

2022-04-07 Thread Andres Freund
pgstat: fix small bug in pgstat_drop_relation(). Just after committing 5891c7a8ed8, a test running with debug_discard_caches=1 failed locally... pgstat_drop_relation() neither checked pgstat_should_count_relation() nor called pgstat_prep_relation_pending(). With debug_discard_caches=1

Re: pgsql: pgstat: store statistics in shared memory.

2022-04-07 Thread Andres Freund
Hi, On 2022-04-07 15:09:27 +0900, Michael Paquier wrote: > On Thu, Apr 07, 2022 at 04:37:02AM +, Andres Freund wrote: > > pgstat: store statistics in shared memory. > > > > Previously the statistics collector received statistics updates via UDP and > > shared statistics data by writing them

pgsql: Fix compilation with WAL_DEBUG.

2022-04-07 Thread Andres Freund
Fix compilation with WAL_DEBUG. Broke with 5c279a6d350. But looks like it had been half-broken since 70e81861fad, because 'rmid' didn't refer to the current record's rmid anymore, but to rmid from "Initialize resource managers" - a constant. Branch -- master Details ---

pgsql: Custom WAL Resource Managers.

2022-04-07 Thread Jeff Davis
Custom WAL Resource Managers. Allow extensions to specify a new custom resource manager (rmgr), which allows specialized WAL. This is meant to be used by a Table Access Method or Index Access Method. Prior to this commit, only Generic WAL was available, which offers support for recovery and

Re: pgsql: pgstat: store statistics in shared memory.

2022-04-07 Thread Michael Paquier
On Thu, Apr 07, 2022 at 04:37:02AM +, Andres Freund wrote: > pgstat: store statistics in shared memory. > > Previously the statistics collector received statistics updates via UDP and > shared statistics data by writing them out to temporary files regularly. These > files can reach tens of