pgsql: Block signals while allocating DSM memory.

2022-07-14 Thread Thomas Munro
Block signals while allocating DSM memory. On Linux, we call posix_fallocate() on shm_open()'d memory to avoid later potential SIGBUS (see commit 899bd785). Based on field reports of systems stuck in an EINTR retry loop there, there, we made it possible to break out of that loop via slightly odd

pgsql: Block signals while allocating DSM memory.

2022-07-14 Thread Thomas Munro
Block signals while allocating DSM memory. On Linux, we call posix_fallocate() on shm_open()'d memory to avoid later potential SIGBUS (see commit 899bd785). Based on field reports of systems stuck in an EINTR retry loop there, there, we made it possible to break out of that loop via slightly odd

pgsql: Block signals while allocating DSM memory.

2022-07-14 Thread Thomas Munro
Block signals while allocating DSM memory. On Linux, we call posix_fallocate() on shm_open()'d memory to avoid later potential SIGBUS (see commit 899bd785). Based on field reports of systems stuck in an EINTR retry loop there, there, we made it possible to break out of that loop via slightly odd

pgsql: Block signals while allocating DSM memory.

2022-07-14 Thread Thomas Munro
Block signals while allocating DSM memory. On Linux, we call posix_fallocate() on shm_open()'d memory to avoid later potential SIGBUS (see commit 899bd785). Based on field reports of systems stuck in an EINTR retry loop there, there, we made it possible to break out of that loop via slightly odd

pgsql: Block signals while allocating DSM memory.

2022-07-14 Thread Thomas Munro
Block signals while allocating DSM memory. On Linux, we call posix_fallocate() on shm_open()'d memory to avoid later potential SIGBUS (see commit 899bd785). Based on field reports of systems stuck in an EINTR retry loop there, there, we made it possible to break out of that loop via slightly odd

pgsql: Block signals while allocating DSM memory.

2022-07-14 Thread Thomas Munro
Block signals while allocating DSM memory. On Linux, we call posix_fallocate() on shm_open()'d memory to avoid later potential SIGBUS (see commit 899bd785). Based on field reports of systems stuck in an EINTR retry loop there, there, we made it possible to break out of that loop via slightly odd

pgsql: Remove redundant ftruncate() for POSIX DSM memory.

2022-07-14 Thread Thomas Munro
Remove redundant ftruncate() for POSIX DSM memory. In early releases of the DSM infrastructure, it was possible to resize segments. That was removed in release 12 by commit 3c60d0fa. Now the ftruncate() + posix_fallocate() sequence during DSM segment creation has a redundant step: we're always

pgsql: Create a distinct wait event for POSIX DSM allocation.

2022-07-14 Thread Thomas Munro
Create a distinct wait event for POSIX DSM allocation. Previously we displayed "DSMFillZeroWrite" while in posix_fallocate(), because we shared the same wait event for "mmap" and "posix" DSM types. Let's introduce a new wait event "DSMAllocate", to be more accurate. Reported-by: Andres Freund

pgsql: Doc: Acknowledge historically supported CPUs and OSes.

2022-07-11 Thread Thomas Munro
Doc: Acknowledge historically supported CPUs and OSes. PostgreSQL/POSTGRES has run on a huge range of CPUs and OSes. As we're dropping some of the earliest systems the project was founded on, let's provide a place to remember them. Discussion:

pgsql: Further tidy-up for old CPU architectures.

2022-07-11 Thread Thomas Munro
Further tidy-up for old CPU architectures. Further to commit 92d70b77, let's drop the code we carry for the following untested architectures: M68K, M88K, M32R, SuperH. We have no idea if anything actually works there, and surely as vintage hardware and microcontrollers they would be underpowered

pgsql: Fix lock assertions in dshash.c.

2022-07-10 Thread Thomas Munro
Fix lock assertions in dshash.c. dshash.c previously maintained flags to be able to assert that you didn't hold any partition lock. These flags could get out of sync with reality in error scenarios. Get rid of all that, and make assertions about the locks themselves instead. Since

pgsql: Fix lock assertions in dshash.c.

2022-07-10 Thread Thomas Munro
Fix lock assertions in dshash.c. dshash.c previously maintained flags to be able to assert that you didn't hold any partition lock. These flags could get out of sync with reality in error scenarios. Get rid of all that, and make assertions about the locks themselves instead. Since

pgsql: Fix lock assertions in dshash.c.

2022-07-10 Thread Thomas Munro
Fix lock assertions in dshash.c. dshash.c previously maintained flags to be able to assert that you didn't hold any partition lock. These flags could get out of sync with reality in error scenarios. Get rid of all that, and make assertions about the locks themselves instead. Since

pgsql: Fix lock assertions in dshash.c.

2022-07-10 Thread Thomas Munro
Fix lock assertions in dshash.c. dshash.c previously maintained flags to be able to assert that you didn't hold any partition lock. These flags could get out of sync with reality in error scenarios. Get rid of all that, and make assertions about the locks themselves instead. Since

pgsql: Fix lock assertions in dshash.c.

2022-07-10 Thread Thomas Munro
Fix lock assertions in dshash.c. dshash.c previously maintained flags to be able to assert that you didn't hold any partition lock. These flags could get out of sync with reality in error scenarios. Get rid of all that, and make assertions about the locks themselves instead. Since

pgsql: Fix lock assertions in dshash.c.

2022-07-10 Thread Thomas Munro
Fix lock assertions in dshash.c. dshash.c previously maintained flags to be able to assert that you didn't hold any partition lock. These flags could get out of sync with reality in error scenarios. Get rid of all that, and make assertions about the locks themselves instead. Since

pgsql: Tidy up claimed supported CPUs and OSes.

2022-07-10 Thread Thomas Munro
Tidy up claimed supported CPUs and OSes. * Remove arbitrary mention of certain endianness and bitness variants; it's enough to say that applicable variants are expected to work. * List RISC-V (known to work, being tested). * List SuperH and M88K (code exists, unknown status, like M68K). *

pgsql: Fix \watch's interaction with libedit on ^C.

2022-07-09 Thread Thomas Munro
Fix \watch's interaction with libedit on ^C. When you hit ^C, the terminal driver in Unix-like systems echoes "^C" as well as sending an interrupt signal (depending on stty settings). At least libedit (but maybe also libreadline) is then confused about the current cursor location, and corrupts

pgsql: Fix \watch's interaction with libedit on ^C.

2022-07-09 Thread Thomas Munro
Fix \watch's interaction with libedit on ^C. When you hit ^C, the terminal driver in Unix-like systems echoes "^C" as well as sending an interrupt signal (depending on stty settings). At least libedit (but maybe also libreadline) is then confused about the current cursor location, and corrupts

pgsql: Fix \watch's interaction with libedit on ^C.

2022-07-09 Thread Thomas Munro
Fix \watch's interaction with libedit on ^C. When you hit ^C, the terminal driver in Unix-like systems echoes "^C" as well as sending an interrupt signal (depending on stty settings). At least libedit (but maybe also libreadline) is then confused about the current cursor location, and corrupts

pgsql: Fix \watch's interaction with libedit on ^C.

2022-07-09 Thread Thomas Munro
Fix \watch's interaction with libedit on ^C. When you hit ^C, the terminal driver in Unix-like systems echoes "^C" as well as sending an interrupt signal (depending on stty settings). At least libedit (but maybe also libreadline) is then confused about the current cursor location, and corrupts

pgsql: Fix \watch's interaction with libedit on ^C.

2022-07-09 Thread Thomas Munro
Fix \watch's interaction with libedit on ^C. When you hit ^C, the terminal driver in Unix-like systems echoes "^C" as well as sending an interrupt signal (depending on stty settings). At least libedit (but maybe also libreadline) is then confused about the current cursor location, and corrupts

pgsql: Fix \watch's interaction with libedit on ^C.

2022-07-09 Thread Thomas Munro
Fix \watch's interaction with libedit on ^C. When you hit ^C, the terminal driver in Unix-like systems echoes "^C" as well as sending an interrupt signal (depending on stty settings). At least libedit (but maybe also libreadline) is then confused about the current cursor location, and corrupts

pgsql: Fix \watch's interaction with libedit on ^C.

2022-07-09 Thread Thomas Munro
Fix \watch's interaction with libedit on ^C. When you hit ^C, the terminal driver in Unix-like systems echoes "^C" as well as sending an interrupt signal (depending on stty settings). At least libedit (but maybe also libreadline) is then confused about the current cursor location, and corrupts

pgsql: Remove HP-UX port.

2022-07-07 Thread Thomas Munro
Remove HP-UX port. HP-UX hardware is no longer produced, build farm coverage recently ended, and there are no known active maintainers targeting this OS. Since there is a major rewrite of the build system in the pipeline for PostgreSQL 16, and that requires development, testing and maintainance

pgsql: Remove HP/Intel Itanium support.

2022-07-07 Thread Thomas Munro
Remove HP/Intel Itanium support. This CPU architecture has been discontinued. We already removed HP-UX support, we never supported Windows/Itanium, and the open source operating systems that a vintage hardware owner might hope to run have all either ended Itanium support or never fully released

pgsql: Default to dynamic_shared_memory_type=sysv on Solaris.

2022-07-01 Thread Thomas Munro
Default to dynamic_shared_memory_type=sysv on Solaris. POSIX shm_open() can sleep for a long time and fail spuriously because of contention on an internal lock file on Solaris (and presumably illumos). Commit 389869af fixed the main problem with this, namely that we could crash, but it's now

pgsql: Default to dynamic_shared_memory_type=sysv on Solaris.

2022-07-01 Thread Thomas Munro
Default to dynamic_shared_memory_type=sysv on Solaris. POSIX shm_open() can sleep for a long time and fail spuriously because of contention on an internal lock file on Solaris (and presumably illumos). Commit 389869af fixed the main problem with this, namely that we could crash, but it's now

pgsql: Harden dsm_impl.c against unexpected EEXIST.

2022-06-30 Thread Thomas Munro
Harden dsm_impl.c against unexpected EEXIST. Previously, we trusted the OS not to report EEXIST unless we'd passed in IPC_CREAT | IPC_EXCL or O_CREAT | O_EXCL, as appropriate. Solaris's shm_open() can in fact do that, causing us to crash because we didn't ereport and then we blithely assumed the

pgsql: Harden dsm_impl.c against unexpected EEXIST.

2022-06-30 Thread Thomas Munro
Harden dsm_impl.c against unexpected EEXIST. Previously, we trusted the OS not to report EEXIST unless we'd passed in IPC_CREAT | IPC_EXCL or O_CREAT | O_EXCL, as appropriate. Solaris's shm_open() can in fact do that, causing us to crash because we didn't ereport and then we blithely assumed the

pgsql: Harden dsm_impl.c against unexpected EEXIST.

2022-06-30 Thread Thomas Munro
Harden dsm_impl.c against unexpected EEXIST. Previously, we trusted the OS not to report EEXIST unless we'd passed in IPC_CREAT | IPC_EXCL or O_CREAT | O_EXCL, as appropriate. Solaris's shm_open() can in fact do that, causing us to crash because we didn't ereport and then we blithely assumed the

pgsql: Harden dsm_impl.c against unexpected EEXIST.

2022-06-30 Thread Thomas Munro
Harden dsm_impl.c against unexpected EEXIST. Previously, we trusted the OS not to report EEXIST unless we'd passed in IPC_CREAT | IPC_EXCL or O_CREAT | O_EXCL, as appropriate. Solaris's shm_open() can in fact do that, causing us to crash because we didn't ereport and then we blithely assumed the

pgsql: Harden dsm_impl.c against unexpected EEXIST.

2022-06-30 Thread Thomas Munro
Harden dsm_impl.c against unexpected EEXIST. Previously, we trusted the OS not to report EEXIST unless we'd passed in IPC_CREAT | IPC_EXCL or O_CREAT | O_EXCL, as appropriate. Solaris's shm_open() can in fact do that, causing us to crash because we didn't ereport and then we blithely assumed the

pgsql: Harden dsm_impl.c against unexpected EEXIST.

2022-06-30 Thread Thomas Munro
Harden dsm_impl.c against unexpected EEXIST. Previously, we trusted the OS not to report EEXIST unless we'd passed in IPC_CREAT | IPC_EXCL or O_CREAT | O_EXCL, as appropriate. Solaris's shm_open() can in fact do that, causing us to crash because we didn't ereport and then we blithely assumed the

pgsql: Harden dsm_impl.c against unexpected EEXIST.

2022-06-30 Thread Thomas Munro
Harden dsm_impl.c against unexpected EEXIST. Previously, we trusted the OS not to report EEXIST unless we'd passed in IPC_CREAT | IPC_EXCL or O_CREAT | O_EXCL, as appropriate. Solaris's shm_open() can in fact do that, causing us to crash because we didn't ereport and then we blithely assumed the

pgsql: Fix relptr's encoding of the base address.

2022-06-26 Thread Thomas Munro
Fix relptr's encoding of the base address. Previously, we encoded both NULL and the first byte at the base address as 0. That confusion led to the assertion in commit e07d4ddc, which failed when min_dynamic_shared_memory was used. Give them distinct encodings, by switching to 1-based offsets

pgsql: Fix relptr's encoding of the base address.

2022-06-26 Thread Thomas Munro
Fix relptr's encoding of the base address. Previously, we encoded both NULL and the first byte at the base address as 0. That confusion led to the assertion in commit e07d4ddc, which failed when min_dynamic_shared_memory was used. Give them distinct encodings, by switching to 1-based offsets

pgsql: Don't trust signalfd() on illumos.

2022-06-25 Thread Thomas Munro
Don't trust signalfd() on illumos. Since commit 6a2a70a02, we've used signalfd() to receive latch wakeups when building with WAIT_USE_EPOLL (default for Linux and illumos), and our traditional self-pipe when falling back to WAIT_USE_POLL (default for other Unixes with neither epoll() nor

pgsql: Don't trust signalfd() on illumos.

2022-06-25 Thread Thomas Munro
Don't trust signalfd() on illumos. Since commit 6a2a70a02, we've used signalfd() to receive latch wakeups when building with WAIT_USE_EPOLL (default for Linux and illumos), and our traditional self-pipe when falling back to WAIT_USE_POLL (default for other Unixes with neither epoll() nor

pgsql: Add debugging help in OwnLatch().

2022-05-30 Thread Thomas Munro
Add debugging help in OwnLatch(). Build farm animal gharial recently failed a few times in a parallel worker's call to OwnLatch() with "ERROR: latch already owned". Let's turn that into a PANIC and show the PID of the owner, to try to learn more. Discussion:

pgsql: Doc: Document pg_stat_recovery_prefetch.stats_reset.

2022-05-28 Thread Thomas Munro
Doc: Document pg_stat_recovery_prefetch.stats_reset. This column was missing from the table that describes the new view. Author: Shinoda, Noriyoshi (PN Japan FSIP) Discussion: https://postgr.es/m/dm4pr84mb173401c19a0eb9b1caab197cee...@dm4pr84mb1734.namprd84.prod.outlook.com Branch --

pgsql: Fix slow animal timeouts in 032_relfilenode_reuse.pl.

2022-05-13 Thread Thomas Munro
Fix slow animal timeouts in 032_relfilenode_reuse.pl. Per BF animal chipmunk: CREATE DATABASE could apparently fail due to an AV process being in the template database and not quitting fast enough for the 5 second timeout in CountOtherDBBackends(). The test script had autovacuum_naptime=1s to

pgsql: Add logging for excessive ProcSignalBarrier waits.

2022-05-11 Thread Thomas Munro
Add logging for excessive ProcSignalBarrier waits. To enable diagnosis of systems that are not processing ProcSignalBarrier requests promptly, add a LOG message every 5 seconds if we seem to be wedged. Although you could already see this state as a wait event in pg_stat_activity, the log message

pgsql: Fix race in 032_relfilenode_reuse.pl.

2022-05-08 Thread Thomas Munro
Fix race in 032_relfilenode_reuse.pl. Add wait_for_catchup() call to the test added by commit e2f65f42. Per slow build farm animal grison. Also fix a comment. Discussion: https://postgr.es/m/CA%2BhUKGLJ2Vy8hVQmnYotmTaEKZK0%3D-GcXgNAgcHzArZvtS4L_g%40mail.gmail.com Branch -- master

pgsql: Fix old-fd issues using global barriers everywhere.

2022-05-06 Thread Thomas Munro
for now as the infrastructure required is too new and invasive. In master only, since commit aa010514, it could also happen when using CREATE DATABASE with a user-supplied OID or via pg_upgrade. Author: Andres Freund Reviewed-by: Robert Haas Reviewed-by: Thomas Munro Discussion: https

pgsql: Rethink PROCSIGNAL_BARRIER_SMGRRELEASE.

2022-05-06 Thread Thomas Munro
Rethink PROCSIGNAL_BARRIER_SMGRRELEASE. With sufficiently bad luck, it was possible for IssuePendingWritebacks() to reopen a file after we'd processed PROCSIGNAL_BARRIER_SMGRRELEASE and before the file was unlinked by some other backend. That left a small hole in commit 4eb21763's plan to fix

Re: pgsql: Track LLVM 15 changes.

2022-04-16 Thread Thomas Munro
On Wed, Feb 16, 2022 at 9:18 AM Thomas Munro wrote: > On Wed, Feb 16, 2022 at 7:34 AM Tom Lane wrote: > > Although seawasp isn't actually failing at the moment, it's emitting > > a boatload of deprecation warnings, eg > > > > In file included from llvmjit_deform.c:27:

pgsql: Don't retry restore_command while reading ahead.

2022-04-16 Thread Thomas Munro
Don't retry restore_command while reading ahead. Suppress further attempts to read ahead in the WAL if we run out of data, until the records already decoded have been replayed. This restores the traditional behavior for continuous archive recovery, which is to retry the failing restore_command

pgsql: Fix documentation for pg_stat_recovery_prefetch.

2022-04-12 Thread Thomas Munro
. Author: sirisha chamarthi Author: Thomas Munro Discussion: https://postgr.es/m/cakrakevk-lrhmdyt6x_p33ef6dcorm2jed5h_ehdrdv0res...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/4533de358dd5f2a8930e3a73bcb49103b37357be Modified Files -- doc

pgsql: Fix recovery_prefetch docs.

2022-04-07 Thread Thomas Munro
Fix recovery_prefetch docs. Correct a typo and a couple of sentences that weren't updated to reflect recent changes to the code. Reported-by: Justin Pryzby Discussion: https://postgr.es/m/2022040712.GC24419%40telsasoft.com Branch -- master Details ---

pgsql: Include some missing headers.

2022-04-07 Thread Thomas Munro
Include some missing headers. Per headerscheck on BF animal crake, and Andres. Discussion: https://postgr.es/m/20220407083630.n62vgwqfy2v6wsrd%40alap3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/5b186308fbc41d0713fa91426d57a2afe37dd969 Modified

pgsql: Prefetch data referenced by the WAL, take II.

2022-04-07 Thread Thomas Munro
Prefetch data referenced by the WAL, take II. Introduce a new GUC recovery_prefetch. When enabled, look ahead in the WAL and try to initiate asynchronous reading of referenced data blocks that are not yet cached in our buffer pool. For now, this is done with posix_fadvise(), which has several

pgsql: Fix tuplesort optimization for CLUSTER-on-expression.

2022-04-03 Thread Thomas Munro
Fix tuplesort optimization for CLUSTER-on-expression. When dispatching sort operations to specialized variants, commit 69749243 failed to handle the case where CLUSTER-sort decides not to initialize datum1 and isnull1. Fix by hoisting that decision up a level and advertising whether datum1 can

pgsql: Fix pg_waldump docs.

2022-03-27 Thread Thomas Munro
Fix pg_waldump docs. Before 52b5568, the recently added -l option was short for --relation. We changed it to -R, but we forgot to update one place in the documentation. Author: Japin Li Discussion:

pgsql: Improve command line options for pg_waldump.

2022-03-24 Thread Thomas Munro
Improve command line options for pg_waldump. Follow-up improvements for commit 127aea2a based on discussion: * use fork name for --fork, not number * use -R, -B as short switches for --relation, --block * re-alphabetize the list of switches (code, --help and docs) Suggested-by: Peter Eisentraut

pgsql: Add additional filtering options to pg_waldump.

2022-03-24 Thread Thomas Munro
Add additional filtering options to pg_waldump. Allow filtering by RelFileNode, BlockNumber, ForkNum and FPW. Author: David Christensen Reviewed-by: Japin Li Reviewed-by: Bharath Rupireddy Reviewed-by: Cary Huang Reviewed-by: Thomas Munro Discussion: https://postgr.es/m/lzzgmgm6e5.fsf

pgsql: Try to stabilize vacuum test.

2022-03-22 Thread Thomas Munro
Try to stabilize vacuum test. As commits b700f96c and 3414099c did for the reloptions test, make sure VACUUM can always truncate the table as expected. Back-patch to 12, where vacuum_truncate arrived. Discussion:

pgsql: Try to stabilize vacuum test.

2022-03-22 Thread Thomas Munro
Try to stabilize vacuum test. As commits b700f96c and 3414099c did for the reloptions test, make sure VACUUM can always truncate the table as expected. Back-patch to 12, where vacuum_truncate arrived. Discussion:

pgsql: Try to stabilize vacuum test.

2022-03-22 Thread Thomas Munro
Try to stabilize vacuum test. As commits b700f96c and 3414099c did for the reloptions test, make sure VACUUM can always truncate the table as expected. Back-patch to 12, where vacuum_truncate arrived. Discussion:

pgsql: Try to stabilize vacuum test.

2022-03-22 Thread Thomas Munro
Try to stabilize vacuum test. As commits b700f96c and 3414099c did for the reloptions test, make sure VACUUM can always truncate the table as expected. Back-patch to 12, where vacuum_truncate arrived. Discussion:

pgsql: Log regression.diffs in 027_stream_regress.pl.

2022-03-20 Thread Thomas Munro
Log regression.diffs in 027_stream_regress.pl. To help diagnose the reasons for a regression test failure inside this TAP test, dump the contents of regression.diffs to the log. While the CI scripts show it automatically, the build farm client does not. Reviewed-by: Andrew Dunstan Discussion:

pgsql: Add circular WAL decoding buffer, take II.

2022-03-17 Thread Thomas Munro
Add circular WAL decoding buffer, take II. Teach xlogreader.c to decode the WAL into a circular buffer. This will support optimizations based on looking ahead, to follow in a later commit. * XLogReadRecord() works as before, decoding records one by one, and allowing them to be examined via

Re: pgsql: Add option to use ICU as global locale provider

2022-03-17 Thread Thomas Munro
On Fri, Mar 18, 2022 at 4:12 PM Julien Rouhaud wrote: > On Fri, Mar 18, 2022 at 11:01:11AM +0900, Michael Paquier wrote: > > FYI, prion is complaining here: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prion=2022-03-18%2001%3A43%3A13 > > > > Some details: > > # Failed test 'fails

pgsql: Declare aarch64 has single copy atomicity for 8 byte values.

2022-03-17 Thread Thomas Munro
-service.arm.com/static/61fbe8f4fa8173727a1b734e https://developer.arm.com/documentation/ddi0487/latest Author: Yura Sokolov Reviewed-by: Thomas Munro Reviewed-by: Alexander Korotkov Discussion: https://postgr.es/m/f7f3f0febe27862711f924a7b0f39e065e547f4b.camel%40postgrespro.ru Discussion: https://postgr.es

Re: pgsql: Fix pg_basebackup with in-place tablespaces.

2022-03-16 Thread Thomas Munro
On Thu, Mar 17, 2022 at 3:29 AM David Steele wrote: > Sure, but there is a behavioral change whether the GUC is enabled or > not. Before, if there was clutter in pg_tblspc there would at least be a > warning in the log. Now that logging does not happen. If there's clutter that doesn't look like

pgsql: Fix race between DROP TABLESPACE and checkpointing.

2022-03-15 Thread Thomas Munro
) performs memory allocations, it cannot be called within a critical section, so we also need to move SyncPreCheckpoint() to before CreateCheckPoint()'s critical section. This is an old bug, so back-patch to all supported versions. Author: Nathan Bossart Reported-by: Nathan Bossart Reviewed-by: Th

pgsql: Fix race between DROP TABLESPACE and checkpointing.

2022-03-15 Thread Thomas Munro
) performs memory allocations, it cannot be called within a critical section, so we also need to move SyncPreCheckpoint() to before CreateCheckPoint()'s critical section. This is an old bug, so back-patch to all supported versions. Author: Nathan Bossart Reported-by: Nathan Bossart Reviewed-by: Th

pgsql: Fix race between DROP TABLESPACE and checkpointing.

2022-03-15 Thread Thomas Munro
) performs memory allocations, it cannot be called within a critical section, so we also need to move SyncPreCheckpoint() to before CreateCheckPoint()'s critical section. This is an old bug, so back-patch to all supported versions. Author: Nathan Bossart Reported-by: Nathan Bossart Reviewed-by: Th

pgsql: Fix race between DROP TABLESPACE and checkpointing.

2022-03-15 Thread Thomas Munro
) performs memory allocations, it cannot be called within a critical section, so we also need to move SyncPreCheckpoint() to before CreateCheckPoint()'s critical section. This is an old bug, so back-patch to all supported versions. Author: Nathan Bossart Reported-by: Nathan Bossart Reviewed-by: Th

pgsql: Fix race between DROP TABLESPACE and checkpointing.

2022-03-15 Thread Thomas Munro
) performs memory allocations, it cannot be called within a critical section, so we also need to move SyncPreCheckpoint() to before CreateCheckPoint()'s critical section. This is an old bug, so back-patch to all supported versions. Author: Nathan Bossart Reported-by: Nathan Bossart Reviewed-by: Th

pgsql: Fix race between DROP TABLESPACE and checkpointing.

2022-03-15 Thread Thomas Munro
) performs memory allocations, it cannot be called within a critical section, so we also need to move SyncPreCheckpoint() to before CreateCheckPoint()'s critical section. This is an old bug, so back-patch to all supported versions. Author: Nathan Bossart Reported-by: Nathan Bossart Reviewed-by: Th

pgsql: Fix documentation typo in commit 5e6368b4.

2022-03-15 Thread Thomas Munro
Fix documentation typo in commit 5e6368b4. Back-patch to 14. Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/ea70f6945029b622ea28d53c7a32354d1d033246 Modified Files -- doc/src/sgml/monitoring.sgml | 2 +- 1 file changed, 1 insertion(+), 1

pgsql: Fix documentation typo in commit 5e6368b4.

2022-03-15 Thread Thomas Munro
Fix documentation typo in commit 5e6368b4. Back-patch to 14. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/501c66c81b755583fa516c88e1cc7c760dc88bd5 Modified Files -- doc/src/sgml/monitoring.sgml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

pgsql: Fix waiting in RegisterSyncRequest().

2022-03-15 Thread Thomas Munro
Fix waiting in RegisterSyncRequest(). If we run out of space in the checkpointer sync request queue (which is hopefully rare on real systems, but common with very small buffer pool), we wait for it to drain. While waiting, we should report that as a wait event so that users know what is going

pgsql: Fix waiting in RegisterSyncRequest().

2022-03-15 Thread Thomas Munro
Fix waiting in RegisterSyncRequest(). If we run out of space in the checkpointer sync request queue (which is hopefully rare on real systems, but common with very small buffer pool), we wait for it to drain. While waiting, we should report that as a wait event so that users know what is going

pgsql: Fix waiting in RegisterSyncRequest().

2022-03-15 Thread Thomas Munro
Fix waiting in RegisterSyncRequest(). If we run out of space in the checkpointer sync request queue (which is hopefully rare on real systems, but common with very small buffer pool), we wait for it to drain. While waiting, we should report that as a wait event so that users know what is going

pgsql: Fix waiting in RegisterSyncRequest().

2022-03-15 Thread Thomas Munro
Fix waiting in RegisterSyncRequest(). If we run out of space in the checkpointer sync request queue (which is hopefully rare on real systems, but common with very small buffer pool), we wait for it to drain. While waiting, we should report that as a wait event so that users know what is going

pgsql: Wake up for latches in CheckpointWriteDelay().

2022-03-15 Thread Thomas Munro
Wake up for latches in CheckpointWriteDelay(). The checkpointer shouldn't ignore its latch. Other backends may be waiting for it to drain the request queue. Hopefully real systems don't have a full queue often, but the condition is reached easily when shared_buffers is small. This involves

pgsql: Wake up for latches in CheckpointWriteDelay().

2022-03-15 Thread Thomas Munro
Wake up for latches in CheckpointWriteDelay(). The checkpointer shouldn't ignore its latch. Other backends may be waiting for it to drain the request queue. Hopefully real systems don't have a full queue often, but the condition is reached easily when shared_buffers is small. This involves

pgsql: Back-patch LLVM 14 API changes.

2022-03-15 Thread Thomas Munro
Back-patch LLVM 14 API changes. Since LLVM 14 has stopped changing and is about to be released, back-patch the following changes from the master branch: e6a7600202105919bffd62b3dfd941f4a94e082b 807fee1a39de6bb8184082012e643951abb9ad1d a56e7b66010f330782243de9e25ac2a6596be0e1 Back-patch to

pgsql: Back-patch LLVM 14 API changes.

2022-03-15 Thread Thomas Munro
Back-patch LLVM 14 API changes. Since LLVM 14 has stopped changing and is about to be released, back-patch the following changes from the master branch: e6a7600202105919bffd62b3dfd941f4a94e082b 807fee1a39de6bb8184082012e643951abb9ad1d a56e7b66010f330782243de9e25ac2a6596be0e1 Back-patch to

pgsql: Back-patch LLVM 14 API changes.

2022-03-15 Thread Thomas Munro
Back-patch LLVM 14 API changes. Since LLVM 14 has stopped changing and is about to be released, back-patch the following changes from the master branch: e6a7600202105919bffd62b3dfd941f4a94e082b 807fee1a39de6bb8184082012e643951abb9ad1d a56e7b66010f330782243de9e25ac2a6596be0e1 Back-patch to

pgsql: Back-patch LLVM 14 API changes.

2022-03-15 Thread Thomas Munro
Back-patch LLVM 14 API changes. Since LLVM 14 has stopped changing and is about to be released, back-patch the following changes from the master branch: e6a7600202105919bffd62b3dfd941f4a94e082b 807fee1a39de6bb8184082012e643951abb9ad1d a56e7b66010f330782243de9e25ac2a6596be0e1 Back-patch to

pgsql: Silence LLVM 14 API deprecation warnings.

2022-03-15 Thread Thomas Munro
Silence LLVM 14 API deprecation warnings. We are going to need to handle the upcoming opaque pointer API changes[1], possibly in time for LLVM 15, but in the meantime let's silence the warnings produced by LLVM 14. [1] https://llvm.org/docs/OpaquePointers.html Reviewed-by: Andres Freund

Re: pgsql: Fix pg_basebackup with in-place tablespaces.

2022-03-15 Thread Thomas Munro
On Wed, Mar 16, 2022 at 10:28 AM Tom Lane wrote: > David Steele writes: > > On 3/14/22 19:31, Thomas Munro wrote: > >> Fix pg_basebackup with in-place tablespaces. > > > Perhaps I'm being picky, but seems like this logic should be wrapped in: > &g

pgsql: Fix pg_basebackup with in-place tablespaces.

2022-03-14 Thread Thomas Munro
Fix pg_basebackup with in-place tablespaces. Previously, pg_basebackup from a cluster that contained an 'in-place' tablespace, as introduced by commit 7170f215, would produce a harmless warning on Unix and fail completely on Windows. Reported-by: Kyotaro Horiguchi Reviewed-by: Kyotaro Horiguchi

Re: pgsql: Track LLVM 15 changes.

2022-03-14 Thread Thomas Munro
On Wed, Feb 16, 2022 at 11:44 AM Tom Lane wrote: > Andres Freund writes: > > On 2022-02-16 09:18:53 +1300, Thomas Munro wrote: > >> Yeah I mentioned this problem in the other thread. I got as far as > >> finding this write-up: > >> https://llvm.org/

Re: pgsql: Track LLVM 15 changes.

2022-02-15 Thread Thomas Munro
On Wed, Feb 16, 2022 at 7:34 AM Tom Lane wrote: > Although seawasp isn't actually failing at the moment, it's emitting > a boatload of deprecation warnings, eg > > In file included from llvmjit_deform.c:27: > ../../../../src/include/jit/llvmjit_emit.h:112:23: warning: > 'LLVMBuildStructGEP' is

Re: pgsql: Track LLVM 15 changes.

2022-02-14 Thread Thomas Munro
On Tue, Feb 15, 2022 at 10:22 AM Alvaro Herrera wrote: > On 2022-Feb-15, Thomas Munro wrote: > > My general plan for this stuff is to try to do just one back-patch for > > each LLVM release, with all the changes in it, to reduce commit churn. > > Hence commit messages t

Re: pgsql: Track LLVM 15 changes.

2022-02-14 Thread Thomas Munro
On Tue, Feb 15, 2022 at 4:06 AM Fabien COELHO wrote: > > This isn't an API change, it's just a missing #include that we got away > > with before. Per buildfarm animal seawasp. > > If it is a somehow *missing* include, should it be back-patched? Not sure, > just asking. Arguably (I think it'd

pgsql: Add WL_SOCKET_CLOSED for socket shutdown events.

2022-02-13 Thread Thomas Munro
Add WL_SOCKET_CLOSED for socket shutdown events. Provide a way for WaitEventSet to report that the remote peer has shut down its socket, independently of whether there is any buffered data remaining to be read. This works only on systems where the kernel exposes that information, namely: *

pgsql: Use WL_SOCKET_CLOSED for client_connection_check_interval.

2022-02-13 Thread Thomas Munro
Use WL_SOCKET_CLOSED for client_connection_check_interval. Previously we used poll() directly to check for a POLLRDHUP event. Instead, use the WaitEventSet API to poll the socket for WL_SOCKET_CLOSED, which knows how to detect this condition on many more operating systems. Reviewed-by: Zhihong

pgsql: Track LLVM 15 changes.

2022-02-13 Thread Thomas Munro
Track LLVM 15 changes. This isn't an API change, it's just a missing #include that we got away with before. Per buildfarm animal seawasp. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/0052fb489008a68d0f3e0445f52e1ab3166632a4 Modified Files --

pgsql: Remove REGRESS_OUTPUTDIR environment variable.

2022-02-13 Thread Thomas Munro
Remove REGRESS_OUTPUTDIR environment variable. Andres Freund points out that the tmp_check path is already available as perl variable PostgreSQL::Test::Utils::tmp_check, so we can drop the new environment variable introduced by commit f47ed79cc. Discussion:

pgsql: Fix DROP {DATABASE,TABLESPACE} on Windows.

2022-02-11 Thread Thomas Munro
Fix DROP {DATABASE,TABLESPACE} on Windows. Previously, it was possible for DROP DATABASE, DROP TABLESPACE and ALTER DATABASE SET TABLESPACE to fail because other backends still had file handles open for dropped tables. Windows won't allow a directory containing unlinked-but-still-open files to

pgsql: Track LLVM 14 API changes, up to 2022-01-30.

2022-02-03 Thread Thomas Munro
Track LLVM 14 API changes, up to 2022-01-30. Tested with LLVM 11, LLVM 13 and LLVM's main branch at commit 8d8fce87bbd5. There are still some deprecation warnings that will need to be sorted out, but this may be enough to turn "seawasp" green again. Like commit e6a76002, done on master only for

pgsql: Fix recovery conflict in 027_stream_regress.pl.

2022-02-01 Thread Thomas Munro
Fix recovery conflict in 027_stream_regress.pl. To avoid "ERROR: canceling statement due to conflict with recovery", as seen on a couple of slower build farm animals, crank max_standby_streaming_delay right up. In passing, adjust a configuration option that accidentally used a non-standard

pgsql: Add new simple TAP test for tablespaces, attempt II.

2022-01-20 Thread Thomas Munro
Add new simple TAP test for tablespaces, attempt II. See commit message for d1511fe1b040853f6e10d353e56b42bb96ae239d. This new version attempts to fix path translation problem on MSYS/Windows. Discussion: https://postgr.es/m/20220117055326.GD756210%40rfd.leadboat.com Branch -- master

Re: pgsql: Test replay of regression tests, attempt II.

2022-01-20 Thread Thomas Munro
On Thu, Jan 20, 2022 at 6:24 PM Andres Freund wrote: > I wonder if the easiest way to make this test reliable would be to make the > table a temporary one? That now uses very aggressive horizons, there's no > bgwriter that could pin the page, etc. Good idea, thanks. I pushed that minimal

Re: pgsql: TAP tests: check for postmaster.pid anyway when "pg_ctl start" f

2022-01-20 Thread Thomas Munro
On Thu, Jan 20, 2022 at 10:29 AM Tom Lane wrote: > TAP tests: check for postmaster.pid anyway when "pg_ctl start" fails. > > "pg_ctl start" might start a new postmaster and then return failure > anyway, for example if PGCTLTIMEOUT is exceeded. If there is a > postmaster there, it's still

<    1   2   3   4   5   6   7   8   9   10   >