Re: pgsql: Use SIGURG rather than SIGUSR1 for latches.

2021-04-15 Thread Thomas Munro
On Thu, Apr 15, 2021 at 2:26 PM Tom Lane wrote: > It's possible that that argument doesn't apply to the way SIGURG is used > in this patch, but I don't see a good reason to ignore the convention of > setting up the handler this way. Yeah, will fix. I don't think there is a bug here given the

pgsql: Fix potential SSI hazard in heap_update().

2021-04-12 Thread Thomas Munro
Fix potential SSI hazard in heap_update(). Commit 6f38d4dac38 failed to heed a warning about the stability of the value pointed to by "otid". The caller is allowed to pass in a pointer to newtup->t_self, which will be updated during the execution of the function. Instead, the SSI check should

pgsql: Fix potential SSI hazard in heap_update().

2021-04-12 Thread Thomas Munro
Fix potential SSI hazard in heap_update(). Commit 6f38d4dac38 failed to heed a warning about the stability of the value pointed to by "otid". The caller is allowed to pass in a pointer to newtup->t_self, which will be updated during the execution of the function. Instead, the SSI check should

pgsql: Make new GUC short descriptions more consistent.

2021-04-09 Thread Thomas Munro
Make new GUC short descriptions more consistent. Reported-by: Daniel Westermann (DWE) Discussion: https://postgr.es/m/GV0P278MB0483490FEAC879DCA5ED583DD2739%40GV0P278MB0483.CHEP278.PROD.OUTLOOK.COM Branch -- master Details ---

pgsql: Doc: Review for "Optionally prefetch referenced data in recovery

2021-04-09 Thread Thomas Munro
Doc: Review for "Optionally prefetch referenced data in recovery." Typos, corrections and language improvements in the docs, and a few in code comments too. Reported-by: Justin Pryzby Discussion: https://postgr.es/m/20210409033703.GP6592%40telsasoft.com Branch -- master Details ---

pgsql: Remove duplicate typedef.

2021-04-08 Thread Thomas Munro
Remove duplicate typedef. Thinko in commit 323cbe7c, per complaint from BF animal locust's older GCC compiler. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/34399a670a1c559ef8a355ed25d090d73e400ad4 Modified Files -- src/include/replication/logical.h

pgsql: Add circular WAL decoding buffer.

2021-04-08 Thread Thomas Munro
Add circular WAL decoding buffer. Teach xlogreader.c to decode its output into a circular buffer, to support optimizations based on looking ahead. * XLogReadRecord() works as before, consuming records one by one, and allowing them to be examined via the traditional XLogRecGetXXX() macros.

pgsql: Remove read_page callback from XLogReader.

2021-04-08 Thread Thomas Munro
) Reviewed-by: Antonin Houska Reviewed-by: Alvaro Herrera Reviewed-by: Takashi Menjo Reviewed-by: Andres Freund Reviewed-by: Thomas Munro Discussion: https://postgr.es/m/20190418.210257.43726183.horiguchi.kyotaro%40lab.ntt.co.jp Branch -- master Details --- https://git.postgresql.org/pg

pgsql: Optionally prefetch referenced data in recovery.

2021-04-08 Thread Thomas Munro
Optionally prefetch referenced data in recovery. Introduce a new GUC recovery_prefetch, disabled by default. When enabled, look ahead in the WAL and try to initiate asynchronous reading of referenced data blocks that are not yet cached in our buffer pool. For now, this is done with

pgsql: Provide ReadRecentBuffer() to re-pin buffers by ID.

2021-04-07 Thread Thomas Munro
Provide ReadRecentBuffer() to re-pin buffers by ID. If you know the ID of a buffer that recently held a block that you would like to pin, this function can be used check if it's still there. It can be used to avoid a second lookup in the buffer mapping table after PrefetchBuffer() reports a

pgsql: Detect POLLHUP/POLLRDHUP while running queries.

2021-04-02 Thread Thomas Munro
. Author: Sergey Cherkashin Author: Thomas Munro Reviewed-by: Thomas Munro Reviewed-by: Tatsuo Ishii Reviewed-by: Konstantin Knizhnik Reviewed-by: Zhihong Yu Reviewed-by: Andres Freund Reviewed-by: Maksim Milyutin Reviewed-by: Tsunakawa, Takayuki/綱川 貴之 Reviewed-by: Tom Lane (much earlier

pgsql: Provide recovery_init_sync_method=syncfs.

2021-03-19 Thread Thomas Munro
Provide recovery_init_sync_method=syncfs. Since commit 2ce439f3 we have opened every file in the data directory and called fsync() at the start of crash recovery. This can be very slow if there are many files, leading to field complaints of systems taking minutes or even hours to begin crash

pgsql: Revert "Fix race in Parallel Hash Join batch cleanup."

2021-03-17 Thread Thomas Munro
Revert "Fix race in Parallel Hash Join batch cleanup." This reverts commit 0129c56fbe5c26bfec91bfc2c8a3b8818f441d6e. Discussion: https://postgr.es/m/CA%2BhUKGJmcqAE3MZeDCLLXa62cWM0AJbKmp2JrJYaJ86bz36LFA%40mail.gmail.com Branch -- REL_11_STABLE Details ---

pgsql: Revert "Fix race in Parallel Hash Join batch cleanup."

2021-03-17 Thread Thomas Munro
Revert "Fix race in Parallel Hash Join batch cleanup." This reverts commit 8fa2478b407ef867d501fafcdea45fd827f70799. Discussion: https://postgr.es/m/CA%2BhUKGJmcqAE3MZeDCLLXa62cWM0AJbKmp2JrJYaJ86bz36LFA%40mail.gmail.com Branch -- REL_12_STABLE Details ---

pgsql: Revert "Fix race in Parallel Hash Join batch cleanup."

2021-03-17 Thread Thomas Munro
Revert "Fix race in Parallel Hash Join batch cleanup." This reverts commit 4e0f0995e923948631c4114ab353b256b51b58ad. Discussion: https://postgr.es/m/CA%2BhUKGJmcqAE3MZeDCLLXa62cWM0AJbKmp2JrJYaJ86bz36LFA%40mail.gmail.com Branch -- REL_13_STABLE Details ---

pgsql: Revert "Fix race in Parallel Hash Join batch cleanup."

2021-03-17 Thread Thomas Munro
Revert "Fix race in Parallel Hash Join batch cleanup." This reverts commit 378802e3713c6c0fce31d2390c134cd5d7c30157. This reverts commit 3b8981b6e1a2aea0f18384c803e21e9391de669a. Discussion: https://postgr.es/m/CA%2BhUKGJmcqAE3MZeDCLLXa62cWM0AJbKmp2JrJYaJ86bz36LFA%40mail.gmail.com Branch

pgsql: Update the names of Parallel Hash Join phases.

2021-03-16 Thread Thomas Munro
Update the names of Parallel Hash Join phases. Commit 3048898e dropped -ING from some wait event names that correspond to barrier phases. Update the phases' names to match. While we're here making cosmetic changes, also rename "DONE" to "FREE". That pairs better with "ALLOCATE", and describes

pgsql: Fix race in Parallel Hash Join batch cleanup.

2021-03-16 Thread Thomas Munro
Fix race in Parallel Hash Join batch cleanup. With very unlucky timing and parallel_leader_participation off, PHJ could attempt to access per-batch state just as it was being freed. There was code intended to prevent that by checking for a cleared pointer, but it was buggy. Fix, by introducing

pgsql: Fix race in Parallel Hash Join batch cleanup.

2021-03-16 Thread Thomas Munro
Fix race in Parallel Hash Join batch cleanup. With very unlucky timing and parallel_leader_participation off, PHJ could attempt to access per-batch state just as it was being freed. There was code intended to prevent that by checking for a cleared pointer, but it was buggy. Fix, by introducing

pgsql: Fix race in Parallel Hash Join batch cleanup.

2021-03-16 Thread Thomas Munro
Fix race in Parallel Hash Join batch cleanup. With very unlucky timing and parallel_leader_participation off, PHJ could attempt to access per-batch state just as it was being freed. There was code intended to prevent that by checking for a cleared pointer, but it was buggy. Fix, by introducing

pgsql: Fix race in Parallel Hash Join batch cleanup.

2021-03-16 Thread Thomas Munro
Fix race in Parallel Hash Join batch cleanup. With very unlucky timing and parallel_leader_participation off, PHJ could attempt to access per-batch state just as it was being freed. There was code intended to prevent that by checking for a cleared pointer, but it was buggy. Fix, by introducing

pgsql: Fix transaction.sql tests in higher isolation levels.

2021-03-16 Thread Thomas Munro
Fix transaction.sql tests in higher isolation levels. It seems like a useful sanity check to be able to run "installcheck" against a cluster running with default_transaction_level set to serializable or repeatable read. Only one thing currently fails in those configurations, so let's fix that.

pgsql: Enable parallelism in REFRESH MATERIALIZED VIEW.

2021-03-16 Thread Thomas Munro
. Since REFRESH always inserts into a freshly created table before later merging or swapping the data into place with separate operations, we can enable such plans here too. Author: Bharath Rupireddy Reviewed-by: Hou, Zhijie Reviewed-by: Luc Vlaming Reviewed-by: Thomas Munro Discussion: https

pgsql: Drop SERIALIZABLE workaround from parallel query tests.

2021-03-15 Thread Thomas Munro
Drop SERIALIZABLE workaround from parallel query tests. SERIALIZABLE no longer inhibits parallelism, so we can drop some outdated workarounds and comments from regression tests. The change came in release 12, commit bb16aba5, but it's not really worth back-patching. Also fix a typo.

pgsql: Fix new pthread code to respect --disable-thread-safety.

2021-03-12 Thread Thomas Munro
Fix new pthread code to respect --disable-thread-safety. Don't try to compile src/port/pthread_barrier_wait.c if we opted out of threads at configure time. Revealed by build farm member gaur, which can't compile this code because of problems with its pthread implementation. It shouldn't be

pgsql: Specialize checkpointer sort functions.

2021-03-12 Thread Thomas Munro
Specialize checkpointer sort functions. When sorting a potentially large number of dirty buffers, the checkpointer can benefit from a faster sort routine. One reported improvement on a large buffer pool system was 1.4s -> 0.6s. Reviewed-by: Andres Freund Discussion:

pgsql: Add condition variable for recovery resume.

2021-03-11 Thread Thomas Munro
Add condition variable for recovery resume. Replace a sleep loop with a CV, to get a fast reaction time when recovery is resumed or the postmaster exits via standard infrastructure. Unfortunately we still need to wake up every second to perform extra polling during the recovery pause loop.

pgsql: Add condition variable for walreceiver shutdown.

2021-03-11 Thread Thomas Munro
Add condition variable for walreceiver shutdown. Use this new CV to wait for walreceiver shutdown without a sleep/poll loop, while also benefiting from standard postmaster death handling. Discussion: https://postgr.es/m/CA%2BhUKGK1607VmtrDUHQXrsooU%3Dap4g4R2yaoByWOOA3m8xevUQ%40mail.gmail.com

pgsql: Poll postmaster less frequently in recovery.

2021-03-11 Thread Thomas Munro
Poll postmaster less frequently in recovery. Since commits 9f095299 and f98b8476 we don't poll the postmaster pipe at all during crash recovery on Linux and FreeBSD, but on other operating systems we were still doing it for every WAL record. Do it less frequently on operating systems where

pgsql: Minor modernization for README.barrier.

2021-03-11 Thread Thomas Munro
Minor modernization for README.barrier. Itanium is very uncommon and being discontinued. ARM is everywhere. Prefer ARM as an example of an architecture with weak memory ordering. Branch -- master Details ---

pgsql: Improve comment for struct BufferDesc.

2021-03-10 Thread Thomas Munro
Improve comment for struct BufferDesc. Add a note that per-buffer I/O condition variables currently live outside the BufferDesc struct. Follow-up for commit d8725104. Reported-by: Julien Rouhaud Discussion: https://postgr.es/m/20210311031118.hucytmrgwlktjxgq%40nol Branch -- master

pgsql: Replace buffer I/O locks with condition variables.

2021-03-10 Thread Thomas Munro
. Previously, any waiters would wake up (because the I/O lock was automatically released) and then busy-loop until AbortBufferIO() cleared BM_IO_IN_PROGRESS. 3. LWLockMinimallyPadded is removed, as it would now be unused. Author: Robert Haas Reviewed-by: Thomas Munro Reviewed-by: Julien Rouhaud

Re: pgsql: pgbench: Refactor thread portability support.

2021-03-10 Thread Thomas Munro
On Wed, Mar 10, 2021 at 9:49 PM Thomas Munro wrote: > On Wed, Mar 10, 2021 at 9:38 PM Thomas Munro wrote: > > On Wed, Mar 10, 2021 at 8:51 PM Thomas Munro wrote: > > >... I'm now contemplating what's up with hoverfly (some AIX header > > > problem)... > > >

pgsql: Fix another portability bug in recent pgbench commit.

2021-03-10 Thread Thomas Munro
Fix another portability bug in recent pgbench commit. Commit 547f04e7 produced errors on AIX/xlc while building plpython. The new code appears to be incompatible with the hack installed by commit a11cf433. Without access to an AIX system to check, my guess is that _POSIX_C_SOURCE may be

Re: pgsql: pgbench: Refactor thread portability support.

2021-03-10 Thread Thomas Munro
On Wed, Mar 10, 2021 at 9:38 PM Thomas Munro wrote: > On Wed, Mar 10, 2021 at 8:51 PM Thomas Munro wrote: > >... I'm now contemplating what's up with hoverfly (some AIX header > > problem)... > > Pushed something for the other two failures seen, but for this one I'm >

Re: pgsql: pgbench: Refactor thread portability support.

2021-03-10 Thread Thomas Munro
On Wed, Mar 10, 2021 at 8:51 PM Thomas Munro wrote: >... I'm now contemplating what's up with hoverfly (some AIX header > problem)... Pushed something for the other two failures seen, but for this one I'm temporarily stumped. We managed to compile all kinds of stuff that surely in

pgsql: Try to fix portability bugs in recent pgbench commits.

2021-03-10 Thread Thomas Munro
Try to fix portability bugs in recent pgbench commits. 1. pg_time_usec_t needs to be printed with INT64_FORMAT, not %ld, or 32 bit systems complain, per lapwing. 2. Some Windows compilers didn't like a thread function not marked with __stdcall, per whelk; let's see if this fixes the problem.

Re: pgsql: pgbench: Refactor thread portability support.

2021-03-09 Thread Thomas Munro
On Wed, Mar 10, 2021 at 8:14 PM Thomas Munro wrote: > On Wed, Mar 10, 2021 at 8:10 PM Michael Paquier wrote: > > On Wed, Mar 10, 2021 at 04:52:01AM +0000, Thomas Munro wrote: > > > pgbench: Refactor thread portability support. > > > > > > Instead of maintain

Re: pgsql: pgbench: Refactor thread portability support.

2021-03-09 Thread Thomas Munro
On Wed, Mar 10, 2021 at 8:10 PM Michael Paquier wrote: > On Wed, Mar 10, 2021 at 04:52:01AM +0000, Thomas Munro wrote: > > pgbench: Refactor thread portability support. > > > > Instead of maintaining an incomplete emulation of POSIX threads for > > Windows, let's use an

pgsql: pgbench: Synchronize client threads.

2021-03-09 Thread Thomas Munro
pgbench: Synchronize client threads. Wait until all pgbench threads are connected before benchmarking begins. This fixes a problem where some connections could take a very long time to be established because of lock contention from earlier connections, making results unstable and bogus with high

pgsql: Add missing pthread_barrier_t.

2021-03-09 Thread Thomas Munro
Add missing pthread_barrier_t. Supply a simple implementation of the missing pthread_barrier_t type and functions, for macOS. Discussion: https://postgr.es/m/20200227180100.zyvjwzcpiokfsqm2%40alap3.anarazel.de Branch -- master Details ---

pgsql: Add sort_template.h for making sort functions.

2021-03-02 Thread Thomas Munro
Add sort_template.h for making sort functions. Move our qsort implementation into a header that can be used to define specialized functions for better performance and reduced duplication. Reviewed-by: Daniel Gustafsson Discussion:

pgsql: Use sort_template.h for qsort() and qsort_arg().

2021-03-02 Thread Thomas Munro
Use sort_template.h for qsort() and qsort_arg(). Reduce duplication by using the new template. Reviewed-by: Daniel Gustafsson Discussion: https://postgr.es/m/CA%2BhUKGJ2-eaDqAum5bxhpMNhvuJmRDZxB_Tow0n-gse%2BHG0Yig%40mail.gmail.com Branch -- master Details ---

pgsql: Use sort_template.h for qsort_tuple() and qsort_ssup().

2021-03-02 Thread Thomas Munro
Use sort_template.h for qsort_tuple() and qsort_ssup(). Replace the Perl code previously used to generate specialized sort functions with sort_template.h. Reviewed-by: Daniel Gustafsson Discussion: https://postgr.es/m/CA%2BhUKGJ2-eaDqAum5bxhpMNhvuJmRDZxB_Tow0n-gse%2BHG0Yig%40mail.gmail.com

pgsql: Remove obsolete comment for WaitForProcSignalBarrier().

2021-03-01 Thread Thomas Munro
Remove obsolete comment for WaitForProcSignalBarrier(). Commit 814f1d8b removed the behavior described. Reported-by: Amit Kapila Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/bd69ddfcdbf650f11af4317f3f6686c012cf66d0 Modified Files --

pgsql: Use condition variables for ProcSignalBarriers.

2021-02-28 Thread Thomas Munro
Use condition variables for ProcSignalBarriers. Instead of a poll/sleep loop, use a condition variable for precise wake-up whenever a backend's pss_barrierGeneration advances. Discussion: https://postgr.es/m/ca+hukgldemy2gbm80kz20gte6hnvwoere8kwcjk6-u56ost...@mail.gmail.com Branch --

pgsql: Allow condition variables to be used in interrupt code.

2021-02-28 Thread Thomas Munro
Allow condition variables to be used in interrupt code. Adjust the condition variable sleep loop to work correctly when code reached by its internal CHECK_FOR_INTERRUPTS() call interacts with another condition variable. There are no such cases currently, but a proposed patch would do this.

pgsql: Introduce symbolic names for FeBeWaitSet positions.

2021-02-28 Thread Thomas Munro
Introduce symbolic names for FeBeWaitSet positions. Previously we used 0 and 1 to refer to the socket and latch in far flung parts of the tree, without any explanation. Also use PGINVALID_SOCKET rather than -1 in a couple of places that didn't already do that. Reviewed-by: Kyotaro Horiguchi

pgsql: Use FeBeWaitSet for walsender.c.

2021-02-28 Thread Thomas Munro
Use FeBeWaitSet for walsender.c. This avoids the need to set up and tear down a fresh WaitEventSet every time we need need to wait. We have to add an explicit exit on postmaster exit (FeBeWaitSet isn't set up to do that automatically), so move the code to do that into a new function to avoid

pgsql: Use EVFILT_SIGNAL for kqueue latches.

2021-02-28 Thread Thomas Munro
Use EVFILT_SIGNAL for kqueue latches. Cut down on system calls and other overheads by waiting for SIGURG explicitly with kqueue instead of using a signal handler and self-pipe. Affects *BSD and macOS systems. This leaves only the poll implementation with a signal handler and the traditional

pgsql: Use signalfd(2) for epoll latches.

2021-02-28 Thread Thomas Munro
Use signalfd(2) for epoll latches. Cut down on system calls and other overheads by reading from a signalfd instead of using a signal handler and self-pipe. Affects Linux sytems, and possibly others including illumos that implement the Linux epoll and signalfd interfaces. Reviewed-by: Andres

pgsql: Optimize latches to send fewer signals.

2021-02-28 Thread Thomas Munro
Optimize latches to send fewer signals. Don't send signals to processes that aren't sleeping. Author: Andres Freund Discussion: https://postgr.es/m/ca+hukgjjxpdpzbe0a3hyuywbvazuc89yx3jk9rfzgfv_khu...@mail.gmail.com Branch -- master Details ---

pgsql: Use SIGURG rather than SIGUSR1 for latches.

2021-02-28 Thread Thomas Munro
Use SIGURG rather than SIGUSR1 for latches. Traditionally, SIGUSR1 has been overloaded for ad-hoc signals, procsignal.c signals and latch.c wakeups. Move that last use over to a new dedicated signal. SIGURG is normally used to report out-of-band socket data, but PostgreSQL doesn't use that

pgsql: Remove latch.c workaround for Linux < 2.6.27.

2021-02-28 Thread Thomas Munro
Remove latch.c workaround for Linux < 2.6.27. Commit 82ebbeb0 added a workaround for systems with no epoll_create1() and EPOLL_CLOEXEC. Linux < 2.6.27 and glibc < 2.9 are long gone. Now seems like a good time to drop the extra code, because otherwise we'd have to add similar already-dead

Re: pgsql: pg_collation_actual_version() -> pg_collation_current_version().

2021-02-25 Thread Thomas Munro
On Thu, Feb 25, 2021 at 10:49 PM Peter Eisentraut wrote: > Seeing that explanation, I think that's even more of a reason to avoid > the name "current" and use something strikingly different. > > In any case, this function name has been around for some years now and > renaming it just for taste

pgsql: Revert "pg_collation_actual_version() -> pg_collation_current_ve

2021-02-25 Thread Thomas Munro
Revert "pg_collation_actual_version() -> pg_collation_current_version()." This reverts commit 9cf184cc0599b6e65e7e5ecd9d91cd42e278bcd8. Name change less well received than anticipated. Discussion: https://postgr.es/m/afcfb97e-88a1-a540-db95-6c573b93bc2b%40eisentraut.org Branch -- master

Re: pgsql: pg_collation_actual_version() -> pg_collation_current_version().

2021-02-22 Thread Thomas Munro
On Tue, Feb 23, 2021 at 7:03 PM Peter Eisentraut wrote: > On 22.02.21 12:28, Thomas Munro wrote: > > pg_collation_actual_version() -> pg_collation_current_version(). > > > > The new name seems a bit more natural. > > > > Discussion: https://postgr.es/m/2021011

pgsql: Hide internal error for pg_collation_actual_version().

2021-02-22 Thread Thomas Munro
Hide internal error for pg_collation_actual_version(). Instead of an unsightly internal "cache lookup failed" message, just return NULL for bad OIDs, as is the convention for other similar things. Reported-by: Justin Pryzby Reviewed-by: Michael Paquier Discussion:

pgsql: Refactor get_collation_current_version().

2021-02-22 Thread Thomas Munro
Refactor get_collation_current_version(). The code paths for three different OSes finished up with three different ways of excluding C[.xxx] and POSIX from consideration. Merge them. Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/20210117215940.GE8560%40telsasoft.com Branch

pgsql: Tab-complete CREATE COLLATION.

2021-02-22 Thread Thomas Munro
Tab-complete CREATE COLLATION. Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/20210117215940.GE8560%40telsasoft.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/5bc09a74719dfeb6c4cebb311b81385c508459a8 Modified Files --

pgsql: pg_collation_actual_version() -> pg_collation_current_version().

2021-02-22 Thread Thomas Munro
pg_collation_actual_version() -> pg_collation_current_version(). The new name seems a bit more natural. Discussion: https://postgr.es/m/20210117215940.GE8560%40telsasoft.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/9cf184cc0599b6e65e7e5ecd9d91cd42e278bcd8

pgsql: Remove outdated reference to RAID spindles.

2021-02-21 Thread Thomas Munro
Remove outdated reference to RAID spindles. Commit b09ff536 left behind some outdated advice in the long_desc field of the GUC "effective_io_concurrency". Remove it. Back-patch to 13. Reported-by: Andrew Gierth Reviewed-by: Julien Rouhaud Discussion:

pgsql: Remove outdated reference to RAID spindles.

2021-02-21 Thread Thomas Munro
Remove outdated reference to RAID spindles. Commit b09ff536 left behind some outdated advice in the long_desc field of the GUC "effective_io_concurrency". Remove it. Back-patch to 13. Reported-by: Andrew Gierth Reviewed-by: Julien Rouhaud Discussion:

pgsql: Fix compiler warning in back branches (9.6, 10).

2021-02-15 Thread Thomas Munro
Fix compiler warning in back branches (9.6, 10). Back-patch a tiny bit of commit fbb2e9a0 into 9.6 and 10, to silence an uninitialized variable warning from GCC 10.2. Seen on buildfarm member handfish, and my own development workflow where I like to use -Werror. Discussion:

pgsql: Fix compiler warning in back branches (9.6, 10).

2021-02-15 Thread Thomas Munro
Fix compiler warning in back branches (9.6, 10). Back-patch a tiny bit of commit fbb2e9a0 into 9.6 and 10, to silence an uninitialized variable warning from GCC 10.2. Seen on buildfarm member handfish, and my own development workflow where I like to use -Werror. Discussion:

pgsql: Default to wal_sync_method=fdatasync on FreeBSD.

2021-02-14 Thread Thomas Munro
Default to wal_sync_method=fdatasync on FreeBSD. FreeBSD 13 gained O_DSYNC, which would normally cause wal_sync_method to choose open_datasync as its default value. That may not be a good choice for all systems, and performs worse than fdatasync in some scenarios. Let's preserve the existing

pgsql: Default to wal_sync_method=fdatasync on FreeBSD.

2021-02-14 Thread Thomas Munro
Default to wal_sync_method=fdatasync on FreeBSD. FreeBSD 13 gained O_DSYNC, which would normally cause wal_sync_method to choose open_datasync as its default value. That may not be a good choice for all systems, and performs worse than fdatasync in some scenarios. Let's preserve the existing

pgsql: Default to wal_sync_method=fdatasync on FreeBSD.

2021-02-14 Thread Thomas Munro
Default to wal_sync_method=fdatasync on FreeBSD. FreeBSD 13 gained O_DSYNC, which would normally cause wal_sync_method to choose open_datasync as its default value. That may not be a good choice for all systems, and performs worse than fdatasync in some scenarios. Let's preserve the existing

pgsql: Default to wal_sync_method=fdatasync on FreeBSD.

2021-02-14 Thread Thomas Munro
Default to wal_sync_method=fdatasync on FreeBSD. FreeBSD 13 gained O_DSYNC, which would normally cause wal_sync_method to choose open_datasync as its default value. That may not be a good choice for all systems, and performs worse than fdatasync in some scenarios. Let's preserve the existing

pgsql: Default to wal_sync_method=fdatasync on FreeBSD.

2021-02-14 Thread Thomas Munro
Default to wal_sync_method=fdatasync on FreeBSD. FreeBSD 13 gained O_DSYNC, which would normally cause wal_sync_method to choose open_datasync as its default value. That may not be a good choice for all systems, and performs worse than fdatasync in some scenarios. Let's preserve the existing

pgsql: Default to wal_sync_method=fdatasync on FreeBSD.

2021-02-14 Thread Thomas Munro
Default to wal_sync_method=fdatasync on FreeBSD. FreeBSD 13 gained O_DSYNC, which would normally cause wal_sync_method to choose open_datasync as its default value. That may not be a good choice for all systems, and performs worse than fdatasync in some scenarios. Let's preserve the existing

pgsql: Use pg_pwrite() in pg_test_fsync.

2021-02-14 Thread Thomas Munro
Use pg_pwrite() in pg_test_fsync. For consistency with the PostgreSQL behavior this test program is intended to simulate, use pwrite() instead of lseek() + write(). Also fix the final "non-sync" test, which was opening and closing the file for every write. Discussion:

pgsql: Hold interrupts while running dsm_detach() callbacks.

2021-02-14 Thread Thomas Munro
Hold interrupts while running dsm_detach() callbacks. While cleaning up after a parallel query or parallel index creation that created temporary files, we could be interrupted by a statement timeout. The error handling path would then fail to clean up the files when it ran dsm_detach() again,

pgsql: Hold interrupts while running dsm_detach() callbacks.

2021-02-14 Thread Thomas Munro
Hold interrupts while running dsm_detach() callbacks. While cleaning up after a parallel query or parallel index creation that created temporary files, we could be interrupted by a statement timeout. The error handling path would then fail to clean up the files when it ran dsm_detach() again,

pgsql: Hold interrupts while running dsm_detach() callbacks.

2021-02-14 Thread Thomas Munro
Hold interrupts while running dsm_detach() callbacks. While cleaning up after a parallel query or parallel index creation that created temporary files, we could be interrupted by a statement timeout. The error handling path would then fail to clean up the files when it ran dsm_detach() again,

pgsql: Hold interrupts while running dsm_detach() callbacks.

2021-02-14 Thread Thomas Munro
Hold interrupts while running dsm_detach() callbacks. While cleaning up after a parallel query or parallel index creation that created temporary files, we could be interrupted by a statement timeout. The error handling path would then fail to clean up the files when it ran dsm_detach() again,

pgsql: Hold interrupts while running dsm_detach() callbacks.

2021-02-14 Thread Thomas Munro
Hold interrupts while running dsm_detach() callbacks. While cleaning up after a parallel query or parallel index creation that created temporary files, we could be interrupted by a statement timeout. The error handling path would then fail to clean up the files when it ran dsm_detach() again,

pgsql: Hold interrupts while running dsm_detach() callbacks.

2021-02-14 Thread Thomas Munro
Hold interrupts while running dsm_detach() callbacks. While cleaning up after a parallel query or parallel index creation that created temporary files, we could be interrupted by a statement timeout. The error handling path would then fail to clean up the files when it ran dsm_detach() again,

pgsql: ReadNewTransactionId() -> ReadNextTransactionId().

2021-02-14 Thread Thomas Munro
ReadNewTransactionId() -> ReadNextTransactionId(). The new name conveys the effect better, is more consistent with similar functions ReadNextMultiXactId(), ReadNextFullTransactionId(), and matches the name of the variable that it reads. Reported-by: Peter Geoghegan Discussion:

pgsql: Tab-complete CREATE DATABASE ... LOCALE.

2021-02-04 Thread Thomas Munro
Tab-complete CREATE DATABASE ... LOCALE. Author: Ian Lawrence Barwick Discussion: https://postgr.es/m/CAB8KJ%3Dh0XO2CB4QbLBc1Tm9Bg5wzSGQtT-eunaCmrghJp4nqdA%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/e1c02d92aee30328316c309c3c2f305d77f231a2

pgsql: Fix sample output of EXPLAIN ANALYZE.

2021-01-20 Thread Thomas Munro
Fix sample output of EXPLAIN ANALYZE. Since commit f0f13a3a08b2757997410f3a1c38bdc22973c525, we estimate ModifyTable paths without a RETURNING clause differently. Update an example from the manual that showed the old behavior. Author: Takayuki Tsunakawa Reviewed-by: Laurenz Albe Discussion:

pgsql: Minor header cleanup for the new iovec code.

2021-01-13 Thread Thomas Munro
Minor header cleanup for the new iovec code. Remove redundant function declaration and improve header comment in pg_iovec.h. Move the new declaration in fd.h next to a group of more similar functions. Branch -- master Details ---

pgsql: Move our p{read,write}v replacements into their own files.

2021-01-13 Thread Thomas Munro
Move our p{read,write}v replacements into their own files. macOS's ranlib issued a warning about an empty pread.o file with the previous arrangement, on systems new enough to require no replacement functions. Let's go back to using configure's AC_REPLACE_FUNCS system to build and include each .o

pgsql: Don't use elog() in src/port/pwrite.c.

2021-01-12 Thread Thomas Munro
Don't use elog() in src/port/pwrite.c. Nothing broke because of this oversight yet, but it would fail to link if we tried to use pg_pwrite() in frontend code on a system that lacks pwrite(). Use an assertion instead. Also pgindent while here. Discussion:

pgsql: Fix function prototypes in dependency.h.

2021-01-11 Thread Thomas Munro
Fix function prototypes in dependency.h. Commit 257836a7 accidentally deleted a couple of redundant-but-conventional "extern" keywords on function prototypes. Put them back. Reported-by: Alvaro Herrera Branch -- master Details ---

Re: pgsql: Track collation versions for indexes.

2021-01-11 Thread Thomas Munro
On Tue, Jan 12, 2021 at 10:37 AM Alvaro Herrera wrote: > This was a strange change ... Is this just an inadvertent slip-up? > -extern long changeDependenciesOf(Oid classId, Oid oldObjectId, > +long changeDependenciesOf(Oid classId, Oid oldObjectId, Huh, definitely not done on purpose. Will

pgsql: Use vectored I/O to fill new WAL segments.

2021-01-10 Thread Thomas Munro
Use vectored I/O to fill new WAL segments. Instead of making many block-sized write() calls to fill a new WAL file with zeroes, make a smaller number of pwritev() calls (or various emulations). The actual number depends on the OS's IOV_MAX, which PG_IOV_MAX currently caps at 32. That means

pgsql: Provide pg_preadv() and pg_pwritev().

2021-01-10 Thread Thomas Munro
Provide pg_preadv() and pg_pwritev(). Provide synchronous vectored file I/O routines. These map to preadv() and pwritev(), with fallback implementations for systems that don't have them. Also provide a wrapper pg_pwritev_with_retry() that automatically retries on short writes. Reviewed-by: Tom

pgsql: Replace remaining uses of "whitelist".

2021-01-04 Thread Thomas Munro
Replace remaining uses of "whitelist". Instead describe the action that the list effects, or just use "list" where the meaning is obvious from context. Author: Dagfinn Ilmari Mannsåker Discussion: https://postgr.es/m/20200615182235.x7lch5n6kcjq4aue%40alap3.anarazel.de Branch -- master

pgsql: pgindent: whitelist/blacklist -> additional/excluded.

2021-01-04 Thread Thomas Munro
pgindent: whitelist/blacklist -> additional/excluded. Author: Dagfinn Ilmari Mannsåker Discussion: https://postgr.es/m/20200615182235.x7lch5n6kcjq4aue%40alap3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/fe05b6b620066aec313c43b6b4d6c169d0a346f7

pgsql: Rename "enum blacklist" to "uncommitted enums".

2021-01-04 Thread Thomas Munro
Rename "enum blacklist" to "uncommitted enums". We agreed to remove this terminology and use something more descriptive. Discussion: https://postgr.es/m/20200615182235.x7lch5n6kcjq4aue%40alap3.anarazel.de Branch -- master Details ---

pgsql: Remove unused function prototypes.

2021-01-04 Thread Thomas Munro
Remove unused function prototypes. Cleanup for commit dee663f7. Reported-by: Tomas Vondra Discussion: https://postgr.es/m/CA+hUKGLJ=84yt+nvhkeedauutvhmfq9i-n7k_o50jmq6rpj...@mail.gmail.com Branch -- master Details ---

pgsql: Use truncate(2) where appropriate.

2020-11-30 Thread Thomas Munro
Use truncate(2) where appropriate. When truncating files by name, use truncate(2). Windows hasn't got it, so keep our previous coding based on ftruncate(2) as a fallback. Discussion: https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org Branch -- master Details ---

pgsql: Free disk space for dropped relations on commit.

2020-11-30 Thread Thomas Munro
Free disk space for dropped relations on commit. When committing a transaction that dropped a relation, we previously truncated only the first segment file to free up disk space (the one that won't be unlinked until the next checkpoint). Truncate higher numbered segments too, even though we

pgsql: Free disk space for dropped relations on commit.

2020-11-30 Thread Thomas Munro
Free disk space for dropped relations on commit. When committing a transaction that dropped a relation, we previously truncated only the first segment file to free up disk space (the one that won't be unlinked until the next checkpoint). Truncate higher numbered segments too, even though we

pgsql: Free disk space for dropped relations on commit.

2020-11-30 Thread Thomas Munro
Free disk space for dropped relations on commit. When committing a transaction that dropped a relation, we previously truncated only the first segment file to free up disk space (the one that won't be unlinked until the next checkpoint). Truncate higher numbered segments too, even though we

pgsql: Free disk space for dropped relations on commit.

2020-11-30 Thread Thomas Munro
Free disk space for dropped relations on commit. When committing a transaction that dropped a relation, we previously truncated only the first segment file to free up disk space (the one that won't be unlinked until the next checkpoint). Truncate higher numbered segments too, even though we

pgsql: Free disk space for dropped relations on commit.

2020-11-30 Thread Thomas Munro
Free disk space for dropped relations on commit. When committing a transaction that dropped a relation, we previously truncated only the first segment file to free up disk space (the one that won't be unlinked until the next checkpoint). Truncate higher numbered segments too, even though we

pgsql: Free disk space for dropped relations on commit.

2020-11-30 Thread Thomas Munro
Free disk space for dropped relations on commit. When committing a transaction that dropped a relation, we previously truncated only the first segment file to free up disk space (the one that won't be unlinked until the next checkpoint). Truncate higher numbered segments too, even though we

pgsql: Free disk space for dropped relations on commit.

2020-11-30 Thread Thomas Munro
Free disk space for dropped relations on commit. When committing a transaction that dropped a relation, we previously truncated only the first segment file to free up disk space (the one that won't be unlinked until the next checkpoint). Truncate higher numbered segments too, even though we

<    2   3   4   5   6   7   8   9   10   11   >