pgsql: Fix WaitLatch(NULL) on Windows.

2020-11-24 Thread Thomas Munro
Fix WaitLatch(NULL) on Windows. Further to commit 733fa9aa, on Windows when a latch is triggered but we aren't currently waiting for it, we need to locate the latch's HANDLE rather than calling ResetEvent(NULL). Author: Kyotaro Horiguchi Reported-by: Ranier Vilela Discussion:

pgsql: Add collation versions for FreeBSD.

2020-11-20 Thread Thomas Munro
Add collation versions for FreeBSD. On FreeBSD 13, use querylocale() to read the current version of libc collations. Similar to commits 352f6f2d for Windows and d5ac14f9 for GNU/Linux. Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com

pgsql: Adjust DSM and DSA slot usage constants (back-patch).

2020-11-19 Thread Thomas Munro
Adjust DSM and DSA slot usage constants (back-patch). 1. Previously, a DSA area would create up to four segments at each size before doubling the size. After this commit, it will create only two at each size, so it ramps up faster and therefore needs fewer slots. 2. Previously, the total

pgsql: Adjust DSM and DSA slot usage constants (back-patch).

2020-11-19 Thread Thomas Munro
Adjust DSM and DSA slot usage constants (back-patch). 1. Previously, a DSA area would create up to four segments at each size before doubling the size. After this commit, it will create only two at each size, so it ramps up faster and therefore needs fewer slots. 2. Previously, the total

pgsql: Adjust DSM and DSA slot usage constants (back-patch).

2020-11-19 Thread Thomas Munro
Adjust DSM and DSA slot usage constants (back-patch). 1. Previously, a DSA area would create up to four segments at each size before doubling the size. After this commit, it will create only two at each size, so it ramps up faster and therefore needs fewer slots. 2. Previously, the total

pgsql: Add BarrierArriveAndDetachExceptLast().

2020-11-18 Thread Thomas Munro
Add BarrierArriveAndDetachExceptLast(). Provide a way for one process to continue the remaining phases of a (previously) parallel computation alone. Later patches will use this to extend Parallel Hash Join. Author: Melanie Plageman Reviewed-by: Thomas Munro Discussion: https://postgr.es/m/CA

pgsql: Fix parsePGArray() error checking in pg_dump.

2020-11-08 Thread Thomas Munro
Fix parsePGArray() error checking in pg_dump. Coverity complained about a defect in commit 257836a7: Calling "parsePGArray" without checking return value (as is done elsewhere 11 out of 13 times). Fix, and also check for empty strings explicitly (NULL as represented by PQgetvalue()). That

pgsql: Fix assertion in collation version lookup.

2020-11-07 Thread Thomas Munro
Fix assertion in collation version lookup. Commit 257836a7 included an assertion that a version lookup routine is not trying to look up "C" or "POSIX", but that case is reachable with the user-facing SQL function pg_collation_actual_version(). Remove the assertion. Branch -- master Details

pgsql: Fix unlinking of SLRU segments.

2020-11-04 Thread Thomas Munro
Fix unlinking of SLRU segments. Commit dee663f7 intended to drop any queued up fsync requests before unlinking segment files, but missed a code path. Fix, by centralizing the forget-and-unlink code into a single function. Reported-by: Tomas Vondra Discussion:

pgsql: Tolerate version lookup failure for old style Windows locale nam

2020-11-03 Thread Thomas Munro
Tolerate version lookup failure for old style Windows locale names. Accept that we can't get versions for such locale names for now. Users will need to specify the newer language tag format to enable the collation versioning feature. It's not clear that we can do automatic conversion from the

pgsql: Track collation versions for indexes.

2020-11-02 Thread Thomas Munro
, Christoph Berg, Laurenz Albe, Michael Paquier, Robert Haas, Tom Lane and others for very helpful discussion. Author: Thomas Munro Author: Julien Rouhaud Reviewed-by: Peter Eisentraut (earlier versions) Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ

pgsql: Add pg_depend.refobjversion.

2020-11-02 Thread Thomas Munro
Add pg_depend.refobjversion. Provide a place for the version of referenced database objects to be recorded. A follow-up commit will use this to record dependencies on collation versions for indexes, but similar ideas for other kinds of objects have also been mooted. Author: Thomas Munro

pgsql: Handle EACCES errors from kevent() better.

2020-10-14 Thread Thomas Munro
Handle EACCES errors from kevent() better. While registering for postmaster exit events, we have to handle a couple of edge cases where the postmaster is already gone. Commit 815c2f09 missed one: EACCES must surely imply that PostmasterPid no longer belongs to our postmaster process (or

pgsql: Handle EACCES errors from kevent() better.

2020-10-14 Thread Thomas Munro
Handle EACCES errors from kevent() better. While registering for postmaster exit events, we have to handle a couple of edge cases where the postmaster is already gone. Commit 815c2f09 missed one: EACCES must surely imply that PostmasterPid no longer belongs to our postmaster process (or

pgsql: Make WL_POSTMASTER_DEATH level-triggered on kqueue builds.

2020-10-14 Thread Thomas Munro
Make WL_POSTMASTER_DEATH level-triggered on kqueue builds. If WaitEventSetWait() reports that the postmaster has gone away, later calls to WaitEventSetWait() should continue to report that. Otherwise further waits that occur in the proc_exit() path after we already noticed the postmaster's

pgsql: Make WL_POSTMASTER_DEATH level-triggered on kqueue builds.

2020-10-14 Thread Thomas Munro
Make WL_POSTMASTER_DEATH level-triggered on kqueue builds. If WaitEventSetWait() reports that the postmaster has gone away, later calls to WaitEventSetWait() should continue to report that. Otherwise further waits that occur in the proc_exit() path after we already noticed the postmaster's

pgsql: Fix estimates for ModifyTable paths without RETURNING.

2020-10-12 Thread Thomas Munro
Fix estimates for ModifyTable paths without RETURNING. In the past, we always estimated that a ModifyTable node would emit the same number of rows as its subpaths. Without a RETURNING clause, the correct estimate is zero. Fix, in preparation for a proposed parallel write patch that is sensitive

pgsql: Defer flushing of SLRU files.

2020-09-25 Thread Thomas Munro
Defer flushing of SLRU files. Previously, we called fsync() after writing out individual pg_xact, pg_multixact and pg_commit_ts pages due to cache pressure, leading to regular I/O stalls in user backends and recovery. Collapse requests for the same file into a single system call as part of the

pgsql: Fix missing fsync of SLRU directories.

2020-09-23 Thread Thomas Munro
Fix missing fsync of SLRU directories. Harmonize behavior by moving reponsibility for fsyncing directories down into slru.c. In 10 and later, only the multixact directories were missed (see commit 1b02be21), and in older branches all SLRUs were missed. Back-patch to all supported releases.

pgsql: Fix missing fsync of SLRU directories.

2020-09-23 Thread Thomas Munro
Fix missing fsync of SLRU directories. Harmonize behavior by moving reponsibility for fsyncing directories down into slru.c. In 10 and later, only the multixact directories were missed (see commit 1b02be21), and in older branches all SLRUs were missed. Back-patch to all supported releases.

pgsql: Fix missing fsync of SLRU directories.

2020-09-23 Thread Thomas Munro
Fix missing fsync of SLRU directories. Harmonize behavior by moving reponsibility for fsyncing directories down into slru.c. In 10 and later, only the multixact directories were missed (see commit 1b02be21), and in older branches all SLRUs were missed. Back-patch to all supported releases.

pgsql: Fix missing fsync of SLRU directories.

2020-09-23 Thread Thomas Munro
Fix missing fsync of SLRU directories. Harmonize behavior by moving reponsibility for fsyncing directories down into slru.c. In 10 and later, only the multixact directories were missed (see commit 1b02be21), and in older branches all SLRUs were missed. Back-patch to all supported releases.

pgsql: Fix missing fsync of SLRU directories.

2020-09-23 Thread Thomas Munro
Fix missing fsync of SLRU directories. Harmonize behavior by moving reponsibility for fsyncing directories down into slru.c. In 10 and later, only the multixact directories were missed (see commit 1b02be21), and in older branches all SLRUs were missed. Back-patch to all supported releases.

pgsql: Fix missing fsync of SLRU directories.

2020-09-23 Thread Thomas Munro
Fix missing fsync of SLRU directories. Harmonize behavior by moving reponsibility for fsyncing directories down into slru.c. In 10 and later, only the multixact directories were missed (see commit 1b02be21), and in older branches all SLRUs were missed. Back-patch to all supported releases.

pgsql: Allow WaitLatch() to be used without a latch.

2020-09-22 Thread Thomas Munro
Allow WaitLatch() to be used without a latch. Due to flaws in commit 3347c982bab, using WaitLatch() without WL_LATCH_SET could cause an assertion failure or crash. Repair. While here, also add a check that the latch we're switching to belongs to this backend, when changing from one latch to

pgsql: Code review for dynahash change.

2020-09-18 Thread Thomas Munro
Code review for dynahash change. Commit be0a left behind a comment about the order of some tests that didn't make sense without the expensive division, and in fact we might as well change the order to one that fails more cheaply most of the time as a micro-optimization. Also, remove the "+

pgsql: Remove large fill factor support from dynahash.c.

2020-09-18 Thread Thomas Munro
Herrera Reviewed-by: Tomas Vondra Reviewed-by: Thomas Munro Reviewed-by: David Rowley Discussion: https://postgr.es/m/VI1PR0701MB696044FC35013A96FECC7AC8F62D0%40VI1PR0701MB6960.eurprd07.prod.outlook.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff

pgsql: Add d_type to our Windows dirent emulation.

2020-09-07 Thread Thomas Munro
Add d_type to our Windows dirent emulation. This allows us to skip some stat calls, by extending commit 861c6e7c to cover Windows systems. Author: Juan José Santamaría Flecha Reviewed-by: Alvaro Herrera Reviewed-by: Andres Freund Reviewed-by: Magnus Hagander Reviewed-by: Thomas Munro

pgsql: Skip unnecessary stat() calls in walkdir().

2020-09-07 Thread Thomas Munro
Skip unnecessary stat() calls in walkdir(). Some kernels can tell us the type of a "dirent", so we can avoid a call to stat() or lstat() in many cases. Define a new function get_dirent_type() to contain that logic, for use by the backend and frontend versions of walkdir(), and perhaps other

pgsql: Fix rare failure in LDAP tests.

2020-08-02 Thread Thomas Munro
Fix rare failure in LDAP tests. Instead of writing a query to psql's stdin, use -c. This avoids a failure where psql exits before we write, seen a few times on the build farm. Thanks to Tom Lane for the suggestion. Back-patch to 11, where the LDAP tests arrived. Reviewed-by: Noah Misch

pgsql: Fix rare failure in LDAP tests.

2020-08-02 Thread Thomas Munro
Fix rare failure in LDAP tests. Instead of writing a query to psql's stdin, use -c. This avoids a failure where psql exits before we write, seen a few times on the build farm. Thanks to Tom Lane for the suggestion. Back-patch to 11, where the LDAP tests arrived. Reviewed-by: Noah Misch

pgsql: Fix rare failure in LDAP tests.

2020-08-02 Thread Thomas Munro
Fix rare failure in LDAP tests. Instead of writing a query to psql's stdin, use -c. This avoids a failure where psql exits before we write, seen a few times on the build farm. Thanks to Tom Lane for the suggestion. Back-patch to 11, where the LDAP tests arrived. Reviewed-by: Noah Misch

pgsql: Fix rare failure in LDAP tests.

2020-08-02 Thread Thomas Munro
Fix rare failure in LDAP tests. Instead of writing a query to psql's stdin, use -c. This avoids a failure where psql exits before we write, seen a few times on the build farm. Thanks to Tom Lane for the suggestion. Back-patch to 11, where the LDAP tests arrived. Reviewed-by: Noah Misch

pgsql: Correct comment in simplehash.h.

2020-08-02 Thread Thomas Munro
Correct comment in simplehash.h. Post-commit review for commit 84c0e4b9. Author: David Rowley Discussion: https://postgr.es/m/CAApHDvptBx_%2BUPAzY0uXzopbvPVGKPeZ6Hoy8rnPcWz20Cr0Bw%40mail.gmail.com Branch -- master Details ---

Re: pgsql: Fix some issues with step generation in partition pruning.

2020-08-01 Thread Thomas Munro
On Tue, Jul 28, 2020 at 2:01 PM Etsuro Fujita wrote: > src/backend/partitioning/partprune.c | 187 ++ Hi Fujita-san I wonder if this build farm failure is related? Program terminated with signal 11, Segmentation fault. ... #0 0x59c15c in

pgsql: Use pg_pread() and pg_pwrite() in slru.c.

2020-08-01 Thread Thomas Munro
Use pg_pread() and pg_pwrite() in slru.c. This avoids lseek() system calls at every SLRU I/O, as was done for relation files in commit c24dcd0c. Reviewed-by: Ashwin Agrawal Reviewed-by: Andres Freund Discussion:

pgsql: Improve programmer docs for simplehash and dynahash.

2020-07-31 Thread Thomas Munro
Improve programmer docs for simplehash and dynahash. When reading the code it's not obvious when one should prefer dynahash over simplehash and vice-versa, so, for programmer-friendliness, add comments to inform that decision. Show sample simplehash method signatures. Author: James Coleman

Re: pgsql: Preallocate some DSM space at startup.

2020-07-31 Thread Thomas Munro
On Fri, Jul 31, 2020 at 6:26 PM Michael Paquier wrote: > longfin is complaning on this one: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=longfin=2020-07-31%2005%3A59%3A08 > guc.c:2241:38: error: implicit conversion from 'unsigned long' to > 'int' changes value from 17592186044415 to

pgsql: Fix compiler warning from Clang.

2020-07-31 Thread Thomas Munro
Fix compiler warning from Clang. Per build farm. Discussion: https://postgr.es/m/20200731062626.GD3317%40paquier.xyz Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/7be04496a9f763fc4d4c1d06ce9ccc250e52df31 Modified Files --

pgsql: Preallocate some DSM space at startup.

2020-07-30 Thread Thomas Munro
Preallocate some DSM space at startup. Create an optional region in the main shared memory segment that can be used to acquire and release "fast" DSM segments, and can benefit from huge pages allocated at cluster startup time, if configured. Fall back to the existing mechanisms when that space

pgsql: Cache smgrnblocks() results in recovery.

2020-07-30 Thread Thomas Munro
Cache smgrnblocks() results in recovery. Avoid repeatedly calling lseek(SEEK_END) during recovery by caching the size of each fork. For now, we can't use the same technique in other processes, because we lack a shared invalidation mechanism. Do this by generalizing the pre-existing caching used

pgsql: Introduce a WaitEventSet for the stats collector.

2020-07-29 Thread Thomas Munro
Introduce a WaitEventSet for the stats collector. This avoids avoids some epoll/kqueue system calls for every wait. Reviewed-by: Kyotaro Horiguchi Discussion: https://postgr.es/m/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw%40mail.gmail.com Branch -- master Details ---

pgsql: Use a long lived WaitEventSet for WaitLatch().

2020-07-29 Thread Thomas Munro
Use a long lived WaitEventSet for WaitLatch(). Create LatchWaitSet at backend startup time, and use it to implement WaitLatch(). This avoids repeated epoll/kqueue setup and teardown system calls. Reorder SubPostmasterMain() slightly so that we restore the postmaster pipe and Windows signal

pgsql: Use WaitLatch() for condition variables.

2020-07-29 Thread Thomas Munro
Use WaitLatch() for condition variables. Previously, condition_variable.c created a long lived WaitEventSet to avoid extra system calls. WaitLatch() now uses something similar internally, so there is no point in wasting an extra kernel descriptor. Reviewed-by: Kyotaro Horiguchi Discussion:

pgsql: Move syncscan.c to src/backend/access/common.

2020-07-28 Thread Thomas Munro
Move syncscan.c to src/backend/access/common. Since the tableam.c code needs to make use of the syncscan.c routines itself, and since other block-oriented AMs might also want to use it one day, it didn't make sense for it to live under src/backend/access/heap. Reviewed-by: Andres Freund

pgsql: Fix error message.

2020-07-23 Thread Thomas Munro
Fix error message. Remove extra space. Back-patch to all releases, like commit 7897e3bb. Author: Lu, Chenyang Discussion: https://postgr.es/m/795d03c6129844d3803e7eea48f5af0d%40G08CNEXMBPEKD04.g08.fujitsu.local Branch -- REL9_5_STABLE Details ---

pgsql: Fix error message.

2020-07-23 Thread Thomas Munro
Fix error message. Remove extra space. Back-patch to all releases, like commit 7897e3bb. Author: Lu, Chenyang Discussion: https://postgr.es/m/795d03c6129844d3803e7eea48f5af0d%40G08CNEXMBPEKD04.g08.fujitsu.local Branch -- REL_10_STABLE Details ---

pgsql: Fix error message.

2020-07-23 Thread Thomas Munro
Fix error message. Remove extra space. Back-patch to all releases, like commit 7897e3bb. Author: Lu, Chenyang Discussion: https://postgr.es/m/795d03c6129844d3803e7eea48f5af0d%40G08CNEXMBPEKD04.g08.fujitsu.local Branch -- REL9_6_STABLE Details ---

pgsql: Fix error message.

2020-07-23 Thread Thomas Munro
Fix error message. Remove extra space. Back-patch to all releases, like commit 7897e3bb. Author: Lu, Chenyang Discussion: https://postgr.es/m/795d03c6129844d3803e7eea48f5af0d%40G08CNEXMBPEKD04.g08.fujitsu.local Branch -- REL_11_STABLE Details ---

pgsql: Fix error message.

2020-07-23 Thread Thomas Munro
Fix error message. Remove extra space. Back-patch to all releases, like commit 7897e3bb. Author: Lu, Chenyang Discussion: https://postgr.es/m/795d03c6129844d3803e7eea48f5af0d%40G08CNEXMBPEKD04.g08.fujitsu.local Branch -- REL_12_STABLE Details ---

pgsql: Fix error message.

2020-07-23 Thread Thomas Munro
Fix error message. Remove extra space. Back-patch to all releases, like commit 7897e3bb. Author: Lu, Chenyang Discussion: https://postgr.es/m/795d03c6129844d3803e7eea48f5af0d%40G08CNEXMBPEKD04.g08.fujitsu.local Branch -- REL_13_STABLE Details ---

pgsql: Fix error message.

2020-07-23 Thread Thomas Munro
Fix error message. Remove extra space. Back-patch to all releases, like commit 7897e3bb. Author: Lu, Chenyang Discussion: https://postgr.es/m/795d03c6129844d3803e7eea48f5af0d%40G08CNEXMBPEKD04.g08.fujitsu.local Branch -- master Details ---

pgsql: Fix conversion table generator scripts.

2020-07-21 Thread Thomas Munro
Fix conversion table generator scripts. convutils.pm used implicit conversion of undefined value to integer zero. Some of conversion scripts are susceptible to regexp greediness. Fix, avoiding whitespace changes in the output. Also update ICU URLs that moved. No need to back-patch, because the

pgsql: Add huge_page_size setting for use on Linux.

2020-07-17 Thread Thomas Munro
Add huge_page_size setting for use on Linux. This allows the huge page size to be set explicitly. The default is 0, meaning it will use the system default, as before. Author: Odin Ugedal Discussion: https://postgr.es/m/20200608154639.20254-1-odin%40ugedal.com Branch -- master Details

pgsql: Use MinimalTuple for tuple queues.

2020-07-17 Thread Thomas Munro
Use MinimalTuple for tuple queues. This representation saves 8 bytes per tuple compared to HeapTuple, and avoids the need to allocate, copy and free on the receiving side. Gather can emit the returned MinimalTuple directly, but GatherMerge now needs to make an explicit copy because it buffers

pgsql: Make BufFileWrite() void.

2020-06-15 Thread Thomas Munro
Make BufFileWrite() void. It now either returns after it wrote all the data you gave it, or raises an error. Not done in back-branches, because it might cause problems for external code. Discussion: https://postgr.es/m/CA%2BhUKGJE04G%3D8TLK0DLypT_27D9dR8F1RQgNp0jK6qR0tZGWOw%40mail.gmail.com

pgsql: Fix buffile.c error handling.

2020-06-15 Thread Thomas Munro
Fix buffile.c error handling. Convert buffile.c error handling to use ereport. This fixes cases where I/O errors were indistinguishable from EOF or not reported. Also remove "%m" from error messages where errno would be bogus. While we're modifying those strings, add block numbers and short

pgsql: Fix buffile.c error handling.

2020-06-15 Thread Thomas Munro
Fix buffile.c error handling. Convert buffile.c error handling to use ereport. This fixes cases where I/O errors were indistinguishable from EOF or not reported. Also remove "%m" from error messages where errno would be bogus. While we're modifying those strings, add block numbers and short

pgsql: Fix buffile.c error handling.

2020-06-15 Thread Thomas Munro
Fix buffile.c error handling. Convert buffile.c error handling to use ereport. This fixes cases where I/O errors were indistinguishable from EOF or not reported. Also remove "%m" from error messages where errno would be bogus. While we're modifying those strings, add block numbers and short

pgsql: Fix buffile.c error handling.

2020-06-15 Thread Thomas Munro
Fix buffile.c error handling. Convert buffile.c error handling to use ereport. This fixes cases where I/O errors were indistinguishable from EOF or not reported. Also remove "%m" from error messages where errno would be bogus. While we're modifying those strings, add block numbers and short

pgsql: Fix buffile.c error handling.

2020-06-15 Thread Thomas Munro
Fix buffile.c error handling. Convert buffile.c error handling to use ereport. This fixes cases where I/O errors were indistinguishable from EOF or not reported. Also remove "%m" from error messages where errno would be bogus. While we're modifying those strings, add block numbers and short

pgsql: Fix buffile.c error handling.

2020-06-15 Thread Thomas Munro
Fix buffile.c error handling. Convert buffile.c error handling to use ereport. This fixes cases where I/O errors were indistinguishable from EOF or not reported. Also remove "%m" from error messages where errno would be bogus. While we're modifying those strings, add block numbers and short

pgsql: Fix buffile.c error handling.

2020-06-15 Thread Thomas Munro
Fix buffile.c error handling. Convert buffile.c error handling to use ereport. This fixes cases where I/O errors were indistinguishable from EOF or not reported. Also remove "%m" from error messages where errno would be bogus. While we're modifying those strings, add block numbers and short

pgsql: Doc: Add references for SI and SSI.

2020-06-14 Thread Thomas Munro
Doc: Add references for SI and SSI. Our documentation failed to point out that REPEATABLE READ is really snapshot isolation, which might be important to some users. Point to the standard reference paper for this complicated topic. Likewise, add a reference to the VLDB paper about PostgreSQL

pgsql: Doc: Add references for SI and SSI.

2020-06-14 Thread Thomas Munro
Doc: Add references for SI and SSI. Our documentation failed to point out that REPEATABLE READ is really snapshot isolation, which might be important to some users. Point to the standard reference paper for this complicated topic. Likewise, add a reference to the VLDB paper about PostgreSQL

pgsql: Doc: Add references for SI and SSI.

2020-06-14 Thread Thomas Munro
Doc: Add references for SI and SSI. Our documentation failed to point out that REPEATABLE READ is really snapshot isolation, which might be important to some users. Point to the standard reference paper for this complicated topic. Likewise, add a reference to the VLDB paper about PostgreSQL

pgsql: Doc: Add references for SI and SSI.

2020-06-14 Thread Thomas Munro
Doc: Add references for SI and SSI. Our documentation failed to point out that REPEATABLE READ is really snapshot isolation, which might be important to some users. Point to the standard reference paper for this complicated topic. Likewise, add a reference to the VLDB paper about PostgreSQL

pgsql: Doc: Add references for SI and SSI.

2020-06-14 Thread Thomas Munro
Doc: Add references for SI and SSI. Our documentation failed to point out that REPEATABLE READ is really snapshot isolation, which might be important to some users. Point to the standard reference paper for this complicated topic. Likewise, add a reference to the VLDB paper about PostgreSQL

pgsql: Doc: Add references for SI and SSI.

2020-06-14 Thread Thomas Munro
Doc: Add references for SI and SSI. Our documentation failed to point out that REPEATABLE READ is really snapshot isolation, which might be important to some users. Point to the standard reference paper for this complicated topic. Likewise, add a reference to the VLDB paper about PostgreSQL

pgsql: Doc: Add references for SI and SSI.

2020-06-14 Thread Thomas Munro
Doc: Add references for SI and SSI. Our documentation failed to point out that REPEATABLE READ is really snapshot isolation, which might be important to some users. Point to the standard reference paper for this complicated topic. Likewise, add a reference to the VLDB paper about PostgreSQL

pgsql: Improve comments for [Heap]CheckForSerializableConflictOut().

2020-06-11 Thread Thomas Munro
Improve comments for [Heap]CheckForSerializableConflictOut(). Rewrite the documentation of these functions, in light of recent bug fix commit 5940ffb2. Back-patch to 13 where the check-for-conflict-out code was split up into AM-specific and generic parts, and new documentation was added that now

pgsql: Improve comments for [Heap]CheckForSerializableConflictOut().

2020-06-11 Thread Thomas Munro
Improve comments for [Heap]CheckForSerializableConflictOut(). Rewrite the documentation of these functions, in light of recent bug fix commit 5940ffb2. Back-patch to 13 where the check-for-conflict-out code was split up into AM-specific and generic parts, and new documentation was added that now

pgsql: Fix locking bugs that could corrupt pg_control.

2020-06-07 Thread Thomas Munro
ire ControlFileLock before modifying ControlFile and calling UpdateControlFile(). Back-patch to all supported releases. Author: Nathan Bossart Author: Fujii Masao Reviewed-by: Fujii Masao Reviewed-by: Michael Paquier Reviewed-by: Thomas Munro Discussion: https://postgr.es/m/70BF24D6-DC51-4

pgsql: Fix locking bugs that could corrupt pg_control.

2020-06-07 Thread Thomas Munro
ire ControlFileLock before modifying ControlFile and calling UpdateControlFile(). Back-patch to all supported releases. Author: Nathan Bossart Author: Fujii Masao Reviewed-by: Fujii Masao Reviewed-by: Michael Paquier Reviewed-by: Thomas Munro Discussion: https://postgr.es/m/70BF24D6-DC51-4

pgsql: Fix locking bugs that could corrupt pg_control.

2020-06-07 Thread Thomas Munro
ire ControlFileLock before modifying ControlFile and calling UpdateControlFile(). Back-patch to all supported releases. Author: Nathan Bossart Author: Fujii Masao Reviewed-by: Fujii Masao Reviewed-by: Michael Paquier Reviewed-by: Thomas Munro Discussion: https://postgr.es/m/70BF24D6-DC51-4

pgsql: Fix locking bugs that could corrupt pg_control.

2020-06-07 Thread Thomas Munro
ire ControlFileLock before modifying ControlFile and calling UpdateControlFile(). Back-patch to all supported releases. Author: Nathan Bossart Author: Fujii Masao Reviewed-by: Fujii Masao Reviewed-by: Michael Paquier Reviewed-by: Thomas Munro Discussion: https://postgr.es/m/70BF24D6-DC51-4

pgsql: Fix locking bugs that could corrupt pg_control.

2020-06-07 Thread Thomas Munro
ire ControlFileLock before modifying ControlFile and calling UpdateControlFile(). Back-patch to all supported releases. Author: Nathan Bossart Author: Fujii Masao Reviewed-by: Fujii Masao Reviewed-by: Michael Paquier Reviewed-by: Thomas Munro Discussion: https://postgr.es/m/70BF24D6-DC51-4

pgsql: Fix locking bugs that could corrupt pg_control.

2020-06-07 Thread Thomas Munro
ire ControlFileLock before modifying ControlFile and calling UpdateControlFile(). Back-patch to all supported releases. Author: Nathan Bossart Author: Fujii Masao Reviewed-by: Fujii Masao Reviewed-by: Michael Paquier Reviewed-by: Thomas Munro Discussion: https://postgr.es/m/70BF24D6-DC51-4

pgsql: Fix locking bugs that could corrupt pg_control.

2020-06-07 Thread Thomas Munro
ire ControlFileLock before modifying ControlFile and calling UpdateControlFile(). Back-patch to all supported releases. Author: Nathan Bossart Author: Fujii Masao Reviewed-by: Fujii Masao Reviewed-by: Michael Paquier Reviewed-by: Thomas Munro Discussion: https://postgr.es/m/70BF24D6-DC51-4

pgsql: Doc: Update example symptom of systemd misconfiguration.

2020-06-07 Thread Thomas Munro
Doc: Update example symptom of systemd misconfiguration. In PostgreSQL 10, we stopped using System V semaphores on Linux systems. Update the example we give of an error message from a misconfigured system to show what people are most likely to see these days. Back-patch to 10, where

pgsql: Doc: Update example symptom of systemd misconfiguration.

2020-06-07 Thread Thomas Munro
Doc: Update example symptom of systemd misconfiguration. In PostgreSQL 10, we stopped using System V semaphores on Linux systems. Update the example we give of an error message from a misconfigured system to show what people are most likely to see these days. Back-patch to 10, where

pgsql: Doc: Update example symptom of systemd misconfiguration.

2020-06-07 Thread Thomas Munro
Doc: Update example symptom of systemd misconfiguration. In PostgreSQL 10, we stopped using System V semaphores on Linux systems. Update the example we give of an error message from a misconfigured system to show what people are most likely to see these days. Back-patch to 10, where

pgsql: Doc: Update example symptom of systemd misconfiguration.

2020-06-07 Thread Thomas Munro
Doc: Update example symptom of systemd misconfiguration. In PostgreSQL 10, we stopped using System V semaphores on Linux systems. Update the example we give of an error message from a misconfigured system to show what people are most likely to see these days. Back-patch to 10, where

pgsql: Doc: Update example symptom of systemd misconfiguration.

2020-06-07 Thread Thomas Munro
Doc: Update example symptom of systemd misconfiguration. In PostgreSQL 10, we stopped using System V semaphores on Linux systems. Update the example we give of an error message from a misconfigured system to show what people are most likely to see these days. Back-patch to 10, where

pgsql: Doc: Clean up references to obsolete OS versions.

2020-06-07 Thread Thomas Munro
Doc: Clean up references to obsolete OS versions. Remove obsolete instructions for old operating system versions, and update the text to reflect the defaults on modern systems. Reviewed-by: Tom Lane Reviewed-by: Peter Eisentraut Reviewed-by: Magnus Hagander Discussion:

pgsql: Rationalize GetWalRcv{Write,Flush}RecPtr().

2020-04-08 Thread Thomas Munro
Rationalize GetWalRcv{Write,Flush}RecPtr(). GetWalRcvWriteRecPtr() previously reported the latest *flushed* location. Adopt the conventional terminology used elsewhere in the tree by renaming it to GetWalRcvFlushRecPtr(), and likewise for some related variables that used the term "received".

Re: pgsql: Collect statistics about SLRU caches

2020-04-07 Thread Thomas Munro
Hi Tomas, In pgstat.c, + elog(LOG, "msg->m_index = %d", msg->m_index); Looks like unintentional debugging stuff?

pgsql: Introduce xid8-based functions to replace txid_XXX.

2020-04-06 Thread Thomas Munro
Introduce xid8-based functions to replace txid_XXX. The txid_XXX family of fmgr functions exposes 64 bit transaction IDs to users as int8. Now that we have an SQL type xid8 for FullTransactionId, define a new set of functions including pg_current_xact_id() and pg_current_snapshot() based on

pgsql: Add SQL type xid8 to expose FullTransactionId to users.

2020-04-06 Thread Thomas Munro
Add SQL type xid8 to expose FullTransactionId to users. Similar to xid, but 64 bits wide. This new type is suitable for use in various system views and administration functions. Reviewed-by: Fujii Masao Reviewed-by: Takao Fujii Reviewed-by: Yoshikazu Imai Reviewed-by: Mark Dilger

pgsql: Add maintenance_io_concurrency to postgresql.conf.sample.

2020-04-01 Thread Thomas Munro
Add maintenance_io_concurrency to postgresql.conf.sample. New GUC from commit fc34b0d9. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/37b3794dfcfb9d55f7ea83693f50b1484caab21c Modified Files -- src/backend/utils/misc/postgresql.conf.sample | 1 + 1

Re: pgsql: Add kqueue(2) support to the WaitEventSet API.

2020-03-28 Thread Thomas Munro
On Sun, Mar 29, 2020 at 7:43 AM Tom Lane wrote: > Thomas Munro writes: > > Pushed. > > prairiedog just turned up a different issue in this area [1]. > I wondered why it hadn't reported in for awhile, and upon > investigation I found that the test run was stuck in the

pgsql: Allow NULL version for individual collations.

2020-03-24 Thread Thomas Munro
Allow NULL version for individual collations. Remove the documented restriction that collation providers must either return NULL for all collations or non-NULL for all collations. Use NULL for glibc collations like "C.UTF-8", which might otherwise lead future proposed commits to force

pgsql: Add collation versions for Windows.

2020-03-24 Thread Thomas Munro
Add collation versions for Windows. On Vista and later, use GetNLSVersionEx() to request collation version information. Reviewed-by: Juan José Santamaría Flecha Discussion: https://postgr.es/m/CA%2BhUKGJvqup3s%2BJowVTcacZADO6dOhfdBmvOPHLS3KXUJu41Jw%40mail.gmail.com Branch -- master

Re: pgsql: Add kqueue(2) support to the WaitEventSet API.

2020-03-17 Thread Thomas Munro
On Tue, Mar 17, 2020 at 10:21 AM Thomas Munro wrote: > I'm now far away from my home Mac so I can't test until later but I > think we can fix this by double checking with the pipe: Pushed.

pgsql: Fix kqueue support under debugger on macOS.

2020-03-17 Thread Thomas Munro
Fix kqueue support under debugger on macOS. While running under a debugger, macOS's getppid() can return the debugger's PID. That could cause a backend to exit because it falsely believed that the postmaster had died, since commit 815c2f09. Continue to use getppid() as a fast postmaster check

pgsql: Don't use EV_CLEAR for kqueue events.

2020-03-17 Thread Thomas Munro
Don't use EV_CLEAR for kqueue events. For the semantics to match the epoll implementation, we need a socket to continue to appear readable/writable if you wait multiple times without doing I/O in between (in Linux terminology: level-triggered rather than edge-triggered). This distinction will be

Re: pgsql: Add kqueue(2) support to the WaitEventSet API.

2020-03-16 Thread Thomas Munro
On Tue, Mar 17, 2020 at 9:30 AM Alvaro Herrera wrote: > On 2020-Mar-17, Thomas Munro wrote: > > Reproduced here. The problem seems to be that macOS's getppid() > > returns the debugger's PID, while the debugger is attached. This > > doesn't happen on FreeBSD (even tho

Re: pgsql: Add kqueue(2) support to the WaitEventSet API.

2020-03-16 Thread Thomas Munro
On Tue, Mar 17, 2020 at 12:55 AM Alexander Korotkov wrote: > On Wed, Feb 5, 2020 at 7:59 AM Thomas Munro wrote: > > Add kqueue(2) support to the WaitEventSet API. > > > > Use kevent(2) to wait for events on the BSD family of operating > > systems and macOS. Thi

pgsql: Introduce a maintenance_io_concurrency setting.

2020-03-15 Thread Thomas Munro
Introduce a maintenance_io_concurrency setting. Introduce a GUC and a tablespace option to control I/O prefetching, much like effective_io_concurrency, but for work that is done on behalf of many client sessions. Use the new setting in heapam.c instead of the hard-coded formula

pgsql: Simplify the effective_io_concurrency setting.

2020-03-15 Thread Thomas Munro
Simplify the effective_io_concurrency setting. The effective_io_concurrency GUC and equivalent tablespace option were previously passed through a formula based on a theory about RAID spindles and probabilities, to arrive at the number of pages to prefetch in bitmap heap scans. Tomas Vondra,

<    3   4   5   6   7   8   9   10   11   12   >