Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-26 Thread Thomas Munro
On Wed, Jul 27, 2022 at 6:06 PM Thomas Munro wrote: > On Wed, Jul 27, 2022 at 5:01 PM Thomas Munro wrote: > > Someone else still has the old file open, so we can't rename the new > > one to its name? On Windows that should have gone through pgrename() > > in dirmod.c, which would retry 100 times

Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-26 Thread Thomas Munro
On Wed, Jul 27, 2022 at 5:01 PM Thomas Munro wrote: > On Wed, Jul 27, 2022 at 4:35 PM Michael Paquier wrote: > > On Tue, Jul 26, 2022 at 07:10:22PM +, Robert Haas wrote: > > > Remove the restriction that the relmap must be 512 bytes. > > > The CI on Windows is blowing up here and there after

pgsql: Allow "in place" tablespaces.

2022-07-26 Thread Alvaro Herrera
Allow "in place" tablespaces. This is a backpatch to branches 10-14 of the following commits: 7170f2159fb2 Allow "in place" tablespaces. c6f2f01611d4 Fix pg_basebackup with in-place tablespaces. f6f0db4d6240 Fix pg_tablespace_location() with in-place tablespaces 7a7cd84893e0 doc: Remove mention t

pgsql: Allow "in place" tablespaces.

2022-07-26 Thread Alvaro Herrera
Allow "in place" tablespaces. This is a backpatch to branches 10-14 of the following commits: 7170f2159fb2 Allow "in place" tablespaces. c6f2f01611d4 Fix pg_basebackup with in-place tablespaces. f6f0db4d6240 Fix pg_tablespace_location() with in-place tablespaces 7a7cd84893e0 doc: Remove mention t

pgsql: Allow "in place" tablespaces.

2022-07-26 Thread Alvaro Herrera
Allow "in place" tablespaces. This is a backpatch to branches 10-14 of the following commits: 7170f2159fb2 Allow "in place" tablespaces. c6f2f01611d4 Fix pg_basebackup with in-place tablespaces. f6f0db4d6240 Fix pg_tablespace_location() with in-place tablespaces 7a7cd84893e0 doc: Remove mention t

pgsql: Allow "in place" tablespaces.

2022-07-26 Thread Alvaro Herrera
Allow "in place" tablespaces. This is a backpatch to branches 10-14 of the following commits: 7170f2159fb2 Allow "in place" tablespaces. c6f2f01611d4 Fix pg_basebackup with in-place tablespaces. f6f0db4d6240 Fix pg_tablespace_location() with in-place tablespaces 7a7cd84893e0 doc: Remove mention t

pgsql: Allow "in place" tablespaces.

2022-07-26 Thread Alvaro Herrera
Allow "in place" tablespaces. This is a backpatch to branches 10-14 of the following commits: 7170f2159fb2 Allow "in place" tablespaces. c6f2f01611d4 Fix pg_basebackup with in-place tablespaces. f6f0db4d6240 Fix pg_tablespace_location() with in-place tablespaces 7a7cd84893e0 doc: Remove mention t

pgsql: Move memory management away from writetup() and tuplesort_put*()

2022-07-26 Thread Alexander Korotkov
Move memory management away from writetup() and tuplesort_put*() This commit puts some generic work away from sort-variant-specific function. In particular, tuplesort_put*() now doesn't need to decrease available memory and switch to sort context before calling puttuple_common(). writetup() doesn

pgsql: Add new Tuplesortstate.removeabbrev function

2022-07-26 Thread Alexander Korotkov
Add new Tuplesortstate.removeabbrev function This commit is the preparation to move abbreviation logic into puttuple_common(). The new removeabbrev function turns datum1 representation of SortTuple's from the abbreviated key to the first column value. Therefore, it encapsulates the differential

pgsql: Put abbreviation logic into puttuple_common()

2022-07-26 Thread Alexander Korotkov
Put abbreviation logic into puttuple_common() Abbreviation code is very similar along tuplesort_put*() functions. This commit unifies that code and puts it into puttuple_common(). tuplesort_put*() functions differs in the abbreviation condition, so it has been added as an argument to the puttupl

pgsql: Split TuplesortPublic from Tuplesortstate

2022-07-26 Thread Alexander Korotkov
Split TuplesortPublic from Tuplesortstate The new TuplesortPublic data structure contains the definition of sort-variant-specific interface methods and the part of Tuple sort operation state required by their implementations. This will let define Tuple sort variants without knowledge of Tuplesort

pgsql: Remove Tuplesortstate.copytup function

2022-07-26 Thread Alexander Korotkov
Remove Tuplesortstate.copytup function It's currently unclear how do we split functionality between Tuplesortstate.copytup() function and tuplesort_put*() functions. For instance, copytup_index() and copytup_datum() return error while tuplesort_putindextuplevalues() and tuplesort_putdatum() do the

pgsql: Split tuplesortvariants.c from tuplesort.c

2022-07-26 Thread Alexander Korotkov
Split tuplesortvariants.c from tuplesort.c This commit puts the implementation of Tuple sort variants into the separate file tuplesortvariants.c. That gives better separation of the code and serves well as the demonstration that Tuple sort variant can be defined outside of tuplesort.c. Discussio

Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-26 Thread Thomas Munro
On Wed, Jul 27, 2022 at 4:35 PM Michael Paquier wrote: > On Tue, Jul 26, 2022 at 07:10:22PM +, Robert Haas wrote: > > Remove the restriction that the relmap must be 512 bytes. > The CI on Windows is blowing up here and there after something that > looks to come from this commit, as of this ba

pgsql: Add overflow protection for block-related data in WAL records

2022-07-26 Thread Michael Paquier
Add overflow protection for block-related data in WAL records XLogRecordBlockHeader, the header holding the information for the data related to a block, tracks the length of the data appended to the WAL record with data_length (uint16). This limitation in size was not enforced by the public routi

Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-26 Thread Michael Paquier
Hi Robert, On Tue, Jul 26, 2022 at 07:10:22PM +, Robert Haas wrote: > Remove the restriction that the relmap must be 512 bytes. > > Instead of relying on the ability to atomically overwrite the > entire relmap file in one shot, write a new one and durably > rename it into place. Removing the

pgsql: Improve makeArrayTypeName's algorithm for choosing array type na

2022-07-26 Thread Tom Lane
Improve makeArrayTypeName's algorithm for choosing array type names. As before, we start by prepending one underscore (truncating the base name if necessary). But if there is a conflict, then instead of prepending more and more underscores, append an underscore and some digits, in much the same w

pgsql: Fix brain fade in e530be2c5ce77475d56ccf8f4e0c4872b666ad5f.

2022-07-26 Thread Robert Haas
Fix brain fade in e530be2c5ce77475d56ccf8f4e0c4872b666ad5f. The BoolGetDatum() call ended up in the wrong place. It should be applied when we, err, want to convert a bool to a datum. Thanks to Tom Lane for noticing this. Discussion: http://postgr.es/m/2511599.1658861...@sss.pgh.pa.us Branch ---

pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-26 Thread Robert Haas
Remove the restriction that the relmap must be 512 bytes. Instead of relying on the ability to atomically overwrite the entire relmap file in one shot, write a new one and durably rename it into place. Removing the struct padding and the calculation showing why the map is exactly 512 bytes, and ch

pgsql: Do not allow removal of superuser privileges from bootstrap user

2022-07-26 Thread Robert Haas
Do not allow removal of superuser privileges from bootstrap user. A bootstrap user who is not a superuser will still own many important system objects, such as the pg_catalog schema, that will likely allow that user to regain superuser status. Therefore, allowing the superuser property to be remov

pgsql: Force immediate commit after CREATE DATABASE etc in extended pro

2022-07-26 Thread Tom Lane
Force immediate commit after CREATE DATABASE etc in extended protocol. We have a few commands that "can't run in a transaction block", meaning that if they complete their processing but then we fail to COMMIT, we'll be left with inconsistent on-disk state. However, the existing defenses for this a

pgsql: Force immediate commit after CREATE DATABASE etc in extended pro

2022-07-26 Thread Tom Lane
Force immediate commit after CREATE DATABASE etc in extended protocol. We have a few commands that "can't run in a transaction block", meaning that if they complete their processing but then we fail to COMMIT, we'll be left with inconsistent on-disk state. However, the existing defenses for this a

pgsql: Force immediate commit after CREATE DATABASE etc in extended pro

2022-07-26 Thread Tom Lane
Force immediate commit after CREATE DATABASE etc in extended protocol. We have a few commands that "can't run in a transaction block", meaning that if they complete their processing but then we fail to COMMIT, we'll be left with inconsistent on-disk state. However, the existing defenses for this a

pgsql: Force immediate commit after CREATE DATABASE etc in extended pro

2022-07-26 Thread Tom Lane
Force immediate commit after CREATE DATABASE etc in extended protocol. We have a few commands that "can't run in a transaction block", meaning that if they complete their processing but then we fail to COMMIT, we'll be left with inconsistent on-disk state. However, the existing defenses for this a

pgsql: Force immediate commit after CREATE DATABASE etc in extended pro

2022-07-26 Thread Tom Lane
Force immediate commit after CREATE DATABASE etc in extended protocol. We have a few commands that "can't run in a transaction block", meaning that if they complete their processing but then we fail to COMMIT, we'll be left with inconsistent on-disk state. However, the existing defenses for this a

pgsql: Force immediate commit after CREATE DATABASE etc in extended pro

2022-07-26 Thread Tom Lane
Force immediate commit after CREATE DATABASE etc in extended protocol. We have a few commands that "can't run in a transaction block", meaning that if they complete their processing but then we fail to COMMIT, we'll be left with inconsistent on-disk state. However, the existing defenses for this a

pgsql: Force immediate commit after CREATE DATABASE etc in extended pro

2022-07-26 Thread Tom Lane
Force immediate commit after CREATE DATABASE etc in extended protocol. We have a few commands that "can't run in a transaction block", meaning that if they complete their processing but then we fail to COMMIT, we'll be left with inconsistent on-disk state. However, the existing defenses for this a

pgsql: doc: Add note about re-archiving of same WAL files in docs.

2022-07-26 Thread Fujii Masao
doc: Add note about re-archiving of same WAL files in docs. The server may attempt to re-archive a WAL file that was previously archived. This commit adds the note about how an archive library should handle such a re-archiving. Author: Nathan Bossart Reviewed-by: David Steele, Kyotaro Horiguchi D

pgsql: Reduce overhead of renaming archive status files.

2022-07-26 Thread Fujii Masao
Reduce overhead of renaming archive status files. Presently, archive status files are durably renamed from .ready to .done to indicate that a file has been archived. Persisting this rename to disk accounts for a significant amount of the overhead associated with archiving. While durably renaming