Re: First draft of the PG 15 release notes
On Tue, 27 Sept 2022 at 10:45, Tom Lane wrote: > > David Rowley writes: > > On Tue, 13 Sept 2022 at 09:31, Jonathan S. Katz > > wrote: > >> Separately, per[1], including dense_rank() in the list of window > >> functions with optimizations (dense-rank.diff). > > > This one might have been forgotten... ? I can push it shortly if nobody > > objects. > > Yeah, I missed that one. We're theoretically in the wrap freeze for > 15rc1, but I don't have a problem with release-note changes. Thanks. I've just pushed it. David
Re: First draft of the PG 15 release notes
David Rowley writes: > On Tue, 13 Sept 2022 at 09:31, Jonathan S. Katz wrote: >> Separately, per[1], including dense_rank() in the list of window >> functions with optimizations (dense-rank.diff). > This one might have been forgotten... ? I can push it shortly if nobody > objects. Yeah, I missed that one. We're theoretically in the wrap freeze for 15rc1, but I don't have a problem with release-note changes. regards, tom lane
Re: First draft of the PG 15 release notes
On Tue, 13 Sept 2022 at 09:31, Jonathan S. Katz wrote: > Separately, per[1], including dense_rank() in the list of window > functions with optimizations (dense-rank.diff). This one might have been forgotten... ? I can push it shortly if nobody objects. David > [1] > https://www.postgresql.org/message-id/CAApHDvpr6N7egNfSttGdQMfL%2BKYBjUb_Zf%2BrHULb7_2k4V%3DGGg%40mail.gmail.com
Re: First draft of the PG 15 release notes
Justin Pryzby writes: > On Fri, Sep 23, 2022 at 01:33:07PM -0400, Tom Lane wrote: > +Previously such files were left in the current directory, > +requiring manual cleanup. It's still necessary to remove them > +manually afterwards, but now one can just remove that whole > +subdirectory. > If pg_upgrade succeeds, then it removes the dir itself (so it's not > "necessary"). Ah, I'd only ever paid attention to failure cases, so I didn't realize that :-(. Text adjusted: Previously such files were left in the current directory, requiring manual cleanup. Now they are automatically removed on successful completion of pg_upgrade. I took most of your other suggestions, too. Thanks! regards, tom lane
Re: First draft of the PG 15 release notes
On Fri, Sep 23, 2022 at 01:33:07PM -0400, Tom Lane wrote: > "Jonathan S. Katz" writes: > > On 9/23/22 11:25 AM, Tom Lane wrote: > >> I'm planning to do a final(?) pass over the v15 notes today, > >> but I thought it'd be appropriate to push this separately. > > > RE "final pass", there's still an errant "BACKPATCHED"[1] that still > > needs addressing. I didn't have a chance to verify if it was indeed > > backpatched. > > Yeah, that one indeed needs removed (and I've done so). I see a > few other places where Bruce left notes about things that need more > clarification. I'm just finishing a pass of "update for subsequent > commits", and then I'll start on copy-editing. Some possible changes for your consideration. +Store pg_upgrade's log and +temporary files in a subdirectory of the new cluster called +pg_upgrade_output.d (Justin Pryzby) +Previously such files were left in the current directory, +requiring manual cleanup. It's still necessary to remove them +manually afterwards, but now one can just remove that whole +subdirectory. If pg_upgrade succeeds, then it removes the dir itself (so it's not "necessary"). And if it fails after starting to restore the schema, then it's necessary to remove not the "subdirectory" but the whole new-cluster dir. +Make pg_upgrade preserve tablespace +and database OIDs, as well as table relfilenode numbers s/table/relation/ ? You changed this to use spaces: |The new setting is log_destination = jsonlog. but then left these without spaces: |and wal_level=minimal. |This is enabled via --compress=lz4 and requires + value, use the transaction start time not wall clock time to s/not/rather than/ ? +Adjust psql so that Readline's should use Readline ? +Previously a pound marker was inserted, but that's pretty +unhelpful in SQL. This sounds more like a candid commit message than a release note. +Improve performance of dumping databases with many objects s/of/when/ ? + New options are server to write the *The* new options +In some cases a partition child table could appear more than once. Technically "partition child table" is redundant -- Justin
Re: First draft of the PG 15 release notes
On 9/23/22 1:33 PM, Tom Lane wrote: "Jonathan S. Katz" writes: On 9/23/22 11:25 AM, Tom Lane wrote: I'm planning to do a final(?) pass over the v15 notes today, but I thought it'd be appropriate to push this separately. RE "final pass", there's still an errant "BACKPATCHED"[1] that still needs addressing. I didn't have a chance to verify if it was indeed backpatched. Yeah, that one indeed needs removed (and I've done so). I see a few other places where Bruce left notes about things that need more clarification. I'm just finishing a pass of "update for subsequent commits", and then I'll start on copy-editing. ACK. I will available to review during the weekend (Sunday). Jonathan OpenPGP_signature Description: OpenPGP digital signature
Re: First draft of the PG 15 release notes
"Jonathan S. Katz" writes: > On 9/23/22 11:25 AM, Tom Lane wrote: >> I'm planning to do a final(?) pass over the v15 notes today, >> but I thought it'd be appropriate to push this separately. > RE "final pass", there's still an errant "BACKPATCHED"[1] that still > needs addressing. I didn't have a chance to verify if it was indeed > backpatched. Yeah, that one indeed needs removed (and I've done so). I see a few other places where Bruce left notes about things that need more clarification. I'm just finishing a pass of "update for subsequent commits", and then I'll start on copy-editing. regards, tom lane
Re: First draft of the PG 15 release notes
On 9/23/22 11:25 AM, Tom Lane wrote: "Jonathan S. Katz" writes: Adjusted to be similar to your suggestion. Updated patch attached. I pushed this with a bit more copy-editing. I'm planning to do a final(?) pass over the v15 notes today, but I thought it'd be appropriate to push this separately. Thanks! RE "final pass", there's still an errant "BACKPATCHED"[1] that still needs addressing. I didn't have a chance to verify if it was indeed backpatched. Jonathan [1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=doc/src/sgml/release-15.sgml;hb=refs/heads/REL_15_STABLE#l460 OpenPGP_signature Description: OpenPGP digital signature
Re: First draft of the PG 15 release notes
"Jonathan S. Katz" writes: > Adjusted to be similar to your suggestion. Updated patch attached. I pushed this with a bit more copy-editing. I'm planning to do a final(?) pass over the v15 notes today, but I thought it'd be appropriate to push this separately. regards, tom lane
Re: First draft of the PG 15 release notes
On 9/13/22 7:13 AM, Alvaro Herrera wrote: On 2022-Sep-12, Jonathan S. Katz wrote: + + + Column-level and row-level filtering on + logical replication + publications. + + -column-level filtering +the ability to specify column lists Adjusted to be similar to your suggestion. Updated patch attached. Thanks, Jonathan diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index 14440be77f..76fd071332 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -3,7 +3,7 @@ Release date: - AS OF 2022-06-11 + AS OF 2022-09-13 @@ -15,7 +15,38 @@ - + + + Performance improvements for in-memory and on-disk sorting. + + + + + Support for the SQL MERGE + command. + + + + + The ability to specify column-lists and row-level filters on + logical replication + publications. + + + + + More options for compression, including support for Zstandard (zstd) + compression. This includes support for server-side compression for + +pg_basebackup. + + + + + Structured log output using + the JSON format. + + OpenPGP_signature Description: OpenPGP digital signature
Re: First draft of the PG 15 release notes
On 2022-Sep-12, Jonathan S. Katz wrote: > + > + > + Column-level and row-level filtering on > + logical replication > + publications. > + > + -column-level filtering +the ability to specify column lists > + Row-level filtering and the ability to specify column lists on > + logical replication > + publications. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "No tengo por qué estar de acuerdo con lo que pienso" (Carlos Caszeli)
Re: First draft of the PG 15 release notes
On 9/4/22 2:42 PM, Nathan Bossart wrote: I noticed that the v15 release notes still refer to pg_checkpointer, which was renamed to pg_checkpoint in b9eb0ff. diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index d432c2db44..362728753a 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -1255,7 +1255,7 @@ Author: Jeff Davis Add predefined role pg_checkpointer + linkend="predefined-roles-table">pg_checkpoint that allows members to run CHECKPOINT (Jeff Davis) Nudging on folks to review the major features language for the docs (pg15-maj-features.patch). Separately, per[1], including dense_rank() in the list of window functions with optimizations (dense-rank.diff). Thanks, Jonathan [1] https://www.postgresql.org/message-id/CAApHDvpr6N7egNfSttGdQMfL%2BKYBjUb_Zf%2BrHULb7_2k4V%3DGGg%40mail.gmail.com diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index d432c2db44..7387d493d4 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -3,7 +3,7 @@ Release date: - AS OF 2022-06-11 + AS OF 2022-09-04 @@ -15,7 +15,38 @@ - + + + Performance improvements for in-memory and on-disk sorting. + + + + + Support for the SQL MERGE + command. + + + + + Column-level and row-level filtering on + logical replication + publications. + + + + + More options for compression, including support for Zstandard (zstd) + compression. This includes support for server-side compression for + +pg_basebackup. + + + + + Structured log output using + the JSON format. + + diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index 14440be77f..0c049cb47d 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -1016,7 +1047,8 @@ Author: David Rowley Improve the performance of window functions that use row_number(), -rank(), and count() +rank(), dense_rank() and +count() (David Rowley) OpenPGP_signature Description: OpenPGP digital signature
Re: First draft of the PG 15 release notes
I noticed that the v15 release notes still refer to pg_checkpointer, which was renamed to pg_checkpoint in b9eb0ff. diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index d432c2db44..362728753a 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -1255,7 +1255,7 @@ Author: Jeff Davis Add predefined role pg_checkpointer + linkend="predefined-roles-table">pg_checkpoint that allows members to run CHECKPOINT (Jeff Davis) -- Nathan Bossart Amazon Web Services: https://aws.amazon.com
Re: First draft of the PG 15 release notes
On 5/10/22 11:44 AM, Bruce Momjian wrote: I have completed the first draft of the PG 15 release notes I assume there will be major adjustments in the next few weeks based on feedback. I wanted to propose the "major enhancements" section to see if we can get an iteration in prior to Beta 4. Please see attached patched. Do we want to include anything else, or substitute any of the items? Thanks, Jonathan diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index d432c2db44..7387d493d4 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -3,7 +3,7 @@ Release date: - AS OF 2022-06-11 + AS OF 2022-09-04 @@ -15,7 +15,38 @@ - + + + Performance improvements for in-memory and on-disk sorting. + + + + + Support for the SQL MERGE + command. + + + + + Column-level and row-level filtering on + logical replication + publications. + + + + + More options for compression, including support for Zstandard (zstd) + compression. This includes support for server-side compression for + +pg_basebackup. + + + + + Structured log output using + the JSON format. + + OpenPGP_signature Description: OpenPGP digital signature
Re: First draft of the PG 15 release notes
On Wed, Aug 31, 2022 at 04:03:06PM -0400, Bruce Momjian wrote: > On Wed, Aug 31, 2022 at 11:38:33AM +0200, Peter Eisentraut wrote: > > On 30.08.22 22:42, Bruce Momjian wrote: > > > I just can't figure out what the user needs to understand about this, > > > and I understand very little of it. > > > > We already had this feature for (schema-level) collations, now we have it on > > the level of the database collation. > > Okay, I figured out the interplay between OS collation version support, > collation libraries, and collation levels. Here is an updated patch for > the release notes. Patch applied. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Wed, Aug 31, 2022 at 11:38:33AM +0200, Peter Eisentraut wrote: > On 30.08.22 22:42, Bruce Momjian wrote: > > Good question --- the full text is: > > > > > > > > Record and check the collation of each > linkend="sql-createdatabase">database (Peter Eisentraut) > > > > > > > > This is designed to detect collation > > mismatches to avoid data corruption. Function > > pg_database_collation_actual_version() > > reports the underlying operating system collation version, and > > ALTER DATABASE ... REFRESH sets the database > > to match the operating system collation version. DETAILS? > > > > > > > > I just can't figure out what the user needs to understand about this, > > and I understand very little of it. > > We already had this feature for (schema-level) collations, now we have it on > the level of the database collation. Okay, I figured out the interplay between OS collation version support, collation libraries, and collation levels. Here is an updated patch for the release notes. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index e3ab13e53c..dca4550c03 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -578,17 +578,17 @@ Author: Peter Eisentraut - Record and check the collation of each database (Peter Eisentraut) - This is designed to detect collation + This is designed to detect collation version mismatches to avoid data corruption. Function pg_database_collation_actual_version() reports the underlying operating system collation version, and ALTER DATABASE ... REFRESH sets the database - to match the operating system collation version. DETAILS? + to match the operating system collation version. @@ -605,9 +605,11 @@ Author: Peter Eisentraut - Previously, ICU collations could only be - specified in CREATE - COLLATION and used with the + Previously, only libc-based + collations could be set at the cluster and database levels. + ICU collations were previously limited + to CREATE + COLLATION and referenced by the COLLATE clause.
Re: First draft of the PG 15 release notes
On 30.08.22 22:42, Bruce Momjian wrote: Good question --- the full text is: Record and check the collation of each database (Peter Eisentraut) This is designed to detect collation mismatches to avoid data corruption. Function pg_database_collation_actual_version() reports the underlying operating system collation version, and ALTER DATABASE ... REFRESH sets the database to match the operating system collation version. DETAILS? I just can't figure out what the user needs to understand about this, and I understand very little of it. We already had this feature for (schema-level) collations, now we have it on the level of the database collation.
Re: First draft of the PG 15 release notes
On Sat, Aug 27, 2022 at 04:03:02PM +0200, Matthias van de Meent wrote: > Hi, > > I noticed a stray "DETAILS?" marker while going through the release > notes for 15. Is that subsection still under construction or review? > > > > > > > Record and check the collation of each > linkend="sql-createdatabase">database (Peter Eisentraut) > > [...] > > to match the operating system collation version. DETAILS? > > > > Good question --- the full text is: Record and check the collation of each database (Peter Eisentraut) This is designed to detect collation mismatches to avoid data corruption. Function pg_database_collation_actual_version() reports the underlying operating system collation version, and ALTER DATABASE ... REFRESH sets the database to match the operating system collation version. DETAILS? I just can't figure out what the user needs to understand about this, and I understand very little of it. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
Hi, I noticed a stray "DETAILS?" marker while going through the release notes for 15. Is that subsection still under construction or review? > > > Record and check the collation of eachlinkend="sql-createdatabase">database (Peter Eisentraut) > [...] > to match the operating system collation version. DETAILS? > > Kind regards, Matthias van de Meent
Re: First draft of the PG 15 release notes
On Tue, Jul 12, 2022 at 02:47:07PM -0400, Bruce Momjian wrote: > On Mon, Jul 11, 2022 at 11:31:32PM -0700, Noah Misch wrote: > > On Mon, Jul 11, 2022 at 12:39:57PM -0400, Bruce Momjian wrote: > > > I had trouble reading the sentences in the order you used so I > > > restructured it: > > > > > > The new default is one of the secure schema usage patterns that > > linkend="ddl-schemas-patterns"/> has recommended since the security > > > release for CVE-2018-1058. The change applies to newly-created > > > databases in existing clusters and for new clusters. Upgrading a > > > cluster or restoring a database dump will preserve existing permissions. > > > > I agree with the sentence order change. > > Great. > > > > For existing databases, especially those having multiple users, consider > > > issuing REVOKE to adopt this new default. For new > > > databases having zero need to defend against insider threats, granting > > > USAGE permission on their public > > > schemas will yield the behavior of prior releases. > > > > s/USAGE/CREATE/ in the last sentence. Looks good with that change. > > Ah, yes, of course. Patch applied, I also adjusted the second paragraph to be more symmetric. You can see the results here: https://momjian.us/pgsql_docs/release-15.html -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Tue, Jul 19, 2022 at 01:13:07PM -0500, Justin Pryzby wrote: > On Tue, Jul 19, 2022 at 01:24:30PM -0400, Bruce Momjian wrote: > > Well, I put the --no-synchronized-snapshots item in incompatibilities > > since it is a user-visible change that might require script adjustments. > > However, I put the limit of pg_dump to 9.2 and greater into the pg_dump > > section. Are you suggesting I move the--no-synchronized-snapshots item > > down there? That doesn't match with the way I have listed other > > incompatibilities so I am resistant to do that. > > I'd rather see the "limit support to v9.2" be moved or added to the > "incompatibilities" section, maybe with "remove --no-synchronized-snapshots" > as a secondary sentence. Is removing support for an older version an incompatibility --- I didn't think so. > > > > 0. Add support for LZ4 and Zstandard compression of server-side base > > > > backups (Jeevan Ladhe, Robert Haas) > > > > 1. Allow pg_basebackup to use LZ4 and Zstandard compression on > > > > server-side base backup files (Dipesh Pandit, Jeevan Ladhe) > > > > 2. Allow pg_basebackup's --compress option to control the compression > > > > method and options (Michael Paquier, Robert Haas) > > > >New options include server-gzip (gzip on the server), client-gzip > > > > (same as gzip). > > > > 3. Allow pg_basebackup to compress on the server side and decompress on > > > > the client side before storage (Dipesh Pandit) > > > >This is accomplished by specifying compression on the server side > > > > and plain output format. > > > > > > I still think these expose the incremental development rather than the > > > user-facing change. > > > > > 1. It seems wrong to say "server-side" since client-side compression with > > > LZ4/zstd is also supported. > > > > Agreed. I changed it to: > > > >Allow pg_basebackup to do LZ4 and Zstandard server-side compression > >on base backup files (Dipesh Pandit, Jeevan Ladhe) > > This still misses the point that those compression algs are also supported on > the client side, so it seems misleading to mention "server-side" support. I reworked that paragraph in the attached patch. What we did was to add server-side gzip/LZ/ZSTD, and client-side LZ/ZSTD. (We already had client-side gzip.) Hopefully the new text is clearer. You can see the new output here: https://momjian.us/pgsql_docs/release-15.html > > > > Allow custom scan provders to indicate if they support projections > > > > (Sven Klemm) > > > > The default is now that custom scan providers can't support > > > > projections, so they need to be updated for this release. > > > > > > Per the commit message, they don't "need" to be updated. > > > I think this should say "The default now assumes that a custom scan > > > provider > > > does not support projections; to retain optimal performance, they should > > > be > > > updated to indicate whether that's supported. > > > > Okay, I went with this text: > > > > The default is now that custom scan providers are assumed to not > > support projections; those that do need to be updated for this > > release. > > I'd say "those that do *will need to be updated" otherwise the sentence can > sound like it means "those that need to be updated [will] ..." Oh, good point, done. Cumulative patch attached. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml new file mode 100644 index dfa3c74..1cf6375 *** a/doc/src/sgml/release-15.sgml --- b/doc/src/sgml/release-15.sgml *** Author: Tom Lane *** 544,551 The default is now that custom scan providers are assumed to not ! support projections; those that do need to be updated for this ! release. --- 544,551 The default is now that custom scan providers are assumed to not ! support projections; those that do will need to be updated for ! this release. *** Author: Robert Haas 2022-03-08 [7cf085f07] Add support for zstd base backup compression. --> !Allow pg_basebackup to do LZ4 and !Zstandard server-side compression on base backup files (Dipesh !Pandit, Jeevan Ladhe) - - - - !Allow pg_basebackup's !--compress option to control the compression !method and options (Michael Paquier, Robert Haas) --- 2495,2515 2022-02-11 [751b8d23b] pg_basebackup: Allow client-side LZ4 (de)compression. Author: Robert Haas 2022-03-08 [7cf085f07] Add support for zstd base backup compression. + Author: Robert Haas + 2022-01-24 [0ad803291] Server-side gzip compression. --> !Allow
Re: First draft of the PG 15 release notes
On Tue, Jul 19, 2022 at 01:24:30PM -0400, Bruce Momjian wrote: > > > Remove pg_dump's --no-synchronized-snapshots option since all supported > > > server versions support synchronized snapshots (Tom Lane) > > > > It'd be better to put that after the note about dropping support for > > upgrading > > clusters older than v9.2 in psql/pg_dump/pg_upgrade. > > Well, I put the --no-synchronized-snapshots item in incompatibilities > since it is a user-visible change that might require script adjustments. > However, I put the limit of pg_dump to 9.2 and greater into the pg_dump > section. Are you suggesting I move the--no-synchronized-snapshots item > down there? That doesn't match with the way I have listed other > incompatibilities so I am resistant to do that. I'd rather see the "limit support to v9.2" be moved or added to the "incompatibilities" section, maybe with "remove --no-synchronized-snapshots" as a secondary sentence. > > > 0. Add support for LZ4 and Zstandard compression of server-side base > > > backups (Jeevan Ladhe, Robert Haas) > > > 1. Allow pg_basebackup to use LZ4 and Zstandard compression on > > > server-side base backup files (Dipesh Pandit, Jeevan Ladhe) > > > 2. Allow pg_basebackup's --compress option to control the compression > > > method and options (Michael Paquier, Robert Haas) > > >New options include server-gzip (gzip on the server), client-gzip > > > (same as gzip). > > > 3. Allow pg_basebackup to compress on the server side and decompress on > > > the client side before storage (Dipesh Pandit) > > >This is accomplished by specifying compression on the server side and > > > plain output format. > > > > I still think these expose the incremental development rather than the > > user-facing change. > > > 1. It seems wrong to say "server-side" since client-side compression with > > LZ4/zstd is also supported. > > Agreed. I changed it to: > >Allow pg_basebackup to do LZ4 and Zstandard server-side compression >on base backup files (Dipesh Pandit, Jeevan Ladhe) This still misses the point that those compression algs are also supported on the client side, so it seems misleading to mention "server-side" support. > > > Allow custom scan provders to indicate if they support projections (Sven > > > Klemm) > > > The default is now that custom scan providers can't support projections, > > > so they need to be updated for this release. > > > > Per the commit message, they don't "need" to be updated. > > I think this should say "The default now assumes that a custom scan provider > > does not support projections; to retain optimal performance, they should be > > updated to indicate whether that's supported. > > Okay, I went with this text: > > The default is now that custom scan providers are assumed to not > support projections; those that do need to be updated for this > release. I'd say "those that do *will need to be updated" otherwise the sentence can sound like it means "those that need to be updated [will] ..." Thanks, -- Justin
Re: First draft of the PG 15 release notes
On Mon, Jul 18, 2022 at 08:23:23PM -0500, Justin Pryzby wrote: > > Increase hash_mem_multiplier default to 2.0 (Peter Geoghegan) > > This allows query hash operations to use double the amount of work_mem > > memory as other operations. > > I wonder if it's worth pointing out that a query may end up using not just 2x > more memory (since work_mem*hash_mem_multiplier is per node), but 2**N more, > for N nodes. Uh, I said "per operation" so people might realize there can be multiple work_mem memory operations per query. I don't think I can do more in this text. I can't think of a way to improve it without making it more confusing. > > Remove pg_dump's --no-synchronized-snapshots option since all supported > > server versions support synchronized snapshots (Tom Lane) > > It'd be better to put that after the note about dropping support for upgrading > clusters older than v9.2 in psql/pg_dump/pg_upgrade. Well, I put the --no-synchronized-snapshots item in incompatibilities since it is a user-visible change that might require script adjustments. However, I put the limit of pg_dump to 9.2 and greater into the pg_dump section. Are you suggesting I move the--no-synchronized-snapshots item down there? That doesn't match with the way I have listed other incompatibilities so I am resistant to do that. > > Enable system and TOAST btree indexes to efficiently store duplicates > > (Peter Geoghegan) > > Say "btree indexes on system [and TOAST] tables" Okay, updated to: Allow btree indexes on system and TOAST tables to efficiently store duplicates (Peter Geoghegan) > > Prevent changes to columns only indexed by BRIN indexes from disabling HOT > > updates (Josef Simanek) > > This was reverted Ah, yes, removed. > > Generate periodic log message during slow server starts (Nitin Jadhav, > > Robert Haas) > messages plural > > > Messages report the cause of the delay. The time interval for notification > > is controlled by the new server variable log_startup_progress_interval. > *The messages Ah, yes, fixed. > > Add server variable shared_memory_size to report the size of allocated > > shared memory (Nathan Bossart) > > Add server variable shared_memory_size_in_huge_pages to report the number > > of huge memory pages required (Nathan Bossart) > > Maybe these should say server *parameter* since they're not really "variable". Uh, I think of parameters as something passed. We do call them server variables, or read-only server variables. I can add "read-only" but it seems odd. > > 0. Add support for LZ4 and Zstandard compression of server-side base > > backups (Jeevan Ladhe, Robert Haas) > > 1. Allow pg_basebackup to use LZ4 and Zstandard compression on server-side > > base backup files (Dipesh Pandit, Jeevan Ladhe) > > 2. Allow pg_basebackup's --compress option to control the compression > > method and options (Michael Paquier, Robert Haas) > >New options include server-gzip (gzip on the server), client-gzip (same > > as gzip). > > 3. Allow pg_basebackup to compress on the server side and decompress on the > > client side before storage (Dipesh Pandit) > >This is accomplished by specifying compression on the server side and > > plain output format. > > I still think these expose the incremental development rather than the > user-facing change. Well, they are in different parts of the system, though they are clearly all related. I am afraid merging them would be even more confusing. > 1. It seems wrong to say "server-side" since client-side compression with > LZ4/zstd is also supported. Agreed. I changed it to: Allow pg_basebackup to do LZ4 and Zstandard server-side compression on base backup files (Dipesh Pandit, Jeevan Ladhe) > 2. It's confusing to say that the new options are server-gzip and client-gzip, > since it just mentioned new algorithms; I see your point since there will be new options for LZ4 and Zstandard too, so I just removed that paragraph. > 3. I'm not sure this needs to be mentioned at all; maybe it should be a > "detail" following the item about server-side compression. See my concerns above --- it seems too complex to merge into something else. However, I am open to an entire rewrite of these items. > > Tables added to the listed schemas in the future will also be replicated. > > "Tables later added" is clearer. Otherwise "in the future" sounds like maybe > in v16 or v17 we'll start replicating those tables. Agreed, new wording: Tables added later to the listed schemas will also be replicated. > > Allow subscribers to stop logical replication application on error (Osumi > > Takamichi, Mark Dilger) > "application" sounds off. Agreed. New text is: Allow subscribers to stop the application of logical replication changes on error > > Add new default WAL-logged method for database creation (Dilip Kumar) > "New default" sounds off. Say "Add new WAL-logged method for database > creation, used by
Re: First draft of the PG 15 release notes
> Increase hash_mem_multiplier default to 2.0 (Peter Geoghegan) > This allows query hash operations to use double the amount of work_mem memory > as other operations. I wonder if it's worth pointing out that a query may end up using not just 2x more memory (since work_mem*hash_mem_multiplier is per node), but 2**N more, for N nodes. > Remove pg_dump's --no-synchronized-snapshots option since all supported > server versions support synchronized snapshots (Tom Lane) It'd be better to put that after the note about dropping support for upgrading clusters older than v9.2 in psql/pg_dump/pg_upgrade. > Enable system and TOAST btree indexes to efficiently store duplicates (Peter > Geoghegan) Say "btree indexes on system [and TOAST] tables" > Prevent changes to columns only indexed by BRIN indexes from disabling HOT > updates (Josef Simanek) This was reverted > Generate periodic log message during slow server starts (Nitin Jadhav, Robert > Haas) messages plural > Messages report the cause of the delay. The time interval for notification is > controlled by the new server variable log_startup_progress_interval. *The messages > Add server variable shared_memory_size to report the size of allocated shared > memory (Nathan Bossart) > Add server variable shared_memory_size_in_huge_pages to report the number of > huge memory pages required (Nathan Bossart) Maybe these should say server *parameter* since they're not really "variable". > 0. Add support for LZ4 and Zstandard compression of server-side base backups > (Jeevan Ladhe, Robert Haas) > 1. Allow pg_basebackup to use LZ4 and Zstandard compression on server-side > base backup files (Dipesh Pandit, Jeevan Ladhe) > 2. Allow pg_basebackup's --compress option to control the compression method > and options (Michael Paquier, Robert Haas) >New options include server-gzip (gzip on the server), client-gzip (same as > gzip). > 3. Allow pg_basebackup to compress on the server side and decompress on the > client side before storage (Dipesh Pandit) >This is accomplished by specifying compression on the server side and > plain output format. I still think these expose the incremental development rather than the user-facing change. 1. It seems wrong to say "server-side" since client-side compression with LZ4/zstd is also supported. 2. It's confusing to say that the new options are server-gzip and client-gzip, since it just mentioned new algorithms; 3. I'm not sure this needs to be mentioned at all; maybe it should be a "detail" following the item about server-side compression. > Tables added to the listed schemas in the future will also be replicated. "Tables later added" is clearer. Otherwise "in the future" sounds like maybe in v16 or v17 we'll start replicating those tables. > Allow subscribers to stop logical replication application on error (Osumi > Takamichi, Mark Dilger) "application" sounds off. > Add new default WAL-logged method for database creation (Dilip Kumar) "New default" sounds off. Say "Add new WAL-logged method for database creation, used by default". > Have pg_upgrade preserve relfilenodes, tablespace, and database OIDs between > old and new clusters (Shruthi KC, Antonin Houska) "tablespace OIDs" or "tablespace and database OIDs and relfilenodes" > Limit support of pg_upgrade to old servers running PostgreSQL 9.2 and later > (Tom Lane) The word "old" doesn't appear in the 2 release notes items about pg_dump and psql, and "old" makes it sound sounds like "antique" rather than "source". > Some internal-use-only types have also been assigned this column. this *value > Allow custom scan provders to indicate if they support projections (Sven > Klemm) > The default is now that custom scan providers can't support projections, so > they need to be updated for this release. Per the commit message, they don't "need" to be updated. I think this should say "The default now assumes that a custom scan provider does not support projections; to retain optimal performance, they should be updated to indicate whether that's supported.
Re: First draft of the PG 15 release notes
On Thu, Jul 14, 2022 at 01:24:41PM +0700, John Naylor wrote: > Regarding this item: > > "Allow hash lookup for NOT IN clauses with many constants (David Rowley, James > Coleman) > Previously the code always sequentially scanned the list of values." > > The todo list has an entry titled "Planning large IN lists", which links to > > https://www.postgresql.org/message-id/1178821226.6034.63.camel@goldbach > > Did we already have a hash lookup for IN clauses with constants and the above > commit adds NOT IN? If so, maybe we have enough to remove this todo item. Agreed, I have removed it now. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
Regarding this item: "Allow hash lookup for NOT IN clauses with many constants (David Rowley, James Coleman) Previously the code always sequentially scanned the list of values." The todo list has an entry titled "Planning large IN lists", which links to https://www.postgresql.org/message-id/1178821226.6034.63.camel@goldbach Did we already have a hash lookup for IN clauses with constants and the above commit adds NOT IN? If so, maybe we have enough to remove this todo item. -- John Naylor EDB: http://www.enterprisedb.com
Re: First draft of the PG 15 release notes
On Mon, Jul 11, 2022 at 11:31:32PM -0700, Noah Misch wrote: > On Mon, Jul 11, 2022 at 12:39:57PM -0400, Bruce Momjian wrote: > > I had trouble reading the sentences in the order you used so I > > restructured it: > > > > The new default is one of the secure schema usage patterns that > linkend="ddl-schemas-patterns"/> has recommended since the security > > release for CVE-2018-1058. The change applies to newly-created > > databases in existing clusters and for new clusters. Upgrading a > > cluster or restoring a database dump will preserve existing permissions. > > I agree with the sentence order change. Great. > > For existing databases, especially those having multiple users, consider > > issuing REVOKE to adopt this new default. For new > > databases having zero need to defend against insider threats, granting > > USAGE permission on their public > > schemas will yield the behavior of prior releases. > > s/USAGE/CREATE/ in the last sentence. Looks good with that change. Ah, yes, of course. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Mon, Jul 11, 2022 at 12:39:57PM -0400, Bruce Momjian wrote: > I had trouble reading the sentences in the order you used so I > restructured it: > > The new default is one of the secure schema usage patterns thatlinkend="ddl-schemas-patterns"/> has recommended since the security > release for CVE-2018-1058. The change applies to newly-created > databases in existing clusters and for new clusters. Upgrading a > cluster or restoring a database dump will preserve existing permissions. I agree with the sentence order change. > For existing databases, especially those having multiple users, consider > issuing REVOKE to adopt this new default. For new > databases having zero need to defend against insider threats, granting > USAGE permission on their public > schemas will yield the behavior of prior releases. s/USAGE/CREATE/ in the last sentence. Looks good with that change.
Re: First draft of the PG 15 release notes
On Mon, Jul 11, 2022 at 12:39:23PM -0500, Justin Pryzby wrote: > On Tue, May 10, 2022 at 11:41:08PM -0400, Bruce Momjian wrote: > > On Tue, May 10, 2022 at 08:31:17PM -0500, Justin Pryzby wrote: > > > > | Store server-level statistics in shared memory (Kyotaro Horiguchi, > > > Andres Freund, Melanie Plageman) > > > > > > Should this be called "cumulative" statistics? As in > > > b3abca68106d518ce5d3c0d9a1e0ec02a647ceda. > > > > Uh, they are counters, which I guess is cummulative, but that doesn't > > seem very descriptive. The documentation call it the statistics > > collector, but I am not sure we even have that anymore with an in-memory > > implementation. I am kind of not sure what to call it. > > What I was trying to say is that it's now called the cumulative stats system. It is actually called the "cumulative statistics system", so updated; patch attached and applied. > > > | New function > > > > > > "The new function .." (a few times) > > > > Uh, I only see it once. > > There's still a couple of these without "The". Ah, found them, fixed. > > > Should any of these be listed as incompatible changes (some of these I > > > asked > > > before, but the others are from another list). > > > > ccd10a9bfa5 Fix enforcement of PL/pgSQL variable CONSTANT markings (Tom > > > Lane) > > > > I didn't see not enforcing constant as an incompatibility, but rather a > > bug. > > Yes it's a bug, but it's going to be seen as a compatibility issue for someone > whose application breaks. The same goes for other things I mentioned. We don't guarantee that the only breakage is listed in the incompatibilities section, only the most common ones. > > > 376ce3e404b Prefer $HOME when looking up the current user's home > > > directory. > > > > Uh, I didn't think so. > > > > > 7844c9918a4 psql: Show all query results by default > > > > Same. > > > > > 17a856d08be Change aggregated log format of pgbench. > > > > We have a pgbench section and I can't see it. I am trying to keep > > incompatiblities as things related to in-production problems or > > surprises. > > > > > ? 73508475d69 Remove pg_atoi() > > > > I don't see who would care except for internals folks. > > > > > ? aa64f23b029 Remove MaxBackends variable in favor of GetMaxBackends() > > > function. > > > > Same. > > > > > ? d816f366bc4 psql: Make SSL info display more compact > > > > I did look at that but considered that this wouldn't be something that > > would break anything. > > > > > ? 27b02e070fd pg_upgrade: Don't print progress status when output is not > > > a tty. > > > > Same. > > > > > ? ab4fd4f868e Remove 'datlastsysoid'. > > > > Seemed too internal. > > FYI, removal of this column broke a tool one of my coworkers uses (navicat). > I'm told that the fix will be in navicat v16.1 (but their existing users will > need to pay to upgrade from v15). This actually supports my point --- only navicat needs to know about this renaming, it its users. Telling navicat users about this change does not help them. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index 179ad37d9d..cde78b6181 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -1100,8 +1100,8 @@ Author: Andres Freund -Store server-level -statistics in shared memory (Kyotaro Horiguchi, Andres +Store cumulative statistics +system data in shared memory (Kyotaro Horiguchi, Andres Freund, Melanie Plageman) @@ -1248,7 +1248,7 @@ Author: Tom Lane -New function has_parameter_privilege() +The new function has_parameter_privilege() reports on this privilege. @@ -1656,7 +1656,7 @@ Author: Amit Kapila -New function pg_stat_reset_subscription_stats() allows the resetting of subscriber statistics.
Re: First draft of the PG 15 release notes
On Tue, May 10, 2022 at 11:41:08PM -0400, Bruce Momjian wrote: > On Tue, May 10, 2022 at 08:31:17PM -0500, Justin Pryzby wrote: > > | Store server-level statistics in shared memory (Kyotaro Horiguchi, Andres > > Freund, Melanie Plageman) > > > > Should this be called "cumulative" statistics? As in > > b3abca68106d518ce5d3c0d9a1e0ec02a647ceda. > > Uh, they are counters, which I guess is cummulative, but that doesn't > seem very descriptive. The documentation call it the statistics > collector, but I am not sure we even have that anymore with an in-memory > implementation. I am kind of not sure what to call it. What I was trying to say is that it's now called the cumulative stats system. > > | New function > > > > "The new function .." (a few times) > > Uh, I only see it once. There's still a couple of these without "The". > > Should any of these be listed as incompatible changes (some of these I asked > > before, but the others are from another list). > > ccd10a9bfa5 Fix enforcement of PL/pgSQL variable CONSTANT markings (Tom > > Lane) > > I didn't see not enforcing constant as an incompatibility, but rather a > bug. Yes it's a bug, but it's going to be seen as a compatibility issue for someone whose application breaks. The same goes for other things I mentioned. > > 376ce3e404b Prefer $HOME when looking up the current user's home directory. > > Uh, I didn't think so. > > > 7844c9918a4 psql: Show all query results by default > > Same. > > > 17a856d08be Change aggregated log format of pgbench. > > We have a pgbench section and I can't see it. I am trying to keep > incompatiblities as things related to in-production problems or > surprises. > > > ? 73508475d69 Remove pg_atoi() > > I don't see who would care except for internals folks. > > > ? aa64f23b029 Remove MaxBackends variable in favor of GetMaxBackends() > > function. > > Same. > > > ? d816f366bc4 psql: Make SSL info display more compact > > I did look at that but considered that this wouldn't be something that > would break anything. > > > ? 27b02e070fd pg_upgrade: Don't print progress status when output is not a > > tty. > > Same. > > > ? ab4fd4f868e Remove 'datlastsysoid'. > > Seemed too internal. FYI, removal of this column broke a tool one of my coworkers uses (navicat). I'm told that the fix will be in navicat v16.1 (but their existing users will need to pay to upgrade from v15). -- Justin
Re: First draft of the PG 15 release notes
On Sat, Jul 9, 2022 at 08:19:41PM -0700, Noah Misch wrote: > > I think you would need to say "previous behavior" since people might be > > upgrading from releases before PG 14. I also would change "In existing > > I felt "previous behavior" was mildly ambiguous. I've changed it to "the > behavior of prior releases". Sure. > > > databases" to "For existing databases". I think your big risk here is > > Done. New version attached. I had trouble reading the sentences in the order you used so I restructured it: The new default is one of the secure schema usage patterns that has recommended since the security release for CVE-2018-1058. The change applies to newly-created databases in existing clusters and for new clusters. Upgrading a cluster or restoring a database dump will preserve existing permissions. For existing databases, especially those having multiple users, consider issuing REVOKE to adopt this new default. For new databases having zero need to defend against insider threats, granting USAGE permission on their public schemas will yield the behavior of prior releases. > > Is this something we want to get into in the release notes, or perhaps > > do we need to link to a wiki page for these details? > > No supported release has a wiki page link in its release notes. We used wiki > pages in the more-distant past, but I don't recall why. I am not aware of > wiki pages having relevant benefits. I think the wiki was good if you needed a lot of release-specific text, or if you wanted to adjust the wording after the release. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Wed, Jul 06, 2022 at 09:10:53AM -0400, Bruce Momjian wrote: > On Tue, Jul 5, 2022 at 07:45:57PM -0700, Noah Misch wrote: > > On Tue, Jul 05, 2022 at 07:47:52PM -0400, Bruce Momjian wrote: > > > Yes, I think it is a question of practicality vs. desirability. We are > > > basically telling people they have to do research to get the old > > > behavior in their new databases and clusters. > > > > True. I want to maximize the experience for different classes of database: > > > > 1. Databases needing user isolation and unknowingly not getting it. > > 2. Databases not needing user isolation, e.g. automated test environments. > > > > Expecting all of these DBAs to read a 500-word doc section is failure-prone. > > For the benefit of (2), I'm now thinking about adding a release note > > sentence, > > "For a new database having zero need to defend against insider threats, > > granting back the privilege yields the PostgreSQL 14 behavior." > > I think you would need to say "previous behavior" since people might be > upgrading from releases before PG 14. I also would change "In existing I felt "previous behavior" was mildly ambiguous. I've changed it to "the behavior of prior releases". > databases" to "For existing databases". I think your big risk here is Done. New version attached. > trying to explain how to have new clusters get the old or new behavior > in the same text block, e.g.: > > The new default is one of the secure schema usage patterns that > has recommended since the > security release for CVE-2018-1058. Upgrading a cluster or restoring a > database dump will preserve existing permissions. This is a change in > the default for newly-created databases in existing clusters and for new > clusters. For existing databases, especially those having multiple > users, consider issuing a REVOKE to adopt this new > default. (USAGE permission on this schema has not > changed.) For a new database having zero need to defend against insider > threats, granting back the privilege yields the previous behavior. > > Is this something we want to get into in the release notes, or perhaps > do we need to link to a wiki page for these details? No supported release has a wiki page link in its release notes. We used wiki pages in the more-distant past, but I don't recall why. I am not aware of wiki pages having relevant benefits. Author: Noah Misch Commit: Noah Misch diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index 179ad37..3ed986c 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -58,16 +58,17 @@ Author: Noah Misch - This is a change in the default for newly-created databases in - existing clusters and for new clusters; USAGE - permissions on the public schema has not - been changed. Databases restored from previous Postgres releases - will be restored with their current permissions. Users wishing - to have the former permissions will need to grant - CREATE permission for PUBLIC - on the public schema; this change can be made - on template1 to cause all new databases - to have these permissions. + The new default is one of the secure schema usage patterns that + has recommended since the + security release for CVE-2018-1058. Upgrading a cluster or restoring a + database dump will preserve existing permissions. This is a change in + the default for newly-created databases in existing clusters and for new + clusters. For existing databases, especially those having multiple + users, consider issuing REVOKE to adopt this new + default. For a new database having zero need to defend against insider + threats, granting back the privilege yields the behavior of prior + releases. (USAGE permission on this schema has not + changed.)
Re: First draft of the PG 15 release notes
On Tue, Jul 5, 2022 at 07:45:57PM -0700, Noah Misch wrote: > On Tue, Jul 05, 2022 at 07:47:52PM -0400, Bruce Momjian wrote: > > Yes, I think it is a question of practicality vs. desirability. We are > > basically telling people they have to do research to get the old > > behavior in their new databases and clusters. > > True. I want to maximize the experience for different classes of database: > > 1. Databases needing user isolation and unknowingly not getting it. > 2. Databases not needing user isolation, e.g. automated test environments. > > Expecting all of these DBAs to read a 500-word doc section is failure-prone. > For the benefit of (2), I'm now thinking about adding a release note sentence, > "For a new database having zero need to defend against insider threats, > granting back the privilege yields the PostgreSQL 14 behavior." I think you would need to say "previous behavior" since people might be upgrading from releases before PG 14. I also would change "In existing databases" to "For existing databases". I think your big risk here is trying to explain how to have new clusters get the old or new behavior in the same text block, e.g.: The new default is one of the secure schema usage patterns that has recommended since the security release for CVE-2018-1058. Upgrading a cluster or restoring a database dump will preserve existing permissions. This is a change in the default for newly-created databases in existing clusters and for new clusters. For existing databases, especially those having multiple users, consider issuing a REVOKE to adopt this new default. (USAGE permission on this schema has not changed.) For a new database having zero need to defend against insider threats, granting back the privilege yields the previous behavior. Is this something we want to get into in the release notes, or perhaps do we need to link to a wiki page for these details? -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Tue, Jul 05, 2022 at 07:47:52PM -0400, Bruce Momjian wrote: > On Tue, Jul 5, 2022 at 02:57:52PM -0700, Noah Misch wrote: > > Since having too-permissive ACLs is usually symptom-free, I share your > > forecast about the more-common question. Expect questions on mailing lists, > > stackoverflow, etc. The right way to answer those questions is roughly > > this: > > > > > On PostgreSQL 15, my application gets "permission denied for schema > > > public". What should I do? > > > > You have a choice to make. The best selection depends on the security > > needs of your database. See > > > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > for a guide to making that choice. > > > > Recommending GRANT to that two-sentence question would be negligent. One > > should know a database's lack of security needs before recommending GRANT. > > This is a key opportunity to have more users make the right decision while > > their attention is on the topic. > > Yes, I think it is a question of practicality vs. desirability. We are > basically telling people they have to do research to get the old > behavior in their new databases and clusters. True. I want to maximize the experience for different classes of database: 1. Databases needing user isolation and unknowingly not getting it. 2. Databases not needing user isolation, e.g. automated test environments. Expecting all of these DBAs to read a 500-word doc section is failure-prone. For the benefit of (2), I'm now thinking about adding a release note sentence, "For a new database having zero need to defend against insider threats, granting back the privilege yields the PostgreSQL 14 behavior."
Re: First draft of the PG 15 release notes
On Tue, Jul 5, 2022 at 02:57:52PM -0700, Noah Misch wrote: > Since having too-permissive ACLs is usually symptom-free, I share your > forecast about the more-common question. Expect questions on mailing lists, > stackoverflow, etc. The right way to answer those questions is roughly this: > > > On PostgreSQL 15, my application gets "permission denied for schema > > public". What should I do? > > You have a choice to make. The best selection depends on the security > needs of your database. See > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > for a guide to making that choice. > > Recommending GRANT to that two-sentence question would be negligent. One > should know a database's lack of security needs before recommending GRANT. > This is a key opportunity to have more users make the right decision while > their attention is on the topic. Yes, I think it is a question of practicality vs. desirability. We are basically telling people they have to do research to get the old behavior in their new databases and clusters. > > My only stylistic suggestion would be to remove "a" from "a > > REVOKE". > > I'll plan to push with that change. WFM. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Tue, Jul 05, 2022 at 04:35:32PM -0400, Bruce Momjian wrote: > On Tue, Jul 5, 2022 at 12:53:49PM -0700, Noah Misch wrote: > > On Tue, Jul 05, 2022 at 02:35:39PM -0400, Bruce Momjian wrote: > > > On Fri, Jul 1, 2022 at 06:21:28PM -0700, Noah Misch wrote: > > > > Here's what I've been trying to ask: what do you think of linking to > > > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > > > here? The release note text is still vague, and the docs have extensive > > > > coverage of the topic. The notes can just link to that extensive > > > > coverage. > > > > > > Sure. how is this patch? > > > > > --- a/doc/src/sgml/release-15.sgml > > > +++ b/doc/src/sgml/release-15.sgml > > > @@ -63,11 +63,12 @@ Author: Noah Misch > > >permissions on the public schema has not > > >been changed. Databases restored from previous Postgres releases > > >will be restored with their current permissions. Users wishing > > > - to have the former permissions will need to grant > > > + to have the former more-open permissions will need to grant > > >CREATE permission for PUBLIC > > >on the public schema; this change can be made > > >on template1 to cause all new databases > > > - to have these permissions. > > > + to have these permissions. This change was made to increase > > > + security; see . > > > > > > > > > > I think this still puts undue weight on single-user systems moving back to > > the > > old default. The linked documentation does say how to get back to v14 > > permissions (and disclaims security if you do so), so let's not mention it > > here. The attached is how I would write it. I also reworked the "Databases > > restored from previous ..." sentence, since its statement is also true of > > databases restored v15-to-v15 (no "previous" release involved). I also > > moved > > the bit about USAGE to end, since it's just emphasizing what the reader > > should > > already assume. Any concerns? > > I see where you are going --- to talk about how to convert upgraded > clusters to secure clusters, rather than how to revert to the previous > behavior. I assumed that the most common question would be how to get > the previous behavior, rather than how to get the new behavior in > upgraded clusters. However, I am fine with what you think is best. Since having too-permissive ACLs is usually symptom-free, I share your forecast about the more-common question. Expect questions on mailing lists, stackoverflow, etc. The right way to answer those questions is roughly this: > On PostgreSQL 15, my application gets "permission denied for schema > public". What should I do? You have a choice to make. The best selection depends on the security needs of your database. See https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS for a guide to making that choice. Recommending GRANT to that two-sentence question would be negligent. One should know a database's lack of security needs before recommending GRANT. This is a key opportunity to have more users make the right decision while their attention is on the topic. > My only stylistic suggestion would be to remove "a" from "a > REVOKE". I'll plan to push with that change.
Re: First draft of the PG 15 release notes
On Tue, Jul 5, 2022 at 12:53:49PM -0700, Noah Misch wrote: > On Tue, Jul 05, 2022 at 02:35:39PM -0400, Bruce Momjian wrote: > > On Fri, Jul 1, 2022 at 06:21:28PM -0700, Noah Misch wrote: > > > Here's what I've been trying to ask: what do you think of linking to > > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > > here? The release note text is still vague, and the docs have extensive > > > coverage of the topic. The notes can just link to that extensive > > > coverage. > > > > Sure. how is this patch? > > > --- a/doc/src/sgml/release-15.sgml > > +++ b/doc/src/sgml/release-15.sgml > > @@ -63,11 +63,12 @@ Author: Noah Misch > >permissions on the public schema has not > >been changed. Databases restored from previous Postgres releases > >will be restored with their current permissions. Users wishing > > - to have the former permissions will need to grant > > + to have the former more-open permissions will need to grant > >CREATE permission for PUBLIC > >on the public schema; this change can be made > >on template1 to cause all new databases > > - to have these permissions. > > + to have these permissions. This change was made to increase > > + security; see . > > > > > > I think this still puts undue weight on single-user systems moving back to the > old default. The linked documentation does say how to get back to v14 > permissions (and disclaims security if you do so), so let's not mention it > here. The attached is how I would write it. I also reworked the "Databases > restored from previous ..." sentence, since its statement is also true of > databases restored v15-to-v15 (no "previous" release involved). I also moved > the bit about USAGE to end, since it's just emphasizing what the reader should > already assume. Any concerns? I see where you are going --- to talk about how to convert upgraded clusters to secure clusters, rather than how to revert to the previous behavior. I assumed that the most common question would be how to get the previous behavior, rather than how to get the new behavior in upgraded clusters. However, I am fine with what you think is best. My only stylistic suggestion would be to remove "a" from "a REVOKE". -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Tue, Jul 05, 2022 at 02:35:39PM -0400, Bruce Momjian wrote: > On Fri, Jul 1, 2022 at 06:21:28PM -0700, Noah Misch wrote: > > Here's what I've been trying to ask: what do you think of linking to > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > here? The release note text is still vague, and the docs have extensive > > coverage of the topic. The notes can just link to that extensive coverage. > > Sure. how is this patch? > --- a/doc/src/sgml/release-15.sgml > +++ b/doc/src/sgml/release-15.sgml > @@ -63,11 +63,12 @@ Author: Noah Misch >permissions on the public schema has not >been changed. Databases restored from previous Postgres releases >will be restored with their current permissions. Users wishing > - to have the former permissions will need to grant > + to have the former more-open permissions will need to grant >CREATE permission for PUBLIC >on the public schema; this change can be made >on template1 to cause all new databases > - to have these permissions. > + to have these permissions. This change was made to increase > + security; see . > > I think this still puts undue weight on single-user systems moving back to the old default. The linked documentation does say how to get back to v14 permissions (and disclaims security if you do so), so let's not mention it here. The attached is how I would write it. I also reworked the "Databases restored from previous ..." sentence, since its statement is also true of databases restored v15-to-v15 (no "previous" release involved). I also moved the bit about USAGE to end, since it's just emphasizing what the reader should already assume. Any concerns? Author: Noah Misch Commit: Noah Misch diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index 179ad37..aa02ee9 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -58,16 +58,15 @@ Author: Noah Misch - This is a change in the default for newly-created databases in - existing clusters and for new clusters; USAGE - permissions on the public schema has not - been changed. Databases restored from previous Postgres releases - will be restored with their current permissions. Users wishing - to have the former permissions will need to grant - CREATE permission for PUBLIC - on the public schema; this change can be made - on template1 to cause all new databases - to have these permissions. + The new default is one of the secure schema usage patterns that + has recommended since the + security release for CVE-2018-1058. Upgrading a cluster or restoring a + database dump will preserve existing permissions. This is a change in + the default for newly-created databases in existing clusters and for new + clusters. In existing databases, especially those having multiple + users, consider issuing a REVOKE to adopt this new + default. (USAGE permission on this schema has not + changed.)
Re: First draft of the PG 15 release notes
On Tue, Jul 5, 2022 at 11:51:31AM -0700, Peter Geoghegan wrote: > On Tue, Jul 5, 2022 at 11:09 AM Bruce Momjian wrote: > > > > Actually, I was wrong. I thought that we only mentioned that we > > computed a more agressive xid, but now see I was mentioning the _frozen_ > > xid. Reading the commit, we do compute the multi-xid and store that too > > so I have updated the PG 15 release notes with the attached patch. > > It might be worth using the "symbol names" directly, since they appear > in the documentation already (under "Routine Vacuuming"). These are > relfrozenxid and > relminmxid. These are implementation > details, but they're documented in detail (though admittedly the > documentation has *lots* of problems). Well, users can look into the details if they wish. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Tue, Jul 5, 2022 at 11:09 AM Bruce Momjian wrote: > > Actually, I was wrong. I thought that we only mentioned that we > computed a more agressive xid, but now see I was mentioning the _frozen_ > xid. Reading the commit, we do compute the multi-xid and store that too > so I have updated the PG 15 release notes with the attached patch. It might be worth using the "symbol names" directly, since they appear in the documentation already (under "Routine Vacuuming"). These are relfrozenxid and relminmxid. These are implementation details, but they're documented in detail (though admittedly the documentation has *lots* of problems). Here is what I would like this item to hint at, to advanced users with tricky requirements: The new approach to setting relminmxid will improve the behavior of VACUUM in databases that already happen to use lots of MultiXacts. These users will notice that autovacuum now works off of relminmxid values that actually tell us something about each table's consumption of MultiXacts over time. Most individual tables naturally consume *zero* MultiXacts, even in databases that consume many MultiXacts -- due to naturally occuring workload characteristics. The old approach failed to recognize this, leading to very uniform relminmxid values across tables that were in fact very different, MultiXact-wise. The way that we handle relfrozenxid is probably much less likely to make life much easier for any database, at least on its own, in Postgres 15. So from the point of view of a user considering upgrading, the impact on relminmxid is likely to be far more important. Admittedly the most likely scenario by far is that the whole feature just isn't interesting, but a small minority of advanced users (users with painful MultiXact problems) will find the relminmxid thing very compelling. -- Peter Geoghegan
Re: First draft of the PG 15 release notes
On Fri, Jul 1, 2022 at 06:21:28PM -0700, Noah Misch wrote: > Here's what I've been trying to ask: what do you think of linking to > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > here? The release note text is still vague, and the docs have extensive > coverage of the topic. The notes can just link to that extensive coverage. Sure. how is this patch? -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index 179ad37d9d..3ad8c1d996 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -63,11 +63,12 @@ Author: Noah Misch permissions on the public schema has not been changed. Databases restored from previous Postgres releases will be restored with their current permissions. Users wishing - to have the former permissions will need to grant + to have the former more-open permissions will need to grant CREATE permission for PUBLIC on the public schema; this change can be made on template1 to cause all new databases - to have these permissions. + to have these permissions. This change was made to increase + security; see .
Re: First draft of the PG 15 release notes
On Sat, Jul 2, 2022 at 08:13:41PM -0400, Bruce Momjian wrote: > On Fri, Jul 1, 2022 at 09:56:17AM -0700, Peter Geoghegan wrote: > > On Fri, Jul 1, 2022 at 9:41 AM Bruce Momjian wrote: > > > > It might be worth explaining the shift directly in the release notes. > > > > The new approach is simpler and makes a lot more sense -- why should > > > > the relfrozenxid be closely tied to freezing? We don't necessarily > > > > > > I don't think this is an appropriate detail for the release notes. > > > > Okay. What about saying something about relminmxid advancement where > > the database consumes lots of multixacts? > > No. same issue. Actually, I was wrong. I thought that we only mentioned that we computed a more agressive xid, but now see I was mentioning the _frozen_ xid. Reading the commit, we do compute the multi-xid and store that too so I have updated the PG 15 release notes with the attached patch. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index 47ac329e79..179ad37d9d 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -994,7 +994,8 @@ Author: Peter Geoghegan Allow vacuum to be more -aggressive in setting the oldest frozenxid (Peter Geoghegan) +aggressive in setting the oldest frozen and multi transaction id +(Peter Geoghegan)
Re: First draft of the PG 15 release notes
> > Okay, thanks Bruce. -- Peter Geoghegan
Re: First draft of the PG 15 release notes
On Fri, Jul 1, 2022 at 09:56:17AM -0700, Peter Geoghegan wrote: > On Fri, Jul 1, 2022 at 9:41 AM Bruce Momjian wrote: > > > It might be worth explaining the shift directly in the release notes. > > > The new approach is simpler and makes a lot more sense -- why should > > > the relfrozenxid be closely tied to freezing? We don't necessarily > > > > I don't think this is an appropriate detail for the release notes. > > Okay. What about saying something about relminmxid advancement where > the database consumes lots of multixacts? No. same issue. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Fri, Jul 01, 2022 at 02:08:00PM -0400, Bruce Momjian wrote: > On Wed, Jun 29, 2022 at 10:08:08PM -0700, Noah Misch wrote: > > On Tue, Jun 28, 2022 at 04:35:45PM -0400, Bruce Momjian wrote: > > > > > permissions on the public schema has not > > > > > been changed. Databases restored from previous Postgres > > > > > releases > > > > > will be restored with their current permissions. Users wishing > > > > > to have the old permissions on new objects will need to grant > > > > > > > > The phrase "old permissions on new objects" doesn't sound right to me, > > > > but I'm > > > > not sure why. I think you're aiming for the fact that this is just a > > > > default; > > > > one can still change the ACL to anything, including to the old default. > > > > If > > > > these notes are going to mention the old default like they do so far, I > > > > think > > > > they should also urge readers to understand > > > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > > > before returning to the old default. What do you think? > > > > > > Agreed, the new text is: > > > > > > Users wishing to have the former permissions will need to grant > > > CREATE permission for PUBLIC on > > > the public schema; this change can be made on > > > template1 to cause all new databases to have these > > > permissions. > > > > What do you think about the "should also urge readers ..." part of my > > message? > > I see your point, that there is no indication of why you might not want > to restore the old permissions. I created the attached patch which > makes two additions to clarify this. > --- a/doc/src/sgml/release-15.sgml > +++ b/doc/src/sgml/release-15.sgml > @@ -63,12 +63,11 @@ Author: Noah Misch >permissions on the public schema has not >been changed. Databases restored from previous Postgres releases >will be restored with their current permissions. Users wishing > - to have the former more-open permissions will need to grant > + to have the former permissions will need to grant >CREATE permission for PUBLIC >on the public schema; this change can be made >on template1 to cause all new databases > - to have these permissions. This change was made to increase > - security. > + to have these permissions. > > Here's what I've been trying to ask: what do you think of linking to https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS here? The release note text is still vague, and the docs have extensive coverage of the topic. The notes can just link to that extensive coverage.
Re: First draft of the PG 15 release notes
On Wed, Jun 29, 2022 at 10:08:08PM -0700, Noah Misch wrote: > On Tue, Jun 28, 2022 at 04:35:45PM -0400, Bruce Momjian wrote: > > > If you dump/reload an unmodified v14 template1 (as pg_dumpall and > > > pg_upgrade > > > do), your v15 template1 will have a v14 ACL on its public schema. At that > > > point, the fate of "newly-created databases in existing clusters" depends > > > on > > > whether you clone template1 or template0. Does any of that detail belong > > > here, or does the existing text suffice? > > > > I think it is very confusing to have template0 have one value and > > template1 have a different one, but as I understand it template0 will > > only be used for pg_dump comparison, and that will keep template1 with > > the same permissions, so I guess it is okay. > > It's an emergent property of two decisions. In the interest of backward > compatibility, I decided to have v15 pg_dump emit GRANT for the public schema > even when the source is an unmodified v14- database. When that combines with > the ancient decision that a pg_dumpall or pg_upgrade covers template1 but not > template0, one gets the above consequences. I don't see a way to improve on > this outcome. Thanks for the summary. > > > > permissions on the public schema has not > > > > been changed. Databases restored from previous Postgres releases > > > > will be restored with their current permissions. Users wishing > > > > to have the old permissions on new objects will need to grant > > > > > > The phrase "old permissions on new objects" doesn't sound right to me, > > > but I'm > > > not sure why. I think you're aiming for the fact that this is just a > > > default; > > > one can still change the ACL to anything, including to the old default. > > > If > > > these notes are going to mention the old default like they do so far, I > > > think > > > they should also urge readers to understand > > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > > before returning to the old default. What do you think? > > > > Agreed, the new text is: > > > > Users wishing to have the former permissions will need to grant > > CREATE permission for PUBLIC on > > the public schema; this change can be made on > > template1 to cause all new databases to have these > > permissions. > > What do you think about the "should also urge readers ..." part of my message? I see your point, that there is no indication of why you might not want to restore the old permissions. I created the attached patch which makes two additions to clarify this. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index ed7cb89b03..47ac329e79 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -63,12 +63,11 @@ Author: Noah Misch permissions on the public schema has not been changed. Databases restored from previous Postgres releases will be restored with their current permissions. Users wishing - to have the former more-open permissions will need to grant + to have the former permissions will need to grant CREATE permission for PUBLIC on the public schema; this change can be made on template1 to cause all new databases - to have these permissions. This change was made to increase - security. + to have these permissions.
Re: First draft of the PG 15 release notes
On Fri, Jul 1, 2022 at 9:41 AM Bruce Momjian wrote: > > It might be worth explaining the shift directly in the release notes. > > The new approach is simpler and makes a lot more sense -- why should > > the relfrozenxid be closely tied to freezing? We don't necessarily > > I don't think this is an appropriate detail for the release notes. Okay. What about saying something about relminmxid advancement where the database consumes lots of multixacts? -- Peter Geoghegan
Re: First draft of the PG 15 release notes
On Tue, Jun 28, 2022 at 05:32:26PM -0700, Peter Geoghegan wrote: > The main enhancement to VACUUM for Postgres 15 was item 1, which > taught VACUUM to dynamically track the oldest remaining XID (and the > oldest remaining MXID) that will remain in the table at the end of the > same VACUUM operation. These final/oldest XID/MXID values are what we > now use to set relfrozenxid and relminmxid in pg_class. Previously we > just set relfrozenxid/relminmxid to whatever XID/MXID value was used > to determine which XIDs/MXIDs needed to be frozen. These values often > indicated more about VACUUM implementation details (like the > vacuum_freeze_min_age GUc's value) than the actual true contents of the > table at the end of the most recent VACUUM. > > It might be worth explaining the shift directly in the release notes. > The new approach is simpler and makes a lot more sense -- why should > the relfrozenxid be closely tied to freezing? We don't necessarily I don't think this is an appropriate detail for the release notes. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Tue, Jun 28, 2022 at 04:35:45PM -0400, Bruce Momjian wrote: > On Mon, Jun 27, 2022 at 11:37:19PM -0700, Noah Misch wrote: > > On Tue, May 10, 2022 at 11:44:15AM -0400, Bruce Momjian wrote: > > > I have completed the first draft of the PG 15 release notes > > > > > > > > > > > > > > > > > Remove PUBLIC creation permission on the > > linkend="ddl-schemas-public">public schema > > > (Noah Misch) > > > > > > > > > > > > This is a change in the default for newly-created databases in > > > existing clusters and for new clusters; USAGE > > > > If you dump/reload an unmodified v14 template1 (as pg_dumpall and pg_upgrade > > do), your v15 template1 will have a v14 ACL on its public schema. At that > > point, the fate of "newly-created databases in existing clusters" depends on > > whether you clone template1 or template0. Does any of that detail belong > > here, or does the existing text suffice? > > I think it is very confusing to have template0 have one value and > template1 have a different one, but as I understand it template0 will > only be used for pg_dump comparison, and that will keep template1 with > the same permissions, so I guess it is okay. It's an emergent property of two decisions. In the interest of backward compatibility, I decided to have v15 pg_dump emit GRANT for the public schema even when the source is an unmodified v14- database. When that combines with the ancient decision that a pg_dumpall or pg_upgrade covers template1 but not template0, one gets the above consequences. I don't see a way to improve on this outcome. > > > permissions on the public schema has not > > > been changed. Databases restored from previous Postgres releases > > > will be restored with their current permissions. Users wishing > > > to have the old permissions on new objects will need to grant > > > > The phrase "old permissions on new objects" doesn't sound right to me, but > > I'm > > not sure why. I think you're aiming for the fact that this is just a > > default; > > one can still change the ACL to anything, including to the old default. If > > these notes are going to mention the old default like they do so far, I > > think > > they should also urge readers to understand > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > before returning to the old default. What do you think? > > Agreed, the new text is: > > Users wishing to have the former permissions will need to grant > CREATE permission for PUBLIC on > the public schema; this change can be made on > template1 to cause all new databases to have these > permissions. What do you think about the "should also urge readers ..." part of my message?
Re: First draft of the PG 15 release notes
On Tue, Jun 28, 2022 at 1:35 PM Bruce Momjian wrote: > Okay, text updated, thanks. Applied patch attached. I have some notes on these items: 1. "Allow vacuum to be more aggressive in setting the oldest frozenxid (Peter Geoghegan)" 2. "Add additional information to VACUUM VERBOSE and autovacuum logging messages (Peter Geoghegan)" The main enhancement to VACUUM for Postgres 15 was item 1, which taught VACUUM to dynamically track the oldest remaining XID (and the oldest remaining MXID) that will remain in the table at the end of the same VACUUM operation. These final/oldest XID/MXID values are what we now use to set relfrozenxid and relminmxid in pg_class. Previously we just set relfrozenxid/relminmxid to whatever XID/MXID value was used to determine which XIDs/MXIDs needed to be frozen. These values often indicated more about VACUUM implementation details (like the vacuum_freeze_min_age GUc's value) than the actual true contents of the table at the end of the most recent VACUUM. It might be worth explaining the shift directly in the release notes. The new approach is simpler and makes a lot more sense -- why should the relfrozenxid be closely tied to freezing? We don't necessarily have to freeze any tuple to advance relfrozenxid right up to the removable cutoff/OldestXmin used by VACUUM. For example, anti-wraparound VACUUMs that run against static tables now set relfrozenxid/relminmxid to VACUUM's removable cutoff/OldestXmin directly, without freezing anything (after the first time). Same with tables that happen to have every row deleted -- only the actual unfrozen XIDs/MXIDs left in the table matter, and if there happen to be none at all then we can use the same relfrozenxid as we would for a CREATE TABLE. All depends on what the workload allows. There will also be a real practical benefit for users that allocate a lot of MultiXactIds: We'll now have pg_class.relminmxid values that are much more reliable indicators of what is really going on in the table, MultiXactId-wise. I expect that this will make it much less likely that anti-wraparound VACUUMs will run needlessly against the largest tables, where there probably wasn't ever one single MultiXactId. In other words, the implementation will have more accurate information at the level of each table, and won't . I think that very uneven consumption of MultiXactIds at the table level is probably common in real databases. Plus VACUUM can usually remove a non-running MultiXact from a tuple's xmax, regardless of whether or not the mxid happens to be before the vacuum_multixact_freeze_min_age-based MXID cutoff -- VACUUM has always just set xmax to InvalidXid in passing when it's possible to do so easily. MultiXacts are inherently pretty short-lived information about row lockers at a point in time. We don't really need to keep them around for very long. We may now be able to truncate the two MultiXact related SLRUs much more frequently with some workloads. Finally, note that the new VACUUM VERBOSE output (which is now pretty much the same as the autovacuum log output) shows when and how relfrozenxid/relminmxid have advanced. This should make it relatively easy to observe these effects where they exist. Thanks -- Peter Geoghegan
Re: First draft of the PG 15 release notes
On Mon, Jun 27, 2022 at 11:37:19PM -0700, Noah Misch wrote: > On Tue, May 10, 2022 at 11:44:15AM -0400, Bruce Momjian wrote: > > I have completed the first draft of the PG 15 release notes > > > > > > > > > > > Remove PUBLIC creation permission on the > linkend="ddl-schemas-public">public schema > > (Noah Misch) > > > > > > > > This is a change in the default for newly-created databases in > > existing clusters and for new clusters; USAGE > > If you dump/reload an unmodified v14 template1 (as pg_dumpall and pg_upgrade > do), your v15 template1 will have a v14 ACL on its public schema. At that > point, the fate of "newly-created databases in existing clusters" depends on > whether you clone template1 or template0. Does any of that detail belong > here, or does the existing text suffice? I think it is very confusing to have template0 have one value and template1 have a different one, but as I understand it template0 will only be used for pg_dump comparison, and that will keep template1 with the same permissions, so I guess it is okay. > > permissions on the public schema has not > > been changed. Databases restored from previous Postgres releases > > will be restored with their current permissions. Users wishing > > to have the old permissions on new objects will need to grant > > The phrase "old permissions on new objects" doesn't sound right to me, but I'm > not sure why. I think you're aiming for the fact that this is just a default; > one can still change the ACL to anything, including to the old default. If > these notes are going to mention the old default like they do so far, I think > they should also urge readers to understand > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > before returning to the old default. What do you think? Agreed, the new text is: Users wishing to have the former permissions will need to grant CREATE permission for PUBLIC on the public schema; this change can be made on template1 to cause all new databases to have these permissions. > > > CREATE permission for PUBLIC > > on the public schema; this change can be made > > on template1 to cause all new databases > > to have these permissions. template1 > > permissions for pg_dumpall and > > pg_upgrade? > > pg_dumpall will change template1. I think pg_upgrade will too, and neither > program will change template0. Okay, I will remove that question mark sentence. > > > > > > > > > > > > > > > > Change the owner of the public schema to > > pg_database_owner (Noah Misch) > > > > > > > > Previously it was the literal user name of the database owner. > > It was the bootstrap superuser. Okay, text updated, thanks. Applied patch attached. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml index 6da3f89d08..47ac329e79 100644 --- a/doc/src/sgml/release-15.sgml +++ b/doc/src/sgml/release-15.sgml @@ -63,13 +63,11 @@ Author: Noah Misch permissions on the public schema has not been changed. Databases restored from previous Postgres releases will be restored with their current permissions. Users wishing - to have the old permissions on new objects will need to grant + to have the former permissions will need to grant CREATE permission for PUBLIC on the public schema; this change can be made on template1 to cause all new databases - to have these permissions. template1 - permissions for pg_dumpall and - pg_upgrade? + to have these permissions. @@ -85,7 +83,7 @@ Author: Noah Misch - Previously it was the literal user name of the database owner. + Previously it was the literal user name of the bootstrap superuser. Databases restored from previous Postgres releases will be restored with their current owner specification.
Re: First draft of the PG 15 release notes
On Tue, May 10, 2022 at 11:44:15AM -0400, Bruce Momjian wrote: > I have completed the first draft of the PG 15 release notes > > > > > Remove PUBLIC creation permission on thelinkend="ddl-schemas-public">public schema > (Noah Misch) > > > > This is a change in the default for newly-created databases in > existing clusters and for new clusters; USAGE If you dump/reload an unmodified v14 template1 (as pg_dumpall and pg_upgrade do), your v15 template1 will have a v14 ACL on its public schema. At that point, the fate of "newly-created databases in existing clusters" depends on whether you clone template1 or template0. Does any of that detail belong here, or does the existing text suffice? > permissions on the public schema has not > been changed. Databases restored from previous Postgres releases > will be restored with their current permissions. Users wishing > to have the old permissions on new objects will need to grant The phrase "old permissions on new objects" doesn't sound right to me, but I'm not sure why. I think you're aiming for the fact that this is just a default; one can still change the ACL to anything, including to the old default. If these notes are going to mention the old default like they do so far, I think they should also urge readers to understand https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS before returning to the old default. What do you think? > CREATE permission for PUBLIC > on the public schema; this change can be made > on template1 to cause all new databases > to have these permissions. template1 > permissions for pg_dumpall and > pg_upgrade? pg_dumpall will change template1. I think pg_upgrade will too, and neither program will change template0. > > > > > > > > Change the owner of the public schema to > pg_database_owner (Noah Misch) > > > > Previously it was the literal user name of the database owner. It was the bootstrap superuser. > Databases restored from previous Postgres releases will be restored > with their current owner specification. > >
Re: First draft of the PG 15 release notes
On Thu, May 26, 2022 at 11:17 AM Bruce Momjian wrote: > On Thu, May 26, 2022 at 10:31:14AM +0900, Amit Langote wrote: > > * I think it's better to s/...or a LIST partition/...or with a LIST > > partition > > > > * The capitalization of DEFAULT and LIST seems unnecessary. > > > > Updated the patch on those points. > > I went with this text: > > Previously, a partitioned table with a DEFAULT partition or a LIST > partition containing multiple values could not be used for ordered > partition scans. Now they can be used if these partitions are pruned. Good enough for me, thanks. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
Re: First draft of the PG 15 release notes
On Thu, May 26, 2022 at 10:31:14AM +0900, Amit Langote wrote: > * I think it's better to s/...or a LIST partition/...or with a LIST partition > > * The capitalization of DEFAULT and LIST seems unnecessary. > > Updated the patch on those points. I went with this text: Previously, a partitioned table with a DEFAULT partition or a LIST partition containing multiple values could not be used for ordered partition scans. Now they can be used if these partitions are pruned. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Wed, May 25, 2022 at 1:04 PM Amit Langote wrote: > On Wed, May 25, 2022 at 12:44 PM David Rowley wrote: > > On Wed, 25 May 2022 at 15:01, Amit Langote wrote: > > > +Previously, a partitioned table with DEFAULT partition or a LIST > > > partition containing multiple values could not be used for ordered > > > partition scans. Now it can be used at least in the cases where such > > > partitions are pruned. > > > > I think this one is an improvement. I'd drop "at least". > > Okay, I can agree that "at least" sounds a bit extraneous, so removed. * I think it's better to s/...or a LIST partition/...or with a LIST partition * The capitalization of DEFAULT and LIST seems unnecessary. Updated the patch on those points. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com reword-ordered-partition-scan-item_v5.diff Description: Binary data
Re: First draft of the PG 15 release notes
On Wed, May 25, 2022 at 12:44 PM David Rowley wrote: > On Wed, 25 May 2022 at 15:01, Amit Langote wrote: > > +Previously, a partitioned table with DEFAULT partition or a LIST > > partition containing multiple values could not be used for ordered > > partition scans. Now it can be used at least in the cases where such > > partitions are pruned. > > I think this one is an improvement. I'd drop "at least". Okay, I can agree that "at least" sounds a bit extraneous, so removed. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com reword-ordered-partition-scan-item_v4.diff Description: Binary data
Re: First draft of the PG 15 release notes
On Wed, 25 May 2022 at 15:01, Amit Langote wrote: > +Previously, a partitioned table with DEFAULT partition or a LIST > partition containing multiple values could not be used for ordered > partition scans. Now it can be used at least in the cases where such > partitions are pruned. I think this one is an improvement. I'd drop "at least". David
Re: First draft of the PG 15 release notes
On Wed, May 25, 2022 at 8:36 AM Bruce Momjian wrote: > On Thu, May 19, 2022 at 06:13:28PM +0900, Amit Langote wrote: > > Or maybe we could mention that but use a wording that doesn't make it > > sound like an implementation detail, like: > > > > +Previously, an ordered partition scan could not be used for a > > LIST-partitioned table with any partition containing multiple values, > > nor for partitioned tables with DEFAULT partition. Now it can be used > > in those cases at least for queries in which such partitions are > > pruned. > > Sorry, I just don't see this as an improvement because it starts with a > complex term "an ordered partition scan" rather than simply "a > partitioned table". The headline says "Allow ordered scans of partitions to avoid sorting in more cases", so I proposed starting the description too with "an ordered scan". Also, not sure about going with: "previously, could not be used for , but now it can be provided " instead of: "previously, could not be used for , but now it can be provided " as in my proposed wording, but maybe that's just me. Anyway, I still think it would be better to fix the description such that the cases in which ordered scans will continue to not be usable are clear. The existing text doesn't make clear, for example, that a DEFAULT partition if present must have been pruned for an ordered scan to be used. So I propose: +Previously, a partitioned table with DEFAULT partition or a LIST partition containing multiple values could not be used for ordered partition scans. Now it can be used at least in the cases where such partitions are pruned. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com reword-ordered-partition-scan-item_v3.diff Description: Binary data
Re: First draft of the PG 15 release notes
On Thu, May 19, 2022 at 06:13:28PM +0900, Amit Langote wrote: > Or maybe we could mention that but use a wording that doesn't make it > sound like an implementation detail, like: > > +Previously, an ordered partition scan could not be used for a > LIST-partitioned table with any partition containing multiple values, > nor for partitioned tables with DEFAULT partition. Now it can be used > in those cases at least for queries in which such partitions are > pruned. Sorry, I just don't see this as an improvement because it starts with a complex term "an ordered partition scan" rather than simply "a partitioned table". -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Tue, May 24, 2022 at 12:16:10PM +0700, John Naylor wrote: > On Mon, May 16, 2022 at 9:18 PM Bruce Momjian wrote: > > > > Newer wording: > > > > Improve validation of UTF-8 text (even if only ASCII) by processing > > 16 bytes at a time (John Naylor) > > Thanks! I also think Heikki should be mentioned as a coauthor here -- > the ASCII coding was his work in large part. Sure, done. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Mon, May 16, 2022 at 9:18 PM Bruce Momjian wrote: > > Newer wording: > > Improve validation of UTF-8 text (even if only ASCII) by processing > 16 bytes at a time (John Naylor) Thanks! I also think Heikki should be mentioned as a coauthor here -- the ASCII coding was his work in large part. -- John Naylor EDB: http://www.enterprisedb.com
Re: First draft of the PG 15 release notes
On Thu, May 19, 2022 at 2:56 PM David Rowley wrote: > On Thu, 19 May 2022 at 14:41, Amit Langote wrote: > > Though a bit late given beta is now wrapped, I have another partition > > item wording improvement suggestion: > > > > -Previously, a partitioned table with any LIST partition containing > > multiple values could not be used for ordered partition scans. Now > > only non-pruned LIST partitions are checked. This also helps with > > -partitioned tables with DEFAULT partitions. > > > > +Previously, an ordered partition scan would not be considered for a > > LIST-partitioned table with any partition containing multiple values, > > nor for partitioned tables with DEFAULT partition. > > I think your proposed wording does not really improve things. The > "Now only non-pruned LIST partitions are checked" is important and I > think Bruce did the right thing to mention that. Prior to this change, > ordered scans were not possible if there was a DEFAULT or if any LIST > partition allowed >1 value. Now, if the default partition is pruned > and there are no non-pruned partitions that allow Datum values that > are inter-mixed with ones from another non-pruned partition, then an > ordered scan can be performed. > > For example, non-pruned partition a allows IN(1,3), and non-pruned > partition b allows IN(2,4), we cannot do the ordered scan. With > IN(1,2), IN(3,4), we can. I think that's what I understood this change to be about. Before this change, partitions_are_ordered() only returned true if *all* partitions of a parent are known to be ordered, which they're not in the presence of the default partition and of a list partition containing out-of-order values. It didn't matter to partitions_are_ordered() that the caller might not care about those partitions being present in the PartitionDesc because of having been pruned by the query, but that information was not readily available . So, you added PartitionBoundInfo.interleaved_parts to record indexes of partitions containing out-of-order values and RelOptInfo.live_parts to record non-pruned partitions, which made it more feasible for partitions_are_ordered() to address those cases. I suppose you think it's better to be verbose by mentioning that partitions_are_ordered() now considers only non-pruned partitions which allows supporting more cases, but I see that as mentioning implementation details unnecessarily. Or maybe we could mention that but use a wording that doesn't make it sound like an implementation detail, like: +Previously, an ordered partition scan could not be used for a LIST-partitioned table with any partition containing multiple values, nor for partitioned tables with DEFAULT partition. Now it can be used in those cases at least for queries in which such partitions are pruned. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com reword-ordered-partition-scan-item_v2.diff Description: Binary data
Re: First draft of the PG 15 release notes
On Thu, 19 May 2022 at 14:41, Amit Langote wrote: > Though a bit late given beta is now wrapped, I have another partition > item wording improvement suggestion: > > -Previously, a partitioned table with any LIST partition containing > multiple values could not be used for ordered partition scans. Now > only non-pruned LIST partitions are checked. This also helps with > -partitioned tables with DEFAULT partitions. > > +Previously, an ordered partition scan would not be considered for a > LIST-partitioned table with any partition containing multiple values, > nor for partitioned tables with DEFAULT partition. I think your proposed wording does not really improve things. The "Now only non-pruned LIST partitions are checked" is important and I think Bruce did the right thing to mention that. Prior to this change, ordered scans were not possible if there was a DEFAULT or if any LIST partition allowed >1 value. Now, if the default partition is pruned and there are no non-pruned partitions that allow Datum values that are inter-mixed with ones from another non-pruned partition, then an ordered scan can be performed. For example, non-pruned partition a allows IN(1,3), and non-pruned partition b allows IN(2,4), we cannot do the ordered scan. With IN(1,2), IN(3,4), we can. David
Re: First draft of the PG 15 release notes
On Sat, May 14, 2022 at 12:42 AM Bruce Momjian wrote: > On Fri, May 13, 2022 at 10:48:41AM +0900, Amit Langote wrote: > > Btw, perhaps the following should be listed under E.1.3.2.1. Logical > > Replication, not E.1.3.1.1. Partitioning? > > > > > > > > > > > > Remove incorrect duplicate partitions in system view > > pg_publication_tables (Hou Zhijie) > > > > > > > > Attached a patch to do so. > > Agreed, done. Thank you. Though a bit late given beta is now wrapped, I have another partition item wording improvement suggestion: -Previously, a partitioned table with any LIST partition containing multiple values could not be used for ordered partition scans. Now only non-pruned LIST partitions are checked. This also helps with -partitioned tables with DEFAULT partitions. +Previously, an ordered partition scan would not be considered for a LIST-partitioned table with any partition containing multiple values, nor for partitioned tables with DEFAULT partition. I think the "Now only non-pruned LIST partitions are checked" bit in the original wording is really an implementation detail of the actual improvement that ordered partition scans are now possible in more cases -- it simply became easier for the code that implements this optimization to refer to non-pruned partitions, using a bitmapset rather than having to trawl through the whole array of partition rels, which is what I think the commit message of this item mentions. David can correct me if I got that wrong. Attached a patch. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com reword-ordered-partition-scan-item.diff Description: Binary data
Re: First draft of the PG 15 release notes
On Mon, May 16, 2022 at 11:02:39AM -0500, Justin Pryzby wrote: > On Mon, May 16, 2022 at 05:41:08PM +0200, Alvaro Herrera wrote: > > On 2022-May-10, Bruce Momjian wrote: > > > > > I have completed the first draft of the PG 15 release notes and you can > > > see the results here: > > > > > > https://momjian.us/pgsql_docs/release-15.html > > > > Just to be clear -- 15beta1 will be released with the "new features and > > enhancements" list as empty, is that right? > > I assume so. Last year, it was empty until *after* 14rc1. > > https://www.postgresql.org/message-id/flat/cah2-wzntqzn_jjsuzyzmpstgc0c98_mzxybu0txzhlvkiet...@mail.gmail.com Yeah, Jonathan does that, and it is pulled usually from the press release. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Mon, May 16, 2022 at 05:41:08PM +0200, Alvaro Herrera wrote: > On 2022-May-10, Bruce Momjian wrote: > > > I have completed the first draft of the PG 15 release notes and you can > > see the results here: > > > > https://momjian.us/pgsql_docs/release-15.html > > Just to be clear -- 15beta1 will be released with the "new features and > enhancements" list as empty, is that right? I assume so. Last year, it was empty until *after* 14rc1. https://www.postgresql.org/message-id/flat/cah2-wzntqzn_jjsuzyzmpstgc0c98_mzxybu0txzhlvkiet...@mail.gmail.com -- Justin
Re: First draft of the PG 15 release notes
On 2022-May-10, Bruce Momjian wrote: > I have completed the first draft of the PG 15 release notes and you can > see the results here: > > https://momjian.us/pgsql_docs/release-15.html Just to be clear -- 15beta1 will be released with the "new features and enhancements" list as empty, is that right? -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Re: First draft of the PG 15 release notes
On Mon, May 16, 2022 at 10:09:18AM -0400, Bruce Momjian wrote: > On Mon, May 16, 2022 at 01:21:22PM +0700, John Naylor wrote: > > Hi Bruce, > > > > "Improve validation of ASCII and UTF-8 text by processing 16 bytes at > > a time (John Naylor)" > > > > The reader might assume here that ASCII is optimized regardless of > > encoding, but it is only optimized in the context of UTF-8. So I would > > just mention UTF-8. > > I struggled with this item because it seemed to me that even if the > UTF-8 text was only ASCII, it would benefit, so I just rewrote it to: > > Improve validation of UTF-8 text (even ASCII-only) by processing 16 > bytes at a time (John Naylor) Newer wording: Improve validation of UTF-8 text (even if only ASCII) by processing 16 bytes at a time (John Naylor) -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Mon, May 16, 2022 at 01:21:22PM +0700, John Naylor wrote: > Hi Bruce, > > "Improve validation of ASCII and UTF-8 text by processing 16 bytes at > a time (John Naylor)" > > The reader might assume here that ASCII is optimized regardless of > encoding, but it is only optimized in the context of UTF-8. So I would > just mention UTF-8. I struggled with this item because it seemed to me that even if the UTF-8 text was only ASCII, it would benefit, so I just rewrote it to: Improve validation of UTF-8 text (even ASCII-only) by processing 16 bytes at a time (John Naylor) -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
Hi Bruce, "Improve validation of ASCII and UTF-8 text by processing 16 bytes at a time (John Naylor)" The reader might assume here that ASCII is optimized regardless of encoding, but it is only optimized in the context of UTF-8. So I would just mention UTF-8. Thanks! -- John Naylor EDB: http://www.enterprisedb.com
RE: First draft of the PG 15 release notes
On Saturday, May 14, 2022 12:07 AM 'Bruce Momjian' wrote: > On Fri, May 13, 2022 at 01:36:04AM +, osumi.takami...@fujitsu.com wrote: > > > > > > This is enabled with the subscriber option "disable_on_error" > > > and avoids possible infinite loops during stream application. > > > > > > > > Thank you ! > > > > In this last paragraph, how about replacing "infinite loops" > > with "infinite error loops" ? I think it makes the situation somewhat > > clear for readers. > > Agreed, done. Thanks ! Best Regards, Takamichi Osumi
Re: First draft of the PG 15 release notes
On Sat, May 14, 2022 at 10:22:10AM +0530, Amit Kapila wrote: > > I see the point now --- new item: > > > > > > > > > > > > Prevent logical replication of empty transactions (Ajin Cherian, > > Hou Zhijie, Euler Taveira) > > > > > > > > Previously, write transactions would send empty transactions to > > subscribers if subscribed tables were not modified. > > > > > > > > Thanks! I will admit I had a little trouble with the wording of this item. :-) -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Fri, May 13, 2022 at 9:18 PM Bruce Momjian wrote: > > On Fri, May 13, 2022 at 08:24:53AM +0530, Amit Kapila wrote: > > On Fri, May 13, 2022 at 6:02 AM Euler Taveira wrote: > > > > > > On Thu, May 12, 2022, at 11:22 AM, Bruce Momjian wrote: > > > > > > On Thu, May 12, 2022 at 11:12:54AM -0300, Euler Taveira wrote: > > > OB> On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote: > > > > > > > > I looked at that but thought that everyone would already assume we > > > > skipped replication of empty transactions, and I didn't see much > > > > impact > > > > for the user, so I didn't include it. > > > > > > > > It certainly has an impact on heavy workloads that replicate tables > > > > with few > > > > modifications. It receives a high traffic of 'begin' and 'commit' > > > > messages that > > > > the previous Postgres versions have to handle (discard). I would > > > > classify it as > > > > a performance improvement for logical replication. Don't have a strong > > > > opinion > > > > if it should be mentioned or not. > > > > > > Oh, so your point is that a transaction that only has SELECT would > > > previously send an empty transaction? I thought this was only for apps > > > that create literal empty transactions, which seem rare. > > > > > > No. It should be a write transaction. If you have a replication setup that > > > publish only table foo (that isn't modified often) and most of your > > > workload does not contain table foo, Postgres sends 'begin' and 'commit' > > > messages to subscriber even if there is no change to replicate. > > > > > > > It reduces network traffic and improves performance by 3-14% on simple > > tests [1] like the one shown by Euler. I see a value in adding this as > > for the workloads where it hits, it seems more than 99% of network > > traffic [2] is due to these empty messages. > > I see the point now --- new item: > > > > > > Prevent logical replication of empty transactions (Ajin Cherian, > Hou Zhijie, Euler Taveira) > > > > Previously, write transactions would send empty transactions to > subscribers if subscribed tables were not modified. > > > Thanks! -- With Regards, Amit Kapila.
Re: First draft of the PG 15 release notes
On Fri, May 13, 2022 at 08:24:53AM +0530, Amit Kapila wrote: > On Fri, May 13, 2022 at 6:02 AM Euler Taveira wrote: > > > > On Thu, May 12, 2022, at 11:22 AM, Bruce Momjian wrote: > > > > On Thu, May 12, 2022 at 11:12:54AM -0300, Euler Taveira wrote: > > OB> On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote: > > > > > > I looked at that but thought that everyone would already assume we > > > skipped replication of empty transactions, and I didn't see much > > > impact > > > for the user, so I didn't include it. > > > > > > It certainly has an impact on heavy workloads that replicate tables with > > > few > > > modifications. It receives a high traffic of 'begin' and 'commit' > > > messages that > > > the previous Postgres versions have to handle (discard). I would classify > > > it as > > > a performance improvement for logical replication. Don't have a strong > > > opinion > > > if it should be mentioned or not. > > > > Oh, so your point is that a transaction that only has SELECT would > > previously send an empty transaction? I thought this was only for apps > > that create literal empty transactions, which seem rare. > > > > No. It should be a write transaction. If you have a replication setup that > > publish only table foo (that isn't modified often) and most of your > > workload does not contain table foo, Postgres sends 'begin' and 'commit' > > messages to subscriber even if there is no change to replicate. > > > > It reduces network traffic and improves performance by 3-14% on simple > tests [1] like the one shown by Euler. I see a value in adding this as > for the workloads where it hits, it seems more than 99% of network > traffic [2] is due to these empty messages. I see the point now --- new item: Prevent logical replication of empty transactions (Ajin Cherian, Hou Zhijie, Euler Taveira) Previously, write transactions would send empty transactions to subscribers if subscribed tables were not modified. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Fri, May 13, 2022 at 10:48:41AM +0900, Amit Langote wrote: > On Thu, May 12, 2022 at 10:52 PM Bruce Momjian wrote: > > Okay, I went with: > > > > Previously, such updates ran delete actions on the source > > partition and insert actions on the target partition. PostgreSQL > > will > > now run update actions on the referenced partition root. > > WFM, thanks. > > Btw, perhaps the following should be listed under E.1.3.2.1. Logical > Replication, not E.1.3.1.1. Partitioning? > > > > > > Remove incorrect duplicate partitions in system view > pg_publication_tables (Hou Zhijie) > > > > Attached a patch to do so. Agreed, done. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Fri, May 13, 2022 at 01:36:04AM +, osumi.takami...@fujitsu.com wrote: > > > > This is enabled with the subscriber option "disable_on_error" > > and avoids possible infinite loops during stream application. > > > > > Thank you ! > > In this last paragraph, how about replacing "infinite loops" > with "infinite error loops" ? I think it makes the situation somewhat > clear for readers. Agreed, done. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Fri, May 13, 2022 at 6:02 AM Euler Taveira wrote: > > On Thu, May 12, 2022, at 11:22 AM, Bruce Momjian wrote: > > On Thu, May 12, 2022 at 11:12:54AM -0300, Euler Taveira wrote: > OB> On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote: > > > > I looked at that but thought that everyone would already assume we > > skipped replication of empty transactions, and I didn't see much impact > > for the user, so I didn't include it. > > > > It certainly has an impact on heavy workloads that replicate tables with few > > modifications. It receives a high traffic of 'begin' and 'commit' messages > > that > > the previous Postgres versions have to handle (discard). I would classify > > it as > > a performance improvement for logical replication. Don't have a strong > > opinion > > if it should be mentioned or not. > > Oh, so your point is that a transaction that only has SELECT would > previously send an empty transaction? I thought this was only for apps > that create literal empty transactions, which seem rare. > > No. It should be a write transaction. If you have a replication setup that > publish only table foo (that isn't modified often) and most of your > workload does not contain table foo, Postgres sends 'begin' and 'commit' > messages to subscriber even if there is no change to replicate. > It reduces network traffic and improves performance by 3-14% on simple tests [1] like the one shown by Euler. I see a value in adding this as for the workloads where it hits, it seems more than 99% of network traffic [2] is due to these empty messages. [1] - https://www.postgresql.org/message-id/OSZPR01MB63105A71CFAA46F5BD7C9D7CFD1E9%40OSZPR01MB6310.jpnprd01.prod.outlook.com [2] - https://www.postgresql.org/message-id/CAMkU=1yohp9-dv48flosprmqyeyys5zwkazgd41rjr10xin...@mail.gmail.com -- With Regards, Amit Kapila.
Re: First draft of the PG 15 release notes
On Fri, May 13, 2022 at 11:44 AM Amit Kapila wrote: > On Fri, May 13, 2022 at 7:19 AM Amit Langote wrote: > > > > On Thu, May 12, 2022 at 10:52 PM Bruce Momjian wrote: > > > Okay, I went with: > > > > > > Previously, such updates ran delete actions on the source > > > partition and insert actions on the target partition. PostgreSQL > > > will > > > now run update actions on the referenced partition root. > > > > WFM, thanks. > > > > Btw, perhaps the following should be listed under E.1.3.2.1. Logical > > Replication, not E.1.3.1.1. Partitioning? > > > > Right. > > > > > > > > > > > Remove incorrect duplicate partitions in system view > > pg_publication_tables (Hou Zhijie) > > > > > > > > Attached a patch to do so. > > > > I don't see any attachment. Oops, attached this time. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com move-logirep-item.diff Description: Binary data
Re: First draft of the PG 15 release notes
On Fri, May 13, 2022 at 7:19 AM Amit Langote wrote: > > On Thu, May 12, 2022 at 10:52 PM Bruce Momjian wrote: > > Okay, I went with: > > > > Previously, such updates ran delete actions on the source > > partition and insert actions on the target partition. PostgreSQL > > will > > now run update actions on the referenced partition root. > > WFM, thanks. > > Btw, perhaps the following should be listed under E.1.3.2.1. Logical > Replication, not E.1.3.1.1. Partitioning? > Right. > > > > > Remove incorrect duplicate partitions in system view > pg_publication_tables (Hou Zhijie) > > > > Attached a patch to do so. > I don't see any attachment. -- With Regards, Amit Kapila.
Re: First draft of the PG 15 release notes
On Thu, May 12, 2022 at 10:52 PM Bruce Momjian wrote: > Okay, I went with: > > Previously, such updates ran delete actions on the source > partition and insert actions on the target partition. PostgreSQL will > now run update actions on the referenced partition root. WFM, thanks. Btw, perhaps the following should be listed under E.1.3.2.1. Logical Replication, not E.1.3.1.1. Partitioning? Remove incorrect duplicate partitions in system view pg_publication_tables (Hou Zhijie) Attached a patch to do so. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
RE: First draft of the PG 15 release notes
On Thursday, May 12, 2022 11:10 PM 'Bruce Momjian' wrote: > On Thu, May 12, 2022 at 01:35:39PM +, osumi.takami...@fujitsu.com > wrote: > > I'd like to suggest that we mention a new option for subscription > 'disable_on_error'. > > > > > > > https://github.com/postgres/postgres/commit/705e20f8550c0e8e47c0b6b20 > b5f5ffd6ffd9e33 > > Yes, I missed that one, added: > > > > > > Allow subscribers to stop logical replication application on error > (Osumi Takamichi, Mark Dilger) > > > > This is enabled with the subscriber option "disable_on_error" > and avoids possible infinite loops during stream application. > > Thank you ! In this last paragraph, how about replacing "infinite loops" with "infinite error loops" ? I think it makes the situation somewhat clear for readers. Best Regards, Takamichi Osumi
Re: First draft of the PG 15 release notes
On Thu, May 12, 2022, at 11:22 AM, Bruce Momjian wrote:
> On Thu, May 12, 2022 at 11:12:54AM -0300, Euler Taveira wrote:
> OB> On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote:
> >
> > I looked at that but thought that everyone would already assume we
> > skipped replication of empty transactions, and I didn't see much impact
> > for the user, so I didn't include it.
> >
> > It certainly has an impact on heavy workloads that replicate tables with few
> > modifications. It receives a high traffic of 'begin' and 'commit' messages
> > that
> > the previous Postgres versions have to handle (discard). I would classify
> > it as
> > a performance improvement for logical replication. Don't have a strong
> > opinion
> > if it should be mentioned or not.
>
> Oh, so your point is that a transaction that only has SELECT would
> previously send an empty transaction? I thought this was only for apps
> that create literal empty transactions, which seem rare.
No. It should be a write transaction. If you have a replication setup that
publish only table foo (that isn't modified often) and most of your
workload does not contain table foo, Postgres sends 'begin' and 'commit'
messages to subscriber even if there is no change to replicate.
Let me show you an example:
postgres=# CREATE TABLE foo (a integer primary key, b text);
CREATE TABLE
postgres=# CREATE TABLE bar (c integer primary key, d text);
CREATE TABLE
postgres=# CREATE TABLE baz (e integer primary key, f text);
CREATE TABLE
postgres=# CREATE PUBLICATION pubfoo FOR TABLE foo;
CREATE PUBLICATION
postgres=# SELECT pg_create_logical_replication_slot('slotfoo', 'pgoutput');
pg_create_logical_replication_slot
(slotfoo,0/E709AC50)
(1 row)
Let's create a transaction without table foo:
postgres=# BEGIN;
BEGIN
postgres=*# INSERT INTO bar (c, d) VALUES(1, 'blah');
INSERT 0 1
postgres=*# INSERT INTO baz (e, f) VALUES(2, 'xpto');
INSERT 0 1
postgres=*# COMMIT;
COMMIT
As you can see, the replication slot contains messages for that transaction.
Although, table bar and baz are NOT published, the begin (B) and commit (C)
messages that refers to this transaction are sent to subscriber.
postgres=# SELECT chr(get_byte(data, 0)) FROM
pg_logical_slot_peek_binary_changes('slotfoo', NULL, NULL,
'proto_version', '1', 'publication_names', 'pubfoo');
chr
-
B
C
(2 rows)
If you execute another transaction without table foo, there will be another B/C
pair.
postgres=# DELETE FROM baz WHERE e = 2;
DELETE 1
postgres=# SELECT chr(get_byte(data, 0)) FROM
pg_logical_slot_peek_binary_changes('slotfoo', NULL, NULL,
'proto_version', '1', 'publication_names', 'pubfoo');
chr
-
B
C
B
C
(4 rows)
Let's create a transaction that uses table foo but also table bar:
postgres=# BEGIN;
BEGIN
postgres=*# INSERT INTO foo (a, b) VALUES(100, 'asdf');
INSERT 0 1
postgres=*# INSERT INTO bar (c, d) VALUES(200, 'qwert');
INSERT 0 1
postgres=*# COMMIT;
COMMIT
In this case, there will be other messages since the publication pubfoo
publishes table foo. ('I' means there is an INSERT for table foo).
postgres=# SELECT chr(get_byte(data, 0)), length(data) FROM
pg_logical_slot_peek_binary_changes('slotfoo', NULL, NULL,
'proto_version', '1', 'publication_names', 'pubfoo');
chr | length
-+
B | 21
C | 26
B | 21
C | 26
B | 21
R | 41
I | 25
C | 26
(8 rows)
In summary, a logical replication setup sends 47 bytes per skipped transaction.
v15 won't send the first 2 B/C pairs. Discussion started here [1].
[1]
https://postgr.es/m/CAMkU=1yohp9-dv48flosprmqyeyys5zwkazgd41rjr10xin...@mail.gmail.com
--
Euler Taveira
EDB https://www.enterprisedb.com/
Re: First draft of the PG 15 release notes
On Thu, May 12, 2022 at 08:05:40PM +0530, vignesh C wrote: > I wonder if this is worth mentioning: > > Raise a WARNING for missing publications. > commit 8f2e2bbf145384784bad07a96d461c6bbd91f597 > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=8f2e2bbf145384784bad07a96d461c6bbd91f597 Reading the commit message, it looked like only a warning was being added, which was more of a helpful change rather than something we need to mention. However, if this means you could can now create a subscription for a missing publication that you couldn't do before, it should be added --- I couldn't tell from the patch. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
I wonder if this is worth mentioning: Raise a WARNING for missing publications. commit 8f2e2bbf145384784bad07a96d461c6bbd91f597 https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=8f2e2bbf145384784bad07a96d461c6bbd91f597 Regards, Vignesh On Thu, May 12, 2022 at 7:52 PM Bruce Momjian wrote: > > On Thu, May 12, 2022 at 11:12:54AM -0300, Euler Taveira wrote: > OB> On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote: > > > > I looked at that but thought that everyone would already assume we > > skipped replication of empty transactions, and I didn't see much impact > > for the user, so I didn't include it. > > > > It certainly has an impact on heavy workloads that replicate tables with few > > modifications. It receives a high traffic of 'begin' and 'commit' messages > > that > > the previous Postgres versions have to handle (discard). I would classify > > it as > > a performance improvement for logical replication. Don't have a strong > > opinion > > if it should be mentioned or not. > > Oh, so your point is that a transaction that only has SELECT would > previously send an empty transaction? I thought this was only for apps > that create literal empty transactions, which seem rare. > > -- > Bruce Momjian https://momjian.us > EDB https://enterprisedb.com > > Indecision is a decision. Inaction is an action. Mark Batterson > > >
Re: First draft of the PG 15 release notes
On Thu, May 12, 2022 at 11:12:54AM -0300, Euler Taveira wrote: OB> On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote: > > I looked at that but thought that everyone would already assume we > skipped replication of empty transactions, and I didn't see much impact > for the user, so I didn't include it. > > It certainly has an impact on heavy workloads that replicate tables with few > modifications. It receives a high traffic of 'begin' and 'commit' messages > that > the previous Postgres versions have to handle (discard). I would classify it > as > a performance improvement for logical replication. Don't have a strong opinion > if it should be mentioned or not. Oh, so your point is that a transaction that only has SELECT would previously send an empty transaction? I thought this was only for apps that create literal empty transactions, which seem rare. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote: > I looked at that but thought that everyone would already assume we > skipped replication of empty transactions, and I didn't see much impact > for the user, so I didn't include it. It certainly has an impact on heavy workloads that replicate tables with few modifications. It receives a high traffic of 'begin' and 'commit' messages that the previous Postgres versions have to handle (discard). I would classify it as a performance improvement for logical replication. Don't have a strong opinion if it should be mentioned or not. -- Euler Taveira EDB https://www.enterprisedb.com/
Re: First draft of the PG 15 release notes
On Thu, May 12, 2022 at 01:35:39PM +, osumi.takami...@fujitsu.com wrote: > I'd like to suggest that we mention a new option for subscription > 'disable_on_error'. > > > https://github.com/postgres/postgres/commit/705e20f8550c0e8e47c0b6b20b5f5ffd6ffd9e33 Yes, I missed that one, added: Allow subscribers to stop logical replication application on error (Osumi Takamichi, Mark Dilger) This is enabled with the subscriber option "disable_on_error" and avoids possible infinite loops during stream application. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Thu, May 12, 2022 at 09:32:48PM +1000, Ajin Cherian wrote: > On Wed, May 11, 2022 at 1:44 AM Bruce Momjian wrote: > > > > I have completed the first draft of the PG 15 release notes and you can > > see the results here: > > > > https://momjian.us/pgsql_docs/release-15.html > > > > The feature count is similar to recent major releases: > > > > release-10 195 > > release-11 185 > > release-12 198 > > release-13 183 > > release-14 229 > > --> release-15 186 > > > > I assume there will be major adjustments in the next few weeks based on > > feedback. > > > > I wonder if this is worth mentioning: > > Skip empty transactions for logical replication. > commit d5a9d86d8ffcadc52ff3729cd00fbd83bc38643c > > https://github.com/postgres/postgres/commit/d5a9d86d8ffcadc52ff3729cd00fbd83bc38643c I looked at that but thought that everyone would already assume we skipped replication of empty transactions, and I didn't see much impact for the user, so I didn't include it. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Thu, May 12, 2022 at 04:40:49PM +0530, Amit Kapila wrote: > One more point related to logical replication features: > > > > Add SQL functions to monitor the directory contents of replication > slots (Bharath Rupireddy) > > Specifically, the functions are pg_ls_logicalsnapdir(), > pg_ls_logicalmapdir(), and pg_ls_replslotdir(). They can be run by > members of the predefined pg_monitor role. > > > > This feature is currently under the section "Streaming Replication and > Recovery". Shouldn't it be under "Logical Replication"? The function > names themselves seem to indicate that they are used for logical > replication contents. I think the replication slot-related function > would fall under both categories but overall it seems to belong to the > "Logical Replication" section. Oh, very good point! I missed that this is logical-slot-only monitoring, so I moved the item to logical replication and changed the description to: Add SQL functions to monitor the directory contents of logical replication slots (Bharath Rupireddy) -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Thu, May 12, 2022 at 02:25:36PM +0530, Amit Kapila wrote: > On Wed, May 11, 2022 at 2:02 AM Bruce Momjian wrote: > > Yes, sorry, I missed that. Oddly, the unlogged sequence patch was > > retained, even though there is no value for it on the primary. I > > removed the sentence that mentioned that benefit from the release notes > > since it doesn't apply to PG 15 anymore. > > > > + Create unlogged sequences and allow them to be skipped in logical > replication > > Is it right to say the second part of the sentence: "allow them to be > skipped in logical replication" when we are not replicating them in > the first place? Oops, yeah, that second part was reverted; new text: Allow the creation of unlogged sequences (Peter Eisentraut) -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
On Thu, May 12, 2022 at 02:27:26PM +0900, Amit Langote wrote: > > Yes, this is what I needed to know. The updated text is: > > > > > > > > > > > > Improve foreign key behavior of updates on partitioned tables > > that move rows between partitions (Amit Langote) > > > > > > > > Previously, such updates ran delete actions on the source partition > > and insert actions on the target partition. PostgreSQL will now > > run update actions on the partition root. > > > > > > Looks fine to me. Though I think maybe we should write the last > sentence as "PostgreSQL will now run update actions on the partition > root mentioned in the query" to be less ambiguous about which "root", > because it can also mean the actual root table in the partition tree. > A user may be updating only a particular subtree by mentioning that > subtree's root in the query, for example. Okay, I went with: Previously, such updates ran delete actions on the source partition and insert actions on the target partition. PostgreSQL will now run update actions on the referenced partition root. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
RE: First draft of the PG 15 release notes
On Wednesday, May 11, 2022 12:44 AM Bruce Momjian wrote: > I have completed the first draft of the PG 15 release notes and you can see > the > results here: > > https://momjian.us/pgsql_docs/release-15.html > > The feature count is similar to recent major releases: > > release-10 195 > release-11 185 > release-12 198 > release-13 183 > release-14 229 > --> release-15 186 > > I assume there will be major adjustments in the next few weeks based on > feedback. Hi, I'd like to suggest that we mention a new option for subscription 'disable_on_error'. https://github.com/postgres/postgres/commit/705e20f8550c0e8e47c0b6b20b5f5ffd6ffd9e33 Best Regards, Takamichi Osumi
Re: First draft of the PG 15 release notes
On 5/11/22 10:50 PM, Ian Lawrence Barwick wrote: 2022年5月12日(木) 11:46 Bruce Momjian : On Thu, May 12, 2022 at 11:40:17AM +0900, Ian Lawrence Barwick wrote: Looking at the release notes from the point of view of someone who has maybe not been following the long-running debate on removing exclusive backups: "Important-sounding backup thing is suddenly gone! What was that again? Hmm, can't find anything in the now-current Pg 15 docs [*], do I need to worry about this!?" [*] the backup section has removed all mention of the word "exclusive" https://www.postgresql.org/docs/devel/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP versus: "Long-deprecated thing is finally gone, ah OK whatever". I am thinking back here to a point in my working life where the release notes were reviewed (by a team including non-Pg specialists) for potential issues when considering a major upgrade - from experience the more clarity with this kind of change the better so as not to unnecessarily raise alarm bells. Ah, you are right. I thought I had "deprecated" in the text, but I now see I did not, and we do have cases where we mention the deprecated status in previous release notes, so the new text is: Remove long-deprecated exclusive backup mode (David Steele, Nathan Bossart) That works, thanks! A bit late to this conversation, but +1 from me. -- -David da...@pgmasters.net
Re: First draft of the PG 15 release notes
On Wed, May 11, 2022 at 1:44 AM Bruce Momjian wrote: > > I have completed the first draft of the PG 15 release notes and you can > see the results here: > > https://momjian.us/pgsql_docs/release-15.html > > The feature count is similar to recent major releases: > > release-10 195 > release-11 185 > release-12 198 > release-13 183 > release-14 229 > --> release-15 186 > > I assume there will be major adjustments in the next few weeks based on > feedback. > I wonder if this is worth mentioning: Skip empty transactions for logical replication. commit d5a9d86d8ffcadc52ff3729cd00fbd83bc38643c https://github.com/postgres/postgres/commit/d5a9d86d8ffcadc52ff3729cd00fbd83bc38643c regards, Ajin Cherian Fujitsu Australia
Re: First draft of the PG 15 release notes
On Thu, May 12, 2022 at 2:25 PM Amit Kapila wrote: > > On Wed, May 11, 2022 at 2:02 AM Bruce Momjian wrote: > > > > On Tue, May 10, 2022 at 04:17:59PM -0400, Jonathan Katz wrote: > > > On 5/10/22 11:44 AM, Bruce Momjian wrote: > > > > I have completed the first draft of the PG 15 release notes and you can > > > > see the results here: > > > > > > > > https://momjian.us/pgsql_docs/release-15.html > > > > > > Thanks for pulling this together. > > > > > > + Allow logical replication to transfer sequence changes > > > > > > I believe this was reverted in 2c7ea57e5, unless some other parts of this > > > work made it in. > > > > Yes, sorry, I missed that. Oddly, the unlogged sequence patch was > > retained, even though there is no value for it on the primary. I > > removed the sentence that mentioned that benefit from the release notes > > since it doesn't apply to PG 15 anymore. > > > > + Create unlogged sequences and allow them to be skipped in logical > replication > > Is it right to say the second part of the sentence: "allow them to be > skipped in logical replication" when we are not replicating them in > the first place? > One more point related to logical replication features: > Add SQL functions to monitor the directory contents of replication slots (Bharath Rupireddy) Specifically, the functions are pg_ls_logicalsnapdir(), pg_ls_logicalmapdir(), and pg_ls_replslotdir(). They can be run by members of the predefined pg_monitor role. > This feature is currently under the section "Streaming Replication and Recovery". Shouldn't it be under "Logical Replication"? The function names themselves seem to indicate that they are used for logical replication contents. I think the replication slot-related function would fall under both categories but overall it seems to belong to the "Logical Replication" section. -- With Regards, Amit Kapila.
Re: First draft of the PG 15 release notes
On Wed, May 11, 2022 at 2:02 AM Bruce Momjian wrote: > > On Tue, May 10, 2022 at 04:17:59PM -0400, Jonathan Katz wrote: > > On 5/10/22 11:44 AM, Bruce Momjian wrote: > > > I have completed the first draft of the PG 15 release notes and you can > > > see the results here: > > > > > > https://momjian.us/pgsql_docs/release-15.html > > > > Thanks for pulling this together. > > > > + Allow logical replication to transfer sequence changes > > > > I believe this was reverted in 2c7ea57e5, unless some other parts of this > > work made it in. > > Yes, sorry, I missed that. Oddly, the unlogged sequence patch was > retained, even though there is no value for it on the primary. I > removed the sentence that mentioned that benefit from the release notes > since it doesn't apply to PG 15 anymore. > + Create unlogged sequences and allow them to be skipped in logical replication Is it right to say the second part of the sentence: "allow them to be skipped in logical replication" when we are not replicating them in the first place? -- With Regards, Amit Kapila.
Re: First draft of the PG 15 release notes
On Wed, May 11, 2022 at 11:41 PM Bruce Momjian wrote: > On Wed, May 11, 2022 at 04:02:31PM +0900, Amit Langote wrote: > > On Wed, May 11, 2022 at 12:44 AM Bruce Momjian wrote: > > > I have completed the first draft of the PG 15 release notes > > The commit is intended to only change the behavior of RI triggers, > > while leaving user-defined triggers firing as before. I think it > > might be a good idea to be specific by wording this, maybe as follows? > > > > Improve the firing of foreign key triggers during cross-partition > > updates of partitioned tables (Amit Langote) > > > > Previously, such updates fired delete triggers on the source partition > > and insert triggers on the target partition, whereas PostgreSQL will > > now fire update triggers on the partitioned table mentioned in the > > query, which makes the behavior of foreign keys pointing into that > > table more consistent. Note that other user-defined triggers are > > fired as they were before. > > Yes, this is what I needed to know. The updated text is: > > > > > > Improve foreign key behavior of updates on partitioned tables > that move rows between partitions (Amit Langote) > > > > Previously, such updates ran delete actions on the source partition > and insert actions on the target partition. PostgreSQL will now > run update actions on the partition root. > > Looks fine to me. Though I think maybe we should write the last sentence as "PostgreSQL will now run update actions on the partition root mentioned in the query" to be less ambiguous about which "root", because it can also mean the actual root table in the partition tree. A user may be updating only a particular subtree by mentioning that subtree's root in the query, for example. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
Re: First draft of the PG 15 release notes
2022年5月12日(木) 11:46 Bruce Momjian : > > On Thu, May 12, 2022 at 11:40:17AM +0900, Ian Lawrence Barwick wrote: > > Looking at the release notes from the point of view of someone who has maybe > > not been following the long-running debate on removing exclusive backups: > > > > "Important-sounding backup thing is suddenly gone! What was that > > again? Hmm, can't > > find anything in the now-current Pg 15 docs [*], do I need to worry > > about this!?" > > > > [*] the backup section has removed all mention of the word "exclusive" > > https://www.postgresql.org/docs/devel/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP > > > > versus: > > > > "Long-deprecated thing is finally gone, ah OK whatever". > > > > I am thinking back here to a point in my working life where the > > release notes were reviewed > > (by a team including non-Pg specialists) for potential issues when > > considering a major > > upgrade - from experience the more clarity with this kind of change > > the better so > > as not to unnecessarily raise alarm bells. > > Ah, you are right. I thought I had "deprecated" in the text, but I now > see I did not, and we do have cases where we mention the deprecated > status in previous release notes, so the new text is: > > Remove long-deprecated exclusive backup mode (David Steele, Nathan > Bossart) That works, thanks! Regards Ian Barwick -- EnterpriseDB: https://www.enterprisedb.com
Re: First draft of the PG 15 release notes
On Thu, May 12, 2022 at 11:40:17AM +0900, Ian Lawrence Barwick wrote: > Looking at the release notes from the point of view of someone who has maybe > not been following the long-running debate on removing exclusive backups: > > "Important-sounding backup thing is suddenly gone! What was that > again? Hmm, can't > find anything in the now-current Pg 15 docs [*], do I need to worry > about this!?" > > [*] the backup section has removed all mention of the word "exclusive" > https://www.postgresql.org/docs/devel/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP > > versus: > > "Long-deprecated thing is finally gone, ah OK whatever". > > I am thinking back here to a point in my working life where the > release notes were reviewed > (by a team including non-Pg specialists) for potential issues when > considering a major > upgrade - from experience the more clarity with this kind of change > the better so > as not to unnecessarily raise alarm bells. Ah, you are right. I thought I had "deprecated" in the text, but I now see I did not, and we do have cases where we mention the deprecated status in previous release notes, so the new text is: Remove long-deprecated exclusive backup mode (David Steele, Nathan Bossart) -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Re: First draft of the PG 15 release notes
Bruce Momjian writes: > On Thu, May 12, 2022 at 10:37:56AM +0900, Ian Lawrence Barwick wrote: Remove exclusive backup mode (David Steele, Nathan Bossart) >> It'd be useful to mention exclusive backup mode has been deprecated since >> 9.6, >> lest the impression arise that an important-sounding feature has been torn >> out >> suddenly. > Well, the documentation was clear about it being deprecated, so I don't > see a need to mention it in the release notes. Yeah, but somebody reading these notes doesn't necessarily have that old documentation at hand. I think writing "Remove the long-deprecated exclusive backup mode" would do nicely to make this point without many extra words. regards, tom lane
