Re: PostgreSQL 16 release announcement draft

2023-08-25 Thread Erik Rijkers
Op 8/26/23 om 04:51 schreef Jonathan S. Katz: On 8/24/23 11:17 AM, Erik Rijkers wrote: Op 8/24/23 om 16:32 schreef Jonathan S. Katz: On 8/23/23 5:07 PM, David Rowley wrote: On Thu, 24 Aug 2023 at 05:55, Jonathan S. Katz wrote: Hi, When v15 docs have: "27.2.7. Cascading Replication The

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-25 Thread Hayato Kuroda (Fujitsu)
Dear Peter, Thank you for reviewing! The patch can be available in [1]. > Here are my review comments for patch v25-0003. > > == > src/bin/pg_upgrade/check.c > > 1. GENERAL > > +static void check_for_confirmed_flush_lsn(void); > +static void check_for_lost_slots(void); > > For more

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-25 Thread Hayato Kuroda (Fujitsu)
Dear Peter, Thank you for reviewing! PSA new version patch set. > == > Commit message. > > 1. > I felt this should mention the limitation that the slot upgrade > feature is only supported from PG17 slots upwards. Added. The same sentence as doc was used. > doc/src/sgml/ref/pgupgrade.sgml

Re: PSQL error: total cell count of XXX exceeded

2023-08-25 Thread David G. Johnston
On Friday, August 25, 2023, Hongxu Ma wrote: > > > When I tried to select a big amount of rows, psql complains a error "Cannot > add cell to table content: total cell count of 905032704 exceeded." > > We should use long for ncolumns and nrows and give a more obvious error > message here. > > Any

Re: PostgreSQL 16 release announcement draft

2023-08-25 Thread Jonathan S. Katz
On 8/24/23 11:19 AM, Alvaro Herrera wrote: On 2023-Aug-24, Jonathan S. Katz wrote: ### Performance Improvements PostgreSQL 16 improves the performance of existing PostgreSQL functionality through new query planner optimizations. In this latest release, the query planner can parallelize

PSQL error: total cell count of XXX exceeded

2023-08-25 Thread Hongxu Ma
Hi Hackers, When I tried to select a big amount of rows, psql complains a error "Cannot add cell to table content: total cell count of 905032704 exceeded." Here are the reproduce steps: ``` interma=# select version(); version

Re: PostgreSQL 16 release announcement draft

2023-08-25 Thread Jonathan S. Katz
On 8/24/23 11:17 AM, Erik Rijkers wrote: Op 8/24/23 om 16:32 schreef Jonathan S. Katz: On 8/23/23 5:07 PM, David Rowley wrote: On Thu, 24 Aug 2023 at 05:55, Jonathan S. Katz wrote: Hi, When v15 docs have: "27.2.7. Cascading Replication The cascading replication feature allows a standby

Re: PostgreSQL 16 release announcement draft

2023-08-25 Thread Jonathan S. Katz
On 8/24/23 12:54 PM, Dave Cramer wrote: > Postgres, PostgreSQL, and the Elephant Logo (Slonik) are all registered > trademarks of the [PostgreSQL Community Association of Canada](https://www.postgres.ca ). Isn't this just the "PostgreSQL

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-25 Thread Imseih (AWS), Sami
> Here is a new version of the patch that avoids changing the names of the > existing functions. I'm not thrilled about the name > (pgstat_fetch_stat_local_beentry_by_backend_id), so I am open to > suggestions. In any case, I'd like to rename all three of the> > pgstat_fetch_stat_* functions in

Re: [ psql - review request ] review request for \d+ tablename, \d+ indexname indenting

2023-08-25 Thread Cary Huang
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: not tested Spec compliant: not tested Documentation:not tested Hello Thank you for the patch and the effort to enhance \d+ 's output

Re: Make all Perl warnings fatal

2023-08-25 Thread Dagfinn Ilmari Mannsåker
Alvaro Herrera writes: > On 2023-Aug-10, Peter Eisentraut wrote: > >> I wanted to figure put if we can catch these more reliably, in the style of >> -Werror. AFAICT, there is no way to automatically turn all warnings into >> fatal errors. But there is a way to do it per script, by replacing >>

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-25 Thread Nathan Bossart
On Fri, Aug 25, 2023 at 08:32:51AM -0700, Nathan Bossart wrote: > On second thought, renaming these exported functions so close to release is > probably not a great idea. I should probably skip back-patching that one. > Or I could have the existing functions call the new ones in v16 for >

Support prepared statement invalidation when result types change

2023-08-25 Thread Jelte Fennema
The cached plan for a prepared statements can get invalidated when DDL changes the tables used in the query, or when search_path changes. When this happens the prepared statement can still be executed, but it will be replanned in the new context. This means that the prepared statement will do

Re: [PATCH] Add XMLText function (SQL/XML X038)

2023-08-25 Thread Vik Fearing
On 8/25/23 17:56, Chapman Flack wrote: [0] https://wiki.postgresql.org/wiki/PostgreSQL_vs_SQL/XML_Standards I was not aware of this page. What a wealth of information! -- Vik Fearing

Re: initdb caching during tests

2023-08-25 Thread Nathan Bossart
On Thu, Aug 24, 2023 at 03:10:00PM -0700, Andres Freund wrote: > Cool. Pushed that way. I just noticed the tests running about 30% faster on my machine due to this. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: [PATCH] Add XMLText function (SQL/XML X038)

2023-08-25 Thread Chapman Flack
On 2023-08-25 10:49, Vik Fearing wrote: I do not think this should be addressed in this patch because there are quite a lot of functions that need to handle this. Indeed, as described in [0], we still largely provide the SQL/XML:2003 notion of a single XML datatype, not the distinguishable

Re: [PATCH] Add XMLText function (SQL/XML X038)

2023-08-25 Thread Jim Jones
On 25.08.23 16:49, Vik Fearing wrote: I am talking specifically about this: @@ -505,6 +506,10 @@ xmlcomment(PG_FUNCTION_ARGS) appendStringInfoText(, arg); appendStringInfoString(, "-->"); + + + + PG_RETURN_XML_P(stringinfo_to_xmltype());  #else NO_XML_SUPPORT(); I have

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-25 Thread Nathan Bossart
On Fri, Aug 25, 2023 at 03:01:40PM +, Imseih (AWS), Sami wrote: > 1/ cast the return of bsearch. This was done previously and is the common > convention in the code. Will do. > 2/ This will probably be a good time to update the docs for > pg_stat_get_backend_subxact [1] > to call out that

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-25 Thread Nathan Bossart
On Fri, Aug 25, 2023 at 09:36:18AM +0900, Ian Lawrence Barwick wrote: > Thanks for looking at this. In summary we now have these functions: > > extern PgBackendStatus *pgstat_get_beentry_by_backend_id(BackendId > beid); > extern LocalPgBackendStatus >

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-25 Thread Imseih (AWS), Sami
I tested the patch and it does the correct thing. I have a few comments: 1/ cast the return of bsearch. This was done previously and is the common convention in the code. So + return bsearch(, localBackendStatusTable, localNumBackends, +

Re: [PATCH] Add XMLText function (SQL/XML X038)

2023-08-25 Thread Vik Fearing
On 8/25/23 14:42, Jim Jones wrote: Hi Vik Thanks for reviewing my patch! Thank you for writing it! On 25.08.23 12:05, Vik Fearing wrote: I am replying to this email, but my comments are based on the v2 patch. Thank you for working on this, and I think this is a valuable addition.

Re: initdb caching during tests

2023-08-25 Thread Andres Freund
Hi, On 2023-08-25 09:00:24 +0200, Daniel Gustafsson wrote: > > On 25 Aug 2023, at 07:50, Thomas Munro wrote: > > > > On Fri, Aug 25, 2023 at 10:10 AM Andres Freund wrote: > >> Let's see what the buildfarm says - it's not inconceivable that it'll show > >> some issues. > > > > Apparently

Re: Let's make PostgreSQL multi-threaded

2023-08-25 Thread Stephen Frost
Greetings, * David Geier (geidav...@gmail.com) wrote: > On 8/11/23 14:05, Merlin Moncure wrote: > > Hm, noted this upthread, but asking again, does this > > help/benefit interactions with the operating system make oom kill > > situations less likely?   These things are the bane of my existence,

Re: Format list of catalog files in makefile vertically

2023-08-25 Thread Andres Freund
Hi, On August 25, 2023 9:12:51 AM EDT, Peter Eisentraut wrote: >I propose to reformat the catalog lists in src/backend/catalog/Makefile to be >more vertical, one per line. This makes it easier to keep that list in sync >with src/include/catalog/meson.build, and visually compare both lists.

Format list of catalog files in makefile vertically

2023-08-25 Thread Peter Eisentraut
I propose to reformat the catalog lists in src/backend/catalog/Makefile to be more vertical, one per line. This makes it easier to keep that list in sync with src/include/catalog/meson.build, and visually compare both lists. Also, it's easier to read and edit in general. In passing, I'd

Unlogged relation copy is not fsync'd

2023-08-25 Thread Heikki Linnakangas
I noticed another missing fsync() with unlogged tables, similar to the one at [1]. RelationCopyStorage does this: /* * When we WAL-logged rel pages, we must nonetheless fsync them. The * reason is that since we're copying outside shared buffers, a CHECKPOINT

Re: [PATCH] Add XMLText function (SQL/XML X038)

2023-08-25 Thread Daniel Gustafsson
> On 25 Aug 2023, at 14:42, Jim Jones wrote: > Is there any tool like pgindent to take care of it automatically? No, pgindent doesn't address whitespace, only indentation of non-whitespace. -- Daniel Gustafsson

Re: [PATCH] Add XMLText function (SQL/XML X038)

2023-08-25 Thread Jim Jones
Hi Vik Thanks for reviewing my patch! On 25.08.23 12:05, Vik Fearing wrote: I am replying to this email, but my comments are based on the v2 patch. Thank you for working on this, and I think this is a valuable addition. However, I have two issues with it. 1) There seems to be several

Re: list of acknowledgments for PG16

2023-08-25 Thread Daniel Gustafsson
> On 25 Aug 2023, at 14:22, Magnus Hagander wrote: > On Tue, Aug 22, 2023 at 4:03 PM Joe Conway wrote: >> I'm sure we could figure out how to just release the updated docs, but >> with RC1 a week away, is it really worthwhile? > > We've also been pretty strict to say that we don't *want*

Re: list of acknowledgments for PG16

2023-08-25 Thread Magnus Hagander
On Tue, Aug 22, 2023 at 4:03 PM Joe Conway wrote: > > On 8/22/23 09:44, Tom Lane wrote: > > Alvaro Herrera writes: > >> On 2023-Aug-22, Peter Eisentraut wrote: > >>> The list of acknowledgments for the PG16 release notes has been committed. > >>> It should show up here sometime: > >>>

Re: persist logical slots to disk during shutdown checkpoint

2023-08-25 Thread vignesh C
On Sat, 19 Aug 2023 at 11:53, Amit Kapila wrote: > > It's entirely possible for a logical slot to have a confirmed_flush > LSN higher than the last value saved on disk while not being marked as > dirty. It's currently not a problem to lose that value during a clean > shutdown / restart cycle but

Re: Let's make PostgreSQL multi-threaded

2023-08-25 Thread David Geier
Hi, On 8/11/23 14:05, Merlin Moncure wrote: On Thu, Jul 27, 2023 at 8:28 AM David Geier wrote: Hi, On 6/7/23 23:37, Andres Freund wrote: > I think we're starting to hit quite a few limits related to the process model, > particularly on bigger machines. The overhead of

Re: cataloguing NOT NULL constraints

2023-08-25 Thread Alvaro Herrera
On 2023-Aug-25, Alvaro Herrera wrote: > I have now pushed this again. Hopefully it'll stick this time. Hmm, failed under the Czech locale[1]; apparently "inh_grandchld" sorts earlier than "inh_child1" there. I think I'll rename inh_grandchld to inh_child3 or something like that. [1]

Re: [PATCH] Add native windows on arm64 support

2023-08-25 Thread Daniel Gustafsson
> On 25 Aug 2023, at 11:40, Anthony Roberts wrote: > * A machine for you to directly access (via a VPN) That could be very helpful to provide to developers who are trying to debug very wicked bugs which requires access for analysis. Such things are on a case by case basis of course at your

Re: RFC: Logging plan of the running query

2023-08-25 Thread James Coleman
On Thu, Aug 17, 2023 at 10:02 AM torikoshia wrote: > > On 2023-06-16 01:34, James Coleman wrote: > > Attached is v28 > > which sets ProcessLogQueryPlanInterruptActive to false in errfinish > > when necessary. Once built with those two patches I'm simply running > > `make check`. > > With v28-0001

Re: cataloguing NOT NULL constraints

2023-08-25 Thread Alvaro Herrera
I have now pushed this again. Hopefully it'll stick this time. We may want to make some further tweaks to the behavior in some cases -- for example, don't disallow ALTER TABLE DROP NOT NULL when the constraint is both inherited and has a local definition; the other option is to mark the

Re: [PATCH] Add native windows on arm64 support

2023-08-25 Thread Anthony Roberts
Thanks for the link - that looks basically the same as what we do for CMake (option 2), so should be relatively easy. I will have a chat to relevant people about setting a machine up properly for it. Thanks, Anthony On 25/08/2023 11:17, Michael Paquier wrote: On Fri, Aug 25, 2023 at

Re: [PATCH] Add native windows on arm64 support

2023-08-25 Thread Anthony Roberts
Hi, Which of these are you looking for us to provide: * A machine for you to directly access (via a VPN) * A machine we just run a worker script on that automatically picks up the builds as required * Us to do downstream CI All are possible, but preferably not option 1, as it would mean

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-25 Thread Amit Kapila
On Fri, Aug 25, 2023 at 2:14 PM Peter Smith wrote: > > Here are my review comments for patch v25-0002. > > In general, I feel where possible the version checking is best done in > the "check.c" file (the filename is a hint). Most of the review > comments below are repeating this point. > > ==

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-25 Thread Peter Smith
Hi Kuroda-san, Here are my review comments for patch v25-0003. == src/bin/pg_upgrade/check.c 1. GENERAL +static void check_for_confirmed_flush_lsn(void); +static void check_for_lost_slots(void); For more clarity, I wonder if it is better to rename some functions:

Re: [PATCH] Add native windows on arm64 support

2023-08-25 Thread Michael Paquier
On Fri, Aug 25, 2023 at 10:40:39AM +0100, Anthony Roberts wrote: > Which of these are you looking for us to provide: > > * A machine for you to directly access (via a VPN) > * A machine we just run a worker script on that automatically picks up the > builds as required > * Us to do downstream CI

Re: subscription/015_stream sometimes breaks

2023-08-25 Thread Amit Kapila
On Thu, Aug 24, 2023 at 3:48 PM Amit Kapila wrote: > > On Thu, Aug 24, 2023 at 1:20 PM Alvaro Herrera > wrote: > > > > On 2023-Aug-24, Amit Kapila wrote: > > > > > On Wed, Aug 23, 2023 at 1:31 PM Alvaro Herrera > > > wrote: > > > > > > Hmm, I think if worker->in_use is false, we shouldn't

Re: [PATCH] Add XMLText function (SQL/XML X038)

2023-08-25 Thread Vik Fearing
On 3/25/23 12:49, Jim Jones wrote: Hi, This small patch proposes the implementation of the standard SQL/XML function XMLText (X038). It basically converts a text parameter into an xml text node. It uses the libxml2 function xmlEncodeSpecialChars[1] to escape possible predefined entities.

Re: logical_replication_mode

2023-08-25 Thread Amit Kapila
On Fri, Aug 25, 2023 at 12:38 PM Peter Eisentraut wrote: > > On 25.08.23 08:52, Zhijie Hou (Fujitsu) wrote: > > On Friday, August 25, 2023 12:28 PM Amit Kapila > > wrote: > >> > >> On Thu, Aug 24, 2023 at 12:45 PM Peter Eisentraut > >> wrote: > >>> > >>> I suggest we rename this setting to

Re: Testing autovacuum wraparound (including failsafe)

2023-08-25 Thread Daniel Gustafsson
> On 22 Aug 2023, at 07:49, Michael Paquier wrote: > > On Wed, Jul 12, 2023 at 01:47:51PM +0200, Daniel Gustafsson wrote: >> +# bump the query timeout to avoid false negatives on slow test syetems. >> +$ENV{PG_TEST_TIMEOUT_DEFAULT} = 600; >> Does this actually work? Utils.pm read the

Re: Support run-time partition pruning for hash join

2023-08-25 Thread Richard Guo
On Fri, Aug 25, 2023 at 11:03 AM David Rowley wrote: > On Thu, 24 Aug 2023 at 21:27, Richard Guo wrote: > > I think we need to solve this problem first before we can > > make this new partition pruning mechanism some useful in practice, but > > how? Some thoughts currently in my mind include >

Re: Synchronizing slots from primary to standby

2023-08-25 Thread Alvaro Herrera
Wait a minute ... >From bac0fbef8b203c530e5117b0b7cfee13cfab78b9 Mon Sep 17 00:00:00 2001 From: Bharath Rupireddy Date: Sat, 22 Jul 2023 10:17:48 + Subject: [PATCH v13 1/2] Allow logical walsenders to wait for physical standbys @@ -2498,6 +2500,13 @@ ReorderBufferProcessTXN(ReorderBuffer

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-25 Thread Peter Smith
Here are my review comments for patch v25-0002. In general, I feel where possible the version checking is best done in the "check.c" file (the filename is a hint). Most of the review comments below are repeating this point. == Commit message. 1. I felt this should mention the limitation

Re: [PATCH] Add XMLText function (SQL/XML X038)

2023-08-25 Thread Jim Jones
On 25.03.23 13:25, I wrote: I just realized that I forgot to add a few examples to my last message :D postgres=# SELECT xmltext('foo ´/[({bar?})]\`');   xmltext  foo ´/[({bar?})]\` (1 row) postgres=# SELECT xmltext('foo & ');     xmltext ---  

Re: broken master regress tests

2023-08-25 Thread Pavel Stehule
pá 25. 8. 2023 v 8:22 odesílatel Matthias van de Meent < boekewurm+postg...@gmail.com> napsal: > > > On Fri, 25 Aug 2023, 05:54 Pavel Stehule, wrote: > >> Hi >> >> today build is broken on my Fedora 39 >> >> Regards >> >> Pavel >> >> make[2]: Opouští se adresář >>

Re: logical_replication_mode

2023-08-25 Thread Peter Eisentraut
On 25.08.23 08:52, Zhijie Hou (Fujitsu) wrote: On Friday, August 25, 2023 12:28 PM Amit Kapila wrote: On Thu, Aug 24, 2023 at 12:45 PM Peter Eisentraut wrote: I suggest we rename this setting to something starting with debug_. Right now, the name looks much too tempting for users to fiddle

Re: initdb caching during tests

2023-08-25 Thread Daniel Gustafsson
> On 25 Aug 2023, at 07:50, Thomas Munro wrote: > > On Fri, Aug 25, 2023 at 10:10 AM Andres Freund wrote: >> Let's see what the buildfarm says - it's not inconceivable that it'll show >> some issues. > > Apparently Solaris doesn't like "cp -a", per animal "margay". I think > "cp -RPp" should

RE: logical_replication_mode

2023-08-25 Thread Zhijie Hou (Fujitsu)
On Friday, August 25, 2023 12:28 PM Amit Kapila wrote: > > On Thu, Aug 24, 2023 at 12:45 PM Peter Eisentraut > wrote: > > > > I suggest we rename this setting to something starting with debug_. > > Right now, the name looks much too tempting for users to fiddle with. > > I think this is similar

Re: broken master regress tests

2023-08-25 Thread Matthias van de Meent
On Fri, 25 Aug 2023, 05:54 Pavel Stehule, wrote: > Hi > > today build is broken on my Fedora 39 > > Regards > > Pavel > > make[2]: Opouští se adresář > „/home/pavel/src/postgresql.master/src/bin/initdb“ > make -C pg_amcheck check > make[2]: Vstupuje se do adresáře >

Re: broken master regress tests

2023-08-25 Thread Pavel Stehule
pá 25. 8. 2023 v 8:10 odesílatel Pavel Stehule napsal: > Hi > > pá 25. 8. 2023 v 7:38 odesílatel Thomas Munro > napsal: > >> On Fri, Aug 25, 2023 at 3:54 PM Pavel Stehule >> wrote: >> > today build is broken on my Fedora 39 >> >> Has commit 252dcb32 somehow upset some kind of bleeding edge

Re: broken master regress tests

2023-08-25 Thread Pavel Stehule
Hi pá 25. 8. 2023 v 7:38 odesílatel Thomas Munro napsal: > On Fri, Aug 25, 2023 at 3:54 PM Pavel Stehule > wrote: > > today build is broken on my Fedora 39 > > Has commit 252dcb32 somehow upset some kind of bleeding edge btrfs > filesystem? That's a wild guess and I can't really imagine how