Buildfarm failures on urocryon

2023-08-31 Thread vignesh C
Hi, Recently urocryon has been failing with the following errors at [1]: checking for icu-uc icu-i18n... no configure: error: ICU library not found If you have ICU already installed, see config.log for details on the failure. It is possible the compiler isn't looking in the proper directory. Use

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-31 Thread Dilip Kumar
On Fri, Sep 1, 2023 at 9:47 AM Dilip Kumar wrote: > > On Thu, Aug 31, 2023 at 7:56 PM Hayato Kuroda (Fujitsu) > wrote: > Some more comments on 0002 1. + conn = connectToServer(_cluster, "template1"); + + prep_status("Checking for logical replication slots"); + + res = executeQueryOrDie(conn,

Re: persist logical slots to disk during shutdown checkpoint

2023-08-31 Thread vignesh C
On Fri, 1 Sept 2023 at 10:06, Amit Kapila wrote: > > On Thu, Aug 31, 2023 at 6:12 PM Ashutosh Bapat > wrote: > > > > On Thu, Aug 31, 2023 at 2:52 PM Amit Kapila wrote: > > > > > > All but one. Normally, the idea of marking dirty is to indicate that > > > we will actually write/flush the

Re: pg_upgrade fails with in-place tablespace[

2023-08-31 Thread Michael Paquier
On Sat, Aug 19, 2023 at 08:11:28PM +0800, Rui Zhao wrote: > Please refer to the TAP test I have included for a better understanding > of my suggestion. Sure, but it seems to me that my original question is not really answered: what's your use case for being able to support in-place tablespaces

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-31 Thread Hayato Kuroda (Fujitsu)
Dear Amit, Thank you for giving suggestions! I think your fixes are good. New patch set can be available in [1]. > Apart from this, I have addressed some of the comments raised by you > for the 0003 patch. Please find the diff patch attached. I think we > should combine 0002 and 0003 patches.

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-31 Thread Hayato Kuroda (Fujitsu)
Dear Peter, Thanks for giving comments! PSA new version. I replied only comment 8 because others were replied by Amit. > .../t/003_logical_replication_slots.pl > > 8. Consider adding one more test > > Maybe there should also be some "live check" test performed (e.g. > using --check, and a

Re: persist logical slots to disk during shutdown checkpoint

2023-08-31 Thread Amit Kapila
On Thu, Aug 31, 2023 at 6:12 PM Ashutosh Bapat wrote: > > On Thu, Aug 31, 2023 at 2:52 PM Amit Kapila wrote: > > > > All but one. Normally, the idea of marking dirty is to indicate that > > we will actually write/flush the contents at a later point (except > > when required for correctness) as

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-31 Thread Dilip Kumar
On Thu, Aug 31, 2023 at 7:56 PM Hayato Kuroda (Fujitsu) wrote: > Thanks for giving comments! Thanks > > Some comments in 0002 > > > > 1. > > + res = executeQueryOrDie(conn, "SELECT slot_name " > > + "FROM pg_catalog.pg_replication_slots " > > + "WHERE slot_type = 'logical' AND " > > +

Re: SQL:2011 application time

2023-08-31 Thread Corey Huinker
> > The PERIOD patch is not finished and includes some deliberately-failing > tests. I did make some progress here finishing ALTER TABLE ADD PERIOD. > If it's ok with you, I need PERIODs for System Versioning, and planned on developing a highly similar version, albeit closer to the standard. It

Re: Extract numeric filed in JSONB more effectively

2023-08-31 Thread Andy Fan
> An incompatible issue at error message level is found during test: > create table jb(a jsonb); > insert into jb select '{"a": "a"}'::jsonb; > select (a->'a')::int4 from jb; > > master: ERROR: cannot cast jsonb string to type *integer* > patch: ERROR: cannot cast jsonb string to type

Re: Should we use MemSet or {0} for struct initialization?

2023-08-31 Thread Richard Guo
On Thu, Aug 31, 2023 at 7:07 PM John Naylor wrote: > > On Thu, Aug 31, 2023 at 5:34 PM Richard Guo > wrote: > > > > > > While working on a bug in expandRecordVariable() I noticed that in the > > > switch statement for case RTE_SUBQUERY we initialize struct ParseState > > > with {0} while for

Re: should frontend tools use syncfs() ?

2023-08-31 Thread Nathan Bossart
On Fri, Sep 01, 2023 at 10:40:12AM +0900, Michael Paquier wrote: > That should be OK this way. The extra running time is not really > visible, right? AFAICT it is negligible. Presumably it could take a little longer if there is a lot to sync on the file system, but I don't know if that's worth

Re: More new SQL/JSON item methods

2023-08-31 Thread Chapman Flack
On 2023-08-31 20:50, Vik Fearing wrote: — An SQL/JSON item is defined recursively as any of the following: ... • An SQL/JSON array, defined as an ordered list of zero or more SQL/JSON items, called the SQL/JSON elements of the SQL/JSON array. ... — An SQL/JSON sequence is an

Re: should frontend tools use syncfs() ?

2023-08-31 Thread Michael Paquier
On Thu, Aug 31, 2023 at 08:48:58AM -0700, Nathan Bossart wrote: > On Thu, Aug 31, 2023 at 02:30:33PM +0900, Michael Paquier wrote: > > - Should we have some regression tests? We should only need one test > > in one of the binaries to be able to stress the new code paths of > > file_utils.c with

Re: Eliminate redundant tuple visibility check in vacuum

2023-08-31 Thread Peter Geoghegan
On Thu, Aug 31, 2023 at 3:35 PM Melanie Plageman wrote: > Any inserting transaction which aborts after heap_page_prune()'s > visibility check will now be of no concern to lazy_scan_prune(). Since > we don't do the visibility check again, we won't find the tuple > HEAPTUPLE_DEAD and thus won't

Re: Show inline comments from pg_hba lines in the pg_hba_file_rules view

2023-08-31 Thread Michael Paquier
On Fri, Sep 01, 2023 at 12:01:37AM +0200, Jim Jones wrote: > Often we make changes in the pg_hba.conf and leave a #comment there, just in > case we forget why the change was done. To avoid having to open the > configuration file every time just to check the comments, it would be quite > nice to

Re: sandboxing untrusted code

2023-08-31 Thread Jeff Davis
On Thu, 2023-08-31 at 11:25 -0400, Robert Haas wrote: > As a refresher, the scenario I'm talking about is any one in which > one > user, who I'll call Bob, does something that results in executing > code > provided by another user, who I'll call Alice. The most obvious way > that this can happen

Re: More new SQL/JSON item methods

2023-08-31 Thread Vik Fearing
On 8/30/23 19:20, Chapman Flack wrote: On 2023-08-30 12:28, Alvaro Herrera wrote:     b) Otherwise, the result of JAE is the SQL/JSON sequence V_1,    ..., V_n. This has my Spidey sense tingling, as it seems very parallel to SQL/XML where the result of XMLQUERY is to have type

Re: trying again to get incremental backup

2023-08-31 Thread David Steele
Hi Robert, On 8/30/23 10:49, Robert Haas wrote: In the limited time that I've had to work on this project lately, I've been trying to come up with a test case for this feature -- and since I've gotten completely stuck, I thought it might be time to post and see if anyone else has a better idea.

Re: Eliminate redundant tuple visibility check in vacuum

2023-08-31 Thread Melanie Plageman
On Thu, Aug 31, 2023 at 5:39 AM David Geier wrote: > Regarding the 2nd patch (disclaimer: I'm not too familiar with that area > of the code): I don't completely understand why the retry loop is not > needed anymore and how you now detect/handle the possible race > condition? It can still happen

Re: Eliminate redundant tuple visibility check in vacuum

2023-08-31 Thread Melanie Plageman
On Thu, Aug 31, 2023 at 2:03 PM Robert Haas wrote: > > I have a few suggestions: > > - Rather than removing the rather large comment block at the top of > lazy_scan_prune() I'd like to see it rewritten to explain how we now > deal with the problem. I'd suggest leaving the first paragraph ("Prior

Show inline comments from pg_hba lines in the pg_hba_file_rules view

2023-08-31 Thread Jim Jones
Hi, Often we make changes in the pg_hba.conf and leave a #comment there, just in case we forget why the change was done. To avoid having to open the configuration file every time just to check the comments, it would be quite nice to have the option to read these comments in the

Re: Reducing connection overhead in pg_upgrade compat check phase

2023-08-31 Thread Daniel Gustafsson
> On 12 Jul 2023, at 01:36, Nathan Bossart wrote: > > On Wed, Jul 12, 2023 at 12:43:14AM +0200, Daniel Gustafsson wrote: >> I did have coffee before now, but only found time to actually address this >> now >> so here is a v7 with just that change and a fresh rebase. > > Thanks. I think the

Re: Statistics Import and Export

2023-08-31 Thread Corey Huinker
> > > Thanks. I think this may be used with postgres_fdw to import > statistics directly from the foreigns server, whenever possible, > rather than fetching the rows and building it locally. If it's known > that the stats on foreign and local servers match for a foreign table, > we will be one

Re: [17] CREATE SUBSCRIPTION ... SERVER

2023-08-31 Thread Joe Conway
On 8/31/23 12:52, Jeff Davis wrote: On Thu, 2023-08-31 at 10:59 +0530, Ashutosh Bapat wrote: The server's FDW has to be postgres_fdw. So we have to handle the awkward dependency between core and postgres_fdw (an extension). That sounds more than just "awkward". I can't think of any precedent

Re: Document efficient self-joins / UPDATE LIMIT techniques.

2023-08-31 Thread Corey Huinker
On Wed, Jun 28, 2023 at 2:20 PM Corey Huinker wrote: > This patch adds a few examples to demonstrate the following: > Bumping so CF app can see thread. >

Suppressing compiler warning on Debian 12/gcc 12.2.0

2023-08-31 Thread Bruce Momjian
Being a new user of Debian 12/gcc 12.2.0, I wrote the following shell script to conditionally add gmake rules with compiler flags to src/Makefile.custom to suppress warnings for certain files. This allows me to compile all supported Postgres releases without warnings. I actually didn't how

Re: UUID v7

2023-08-31 Thread Andrey M. Borodin
Thanks for interesting ideas, Mat! > On 31 Aug 2023, at 20:32, Mat Arye wrote: > > From a user perspective, it would be great to add 2 things: > - A function to extract the timestamp from a V7 UUID (very useful for > defining constraints if partitioning by the uuid-embedded timestamps, for >

Re: Replace some cstring_to_text to cstring_to_text_with_len

2023-08-31 Thread Dagfinn Ilmari Mannsåker
Peter Eisentraut writes: > On 31.08.23 16:10, Ranier Vilela wrote: >> Em qui., 31 de ago. de 2023 às 09:51, Andrew Dunstan >> mailto:and...@dunslane.net>> escreveu: >> >> On 2023-08-31 Th 07:41, John Naylor wrote: >>> >>> On Thu, Aug 31, 2023 at 6:07 PM Ranier Vilela >>

Re: Improve heapgetpage() performance, overhead from serializable

2023-08-31 Thread Muhammad Malik
Hi, Is there a plan to merge this patch in PG16? Thanks, Muhammad From: Andres Freund Sent: Saturday, July 15, 2023 6:56 PM To: pgsql-hack...@postgresql.org Cc: Thomas Munro Subject: Improve heapgetpage() performance, overhead from serializable Hi, Several

Re: Eliminate redundant tuple visibility check in vacuum

2023-08-31 Thread Robert Haas
On Mon, Aug 28, 2023 at 7:49 PM Melanie Plageman wrote: > While working on a set of patches to combine the freeze and visibility > map WAL records into the prune record, I wrote the attached patches > reusing the tuple visibility information collected in heap_page_prune() > back in

Re: Use virtual tuple slot for Unique node

2023-08-31 Thread Денис Смирнов
Again the new patch hasn't been attached to the thread, so resend it. v3-use-virtual-slots-for-unique-node.patch Description: Binary data

Re: [17] CREATE SUBSCRIPTION ... SERVER

2023-08-31 Thread Jeff Davis
On Thu, 2023-08-31 at 08:37 -0400, Robert Haas wrote: > What I feel is kind of weird about this syntax is that it seems like > it's entangled with the FDW mechanism but doesn't really overlap with > it. I like the fact that it works with user mappings and benefits from the other thinking that's

Re: Replace some cstring_to_text to cstring_to_text_with_len

2023-08-31 Thread Dagfinn Ilmari Mannsåker
Ranier Vilela writes: > Em qui., 31 de ago. de 2023 às 12:12, Dagfinn Ilmari Mannsåker < > ilm...@ilmari.org> escreveu: > >> Ranier Vilela writes: >> >> > Em qui., 31 de ago. de 2023 às 10:12, Dagfinn Ilmari Mannsåker < >> > ilm...@ilmari.org> escreveu: >> > >> >> Andrew Dunstan writes: >> >>

Re: Use virtual tuple slot for Unique node

2023-08-31 Thread Denis Smirnov
I have made a small research and found out that though the patch itself is correct (i.e. we can benefit from replacing TTSOpsMinimalTuple with TTSOpsVirtual for the Unique node), my explanation WHY was wrong.1. We always materialize the new unique tuple in the slot, never mind what type of tuple

Re: Replace some cstring_to_text to cstring_to_text_with_len

2023-08-31 Thread Ranier Vilela
Em qui., 31 de ago. de 2023 às 12:12, Dagfinn Ilmari Mannsåker < ilm...@ilmari.org> escreveu: > Ranier Vilela writes: > > > Em qui., 31 de ago. de 2023 às 10:12, Dagfinn Ilmari Mannsåker < > > ilm...@ilmari.org> escreveu: > > > >> Andrew Dunstan writes: > >> > >> > On 2023-08-31 Th 07:41, John

Re: [17] CREATE SUBSCRIPTION ... SERVER

2023-08-31 Thread Jeff Davis
On Thu, 2023-08-31 at 10:59 +0530, Ashutosh Bapat wrote: > The server's FDW has to be postgres_fdw. So we have to handle the > awkward dependency between core and postgres_fdw (an extension). That sounds more than just "awkward". I can't think of any precedent for that and it seems to violate the

Re: [17] CREATE SUBSCRIPTION ... SERVER

2023-08-31 Thread Jeff Davis
On Wed, 2023-08-30 at 09:09 -0700, Jeff Davis wrote: > Admittedly, I didn't complete the dummy-FDW approach, so perhaps it > works out better overall. I can give it a try. We need to hide the dummy FDW from pg_dump. And we need to hide it from psql's \dew, because that's used in tests and prints

Re: Initdb-time block size specification

2023-08-31 Thread David Christensen
> I was definitely hand-waving additional implementation here for > non-native 128 bit support; the modulus algorithm as presented > requires 4 times the space as the divisor, so a uint16 implementation > should work for all 64-bit machines. Certainly open to other ideas or > implementations,

Re: Initdb-time block size specification

2023-08-31 Thread David Christensen
> + * pg_fastmod - calculates the modulus of a 32-bit number against a constant > + * divisor without using the division operator > + */ > +static inline uint32 pg_fastmod(uint32 n, uint32 divisor, uint64 fastinv) > +{ > +#ifdef HAVE_INT128 > + uint64_t lowbits = fastinv * n; > + return

Re: Initdb-time block size specification

2023-08-31 Thread John Naylor
On Thu, Aug 31, 2023 at 8:51 AM David Christensen < david.christen...@crunchydata.com> wrote: > 0005 - utility functions for fast div/mod operations; basically > montgomery multiplication +/* + * pg_fastmod - calculates the modulus of a 32-bit number against a constant + * divisor without using

Re: should frontend tools use syncfs() ?

2023-08-31 Thread Nathan Bossart
On Thu, Aug 31, 2023 at 02:30:33PM +0900, Michael Paquier wrote: > - Should we have some regression tests? We should only need one test > in one of the binaries to be able to stress the new code paths of > file_utils.c with syncfs. The cheapest one may be pg_dump with a > dump in directory

Re: UUID v7

2023-08-31 Thread Mat Arye
Andrey, Thanks for all your work on this. I think this will be really useful. >From a user perspective, it would be great to add 2 things: - A function to extract the timestamp from a V7 UUID (very useful for defining constraints if partitioning by the uuid-embedded timestamps, for instance). -

Re: Replace some cstring_to_text to cstring_to_text_with_len

2023-08-31 Thread Peter Eisentraut
On 31.08.23 16:10, Ranier Vilela wrote: Em qui., 31 de ago. de 2023 às 09:51, Andrew Dunstan mailto:and...@dunslane.net>> escreveu: On 2023-08-31 Th 07:41, John Naylor wrote: On Thu, Aug 31, 2023 at 6:07 PM Ranier Vilela mailto:ranier...@gmail.com>> wrote: > > Em qui., 31 de

sandboxing untrusted code

2023-08-31 Thread Robert Haas
On Mon, Feb 27, 2023 at 7:37 PM Jeff Davis wrote: > On Mon, 2023-02-27 at 16:13 -0500, Robert Haas wrote: > > On Mon, Feb 27, 2023 at 1:25 PM Jeff Davis wrote: > > > I think you are saying that we should still run Alice's code with > > > the > > > privileges of Bob, but somehow make that safe(r)

Re: Replace some cstring_to_text to cstring_to_text_with_len

2023-08-31 Thread Dagfinn Ilmari Mannsåker
Ranier Vilela writes: > Em qui., 31 de ago. de 2023 às 10:12, Dagfinn Ilmari Mannsåker < > ilm...@ilmari.org> escreveu: > >> Andrew Dunstan writes: >> >> > On 2023-08-31 Th 07:41, John Naylor wrote: >> >> >> >> On Thu, Aug 31, 2023 at 6:07 PM Ranier Vilela >> wrote: >> >> > >> >> > Em qui., 31

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-31 Thread Hayato Kuroda (Fujitsu)
Dear Dilip, Thanks for giving comments! > Some comments in 0002 > > 1. > + res = executeQueryOrDie(conn, "SELECT slot_name " > + "FROM pg_catalog.pg_replication_slots " > + "WHERE slot_type = 'logical' AND " > + "temporary IS FALSE;"); > > What is the reason we are ignoring temporary slots

Re: Replace some cstring_to_text to cstring_to_text_with_len

2023-08-31 Thread Ranier Vilela
Em qui., 31 de ago. de 2023 às 10:12, Dagfinn Ilmari Mannsåker < ilm...@ilmari.org> escreveu: > Andrew Dunstan writes: > > > On 2023-08-31 Th 07:41, John Naylor wrote: > >> > >> On Thu, Aug 31, 2023 at 6:07 PM Ranier Vilela > wrote: > >> > > >> > Em qui., 31 de ago. de 2023 às 00:22, Michael

Re: Replace some cstring_to_text to cstring_to_text_with_len

2023-08-31 Thread Ranier Vilela
Em qui., 31 de ago. de 2023 às 09:51, Andrew Dunstan escreveu: > > On 2023-08-31 Th 07:41, John Naylor wrote: > > > On Thu, Aug 31, 2023 at 6:07 PM Ranier Vilela wrote: > > > > Em qui., 31 de ago. de 2023 às 00:22, Michael Paquier < > mich...@paquier.xyz> escreveu: > >> > >> On Wed, Aug 30,

Re: Replace some cstring_to_text to cstring_to_text_with_len

2023-08-31 Thread Ranier Vilela
Em qui., 31 de ago. de 2023 às 08:41, John Naylor < john.nay...@enterprisedb.com> escreveu: > > On Thu, Aug 31, 2023 at 6:07 PM Ranier Vilela wrote: > > > > Em qui., 31 de ago. de 2023 às 00:22, Michael Paquier < > mich...@paquier.xyz> escreveu: > >> > >> On Wed, Aug 30, 2023 at 03:00:13PM

Re: persist logical slots to disk during shutdown checkpoint

2023-08-31 Thread Amit Kapila
On Thu, Aug 31, 2023 at 6:12 PM Ashutosh Bapat wrote: > > On Thu, Aug 31, 2023 at 2:52 PM Amit Kapila wrote: > > > > > I > > > > > think we should shut down subscriber, restart publisher and then make > > > > > this > > > > > check based on the contents of the replication slot instead of server

Re: remaining sql/json patches

2023-08-31 Thread Erik Rijkers
Op 8/31/23 om 14:57 schreef Amit Langote: Hello, On Wed, Aug 16, 2023 at 1:27 PM Amit Langote wrote: I will post a new version after finishing working on a few other improvements I am working on. Sorry about the delay. Here's a new version. Hi, While compiling the new set

Re: Adding a pg_get_owned_sequence function?

2023-08-31 Thread Dagfinn Ilmari Mannsåker
On Fri, 9 Jun 2023, at 20:19, Dagfinn Ilmari Mannsåker wrote: > Hi hackers, > > I've always been annoyed by the fact that pg_get_serial_sequence takes > the table and returns the sequence as strings rather than regclass. And > since identity columns were added, the name is misleading as well

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-31 Thread Dilip Kumar
On Tue, Aug 29, 2023 at 5:28 PM Hayato Kuroda (Fujitsu) wrote: Some comments in 0002 1. + res = executeQueryOrDie(conn, "SELECT slot_name " + "FROM pg_catalog.pg_replication_slots " + "WHERE slot_type = 'logical' AND " + "temporary IS FALSE;"); What is the reason we are ignoring temporary

Re: Replace some cstring_to_text to cstring_to_text_with_len

2023-08-31 Thread Dagfinn Ilmari Mannsåker
Andrew Dunstan writes: > On 2023-08-31 Th 07:41, John Naylor wrote: >> >> On Thu, Aug 31, 2023 at 6:07 PM Ranier Vilela wrote: >> > >> > Em qui., 31 de ago. de 2023 às 00:22, Michael Paquier >> escreveu: >> >> >> >> On Wed, Aug 30, 2023 at 03:00:13PM -0300, Ranier Vilela wrote: >> >> >

Re: Initdb-time block size specification

2023-08-31 Thread David Christensen
Enclosed are TPC-H results for 1GB shared_buffers, 64MB work_mem on a 64GB laptop with SSD storage; everything else is default settings. TL;DR: unpatched version: 17.30 seconds, patched version: 17.15; there are some slight variations in runtime, but seems to be within the noise level at this

Re: Replace some cstring_to_text to cstring_to_text_with_len

2023-08-31 Thread Andrew Dunstan
On 2023-08-31 Th 07:41, John Naylor wrote: On Thu, Aug 31, 2023 at 6:07 PM Ranier Vilela wrote: > > Em qui., 31 de ago. de 2023 às 00:22, Michael Paquier escreveu: >> >> On Wed, Aug 30, 2023 at 03:00:13PM -0300, Ranier Vilela wrote: >> > cstring_to_text has a small overhead, because call

Re: persist logical slots to disk during shutdown checkpoint

2023-08-31 Thread Ashutosh Bapat
On Thu, Aug 31, 2023 at 2:52 PM Amit Kapila wrote: > > > > I > > > > think we should shut down subscriber, restart publisher and then make > > > > this > > > > check based on the contents of the replication slot instead of server > > > > log. > > > > Shutting down subscriber will ensure that

Re: [17] CREATE SUBSCRIPTION ... SERVER

2023-08-31 Thread Robert Haas
On Wed, Aug 30, 2023 at 1:19 PM Jeff Davis wrote: > On Wed, 2023-08-30 at 09:49 -0400, Tom Lane wrote: > > This seems like it requires a whole lot of new mechanism (parser > > and catalog infrastructure) that could be done far more easily > > in other ways. In particular, how about inventing a

Re: Replace some cstring_to_text to cstring_to_text_with_len

2023-08-31 Thread John Naylor
On Thu, Aug 31, 2023 at 6:07 PM Ranier Vilela wrote: > > Em qui., 31 de ago. de 2023 às 00:22, Michael Paquier escreveu: >> >> On Wed, Aug 30, 2023 at 03:00:13PM -0300, Ranier Vilela wrote: >> > cstring_to_text has a small overhead, because call strlen for >> > pointer to char parameter. >> > >>

Re: Should we use MemSet or {0} for struct initialization?

2023-08-31 Thread Junwang Zhao
On Thu, Aug 31, 2023 at 7:07 PM John Naylor wrote: > > > On Thu, Aug 31, 2023 at 5:34 PM Richard Guo wrote: > > > > > > While working on a bug in expandRecordVariable() I noticed that in the > > > switch statement for case RTE_SUBQUERY we initialize struct ParseState > > > with {0} while for

Re: Commitfest 2023-09 starts soon

2023-08-31 Thread Aleksander Alekseev
Hi, > > There are a number of patches carried over from the PG16 development > > cycle that have been in "Waiting on author" for several months. I will > > aggressively prune those after the start of this commitfest if there > > hasn't been any author activity by then. > > The "64-bit TOAST

Re: generate syscache info automatically

2023-08-31 Thread Peter Eisentraut
I have committed 0002 and 0003, and also a small bug fix in the ObjectProperty entry for "transforms". I have also gotten the automatic generation of the ObjectProperty lookup table working (with some warts). Attached is an updated patch set. One win here is that the ObjectProperty lookup

Re: Commitfest 2023-09 starts soon

2023-08-31 Thread Aleksander Alekseev
Hi Peter, > Commitfest 2023-09 (https://commitfest.postgresql.org/44/) starts in > less than 28 hours. > > If you have any patches you would like considered, be sure to add them > in good time. > > All patch authors, and especially experienced hackers, are requested to > make sure the patch

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-31 Thread Dilip Kumar
On Thu, Aug 31, 2023 at 4:38 AM Nathan Bossart wrote: > > On Wed, Aug 30, 2023 at 10:56:22AM -0400, Robert Haas wrote: > > On Wed, Aug 30, 2023 at 10:27 AM Nathan Bossart > > wrote: > >> I'm about to spend way too much time writing the commit message for 0002, > >> but I plan to commit both

Re: lockup in parallel hash join on dikkop (freebsd 14.0-current)

2023-08-31 Thread Thomas Munro
On Thu, Aug 31, 2023 at 2:32 PM Thomas Munro wrote: > On Thu, Aug 31, 2023 at 12:16 AM Tomas Vondra > wrote: > > I have another case of this on dikkop (on 11 again). Is there anything > > else we'd want to try? Or maybe someone would want access to the machine > > and do some investigation

Re: Should we use MemSet or {0} for struct initialization?

2023-08-31 Thread John Naylor
> On Thu, Aug 31, 2023 at 5:34 PM Richard Guo wrote: > > > > While working on a bug in expandRecordVariable() I noticed that in the > > switch statement for case RTE_SUBQUERY we initialize struct ParseState > > with {0} while for case RTE_CTE we do that with MemSet. I understand > > that there

Re: Replace some cstring_to_text to cstring_to_text_with_len

2023-08-31 Thread Ranier Vilela
Em qui., 31 de ago. de 2023 às 00:22, Michael Paquier escreveu: > On Wed, Aug 30, 2023 at 03:00:13PM -0300, Ranier Vilela wrote: > > cstring_to_text has a small overhead, because call strlen for > > pointer to char parameter. > > > > Is it worth the effort to avoid this, where do we know the

Re: cataloguing NOT NULL constraints

2023-08-31 Thread Alexander Lakhin
31.08.2023 13:26, Alvaro Herrera wrote: Hmm, that's some weird code I left there all right. Can you please try this patch? (Not final; I'll review it more completely later, particularly to add this test case.) Yes, your patch fixes the issue. I get the same error now: ERROR:  column "a" in

RE: Initial Schema Sync for Logical Replication

2023-08-31 Thread Kumar, Sachin
Hi Everyone, based on internal discussion with Masahiko I have implemented concurrent DDL support for initial schema sync. Concurrent Patch workflow 1. When TableSync worker creates a replicaton slot, It will save the slot lsn into pg_subscription_rel with SUBREL_SYNC_SCHEMA_DATA_SYNC state, and

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-31 Thread Amit Kapila
On Wed, Aug 30, 2023 at 10:58 AM Peter Smith wrote: > > Here are some review comments for v28-0003. > > == > src/bin/pg_upgrade/check.c > > 1. check_and_dump_old_cluster > + /* > + * Logical replication slots can be migrated since PG17. See comments atop > + *

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-31 Thread Amit Kapila
On Wed, Aug 30, 2023 at 7:55 AM Peter Smith wrote: > > Here are some minor review comments for patch v28-0002 > > == > src/sgml/ref/pgupgrade.sgml > > 1. > - with the primary.) Replication slots are not copied and must > - be recreated. > + with the primary.) Replication

Re: cataloguing NOT NULL constraints

2023-08-31 Thread Alvaro Herrera
Hello Alexander, Thanks for testing. On 2023-Aug-31, Alexander Lakhin wrote: > 25.08.2023 14:38, Alvaro Herrera wrote: > > I have now pushed this again. Hopefully it'll stick this time. > > I've found that after that commit the following query: > CREATE TABLE t(a int PRIMARY KEY) PARTITION BY

Re: Sync scan & regression tests

2023-08-31 Thread Heikki Linnakangas
On 31/08/2023 02:37, Melanie Plageman wrote: On Wed, Aug 30, 2023 at 5:15 PM David Rowley wrote: I just looked at v15's code and I agree that the ss_report_location() would be called even when the scan is finished. It wasn't intentional that that was changed in v16, so I'm happy for your

Re: cataloguing NOT NULL constraints

2023-08-31 Thread Alvaro Herrera
On 2023-Mar-29, Peter Eisentraut wrote: > On 27.03.23 15:55, Peter Eisentraut wrote: > > The information schema should be updated.  I think the following views: > > > > - CHECK_CONSTRAINTS > > - CONSTRAINT_COLUMN_USAGE > > - DOMAIN_CONSTRAINTS > > - TABLE_CONSTRAINTS > > > > It looks like these

Re: cataloguing NOT NULL constraints

2023-08-31 Thread Alexander Lakhin
Hi Alvaro, 25.08.2023 14:38, Alvaro Herrera wrote: I have now pushed this again. Hopefully it'll stick this time. I've found that after that commit the following query: CREATE TABLE t(a int PRIMARY KEY) PARTITION BY RANGE (a); CREATE TABLE tp1(a int); ALTER TABLE t ATTACH PARTITION tp1 FOR

Re: Should we use MemSet or {0} for struct initialization?

2023-08-31 Thread Junwang Zhao
On Thu, Aug 31, 2023 at 5:34 PM Richard Guo wrote: > > While working on a bug in expandRecordVariable() I noticed that in the > switch statement for case RTE_SUBQUERY we initialize struct ParseState > with {0} while for case RTE_CTE we do that with MemSet. I understand > that there is nothing

Re: Sync scan & regression tests

2023-08-31 Thread Heikki Linnakangas
On 29/08/2023 13:35, Heikki Linnakangas wrote: On 07/08/2023 03:55, Tom Lane wrote: This is possibly explained by the fact that it uses (per its extra_config) 'shared_buffers = 10MB', although it's done that for a long time and portals.out hasn't changed since before

Re: Eliminate redundant tuple visibility check in vacuum

2023-08-31 Thread David Geier
Hi Melanie, On 8/31/23 02:59, Melanie Plageman wrote: I created a large table and then updated a tuple on every page in the relation and vacuumed it. I saw a consistent slight improvement in vacuum execution time. I profiled a bit with perf stat as well. The difference is relatively small for

Re: persist logical slots to disk during shutdown checkpoint

2023-08-31 Thread Amit Kapila
On Thu, Aug 31, 2023 at 12:25 PM Ashutosh Bapat wrote: > > On Thu, Aug 31, 2023 at 12:10 PM Amit Kapila wrote: > > > > > + > > > +/* > > > + * We won't ensure that the slot is persisted after the > > > confirmed_flush > > > + * LSN is updated as that could lead to frequent writes.

pg_basebackup: Always return valid temporary slot names

2023-08-31 Thread Jelte Fennema
With PgBouncer in the middle PQbackendPID can return negative values due to it filling all 32 bits of the be_pid with random bits. When this happens it results in pg_basebackup generating an invalid slot name (when no specific slot name is passed in) and thus throwing an error like this:

Re: Extract numeric filed in JSONB more effectively

2023-08-31 Thread Andy Fan
Hi Chap, The v11 attached, mainly changes are: 1. use the jsonb_xx_start and jsonb_finish_numeric style. 2. improve the test case a bit. It doesn't include: 1. the jsonb_finish_text function, since we have a operator ->> for text already and the performance for it is OK and there is no cast

Re: Allow specifying a dbname in pg_basebackup connection string

2023-08-31 Thread Jelte Fennema
Attached is a new version with some slightly updated wording in the docs v4-0001-Allow-specifying-a-dbname-in-pg_basebackup-connec.patch Description: Binary data

Commitfest 2023-09 starts soon

2023-08-31 Thread Peter Eisentraut
Commitfest 2023-09 (https://commitfest.postgresql.org/44/) starts in less than 28 hours. If you have any patches you would like considered, be sure to add them in good time. All patch authors, and especially experienced hackers, are requested to make sure the patch status is up to date. If

Should we use MemSet or {0} for struct initialization?

2023-08-31 Thread Richard Guo
While working on a bug in expandRecordVariable() I noticed that in the switch statement for case RTE_SUBQUERY we initialize struct ParseState with {0} while for case RTE_CTE we do that with MemSet. I understand that there is nothing wrong with this, just cannot get away with the inconsistency

Re: Make --help output fit within 80 columns per line

2023-08-31 Thread torikoshia
On Mon, Aug 21, 2023 at 1:09 PM Masahiro Ikeda wrote: (1) Why don't you add test for the purpose? It could be overkill... I though the following function is the best place. Added the test. BTW, psql --help outputs the content of PGHOST, which caused a failure in the test: ``` -h,

Re: Statistics Import and Export

2023-08-31 Thread Ashutosh Bapat
On Thu, Aug 31, 2023 at 12:17 PM Corey Huinker wrote: > > While the primary purpose of the import function(s) are to reduce downtime > during an upgrade, it is not hard to see that they could also be used to > facilitate tuning and development operations, asking questions like "how might > this

Re: persist logical slots to disk during shutdown checkpoint

2023-08-31 Thread Ashutosh Bapat
On Thu, Aug 31, 2023 at 12:10 PM Amit Kapila wrote: > > > + > > +/* > > + * We won't ensure that the slot is persisted after the confirmed_flush > > + * LSN is updated as that could lead to frequent writes. However, we > > need > > + * to ensure that we do persist the slots at

Statistics Import and Export

2023-08-31 Thread Corey Huinker
pg_stats_export is a view that aggregates pg_statistic data by relation oid and stores all of the column statistical data in a system-indepdent (i.e. no oids, collation information removed, all MCV values rendered as text) jsonb format, along with the relation's relname, reltuples, and relpages

Re: persist logical slots to disk during shutdown checkpoint

2023-08-31 Thread Amit Kapila
On Wed, Aug 30, 2023 at 6:33 PM Ashutosh Bapat wrote: > > On Tue, Aug 29, 2023 at 5:40 PM Ashutosh Bapat > wrote: > > > > I am looking at it. If you can wait till the end of the week, that > > will be great. > > /* > * Successfully wrote, unset dirty bit, unless somebody dirtied again