Re: bogus: logical replication rows/cols combinations

2022-05-06 Thread Amit Kapila
On Mon, May 2, 2022 at 6:11 PM Alvaro Herrera wrote: > > On 2022-May-02, Amit Kapila wrote: > > > > I think it is possible to expose a list of publications for each > > walsender as it is stored in each walsenders > > LogicalDecodingContext->output_plugin_private. AFAIK, each walsender > > can hav

Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-05-06 Thread Thomas Munro
On Wed, May 4, 2022 at 2:23 PM Thomas Munro wrote: > Assuming no > objections or CI failures show up, I'll consider pushing the first two > patches tomorrow. Done. Time to watch the build farm. It's possible that these changes will produce some blowback, now that we're using PROCSIGNAL_BARRIER_

Re: Support logical replication of DDLs

2022-05-06 Thread Amit Kapila
On Fri, May 6, 2022 at 10:21 PM Zheng Li wrote: > > > Attached is a set of two patches as an attempt to evaluate this approach. > > > > Thanks for exploring this direction. > > I read the deparsing thread and your patch. Here is my thought: > 1. The main concern on maintainability of the deparsing

Re: Collation version tracking for macOS

2022-05-06 Thread Thomas Munro
On Mon, Feb 14, 2022 at 10:00 PM Peter Eisentraut wrote: > During development, I have been using the attached patch to simulate > libc collation versions on macOS. It just uses the internal major OS > version number. I don't know to what the extend the libc locales on > macOS are maintained or u

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Robert Haas
On Fri, May 6, 2022 at 5:15 PM Nathan Bossart wrote: > On Fri, May 06, 2022 at 08:27:11AM -0700, Nathan Bossart wrote: > > +1, I'll put together a new patch set. > > As promised... Looks reasonable to me. Anyone else have thoughts? -- Robert Haas EDB: http://www.enterprisedb.com

Re: Mark all GUC variable as PGDLLIMPORT

2022-05-06 Thread Andres Freund
Hi, On 2022-04-08 08:42:38 -0400, Robert Haas wrote: > On Wed, Apr 6, 2022 at 7:56 PM Michael Paquier wrote: > > On Wed, Apr 06, 2022 at 12:57:29AM +0700, John Naylor wrote: > > > For these two patches, I'd say a day or two after feature freeze is a > > > reasonable goal. > > > > Yeah. For patch

Re: Possible corruption by CreateRestartPoint at promotion

2022-05-06 Thread Michael Paquier
On Fri, May 06, 2022 at 08:52:45AM -0700, Nathan Bossart wrote: > I was looking at other changes in this area (e.g., 3c64dcb), and now I'm > wondering if we actually should invalidate the minRecoveryPoint when the > control file no longer indicates archive recovery. Specifically, what > happens if

Re: [RFC] building postgres with meson -v8

2022-05-06 Thread Andres Freund
Hi, On 2022-05-06 14:27:24 -0700, Andres Freund wrote: > > 0003-meson-Install-all-server-headers.patch > > > > With this, all the server headers installed by a makefile-based build are > > installed. I tried to strike a balance between using install_subdir() with > > exclude list versus listing

Re: [RFC] building postgres with meson -v8

2022-05-06 Thread Andres Freund
Hi, On 2022-04-21 17:34:47 -0400, Tom Lane wrote: > FWIW, I don't think that either gaur or prairiedog need be factored into > this conversation. They cannot build ninja at all for lack of , > so whether they could run meson is pretty much beside the point. Yea. > (I wonder if we should stick

Re: [RFC] building postgres with meson -v8

2022-05-06 Thread Andres Freund
Hi, On 2022-05-04 13:53:54 +0200, Peter Eisentraut wrote: > 0001-meson-Assorted-compiler-test-tweaks.patch > > I was going through a diff of pg_config.h between old and new build and > found a few omissions and small differences. Thanks, merged that. > is of course annoying and can be removed

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Nathan Bossart
On Fri, May 06, 2022 at 08:27:11AM -0700, Nathan Bossart wrote: > +1, I'll put together a new patch set. As promised... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 831218d6e0c04d6342bf593dbf6799efdd831b6b Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 18 Apr 20

Re: configure openldap crash warning

2022-05-06 Thread Tom Lane
I wrote: > After thinking about this for awhile, it seems like the best solution > is to make configure proceed like this: > 1. Find libldap. > 2. Detect whether it's OpenLDAP 2.5 or newer. > 3. If not, try to find libldap_r. Here's a proposed patch for that. It seems to do the right thing with

Re: configure openldap crash warning

2022-05-06 Thread Tom Lane
I wrote: > We still have a bit of work to do, because this setup isn't getting > all the way through src/test/ldap/: > 2022-05-04 11:01:33.459 EDT [21304] LOG: server process (PID 21312) was > terminated by signal 11: Segmentation fault: 11 > Many of the test cases pass, but it looks like ldaps-r

Re: Configuration Parameter/GUC value validation hook

2022-05-06 Thread Euler Taveira
On Wed, May 4, 2022, at 8:12 AM, Bharath Rupireddy wrote: > How about we provide a sample extension (limiting some important > parameters say shared_buffers, work_mem and so on to some > "reasonable/recommended" limits) in the core along with the > set_config_option_hook? This way, all the people u

Re: failures in t/031_recovery_conflict.pl on CI

2022-05-06 Thread Tom Lane
Andres Freund writes: > Done. Perhaps you could trigger a run on longfin, that seems to have been the > most reliably failing animal? No need, its cron job launched already. regards, tom lane

Re: configure openldap crash warning

2022-05-06 Thread Tom Lane
I wrote: > Oh, I have a theory about this: I bet your Homebrew installation > has a recent OpenLDAP version that only supplies libldap not libldap_r. > In that case, configure will still find libldap_r available and will > bind libpq to it, and you get the observed result. The configure > check is

Re: failures in t/031_recovery_conflict.pl on CI

2022-05-06 Thread Andres Freund
On 2022-05-06 12:12:19 -0400, Tom Lane wrote: > Andres Freund writes: > > I looked through all the failures I found and it's two kinds of failures, > > both > > related to the deadlock test. So I'm thinking of skipping just that test as > > in > > the attached. > > > Working on committing / bac

Re: Estimating HugePages Requirements?

2022-05-06 Thread Nathan Bossart
On Tue, Apr 26, 2022 at 10:34:06AM +0900, Michael Paquier wrote: > Yes, the redirection issue would apply to all the run-time GUCs. Should this be tracked as an open item for v15? There was another recent report about the extra log output [0]. [0] https://www.postgresql.org/message-id/YnARlI5nvb

Re: Support logical replication of DDLs

2022-05-06 Thread Zheng Li
> Attached is a set of two patches as an attempt to evaluate this approach. > > The first patch provides functions to deparse DDL commands. Currently, > it is restricted to just a simple CREATE TABLE statement, the required > code is extracted from one of the patches posted in the thread [1]. > > T

Re: Can postgres ever delete the recycled future WAL files to free-up disk space if max_wal_size is reduced or wal_recycle is set to off?

2022-05-06 Thread Tom Lane
Bharath Rupireddy writes: > Can postgres delete the recycled future WAL files once max_wal_size is > reduced and/or wal_recycle is set to off? A checkpoint should do that, see RemoveOldXlogFiles. Maybe you have a broken WAL archiving setup, or something else preventing removal of old WAL files?

Can postgres ever delete the recycled future WAL files to free-up disk space if max_wal_size is reduced or wal_recycle is set to off?

2022-05-06 Thread Bharath Rupireddy
Hi, I have a scenario where max_wal_size = 10GB and wal_recycle = on, the postgres starts to recycle and keep WAL files for future use, eventually around 600~ WAL files have been kept in the pg_wal directory. The checkpoints were happening at regular intervals. But the disk was about to get full (

Re: failures in t/031_recovery_conflict.pl on CI

2022-05-06 Thread Tom Lane
Andres Freund writes: > I looked through all the failures I found and it's two kinds of failures, both > related to the deadlock test. So I'm thinking of skipping just that test as in > the attached. > Working on committing / backpatching that, unless somebody suggests changes > quickly... WFM.

Re: failures in t/031_recovery_conflict.pl on CI

2022-05-06 Thread Andres Freund
Hi, On 2022-05-05 23:57:28 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2022-05-05 23:36:22 -0400, Tom Lane wrote: > >> So I reluctantly vote for removing 031_recovery_conflict.pl in the > >> back branches for now, with the expectation that we'll fix the > >> infrastructure and put it ba

Re: Possible corruption by CreateRestartPoint at promotion

2022-05-06 Thread Nathan Bossart
On Fri, May 06, 2022 at 07:58:43PM +0900, Michael Paquier wrote: > And I have spent a bit of this stuff to finish with the attached. It > will be a plus to get that done on HEAD for beta1, so I'll try to deal > with it on Monday. I am still a bit stressed about the back branches > as concurrent c

Re: libpq async duplicate error results

2022-05-06 Thread Tom Lane
I wrote: > What is happening is that this bit in PQsendQueryStart is deciding > not to clear the message buffer because conn->cmd_queue_head isn't > NULL: > /* > * If this is the beginning of a query cycle, reset the error state. > * However, in pipeline mode with something already q

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Nathan Bossart
On Fri, May 06, 2022 at 10:43:21AM -0400, Tom Lane wrote: > Robert Haas writes: >> On Fri, May 6, 2022 at 9:57 AM Tom Lane wrote: >>> I agree that _PG_fini functions as they stand are worthless. >>> What I'm not getting is why we should care enough about that >>> to break just about everybody's e

Re: gcc 12.1.0 warning

2022-05-06 Thread Tom Lane
Erik Rijkers writes: > Not sure if these compiler-mutterings are worth reporting but I guess > we're trying to get a silent compile. > System: Debian 4.9.303-1 (2022-03-07) x86_64 GNU/Linux > Compiling with gcc 12.1.0 causes the below 'warning' and 'note'. > Compiling with --enable-cassert --ena

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Tom Lane
Robert Haas writes: > On Fri, May 6, 2022 at 9:57 AM Tom Lane wrote: >> I agree that _PG_fini functions as they stand are worthless. >> What I'm not getting is why we should care enough about that >> to break just about everybody's extension. Even if unloading >> extensions were feasible, who wo

gcc 12.1.0 warning

2022-05-06 Thread Erik Rijkers
Hi, Not sure if these compiler-mutterings are worth reporting but I guess we're trying to get a silent compile. System: Debian 4.9.303-1 (2022-03-07) x86_64 GNU/Linux Compiling with gcc 12.1.0 causes the below 'warning' and 'note'. Compiling with --enable-cassert --enable-debug is silent, no

Re: libpq async duplicate error results

2022-05-06 Thread Tom Lane
Peter Eisentraut writes: > I took another look at this. The output from the test program at the > beginning of the thread is now (master branch): > ... > command = SELECT 'after'; > PQsendQuery() error: FATAL: terminating connection due to administrator > command > server closed the connection

Re: BufferAlloc: don't take two simultaneous locks

2022-05-06 Thread Robert Haas
On Thu, Apr 21, 2022 at 6:58 PM Yura Sokolov wrote: > At the master state: > - SharedBufHash is not declared as HASH_FIXED_SIZE > - get_hash_entry falls back to element_alloc too fast (just if it doesn't > found free entry in current freelist partition). > - get_hash_entry has races. > - if ther

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Robert Haas
On Fri, May 6, 2022 at 9:57 AM Tom Lane wrote: > Robert Haas writes: > > perhaps for v16 or some future release we can think about redoing a > > bunch of hooks this way. There would be some migration pain for > > extension authors for sure, but then we might get to a point where > > extensions ca

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Tom Lane
Robert Haas writes: > perhaps for v16 or some future release we can think about redoing a > bunch of hooks this way. There would be some migration pain for > extension authors for sure, but then we might get to a point where > extensions can be cleanly unloaded in at least some circumstances, > in

Re: bogus: logical replication rows/cols combinations

2022-05-06 Thread Tomas Vondra
On 5/6/22 15:40, Amit Kapila wrote: > On Fri, May 6, 2022 at 5:56 PM Tomas Vondra > wrote: >> I could think of below two options: 1. Forbid any case where column list is different for the same table when combining publications. 2. Forbid if the column list and row filte

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Robert Haas
On Tue, Apr 19, 2022 at 11:47 AM Nathan Bossart wrote: > Okay, I did it this way in v5. I pushed 0001. Regarding 0002, I think there's no point in adding a _PG_fini(). The code to call _PG_fini() has been surrounded by #ifdef NOT_USED forever, and that seems unlikely to change any time soon as lo

Re: libpq async duplicate error results

2022-05-06 Thread Peter Eisentraut
I took another look at this. The output from the test program at the beginning of the thread is now (master branch): command = SELECT 'before'; result 1 status = PGRES_TUPLES_OK error message = "" command = SELECT pg_terminate_backend(pg_backend_pid()); result 1 status = PGRES_FATAL_ERROR erro

Re: Trying to add more tests to gistbuild.c

2022-05-06 Thread Matheus Alcantara
The attached patch is failing on make check due to a typo, resubmitting the correct one. -- Matheus Alcantaradiff --git a/src/test/regress/expected/gist.out b/src/test/regress/expected/gist.out index a36b4c9c56..b5edc44250 100644 --- a/src/test/regress/expected/gist.out +++ b/src/test/regress/exp

Re: bogus: logical replication rows/cols combinations

2022-05-06 Thread Amit Kapila
On Fri, May 6, 2022 at 5:56 PM Tomas Vondra wrote: > > >> > >> I could think of below two options: > >> 1. Forbid any case where column list is different for the same table > >> when combining publications. > >> 2. Forbid if the column list and row filters for a table are different > >> in the set

Re: Fix typo in comment

2022-05-06 Thread Tom Lane
Alvaro Herrera writes: > On 2022-May-06, Zaorang Yang wrote: >> Maybe, the first letter of comments in postinit.c should be capitalized. > Hmm, typically these one-line comments are not "full sentences", so they > don't have capitals and no ending periods either. I wouldn't like the > endless st

Trying to add more tests to gistbuild.c

2022-05-06 Thread Matheus Alcantara
I'm studying how the gist index works trying to improve the test coverage of gistbuild.c. Reading the source code I noticed that the gistInitBuffering function is not covered, so I decided to start with it. Reading the documentation and the source I understood that for this function to be exec

Re: Fix typo in comment

2022-05-06 Thread Daniel Gustafsson
> On 6 May 2022, at 14:50, Alvaro Herrera wrote: > > On 2022-May-06, Zaorang Yang wrote: > >> Maybe, the first letter of comments in postinit.c should be capitalized. > > Hmm, typically these one-line comments are not "full sentences", so they > don't have capitals and no ending periods either.

Re: Fix typo in comment

2022-05-06 Thread Alvaro Herrera
On 2022-May-06, Zaorang Yang wrote: > Maybe, the first letter of comments in postinit.c should be capitalized. Hmm, typically these one-line comments are not "full sentences", so they don't have capitals and no ending periods either. I wouldn't like the endless stream of patches that would resul

Fix typo in comment

2022-05-06 Thread Zaorang Yang
Maybe, the first letter of comments in postinit.c should be capitalized. Attaching a tiny patch to fix it. 0001-Fix-typo-in-comment.patch Description: Binary data

Re: bogus: logical replication rows/cols combinations

2022-05-06 Thread Tomas Vondra
On 5/6/22 05:23, houzj.f...@fujitsu.com wrote: > On Tuesday, May 3, 2022 11:31 AM Amit Kapila wrote: >> >> On Tue, May 3, 2022 at 12:10 AM Tomas Vondra >> wrote: >>> >>> On 5/2/22 19:51, Alvaro Herrera wrote: > Why would we need to know publications replicated by other >> walsenders? >

Re: Fix typo in code comment - origin.c

2022-05-06 Thread Michael Paquier
On Fri, May 06, 2022 at 07:25:34PM +1000, Peter Smith wrote: > PSA trivial patch to fix some very old comment typo. Thanks, fixed. -- Michael signature.asc Description: PGP signature

Re: Possible corruption by CreateRestartPoint at promotion

2022-05-06 Thread Michael Paquier
On Thu, Apr 28, 2022 at 03:49:42PM +0900, Michael Paquier wrote: > I am not sure what you mean here. FWIW, I am translating the > suggestion of Nathan to split the existing check in > CreateRestartPoint() that we are discussing here into two if blocks, > instead of just one: > - Move the update of

Re: strange slow query - lost lot of time somewhere

2022-05-06 Thread Pavel Stehule
pá 6. 5. 2022 v 10:05 odesílatel David Rowley napsal: > On Fri, 6 May 2022 at 17:52, Pavel Stehule > wrote: > > Breakpoint 1, build_hash_table (size=4369066, mstate=0xfc7f08) at > nodeMemoize.c:268 > > 268 if (size == 0) > > (gdb) p size > > $1 = 4369066 > > Thanks for the report. I think I now

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-05-06 Thread Etsuro Fujita
On Thu, May 5, 2022 at 6:39 AM David Zhang wrote: > On 2022-05-02 1:25 a.m., Etsuro Fujita wrote: > > On Wed, Apr 20, 2022 at 4:55 AM David Zhang wrote: > >> I tried to apply the patch to master and plan to run some tests, but got > >> below errors due to other commits. > > I rebased the patch ag

Re: Configuration Parameter/GUC value validation hook

2022-05-06 Thread Robert Haas
On Wed, May 4, 2022 at 7:12 AM Bharath Rupireddy wrote: > Thanks Tom and Robert for your responses. > > How about we provide a sample extension (limiting some important > parameters say shared_buffers, work_mem and so on to some > "reasonable/recommended" limits) in the core along with the > set_c

Re: strange slow query - lost lot of time somewhere

2022-05-06 Thread David Rowley
On Fri, 6 May 2022 at 20:04, David Rowley wrote: > Thanks for the report. I think I now see the problem. Looking at > [1], it seems that's a bushy plan. That's fine, but less common than a > left deep plan. On second thoughts, it does not need to be a bushy plan for the outer side of the nested

Fix typo in code comment - origin.c

2022-05-06 Thread Peter Smith
PSA trivial patch to fix some very old comment typo. -- Kind Regards, Peter Smith. Fujitsu Australia v1-0001-Fix-typo-in-comment.patch Description: Binary data

Re: Perform streaming logical transactions by background workers and parallel apply

2022-05-06 Thread Peter Smith
On Fri, Apr 29, 2022 at 3:22 PM shiy.f...@fujitsu.com wrote: ... > Thanks for your patch. > > The patch modified streaming option in logical replication, it can be set to > 'on', 'off' and 'apply'. The new option 'apply' haven't been tested in the > tap test. > Attach a patch which modified the s

Re: strange slow query - lost lot of time somewhere

2022-05-06 Thread David Rowley
On Fri, 6 May 2022 at 17:52, Pavel Stehule wrote: > Breakpoint 1, build_hash_table (size=4369066, mstate=0xfc7f08) at > nodeMemoize.c:268 > 268 if (size == 0) > (gdb) p size > $1 = 4369066 Thanks for the report. I think I now see the problem. Looking at [1], it seems that's a bushy plan. That'

RE: Logical replication timeout problem

2022-05-06 Thread wangw.f...@fujitsu.com
On Fri, May 6, 2022 at 9:54 AM Masahiko Sawada wrote: > On Wed, May 4, 2022 at 7:18 PM Amit Kapila wrote: > > > > On Mon, May 2, 2022 at 8:07 AM Masahiko Sawada > wrote: > > > > > > On Mon, May 2, 2022 at 11:32 AM Amit Kapila > wrote: > > > > > > > > > > > > So, shall we go back to the previous