RE: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread houzj.f...@fujitsu.com
On Wednesday, January 11, 2023 1:25 AM Ted Yu wrote: > I was reading src/backend/replication/logical/applyparallelworker.c . > In `pa_allocate_worker`, when pa_launch_parallel_worker returns NULL, I think > the `ParallelApplyTxnHash` should be released. Thanks for reporting. ParallelApplyTxnHa

Re: [PATCH] Add overlaps geometric operators that ignore point overlaps

2023-01-10 Thread Kyotaro Horiguchi
Hello. At Sun, 1 Jan 2023 01:13:24 +0530, Ankit Kumar Pandey wrote in > This is patch for todo item: Add overlaps geometric operators that > ignore point overlaps > > Issue: > > SELECT circle '((0,0), 1)' && circle '((2,0),1) returns True > > Expectation: In above case, both figures touch ot

Re: Handle infinite recursion in logical replication setup

2023-01-10 Thread Amit Kapila
On Tue, Jan 10, 2023 at 11:24 PM Jonathan S. Katz wrote: > > On 1/10/23 10:17 AM, Amit Kapila wrote: > > On Tue, Jan 10, 2023 at 8:13 AM Jonathan S. Katz > > wrote: > > > One can use local or higher > > for reducing the latency for COMMIT when synchronous replication is > > used in the publisher

Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread Ted Yu
On Tue, Jan 10, 2023 at 6:13 PM houzj.f...@fujitsu.com < houzj.f...@fujitsu.com> wrote: > On Wednesday, January 11, 2023 1:25 AM Ted Yu wrote: > > > I was reading src/backend/replication/logical/applyparallelworker.c . > > In `pa_allocate_worker`, when pa_launch_parallel_worker returns NULL, I >

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 8:47 PM Tom Lane wrote: > [ squint ... ] Are you sure it's not a security *hazard*, though? I think you have to squint pretty hard to find a security hazard here. The effect of this GUC is to control the set of privileges that a CREATEROLE user automatically grants to the

Re: [PATCH] support tab-completion for single quote input with equal sign

2023-01-10 Thread Tom Lane
I wrote: > I've spent some effort previously on getting tab-completion to deal > sanely with single-quoted strings, but everything I've tried has > crashed and burned :-(, mainly because it's not clear when to take > the whole literal as one "word" and when not. After a little further thought, a n

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-10 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 10, 2023 at 8:47 PM Tom Lane wrote: >> [ squint ... ] Are you sure it's not a security *hazard*, though? > I think you have to squint pretty hard to find a security hazard here. Maybe, but I'd be sad if somebody manages to find one after this is out in the wil

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-10 Thread Richard Guo
On Tue, Jan 10, 2023 at 6:12 PM Dean Rasheed wrote: > While doing some random testing, I noticed that the following is broken in > HEAD: > > SELECT COUNT(DISTINCT random()) FROM generate_series(1,10); > > ERROR: ORDER/GROUP BY expression not found in targetlist > > It appears to have been broken

Re: Allow tailoring of ICU locales with custom rules

2023-01-10 Thread vignesh C
On Thu, 5 Jan 2023 at 20:45, Peter Eisentraut wrote: > > Patch needed a rebase; no functionality changes. The patch does not apply on top of HEAD as in [1], please post a rebased patch: === Applying patches on top of PostgreSQL commit ID d952373a987bad331c0e499463159dd142ced1ef === === applying

Re: Add SHELL_EXIT_CODE to psql

2023-01-10 Thread vignesh C
On Tue, 10 Jan 2023 at 00:06, Corey Huinker wrote: > > On Mon, Jan 9, 2023 at 10:01 AM Maxim Orlov wrote: >> >> Hi! >> >> In overall, I think we move in the right direction. But we could make code >> better, should we? >> >> + /* Capture exit code for SHELL_EXIT_CODE */ >> +

Re: Using WaitEventSet in the postmaster

2023-01-10 Thread Thomas Munro
On Sun, Jan 8, 2023 at 11:55 AM Andres Freund wrote: > On 2023-01-07 18:08:11 +1300, Thomas Munro wrote: > > On Sat, Jan 7, 2023 at 12:25 PM Andres Freund wrote: > > > On 2023-01-07 11:08:36 +1300, Thomas Munro wrote: > > > > 3. Is it OK to clobber the shared pending flag for SIGQUIT, SIGTERM, >

Re: mprove tab completion for ALTER EXTENSION ADD/DROP

2023-01-10 Thread Kyotaro Horiguchi
At Mon, 2 Jan 2023 13:19:50 +0530, vignesh C wrote in > On Mon, 5 Dec 2022 at 06:53, Michael Paquier wrote: > > > > The DROP could be matched with the objects that are actually part of > > the so-said extension? > > The modified v2 version has the changes to handle the same. Sorry for > the del

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 9:40 PM Tom Lane wrote: > Yeah. I concur that a SUSET GUC isn't much fun for a non-superuser > CREATEROLE holder who might wish to adjust the default behavior they get. > I also concur that it seems a bit far-fetched that a CREATEROLE holder > might create a SECURITY DEFIN

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-10 Thread John Naylor
On Tue, Jan 10, 2023 at 7:08 PM Masahiko Sawada wrote: > It looks no problem in terms of vacuum integration, although I've not > fully tested yet. TID store uses the radix tree as the main storage, > and with the template radix tree, the data types for shared and > non-shared will be different. T

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-10 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 10, 2023 at 9:40 PM Tom Lane wrote: >> The scenario I'm worried about could be closed, mostly, if we were willing >> to invent an intermediate GUC privilege level "can be set interactively >> but only by CREATEROLE holders" ("PGC_CRSET"?). > Of course, if it's p

Re: ATTACH PARTITION seems to ignore column generation status

2023-01-10 Thread Amit Langote
On Wed, Jan 11, 2023 at 7:13 AM Tom Lane wrote: > I wrote: > > Amit Langote writes: > >> Thanks for the patch. It looks good, though I guess you said that we > >> should also change the error message that CREATE TABLE ... PARTITION > >> OF emits to match the other cases while we're here. I've a

Re: [PATCH] Simple code cleanup in tuplesort.c.

2023-01-10 Thread John Naylor
On Mon, Jan 9, 2023 at 7:29 PM Xing Guo wrote: > > Thank you John. This is my first patch, I'll keep it in mind that > adding a version number next time I sending the patch. Welcome to the community! You may also consider reviewing a patch from the current commitfest, since we can always use addi

Re: Can we let extensions change their dumped catalog schemas?

2023-01-10 Thread Tom Lane
Jacob Champion writes: > We'd like to be allowed to change the schema for a table that's been > marked in the past with pg_extension_config_dump(). > Unless I'm missing something obvious (please, let it be that) there's no > way to do this safely. Once you've marked an internal table as dumpable,

RE: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread houzj.f...@fujitsu.com
On Wednesday, January 11, 2023 10:21 AM Ted Yu wrote: > /* First time through, initialize parallel apply worker state > hashtable. */ > if (!ParallelApplyTxnHash) > > I think it would be better if `ParallelApplyTxnHash` is created by the first > successful parallel apply worker.

Re: [PATCH] Improve ability to display optimizer analysis using OPTIMIZER_DEBUG

2023-01-10 Thread John Naylor
On Tue, Jan 3, 2023 at 1:59 PM Ankit Kumar Pandey wrote: > > > On 03/01/23 08:38, David Rowley wrote: > > > > Do you actually have a need for this or are you just trying to tick > > off some TODO items? > > > I would say Iatter but reason I picked it up was more on side of > learning optimizer bet

Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread Ted Yu
On Tue, Jan 10, 2023 at 7:55 PM houzj.f...@fujitsu.com < houzj.f...@fujitsu.com> wrote: > On Wednesday, January 11, 2023 10:21 AM Ted Yu > wrote: > > /* First time through, initialize parallel apply worker state > hashtable. */ > > if (!ParallelApplyTxnHash) > > > > I think it wou

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-10 Thread houzj.f...@fujitsu.com
On Tuesday, January 10, 2023 7:48 PM Dilip Kumar wrote: > > On Tue, Jan 10, 2023 at 10:26 AM houzj.f...@fujitsu.com > wrote: > > > > On Monday, January 9, 2023 4:51 PM Amit Kapila > wrote: > > > > > > On Sun, Jan 8, 2023 at 11:32 AM houzj.f...@fujitsu.com > > > wrote: > > > > > > > > On Sunday

RE: [PATCH] Support % wildcard in extension upgrade filenames

2023-01-10 Thread Regina Obe
> Sandro Santilli writes: > > On Mon, Jan 09, 2023 at 05:51:49PM -0500, Tom Lane wrote: > >> ... you still need one script file for each supported upgrade step > > > That's exactly the problem we're trying to solve here. > > The include support is nice on itself, but won't solve our problem. > >

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-10 Thread David Rowley
On Wed, 11 Jan 2023 at 15:46, Richard Guo wrote: > However the scan/join plan's > tlist does not contain random(), which I think we need to fix. I was wondering if that's true and considered that we don't want to evaluate random() for the sort then again when doing the aggregate transitions, but

Re: Strengthen pg_waldump's --save-fullpage tests

2023-01-10 Thread Bharath Rupireddy
On Wed, Jan 11, 2023 at 6:32 AM Michael Paquier wrote: > > On Tue, Jan 10, 2023 at 05:25:44PM +0100, Drouvot, Bertrand wrote: > > I like the idea of comparing the full page (and not just the LSN) but > > I'm not sure that adding the pageinspect dependency is a good thing. > > > > What about extrac

Re: delay starting WAL receiver

2023-01-10 Thread Thomas Munro
On Wed, Jan 11, 2023 at 2:08 PM Nathan Bossart wrote: > I discussed this a bit in a different thread [0], but I thought it deserved > its own thread. > > After setting wal_retrieve_retry_interval to 1ms in the tests, I noticed > that the recovery tests consistently take much longer. Upon further

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-10 Thread Amit Kapila
On Wed, Jan 11, 2023 at 9:34 AM houzj.f...@fujitsu.com wrote: > > On Tuesday, January 10, 2023 7:48 PM Dilip Kumar > wrote: > > > > I was looking into 0001, IMHO the pid should continue to represent the main > > apply worker. So the pid will always show the main apply worker which is > > actuall

Re: Fix pg_publication_tables to exclude generated columns

2023-01-10 Thread Amit Kapila
On Tue, Jan 10, 2023 at 8:38 AM shiy.f...@fujitsu.com wrote: > > On Mon, Jan 9, 2023 11:06 PM Tom Lane wrote: > > > > Amit Kapila writes: > > > On Mon, Jan 9, 2023 at 5:29 PM shiy.f...@fujitsu.com > > > wrote: > > >> I think one way to fix it is to modify pg_publication_tables query to > > >>

Re: delay starting WAL receiver

2023-01-10 Thread Thomas Munro
On Wed, Jan 11, 2023 at 5:20 PM Thomas Munro wrote: > (I don't know why you didn't make it 0) (Oh, I see why it had to be non-zero to avoiding burning CPU, ignore that part.)

Re: [PATCH] Improve ability to display optimizer analysis using OPTIMIZER_DEBUG

2023-01-10 Thread Tom Lane
John Naylor writes: > Note that the TODO list has accumulated some cruft over the years. Some > time ago I started an effort to remove outdated/undesirable entries, and I > should get back to that, but for the present, please take the warning at > the top to heart: > "WARNING for Developers: Unfo

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-10 Thread Tom Lane
David Rowley writes: > I think whatever the fix is here, we should likely ensure that the > results are consistent regardless of which Aggrefs are the presorted > ones. Perhaps the easiest way to do that, and to ensure we call the > volatile functions are called the same number of times would jus

Re: Fix pg_publication_tables to exclude generated columns

2023-01-10 Thread Tom Lane
Amit Kapila writes: >> On Mon, Jan 9, 2023 11:06 PM Tom Lane wrote: >>> We could just not fix it in the back branches. I'd argue that this is >>> as much a definition change as a bug fix, so it doesn't really feel >>> like something to back-patch anyway. > So, if we don't backpatch then it coul

Re: Add a new pg_walinspect function to extract FPIs from WAL records

2023-01-10 Thread Michael Paquier
On Tue, Jan 10, 2023 at 09:29:03AM +0100, Drouvot, Bertrand wrote: > Thanks for updating the patch! > > +-- Compare FPI from WAL record and page from table, they must be same > > I think "must be the same" or "must be identical" sounds better (but not 100% > sure). > > Except this nit, V4 looks

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-10 Thread David Rowley
On Wed, 11 Jan 2023 at 17:32, Tom Lane wrote: > > David Rowley writes: > > I think whatever the fix is here, we should likely ensure that the > > results are consistent regardless of which Aggrefs are the presorted > > ones. Perhaps the easiest way to do that, and to ensure we call the > > volat

Re: [PATCH] Support using "all" for the db user in pg_ident.conf

2023-01-10 Thread Michael Paquier
On Wed, Dec 28, 2022 at 09:11:05AM +, Jelte Fennema wrote: > That's totally fair, I attached a new iteration of this patchset where this > refactoring and the new functionality are split up in two patches. The confusion that 0001 is addressing is fair (cough, fc579e1, cough), still I am wonder

Re: Spinlock is missing when updating two_phase of ReplicationSlot

2023-01-10 Thread Michael Paquier
On Wed, Jan 11, 2023 at 11:07:05AM +0900, Masahiko Sawada wrote: > I think we should acquire the spinlock when updating fields of the > replication slot even by its owner. Otherwise readers could see > inconsistent results. Looking at another place where we update > two_phase_at, we acquire the spi

Re: delay starting WAL receiver

2023-01-10 Thread Nathan Bossart
On Wed, Jan 11, 2023 at 05:20:38PM +1300, Thomas Munro wrote: > Is the problem here that SIGCHLD is processed ... > > PG_SETMASK(&UnBlockSig); <--- here? > > selres = select(nSockets, &rmask, NULL, NULL, &timeout); > > Meanwhile the SIGCHLD handler code says: > >

Re: MultiXact\SLRU buffers configuration

2023-01-10 Thread Dilip Kumar
On Mon, Jan 9, 2023 at 9:49 AM Andrey Borodin wrote: > > On Tue, Jan 3, 2023 at 5:02 AM vignesh C wrote: > > does not apply on top of HEAD as in [1], please post a rebased patch: > > > Thanks! Here's the rebase. I was looking into this patch, it seems like three different optimizations are squee

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-10 Thread Dilip Kumar
On Wed, Jan 11, 2023 at 9:34 AM houzj.f...@fujitsu.com wrote: > > > I was looking into 0001, IMHO the pid should continue to represent the main > > apply worker. So the pid will always show the main apply worker which is > > actually receiving all the changes for the subscription (in short workin

low wal_retrieve_retry_interval causes missed signals on Windows

2023-01-10 Thread Nathan Bossart
I discussed this elsewhere [0], but I thought it deserved its own thread. After setting wal_retrieve_retry_interval to 1ms in the tests, I noticed that some of the archiving tests began consistently failing on Windows. I believe the problem is that WaitForWALToBecomeAvailable() depends on the cal

Re: [DOCS] Stats views and functions not in order?

2023-01-10 Thread Peter Smith
On Wed, Jan 4, 2023 at 6:08 PM vignesh C wrote: > > On Mon, 2 Jan 2023 at 13:47, Peter Eisentraut > wrote: > > > > On 08.12.22 03:30, Peter Smith wrote: > > > PSA patches for v9* > > > > > > v9-0001 - Now the table rows are ordered per PeterE's suggestions [1] > > > > committed Thanks for pushin

Re: split TOAST support out of postgres.h

2023-01-10 Thread Noah Misch
On Tue, Jan 10, 2023 at 12:00:46PM -0500, Robert Haas wrote: > On Tue, Jan 10, 2023 at 9:46 AM Tom Lane wrote: > > Now, there is a fair question whether splitting this code out of > > postgres.h is worth any trouble at all. TBH my initial reaction > > had been "no". But once we found that only 4

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-10 Thread Ankit Kumar Pandey
On 11/01/23 06:18, David Rowley wrote: Not sure if we should be trying to improve that in this patch. I just wanted to identify it as something else that perhaps could be done. This could be within reach but still original problem of having hashagg removing any gains from this remains.

Re: typos

2023-01-10 Thread Michael Paquier
On Tue, Jan 10, 2023 at 01:55:56PM +0530, Amit Kapila wrote: > I have not yet started, so please go ahead. Okay, I have looked at that and fixed the whole new things, including the typo you have introduced. 0001~0004 have been left out, as of the same reasons as upthread. -- Michael signature.a

Re: Avoiding "wrong tuple length" errors at the end of VACUUM on pg_database update (Backpatch of 947789f to v12 and v13)

2023-01-10 Thread Michael Paquier
On Tue, Jan 10, 2023 at 09:54:31AM -0800, Nathan Bossart wrote: > +1 Okay, thanks. Done this part as of c0ee694 and 72b6098, then. -- Michael signature.asc Description: PGP signature

Re: [PATCH] Improve ability to display optimizer analysis using OPTIMIZER_DEBUG

2023-01-10 Thread Ankit Kumar Pandey
On 11/01/23 09:57, Tom Lane wrote: IME it's typically a lot more productive to approach things via "scratch your own itch". If a problem is biting you directly, then at least you have some clear idea of what it is that needs to be fixed. You might have to work up to an understanding of how to

Re: Fix pg_publication_tables to exclude generated columns

2023-01-10 Thread Amit Kapila
On Wed, Jan 11, 2023 at 10:07 AM Tom Lane wrote: > > Amit Kapila writes: > >> On Mon, Jan 9, 2023 11:06 PM Tom Lane wrote: > >>> We could just not fix it in the back branches. I'd argue that this is > >>> as much a definition change as a bug fix, so it doesn't really feel > >>> like something t

Re: Avoiding "wrong tuple length" errors at the end of VACUUM on pg_database update (Backpatch of 947789f to v12 and v13)

2023-01-10 Thread Michael Paquier
On Tue, Jan 10, 2023 at 11:05:04AM -0800, Andres Freund wrote: > What about a define that forces external toasting very aggressively for > catalog tables, iff they have a toast table? I suspect doing so for > non-catalog tables as well would trigger test changes. Running a buildfarm > animal with t

Re: mprove tab completion for ALTER EXTENSION ADD/DROP

2023-01-10 Thread Michael Paquier
On Wed, Jan 11, 2023 at 12:10:33PM +0900, Kyotaro Horiguchi wrote: > It suggests the *kinds* of objects that are part of the extension, but > lists the objects of that kind regardless of dependency. I read > Michael suggested (and I agree) to restrict the objects (not kinds) to > actually be a par

Re: SQL/JSON revisited

2023-01-10 Thread Elena Indrupskaya
Tags in the patch follow the markup of the XMLTABLE function: XMLTABLE (     XMLNAMESPACES ( namespace_uri AS namespace_name , ... ),     row_expression PASSING BY {REF|VALUE} document_expression BY {REF|VALUE}     COLUMNS name { type PATH column_expression DEFAULT default_expressio

Re: cutting down the TODO list thread

2023-01-10 Thread John Naylor
So, I had intended to spend some time on this at least three times a year. I've clearly failed at that, but now is as good a time as any to pick it back up again. Over in [1], Tom opined: > John Naylor writes: > > > "WARNING for Developers: Unfortunately this list does not contain all the > > in

Re: SQL/JSON revisited

2023-01-10 Thread John Naylor
On Wed, Jan 11, 2023 at 2:02 PM Elena Indrupskaya < e.indrupsk...@postgrespro.ru> wrote: > > Sorry for upsetting your bot. :( What I do in these cases is save the incremental patch as a .txt file -- that way people can read it, but the cf bot doesn't try to launch a CI run. And if I forget that de

Re: Minimal logical decoding on standbys

2023-01-10 Thread Bharath Rupireddy
On Tue, Jan 10, 2023 at 2:03 PM Drouvot, Bertrand wrote: > > Please find attached, V37 taking care of: Thanks. I started to digest the design specified in the commit message and these patches. Here are some quick comments: 1. Does logical decoding on standby work without any issues if the standb

RE: [Proposal] Add foreign-server health checks infrastructure

2023-01-10 Thread Hayato Kuroda (Fujitsu)
Dear Ted, Thank you for reviewing! PSA new version. > + /* quick exit if connection cache has been not initialized yet. */ > > been not initialized -> not been initialized Fixed. > + > (errcode(ERRCODE_CONNECTION_FAILURE), > +

Re: recovery modules

2023-01-10 Thread Michael Paquier
On Tue, Jan 03, 2023 at 11:05:38AM -0800, Nathan Bossart wrote: > I noticed that cfbot's Windows tests are failing because the backslashes in > the archive directory path are causing escaping problems. Here is an > attempt to fix that by converting all backslashes to forward slashes, which > is wh

<    1   2