Re: recovery modules

2023-01-10 Thread Michael Paquier
On Tue, Jan 03, 2023 at 11:05:38AM -0800, Nathan Bossart wrote: > I noticed that cfbot's Windows tests are failing because the backslashes in > the archive directory path are causing escaping problems. Here is an > attempt to fix that by converting all backslashes to forward slashes, which > is

RE: [Proposal] Add foreign-server health checks infrastructure

2023-01-10 Thread Hayato Kuroda (Fujitsu)
Dear Ted, Thank you for reviewing! PSA new version. > + /* quick exit if connection cache has been not initialized yet. */ > > been not initialized -> not been initialized Fixed. > + > (errcode(ERRCODE_CONNECTION_FAILURE), > +

Re: Minimal logical decoding on standbys

2023-01-10 Thread Bharath Rupireddy
On Tue, Jan 10, 2023 at 2:03 PM Drouvot, Bertrand wrote: > > Please find attached, V37 taking care of: Thanks. I started to digest the design specified in the commit message and these patches. Here are some quick comments: 1. Does logical decoding on standby work without any issues if the

Re: SQL/JSON revisited

2023-01-10 Thread John Naylor
On Wed, Jan 11, 2023 at 2:02 PM Elena Indrupskaya < e.indrupsk...@postgrespro.ru> wrote: > > Sorry for upsetting your bot. :( What I do in these cases is save the incremental patch as a .txt file -- that way people can read it, but the cf bot doesn't try to launch a CI run. And if I forget that

Re: cutting down the TODO list thread

2023-01-10 Thread John Naylor
So, I had intended to spend some time on this at least three times a year. I've clearly failed at that, but now is as good a time as any to pick it back up again. Over in [1], Tom opined: > John Naylor writes: > > > "WARNING for Developers: Unfortunately this list does not contain all the > >

Re: SQL/JSON revisited

2023-01-10 Thread Elena Indrupskaya
Tags in the patch follow the markup of the XMLTABLE function: XMLTABLE (     XMLNAMESPACES ( namespace_uri AS namespace_name , ... ),     row_expression PASSING BY {REF|VALUE} document_expression BY {REF|VALUE}     COLUMNS name { type PATH column_expression DEFAULT

Re: mprove tab completion for ALTER EXTENSION ADD/DROP

2023-01-10 Thread Michael Paquier
On Wed, Jan 11, 2023 at 12:10:33PM +0900, Kyotaro Horiguchi wrote: > It suggests the *kinds* of objects that are part of the extension, but > lists the objects of that kind regardless of dependency. I read > Michael suggested (and I agree) to restrict the objects (not kinds) to > actually be a

Re: Avoiding "wrong tuple length" errors at the end of VACUUM on pg_database update (Backpatch of 947789f to v12 and v13)

2023-01-10 Thread Michael Paquier
On Tue, Jan 10, 2023 at 11:05:04AM -0800, Andres Freund wrote: > What about a define that forces external toasting very aggressively for > catalog tables, iff they have a toast table? I suspect doing so for > non-catalog tables as well would trigger test changes. Running a buildfarm > animal with

Re: Fix pg_publication_tables to exclude generated columns

2023-01-10 Thread Amit Kapila
On Wed, Jan 11, 2023 at 10:07 AM Tom Lane wrote: > > Amit Kapila writes: > >> On Mon, Jan 9, 2023 11:06 PM Tom Lane wrote: > >>> We could just not fix it in the back branches. I'd argue that this is > >>> as much a definition change as a bug fix, so it doesn't really feel > >>> like something

Re: [PATCH] Improve ability to display optimizer analysis using OPTIMIZER_DEBUG

2023-01-10 Thread Ankit Kumar Pandey
On 11/01/23 09:57, Tom Lane wrote: IME it's typically a lot more productive to approach things via "scratch your own itch". If a problem is biting you directly, then at least you have some clear idea of what it is that needs to be fixed. You might have to work up to an understanding of how to

Re: Avoiding "wrong tuple length" errors at the end of VACUUM on pg_database update (Backpatch of 947789f to v12 and v13)

2023-01-10 Thread Michael Paquier
On Tue, Jan 10, 2023 at 09:54:31AM -0800, Nathan Bossart wrote: > +1 Okay, thanks. Done this part as of c0ee694 and 72b6098, then. -- Michael signature.asc Description: PGP signature

Re: typos

2023-01-10 Thread Michael Paquier
On Tue, Jan 10, 2023 at 01:55:56PM +0530, Amit Kapila wrote: > I have not yet started, so please go ahead. Okay, I have looked at that and fixed the whole new things, including the typo you have introduced. 0001~0004 have been left out, as of the same reasons as upthread. -- Michael

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-10 Thread Ankit Kumar Pandey
On 11/01/23 06:18, David Rowley wrote: Not sure if we should be trying to improve that in this patch. I just wanted to identify it as something else that perhaps could be done. This could be within reach but still original problem of having hashagg removing any gains from this remains.

Re: split TOAST support out of postgres.h

2023-01-10 Thread Noah Misch
On Tue, Jan 10, 2023 at 12:00:46PM -0500, Robert Haas wrote: > On Tue, Jan 10, 2023 at 9:46 AM Tom Lane wrote: > > Now, there is a fair question whether splitting this code out of > > postgres.h is worth any trouble at all. TBH my initial reaction > > had been "no". But once we found that only

Re: [DOCS] Stats views and functions not in order?

2023-01-10 Thread Peter Smith
On Wed, Jan 4, 2023 at 6:08 PM vignesh C wrote: > > On Mon, 2 Jan 2023 at 13:47, Peter Eisentraut > wrote: > > > > On 08.12.22 03:30, Peter Smith wrote: > > > PSA patches for v9* > > > > > > v9-0001 - Now the table rows are ordered per PeterE's suggestions [1] > > > > committed Thanks for

low wal_retrieve_retry_interval causes missed signals on Windows

2023-01-10 Thread Nathan Bossart
I discussed this elsewhere [0], but I thought it deserved its own thread. After setting wal_retrieve_retry_interval to 1ms in the tests, I noticed that some of the archiving tests began consistently failing on Windows. I believe the problem is that WaitForWALToBecomeAvailable() depends on the

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-10 Thread Dilip Kumar
On Wed, Jan 11, 2023 at 9:34 AM houzj.f...@fujitsu.com wrote: > > > I was looking into 0001, IMHO the pid should continue to represent the main > > apply worker. So the pid will always show the main apply worker which is > > actually receiving all the changes for the subscription (in short

Re: MultiXact\SLRU buffers configuration

2023-01-10 Thread Dilip Kumar
On Mon, Jan 9, 2023 at 9:49 AM Andrey Borodin wrote: > > On Tue, Jan 3, 2023 at 5:02 AM vignesh C wrote: > > does not apply on top of HEAD as in [1], please post a rebased patch: > > > Thanks! Here's the rebase. I was looking into this patch, it seems like three different optimizations are

Re: delay starting WAL receiver

2023-01-10 Thread Nathan Bossart
On Wed, Jan 11, 2023 at 05:20:38PM +1300, Thomas Munro wrote: > Is the problem here that SIGCHLD is processed ... > > PG_SETMASK(); <--- here? > > selres = select(nSockets, , NULL, NULL, ); > > Meanwhile the SIGCHLD handler code says: > > * Was it the wal

Re: Spinlock is missing when updating two_phase of ReplicationSlot

2023-01-10 Thread Michael Paquier
On Wed, Jan 11, 2023 at 11:07:05AM +0900, Masahiko Sawada wrote: > I think we should acquire the spinlock when updating fields of the > replication slot even by its owner. Otherwise readers could see > inconsistent results. Looking at another place where we update > two_phase_at, we acquire the

Re: [PATCH] Support using "all" for the db user in pg_ident.conf

2023-01-10 Thread Michael Paquier
On Wed, Dec 28, 2022 at 09:11:05AM +, Jelte Fennema wrote: > That's totally fair, I attached a new iteration of this patchset where this > refactoring and the new functionality are split up in two patches. The confusion that 0001 is addressing is fair (cough, fc579e1, cough), still I am

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-10 Thread David Rowley
On Wed, 11 Jan 2023 at 17:32, Tom Lane wrote: > > David Rowley writes: > > I think whatever the fix is here, we should likely ensure that the > > results are consistent regardless of which Aggrefs are the presorted > > ones. Perhaps the easiest way to do that, and to ensure we call the > >

Re: Add a new pg_walinspect function to extract FPIs from WAL records

2023-01-10 Thread Michael Paquier
On Tue, Jan 10, 2023 at 09:29:03AM +0100, Drouvot, Bertrand wrote: > Thanks for updating the patch! > > +-- Compare FPI from WAL record and page from table, they must be same > > I think "must be the same" or "must be identical" sounds better (but not 100% > sure). > > Except this nit, V4

Re: Fix pg_publication_tables to exclude generated columns

2023-01-10 Thread Tom Lane
Amit Kapila writes: >> On Mon, Jan 9, 2023 11:06 PM Tom Lane wrote: >>> We could just not fix it in the back branches. I'd argue that this is >>> as much a definition change as a bug fix, so it doesn't really feel >>> like something to back-patch anyway. > So, if we don't backpatch then it

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-10 Thread Tom Lane
David Rowley writes: > I think whatever the fix is here, we should likely ensure that the > results are consistent regardless of which Aggrefs are the presorted > ones. Perhaps the easiest way to do that, and to ensure we call the > volatile functions are called the same number of times would

Re: [PATCH] Improve ability to display optimizer analysis using OPTIMIZER_DEBUG

2023-01-10 Thread Tom Lane
John Naylor writes: > Note that the TODO list has accumulated some cruft over the years. Some > time ago I started an effort to remove outdated/undesirable entries, and I > should get back to that, but for the present, please take the warning at > the top to heart: > "WARNING for Developers:

Re: delay starting WAL receiver

2023-01-10 Thread Thomas Munro
On Wed, Jan 11, 2023 at 5:20 PM Thomas Munro wrote: > (I don't know why you didn't make it 0) (Oh, I see why it had to be non-zero to avoiding burning CPU, ignore that part.)

Re: Fix pg_publication_tables to exclude generated columns

2023-01-10 Thread Amit Kapila
On Tue, Jan 10, 2023 at 8:38 AM shiy.f...@fujitsu.com wrote: > > On Mon, Jan 9, 2023 11:06 PM Tom Lane wrote: > > > > Amit Kapila writes: > > > On Mon, Jan 9, 2023 at 5:29 PM shiy.f...@fujitsu.com > > > wrote: > > >> I think one way to fix it is to modify pg_publication_tables query to > > >>

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-10 Thread Amit Kapila
On Wed, Jan 11, 2023 at 9:34 AM houzj.f...@fujitsu.com wrote: > > On Tuesday, January 10, 2023 7:48 PM Dilip Kumar > wrote: > > > > I was looking into 0001, IMHO the pid should continue to represent the main > > apply worker. So the pid will always show the main apply worker which is > >

Re: delay starting WAL receiver

2023-01-10 Thread Thomas Munro
On Wed, Jan 11, 2023 at 2:08 PM Nathan Bossart wrote: > I discussed this a bit in a different thread [0], but I thought it deserved > its own thread. > > After setting wal_retrieve_retry_interval to 1ms in the tests, I noticed > that the recovery tests consistently take much longer. Upon further

Re: Strengthen pg_waldump's --save-fullpage tests

2023-01-10 Thread Bharath Rupireddy
On Wed, Jan 11, 2023 at 6:32 AM Michael Paquier wrote: > > On Tue, Jan 10, 2023 at 05:25:44PM +0100, Drouvot, Bertrand wrote: > > I like the idea of comparing the full page (and not just the LSN) but > > I'm not sure that adding the pageinspect dependency is a good thing. > > > > What about

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-10 Thread David Rowley
On Wed, 11 Jan 2023 at 15:46, Richard Guo wrote: > However the scan/join plan's > tlist does not contain random(), which I think we need to fix. I was wondering if that's true and considered that we don't want to evaluate random() for the sort then again when doing the aggregate transitions, but

RE: [PATCH] Support % wildcard in extension upgrade filenames

2023-01-10 Thread Regina Obe
> Sandro Santilli writes: > > On Mon, Jan 09, 2023 at 05:51:49PM -0500, Tom Lane wrote: > >> ... you still need one script file for each supported upgrade step > > > That's exactly the problem we're trying to solve here. > > The include support is nice on itself, but won't solve our problem. >

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-10 Thread houzj.f...@fujitsu.com
On Tuesday, January 10, 2023 7:48 PM Dilip Kumar wrote: > > On Tue, Jan 10, 2023 at 10:26 AM houzj.f...@fujitsu.com > wrote: > > > > On Monday, January 9, 2023 4:51 PM Amit Kapila > wrote: > > > > > > On Sun, Jan 8, 2023 at 11:32 AM houzj.f...@fujitsu.com > > > wrote: > > > > > > > > On

Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread Ted Yu
On Tue, Jan 10, 2023 at 7:55 PM houzj.f...@fujitsu.com < houzj.f...@fujitsu.com> wrote: > On Wednesday, January 11, 2023 10:21 AM Ted Yu > wrote: > > /* First time through, initialize parallel apply worker state > hashtable. */ > > if (!ParallelApplyTxnHash) > > > > I think it

Re: [PATCH] Improve ability to display optimizer analysis using OPTIMIZER_DEBUG

2023-01-10 Thread John Naylor
On Tue, Jan 3, 2023 at 1:59 PM Ankit Kumar Pandey wrote: > > > On 03/01/23 08:38, David Rowley wrote: > > > > Do you actually have a need for this or are you just trying to tick > > off some TODO items? > > > I would say Iatter but reason I picked it up was more on side of > learning optimizer

RE: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread houzj.f...@fujitsu.com
On Wednesday, January 11, 2023 10:21 AM Ted Yu wrote: > /* First time through, initialize parallel apply worker state > hashtable. */ > if (!ParallelApplyTxnHash) > > I think it would be better if `ParallelApplyTxnHash` is created by the first > successful parallel apply worker.

Re: Can we let extensions change their dumped catalog schemas?

2023-01-10 Thread Tom Lane
Jacob Champion writes: > We'd like to be allowed to change the schema for a table that's been > marked in the past with pg_extension_config_dump(). > Unless I'm missing something obvious (please, let it be that) there's no > way to do this safely. Once you've marked an internal table as

Re: [PATCH] Simple code cleanup in tuplesort.c.

2023-01-10 Thread John Naylor
On Mon, Jan 9, 2023 at 7:29 PM Xing Guo wrote: > > Thank you John. This is my first patch, I'll keep it in mind that > adding a version number next time I sending the patch. Welcome to the community! You may also consider reviewing a patch from the current commitfest, since we can always use

Re: ATTACH PARTITION seems to ignore column generation status

2023-01-10 Thread Amit Langote
On Wed, Jan 11, 2023 at 7:13 AM Tom Lane wrote: > I wrote: > > Amit Langote writes: > >> Thanks for the patch. It looks good, though I guess you said that we > >> should also change the error message that CREATE TABLE ... PARTITION > >> OF emits to match the other cases while we're here. I've

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-10 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 10, 2023 at 9:40 PM Tom Lane wrote: >> The scenario I'm worried about could be closed, mostly, if we were willing >> to invent an intermediate GUC privilege level "can be set interactively >> but only by CREATEROLE holders" ("PGC_CRSET"?). > Of course, if it's

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-10 Thread John Naylor
On Tue, Jan 10, 2023 at 7:08 PM Masahiko Sawada wrote: > It looks no problem in terms of vacuum integration, although I've not > fully tested yet. TID store uses the radix tree as the main storage, > and with the template radix tree, the data types for shared and > non-shared will be different.

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 9:40 PM Tom Lane wrote: > Yeah. I concur that a SUSET GUC isn't much fun for a non-superuser > CREATEROLE holder who might wish to adjust the default behavior they get. > I also concur that it seems a bit far-fetched that a CREATEROLE holder > might create a SECURITY

Re: mprove tab completion for ALTER EXTENSION ADD/DROP

2023-01-10 Thread Kyotaro Horiguchi
At Mon, 2 Jan 2023 13:19:50 +0530, vignesh C wrote in > On Mon, 5 Dec 2022 at 06:53, Michael Paquier wrote: > > > > The DROP could be matched with the objects that are actually part of > > the so-said extension? > > The modified v2 version has the changes to handle the same. Sorry for > the

Re: Using WaitEventSet in the postmaster

2023-01-10 Thread Thomas Munro
On Sun, Jan 8, 2023 at 11:55 AM Andres Freund wrote: > On 2023-01-07 18:08:11 +1300, Thomas Munro wrote: > > On Sat, Jan 7, 2023 at 12:25 PM Andres Freund wrote: > > > On 2023-01-07 11:08:36 +1300, Thomas Munro wrote: > > > > 3. Is it OK to clobber the shared pending flag for SIGQUIT, SIGTERM,

Re: Add SHELL_EXIT_CODE to psql

2023-01-10 Thread vignesh C
On Tue, 10 Jan 2023 at 00:06, Corey Huinker wrote: > > On Mon, Jan 9, 2023 at 10:01 AM Maxim Orlov wrote: >> >> Hi! >> >> In overall, I think we move in the right direction. But we could make code >> better, should we? >> >> + /* Capture exit code for SHELL_EXIT_CODE */ >> +

Re: Allow tailoring of ICU locales with custom rules

2023-01-10 Thread vignesh C
On Thu, 5 Jan 2023 at 20:45, Peter Eisentraut wrote: > > Patch needed a rebase; no functionality changes. The patch does not apply on top of HEAD as in [1], please post a rebased patch: === Applying patches on top of PostgreSQL commit ID d952373a987bad331c0e499463159dd142ced1ef === === applying

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-10 Thread Richard Guo
On Tue, Jan 10, 2023 at 6:12 PM Dean Rasheed wrote: > While doing some random testing, I noticed that the following is broken in > HEAD: > > SELECT COUNT(DISTINCT random()) FROM generate_series(1,10); > > ERROR: ORDER/GROUP BY expression not found in targetlist > > It appears to have been

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-10 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 10, 2023 at 8:47 PM Tom Lane wrote: >> [ squint ... ] Are you sure it's not a security *hazard*, though? > I think you have to squint pretty hard to find a security hazard here. Maybe, but I'd be sad if somebody manages to find one after this is out in the

Re: [PATCH] support tab-completion for single quote input with equal sign

2023-01-10 Thread Tom Lane
I wrote: > I've spent some effort previously on getting tab-completion to deal > sanely with single-quoted strings, but everything I've tried has > crashed and burned :-(, mainly because it's not clear when to take > the whole literal as one "word" and when not. After a little further thought, a

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 8:47 PM Tom Lane wrote: > [ squint ... ] Are you sure it's not a security *hazard*, though? I think you have to squint pretty hard to find a security hazard here. The effect of this GUC is to control the set of privileges that a CREATEROLE user automatically grants to

Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread Ted Yu
On Tue, Jan 10, 2023 at 6:13 PM houzj.f...@fujitsu.com < houzj.f...@fujitsu.com> wrote: > On Wednesday, January 11, 2023 1:25 AM Ted Yu wrote: > > > I was reading src/backend/replication/logical/applyparallelworker.c . > > In `pa_allocate_worker`, when pa_launch_parallel_worker returns NULL, I >

Re: Handle infinite recursion in logical replication setup

2023-01-10 Thread Amit Kapila
On Tue, Jan 10, 2023 at 11:24 PM Jonathan S. Katz wrote: > > On 1/10/23 10:17 AM, Amit Kapila wrote: > > On Tue, Jan 10, 2023 at 8:13 AM Jonathan S. Katz > > wrote: > > > One can use local or higher > > for reducing the latency for COMMIT when synchronous replication is > > used in the

Re: [PATCH] Add overlaps geometric operators that ignore point overlaps

2023-01-10 Thread Kyotaro Horiguchi
Hello. At Sun, 1 Jan 2023 01:13:24 +0530, Ankit Kumar Pandey wrote in > This is patch for todo item: Add overlaps geometric operators that > ignore point overlaps > > Issue: > > SELECT circle '((0,0), 1)' && circle '((2,0),1) returns True > > Expectation: In above case, both figures touch

RE: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread houzj.f...@fujitsu.com
On Wednesday, January 11, 2023 1:25 AM Ted Yu wrote: > I was reading src/backend/replication/logical/applyparallelworker.c . > In `pa_allocate_worker`, when pa_launch_parallel_worker returns NULL, I think > the `ParallelApplyTxnHash` should be released. Thanks for reporting.

Spinlock is missing when updating two_phase of ReplicationSlot

2023-01-10 Thread Masahiko Sawada
Hi, I realized that in CreateDecodingContext() function, we update both slot->data.two_phase and two_phase_at without acquiring the spinlock: /* Mark slot to allow two_phase decoding if not already marked */ if (ctx->twophase && !slot->data.two_phase) { slot->data.two_phase =

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Peter Geoghegan
On Tue, Jan 10, 2023 at 4:39 PM Peter Geoghegan wrote: > * Run VACUUM FREEZE. We need FREEZE in order to be able to hit the > relevant visibilitymap_set() call site (the one that passes > VISIBILITYMAP_ALL_FROZEN as its flags, without also passing > VISIBILITYMAP_ALL_VISIBLE). > > Now

Re: Infinite Interval

2023-01-10 Thread Joseph Koshakow
On Sun, Jan 8, 2023 at 11:17 PM jian he wrote: > > > > On Sun, Jan 8, 2023 at 4:22 AM Joseph Koshakow wrote: >> >> On Sat, Jan 7, 2023 at 3:05 PM Joseph Koshakow wrote: >> > >> > On Sat, Jan 7, 2023 at 3:04 PM Joseph Koshakow wrote: >> > > >> > > I think this patch is just about ready for

Re: Allow +group in pg_ident.conf

2023-01-10 Thread Michael Paquier
On Tue, Jan 10, 2023 at 09:42:19AM -0500, Andrew Dunstan wrote: > Ok, that sounds reasonable, but the cfbot doesn't like patches that > depend on other patches in a different email. Maybe you can roll this up > as an extra patch in your next version? It's pretty small. This can go two ways if

delay starting WAL receiver

2023-01-10 Thread Nathan Bossart
I discussed this a bit in a different thread [0], but I thought it deserved its own thread. After setting wal_retrieve_retry_interval to 1ms in the tests, I noticed that the recovery tests consistently take much longer. Upon further inspection, it looks like a similar race condition to the one

Re: Strengthen pg_waldump's --save-fullpage tests

2023-01-10 Thread Michael Paquier
On Tue, Jan 10, 2023 at 05:25:44PM +0100, Drouvot, Bertrand wrote: > I like the idea of comparing the full page (and not just the LSN) but > I'm not sure that adding the pageinspect dependency is a good thing. > > What about extracting the block directly from the relation file and > comparing it

Re: [PATCH] support tab-completion for single quote input with equal sign

2023-01-10 Thread Tom Lane
torikoshia writes: > I updated the patch going along with the v3 direction. I think this adds about as many failure modes as it removes, if not more. * The connection string doesn't necessarily end with "'"; it could be a dollar-quoted string. * If it is a dollar-quoted string, there could be

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-10 Thread David Rowley
On Tue, 10 Jan 2023 at 18:36, Tom Lane wrote: > > David Rowley writes: > > Ideally, our sort costing would just be better, but I think that > > raises the bar a little too high to start thinking of making > > improvements to that for this patch. > > It's trickier than it looks, cf f4c7c410e.

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-10 Thread David Rowley
On Wed, 11 Jan 2023 at 08:17, Ankit Kumar Pandey wrote: > > > > On 10/01/23 10:53, David Rowley wrote: > > > the total cost is the same for both of these, but the execution time > > seems to vary quite a bit. > > This is really weird, I tried it different ways (to rule out any issues > due to > >

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Peter Geoghegan
On Tue, Jan 10, 2023 at 12:08 PM Peter Geoghegan wrote: > Actually, FreezeMultiXactId() can fully remove an xmax that has some > member XIDs >= OldestXmin, provided FRM_NOOP processing isn't > possible, at least when no individual member is still running. Doesn't > have to involve transaction

Re: Use windows VMs instead of windows containers on the CI

2023-01-10 Thread Thomas Munro
On Wed, Jan 11, 2023 at 8:20 AM Andres Freund wrote: > On 2023-01-10 09:22:12 -0600, Justin Pryzby wrote: > > > There is more than 2x speed gain when VMs are used. > > > > One consideration is that if windows runs twice as fast, we'll suddenly > > start using twice as many resources at

Can we let extensions change their dumped catalog schemas?

2023-01-10 Thread Jacob Champion
Hi all, I've been talking to other Timescale devs about a requested change to pg_dump, and there's been quite a bit of back-and-forth to figure out what, exactly, we want. Any mistakes here are mine, but I think we've been able to distill it down to the following request: We'd like to be allowed

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-01-10 Thread Tom Lane
Sandro Santilli writes: > On Mon, Jan 09, 2023 at 05:51:49PM -0500, Tom Lane wrote: >> ... you still need one script file for each supported upgrade step > That's exactly the problem we're trying to solve here. > The include support is nice on itself, but won't solve our problem. The

Re: verbose mode for pg_input_error_message?

2023-01-10 Thread Nathan Bossart
On Wed, Jan 04, 2023 at 04:18:59PM -0500, Andrew Dunstan wrote: > On 2023-01-02 Mo 10:44, Tom Lane wrote: >> I don't think that just concatenating those strings would make for a >> pleasant API. More sensible, perhaps, to have a separate function >> that returns a record. Or we could redefine

wal_compression = method:level

2023-01-10 Thread Justin Pryzby
Is it desirable to support specifying a level ? Maybe there's a concern about using high compression levels, but I'll start by asking if the feature is wanted at all. Previous discussion at: 20210614012412.ga31...@telsasoft.com >From cb30e17cf19fffa370a887d28d6d7e683d588b71 Mon Sep 17 00:00:00

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-01-10 Thread Jacob Champion
On Mon, Jan 9, 2023 at 7:07 AM Jelte Fennema wrote: > I also took a closer look at the code, and the only comment I have is: > > > appendPQExpBuffer(>errorMessage, > > These calls can all be replaced by the recently added libpq_append_conn_error Argh, thanks for the catch. Fixed. > Finally I

Re: ATTACH PARTITION seems to ignore column generation status

2023-01-10 Thread Tom Lane
I wrote: > Amit Langote writes: >> Thanks for the patch. It looks good, though I guess you said that we >> should also change the error message that CREATE TABLE ... PARTITION >> OF emits to match the other cases while we're here. I've attached a >> delta patch. > Thanks. I hadn't touched

Wasted Vacuum cycles when OldestXmin is not moving

2023-01-10 Thread sirisha chamarthi
Hi Hackers, vacuum is not able to clean up dead tuples when OldestXmin is not moving (because of a long running transaction or when hot_standby_feedback is behind). Even though OldestXmin is not moved from the last time it checked, it keeps retrying every autovacuum_naptime and wastes CPU cycles

Re: Allow DISTINCT to use Incremental Sort

2023-01-10 Thread David Rowley
On Tue, 10 Jan 2023 at 16:07, Richard Guo wrote: > Sorry I didn't make myself clear. I mean currently on HEAD in planner.c > from line 4847 to line 4857, we have the code to make sure we always use > the more rigorous clause for explicit-sort case. I think this code is > not necessary, because

Re: allowing for control over SET ROLE

2023-01-10 Thread Jeff Davis
On Tue, 2023-01-10 at 11:45 -0500, Robert Haas wrote: > So the risks, which in theory are all very similar, are in practice > far greater in the PostgreSQL context, basically because our default > setup is about 40 years behind the times in terms of implementing > best > practices. I agree that

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-01-10 Thread Jacob Champion
On Mon, Jan 9, 2023 at 7:40 AM Andrew Dunstan wrote: > I'm confused. A client cert might not have a hostname at all, and isn't > used to verify the connecting address, but to verify the username. It > needs to have a CN/DN equal to the user name of the connection, or that > maps to that name via

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-01-10 Thread Sandro Santilli
On Mon, Jan 09, 2023 at 05:51:49PM -0500, Tom Lane wrote: > Have you considered the idea of instead inventing a "\include" facility [...] > cases, you still need one script file for each supported upgrade step That's exactly the problem we're trying to solve here. The include support is nice

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Peter Geoghegan
On Tue, Jan 10, 2023 at 12:19 PM Robert Haas wrote: > I don't understand what distinction you're making. It seems like > hair-splitting to me. We should be able to reproduce problems like > this reliably, at least with the aid of a debugger and some > breakpoints, before we go changing the code.

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-10 Thread Matthias van de Meent
On Tue, 10 Jan 2023 at 20:14, Andres Freund wrote: > > Hi, > > On 2023-01-10 15:03:42 +0100, Matthias van de Meent wrote: > > On Mon, 9 Jan 2023 at 20:34, Andres Freund wrote: > > > It's not too hard to fix in individual places, but I suspect that we'll > > > introduce the bug in future places

Re: Add SHELL_EXIT_CODE to psql

2023-01-10 Thread Corey Huinker
On Tue, Jan 10, 2023 at 3:54 AM Maxim Orlov wrote: > > > On Mon, 9 Jan 2023 at 21:36, Corey Huinker > wrote: > >> >> I chose a name that would avoid collisions with anything a user might >> potentially throw into their environment, so if the var "OS" is fairly >> standard is a reason to avoid

Re: heapgettup refactoring

2023-01-10 Thread Melanie Plageman
On Thu, Jan 5, 2023 at 8:52 AM Peter Eisentraut wrote: > > Ok, let's look through these patches starting from the top then. > > v4-0001-Add-no-movement-scan-helper.patch > > This makes sense overall; there is clearly some duplicate code that can > be unified. > > It appears that during your

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 2:48 PM Peter Geoghegan wrote: > What I actually said was that there is no reason to declare up front > that the only circumstances under which a fix could be committed is > when a clean repro is available. I never said that a test case has > little or no value, and I

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Peter Geoghegan
On Tue, Jan 10, 2023 at 11:47 AM Peter Geoghegan wrote: > In summary, I think that there is currently no way that we can have > the VM (or the PD_ALL_VISIBLE flag) concurrently unset, while leaving > the page all_frozen. It can happen and leave the page all_visible, but > not all_frozen, due to

Re: logical decoding and replication of sequences, take 2

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 1:32 PM Tomas Vondra wrote: > 0001 is a fix for the pre-existing issue in logicalmsg_decode, > attempting to build a snapshot before getting into a consistent state. > AFAICS this only affects assert-enabled builds and is otherwise > harmless, because we are not actually

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Peter Geoghegan
On Tue, Jan 10, 2023 at 10:50 AM Robert Haas wrote: > Look, I don't want to spend time arguing about what seem to me to be > basic principles of good software engineering. When I don't put test > cases into my patches, people complain at me and tell me that I'm a > bad software engineer because I

Re: Transparent column encryption

2023-01-10 Thread Mark Dilger
> On Jan 10, 2023, at 9:26 AM, Mark Dilger wrote: > >-- Cryptographically connected to the encrypted record >patient_id BIGINT NOT NULL, >patient_ssn CHAR(11), > >-- The encrypted record >patient_record TEXT ENCRYPTED WITH (column_encryption_key = cek1, >

Re: RFC: logical publication via inheritance root?

2023-01-10 Thread Jacob Champion
On Mon, Jan 9, 2023 at 12:41 AM Aleksander Alekseev wrote: > I would like to point out that we shouldn't necessarily support > multiple inheritance in all the possible cases, at least not in the > first implementation. Supporting simple cases of inheritance would be > already a valuable feature

Re: Show various offset arrays for heap WAL records

2023-01-10 Thread Andres Freund
Hi, On 2023-01-09 19:59:42 -0800, Peter Geoghegan wrote: > On Mon, Jan 9, 2023 at 1:58 PM Andres Freund wrote: > > A couple times when investigating data corruption issues, the last time just > > yesterday in [1], I needed to see the offsets affected by PRUNE and VACUUM > > records. As that's

Re: Use windows VMs instead of windows containers on the CI

2023-01-10 Thread Andres Freund
Hi, On 2023-01-10 09:22:12 -0600, Justin Pryzby wrote: > > There is more than 2x speed gain when VMs are used. > > One consideration is that if windows runs twice as fast, we'll suddenly > start using twice as many resources at cirrus/google/amazon - the > windows task has been throttling

Re: psql: Add role's membership options to the \du+ command

2023-01-10 Thread Pavel Luzanov
Added the patch to the open commitfest: https://commitfest.postgresql.org/42/4116/ Feel free to reject if it's not interesting. -- Pavel Luzanov

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-10 Thread Ankit Kumar Pandey
On 10/01/23 10:53, David Rowley wrote: the total cost is the same for both of these, but the execution time seems to vary quite a bit. This is really weird, I tried it different ways (to rule out any issues due to caching) and execution time varied in spite of having same cost. Maybe

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-10 Thread Andres Freund
Hi, On 2023-01-10 15:03:42 +0100, Matthias van de Meent wrote: > On Mon, 9 Jan 2023 at 20:34, Andres Freund wrote: > > On 2023-01-09 17:50:10 +0100, Matthias van de Meent wrote: > > > Wouldn't it be enough to only fix the constructions in > > > FullXidRelativeTo() and widen_snapshot_xid() (as

Re: Avoiding "wrong tuple length" errors at the end of VACUUM on pg_database update (Backpatch of 947789f to v12 and v13)

2023-01-10 Thread Andres Freund
Hi, On 2023-01-10 02:57:43 -0500, Tom Lane wrote: > No objections to back-patching the fix, but I wonder if we can find > some mechanical way to prevent this sort of error in future. What about a define that forces external toasting very aggressively for catalog tables, iff they have a toast

Re: can while loop in ClockSweepTick function be kind of infinite loop in some cases?

2023-01-10 Thread Andres Freund
Hi, On 2023-01-10 13:11:35 -0500, Robert Haas wrote: > On Tue, Jan 10, 2023 at 12:40 PM Andres Freund wrote: > > > I think. `expected = originalVictim + 1;` line should be in while loop > > > (before acquiring spin lock) so that, even in the case above, expected > > > variable is incremented for

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Robert Haas
On Mon, Jan 9, 2023 at 5:59 PM Peter Geoghegan wrote: > On Mon, Jan 9, 2023 at 12:51 PM Robert Haas wrote: > > I feel that you should at least have a reproducer for these problems > > posted to the thread, and ideally a regression test, before committing > > things. I think it's very hard to

Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread Ted Yu
On Tue, Jan 10, 2023 at 9:43 AM Ted Yu wrote: > > > On Tue, Jan 10, 2023 at 9:26 AM Ted Yu wrote: > >> >> >> On Tue, Jan 10, 2023 at 9:25 AM Ted Yu wrote: >> >>> Hi, >>> I was reading src/backend/replication/logical/applyparallelworker.c . >>> In `pa_allocate_worker`, when

Re: can while loop in ClockSweepTick function be kind of infinite loop in some cases?

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 12:40 PM Andres Freund wrote: > > I think. `expected = originalVictim + 1;` line should be in while loop > > (before acquiring spin lock) so that, even in the case above, expected > > variable is incremented for each loop and CAS operation will be successful > > at some

Re: Handle infinite recursion in logical replication setup

2023-01-10 Thread Jonathan S. Katz
On 1/10/23 10:17 AM, Amit Kapila wrote: On Tue, Jan 10, 2023 at 8:13 AM Jonathan S. Katz wrote: This consistently created the deadlock in my testing. Discussing with Masahiko off-list, this is due to a deadlock from 4 processes: the walsenders on A and B, and the apply workers on A and B.

Re: Avoiding "wrong tuple length" errors at the end of VACUUM on pg_database update (Backpatch of 947789f to v12 and v13)

2023-01-10 Thread Nathan Bossart
On Tue, Jan 10, 2023 at 02:57:43AM -0500, Tom Lane wrote: > Michael Paquier writes: >> Any objections about getting 947789f applied to REL_13_STABLE and >> REL_12_STABLE and see this issue completely gone from all the versions >> supported? > > No objections to back-patching the fix... +1 --

Re: fixing CREATEROLE

2023-01-10 Thread Robert Haas
On Thu, Jan 5, 2023 at 2:53 PM Robert Haas wrote: > On Tue, Jan 3, 2023 at 3:11 PM Robert Haas wrote: > > Committed and back-patched 0001 with fixes for the issues that you pointed > > out. > > > > Here's a trivial rebase of the rest of the patch set. > > I committed 0001 and 0002 after

  1   2   >