Re: MSVC SSL test failure

2021-12-06 Thread Alexander Lakhin
06.12.2021 23:51, Andrew Dunstan wrote: > I have been getting 100% failures on the SSL tests with closesocket() > alone, and 100% success over 10 tests with this: > > > diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c > index 96ab37c7d0..5998c089b0 100644 > ---

RE: row filtering for logical replication

2021-12-06 Thread tanghy.f...@fujitsu.com
On Friday, December 3, 2021 10:09 AM Peter Smith wrote: > > On Thu, Dec 2, 2021 at 2:32 PM tanghy.f...@fujitsu.com > wrote: > > > > On Thursday, December 2, 2021 5:21 AM Peter Smith > wrote: > > > > > > PSA the v44* set of patches. > > > > > > > Thanks for the new patch. Few comments: > > > >

Re: Do we need pre-allocate WAL files during end-of-recovery checkpoint?

2021-12-06 Thread Noah Misch
On Mon, Dec 06, 2021 at 06:21:40PM +0530, Bharath Rupireddy wrote: > The function PreallocXlogFiles doesn't get called during > end-of-recovery checkpoint in CreateCheckPoint, see [1]. The server > becomes operational after the end-of-recovery checkpoint and may need > WAL files.

Re: row filtering for logical replication

2021-12-06 Thread Peter Smith
Hi Euler – As you know we have been posting patch update versions to the Row-Filter thread several times a week now for a few months. We are carefully tracking all open review comments of the thread and fixing as many as possible with each version posted. ~~ It is true that the multiple patches

Re: pg_get_publication_tables() output duplicate relid

2021-12-06 Thread Amit Kapila
On Tue, Dec 7, 2021 at 10:52 AM Amit Langote wrote: > > On Tue, Dec 7, 2021 at 1:01 PM Amit Kapila wrote: > > On Mon, Dec 6, 2021 at 8:59 PM Amit Langote wrote: > > > On Mon, Dec 6, 2021 at 6:05 PM Amit Kapila > > > wrote: > > > > Your concern is not very clear to me. Can you be more specific

Re: Add sub-transaction overflow status in pg_stat_activity

2021-12-06 Thread Justin Pryzby
> + > + subxact_count xid > + > + > + The current backend's active subtransactions count. subtransaction (no s) > + Set to true if current backend's subtransaction cache is overflowed. Say "has overflowed" > + if (local_beentry->subxact_count >

Re: pg_get_publication_tables() output duplicate relid

2021-12-06 Thread Amit Langote
On Tue, Dec 7, 2021 at 1:01 PM Amit Kapila wrote: > On Mon, Dec 6, 2021 at 8:59 PM Amit Langote wrote: > > On Mon, Dec 6, 2021 at 6:05 PM Amit Kapila wrote: > > > Your concern is not very clear to me. Can you be more specific in what > > > kind of problem you see with such a design for row

Re: Fix a bug in DecodeAbort() and improve input data check on subscriber.

2021-12-06 Thread Amit Kapila
On Tue, Dec 7, 2021 at 6:06 AM Masahiko Sawada wrote: > > Hi all, > > While updating the patch I recently posted[1] to make pg_waldump > report replication origin ID, LSN, and timestamp, I found a bug that > replication origin timestamp is not set in ROLLBACK PREPARED case. > Commit 8bdb1332eb5

Re: pg_get_publication_tables() output duplicate relid

2021-12-06 Thread Amit Langote
On Tue, Dec 7, 2021 at 12:45 PM Amit Kapila wrote: > On Mon, Dec 6, 2021 at 8:04 PM Amit Langote wrote: > > So IIUC the scenario of concern is when a table to be attached as a > > partition is in a schema that's present in pg_publication_namespace. > > The only way to stop it from being

Re: Add sub-transaction overflow status in pg_stat_activity

2021-12-06 Thread Nikolay Samokhvalov
On Mon, Dec 6, 2021 at 8:16 PM Dilip Kumar wrote: > If the subtransaction cache is overflowed in some of the transactions > then it will affect all the concurrent queries as they need to access > the SLRU for checking the visibility of each tuple. But currently > there is no way to identify

Re: Add sub-transaction overflow status in pg_stat_activity

2021-12-06 Thread Zhihong Yu
On Mon, Dec 6, 2021 at 8:17 PM Dilip Kumar wrote: > If the subtransaction cache is overflowed in some of the transactions > then it will affect all the concurrent queries as they need to access > the SLRU for checking the visibility of each tuple. But currently > there is no way to identify

Add sub-transaction overflow status in pg_stat_activity

2021-12-06 Thread Dilip Kumar
If the subtransaction cache is overflowed in some of the transactions then it will affect all the concurrent queries as they need to access the SLRU for checking the visibility of each tuple. But currently there is no way to identify whether in any backend subtransaction is overflowed or what is

Re: pg_get_publication_tables() output duplicate relid

2021-12-06 Thread Amit Kapila
On Mon, Dec 6, 2021 at 8:59 PM Amit Langote wrote: > > On Mon, Dec 6, 2021 at 6:05 PM Amit Kapila wrote: > > On Mon, Dec 6, 2021 at 1:00 PM Amit Langote wrote: > > > On Mon, Dec 6, 2021 at 3:55 PM houzj.f...@fujitsu.com > > > wrote: > > > > After thinking more on this. I find there might be

Re: pg_get_publication_tables() output duplicate relid

2021-12-06 Thread Amit Kapila
On Mon, Dec 6, 2021 at 8:04 PM Amit Langote wrote: > > So IIUC the scenario of concern is when a table to be attached as a > partition is in a schema that's present in pg_publication_namespace. > The only way to stop it from being published is to move it to another > schema that is not published

Re: Triage for unimplemented geometric operators

2021-12-06 Thread Laurenz Albe
On Mon, 2021-12-06 at 18:18 -0500, Tom Lane wrote: > Over in [1] it was pointed out that I overenthusiastically > documented several geometric operators that, in fact, are > only stubs that throw errors when called.  Specifically > these are > > dist_lb:<->(line,box) > dist_bl:

Re: parallel vacuum comments

2021-12-06 Thread Amit Kapila
On Tue, Dec 7, 2021 at 6:54 AM Masahiko Sawada wrote: > > On Mon, Dec 6, 2021 at 1:47 PM Amit Kapila wrote: > > > > On Fri, Dec 3, 2021 at 6:06 PM Masahiko Sawada > > wrote: > > > > > > > > > > > 3. /* > > > > * Copy the index bulk-deletion result returned from ambulkdelete and > > > > @@

Re: Alter all tables in schema owner fix

2021-12-06 Thread vignesh C
On Fri, Dec 3, 2021 at 10:50 PM Bossart, Nathan wrote: > > On 12/2/21, 11:57 PM, "tanghy.f...@fujitsu.com" > wrote: > > Thanks for your patch. > > I tested it and it fixed this problem as expected. It also passed "make > > check-world". > > +1, the patch looks good to me, too. My only other

Re: Why doesn't pgstat_report_analyze() focus on not-all-visible-page dead tuple counts, specifically?

2021-12-06 Thread Peter Geoghegan
On Mon, Dec 6, 2021 at 6:11 PM Robert Haas wrote: > This doesn't seem convincing. Launching autovacuum too soon surely has > costs that someone might not want to pay. Clearly in the degenerate > case where we always autovacuum every table every time an autovacuum > worker is launched, we have

Re: row filtering for logical replication

2021-12-06 Thread Amit Kapila
On Mon, Dec 6, 2021 at 6:18 PM Euler Taveira wrote: > > On Mon, Dec 6, 2021, at 3:44 AM, Amit Kapila wrote: > > True, but that is the main reason the review and development are being > done as separate sub-features. I suggest still keeping the similar > separation till some of the reviews of each

Re: Why doesn't pgstat_report_analyze() focus on not-all-visible-page dead tuple counts, specifically?

2021-12-06 Thread Robert Haas
On Mon, Dec 6, 2021 at 8:14 PM Peter Geoghegan wrote: > Suppose we believe that not-all-visible pages have 20 LP_DEAD items on > average, and they turn out to only have 3 or 5. Theoretically we've > done the wrong thing by launching autovacuum workers sooner -- we > introduce bias. But we also

Re: parallel vacuum comments

2021-12-06 Thread Masahiko Sawada
On Mon, Dec 6, 2021 at 1:47 PM Amit Kapila wrote: > > On Fri, Dec 3, 2021 at 6:06 PM Masahiko Sawada wrote: > > > > On Fri, Dec 3, 2021 at 6:03 PM Amit Kapila wrote: > > > > > > On Thu, Dec 2, 2021 at 6:01 PM Masahiko Sawada > > > wrote: > > > > > > > > I've attached updated patches. > > > >

Re: Why doesn't pgstat_report_analyze() focus on not-all-visible-page dead tuple counts, specifically?

2021-12-06 Thread Peter Geoghegan
On Mon, Dec 6, 2021 at 2:37 PM Peter Geoghegan wrote: > On Mon, Dec 6, 2021 at 12:07 PM Robert Haas wrote: > > So does this. If some of the table is now all-visible when it wasn't > > before, it's certainly a good guess that the portions that still > > aren't have about the same distribution of

Re: Make pg_waldump report replication origin ID, LSN, and timestamp.

2021-12-06 Thread Masahiko Sawada
On Mon, Dec 6, 2021 at 11:24 PM Masahiko Sawada wrote: > > On Mon, Dec 6, 2021 at 5:09 PM Michael Paquier wrote: > > > > On Mon, Dec 06, 2021 at 04:35:07PM +0900, Masahiko Sawada wrote: > > > I've attached a patch to add replication origin information to > > > xact_desc_prepare(). > > > > Yeah.

Re: Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?

2021-12-06 Thread Kyotaro Horiguchi
At Mon, 6 Dec 2021 19:28:03 +, "Bossart, Nathan" wrote in > On 12/6/21, 4:34 AM, "Bharath Rupireddy" > wrote: > > While the database is performing end-of-recovery checkpoint, the > > control file gets updated with db state as "shutting down" in > > CreateCheckPoint (see the code snippet

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-06 Thread Ashutosh Sharma
Thanks Robert for sharing your thoughts. On Mon, Dec 6, 2021 at 11:16 PM Robert Haas wrote: > On Mon, Dec 6, 2021 at 9:23 AM Ashutosh Sharma > wrote: > > One last point - If we try to clone a huge database, as expected CREATE > DATABASE emits a lot of WALs, causing a lot of intermediate

Fix a bug in DecodeAbort() and improve input data check on subscriber.

2021-12-06 Thread Masahiko Sawada
Hi all, While updating the patch I recently posted[1] to make pg_waldump report replication origin ID, LSN, and timestamp, I found a bug that replication origin timestamp is not set in ROLLBACK PREPARED case. Commit 8bdb1332eb5 (CC'ed Amit) added an argument to ReorderBufferFinishPrepared() but

Re: Optionally automatically disable logical replication subscriptions on error

2021-12-06 Thread Greg Nancarrow
On Tue, Dec 7, 2021 at 3:06 AM Mark Dilger wrote: > > My concern about disabling a subscription in response to *any* error is that > people may find the feature does more harm than good. Disabling the > subscription in response to an occasional deadlock against other database > users, or

Triage for unimplemented geometric operators

2021-12-06 Thread Tom Lane
Over in [1] it was pointed out that I overenthusiastically documented several geometric operators that, in fact, are only stubs that throw errors when called. Specifically these are dist_lb:<->(line,box) dist_bl:<->(box,line) close_sl: lseg ## line close_lb: line ##

Re: Why doesn't pgstat_report_analyze() focus on not-all-visible-page dead tuple counts, specifically?

2021-12-06 Thread Peter Geoghegan
On Mon, Dec 6, 2021 at 12:07 PM Robert Haas wrote: > So does this. If some of the table is now all-visible when it wasn't > before, it's certainly a good guess that the portions that still > aren't have about the same distribution of dead tuples that they did > before ... although the other

Re: GiST operator class for bool

2021-12-06 Thread Pavel Luzanov
Hello, I don't see any changes in the documentation.[1] Should bool appear in the looong list of supported operator classes? [1] https://www.postgresql.org/docs/devel/btree-gist.html -- Pavel Luzanov Postgres Professional: https://postgrespro.com The Russian Postgres Company

Re: pg_dump versus ancient server versions

2021-12-06 Thread Tom Lane
Robert Haas writes: > On Sun, Dec 5, 2021 at 7:41 PM Tom Lane wrote: >> Based on these results, I think maybe we should raise our ambitions >> a bit compared to Peter's original proposal. Specifically, >> I wonder if it wouldn't be wise to try to silence compile warnings >> in these branches.

Re: parse_subscription_options - suggested improvements

2021-12-06 Thread Peter Smith
On Tue, Dec 7, 2021 at 6:07 AM Bossart, Nathan wrote: > > On 12/5/21, 9:21 PM, "Michael Paquier" wrote: > > On Mon, Dec 06, 2021 at 11:28:12AM +1100, Peter Smith wrote: > >> For the initialization of opts I put memset within the function to > >> make it explicit that the bit-masks will work as

Re: ExecTypeSetColNames is fundamentally broken

2021-12-06 Thread Tom Lane
Robert Haas writes: > I don't understand the code so I can't comment on the code, but I find > the regression test changes pretty suspect. Attaching any alias list > to the RTE ought to rename the output columns for all purposes, not > just the ones we as implementers find convenient. Well, that

Re: MSVC SSL test failure

2021-12-06 Thread Andrew Dunstan
On 12/6/21 10:30, Alexander Lakhin wrote: > Hello Andrew, > 06.12.2021 17:56, Andrew Dunstan wrote: >> Yeah, quite annoying, especially because only some combinations of MSVC >> runtime / openssl version seem to trigger the problem. >> >> >> Adding a shutdown() before the closesocket() also

Re: Transparent column encryption

2021-12-06 Thread Jacob Champion
On Fri, 2021-12-03 at 22:32 +0100, Peter Eisentraut wrote: > This feature does support deterministic > encryption as an alternative to the default randomized encryption, so > in that mode you can do equality lookups, at the cost of some > security. > + if (enc_det) > +

Re: pg_dump versus ancient server versions

2021-12-06 Thread Robert Haas
On Sun, Dec 5, 2021 at 7:41 PM Tom Lane wrote: > Based on these results, I think maybe we should raise our ambitions > a bit compared to Peter's original proposal. Specifically, > I wonder if it wouldn't be wise to try to silence compile warnings > in these branches. The argument for this is

Re: ExecTypeSetColNames is fundamentally broken

2021-12-06 Thread Robert Haas
On Sun, Dec 5, 2021 at 1:46 PM Tom Lane wrote: > So 0001 attached fixes this by revoking the decision to apply > ExecTypeSetColNames in cases where a Var or RowExpr is declared > to return a named composite type. This causes a couple of regression > test results to change, but they are ones that

Re: Why doesn't pgstat_report_analyze() focus on not-all-visible-page dead tuple counts, specifically?

2021-12-06 Thread Robert Haas
On Sun, Dec 5, 2021 at 12:28 AM Peter Geoghegan wrote: > I wonder why we're counting the number of dead tuples (or LP_DEAD stub > items) in the relation as a whole in ANALYZE's acquire_sample_rows() > function. Wouldn't it make more sense to focus on the "live vs dead > tuple properties" of heap

Re: Do we need pre-allocate WAL files during end-of-recovery checkpoint?

2021-12-06 Thread Bossart, Nathan
On 12/6/21, 4:54 AM, "Bharath Rupireddy" wrote: > The function PreallocXlogFiles doesn't get called during > end-of-recovery checkpoint in CreateCheckPoint, see [1]. The server > becomes operational after the end-of-recovery checkpoint and may need > WAL files. However, I'm not sure how

Re: Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?

2021-12-06 Thread Bossart, Nathan
On 12/6/21, 4:34 AM, "Bharath Rupireddy" wrote: > While the database is performing end-of-recovery checkpoint, the > control file gets updated with db state as "shutting down" in > CreateCheckPoint (see the code snippet at [1]) and at the end it sets > it back to "shut down" for a brief moment

Re: O(n) tasks cause lengthy startups and checkpoints

2021-12-06 Thread Bossart, Nathan
On 12/6/21, 3:44 AM, "Bharath Rupireddy" wrote: > On Fri, Dec 3, 2021 at 11:50 PM Bossart, Nathan wrote: >> I might hack something together for the separate worker approach, if >> for no other reason than to make sure I really understand how these >> functions work. If/when a better idea

Re: pg_replslotdata - a tool for displaying replication slot information

2021-12-06 Thread Bossart, Nathan
On 12/5/21, 11:10 PM, "Michael Paquier" wrote: > On Thu, Dec 02, 2021 at 08:32:08AM +0530, Bharath Rupireddy wrote: >> On Thu, Dec 2, 2021 at 4:22 AM Andres Freund wrote: I don't have any other compelling use- cases at the moment, but I will say that it is typically nice from an

Re: Do we need pre-allocate WAL files during end-of-recovery checkpoint?

2021-12-06 Thread SATYANARAYANA NARLAPURAM
If the segment size is 16MB it shouldn't take much time but higher segment values this can be a problem. But again, the current segment has to be filled 75% to precreate new one. I am not sure how much we gain. Do you have some numbers with different segment sizes? On Mon, Dec 6, 2021 at 4:51 AM

Re: PostgreSQL server: authentication method 10 not supported

2021-12-06 Thread Tom Lane
Ram Pratap Maurya writes: > "PHP Warning: pg_connect(): Unable to connect to PostgreSQL server: > authentication method 10 not supported in > /var/www/html/myLavaUat/app/webroot/myLavaCronDirect/cron/mylava_stg_arch.php > on line 9 " You need to update your client's libpq to a version that

Re: Transparent column encryption

2021-12-06 Thread Robert Haas
On Fri, Dec 3, 2021 at 4:32 PM Peter Eisentraut wrote: > But it's missing the remaining 90% of the work, > including additional DDL support, error handling, robust memory > management, protocol versioning, forward and backward compatibility, > pg_dump support, psql \d support, refinement of the

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-12-06 Thread Robert Haas
On Sun, Dec 5, 2021 at 11:44 PM Sadhuprasad Patro wrote: > 1. > --- a/doc/src/sgml/ref/create_database.sgml > +++ b/doc/src/sgml/ref/create_database.sgml > @@ -31,7 +31,8 @@ CREATE DATABASE class="parameter">name > - [ IS_TEMPLATE [=] class="parameter">istemplate ] ] > + [

[PATCH] Document heuristics for parameterized paths

2021-12-06 Thread Steinar H. Gunderson
Hi, A small patch to the documentation about how to reduce the number of parameterized paths, because it kept me searching for a long time :-) (The code this documents is in add_paths_to_joinrel(), the loop foreach(lc, root->join_info_list).) diff --git a/src/backend/optimizer/README

PostgreSQL server: authentication method 10 not supported

2021-12-06 Thread Ram Pratap Maurya
Hi Team, I am facing connectivity issue with PostgreSQL -13 , can you please suggest . Error coming: "PHP Warning: pg_connect(): Unable to connect to PostgreSQL server: authentication method 10 not supported in

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-06 Thread Robert Haas
On Mon, Dec 6, 2021 at 9:23 AM Ashutosh Sharma wrote: > One last point - If we try to clone a huge database, as expected CREATE > DATABASE emits a lot of WALs, causing a lot of intermediate checkpoints which > seems to be affecting the performance slightly. Yes, I think this needs to be

Re: Optionally automatically disable logical replication subscriptions on error

2021-12-06 Thread Mark Dilger
> On Dec 5, 2021, at 10:56 PM, osumi.takami...@fujitsu.com wrote: > > In my humble opinion, I felt the original purpose of the patch was to > partially remedy > the situation that during the failure of apply, the apply process keeps going > into the infinite error loop. I agree. > I'd say

Re: should we document an example to set multiple libraries in shared_preload_libraries?

2021-12-06 Thread Tom Lane
Bharath Rupireddy writes: > Am I missing something here? Or is there a distinction between parsing > of postgresql.conf and ALTER SYSTEM SET command for GUC_LIST_QUOTE > values? If so, what is it? One context is SQL, the other is not. The quoting rules are really quite different.

Re: Non-superuser subscription owners

2021-12-06 Thread Mark Dilger
> On Dec 6, 2021, at 2:19 AM, Amit Kapila wrote: > >>> If we want to maintain the property that subscriptions can only be >>> owned by superuser We don't want to maintain such a property, or at least, that's not what I want. I don't think that's what Jeff wants, either. To clarify, I'm

Re: should we document an example to set multiple libraries in shared_preload_libraries?

2021-12-06 Thread Bharath Rupireddy
On Fri, Dec 3, 2021 at 11:25 PM Bossart, Nathan wrote: > > On 12/3/21, 6:21 AM, "Bharath Rupireddy" > wrote: > > +1 to add here in the "Parameter Names and Values section", but do we > > want to backlink every string parameter to this section? I think it > > needs more effort. IMO, we can just

Re: MSVC SSL test failure

2021-12-06 Thread Alexander Lakhin
Hello Andrew, 06.12.2021 17:56, Andrew Dunstan wrote: > Yeah, quite annoying, especially because only some combinations of MSVC > runtime / openssl version seem to trigger the problem. > > > Adding a shutdown() before the closesocket() also fixes the issue. > Can you confirm that adding

Re: pg_get_publication_tables() output duplicate relid

2021-12-06 Thread Amit Langote
On Mon, Dec 6, 2021 at 6:05 PM Amit Kapila wrote: > On Mon, Dec 6, 2021 at 1:00 PM Amit Langote wrote: > > On Mon, Dec 6, 2021 at 3:55 PM houzj.f...@fujitsu.com > > wrote: > > > After thinking more on this. I find there might be some usage about > > > adding both > > > child and parent to the

Re: MSVC SSL test failure

2021-12-06 Thread Daniel Gustafsson
> On 6 Dec 2021, at 15:56, Andrew Dunstan wrote: > Yeah, quite annoying, especially because only some combinations of MSVC > runtime / openssl version seem to trigger the problem. > > Adding a shutdown() before the closesocket() also fixes the issue. If you have a patch you're testing I'm

psql: exit status with multiple -c and -f

2021-12-06 Thread Justin Pryzby
I raised this issue a few years ago. https://www.postgresql.org/message-id/20181217175841.GS13019%40telsasoft.com |[pryzbyj@database ~]$ psql -v VERBOSITY=terse ts -xtc 'ONE' -c "SELECT 'TWO'"; echo "exit status $?" |ERROR: syntax error at or near "ONE" at character 1 |?column? | TWO | |exit

Re: MSVC SSL test failure

2021-12-06 Thread Andrew Dunstan
On 12/6/21 01:02, Tom Lane wrote: > Andrew Dunstan writes: >> On 12/5/21 12:50, Tom Lane wrote: >>> This looks quite a bit like the sort of failure that commit >>> 6051857fc was meant to forestall. I wonder whether reverting >>> that commit changes the results? You might also try inserting

Re: Multi-Column List Partitioning

2021-12-06 Thread Amul Sul
On Mon, Dec 6, 2021 at 7:27 PM Nitin Jadhav wrote: > > Thank you for reviewing the patch. > > > partbounds.c: In function ‘get_qual_for_list.isra.18’: > > partbounds.c:4284:29: warning: ‘boundinfo’ may be used uninitialized > > in this function [-Wmaybe-uninitialized] > >

Re: pg_get_publication_tables() output duplicate relid

2021-12-06 Thread Amit Langote
On Mon, Dec 6, 2021 at 1:59 PM Amit Kapila wrote: > On Fri, Dec 3, 2021 at 6:04 PM Amit Langote wrote: > > On Fri, Dec 3, 2021 at 12:37 PM Amit Kapila wrote: > > > On Thu, Dec 2, 2021 at 7:18 PM Amit Langote > > > wrote: > > > > Okay, I did write a PoC patch this morning after sending out my

Re: Failed transaction statistics to measure the logical replication progress

2021-12-06 Thread vignesh C
On Sat, Dec 4, 2021 at 6:32 PM osumi.takami...@fujitsu.com wrote: > > On Friday, December 3, 2021 3:12 PM vignesh C wrote: > > Thanks for the updated patch. > > Currently we are storing the commit count, error_count and abort_count for > > each table of the table sync operation. If we have

Re: Make pg_waldump report replication origin ID, LSN, and timestamp.

2021-12-06 Thread Masahiko Sawada
On Mon, Dec 6, 2021 at 5:09 PM Michael Paquier wrote: > > On Mon, Dec 06, 2021 at 04:35:07PM +0900, Masahiko Sawada wrote: > > I've attached a patch to add replication origin information to > > xact_desc_prepare(). > > Yeah. > > + if (origin_id != InvalidRepOriginId) > +

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-06 Thread Ashutosh Sharma
Thank you, Dilip for the quick response. I am okay with the changes done in the v7 patch. One last point - If we try to clone a huge database, as expected CREATE DATABASE emits a lot of WALs, causing a lot of intermediate checkpoints which seems to be affecting the performance slightly. -- With

Re: MSVC SSL test failure

2021-12-06 Thread Daniel Gustafsson
> On 6 Dec 2021, at 07:02, Tom Lane wrote: > > Andrew Dunstan writes: >> On 12/5/21 12:50, Tom Lane wrote: >>> This looks quite a bit like the sort of failure that commit >>> 6051857fc was meant to forestall. I wonder whether reverting >>> that commit changes the results? You might also try

Re: GUC flags

2021-12-06 Thread Justin Pryzby
On Mon, Dec 06, 2021 at 03:58:39PM +0900, Michael Paquier wrote: > On Sun, Dec 05, 2021 at 11:38:05PM -0600, Justin Pryzby wrote: > > Thanks. One more item. The check_guc script currently outputs 68 false > > positives - even though it includes a list of 20 exceptions. This is not > > useful. >

Do we need pre-allocate WAL files during end-of-recovery checkpoint?

2021-12-06 Thread Bharath Rupireddy
Hi, The function PreallocXlogFiles doesn't get called during end-of-recovery checkpoint in CreateCheckPoint, see [1]. The server becomes operational after the end-of-recovery checkpoint and may need WAL files. However, I'm not sure how beneficial it is going to be if the WAL is pre-allocated (as

Re: row filtering for logical replication

2021-12-06 Thread Euler Taveira
On Mon, Dec 6, 2021, at 3:44 AM, Amit Kapila wrote: > I think what you said as (b) is wrong because we want to allow builtin > immutable functions. See discussion [1]. It was a typo. I mean "non-immutable" function. > True, but that is the main reason the review and development are being > done

Re: Fix inappropriate uses of PG_GETARG_UINT32()

2021-12-06 Thread Peter Eisentraut
On 01.12.21 22:59, Bossart, Nathan wrote: On 12/1/21, 10:29 AM, "Peter Eisentraut" wrote: The attached patch fixes this by accepting the argument using PG_GETARG_INT32(), doing some checks, and then casting it to unsigned for the rest of the code. The patch also fixes another inappropriate

Re: row filtering for logical replication

2021-12-06 Thread Euler Taveira
On Mon, Dec 6, 2021, at 3:35 AM, Dilip Kumar wrote: > On Mon, Dec 6, 2021 at 6:49 AM Euler Taveira wrote: > > > > On Fri, Dec 3, 2021, at 8:12 PM, Euler Taveira wrote: > > > > PS> I will update the commit message in the next version. I barely changed > > the > > documentation to reflect the

Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?

2021-12-06 Thread Bharath Rupireddy
Hi, While the database is performing end-of-recovery checkpoint, the control file gets updated with db state as "shutting down" in CreateCheckPoint (see the code snippet at [1]) and at the end it sets it back to "shut down" for a brief moment and then finally to "in production". If the

Re: Shouldn't postgres_fdw report warning when it gives up getting result from foreign server?

2021-12-06 Thread Bharath Rupireddy
On Mon, Dec 6, 2021 at 1:47 PM Fujii Masao wrote: > Yeah, I agree that's not elegant.. > > So I'd like to propose new patch with different design from > what I proposed before. Patch attached. > > This patch changes pgfdw_exec_cleanup_query() so that it tells > its callers the information about

Re: O(n) tasks cause lengthy startups and checkpoints

2021-12-06 Thread Bharath Rupireddy
On Fri, Dec 3, 2021 at 11:50 PM Bossart, Nathan wrote: > > On 12/3/21, 5:57 AM, "Bharath Rupireddy" > wrote: > > On Fri, Dec 3, 2021 at 3:01 AM Bossart, Nathan wrote: > >> > >> On 12/1/21, 6:48 PM, "Bharath Rupireddy" > >> wrote: > >> > +1 for the overall idea of making the checkpoint

Re: preserve timestamps when installing headers

2021-12-06 Thread Peter Eisentraut
On 06.12.21 12:15, Michael Paquier wrote: FWIW, I am not on board with changing build semantics or any assumptions the header installation relies on either, but I could see a point in switching back to INSTALL_DATA instead of cp to be consistent with the rest of the build, iff the argument made

Re: preserve timestamps when installing headers

2021-12-06 Thread Peter Eisentraut
On 06.12.21 07:51, Tom Lane wrote: TBH, I am not convinced that the complained-of case is enough of a problem to justify any change in our build rules, even if there weren't any semantic issues. If you are worried about build times, you should be using ccache, and IME builds using ccache are

Re: preserve timestamps when installing headers

2021-12-06 Thread Michael Paquier
On Mon, Dec 06, 2021 at 01:51:39AM -0500, Tom Lane wrote: > TBH, I am not convinced that the complained-of case is enough of a > problem to justify any change in our build rules, even if there > weren't any semantic issues. If you are worried about build times, > you should be using ccache, and

Re: [Proposal] Add foreign-server health checks infrastructure

2021-12-06 Thread Shinya Kato
On 2021-11-29 21:36, Zhihong Yu wrote: On Mon, Nov 29, 2021 at 12:51 AM kuroda.hay...@fujitsu.com wrote: Dear Zhihong, Thank you for giving comments! I'll post new patches later. +#define HOLD_CHECKING_REMOTE_SERVERS_INTERRUPTS() (CheckingRemoteServersHoldoffCount++) The macro contains

Re: Optionally automatically disable logical replication subscriptions on error

2021-12-06 Thread Amit Kapila
On Mon, Dec 6, 2021 at 10:07 AM Mark Dilger wrote: > > > On Dec 1, 2021, at 8:48 PM, Amit Kapila wrote: > > > > The patch disables the subscription for non-transient errors. I am not > > sure if we can easily make the call to decide whether any particular > > error is transient or not. For

RE: Optionally automatically disable logical replication subscriptions on error

2021-12-06 Thread osumi.takami...@fujitsu.com
On Monday, December 6, 2021 1:16 PM Greg Nancarrow wrote: > On Sat, Dec 4, 2021 at 12:20 AM osumi.takami...@fujitsu.com > wrote: > > > > Hi, I've made a new patch v11 that incorporated suggestions described > above. > > > > Some review comments for the v11 patch: Thank you for your reviews !

Re: row filtering for logical replication

2021-12-06 Thread Peter Smith
On Sat, Dec 4, 2021 at 10:13 AM Euler Taveira wrote: > > On Thu, Dec 2, 2021, at 4:18 AM, Peter Smith wrote: > > PSA a new v44* patch set. > ... > I used the last patch series (v44) posted by Peter Smith [1]. I did a lot of > improvements in this new version (v45). I merged 0001 (it is basically

Re: Non-superuser subscription owners

2021-12-06 Thread Amit Kapila
On Fri, Dec 3, 2021 at 10:37 PM Mark Dilger wrote: > > > On Dec 2, 2021, at 1:29 AM, Amit Kapila wrote: > > > > If we want to maintain the property that subscriptions can only be > > owned by superuser for your first version then isn't a simple check > > like ((!superuser()) for each of the

Re: row filtering for logical replication

2021-12-06 Thread Amit Kapila
On Mon, Dec 6, 2021 at 6:49 AM Euler Taveira wrote: > > On Fri, Dec 3, 2021, at 8:12 PM, Euler Taveira wrote: > > PS> I will update the commit message in the next version. I barely changed the > documentation to reflect the current behavior. I probably missed some changes > but I will fix in the

Re: pg_get_publication_tables() output duplicate relid

2021-12-06 Thread Amit Kapila
On Mon, Dec 6, 2021 at 1:00 PM Amit Langote wrote: > > On Mon, Dec 6, 2021 at 3:55 PM houzj.f...@fujitsu.com > wrote: > > On Thursday, December 2, 2021 9:48 PM Amit Langote > > wrote: > > > On Thu, Dec 2, 2021 at 2:27 PM Amit Kapila > > > wrote: > > > > On Thu, Dec 2, 2021 at 8:42 AM Amit

Re: Alter all tables in schema owner fix

2021-12-06 Thread Amit Kapila
On Mon, Dec 6, 2021 at 11:46 AM Michael Paquier wrote: > > On Fri, Dec 03, 2021 at 05:20:35PM +, Bossart, Nathan wrote: > > On 12/2/21, 11:57 PM, "tanghy.f...@fujitsu.com" > > wrote: > > > Thanks for your patch. > > > I tested it and it fixed this problem as expected. It also passed "make

Re: stat() vs ERROR_DELETE_PENDING, round N + 1

2021-12-06 Thread Thomas Munro
On Sat, Dec 4, 2021 at 6:18 PM Thomas Munro wrote: > On Thu, Dec 2, 2021 at 3:11 AM Daniel Gustafsson wrote: > > > This patch doesn't compile on Windows according to Appveyor, seemingly > > > because > > > of a syntax error in the new win32ntdll.h file, but the MSVC logs are > > > hard on > >

Re: Shouldn't postgres_fdw report warning when it gives up getting result from foreign server?

2021-12-06 Thread Fujii Masao
On 2021/12/03 23:04, Bharath Rupireddy wrote: Let's not use the boolean just for the cancel request which isn't scalable IMO. Maybe a macro/enum? Otherwise, we could just do, although it doesn't look elegant: if (pgfdw_get_cleanup_result(conn, endtime, , "(cancel request)")) if

Re: Make pg_waldump report replication origin ID, LSN, and timestamp.

2021-12-06 Thread Michael Paquier
On Mon, Dec 06, 2021 at 04:35:07PM +0900, Masahiko Sawada wrote: > I've attached a patch to add replication origin information to > xact_desc_prepare(). Yeah. + if (origin_id != InvalidRepOriginId) + appendStringInfo(buf, "; origin: node %u, lsn %X/%X, at %s", +