Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-03-23 Thread Amul Sul
On Tue, Mar 23, 2021 at 8:59 PM Tom Lane wrote: > > I wrote: > > Michael Paquier writes: > >> One bisect later, the winner is: > >> commit: 3d351d916b20534f973eda760cde17d96545d4c4 > >> author: Tom Lane > >> date: Sun, 30 Aug 2020 12:21:51 -0400 > >> Redefine pg_class.reltuples to be -1 before

Re: Protect syscache from bloating with negative cache entries

2021-03-23 Thread Kyotaro Horiguchi
At Mon, 22 Mar 2021 13:12:10 -0400, Bruce Momjian wrote in > On Thu, Jan 28, 2021 at 05:16:52PM +0900, Kyotaro Horiguchi wrote: > > At Thu, 28 Jan 2021 16:50:44 +0900 (JST), Kyotaro Horiguchi > > wrote in > > > I was going to write in the doc something like "you can inspect memory > > >

Re: Failed assertion on standby while shutdown

2021-03-23 Thread Maxim Orlov
On 2021-03-22 16:40, Fujii Masao wrote: On 2021/03/20 2:25, Maxim Orlov wrote: Hi, haсkers! Recently, I was doing some experiments with primary/standby instances interaction. In certain conditions I’ve got and was able to reproduce crash on failed assertion. The scenario is the following:

Re: Add client connection check during the execution of the query

2021-03-23 Thread Andres Freund
Hi, On 2021-03-24 16:08:13 +1300, Thomas Munro wrote: > ... Andres just asked me the same question, when we were discussing > the pq_peekmessage() patch (v7). I had remembered that POLLHUP didn't > work for this type of thing, from some earlier attempt at something > similar, and indeed on my

Re: Allow matching whole DN from a client certificate

2021-03-23 Thread Michael Paquier
On Fri, Mar 05, 2021 at 04:01:38PM -0500, Andrew Dunstan wrote: > Yeah, fixed this, added a bit more docco, and got rid of some bitrot. I had a look at this patch. What you have here looks in good shape, and I have come comments. > + This option is probably best used in conjunction with a

Re: SQL-standard function body

2021-03-23 Thread Jaime Casanova
On Fri, Mar 19, 2021 at 8:49 AM Peter Eisentraut wrote: > > Right. Here is a new patch with that fix added and a small conflict > resolved. Great. It seems print_function_sqlbody() is not protected to avoid receiving a function that hasn't a standard sql body in

Re: Support for NSS as a libpq TLS backend

2021-03-23 Thread Michael Paquier
On Tue, Mar 23, 2021 at 12:38:50AM +0100, Daniel Gustafsson wrote: > Thanks again for reviewing, another version which addresses the remaining > issues will be posted soon but I wanted to get this out to give further > reviews > something that properly works. I have been looking at the

Re: Autovacuum worker doesn't immediately exit on postmaster death

2021-03-23 Thread Michael Paquier
On Mon, Mar 22, 2021 at 04:07:12PM -0400, Robert Haas wrote: > On Mon, Mar 22, 2021 at 1:48 PM Stephen Frost wrote: >> Thanks for that. Attached is just a rebased version with a commit >> message added. If there aren't any other concerns, I'll commit this in >> the next few days and back-patch

Re: New IndexAM API controlling index vacuum strategies

2021-03-23 Thread Masahiko Sawada
On Wed, Mar 24, 2021 at 11:44 AM Peter Geoghegan wrote: > > On Tue, Mar 23, 2021 at 4:02 AM Masahiko Sawada wrote: > > Here are review comments on 0003 patch: > > Attached is a new revision, v5. It fixes bit rot caused by recent > changes (your index autovacuum logging stuff). It has also been >

Re: 64-bit XIDs in deleted nbtree pages

2021-03-23 Thread Peter Geoghegan
On Tue, Mar 23, 2021 at 8:14 AM Masahiko Sawada wrote: > By this patch series, btree indexes became like hash indexes in terms > of amvacuumcleanup. We do an index scan at btvacuumcleanup() in the > two cases: metapage upgrading and more than 5% > deleted-but-not-yet-recycled pages. Both cases

Re: Add client connection check during the execution of the query

2021-03-23 Thread Thomas Munro
Going back a couple of years to something Konstantin said: On Sat, Aug 3, 2019 at 4:40 AM Konstantin Knizhnik wrote: > But I wonder why we can not perform just pool with POLLOUT flag and zero > timeout. > If OS detected closed connection, it should return POLLHUP, should not it? > I am not sure

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-23 Thread Julien Rouhaud
On Tue, Mar 23, 2021 at 12:27:10PM -0400, Bruce Momjian wrote: > > No, I was thinking of just doing a single commit. Should I do three > commits? I posted it as three patches since that is how it was posted > by the author, and reviewing is easier. It also will need a catversion > bump. Yes,

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-23 Thread Julien Rouhaud
On Tue, Mar 23, 2021 at 10:34:38AM -0400, Bruce Momjian wrote: > On Tue, Mar 23, 2021 at 02:36:27PM +0800, Julien Rouhaud wrote: > > > > Is that an oversight in ca3b37487be333a1d241dab1bbdd17a211a88f43, at least > > for > > non .po files? > > No, I don't think so. We don't change the Free

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-23 Thread Fujii Masao
On 2021/03/24 5:59, Tom Lane wrote: Alvaro Herrera writes: FATAL: the database system is starting up DETAIL: WAL is being applied to recover from a system crash. or DETAIL: The system is applying WAL to recover from a system crash. or DETAIL: The startup process is applying WAL to

Re: PostmasterIsAlive() in recovery (non-USE_POST_MASTER_DEATH_SIGNAL builds)

2021-03-23 Thread Fujii Masao
On 2021/03/23 14:49, Fujii Masao wrote: On 2021/03/23 10:52, Thomas Munro wrote: On Tue, Mar 23, 2021 at 2:44 PM Fujii Masao wrote: I found 0001 patch was committed in de829ddf23, and which added new wait event WalrcvExit. This name seems not consistent with other wait events. I'm

Re: New IndexAM API controlling index vacuum strategies

2021-03-23 Thread Peter Geoghegan
On Mon, Mar 22, 2021 at 6:40 AM Masahiko Sawada wrote: > I've looked at this 0001 patch and here are some review comments: > + * Since we might have to prune a second time here, the code is structured to > + * use a local per-page copy of the counters that caller accumulates. We add > + * our

Re: A reloption for partitioned tables - parallel_workers

2021-03-23 Thread David Rowley
On Fri, 19 Mar 2021 at 02:07, Amit Langote wrote: > Attached a new version rebased over c8f78b616, with the grouping > relation partitioning enhancements as a separate patch 0001. Sorry > about the delay. I had a quick look at this and wondered if the partitioned table's parallel workers

Re: pgsql: Move tablespace path re-creation from the makefiles to pg_regres

2021-03-23 Thread Michael Paquier
On Tue, Mar 23, 2021 at 12:50:29PM +0100, Christoph Berg wrote: > I'm working around the problem now by running the tests as user > "postgres", but does completely break in environments where users want > to run the testsuite from a separate compilation user but don't have root. > > Old code: >

Re: Support for NSS as a libpq TLS backend

2021-03-23 Thread Michael Paquier
On Wed, Mar 24, 2021 at 12:05:35AM +, Jacob Champion wrote: > The first database loaded by NSS_InitContext() becomes the "default" > database. This is what I'm currently hung up on. I can't figure out how > to get NSS to use the database that was loaded for the current > connection, so in my

Re: Support for NSS as a libpq TLS backend

2021-03-23 Thread Jacob Champion
On Tue, 2021-03-23 at 00:38 +0100, Daniel Gustafsson wrote: > This rebase also includes a fix for pgtls_init which was sent offlist by > Jacob. > The changes in pgtls_init can potentially be used to initialize the crypto > context for NSS to clean up this patch, Jacob is currently looking at

Re: pg_amcheck contrib application

2021-03-23 Thread Peter Geoghegan
On Tue, Mar 23, 2021 at 12:53 PM Peter Geoghegan wrote: > One of the advantages of this design is that we verify practically all > of the work involved in deleting an entire subtree up-front, inside > _bt_lock_subtree_parent(). It's clearly safe to back out of it if it > looks dicey. That's

Re: multi-install PostgresNode

2021-03-23 Thread Andrew Dunstan
On 3/23/21 6:36 PM, Michael Paquier wrote: > On Thu, Jan 28, 2021 at 10:19:57AM -0500, Andrew Dunstan wrote: >> +BEGIN >> +{ >> + >> +# putting this in a BEGIN block means it's run and checked by perl -c >> + >> + >> +# everything other than info and get_new_node that we need to

Re: Change default of checkpoint_completion_target

2021-03-23 Thread Bossart, Nathan
On 3/23/21, 12:19 PM, "Stephen Frost" wrote: > * Bossart, Nathan (bossa...@amazon.com) wrote: > > LGTM. I just have a few small wording suggestions. > > Agreed, those looked like good suggestions and so I've incorporated > them. > > Updated patch attached. Looks good! Nathan

Re: Add client connection check during the execution of the query

2021-03-23 Thread Zhihong Yu
Hi, In the description: Provide a new optional GUC that can be used to check whether the client connection has gone away periodically while running very long queries. I think moving 'periodically' to the vicinity of 'to check' would make the sentence more readable. +the operating

Re: multi-install PostgresNode

2021-03-23 Thread Michael Paquier
On Thu, Jan 28, 2021 at 10:19:57AM -0500, Andrew Dunstan wrote: > +BEGIN > +{ > + > +# putting this in a BEGIN block means it's run and checked by perl -c > + > + > +# everything other than info and get_new_node that we need to override. > +# they are all instance methods, so we can

Re: Minimal logical decoding on standbys

2021-03-23 Thread Fabrízio de Royes Mello
> > done in v13 attached. > All tests passed and everything looks good to me... just a final minor fix on regression tests: diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index b0e17d4e1d..961ec869a6 100644 --- a/src/test/regress/expected/rules.out +++

Re: Add client connection check during the execution of the query

2021-03-23 Thread Thomas Munro
On Tue, Mar 23, 2021 at 11:47 PM Thomas Munro wrote: > That leaves the thorny problem Tom mentioned at the top of this > thread[1]: this socket-level approach can be fooled by an 'X' sitting > in the socket buffer, if a client that did PQsendQuery() and then > PQfinish(). Or perhaps even by SSL

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-23 Thread Tom Lane
Alvaro Herrera writes: > FATAL: the database system is starting up > DETAIL: WAL is being applied to recover from a system crash. > or > DETAIL: The system is applying WAL to recover from a system crash. > or > DETAIL: The startup process is applying WAL to recover from a system crash. I

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Tom Lane
Jan Wieck writes: > Have we even reached a consensus yet on that doing it the way, my patch > is proposing, is the right way to go? Like that emitting BLOB TOC > entries into SECTION_DATA when in binary upgrade mode is a good thing? > Or that bunching all the SQL statements for creating the

Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch

2021-03-23 Thread Tom Lane
Peter Eisentraut writes: > On 19.03.21 21:06, Tom Lane wrote: >> I guess the immediate question is how much of a performance gap there >> is now between the float and numeric implementations. > Attached are my test script and the full output. OK ... I prefer to do this sort of timing in a way

Re: [PATCH] pg_permissions

2021-03-23 Thread Alvaro Herrera
On 2021-Mar-08, Joel Jacobson wrote: > $ dropuser test > dropuser: error: removal of role "test" failed: ERROR: role "test" cannot be > dropped because some objects depend on it > DETAIL: 1 object in database joel > > Hmmm. I wonder which 1 object that could be? BTW the easiest way to find

Re: [PATCH] pg_permissions

2021-03-23 Thread Alvaro Herrera
On 2021-Mar-23, Joel Jacobson wrote: > On Thu, Mar 11, 2021, at 08:00, Joel Jacobson wrote: > > 0004-pg_permissions-and-pg_ownerships.patch > > Having gotten some hands-on experience of these views for a while, > I notice I quite often want to check the ownerships/permissions > for some specific

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-23 Thread Alvaro Herrera
On 2021-Mar-24, Fujii Masao wrote: > On 2021/03/24 1:20, Alvaro Herrera wrote: > > Please note that PM_STARTUP mode is very very short-lived. It only > > starts happening when postmaster launches the startup process, and > > before the startup process begins WAL replay (as changed by > >

Re: [PATCH] pg_permissions

2021-03-23 Thread Joel Jacobson
On Thu, Mar 11, 2021, at 08:00, Joel Jacobson wrote: > 0004-pg_permissions-and-pg_ownerships.patch Having gotten some hands-on experience of these views for a while, I notice I quite often want to check the ownerships/permissions for some specific type of objects, or in some specific schema. The

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Jan Wieck
On 3/23/21 3:35 PM, Tom Lane wrote: Jan Wieck writes: The problem here is that pg_upgrade itself is invoking a shell again. It is not assembling an array of arguments to pass into exec*(). I'd be a happy camper if it did the latter. But as things are we'd have to add full shell escapeing for

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-23 Thread Fujii Masao
On 2021/03/24 1:20, Alvaro Herrera wrote: On 2021-Mar-23, James Coleman wrote: On Tue, Mar 23, 2021 at 1:46 AM Fujii Masao wrote: Therefore for now what we've not reached the consensus is what message should be logged at PM_STARTUP. I'm thinking it's better to log "the database system

Re: pg_amcheck contrib application

2021-03-23 Thread Peter Geoghegan
On Tue, Mar 23, 2021 at 12:44 PM Tom Lane wrote: > > I will make this change to HEAD soon, barring objections. > > +1. Not deleting the upper page seems better than the alternatives. FWIW it might also work that way as a holdover from the old page deletion algorithm. These days we decide

Re: pg_amcheck contrib application

2021-03-23 Thread Tom Lane
Peter Geoghegan writes: > That being said, I should make _bt_lock_subtree_parent() return false > and back out of page deletion without raising an error in the case > where we really cannot locate a valid downlink. We really ought to > soldier on when that happens, since we'll do that for a bunch

Re: pg_amcheck contrib application

2021-03-23 Thread Peter Geoghegan
On Tue, Mar 23, 2021 at 12:05 PM Robert Haas wrote: > Right, good point. But... does that really apply to > 005_opclass_damage.pl? I feel like with the kind of physical damage > we're doing in 003_check.pl, it makes a lot of sense to stop vacuum > from crashing headlong into that table. But, 005

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Tom Lane
Jan Wieck writes: > The problem here is that pg_upgrade itself is invoking a shell again. It > is not assembling an array of arguments to pass into exec*(). I'd be a > happy camper if it did the latter. But as things are we'd have to add > full shell escapeing for arbitrary strings. Surely we

Re: [PATCH] Allow multiple recursive self-references

2021-03-23 Thread Tom Lane
Denis Hirn writes: >> I am not at all sure what the standard says about such recursion [...] > as far as I know, the standard does not constraint the number of > self-references > of recursive common table expressions. However, I could be wrong here. As far as I can see, the spec flat-out

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Jan Wieck
On 3/23/21 2:59 PM, Tom Lane wrote: Jan Wieck writes: On 3/23/21 2:35 PM, Tom Lane wrote: If you're passing multiple options, that is --pg-dump-options "--foo=x --bar=y" it seems just horribly fragile. Lose the double quotes and suddenly --bar is a separate option to pg_upgrade itself, not

Re: pg_amcheck contrib application

2021-03-23 Thread Mark Dilger
> On Mar 23, 2021, at 12:05 PM, Robert Haas wrote: > > 005 is doing "logical" > damage rather than "physical" damage, and I don't see why autovacuum > should misbehave in that kind of case. In fact, the fact that > autovacuum can handle such cases is one of the selling points for the > whole

Re: Change default of checkpoint_completion_target

2021-03-23 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > LGTM. I just have a few small wording suggestions. Agreed, those looked like good suggestions and so I've incorporated them. Updated patch attached. Thanks! Stephen From 40a529bc0a129e90c9917c1a3df2297ac7f2e073 Mon Sep 17 00:00:00

Re: pg_amcheck contrib application

2021-03-23 Thread Robert Haas
On Thu, Mar 18, 2021 at 12:12 AM Tom Lane wrote: > Mark Dilger writes: > >> On Mar 16, 2021, at 12:52 PM, Robert Haas wrote: > >> Since we now know that shutting autovacuum off makes the problem go > >> away, I don't see a reason to commit 0001. We should fix pg_amcheck > >> instead, if, as

Re: Support for NSS as a libpq TLS backend

2021-03-23 Thread Stephen Frost
Greetings, * Daniel Gustafsson (dan...@yesql.se) wrote: > > On 22 Mar 2021, at 00:49, Stephen Frost wrote: > > Thanks for the review! Below is a partial response, I haven't had time to > address all your review comments yet but I wanted to submit a rebased patchset > directly since the current

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Tom Lane
Jan Wieck writes: > On 3/23/21 2:35 PM, Tom Lane wrote: >> If you're passing multiple options, that is >> --pg-dump-options "--foo=x --bar=y" >> it seems just horribly fragile. Lose the double quotes and suddenly >> --bar is a separate option to pg_upgrade itself, not part of the argument >> for

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Jan Wieck
On 3/23/21 2:35 PM, Tom Lane wrote: Jan Wieck writes: So the question remains, how do we name this? --pg-dump-options "" --pg-restore-options "" If you're passing multiple options, that is --pg-dump-options "--foo=x --bar=y" it seems just horribly fragile. Lose the

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-23 Thread Andres Freund
Hi, On 2021-03-23 15:50:46 +0900, Fujii Masao wrote: > This fact makes me wonder that if we collect the statistics about WAL writing > from walreceiver as we discussed in other thread, the stats collector should > be invoked at more earlier stage. IIUC walreceiver can be invoked before >

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Tom Lane
Jan Wieck writes: > So the question remains, how do we name this? > --pg-dump-options "" > --pg-restore-options "" If you're passing multiple options, that is --pg-dump-options "--foo=x --bar=y" it seems just horribly fragile. Lose the double quotes and suddenly --bar is a

Re: Change default of checkpoint_completion_target

2021-03-23 Thread Bruce Momjian
On Tue, Mar 23, 2021 at 06:24:07PM +, Bossart, Nathan wrote: > LGTM. I just have a few small wording suggestions. > > +completion overhead. Reducing this parameter is not recommended as > that > +causes the I/O from the checkpoint to have to complete faster, > resulting >

Re: WIP: BRIN multi-range indexes

2021-03-23 Thread Alvaro Herrera
On 2021-Mar-23, Tomas Vondra wrote: > FWIW there's yet another difference between the current BRIN opclass > definition, compared to what CREATE OPERATOR CLASS would do. Or more > precisely, how we'd define opfamily for the cross-type cases (integer, > float and timestamp cases). > > AFAICS we

Re: Change default of checkpoint_completion_target

2021-03-23 Thread Bossart, Nathan
LGTM. I just have a few small wording suggestions. +completion overhead. Reducing this parameter is not recommended as that +causes the I/O from the checkpoint to have to complete faster, resulting +in a higher I/O rate, while then having a period of less I/O between

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Bruce Momjian
On Tue, Mar 23, 2021 at 02:23:03PM -0400, Jan Wieck wrote: > On 3/23/21 2:06 PM, Bruce Momjian wrote: > > We have the postmaster which can pass arbitrary arguments to postgres > > processes using -o. > > Right, and -o is already taken in pg_upgrade for sending options to the old > postmaster. >

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Jan Wieck
On 3/23/21 2:06 PM, Bruce Momjian wrote: We have the postmaster which can pass arbitrary arguments to postgres processes using -o. Right, and -o is already taken in pg_upgrade for sending options to the old postmaster. What we are looking for are options for sending options to pg_dump and

Re: [PATCH] Allow multiple recursive self-references

2021-03-23 Thread Denis Hirn
Hey Pantelis, > I am not at all sure what the standard says about such recursion [...] as far as I know, the standard does not constraint the number of self-references of recursive common table expressions. However, I could be wrong here. > [...] but it looks like the two t's are treated in

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Bruce Momjian
On Tue, Mar 23, 2021 at 01:25:15PM -0400, Jan Wieck wrote: > On 3/23/21 10:56 AM, Bruce Momjian wrote: > > Would it be better to allow pg_upgrade to pass arbitrary arguments to > > pg_restore, instead of just these specific ones? > > > > That would mean arbitrary parameters to pg_dump as well as

Re: Disable WAL logging to speed up data loading

2021-03-23 Thread Stephen Frost
Greetings, * tsunakawa.ta...@fujitsu.com (tsunakawa.ta...@fujitsu.com) wrote: > From: Stephen Frost > > First- what are you expecting would actually happen during crash recovery in > > this specific case with your proposed new WAL level? > ... > > I'm not suggesting it's somehow more crash safe-

Re: WIP: BRIN multi-range indexes

2021-03-23 Thread Tomas Vondra
On 3/23/21 2:36 PM, Alvaro Herrera wrote: > On 2021-Mar-22, Tomas Vondra wrote: > >> I don't know what's the right fix, but it seems like this patch has >> nothing to do with it. If we want to move the opclasses into an >> extension, we can comment out that one (cidr/inet) case for now. > > I

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Jan Wieck
On 3/23/21 10:56 AM, Bruce Momjian wrote: On Tue, Mar 23, 2021 at 08:51:32AM -0400, Jan Wieck wrote: On 3/22/21 7:18 PM, Jan Wieck wrote: > On 3/22/21 5:36 PM, Zhihong Yu wrote: > > Hi, > > > > w.r.t. pg_upgrade_improvements.v2.diff. > > > > +       blobBatchCount = 0; > > +      

Re: Replication slot stats misgivings

2021-03-23 Thread Andres Freund
Hi, On 2021-03-23 23:37:14 +0900, Masahiko Sawada wrote: > On Tue, Mar 23, 2021 at 3:09 PM Amit Kapila wrote: > > > > On Mon, Mar 22, 2021 at 12:20 PM Masahiko Sawada > > wrote: > > > > > > On Mon, Mar 22, 2021 at 1:25 PM Masahiko Sawada > > > wrote: > > > > > > > > On Sat, Mar 20, 2021 at

Re: making update/delete of inheritance trees scale better

2021-03-23 Thread Tom Lane
Robert Haas writes: > I spent some time studying this patch this morning. As far as I can > see, 0001 is a relatively faithful implementation of the design Tom > proposed back in early 2019. I think it would be nice to either get > this committed or else decide that we don't want it and what

Re: proposal - psql - use pager for \watch command

2021-03-23 Thread Pavel Stehule
po 22. 3. 2021 v 13:13 odesílatel Thomas Munro napsal: > On Mon, Mar 22, 2021 at 5:10 PM Pavel Stehule > wrote: > > probably there will not be an issue inside ncurses - the most complex > part of get_event is polling of input sources - tty and some other. The > pspg should not to stop there on

Re: new release pspg

2021-03-23 Thread Pavel Stehule
Hi ne 21. 3. 2021 v 7:40 odesílatel Greg Stark napsal: > This is really cool. Now I just need to figure out how to > integrate it with using Emacs for my terminal. I still want to use > emacs enter and edit my queries but it would be cool to be able to hit > a key and launch an xterm and

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-23 Thread Bruce Momjian
On Tue, Mar 23, 2021 at 12:12:03PM -0300, Álvaro Herrera wrote: > On 2021-Mar-22, Bruce Momjian wrote: > > > --- a/doc/src/sgml/ref/explain.sgml > > +++ b/doc/src/sgml/ref/explain.sgml > > @@ -136,8 +136,10 @@ ROLLBACK; > >the output column list for each node in the plan tree,

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-23 Thread James Coleman
On Tue, Mar 23, 2021 at 12:34 PM Tom Lane wrote: > > Alvaro Herrera writes: > > However, for this one > > > + case CAC_NOTCONSISTENT: > > + if (EnableHotStandby) > > + ereport(FATAL, > > + (errcode(ERRCODE_CANNOT_CONNECT_NOW), > > +

Re: making update/delete of inheritance trees scale better

2021-03-23 Thread Robert Haas
On Wed, Mar 3, 2021 at 9:39 AM Amit Langote wrote: > Just noticed that a test added by the recent 927f453a941 fails due to > 0002. We no longer allow moving a row into a postgres_fdw partition > if it is among the UPDATE's result relations, whereas previously we > would if the UPDATE on that

Query about pg asynchronous processing support

2021-03-23 Thread ??????
Dear hacker: I am anundergraduatefrom Nanjing University. I use pgsql source code for my own development. During processing each sql query in function 'exec_simple_query', I'm going to add some extra functions such as index recommendation, which should be asynchronous in respect to the sql

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-23 Thread Tom Lane
Alvaro Herrera writes: > However, for this one > + case CAC_NOTCONSISTENT: > + if (EnableHotStandby) > + ereport(FATAL, > + (errcode(ERRCODE_CANNOT_CONNECT_NOW), > +errmsg("the database system is not accepting >

Re: [PATCH] Allow multiple recursive self-references

2021-03-23 Thread Pantelis Theodosiou
On Tue, Mar 23, 2021 at 1:03 PM Denis Hirn wrote: > > Hey everyone, > > As you know, Postgres currently supports SQL:1999 recursive common table > expressions, using WITH RECURSIVE. However, Postgres does not allow more > than > one recursive self-reference in the recursive term. This

Re: Handling of opckeytype / CREATE OPERATOR CLASS (bug?)

2021-03-23 Thread Tom Lane
Tomas Vondra writes: > On 3/23/21 6:15 AM, Tom Lane wrote: >> Digging in our git history, the rule about zero opckeytype dates to >> 2001 (f933766ba), which precedes our invention of polymorphic types >> in 2003 (somewhere around 730840c9b). So I'm pretty sure that that >> was a poor man's

Re: Change default of checkpoint_completion_target

2021-03-23 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Mon, Mar 22, 2021 at 01:11:00PM -0400, Stephen Frost wrote: > > Unless there's anything further on this, I'll plan to commit it tomorrow > > or Wednesday. > > Cool, looks fine to me. > > This version of the patch has forgotten to

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-23 Thread Alvaro Herrera
On 2021-Mar-23, James Coleman wrote: > On Tue, Mar 23, 2021 at 1:46 AM Fujii Masao > wrote: > > Therefore for now what we've not reached the consensus is what message > > should be logged at PM_STARTUP. I'm thinking it's better to log > > "the database system is starting up" in that case

Re: Add Nullif case for eval_const_expressions_mutator

2021-03-23 Thread Tom Lane
David Steele writes: > Peter, thoughts on the new patch in [1]? I'm not Peter, but I have a complaint about this bit: + if (!has_nonconst_input) + return ece_evaluate_expr(expr); That's not okay without a further check to see

Re: multi-install PostgresNode

2021-03-23 Thread Andrew Dunstan
On 1/13/21 7:25 AM, Daniel Gustafsson wrote: >> On 17 Dec 2020, at 22:37, Andrew Dunstan wrote: >> I've been giving some thought to $subject. The initial impetus is the >> promise I made to assist with testing of clients built with NSS against >> servers built with openssl, and vice versa. >

Re: Handling of opckeytype / CREATE OPERATOR CLASS (bug?)

2021-03-23 Thread Tomas Vondra
On 3/23/21 6:15 AM, Tom Lane wrote: > Tomas Vondra writes: >> On 3/23/21 3:13 AM, Tom Lane wrote: >>> Hm. Both catalogs.sgml and pg_opclass.h say specifically that >>> opckeytype should be zero if it's to be the same as the input >>> column type. I don't think just dropping the enforcement

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-03-23 Thread Tom Lane
I wrote: > Michael Paquier writes: >> One bisect later, the winner is: >> commit: 3d351d916b20534f973eda760cde17d96545d4c4 >> author: Tom Lane >> date: Sun, 30 Aug 2020 12:21:51 -0400 >> Redefine pg_class.reltuples to be -1 before the first VACUUM or ANALYZE. > I think that's an artifact. That

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-03-23 Thread Alvaro Herrera
I'm coming around to the idea that the fact that you can cancel the wait phase of DETACH CONCURRENTLY creates quite a disaster, and it's not easy to get away from it. The idea that REPEATABLE READ mode means that you now see detached partitions as if they were in normal condition, is completely

Re: 64-bit XIDs in deleted nbtree pages

2021-03-23 Thread Masahiko Sawada
On Mon, Mar 22, 2021 at 7:27 AM Peter Geoghegan wrote: > > On Wed, Mar 10, 2021 at 5:34 PM Peter Geoghegan wrote: > > Here is another bitrot-fix-only revision, v9. Just the recycling patch > > again. > > I committed the final nbtree page deletion patch just now -- the one > that attempts to

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-23 Thread Alvaro Herrera
On 2021-Mar-22, Bruce Momjian wrote: > --- a/doc/src/sgml/ref/explain.sgml > +++ b/doc/src/sgml/ref/explain.sgml > @@ -136,8 +136,10 @@ ROLLBACK; >the output column list for each node in the plan tree, schema-qualify >table and function names, always label variables in expressions

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-03-23 Thread Japin Li
On Mon, 22 Mar 2021 at 11:14, Bharath Rupireddy wrote: > On Sun, Mar 7, 2021 at 7:21 PM Japin Li wrote: >> Thank you point out this. Fixed it in v7 patch set. >> >> Please consider the v7 patch for futher review. > > Thanks for the patches. I just found the following behaviour with the > new

Re: tool to migrate database

2021-03-23 Thread Bruce Momjian
On Tue, Mar 23, 2021 at 09:49:57AM +0100, Joel Jacobson wrote: > I recently read an interesting real-life story from a very big company, Adyen, > and how they upgraded their 50 terrabyte PostgreSQL database. The article is > from 2018 but I still think it's relevant: > > https://medium.com/adyen/

Re: pgsql: Move tablespace path re-creation from the makefiles to pg_regres

2021-03-23 Thread Christoph Berg
Re: Michael Paquier > Move tablespace path re-creation from the makefiles to pg_regress > > Moving this logic into pg_regress fixes a potential failure with > parallel tests when pg_upgrade and the main regression test suite both > trigger the makefile rule that cleaned up testtablespace/ under >

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Bruce Momjian
On Tue, Mar 23, 2021 at 08:51:32AM -0400, Jan Wieck wrote: > On 3/22/21 7:18 PM, Jan Wieck wrote: > > On 3/22/21 5:36 PM, Zhihong Yu wrote: > > > Hi, > > > > > > w.r.t. pg_upgrade_improvements.v2.diff. > > > > > > +       blobBatchCount = 0; > > > +       blobInXact = false; > > > > > > The

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-03-23 Thread Alvaro Herrera
On 2021-Mar-23, Alvaro Herrera wrote: > So I was about ready to get these patches pushed, when I noticed that in > REPEATABLE READ isolation mode it is possible to insert rows violating > an FK referencing the partition that is being detached. I'm not sure > what is a good solution to this

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-03-23 Thread Tom Lane
Michael Paquier writes: > On Tue, Mar 23, 2021 at 04:12:01PM +0900, Michael Paquier wrote: >> It takes some time to initialize a cluster under CLOBBER_CACHE_ALWAYS, >> but the test is quick enough to reproduce. It would be good to bisect >> the origin point here as a first step. > One bisect

Re: Replication slot stats misgivings

2021-03-23 Thread Masahiko Sawada
On Tue, Mar 23, 2021 at 3:09 PM Amit Kapila wrote: > > On Mon, Mar 22, 2021 at 12:20 PM Masahiko Sawada > wrote: > > > > On Mon, Mar 22, 2021 at 1:25 PM Masahiko Sawada > > wrote: > > > > > > On Sat, Mar 20, 2021 at 3:52 AM Andres Freund wrote: > > > > > > > > - If max_replication_slots was

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-23 Thread Bruce Momjian
On Tue, Mar 23, 2021 at 02:36:27PM +0800, Julien Rouhaud wrote: > On Mon, Mar 22, 2021 at 08:43:40PM -0400, Bruce Momjian wrote: > > On Mon, Mar 22, 2021 at 05:17:15PM -0700, Zhihong Yu wrote: > > > Hi, > > > For queryjumble.c : > > > > > > + * Portions Copyright (c) 1996-2020, PostgreSQL Global

Re: Add Nullif case for eval_const_expressions_mutator

2021-03-23 Thread David Steele
On 1/19/21 8:16 PM, Hou, Zhijie wrote: Attatching v3 patch, please consider it for further review. Peter, thoughts on the new patch in [1]? -- -David da...@pgmasters.net [1] https://www.postgresql.org/message-id/ab53b3dbdbd6436f970f33b51ccd7dd3%40G08CNEXMBPEKD05.g08.fujitsu.local

Re: Minimal logical decoding on standbys

2021-03-23 Thread Fabrízio de Royes Mello
On Tue, Mar 23, 2021 at 10:18 AM Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > > LGTM too... Reviewing new changes now to move it forward and make this patch set ready for commiter review. > According to the feature LGTM and all tests passed. Documentation is also OK. Some minor

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-03-23 Thread Alvaro Herrera
So I was about ready to get these patches pushed, when I noticed that in REPEATABLE READ isolation mode it is possible to insert rows violating an FK referencing the partition that is being detached. I'm not sure what is a good solution to this problem. The problem goes like this: /* setup */

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2021-03-23 Thread Andrei Zubkov
Dear Kuroda, > I don't like the idea because such a column has no meaning for the > specific row. > I prefer storing timestamp if GetCurrentTimestamp() is cheap. I agree. New version attached. -- Andrei Zubkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company From

Re: [PATCH] Partial foreign key updates in referential integrity triggers

2021-03-23 Thread David Steele
On 3/18/21 9:52 AM, David Steele wrote: On 1/5/21 4:40 PM, Paul Martinez wrote: I've created a patch to better support referential integrity constraints when using composite primary and foreign keys. This patch allows creating a foreign key using the syntax: I previously proposed this

Re: WIP: BRIN multi-range indexes

2021-03-23 Thread Alvaro Herrera
On 2021-Mar-22, Tomas Vondra wrote: > I don't know what's the right fix, but it seems like this patch has > nothing to do with it. If we want to move the opclasses into an > extension, we can comment out that one (cidr/inet) case for now. I don't know what would be a good reason to define the

Re: Minimal logical decoding on standbys

2021-03-23 Thread Fabrízio de Royes Mello
On Tue, Mar 23, 2021 at 8:47 AM Drouvot, Bertrand wrote: > > I have one remark regarding the conflicts: > > The logical slots are dropped if a conflict is detected. > > But, if the slot is not active before being dropped (say wal_level is changed to < logical on master and a logical slot is not

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-23 Thread James Coleman
On Tue, Mar 23, 2021 at 1:46 AM Fujii Masao wrote: > > > > On 2021/03/23 3:59, James Coleman wrote: > > Are you saying we should only change the message for a single case: > > the case where we'd otherwise allow connections but EnableHotStandby > > is false? > > No. Let me clarify my opinion. > >

Re: PATCH: Attempt to make dbsize a bit more consistent

2021-03-23 Thread gkokolatos
‐‐‐ Original Message ‐‐‐ On Wednesday, March 17, 2021 6:35 AM, Michael Paquier wrote: > On Mon, Mar 15, 2021 at 03:10:59PM +0900, Michael Paquier wrote: > > > Anyway, as mentioned by other people upthread, I am not really > > convinced either that we should have more flavors of

[PATCH] Allow multiple recursive self-references

2021-03-23 Thread Denis Hirn
Hey everyone, As you know, Postgres currently supports SQL:1999 recursive common table expressions, using WITH RECURSIVE. However, Postgres does not allow more than one recursive self-reference in the recursive term. This restriction seems to be unnecessary. In this mail, I'd like to propose a

Re: Wired if-statement in gen_partprune_steps_internal

2021-03-23 Thread Amit Langote
Hi Ryan, On Tue, Mar 23, 2021 at 2:24 AM Ryan Lambert wrote: > Should the status of this patch be updated to ready for comitter to get in > line for Pg 14 deadline? Yes, I've done that. Thanks for the reminder. -- Amit Langote EDB: http://www.enterprisedb.com

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-23 Thread Jan Wieck
On 3/22/21 7:18 PM, Jan Wieck wrote: On 3/22/21 5:36 PM, Zhihong Yu wrote: Hi, w.r.t. pg_upgrade_improvements.v2.diff. +       blobBatchCount = 0; +       blobInXact = false; The count and bool flag are always reset in tandem. It seems variable blobInXact is not needed. You are right.

  1   2   >