Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-28 Thread Andres Freund
On 2022-11-28 13:08:57 +, Simon Riggs wrote: > On Sun, 27 Nov 2022 at 23:34, Nathan Bossart wrote: > > > Rather than explicitly use DEBUG1 everywhere I would have an > > > #define CUSTODIAN_LOG_LEVEL LOG > > > so we can run with it in LOG mode and then set it to DEBUG1 with a one > > >

Re: TAP output format in pg_regress

2022-11-28 Thread Andres Freund
On 2022-11-28 14:13:16 +0100, Daniel Gustafsson wrote: > > On 27 Nov 2022, at 11:22, Nikolay Shaplov wrote: > > В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel > > Gustafsson написал: > > > I wold suggest to use word immediate instead of noatexit. This will do the > > code

Re: Decouple last important WAL record LSN from WAL insert locks

2022-11-28 Thread Andres Freund
Hi, On 2022-11-28 11:42:19 +0530, Bharath Rupireddy wrote: > On Sun, Nov 27, 2022 at 2:43 AM Andres Freund wrote: > > On 2022-11-23 19:12:07 +0530, Bharath Rupireddy wrote: > > > While working on something else, I noticed that each WAL insert lock > > > tracks its own last important WAL record's

Re: Introduce a new view for checkpointer related stats

2022-11-28 Thread Robert Haas
On Tue, Nov 22, 2022 at 3:53 PM Andres Freund wrote: > I think we should consider deprecating the pg_stat_bgwriter columns but > leaving them in place for a few years. New stuff should only be added to > pg_stat_checkpointer, but we don't need to break old monitoring queries. I vote to just

Re: Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Robert Haas
On Mon, Nov 28, 2022 at 10:28 AM Simon Riggs wrote: > So we have these options > > 1. Removing the XactLockTableDelete() call in CommitSubTransaction(). > That releases lock waiters earlier than expected, which requires > pushups in XactLockTableWait() to cope with that (which are currently >

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-28 Thread Andrew Dunstan
On 2022-11-23 We 18:54, Nathan Bossart wrote: > On Wed, Nov 23, 2022 at 02:56:28PM -0500, Andrew Dunstan wrote: >> I have committed the first couple of these to get them out of the way. > Thanks! > >> But I think we need a bit of cleanup in the next patch. >> vacuum_is_relation_owner() looks

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Robert Haas
On Sat, Nov 26, 2022 at 4:08 AM Chris Travers wrote: > I didn't see any changes to pg_upgrade to make this change possible on > upgrade. Is that also outside of the scope of your patch set? If so how is > that continuity supposed to be ensured? The scheme is documented in their 0006 patch,

Re: Removing another gen_node_support.pl special case

2022-11-28 Thread Tom Lane
Peter Eisentraut writes: > On 27.11.22 02:39, Tom Lane wrote: >> I got confused about how we were managing EquivalenceClass pointers >> in the copy/equal infrastructure, and it took me awhile to remember >> that the reason it works is that gen_node_support.pl has hard-wired >> knowledge about

Re: Removing another gen_node_support.pl special case

2022-11-28 Thread Peter Eisentraut
On 27.11.22 02:39, Tom Lane wrote: I got confused about how we were managing EquivalenceClass pointers in the copy/equal infrastructure, and it took me awhile to remember that the reason it works is that gen_node_support.pl has hard-wired knowledge about that. I think that's something we'd be

Re: fixing CREATEROLE

2022-11-28 Thread Robert Haas
On Thu, Nov 24, 2022 at 2:41 AM wrote: > INHERITCREATEDROLES and SETCREATEDROLES behave much like DEFAULT > PRIVILEGES. What about something like: > > ALTER DEFAULT PRIVILEGES FOR alice > GRANT TO alice WITH INHERIT FALSE, SET TRUE, ADMIN TRUE > > The "abbreviated grant" is very much abbreviated,

[PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Nikita Malakhov
Hi hackers! While working on Pluggable TOAST we've detected a defective behavior on tables with large amounts of TOASTed data - queries freeze and DB stalls. Further investigation led us to the loop with GetNewOidWithIndex function call - when all available Oid already exist in the related TOAST

Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Simon Riggs
Issue in current XactLockTableWait, starting with 3c27944fb2141, discovered while reviewing https://commitfest.postgresql.org/40/3806/ Test demonstrating the problem is 001-isotest-tuplelock-subxact.v1.patch A narrative description of the issue follows: session1 - requests multiple nested

Re: Remove a unused argument from qual_is_pushdown_safe

2022-11-28 Thread Daniel Gustafsson
> On 28 Nov 2022, at 15:15, Tom Lane wrote: > My viewpoint is that this change is misguided. Even if the current > coding of qual_is_pushdown_safe doesn't happen to reference the > subquery, it might need to in future. If I understand the code correctly the variable has some value in terms of

Re: Remove a unused argument from qual_is_pushdown_safe

2022-11-28 Thread Tom Lane
Michael Paquier writes: > On Mon, Nov 28, 2022 at 11:54:45AM +0900, Michael Paquier wrote: >> On Fri, Nov 25, 2022 at 04:05:13PM +0800, Richard Guo wrote: > I wonder if we need to revise the comment atop qual_is_pushdown_safe() > too which says > > * rinfo is a restriction clause applying to the

Re: Reducing power consumption on idle servers

2022-11-28 Thread Tom Lane
Robert Haas writes: > I think we should remove those mentions. Otherwise the documentation > just collects mentions of an increasing number of things that are no > longer relevant. Yeah, I think the same. There will be a release-note entry, and I don't object to having something about it in

Re: Non-decimal integer literals

2022-11-28 Thread Peter Eisentraut
On 23.11.22 17:25, Dean Rasheed wrote: Taking a quick look, I noticed that there are no tests for negative values handled in the parser. Giving that a spin shows that make_const() fails to correctly identify the base of negative non-decimal integers in the T_Float case, causing it to fall

Re: TAP output format in pg_regress

2022-11-28 Thread Daniel Gustafsson
> On 27 Nov 2022, at 11:22, Nikolay Shaplov wrote: > В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel > Gustafsson написал: > I wold suggest to use word immediate instead of noatexit. This will do the > code much more sensible for me. I think noatexit is clearer since

Re: Perform streaming logical transactions by background workers and parallel apply

2022-11-28 Thread Amit Kapila
On Mon, Nov 28, 2022 at 12:49 PM Peter Smith wrote: > ... > > 17. > @@ -388,10 +401,9 @@ static inline void cleanup_subxact_info(void); > /* > * Serialize and deserialize changes for a toplevel transaction. > */ > -static void stream_cleanup_files(Oid subid, TransactionId xid); > static

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-28 Thread Simon Riggs
On Sun, 27 Nov 2022 at 23:34, Nathan Bossart wrote: > > Thanks for taking a look! > > On Thu, Nov 24, 2022 at 05:31:02PM +, Simon Riggs wrote: > > * not sure I believe that everything it does can always be aborted out > > of and shutdown - to achieve that you will need a > >

Re: Reducing power consumption on idle servers

2022-11-28 Thread Robert Haas
On Sun, Nov 27, 2022 at 6:57 PM Thomas Munro wrote: > The main documentation of pg_promote() etc now has "The parameter > promote_trigger_file has been removed" in the > places where the GUC was previously mentioned. Perhaps we should just > remove the mentions completely (it's somehow either

Re: Logical Replication Custom Column Expression

2022-11-28 Thread Stavros Koureas
Sure I understand and neither do I have good knowledge of what else could be influenced by such a change. If the value of the column is the subscriber name has no benefit to this idea of merging multiple upstreams with same primary keys, later you describe the "connection dbname", yes this could

Re: Perform streaming logical transactions by background workers and parallel apply

2022-11-28 Thread Amit Kapila
On Sun, Nov 27, 2022 at 9:43 AM houzj.f...@fujitsu.com wrote: > > Attach the new version patch which addressed all comments so far. > Few comments on v52-0001* 1. pa_free_worker() { ... + /* Free the worker information if the worker exited cleanly. */ + if

Re: Introduce a new view for checkpointer related stats

2022-11-28 Thread Bharath Rupireddy
On Sat, Nov 26, 2022 at 4:32 AM Andres Freund wrote: > Thanks Andres for reviewing. > > May I know what it means to deprecate pg_stat_bgwriter columns? > > Add a note to the docs saying that the columns will be removed. Done. > > Are > > you suggesting to add deprecation warnings to

Re: [PATCH] Check snapshot argument of index_beginscan and family

2022-11-28 Thread Aleksander Alekseev
Hi Pavel, Thanks for the feedback! > I think it's a nice catch and worth fixing. The one thing I don't > agree with is using asserts for handling the error that can appear > because most probably the server is built with assertions off and in > this case, there still will be a crash in this

Re: Another multi-row VALUES bug

2022-11-28 Thread Dean Rasheed
On Wed, 23 Nov 2022 at 18:56, Tom Lane wrote: > > I wonder if somehow we could just make one pass over > all the VALUES RTEs, and process each one as needed? The problem > is to identify the relevant target relation, I guess. > I have been thinking about that some more, but I think it would be

Re: Failed Assert in pgstat_assoc_relation

2022-11-28 Thread Ajin Cherian
On Mon, Nov 28, 2022 at 8:10 PM vignesh C wrote: > > Hi, > > While reviewing/testing one of the patches I found the following Assert: > #0 __pthread_kill_implementation (no_tid=0, signo=6, > threadid=139624429171648) at ./nptl/pthread_kill.c:44 > #1 __pthread_kill_internal (signo=6,

Re: [PATCH] Check snapshot argument of index_beginscan and family

2022-11-28 Thread Pavel Borisov
Hi, Alexander! > A colleague of mine (cc'ed) reported that he was able to pass a NULL > snapshot to index_beginscan() and it even worked to a certain degree. > > I took my toy extension [1] and replaced the argument with NULL as an > experiment: > > ``` > eax=# CREATE EXTENSION experiment; >

[PATCH] Check snapshot argument of index_beginscan and family

2022-11-28 Thread Aleksander Alekseev
Hi hackers, A colleague of mine (cc'ed) reported that he was able to pass a NULL snapshot to index_beginscan() and it even worked to a certain degree. I took my toy extension [1] and replaced the argument with NULL as an experiment: ``` eax=# CREATE EXTENSION experiment; CREATE EXTENSION eax=#

Re: Bug in row_number() optimization

2022-11-28 Thread Sergey Shinderuk
On 28.11.2022 03:23, David Rowley wrote: On Sat, 26 Nov 2022 at 05:19, Tom Lane wrote: Sergey Shinderuk writes: What about user-defined operators? I created my own <= operator for int8 which returns true on null input, and put it in a btree operator class. Admittedly, it's weird that (null

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-28 Thread Dilip Kumar
On Mon, Nov 28, 2022 at 1:46 PM shiy.f...@fujitsu.com wrote: > > Thanks for your patch. > > I saw that the patch added a check when selecting largest transaction, but in > addition to ReorderBufferCheckMemoryLimit(), the transaction can also be > streamed in ReorderBufferProcessPartialChange().

Failed Assert in pgstat_assoc_relation

2022-11-28 Thread vignesh C
Hi, While reviewing/testing one of the patches I found the following Assert: #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=139624429171648) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=139624429171648) at ./nptl/pthread_kill.c:78 #2

Re: Report roles in pg_upgrade pg_ prefix check

2022-11-28 Thread Daniel Gustafsson
> On 28 Nov 2022, at 02:18, Michael Paquier wrote: > > On Thu, Nov 24, 2022 at 12:31:09PM +0100, Daniel Gustafsson wrote: >> Looking at a recent pg_upgrade thread I happened to notice that the check for >> roles with a pg_ prefix only reports the error, not the roles it found. >> Other >>

Re: Understanding, testing and improving our Windows filesystem code

2022-11-28 Thread Thomas Munro
On Mon, Nov 28, 2022 at 8:58 PM Ian Lawrence Barwick wrote: > For my understanding, does this entry supersede the proposal in > https://commitfest.postgresql.org/40/3347/ ? I think so (Victor hasn't commented). Patch 0004 derives from Victor's patch and has him as primary author still, but I

Failed Assert while pgstat_unlink_relation

2022-11-28 Thread vignesh C
Hi, While reviewing/testing one of the patches I found the following Assert: #0 0x55c6312ba639 in pgstat_unlink_relation (rel=0x7fb73bcbac58) at pgstat_relation.c:161 #1 0x55c6312bbb5a in pgstat_relation_delete_pending_cb (entry_ref=0x55c6335563a8) at pgstat_relation.c:839 #2

Re: Logical Replication Custom Column Expression

2022-11-28 Thread Peter Smith
On Fri, Nov 25, 2022 at 9:43 PM Stavros Koureas wrote: > > Yes, if the property is on the subscription side then it should be applied > for all the tables that the connected publication is exposing. > So if the property is enabled you should be sure that this origin column > exists to all of

RE: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-28 Thread shiy.f...@fujitsu.com
On Sun, Nov 27, 2022 1:33 PM Dilip Kumar wrote: > > On Sat, Nov 26, 2022 at 12:15 PM Amit Kapila > wrote: > > > > On Fri, Nov 25, 2022 at 5:38 PM Amit Kapila > wrote: > > > > > > On Fri, Nov 25, 2022 at 1:35 PM Dilip Kumar > wrote: > > > > > > > > During DecodeCommit() for skipping a

Re: Remove a unused argument from qual_is_pushdown_safe

2022-11-28 Thread Richard Guo
On Mon, Nov 28, 2022 at 3:40 PM Michael Paquier wrote: > > On Fri, Nov 25, 2022 at 04:05:13PM +0800, Richard Guo wrote: > >> I wonder if we need to revise the comment atop qual_is_pushdown_safe() > >> too which says > >> > >> * rinfo is a restriction clause applying to the given subquery (whose

<    1   2