Re: ExecRTCheckPerms() and many prunable partitions

2022-11-29 Thread Amit Langote
On Wed, Nov 30, 2022 at 3:04 AM Alvaro Herrera wrote: > On 2022-Nov-29, Amit Langote wrote: > > > Maybe, we should have the following there so that the PlannedStmt's > > contents don't point into the Query? > > > > newperminfo = copyObject(perminfo); > > Hmm, I suppose if we want a separate RT

RE: Partial aggregates pushdown

2022-11-29 Thread fujii.y...@df.mitsubishielectric.co.jp
Hi Mr.Pyhalov. Thank you for comments. > I've looked through the patch. Overall I like this approach, but have > the following comments. > > 1) Why should we require partialaggfn for min()/max()/count()? We could > just use original functions for a lot of aggregates, and so it would be > possibl

Re: Reducing power consumption on idle servers

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 1:32 AM Simon Riggs wrote: > Re-attaching patch for bgwriter and walwriter, so it is clear this is > not yet committed. I'm just curious, and not suggesting that 60s wakeups are a problem for the polar ice caps, but why even time out at all? Are the latch protocols involv

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-29 Thread Nathan Bossart
On Tue, Nov 29, 2022 at 12:02:44PM +, Simon Riggs wrote: > The last important point for me is tests, in src/test/modules > probably. It might be possible to reuse the final state of other > modules' tests to test cleanup, or at least integrate a custodian test > into each module. Of course. I

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-29 Thread Nathan Bossart
I spent some more time on the prevent-unnecessary-wakeups patch for logical/worker.c that I've been alluding to in this thread, and I found a few more places where we depend on the worker periodically waking up. This seems to be a common technique, so I'm beginning to wonder whether these changes

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 5:10 PM Nathan Bossart wrote: > I spent some more time on the prevent-unnecessary-wakeups patch for > logical/worker.c that I've been alluding to in this thread, and I found a > few more places where we depend on the worker periodically waking up. This > seems to be a comm

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 5:23 PM Thomas Munro wrote: > On Wed, Nov 30, 2022 at 5:10 PM Nathan Bossart > wrote: > > I spent some more time on the prevent-unnecessary-wakeups patch for > > logical/worker.c that I've been alluding to in this thread, and I found a > > few more places where we depend

pg_dump bugs reported as pg_upgrade bugs

2022-11-29 Thread Bruce Momjian
FYI, you might wonder why so many bugs reported on pg_upgrade eventually are bugs in pg_dump. Well, of course, partly is it because pg_upgrade relies on pg_dump, but a bigger issue is that pg_upgrade will fail if pg_dump or its restoration generate _any_ errors. My guess is that many people are u

RE: wake up logical workers after ALTER SUBSCRIPTION

2022-11-29 Thread Hayato Kuroda (Fujitsu)
Dear Nathan, > I spent some more time on the prevent-unnecessary-wakeups patch for > logical/worker.c that I've been alluding to in this thread, and I found a > few more places where we depend on the worker periodically waking up. This > seems to be a common technique, so I'm beginning to wonder

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-29 Thread Nathan Bossart
On Wed, Nov 30, 2022 at 05:27:40PM +1300, Thomas Munro wrote: > On Wed, Nov 30, 2022 at 5:23 PM Thomas Munro wrote: >> On Wed, Nov 30, 2022 at 5:10 PM Nathan Bossart >> wrote: >> > I spent some more time on the prevent-unnecessary-wakeups patch for >> > logical/worker.c that I've been alluding t

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-29 Thread Nathan Bossart
On Tue, Nov 29, 2022 at 07:56:53PM -0800, Nathan Bossart wrote: > On Tue, Nov 29, 2022 at 12:02:44PM +, Simon Riggs wrote: >> The last important point for me is tests, in src/test/modules >> probably. It might be possible to reuse the final state of other >> modules' tests to test cleanup, or a

Re: pg_dump bugs reported as pg_upgrade bugs

2022-11-29 Thread Tom Lane
Bruce Momjian writes: > FYI, you might wonder why so many bugs reported on pg_upgrade eventually > are bugs in pg_dump. Well, of course, partly is it because pg_upgrade > relies on pg_dump, but a bigger issue is that pg_upgrade will fail if > pg_dump or its restoration generate _any_ errors. My

Re: Strange failure on mamba

2022-11-29 Thread Andres Freund
Hi, On 2022-11-29 20:44:34 -0500, Tom Lane wrote: > Thanks to commit 51b5834cd I've now been able to capture some info > from mamba's last couple of failures [1][2]. Sure enough, what is > happening is that postmaster children are getting stuck in recursive > rtld symbol resolution. A couple of

Re: Non-decimal integer literals

2022-11-29 Thread David Rowley
On Wed, 23 Nov 2022 at 22:19, John Naylor wrote: > > > On Wed, Nov 23, 2022 at 3:54 PM David Rowley wrote: > > > > Going by [1], clang will actually use multiplication by 16 to > > implement the former. gcc is better and shifts left by 4, so likely > > won't improve things for gcc. It seems wort

Re: Tests for psql \g and \o

2022-11-29 Thread Michael Paquier
On Wed, Nov 23, 2022 at 09:18:57PM +0100, Daniel Verite wrote: > PFA a new patch addressing these issues. Thanks, the tests part of the main regression test suite look good to me, so I have applied them after fixing a few typos and tweaking the style of the test. Regarding the tests with pipes, I

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-29 Thread John Naylor
There are a few things up in the air, so I'm coming back to this list to summarize and add a recent update: On Mon, Nov 14, 2022 at 7:59 PM John Naylor wrote: > > - See how much performance we actually gain from tagging the node kind. Needs a benchmark that has enough branch mispredicts and L2/3

Re: Strange failure on mamba

2022-11-29 Thread Tom Lane
Andres Freund writes: > On 2022-11-29 20:44:34 -0500, Tom Lane wrote: >> Backtrace stopped: frame did not save the PC > Do you have any idea why the stack can't be unwound further here? Is it > possibly indicative of a corrupted stack? I guess we'd need to dig into > the netbsd libc code :( I di

Re: Strange failure on mamba

2022-11-29 Thread Andres Freund
Hi, On 2022-11-29 20:44:34 -0500, Tom Lane wrote: > It's also strange that we're apparently running with signals enabled > whereever it is that rtld_bind is getting called from. Could it be that > sigaction is failing to install the requested signal mask, so that one > postmaster signal handler i

Re: Printing backtrace of postgres processes

2022-11-29 Thread Bharath Rupireddy
On Fri, Nov 11, 2022 at 6:04 PM Bharath Rupireddy wrote: > > On Fri, Nov 11, 2022 at 7:59 AM Kyotaro Horiguchi > wrote: > > > > At Thu, 10 Nov 2022 15:56:35 +0530, Bharath Rupireddy > > wrote in > > > On Mon, Apr 18, 2022 at 9:10 AM vignesh C wrote: > > > > > > > > The attached v21 patch has t

Re: Strange failure on mamba

2022-11-29 Thread Tom Lane
Andres Freund writes: > On 2022-11-29 20:44:34 -0500, Tom Lane wrote: >> It's also strange that we're apparently running with signals enabled >> whereever it is that rtld_bind is getting called from. Could it be that >> sigaction is failing to install the requested signal mask, so that one >> pos

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-29 Thread Simon Riggs
On Wed, 30 Nov 2022 at 03:56, Nathan Bossart wrote: > > On Tue, Nov 29, 2022 at 12:02:44PM +, Simon Riggs wrote: > > The last important point for me is tests, in src/test/modules > > probably. It might be possible to reuse the final state of other > > modules' tests to test cleanup, or at leas

Re: Strange failure on mamba

2022-11-29 Thread Andres Freund
Hi, On 2022-11-30 00:55:42 -0500, Tom Lane wrote: > Andres Freund writes: > > What libraries is postgres linked against? I don't know whether -z now only > > affects the "top-level" dependencies of postgres, or also the dependencies > > of > > shared libraries that haven't been built with -z now

Re: New docs chapter on Transaction Management and related changes

2022-11-29 Thread Peter Eisentraut
On 30.11.22 02:51, Bruce Momjian wrote: On Fri, Nov 18, 2022 at 02:11:50PM -0500, Bruce Momjian wrote: On Mon, Nov 7, 2022 at 11:04:46PM +0100, Laurenz Albe wrote: On Sat, 2022-11-05 at 10:08 +, Simon Riggs wrote: Agreed; new compilation patch attached, including mine and then Robert's su

Re: Introduce a new view for checkpointer related stats

2022-11-29 Thread Bharath Rupireddy
On Wed, Nov 30, 2022 at 6:01 AM Andres Freund wrote: > > Hi, > > On 2022-11-28 12:58:48 -0500, Robert Haas wrote: > > On Tue, Nov 22, 2022 at 3:53 PM Andres Freund wrote: > > > I think we should consider deprecating the pg_stat_bgwriter columns but > > > leaving them in place for a few years. New

Re: Reducing power consumption on idle servers

2022-11-29 Thread Simon Riggs
On Wed, 30 Nov 2022 at 03:50, Thomas Munro wrote: > > On Wed, Nov 30, 2022 at 1:32 AM Simon Riggs > wrote: > > Re-attaching patch for bgwriter and walwriter, so it is clear this is > > not yet committed. > > I'm just curious, and not suggesting that 60s wakeups are a problem > for the polar ice c

Re: ExecRTCheckPerms() and many prunable partitions

2022-11-29 Thread Amit Langote
On Wed, Nov 30, 2022 at 11:56 AM Amit Langote wrote: > On Wed, Nov 30, 2022 at 3:04 AM Alvaro Herrera > wrote: > > On 2022-Nov-29, Amit Langote wrote: > > > > > Maybe, we should have the following there so that the PlannedStmt's > > > contents don't point into the Query? > > > > > > newpermi

Re: Perform streaming logical transactions by background workers and parallel apply

2022-11-29 Thread Amit Kapila
On Tue, Nov 29, 2022 at 6:03 PM Amit Kapila wrote: > > 12. Apart from the above, I have made a few changes in the comments > and some other cosmetic changes in the attached patch. > I have made some additional changes in the comments at various places. Kindly check the attached and let me know yo

Re: Support tls-exporter as channel binding for TLSv1.3

2022-11-29 Thread Michael Paquier
On Fri, Oct 14, 2022 at 11:00:10AM +0900, Michael Paquier wrote: > Oh, okay. That's an argument in favor of not doing that, then. > Perhaps we'd better revisit the introduction of tls-exporter once we > know more about all that, and it looks like we would need a way to be > able to negotiate which

Re: Slow standby snapshot

2022-11-29 Thread Simon Riggs
On Tue, 29 Nov 2022 at 20:46, Tom Lane wrote: > > I wrote: > > That seems like a fairly bad idea: it will add extra contention > > on ProcArrayLock, and I see no real strong argument that the path > > can't get traversed often enough for that to matter. It would > > likely be better for KnownAssi

Re: Query Jumbling for CALL and SET utility statements

2022-11-29 Thread Michael Paquier
On Wed, Oct 12, 2022 at 09:13:20AM +0900, Michael Paquier wrote: > Thanks for mentioning that. With an automated way to generate this > code, cursors would be handled, at the advantage of making sure that > no fields are missing in the jumbled structures (is_local was missed > for example on SET).

Re: Strange failure on mamba

2022-11-29 Thread Andres Freund
Hi, On 2022-11-29 22:31:50 -0800, Andres Freund wrote: > On 2022-11-30 00:55:42 -0500, Tom Lane wrote: > > Andres Freund writes: > > > What libraries is postgres linked against? I don't know whether -z now > > > only > > > affects the "top-level" dependencies of postgres, or also the > > > depe

Re: pg_get_constraintdef: Schema qualify foreign tables unless pretty printing is enabled

2022-11-29 Thread Michael Paquier
On Wed, Oct 12, 2022 at 02:19:25PM +0900, Michael Paquier wrote: > Lukas, please note that this patch is waiting for your input for a few > weeks now. Could you reply to the reviews provided? This has stalled for six weeks, so I have marked the patch as returned with feedback. -- Michael signat

Re: real/float example for testlibpq3

2022-11-29 Thread Michael Paquier
On Wed, Nov 09, 2022 at 12:47:23PM -0800, Mark Wong wrote: > I'll take a stab at adding a new chapter and share how that looks. This thread has stalled for three weeks, so I have marked it as returned with feedback. Once you have a new patch, please feel free to submit it. -- Michael signature.

Re: New docs chapter on Transaction Management and related changes

2022-11-29 Thread Simon Riggs
On Wed, 30 Nov 2022 at 01:51, Bruce Momjian wrote: > Thanks to Simon for getting this important > information in our docs, and for the valuable feedback from others that > made this even better. And thanks to you for pulling that all together Bruce. -- Simon Riggshttp://www.Ent

Re: Making pg_rewind faster

2022-11-29 Thread Michael Paquier
On Thu, Oct 06, 2022 at 04:08:45PM +0900, Michael Paquier wrote: > file_entry_t has an entry to track if a file is a relation file. I > think that it would be much cleaner to track if we are handling a WAL > segment when inserting an entry in insert_filehash_entry(), so > isrelfile could be replac

pg_dump: Remove "blob" terminology

2022-11-29 Thread Peter Eisentraut
For historical reasons, pg_dump refers to large objects as "BLOBs". This term is not used anywhere else in PostgreSQL, and it also means something different in the SQL standard and other SQL systems. This patch renames internal functinos, code comments, documentation, etc. to use the "large ob

Re: pgsql: Revoke PUBLIC CREATE from public schema, now owned by pg_databas

2022-11-29 Thread Noah Misch
On Tue, Nov 29, 2022 at 02:22:59PM -0500, Robert Haas wrote: > On Fri, Sep 10, 2021 at 2:39 AM Noah Misch wrote: > > Revoke PUBLIC CREATE from public schema, now owned by pg_database_owner. > > > > This switches the default ACL to what the documentation has recommended > > since CVE-2018-1058. Up

Re: Introduce a new view for checkpointer related stats

2022-11-29 Thread Drouvot, Bertrand
Hi, On 11/30/22 7:34 AM, Bharath Rupireddy wrote: On Wed, Nov 30, 2022 at 6:01 AM Andres Freund wrote: Hi, On 2022-11-28 12:58:48 -0500, Robert Haas wrote: On Tue, Nov 22, 2022 at 3:53 PM Andres Freund wrote: I think we should consider deprecating the pg_stat_bgwriter columns but leaving

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-29 Thread Masahiko Sawada
On Fri, Nov 25, 2022 at 5:00 PM John Naylor wrote: > > On Thu, Nov 24, 2022 at 9:54 PM Masahiko Sawada wrote: > > > > So it seems that there are two candidates of rt_node structure: (1) > > all nodes except for node256 are variable-size nodes and use pointer > > tagging, and (2) node32 and node12

Re: Large Pages and Super Pages for PostgreSQL

2022-11-29 Thread ZHU XIAN WEN
Hi Thomas Thank you very much for the work. I just got latest FreeBSD 13.1 environment, and I'm going to test and verify it. so would you please rebase latest patch? best wishes Tony On 2022/11/8 06:59, Thomas Munro wrote: On Sun, Jan 16, 2022 at 8:32 PM Thomas Munro wrote: On Sun, Jan

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-29 Thread Masahiko Sawada
On Fri, Nov 25, 2022 at 6:47 PM John Naylor wrote: > > > > On Thu, Nov 24, 2022 at 9:54 PM Masahiko Sawada wrote: > > > > [v11] > > There is one more thing that just now occurred to me: In expanding the use of > size classes, that makes rebasing and reworking the shared memory piece more > work

Re: psql - factor out echo code

2022-11-29 Thread Michael Paquier
On Sun, Jul 24, 2022 at 10:23:39PM +0200, Pavel Stehule wrote: > I had just one question - with this patch, the format of output of modes > ECHO ALL and ECHO QUERIES will be different, and that can be a little bit > messy. On second hand, the prefix --QUERY can be disturbing in echo queries > mode.

<    1   2