Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-02-12 Thread Masahiko Sawada
On Sat, Feb 11, 2023 at 2:33 PM John Naylor wrote: > > I didn't get any closer to radix-tree regression, Me neither. It seems that in v26, inserting chunks into node-32 is slow but needs more analysis. I'll share if I found something interesting. > but I did find some inefficiencies in

Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry

2023-02-12 Thread Kyotaro Horiguchi
At Mon, 13 Feb 2023 08:09:50 +0100, "Drouvot, Bertrand" wrote in >> I think this is useful beyond being able to generate those functions >> with >> macros. The fact that we had to deal with transactional code in >> pgstatfuncs.c >> meant that a lot of the relevant itnernals had to be exposed

Re: recovery modules

2023-02-12 Thread Michael Paquier
On Thu, Feb 09, 2023 at 02:48:26PM -0800, Nathan Bossart wrote: > On Thu, Feb 09, 2023 at 12:18:55PM -0800, Andres Freund wrote: >>> >>> -typedef bool (*ArchiveCheckConfiguredCB) (void); >>> +typedef bool (*ArchiveCheckConfiguredCB) (ArchiveModuleState *state); >>> >>> >>> If true is

Re: Making Vars outer-join aware

2023-02-12 Thread Richard Guo
On Mon, Feb 13, 2023 at 7:58 AM Justin Pryzby wrote: > The patch broke this query: > > select from pg_inherits inner join information_schema.element_types > right join (select from pg_constraint as sample_2) on true > on false, lateral (select scope_catalog, inhdetachpending from >

Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry

2023-02-12 Thread Drouvot, Bertrand
Hi, On 2/10/23 10:46 PM, Andres Freund wrote: Hi, On 2023-02-09 11:38:18 +0100, Drouvot, Bertrand wrote: Please find attached a patch proposal for $SUBJECT. The idea has been raised in [1] by Andres: it would allow to simplify even more the work done to generate pg_stat_get_xact*()

Re: Introduce a new view for checkpointer related stats

2023-02-12 Thread Bharath Rupireddy
On Fri, Feb 10, 2023 at 10:00 PM Bharath Rupireddy wrote: > > Thanks. FWIW, I've attached the patch introducing pg_stat_checkpointer > as 0003 here. > > Please review the attached v7 patch set. Needed a rebase. Please review the attached v8 patch set. -- Bharath Rupireddy PostgreSQL

Force testing of query jumbling code in TAP tests

2023-02-12 Thread Michael Paquier
Hi all, As mentioned two times on this thread, there is not much coverage for the query jumbling code, even if it is in core: https://www.postgresql.org/message-id/Y5BHOUhX3zTH/i...@paquier.xyz Ths issue is that we have the options to enable it, but only pg_stat_statements is able to enable and

Re: Support logical replication of DDLs

2023-02-12 Thread Peter Smith
On Sat, Feb 11, 2023 at 3:31 AM vignesh C wrote: > > On Fri, 10 Feb 2023 at 21:50, vignesh C wrote: > > The attached v68 version patch has the changes for the same. > > I was not sure if we should support ddl replication of > create/alter/drop subscription commands as there might be some data >

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-02-12 Thread Bharath Rupireddy
On Sun, Feb 12, 2023 at 11:01 PM Andres Freund wrote: > > On 2023-02-12 19:59:00 +0530, Bharath Rupireddy wrote: > > Thanks for looking at it. We know that we don't change the zbuffer in > > the function, so can we avoid static const and have just a static > > variable, like the attached > >

Re: Support logical replication of DDLs

2023-02-12 Thread Zheng Li
On Fri, Feb 10, 2023 at 11:31 AM vignesh C wrote: > > On Fri, 10 Feb 2023 at 21:50, vignesh C wrote: > > The attached v68 version patch has the changes for the same. > > I was not sure if we should support ddl replication of > create/alter/drop subscription commands as there might be some data >

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-12 Thread Takamichi Osumi (Fujitsu)
Hi, Horiguchi-san On Monday, February 13, 2023 10:26 AM Kyotaro Horiguchi wrote: > At Fri, 10 Feb 2023 10:34:49 +0530, Amit Kapila > wrote in > > On Fri, Feb 10, 2023 at 10:11 AM Amit Kapila > wrote: > > > > > > On Fri, Feb 10, 2023 at 6:27 AM Kyotaro Horiguchi > > > wrote: > > > > We have

Re: Exit walsender before confirming remote flush in logical replication

2023-02-12 Thread Peter Smith
Here are some comments for patch v7-0002. == Commit Message 1. This commit extends START_REPLICATION to accept SHUTDOWN_MODE clause. It is currently implemented only for logical replication. ~ "to accept SHUTDOWN_MODE clause." --> "to accept a SHUTDOWN_MODE clause." ==

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-02-12 Thread Michael Paquier
On Sun, Feb 12, 2023 at 09:31:36AM -0800, Andres Freund wrote: > Another thing: I think we should either avoid iterating over all the IOVs if > we don't need them, or, even better, initialize the array as a constant, once. Where you imply that we'd still use memset() once on iov[PG_IOV_MAX],

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-02-12 Thread Michael Paquier
On Mon, Feb 13, 2023 at 11:27:58AM +0900, Kyotaro Horiguchi wrote: > I think currently the output by --describe-config can be used only for > consulting while editing a (possiblly broken) config file. Thus I > think it's no use showing GIC_DISALLOW_IN_FILE items there unless we > use

Re: Exit walsender before confirming remote flush in logical replication

2023-02-12 Thread Amit Kapila
On Mon, Feb 13, 2023 at 7:26 AM Kyotaro Horiguchi wrote: > > At Fri, 10 Feb 2023 12:40:43 +, "Hayato Kuroda (Fujitsu)" > wrote in > > Dear Amit, > > > > > Can't we have this option just as a bool (like shutdown_immediate)? > > > Why do we want to keep multiple modes? > > > > Of course we

Re: Rework LogicalOutputPluginWriterUpdateProgress

2023-02-12 Thread Amit Kapila
On Sat, Feb 11, 2023 at 3:04 AM Andres Freund wrote: > > > One difference I see with the patch is that I think we will end up > > sending keepalive for empty prepared transactions even though we don't > > skip sending begin/prepare messages for those. > > With the proposed approach we reliably

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-02-12 Thread Kyotaro Horiguchi
At Fri, 10 Feb 2023 16:43:15 +0900, Michael Paquier wrote in > On Wed, Feb 08, 2023 at 09:42:13PM -0500, Tom Lane wrote: > > Hm. On the one hand, if it is in fact not in postgresql.conf.sample, > > then that flag should be set for sure. OTOH I see that that flag > > isn't purely

Re: Exit walsender before confirming remote flush in logical replication

2023-02-12 Thread Kyotaro Horiguchi
At Fri, 10 Feb 2023 12:40:43 +, "Hayato Kuroda (Fujitsu)" wrote in > Dear Amit, > > > Can't we have this option just as a bool (like shutdown_immediate)? > > Why do we want to keep multiple modes? > > Of course we can use boolean instead, but current style is motivated by the > post[1].

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-12 Thread Kyotaro Horiguchi
At Fri, 10 Feb 2023 10:34:49 +0530, Amit Kapila wrote in > On Fri, Feb 10, 2023 at 10:11 AM Amit Kapila wrote: > > > > On Fri, Feb 10, 2023 at 6:27 AM Kyotaro Horiguchi > > wrote: > > > We have suffered this kind of feedback silence many times. Thus I > > > don't want to rely on luck here. I

Re: [PATCH] Add pretty-printed XML output option

2023-02-12 Thread Peter Smith
Something is misbehaving. Using the latest HEAD, and before applying the v6 patch, 'make check' is working OK. But after applying the v6 patch, some XML regression tests are failing because the DETAIL part of the message indicating the wrong syntax position is not getting displayed. Not only for

Re: Generating code for query jumbling through gen_node_support.pl

2023-02-12 Thread Michael Paquier
On Fri, Feb 10, 2023 at 04:40:08PM -0500, Tom Lane wrote: > v2 looks good to me as far as it goes. Thanks. I have applied that after an extra lookup. > I agree these other questions deserve a separate look. Okay, I may be able to come back to that. Another point is that we need to do a better

Re: Making Vars outer-join aware

2023-02-12 Thread Justin Pryzby
On Mon, Jan 23, 2023 at 03:38:06PM -0500, Tom Lane wrote: > Richard, are you planning to review this any more? I'm getting > a little antsy to get it committed. For such a large patch, > it's surprising it's had so few conflicts to date. The patch broke this query: select from pg_inherits

Re: ExecRTCheckPerms() and many prunable partitions (sqlsmith)

2023-02-12 Thread Justin Pryzby
On Tue, Nov 29, 2022 at 10:37:56PM +0900, Amit Langote wrote: > 0002 contains changes that has to do with changing how we access > checkAsUser in some foreign table planning/execution code sites. > Thought it might be better to describe it separately too. This was committed as 599b33b94: Stop

Re: run pgindent on a regular basis / scripted manner

2023-02-12 Thread Tom Lane
Andrew Dunstan writes: > On 2023-02-12 Su 11:24, Tom Lane wrote: >> It seems like "indent the whole tree" is about to become a minority >> use-case. Maybe instead of continuing to privilege that case, we >> should say that it's invoked by some new switch like --all-files, >> and without that

Re: run pgindent on a regular basis / scripted manner

2023-02-12 Thread Justin Pryzby
On Sun, Feb 12, 2023 at 11:24:14AM -0500, Tom Lane wrote: > Andrew Dunstan writes: > > ... then you could do > >     pgindent `git dirty` > > The only danger would be if there were no dirty files. Maybe we need a > > switch to inhibit using the current directory if there are no command > >

Re: run pgindent on a regular basis / scripted manner

2023-02-12 Thread Andrew Dunstan
On 2023-02-12 Su 11:24, Tom Lane wrote: Andrew Dunstan writes: ... then you could do     pgindent `git dirty` The only danger would be if there were no dirty files. Maybe we need a switch to inhibit using the current directory if there are no command line files. It seems like "indent the

Re: pgindent vs variable declaration across multiple lines

2023-02-12 Thread Tom Lane
Now that pg_bsd_indent is in our tree, we can format this as a patch against Postgres sources. I'll stick it in the March CF so we don't forget about it. regards, tom lane diff --git a/src/tools/pg_bsd_indent/args.c b/src/tools/pg_bsd_indent/args.c index

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-02-12 Thread Andres Freund
Hi, On 2023-02-12 19:59:00 +0530, Bharath Rupireddy wrote: > Thanks for looking at it. We know that we don't change the zbuffer in > the function, so can we avoid static const and have just a static > variable, like the attached > v1-0001-Use-static-variable-to-avoid-memset-calls-in-pg_p.patch?

Re: run pgindent on a regular basis / scripted manner

2023-02-12 Thread Tom Lane
Andrew Dunstan writes: > ... then you could do >     pgindent `git dirty` > The only danger would be if there were no dirty files. Maybe we need a > switch to inhibit using the current directory if there are no command > line files. It seems like "indent the whole tree" is about to become a

Re: Transparent column encryption

2023-02-12 Thread Mark Dilger
> On Feb 11, 2023, at 1:54 PM, Mark Dilger wrote: > > Here are some observations I should mention, src/sgml/html/libpq-exec.html needs clarification: > paramFormats[]Specifies whether parameters are text (put a zero in the array > entry for the corresponding parameter) or binary (put a

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-02-12 Thread Bharath Rupireddy
On Sun, Feb 12, 2023 at 4:14 AM Andres Freund wrote: > > > +ssize_t > > +pg_pwrite_zeros(int fd, size_t size) > > +{ > > + PGAlignedBlock zbuffer; > > + size_t zbuffer_sz; > > + struct ioveciov[PG_IOV_MAX]; > > + int blocks; > > + size_t remaining_size = 0;

Re: run pgindent on a regular basis / scripted manner

2023-02-12 Thread Andrew Dunstan
On 2023-02-10 Fr 10:21, Andrew Dunstan wrote: On 2023-02-10 Fr 04:25, Jelte Fennema wrote: Ah yes, I had seen that when I read the initial --commit patch but then forgot about it when the flag didn't work at all when I tried it. Attached is a patch that fixes the issue. And also implements

Re: Making aggregate deserialization (and WAL receive) functions slightly faster

2023-02-12 Thread David Rowley
On Sun, 12 Feb 2023 at 19:39, Tom Lane wrote: > I find this patch horribly dangerous. I see LogicalRepApplyLoop() does something similar with a StringInfoData. Maybe it's just scarier having an external function in stringinfo.c which does this as having it increases the chances of someone using

Re: Sort optimizations: Making in-memory sort cache-aware

2023-02-12 Thread Ankit Kumar Pandey
On 12/02/23 01:59, Andres Freund wrote: However, tuplesort.c completely breaks that for > 1 column sorts. While spatial locality for accesses to the ->memtuples array is decent during sorting, due to qsort's subdividing of the problem, the locality for access to the tuples is *awful*.