Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-09-01 Thread Amit Kapila
On Thu, Aug 29, 2024 at 11:31 AM Bharath Rupireddy wrote: > > Thanks for looking into this. > > On Mon, Aug 26, 2024 at 4:35 PM Amit Kapila wrote: > > > > Few comments on 0001: > > 1. > > @@ -651,6 +651,13 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid > > > > + /* > > + * Skip the sync if

Invalid Assert while validating REPLICA IDENTITY?

2024-09-01 Thread Dilip Kumar
While working on some other code I noticed that in FindReplTupleInLocalRel() there is an assert [1] that seems to be passing IndexRelation to GetRelationIdentityOrPK() whereas it should be passing normal relation. [1] if (OidIsValid(localidxoid)) { #ifdef USE_ASSERT_CHECKING Relation idxrel =

Re: Track the amount of time waiting due to cost_delay

2024-09-01 Thread Bertrand Drouvot
Hi, On Tue, Aug 20, 2024 at 12:48:29PM +, Bertrand Drouvot wrote: > As it looks like we have a consensus not to wait on [0] (as reducing the > number > of interrupts makes sense on its own), then please find attached v4, a rebase > version (that also makes clear in the doc that that new field

Regarding canditate_restart_lsn in logical decoding.

2024-09-01 Thread reddy manjunath
Hi, I have created my own output plugin for logical decoding. I am storing decoded data in Apache kafka via pg_recvlogical utility. Using pg_recvlogical I am updating confirmed_flush_lsn of slot based on the value that I'm storing in kafka,this is done for every 10 secs. In case of walsender shutd

Re: long-standing data loss bug in initial sync of logical replication

2024-09-01 Thread Amit Kapila
On Fri, Aug 30, 2024 at 3:06 PM Shlok Kyal wrote: > > Next I am planning to test solely on the logical decoding side and > will share the results. > Thanks, the next set of proposed tests makes sense to me. It will also be useful to generate some worst-case scenarios where the number of invalidat

Re: DOCS - pg_replication_slot . Fix the 'inactive_since' description

2024-09-01 Thread shveta malik
On Mon, Sep 2, 2024 at 5:47 AM Peter Smith wrote: > > Hi hackers. While reviewing another thread I had cause to look at the > docs for the pg_replication_slot 'inactive_since' field [1] for the > first time. > > I was confused by the description, which is as follows: > > inactive_since timest

Re: ANALYZE ONLY

2024-09-01 Thread torikoshia
On 2024-09-01 10:31, Michael Harris wrote: Hello Atsushi & Melih Thank you both for your further feedback. On Thu, 29 Aug 2024 at 21:31, Melih Mutlu wrote: I believe moving "[ ONLY ]" to the definitions of table_and_columns below, as you did with "[ * ]", might be better to be consistent wit

Re: Collect statistics about conflicts in logical replication

2024-09-01 Thread shveta malik
On Mon, Sep 2, 2024 at 4:20 AM Peter Smith wrote: > > On Fri, Aug 30, 2024 at 4:24 PM shveta malik wrote: > > > > On Fri, Aug 30, 2024 at 10:53 AM Peter Smith wrote: > > > > ... > > > 2. Arrange all the counts into an intuitive/natural order > > > > > > There is an intuitive/natural ordering for

Re: Make printtup a bit faster

2024-09-01 Thread Andy Fan
Andy Fan writes: > The attached is PoC of this idea, not matter which method are adopted > (rewrite all the outfunction or a optional print function), I think the > benefit will be similar. In the blew test case, it shows us 10%+ > improvements. (0.134ms vs 0.110ms) After working on more {type}_

Re: JIT: Remove some unnecessary instructions.

2024-09-01 Thread Xing Guo
On Fri, Aug 30, 2024 at 8:50 PM Andreas Karlsson wrote: > > On 8/30/24 5:55 AM, Xing Guo wrote: > > I find there are some unnecessary load/store instructions being > > emitted by the JIT compiler. > > Well spotted! All of these are obvious dead instructions and while LLVM > might be able to optimi

Re: Re: bt Scankey in another contradictory case

2024-09-01 Thread bigbro...@hotmail.com
Hi, I have reanalysed the code of function _bt_first. I notice that using a multi-attribute index if we can't identify the starting boundaries and the following attributes markded not required , that means we need start at first or last page in the index to examine every tuple to satisfy the qu

Re: query_id, pg_stat_activity, extended query protocol

2024-09-01 Thread Michael Paquier
On Sat, Aug 31, 2024 at 09:47:41AM +0800, jian he wrote: > /* test \bind queryid exists */ > select query_id is not null as query_id_exist > from pg_stat_activity where pid = pg_backend_pid() \bind \g > > /* test that \parse \bind_named queryid exists */ > select pg_backend_pid() as current_pid \g

Re: Test 041_checkpoint_at_promote.pl faild in installcheck due to missing injection_points

2024-09-01 Thread Michael Paquier
On Fri, Aug 23, 2024 at 06:45:02PM -0400, Tom Lane wrote: > It exports it twice, though, which is pretty confusing. Right. I am not sure what was my state of mind back then, but this pattern has spread a bit. REL_17_STABLE is frozen for a few more days, so I'll address all the items of this thre

Re: Minor refactor: Use more consistent names for the labels of PG_Locale_Strategy

2024-09-01 Thread Michael Paquier
On Thu, Aug 29, 2024 at 10:06:32AM +0900, Michael Paquier wrote: > +1 for your suggestion, as you are suggesting. The original intention > when PG_Locale_Strategy got introduced was to have everything named as > PG_REGEX_LOCALE_*, but with the built-in part coming in play in this > code adding "ST

Re: In-placre persistance change of a relation

2024-09-01 Thread Michael Paquier
On Sun, Sep 01, 2024 at 10:15:00PM +0300, Heikki Linnakangas wrote: > I wonder if the twophase state files and undo log files should be merged > into one file. They're similar in many ways: there's one file per > transaction, named using the XID. I haven't thought this fully through, just > a thoug

DOCS - pg_replication_slot . Fix the 'inactive_since' description

2024-09-01 Thread Peter Smith
Hi hackers. While reviewing another thread I had cause to look at the docs for the pg_replication_slot 'inactive_since' field [1] for the first time. I was confused by the description, which is as follows: inactive_since timestamptz The time since the slot has become inactive. NULL if the slo

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2024-09-01 Thread Michael Paquier
On Fri, Aug 30, 2024 at 04:10:32PM -0400, Andrew Dunstan wrote: > > On 2024-08-29 Th 4:44 PM, Jacob Champion wrote: > > As for the other patches, I'll ping Andrew about 0001, > > > Patch 0001 looks sane to me. So does 0002 to me. I'm not much a fan of the addition of pgstat_bestart_pre_auth(),

Re: scalability bottlenecks with (many) partitions (and more)

2024-09-01 Thread Robert Haas
On Sun, Sep 1, 2024 at 3:30 PM Tomas Vondra wrote: > I don't think that's possible with hard-coded size of the array - that > allocates the memory for everyone. We'd need to make it variable-length, > and while doing those benchmarks I think we actually already have a GUC > for that - max_locks_pe

Re: Pgstattuple on Sequences: Seeking Community Feedback on Potential Patch

2024-09-01 Thread Michael Paquier
On Fri, Aug 30, 2024 at 04:06:03PM -0500, Nathan Bossart wrote: > I suppose it would be difficult to argue that it is actually useful, given > it hasn't worked since v11 and apparently nobody noticed until recently. > If we're content to leave it unsupported, then sure, let's just remove the > "rel

Re: Collect statistics about conflicts in logical replication

2024-09-01 Thread Peter Smith
On Fri, Aug 30, 2024 at 4:24 PM shveta malik wrote: > > On Fri, Aug 30, 2024 at 10:53 AM Peter Smith wrote: > > ... > > 2. Arrange all the counts into an intuitive/natural order > > > > There is an intuitive/natural ordering for these counts. For example, > > the 'confl_*' count fields are in the

Re: Collect statistics about conflicts in logical replication

2024-09-01 Thread Peter Smith
On Fri, Aug 30, 2024 at 6:36 PM shveta malik wrote: > > On Fri, Aug 30, 2024 at 12:15 PM Zhijie Hou (Fujitsu) > wrote: > > > > > > Here is V5 patch which addressed above and Shveta's[1] comments. > > > > The patch looks good to me. > Patch v5 LGTM. == Kind Regards, Peter Smith. Fujitsu Aust

Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2024-09-01 Thread Michail Nikolaev
Hello, Matthias! Just wanted to update you with some information about the next steps in work. > In heapam_index_build_range_scan, it seems like you're popping the > snapshot and registering a new one while holding a tuple from > heap_getnext(), thus while holding a page lock. I'm not so sure tha

Re: optimizing pg_upgrade's once-in-each-database steps

2024-09-01 Thread Nathan Bossart
On Sat, Aug 31, 2024 at 01:18:10AM +0100, Ilya Gladyshev wrote: > LGTM in general, but here are some final nitpicks: Thanks for reviewing. > + if (maxFd != 0) > + (void) select(maxFd + 1, &input_mask, &output_mask, > &except_mask, NULL); > > It´s a good idea to check for the ret

Re: scalability bottlenecks with (many) partitions (and more)

2024-09-01 Thread Tomas Vondra
Hi, While discussing this patch with Robert off-list, one of the questions he asked was is there's some size threshold after which it starts to have negative impact. I didn't have a good answer to that - I did have some intuition (that making it too large would not hurt), but I haven't done any te

Re: In-placre persistance change of a relation

2024-09-01 Thread Heikki Linnakangas
On 31/08/2024 19:09, Kyotaro Horiguchi wrote: - UNDO log(0002): This handles file deletion during transaction aborts, which was previously managed, in part, by the commit XLOG record at the end of a transaction. - Prevent orphan files after a crash (0005): This is another use-case of th

Re: Improving tracking/processing of buildfarm test failures

2024-09-01 Thread sia kc
Hello everyone I am a developer interested in this project. Had a little involvement with MariaDB and now I like to work on Postgres. Never worked with mailing lists so I am not sure if this is the way I should interact. Liked to be pointed to some tasks and documents to get started.

Re: Improving tracking/processing of buildfarm test failures

2024-09-01 Thread Alexander Lakhin
Hello hackers, Please take a look at the August report on buildfarm failures: # SELECT br, count(*) FROM failures WHERE dt >= '2024-08-01' AND  dt < '2024-09-01' GROUP BY br; REL_12_STABLE: 2 REL_13_STABLE: 2 REL_14_STABLE: 12 REL_15_STABLE: 3 REL_16_STABLE: 5 REL_17_STABLE: 17 HEAD: 38 -- Total:

Add ExprState hashing for GROUP BY and hashed SubPlans

2024-09-01 Thread David Rowley
adf97c156 added support to allow ExprStates to support hashing and adjusted Hash Join to make use of that. That allowed a speedup in hash value generation as it allowed JIT compilation of hash values. It also allowed more efficient tuple deforming as all required attributes are deformed in one go r

Re: ANALYZE ONLY

2024-09-01 Thread David Rowley
On Sun, 1 Sept 2024 at 13:41, Michael Harris wrote: > https://commitfest.postgresql.org/49/5226/ > > I was not sure what to put for some of the fields - for 'reviewer' should > I have put the people who have provided feedback on this thread? I think it's fairly common that only reviewers either