Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Bertrand Drouvot
Hi, On Wed, Mar 27, 2024 at 10:08:33AM +0530, Bharath Rupireddy wrote: > On Tue, Mar 26, 2024 at 11:22 PM Bertrand Drouvot > wrote: > > > > - if (!(RecoveryInProgress() && slot->data.synced)) > > + if (!(InRecovery && slot->data.synced)) > >

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Bharath Rupireddy
On Wed, Mar 27, 2024 at 10:08 AM Bharath Rupireddy wrote: > > Please find the attached v25-0001 (made this 0001 patch now as > inactive_since patch is committed) patch with the above changes. Fixed an issue in synchronize_slots where DatumGetLSN is being used in place of DatumGetTimestampTz.

Re: Why is parula failing?

2024-03-26 Thread Tom Lane
David Rowley writes: > Unfortunately, REL_16_STABLE does not have the additional debugging, > so don't get to know what reltuples was set to. Let's wait a bit to see if it fails in HEAD ... but if not, would it be reasonable to back-patch the additional debugging output?

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread shveta malik
On Tue, Mar 26, 2024 at 6:05 PM Bertrand Drouvot wrote: > > > > We can think on that later if we really need another > > field which give us sync time. > > I think that calling GetCurrentTimestamp() so frequently could be too costly, > so > I'm not sure we should. Agreed. > > In my second

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-03-26 Thread Will Mortensen
Rebased, fixed a couple typos, and reordered the isolation tests to put the most elaborate pair last. v11-0001-Refactor-GetLockConflicts-into-more-general-GetL.patch Description: Binary data v11-0002-Allow-specifying-single-lockmode-in-WaitForLocke.patch Description: Binary data

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Bharath Rupireddy
On Wed, Mar 27, 2024 at 10:24 AM shveta malik wrote: > > On Wed, Mar 27, 2024 at 10:22 AM Amit Kapila wrote: > > > > On Wed, Mar 27, 2024 at 10:08 AM Bharath Rupireddy > > wrote: > > > > > > On Tue, Mar 26, 2024 at 11:22 PM Bertrand Drouvot > > > wrote: > > > > > > > 3) > > > >

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread shveta malik
On Wed, Mar 27, 2024 at 10:22 AM Amit Kapila wrote: > > On Wed, Mar 27, 2024 at 10:08 AM Bharath Rupireddy > wrote: > > > > On Tue, Mar 26, 2024 at 11:22 PM Bertrand Drouvot > > wrote: > > > > > 3) > > > update_synced_slots_inactive_time(): > > > > > > This assert is removed, is it intentional?

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Amit Kapila
On Wed, Mar 27, 2024 at 10:08 AM Bharath Rupireddy wrote: > > On Tue, Mar 26, 2024 at 11:22 PM Bertrand Drouvot > wrote: > > > 3) > > update_synced_slots_inactive_time(): > > > > This assert is removed, is it intentional? > > Assert(s->active_pid == 0); > > Yes, the slot can get acquired in the

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-26 Thread Amit Kapila
On Tue, Mar 26, 2024 at 9:10 PM Alvaro Herrera wrote: > > On 2024-Mar-26, Nathan Bossart wrote: > > > FWIW I'd really prefer to have something like max_slot_xid_age for this. A > > time-based parameter would likely help with most cases, but transaction ID > > usage will vary widely from server

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Bharath Rupireddy
On Tue, Mar 26, 2024 at 11:22 PM Bertrand Drouvot wrote: > > > I'm attaching v24 patches. It implements the above idea proposed > > upthread for synced slots. > > v24-0002 > > 1 === > > This commit does two things: > 1) Updates inactive_since for sync slots with the value >

Re: remaining sql/json patches

2024-03-26 Thread Amit Langote
On Wed, Mar 27, 2024 at 12:42 PM jian he wrote: > hi. > I don't fully understand all the code in json_table patch. > maybe we can split it into several patches, I'm working on exactly that atm. > like: > * no nested json_table_column. > * nested json_table_column, with PLAN DEFAULT > * nested

Re: remaining sql/json patches

2024-03-26 Thread jian he
On Tue, Mar 26, 2024 at 6:16 PM jian he wrote: > > On Fri, Mar 22, 2024 at 12:08 AM Amit Langote wrote: > > > > On Wed, Mar 20, 2024 at 9:53 PM Amit Langote > > wrote: > > > I'll push 0001 tomorrow. > > > > Pushed that one. Here's the remaining JSON_TABLE() patch. > > hi. I don't fully

Re: Can't find not null constraint, but \d+ shows that

2024-03-26 Thread Tender Wang
Alvaro Herrera 于2024年3月26日周二 23:25写道: > On 2024-Mar-26, Tender Wang wrote: > > > postgres=# CREATE TABLE t1(c0 int, c1 int); > > postgres=# ALTER TABLE t1 ADD CONSTRAINT Q PRIMARY KEY(c0, c1); > > postgres=# ALTER TABLE t1 DROP c1; > > > > postgres=# ALTER TABLE t1 ALTER c0 DROP NOT NULL; >

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread shveta malik
On Tue, Mar 26, 2024 at 9:59 PM Bharath Rupireddy wrote: > > On Tue, Mar 26, 2024 at 4:35 PM Bharath Rupireddy > wrote: > > > > If we just sync inactive_since value for synced slots while in > > recovery from the primary, so be it. Why do we need to update it to > > the current time when the

Re: Properly pathify the union planner

2024-03-26 Thread Richard Guo
On Wed, Mar 27, 2024 at 6:23 AM David Rowley wrote: > Because this field is set, it plans the CTE thinking it's a UNION > child and breaks when it can't find a SortGroupClause for the CTE's > target list item. Right. The problem here is that we mistakenly think that the CTE query is a

Re: recovery modules

2024-03-26 Thread Nathan Bossart
another rebase for cfbot -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 0ccdbb0a010830380e6ff4b7a052198d50f0680f Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 15 Feb 2023 14:28:53 -0800 Subject: [PATCH v20 1/5] introduce routine for checking mutually exclusive

Re: [PATCH] LockAcquireExtended improvement

2024-03-26 Thread Will Mortensen
On Thu, Mar 14, 2024 at 1:15 PM Robert Haas wrote: > Seeing no further discussion, I have committed my version of this > patch, with your test case. This comment on ProcSleep() seems to have the values of dontWait backward (double negatives are tricky): * Result: PROC_WAIT_STATUS_OK if we

Re: Add new error_action COPY ON_ERROR "log"

2024-03-26 Thread Masahiko Sawada
On Tue, Mar 26, 2024 at 6:36 PM Bharath Rupireddy wrote: > > On Tue, Mar 26, 2024 at 1:46 PM Masahiko Sawada wrote: > > > > Thank you for updating the patch! Looks good to me. > > > > Please find the attached patch. I've made some changes for the > > documentation and the commit message. I'll

Re: Why is parula failing?

2024-03-26 Thread David Rowley
On Tue, 26 Mar 2024 at 21:03, Tharakan, Robins wrote: > > > David Rowley writes: > > It would be good to have log_autovacuum_min_duration = 0 on this machine > > for a while. > > - Have set log_autovacuum_min_duration=0 on parula and a test run came out > okay. > - Also added REL_16_STABLE to

Re: add AVX2 support to simd.h

2024-03-26 Thread Tom Lane
Nathan Bossart writes: > On Tue, Mar 26, 2024 at 06:55:54PM -0500, Nathan Bossart wrote: >> On Tue, Mar 26, 2024 at 07:28:24PM -0400, Tom Lane wrote: >>> A significant fraction of the buildfarm is issuing warnings about >>> this. > Done. I'll keep an eye on the farm. Thanks. > I just did the

Re: add AVX2 support to simd.h

2024-03-26 Thread Nathan Bossart
On Tue, Mar 26, 2024 at 06:55:54PM -0500, Nathan Bossart wrote: > On Tue, Mar 26, 2024 at 07:28:24PM -0400, Tom Lane wrote: >> A significant fraction of the buildfarm is issuing warnings about >> this. > > Thanks for the heads-up. Will fix. Done. I'll keep an eye on the farm. I just did the

Re: Possibility to disable `ALTER SYSTEM`

2024-03-26 Thread Andrew Dunstan
> On Mar 27, 2024, at 3:53 AM, Tom Lane wrote: > > Bruce Momjian writes: >> I am thinking "enable_alter_system_command" is probably good because we >> already use "enable" so why not reuse that idea, and I think "command" >> is needed because we need to clarify we are talking about the

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-26 Thread John Naylor
On Mon, Mar 25, 2024 at 8:07 PM Masahiko Sawada wrote: > > On Mon, Mar 25, 2024 at 3:25 PM John Naylor wrote: > > > > On Fri, Mar 22, 2024 at 12:20 PM Masahiko Sawada > > wrote: > > - * remaining LP_DEAD line pointers on the page in the dead_items > > - * array. These dead items include those

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-26 Thread Alexander Korotkov
On Sun, Mar 24, 2024 at 4:39 AM Bharath Rupireddy wrote: > I share the same concern as yours and had proposed something upthread > [1]. The idea is something like how each query takes a snapshot at the > beginning of txn/query (depending on isolation level), the same way > the standby can wait

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-26 Thread Alexander Korotkov
On Fri, Mar 22, 2024 at 12:58 AM Peter Eisentraut wrote: > On 17.03.24 15:09, Alexander Korotkov wrote: > > My current attempt was to commit minimal implementation as less > > invasive as possible. A new clause for BEGIN doesn't require > > additional keywords and doesn't introduce additional

Re: pg_stat_statements and "IN" conditions

2024-03-26 Thread Yasuo Honda
Thanks for the useful info. Ruby on Rails uses bigint as a default data type for the primary key and prepared statements have been enabled by default for PostgreSQL. I'm looking forward to these current patches being merged as a first step and future versions of pg_stat_statements will support

Re: add AVX2 support to simd.h

2024-03-26 Thread Nathan Bossart
On Tue, Mar 26, 2024 at 07:28:24PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> I've committed v9, and I've marked the commitfest entry as "Committed," >> although we may want to revisit AVX2, etc. in the future. > > A significant fraction of the buildfarm is issuing warnings about > this.

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-26 Thread Alexander Korotkov
On Fri, Mar 22, 2024 at 12:50 AM Peter Eisentraut wrote: > On 19.03.24 18:38, Kartyshov Ivan wrote: > > CALL pg_wait_lsn('0/3002AE8', 1); > > BEGIN; > > SELECT * FROM tbl; // read fresh insertions > > COMMIT; > > I'm not endorsing this

Re: add AVX2 support to simd.h

2024-03-26 Thread Tom Lane
Nathan Bossart writes: > I've committed v9, and I've marked the commitfest entry as "Committed," > although we may want to revisit AVX2, etc. in the future. A significant fraction of the buildfarm is issuing warnings about this. adder | 2024-03-26 21:04:33 |

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-03-26 Thread Justin Pryzby
On Thu, Mar 21, 2024 at 01:07:01PM +0100, Alvaro Herrera wrote: > Given that Michaël is temporarily gone, I propose to push the attached > tomorrow. Thanks. On Tue, Mar 26, 2024 at 12:05:47PM +0100, Alvaro Herrera wrote: > On 2024-Mar-26, Alexander Lakhin wrote: > > > Hello Alvaro, > > > >

Re: Properly pathify the union planner

2024-03-26 Thread David Rowley
On Wed, 27 Mar 2024 at 06:00, Alexander Lakhin wrote: > SELECT count(*) FROM ( >WITH q1(x) AS (SELECT 1) >SELECT FROM q1 UNION SELECT FROM q1 > ) qu; > > TRAP: failed Assert("lg != NULL"), File: "planner.c", Line: 7941, PID: 1133017 Thanks for finding that. There's something weird going

Re: WIP Incremental JSON Parser

2024-03-26 Thread Jacob Champion
On Mon, Mar 25, 2024 at 3:14 PM Jacob Champion wrote: > - add Assert calls in impossible error cases [2] To expand on this one, I think these parts of the code (marked with `<- here`) are impossible to reach: > switch (top) > { > case JSON_TOKEN_STRING: > if (next_prediction(pstack)

Re: speed up a logical replica setup

2024-03-26 Thread Tomas Vondra
On 3/26/24 21:17, Euler Taveira wrote: > On Tue, Mar 26, 2024, at 4:12 PM, Tomas Vondra wrote: >> Perhaps I'm missing something, but why is NUM_CONN_ATTEMPTS even needed? >> Why isn't recovery_timeout enough to decide if wait_for_end_recovery() >> waited long enough? > > It was an attempt to

Re: pg_stat_statements and "IN" conditions

2024-03-26 Thread Dmitry Dolgov
> On Tue, Mar 26, 2024 at 04:21:46PM +0900, Yasuo Honda wrote: > Yes. The script uses prepared statements because Ruby on Rails enables > prepared statements by default for PostgreSQL databases. > > Then I tested this branch > https://github.com/yahonda/postgres/tree/pg_stat_statements without >

Re: speed up a logical replica setup

2024-03-26 Thread Euler Taveira
On Tue, Mar 26, 2024, at 4:12 PM, Tomas Vondra wrote: > Perhaps I'm missing something, but why is NUM_CONN_ATTEMPTS even needed? > Why isn't recovery_timeout enough to decide if wait_for_end_recovery() > waited long enough? It was an attempt to decoupled a connection failure (that keeps streaming

Re: Remove some redundant set_cheapest() calls

2024-03-26 Thread Tom Lane
Richard Guo writes: > I happened to notice that the set_cheapest() calls in functions > set_namedtuplestore_pathlist() and set_result_pathlist() are redundant, > as we've centralized the set_cheapest() calls in set_rel_pathlist(). > Attached is a trivial patch to remove these calls. Agreed, and

Re: New Table Access Methods for Multi and Single Inserts

2024-03-26 Thread Bharath Rupireddy
On Tue, Mar 26, 2024 at 9:07 PM Jeff Davis wrote: > > On Tue, 2024-03-26 at 01:28 +0530, Bharath Rupireddy wrote: > > I'm thinking > > of dropping the COPY FROM patch using the new multi insert API for > > the > > following reasons: ... > > I agree with all of this. We do want COPY ... FROM

Re: Slow GRANT ROLE on PostgreSQL 16 with thousands of ROLEs

2024-03-26 Thread Nathan Bossart
On Tue, Mar 26, 2024 at 03:08:00PM -0400, Tom Lane wrote: > My one remaining suggestion is that this comment isn't very precise > about what's happening: > > * If there is a previously-created Bloom filter, use it to determine > * whether the role is missing from the list. Otherwise,

Re: archive modules loose ends

2024-03-26 Thread Nathan Bossart
On Mon, Jan 15, 2024 at 08:50:25AM -0600, Nathan Bossart wrote: > Thanks for reviewing. I've marked this as ready-for-committer, and I'm > hoping to commit it in the near future. This one probably ought to go into v17, but I wanted to do one last call for feedback prior to committing. --

Re: speed up a logical replica setup

2024-03-26 Thread Tomas Vondra
On 3/26/24 03:53, Euler Taveira wrote: > On Mon, Mar 25, 2024, at 1:06 PM, Hayato Kuroda (Fujitsu) wrote: >> ## Analysis for failure 1 >> >> The failure caused by a time lag between walreceiver finishes and >> pg_is_in_recovery() >> returns true. >> >> According to the output [1], it seems that

Re: add AVX2 support to simd.h

2024-03-26 Thread Nathan Bossart
I've committed v9, and I've marked the commitfest entry as "Committed," although we may want to revisit AVX2, etc. in the future. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Slow GRANT ROLE on PostgreSQL 16 with thousands of ROLEs

2024-03-26 Thread Tom Lane
Nathan Bossart writes: > On Tue, Mar 26, 2024 at 02:16:03PM -0400, Tom Lane wrote: >> ... I'm not sold on your "ROLES_LIST_BLOOM_THRESHOLD * 10" >> value. Maybe it doesn't matter though. > Yeah, I wasn't sure how much to worry about this. I figured that we might > as well set it to a

Re: Slow GRANT ROLE on PostgreSQL 16 with thousands of ROLEs

2024-03-26 Thread Nathan Bossart
On Tue, Mar 26, 2024 at 02:16:03PM -0400, Tom Lane wrote: > I did a little experimentation using the attached quick-hack C > function, and came to the conclusion that setting up the bloom filter > costs more or less as much as inserting 1000 or so OIDs the dumb way. > So we definitely want a

Re: Slow GRANT ROLE on PostgreSQL 16 with thousands of ROLEs

2024-03-26 Thread Tom Lane
Nathan Bossart writes: > I spent some time trying to get some ballpark figures but have thus far > been unsuccessful. Even if I was able to get good numbers, I'm not sure > how much they'd help us, as we'll still need to decide how much overhead we > are willing to take in comparison to the

Re: pg_upgrade --copy-file-range

2024-03-26 Thread Tomas Vondra
On 3/25/24 15:31, Robert Haas wrote: > On Sat, Mar 23, 2024 at 9:37 AM Tomas Vondra > wrote: >> OK, that makes sense. Here's a patch that should work like this - in >> copy_file we check if we need to calculate checksums, and either use the >> requested copy method, or fall back to the

Re: pg_combinebackup --copy-file-range

2024-03-26 Thread Tomas Vondra
On 3/26/24 15:09, Jakub Wartak wrote: > On Sat, Mar 23, 2024 at 6:57 PM Tomas Vondra > wrote: > >> On 3/23/24 14:47, Tomas Vondra wrote: >>> On 3/23/24 13:38, Robert Haas wrote: On Fri, Mar 22, 2024 at 8:26 PM Thomas Munro wrote: > [..] >>> Yeah, that's in

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Bertrand Drouvot
Hi, On Tue, Mar 26, 2024 at 09:59:23PM +0530, Bharath Rupireddy wrote: > On Tue, Mar 26, 2024 at 4:35 PM Bharath Rupireddy > wrote: > > > > If we just sync inactive_since value for synced slots while in > > recovery from the primary, so be it. Why do we need to update it to > > the current time

Re: UUID v7

2024-03-26 Thread Andrey M. Borodin
Sorry for this long reply. I was looking on refactoring around pg_strong_random() and could not decide what to do. Finally, I decided to post at least something. > On 22 Mar 2024, at 19:15, Peter Eisentraut wrote: > > I have been studying the uuidv() function. > > I find this code extremely

Re: Possibility to disable `ALTER SYSTEM`

2024-03-26 Thread Tom Lane
Bruce Momjian writes: > I am thinking "enable_alter_system_command" is probably good because we > already use "enable" so why not reuse that idea, and I think "command" > is needed because we need to clarify we are talking about the command, > and not generic altering of the system. We could use

Re: Propagate pathkeys from CTEs up to the outer query

2024-03-26 Thread Tom Lane
Richard Guo writes: > I agree with your points. Previously I was thinking that CTEs were the > only scenario where we needed to remember the best path and only > required the best path's pathkeys. However, considering potential > future use cases as you mentioned, I concur that having a

Re: Properly pathify the union planner

2024-03-26 Thread Alexander Lakhin
Hello David, 25.03.2024 04:43, David Rowley wrote: I didn't see that as a reason not to push this patch as this occurs both with and without this change, so I've now pushed this patch. Please look at a new assertion failure, that is triggered by the following query: SELECT count(*) FROM (  

Re: Slow GRANT ROLE on PostgreSQL 16 with thousands of ROLEs

2024-03-26 Thread Nathan Bossart
Here is a new version of the patch that I feel is in decent shape. On Mon, Mar 25, 2024 at 10:16:47AM -0500, Nathan Bossart wrote: > On Mon, Mar 25, 2024 at 11:08:39AM -0400, Tom Lane wrote: >> * The magic constants (crossover list length and bloom filter size) >> need some testing to see if

Re: Possibility to disable `ALTER SYSTEM`

2024-03-26 Thread Bruce Momjian
On Tue, Mar 26, 2024 at 10:23:51AM -0400, Tom Lane wrote: > Robert Haas writes: > > On Mon, Mar 25, 2024 at 5:04 PM Bruce Momjian wrote: > >> To me, externally_managed_configuration is promising a lot more than it > >> delivers because there is still a lot of ocnfiguration it doesn't > >>

Re: pgsql: Allow using syncfs() in frontend utilities.

2024-03-26 Thread Nathan Bossart
On Tue, Mar 26, 2024 at 10:11:31AM -0500, Nathan Bossart wrote: > On Tue, Mar 26, 2024 at 11:18:57AM +0100, Peter Eisentraut wrote: >> On 22.03.24 17:52, Robert Haas wrote: >>> I'd like to complain about this commit's addition of a new appendix. >> >> I already complained about that at >>

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Bharath Rupireddy
On Tue, Mar 26, 2024 at 4:35 PM Bharath Rupireddy wrote: > > If we just sync inactive_since value for synced slots while in > recovery from the primary, so be it. Why do we need to update it to > the current time when the slot is being created? We don't expose slot > creation time, no? Aren't we

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-26 Thread Bertrand Drouvot
Hi, On Tue, Mar 26, 2024 at 04:39:55PM +0100, Alvaro Herrera wrote: > On 2024-Mar-26, Nathan Bossart wrote: > > I don't object to a > > time-based setting as well, but that won't always work as well for this > > particular use-case, especially if we are relying on users to set a > > slot-level

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-26 Thread Alvaro Herrera
On 2024-Mar-26, Nathan Bossart wrote: > FWIW I'd really prefer to have something like max_slot_xid_age for this. A > time-based parameter would likely help with most cases, but transaction ID > usage will vary widely from server to server, so it'd be nice to have > something to protect against

Re: New Table Access Methods for Multi and Single Inserts

2024-03-26 Thread Jeff Davis
On Tue, 2024-03-26 at 01:28 +0530, Bharath Rupireddy wrote: > I'm thinking > of dropping the COPY FROM patch using the new multi insert API for > the > following reasons: ... I agree with all of this. We do want COPY ... FROM support, but there are some details to work out and we don't want to

Re: Can't find not null constraint, but \d+ shows that

2024-03-26 Thread Alvaro Herrera
On 2024-Mar-26, Tender Wang wrote: > postgres=# CREATE TABLE t1(c0 int, c1 int); > postgres=# ALTER TABLE t1 ADD CONSTRAINT Q PRIMARY KEY(c0, c1); > postgres=# ALTER TABLE t1 DROP c1; > > postgres=# ALTER TABLE t1 ALTER c0 DROP NOT NULL; > ERROR: could not find not-null constraint on column

Re: pgsql: Allow using syncfs() in frontend utilities.

2024-03-26 Thread Nathan Bossart
On Tue, Mar 26, 2024 at 11:18:57AM +0100, Peter Eisentraut wrote: > On 22.03.24 17:52, Robert Haas wrote: >> I'd like to complain about this commit's addition of a new appendix. > > I already complained about that at >

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-26 Thread Nathan Bossart
On Tue, Mar 26, 2024 at 03:44:29PM +0530, Amit Kapila wrote: > On Tue, Mar 26, 2024 at 2:11 PM Alvaro Herrera > wrote: >> Well, I think a GUC is good to have regardless of the slot parameter, >> because the GUC can be used as an instance-wide protection against going >> out of disk space because

Re: pgsql: Allow using syncfs() in frontend utilities.

2024-03-26 Thread Nathan Bossart
On Fri, Mar 22, 2024 at 12:52:15PM -0400, Robert Haas wrote: > I'd like to complain about this commit's addition of a new appendix. I > do understand the temptation to document caveats like this centrally > instead of in multiple places, but as I've been complaining about over > in the

SQL function which allows to distinguish a server being in point in time recovery mode and an ordinary replica

2024-03-26 Thread m . litsarev
Hi, At present time, an existing pg_is_in_recovery() method is not enough to distinguish a server being in point in time recovery (PITR) mode and an ordinary replica because it returns true in both cases. That is why pg_is_standby_requested() function introduced in attached patch might help.

Re: make dist using git archive

2024-03-26 Thread Tristan Partin
On Tue Mar 26, 2024 at 2:56 AM CDT, Andres Freund wrote: Hi, On 2024-03-26 08:36:58 +0100, Peter Eisentraut wrote: > On 26.03.24 01:23, Andres Freund wrote: > > On 2024-03-25 06:44:33 +0100, Peter Eisentraut wrote: > > > Done and committed. > > > > This triggered a new warning for me: > > > >

Re: Possibility to disable `ALTER SYSTEM`

2024-03-26 Thread Tom Lane
Robert Haas writes: > On Mon, Mar 25, 2024 at 5:04 PM Bruce Momjian wrote: >> To me, externally_managed_configuration is promising a lot more than it >> delivers because there is still a lot of ocnfiguration it doesn't >> control. I am also confused why the purpose of the feature, external >>

Re: pg_upgrade --copy-file-range

2024-03-26 Thread Jakub Wartak
On Sat, Mar 23, 2024 at 6:57 PM Tomas Vondra wrote: > On 3/23/24 14:47, Tomas Vondra wrote: > > On 3/23/24 13:38, Robert Haas wrote: > >> On Fri, Mar 22, 2024 at 8:26 PM Thomas Munro > >> wrote: [..] > > Yeah, that's in write_reconstructed_file() and the patch does not touch > > that at all. I

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-26 Thread Kartyshov Ivan
Thank you for your interest to the patch. I understand you questions, but I fully support Alexander Korotkov idea to commit the minimal required functionality. And then keep working on other improvements. On 2024-03-24 05:39, Bharath Rupireddy wrote: On Fri, Mar 22, 2024 at 4:28 AM Peter

Re: Parallel Aggregates for string_agg and array_agg

2024-03-26 Thread Alexander Lakhin
Hello David, 23.01.2023 07:37, David Rowley wrote: I've now pushed this. I've discovered that the test query: -- Ensure parallel aggregation is actually being used. explain (costs off) select * from v_pagg_test order by y; added by 16fd03e95 fails sometimes. For instance:

Re: Possibility to disable `ALTER SYSTEM`

2024-03-26 Thread Robert Haas
On Tue, Mar 26, 2024 at 8:55 AM Abhijit Menon-Sen wrote: > Yes, "externally_managed_configuration" raises far more questions than > it answers. "enable_alter_system" is clearer in terms of what to expect > when you set it. "enable_alter_system_command" is rather long, but even > better in that it

Re: Possibility to disable `ALTER SYSTEM`

2024-03-26 Thread Abhijit Menon-Sen
At 2024-03-26 08:11:33 -0400, robertmh...@gmail.com wrote: > > On Mon, Mar 25, 2024 at 5:04 PM Bruce Momjian wrote: > > > > Isn't "configuration" too generic a term for disabling ALTER SYSTEM? > > > > > > maybe "externally_managed_auto_config" > > > > How many people associate "auto" with ALTER

Re: Streaming I/O, vectored I/O (WIP)

2024-03-26 Thread Heikki Linnakangas
On 24/03/2024 15:02, Thomas Munro wrote: On Wed, Mar 20, 2024 at 4:04 AM Heikki Linnakangas wrote: Maybe 'pg_streaming_read_next_buffer' or just 'pg_streaming_read_next', for a shorter name. Hmm. The idea of 'buffer' appearing in a couple of names is that there are conceptually other kinds

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Bertrand Drouvot
Hi, On Tue, Mar 26, 2024 at 04:17:53PM +0530, shveta malik wrote: > On Tue, Mar 26, 2024 at 3:50 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > > I think there may have been some misunderstanding here. > > > > Indeed ;-) > > > > > But now if I > > > rethink this, I am fine with

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Bertrand Drouvot
Hi, On Tue, Mar 26, 2024 at 04:49:18PM +0530, shveta malik wrote: > On Tue, Mar 26, 2024 at 4:35 PM Bharath Rupireddy > wrote: > > > > On Tue, Mar 26, 2024 at 4:18 PM shveta malik wrote: > > > > > > > What about another approach?: inactive_since gives data synced from > > > > primary for > > >

Re: REVOKE FROM warning on grantor

2024-03-26 Thread Robert Haas
On Tue, Mar 26, 2024 at 5:22 AM Étienne BERSAC wrote: > > ldap2pg really ought to issue REVOKE x FROM y GRANTED BY z. > > Thanks for this. I missed this notation and it is exactly what I need. > > You could consider this subject as closed. Thanks for your time and > explanation. No problem. Glad

Re: Possibility to disable `ALTER SYSTEM`

2024-03-26 Thread Daniel Gustafsson
> On 26 Mar 2024, at 13:11, Robert Haas wrote: > On Mon, Mar 25, 2024 at 5:04 PM Bruce Momjian wrote: >> To me, externally_managed_configuration is promising a lot more than it >> delivers because there is still a lot of ocnfiguration it doesn't >> control. I am also confused why the purpose

Re: Possibility to disable `ALTER SYSTEM`

2024-03-26 Thread Robert Haas
On Mon, Mar 25, 2024 at 5:04 PM Bruce Momjian wrote: > > > Isn't "configuration" too generic a term for disabling ALTER SYSTEM? > > > > maybe "externally_managed_auto_config" > > How many people associate "auto" with ALTER SYSTEM? I assume not many. > > To me, externally_managed_configuration is

Can't find not null constraint, but \d+ shows that

2024-03-26 Thread Tender Wang
Hi Alvaro, I met an issue related to Catalog not-null commit on HEAD. postgres=# CREATE TABLE t1(c0 int, c1 int); CREATE TABLE postgres=# ALTER TABLE t1 ADD CONSTRAINT Q PRIMARY KEY(c0, c1); ALTER TABLE postgres=# \d+ t1 Table "public.t1" Column |

Re: Refactoring of pg_resetwal/t/001_basic.pl

2024-03-26 Thread Svetlana Derevyanko
Peter Eisentraut писал(а) 2024-03-25 17:10: But MXOFF_SIZE doesn't exist anywhere else. The actual formula uses sizeof(MultiXactOffset), which isn't obvious from your patch. So this just moves the magic constants around by one level. I think if we're going to add more symbols, then it has

Re: Use streaming read API in ANALYZE

2024-03-26 Thread Nazir Bilal Yavuz
Hi, On Wed, 28 Feb 2024 at 14:42, Nazir Bilal Yavuz wrote: > > > The new version of the streaming read API [1] is posted. I updated the > streaming read API changes patch (0001), using the streaming read API > in ANALYZE patch (0002) remains the same. This should make it easier > to review as it

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread shveta malik
On Tue, Mar 26, 2024 at 4:35 PM Bharath Rupireddy wrote: > > On Tue, Mar 26, 2024 at 4:18 PM shveta malik wrote: > > > > > What about another approach?: inactive_since gives data synced from > > > primary for > > > synced slots and another dedicated field (could be added later...) could > > >

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-03-26 Thread Alvaro Herrera
On 2024-Mar-26, Alexander Lakhin wrote: > Hello Alvaro, > > 21.03.2024 15:07, Alvaro Herrera wrote: > > Given that Michaël is temporarily gone, I propose to push the attached > > tomorrow. > > Please look at a new anomaly introduced with 374c7a229. > Starting from that commit, the following

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Bharath Rupireddy
On Tue, Mar 26, 2024 at 4:18 PM shveta malik wrote: > > > What about another approach?: inactive_since gives data synced from primary > > for > > synced slots and another dedicated field (could be added later...) could > > represent what you suggest as the other option. > > Yes, okay with me. I

Remove some redundant set_cheapest() calls

2024-03-26 Thread Richard Guo
I happened to notice that the set_cheapest() calls in functions set_namedtuplestore_pathlist() and set_result_pathlist() are redundant, as we've centralized the set_cheapest() calls in set_rel_pathlist(). Attached is a trivial patch to remove these calls. BTW, I suspect that the set_cheapest()

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread shveta malik
On Tue, Mar 26, 2024 at 3:50 PM Bertrand Drouvot wrote: > > Hi, > > > I think there may have been some misunderstanding here. > > Indeed ;-) > > > But now if I > > rethink this, I am fine with 'inactive_since' getting synced from > > primary to standby. But if we do that, we need to add docs

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Ajin Cherian
On Tue, Mar 26, 2024 at 7:57 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > Please see the attached v23 patches. I've addressed all the review > comments received so far from Amit and Shveta. > > In patch 0003: + SpinLockAcquire(>mutex); + } + +

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Amit Kapila
On Tue, Mar 26, 2024 at 3:12 PM Bertrand Drouvot wrote: > > On Tue, Mar 26, 2024 at 02:27:17PM +0530, Bharath Rupireddy wrote: > > Please use the v22 patch set. > > Thanks! > > 1 === > > +reset_synced_slots_info(void) > > I'm not sure "reset" is the right word, what about >

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Bertrand Drouvot
Hi, On Tue, Mar 26, 2024 at 03:17:36PM +0530, shveta malik wrote: > On Tue, Mar 26, 2024 at 1:54 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Tue, Mar 26, 2024 at 01:37:21PM +0530, Amit Kapila wrote: > > > On Tue, Mar 26, 2024 at 1:15 PM Bertrand Drouvot > > > wrote: > > > > > > > > 2

Re: pgsql: Allow using syncfs() in frontend utilities.

2024-03-26 Thread Peter Eisentraut
On 22.03.24 17:52, Robert Haas wrote: On Wed, Sep 6, 2023 at 7:28 PM Nathan Bossart wrote: Allow using syncfs() in frontend utilities. This commit allows specifying a --sync-method in several frontend utilities that must synchronize many files to disk (initdb, pg_basebackup, pg_checksums,

Re: remaining sql/json patches

2024-03-26 Thread jian he
On Fri, Mar 22, 2024 at 12:08 AM Amit Langote wrote: > > On Wed, Mar 20, 2024 at 9:53 PM Amit Langote wrote: > > I'll push 0001 tomorrow. > > Pushed that one. Here's the remaining JSON_TABLE() patch. > hi. minor issues i found json_table patch. + if (!IsA($5, A_Const) || + castNode(A_Const,

Re: Improve readability by using designated initializers when possible

2024-03-26 Thread Peter Eisentraut
On 25.03.24 06:00, jian he wrote: looking through v4 again. v4 looks good to me. Thanks, I have committed this.

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-26 Thread Amit Kapila
On Tue, Mar 26, 2024 at 2:11 PM Alvaro Herrera wrote: > > On 2024-Mar-26, Amit Kapila wrote: > > > On Tue, Mar 26, 2024 at 1:09 PM Alvaro Herrera > > wrote: > > > On 2024-Mar-26, Amit Kapila wrote: > > > > I would also like to solicit your opinion on the other slot-level > > > > parameter we

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread shveta malik
On Tue, Mar 26, 2024 at 1:54 PM Bertrand Drouvot wrote: > > Hi, > > On Tue, Mar 26, 2024 at 01:37:21PM +0530, Amit Kapila wrote: > > On Tue, Mar 26, 2024 at 1:15 PM Bertrand Drouvot > > wrote: > > > > > > 2 === > > > > > > It looks like inactive_since is set to the current timestamp on the > >

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Bertrand Drouvot
Hi, On Tue, Mar 26, 2024 at 02:27:17PM +0530, Bharath Rupireddy wrote: > Please use the v22 patch set. Thanks! 1 === +reset_synced_slots_info(void) I'm not sure "reset" is the right word, what about slot_sync_shutdown_update()? 2 === + for (int i = 0; i < max_replication_slots; i++) +

Re: Add new error_action COPY ON_ERROR "log"

2024-03-26 Thread Bharath Rupireddy
On Tue, Mar 26, 2024 at 1:46 PM Masahiko Sawada wrote: > > Thank you for updating the patch! Looks good to me. > > Please find the attached patch. I've made some changes for the > documentation and the commit message. I'll push it, barring any > objections. Thanks. v12 patch LGTM. -- Bharath

Re: REVOKE FROM warning on grantor

2024-03-26 Thread Étienne BERSAC
Hi, > ldap2pg really ought to issue REVOKE x FROM y GRANTED BY z. Thanks for this. I missed this notation and it is exactly what I need. You could consider this subject as closed. Thanks for your time and explanation. Regards, Étienne

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread shveta malik
On Tue, Mar 26, 2024 at 2:27 PM Bharath Rupireddy wrote: > > > > > 1) > > Commti msg: > > > > ensures the value is set to current timestamp during the > > shutdown to help correctly interpret the time if the standby gets > > promoted without a restart. > > > > shutdown --> shutdown of slot sync

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-26 Thread Bertrand Drouvot
Hi, On Tue, Mar 26, 2024 at 01:45:23PM +0530, Amit Kapila wrote: > On Tue, Mar 26, 2024 at 1:09 PM Alvaro Herrera > wrote: > > > > On 2024-Mar-26, Amit Kapila wrote: > > > > > We have a consensus on inactive_since, so I'll make that change. > > > > Sounds reasonable. So this is a timestamptz

Re: Catalog domain not-null constraints

2024-03-26 Thread Dean Rasheed
On Tue, 26 Mar 2024 at 07:30, Alvaro Herrera wrote: > > On 2024-Mar-25, Dean Rasheed wrote: > > > Also (not this patch's fault), psql doesn't seem to offer a way to > > display domain constraint names -- something you need to know to drop > > or alter them. Perhaps \dD+ could be made to do that?

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-26 Thread Bharath Rupireddy
On Tue, Mar 26, 2024 at 11:26 AM Amit Kapila wrote: > > Review comments on v18_0002 and v18_0005 > === > > 1. > We have decided to update inactive_since for temporary slots. So, > unless there is some reason, we should allow inactive_timeout to also > be set

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-26 Thread Alvaro Herrera
On 2024-Mar-26, Amit Kapila wrote: > On Tue, Mar 26, 2024 at 1:09 PM Alvaro Herrera > wrote: > > On 2024-Mar-26, Amit Kapila wrote: > > > I would also like to solicit your opinion on the other slot-level > > > parameter we are planning to introduce. This new slot-level parameter > > > will be

  1   2   >