Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Bharath Rupireddy
On Mon, Mar 25, 2024 at 10:28 AM Amit Kapila wrote: > > On Mon, Mar 25, 2024 at 9:48 AM Amit Kapila wrote: > > > > > > Such a test looks reasonable but shall we add equal to in the second > > part of the test (like '$last_inactive_time'::timestamptz >= > > > '$slot_creation_time'::timestamptz;).

Re: Built-in CTYPE provider

2024-03-25 Thread Laurenz Albe
There is no technical content in this mail, but I'd like to show appreciation for your work on this. I hope this will eventually remove one of the great embarrassments when using PostgreSQL: the dependency on operation system collations. Yours, Laurenz Albe

Re: Built-in CTYPE provider

2024-03-25 Thread Peter Eisentraut
On 22.03.24 18:26, Jeff Davis wrote: On Fri, 2024-03-22 at 15:51 +0100, Peter Eisentraut wrote: I think this might be too big of a compatibility break.  So far, initcap('123abc') has always returned '123abc'.  If the new collation returns '123Abc' now, then that's quite a change.  These are not

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Amit Kapila
On Mon, Mar 25, 2024 at 12:43 PM shveta malik wrote: > > On Mon, Mar 25, 2024 at 11:53 AM shveta malik wrote: > > > > On Mon, Mar 25, 2024 at 10:33 AM shveta malik > > wrote: > > > > > > On Sun, Mar 24, 2024 at 3:06 PM Bharath Rupireddy > > > wrote: > > > > > > > > I've attached the v18 patch

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Bertrand Drouvot
Hi, On Mon, Mar 25, 2024 at 12:25:21PM +0530, Bharath Rupireddy wrote: > On Mon, Mar 25, 2024 at 10:28 AM Amit Kapila wrote: > > > > On Mon, Mar 25, 2024 at 9:48 AM Amit Kapila wrote: > > > > > > > > > Such a test looks reasonable but shall we add equal to in the second > > > part of the test

RE: speed up a logical replica setup

2024-03-25 Thread Hayato Kuroda (Fujitsu)
Dear Amit, Euler, I also want to share my opinion, just in case. > On Thu, Mar 21, 2024 at 8:00 PM Euler Taveira wrote: > > > > On Thu, Mar 21, 2024, at 10:33 AM, vignesh C wrote: > > > > If we commit this we might not be able to change the way the option > > behaves once the customers starts

Re: Partial aggregates pushdown

2024-03-25 Thread Alexander Pyhalov
fujii.y...@df.mitsubishielectric.co.jp писал(а) 2024-03-16 05:28: Hi. Mr.Pyhalov. >> >> I don't see it that way. What we would push to the foreign server >> would be something like SELECT count(a) FROM t. Then, after we get >> the results back and combine the various partial counts locally, we

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Bertrand Drouvot
Hi, On Mon, Mar 25, 2024 at 12:59:52PM +0530, Amit Kapila wrote: > On Mon, Mar 25, 2024 at 12:43 PM shveta malik wrote: > > > > On Mon, Mar 25, 2024 at 11:53 AM shveta malik > > wrote: > > > > > > On Mon, Mar 25, 2024 at 10:33 AM shveta malik > > > wrote: > > > > > > > > On Sun, Mar 24, 2024

Re: Properly pathify the union planner

2024-03-25 Thread Richard Guo
On Mon, Mar 25, 2024 at 9:44 AM David Rowley wrote: > It seems ok that > the ec_indexes are not set for the top-level set RelOptInfo as > get_eclass_for_sort_expr() does not make use of ec_indexes while > searching for an existing EquivalenceClass. Maybe we should fix this > varno == 0 hack and

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread shveta malik
On Mon, Mar 25, 2024 at 10:33 AM shveta malik wrote: > > On Sun, Mar 24, 2024 at 3:06 PM Bharath Rupireddy > wrote: > > > > I've attached the v18 patch set here. > I have a question. Don't we allow creating subscriptions on an existing slot with a non-null 'inactive_timeout' set where

Re: altering a column's collation leaves an invalid foreign key

2024-03-25 Thread Paul Jungwirth
On 3/23/24 10:04, Paul Jungwirth wrote: Perhaps if the previous collation was nondeterministic we should force a re-check. Here is a patch implementing this. It was a bit more fuss than I expected, so maybe someone has a better way. We have had nondeterministic collations since v12, so

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-25 Thread John Naylor
On Fri, Mar 22, 2024 at 12:20 PM Masahiko Sawada wrote: > > On Thu, Mar 21, 2024 at 7:48 PM John Naylor wrote: > > v77-0001 > > > > - dead_items = (VacDeadItems *) > > palloc(vac_max_items_to_alloc_size(max_items)); > > - dead_items->max_items = max_items; > > - dead_items->num_items = 0; > >

Re: Add Index-level REINDEX with multiple jobs

2024-03-25 Thread Alexander Korotkov
On Mon, Mar 25, 2024 at 4:47 AM Richard Guo wrote: > On Mon, Mar 25, 2024 at 10:07 AM Tom Lane wrote: >> >> Alexander Korotkov writes: >> > Here goes the revised patch. I'm going to push this if there are no >> > objections. >> >> Quite a lot of the buildfarm is complaining about this: >> >>

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread shveta malik
On Mon, Mar 25, 2024 at 11:53 AM shveta malik wrote: > > On Mon, Mar 25, 2024 at 10:33 AM shveta malik wrote: > > > > On Sun, Mar 24, 2024 at 3:06 PM Bharath Rupireddy > > wrote: > > > > > > I've attached the v18 patch set here. I have one concern, for synced slots on standby, how do we

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-03-25 Thread Donghang Lin
Hi Tomas, > > On Sat, Feb 17, 2024 at 2:31 PM Tomas Vondra < tomas.von...@enterprisedb.com> wrote: > Hi David, > > Do you plan to work continue working on this patch? I did take a look, > and on the whole it looks reasonable - it modifies the right places etc. > > I think there are two things that

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Bharath Rupireddy
On Mon, Mar 25, 2024 at 1:37 PM Bertrand Drouvot wrote: > > > > I have one concern, for synced slots on standby, how do we disallow > > > invalidation due to inactive-timeout immediately after promotion? > > > > > > For synced slots, last_inactive_time and inactive_timeout are both > > > set. > >

Re: Statistics Import and Export

2024-03-25 Thread Ashutosh Bapat
Hi Corey, On Sat, Mar 23, 2024 at 7:21 AM Corey Huinker wrote: > v12 attached. > > 0001 - > > Some random comments +SELECT +format('SELECT pg_catalog.pg_set_attribute_stats( ' +|| 'relation => %L::regclass::oid, attname => %L::name, ' +|| 'inherited => %L::boolean,

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-03-25 Thread Alexander Korotkov
On Tue, Sep 20, 2022 at 3:21 PM Robert Haas wrote: > On Mon, Sep 19, 2022 at 4:42 PM Dmitry Koval wrote: > > Thanks for comments and advice! > > I thought about this problem and discussed about it with colleagues. > > Unfortunately, I don't know of a good general solution. > > Yeah, me neither.

Re: A problem about partitionwise join

2024-03-25 Thread Ashutosh Bapat
On Mon, Mar 25, 2024 at 9:01 AM Richard Guo wrote: > > create table p (k1 int, k2 int, val int) partition by range(k1, k2); > create table p_1 partition of p for values from (1,1) to (10,100); > create table p_2 partition of p for values from (10,100) to (20,200); > > set

Re: session username in default psql prompt?

2024-03-25 Thread Jelte Fennema-Nio
On Sat, 23 Mar 2024 at 00:34, Tom Lane wrote: > Prompt space is > expensive and precious, at least for people who aren't in the > habit of working in very wide windows. That problem seems easy to address by adding a newline into the default prompt. Something like this: \set PROMPT1 '%n@%~%R%\n#

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2024-03-25 Thread Richard Guo
On Mon, Mar 25, 2024 at 5:17 PM Amit Langote wrote: > I've pushed this now. > > When updating the commit message, I realized that you had made the > right call to divide the the changes around not translating the dummy > SpecialJoinInfos into a separate patch. So, I pushed it like that. Sorry

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2024-03-25 Thread Amit Langote
On Mon, Mar 25, 2024 at 7:25 PM Richard Guo wrote: > On Mon, Mar 25, 2024 at 5:17 PM Amit Langote wrote: >> I've pushed this now. >> >> When updating the commit message, I realized that you had made the >> right call to divide the the changes around not translating the dummy >> SpecialJoinInfos

Re: DOCS: add helpful partitioning links

2024-03-25 Thread Ashutosh Bapat
On Fri, Mar 22, 2024 at 10:58 PM Robert Treat wrote: > > v5 patch attached which I think further improves clarity/brevity of > this section. I've left the patch name the same for simplicity, but > I'd agree that the commit would now be more along the lines of editing > / improvements /

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread shveta malik
On Mon, Mar 25, 2024 at 5:10 PM Amit Kapila wrote: > > I think we should keep pg_alter_replication_slot() as the last > priority among the remaining patches for this release. Let's try to > first finish the primary functionality of inactive_timeout patch. > Otherwise, I agree that the problem

Re: speed up a logical replica setup

2024-03-25 Thread Peter Eisentraut
On 25.03.24 13:36, Bharath Rupireddy wrote: On Mon, Mar 25, 2024 at 5:25 PM Peter Eisentraut wrote: I have committed your version v33. Looks like BF animals aren't happy, please check - https://buildfarm.postgresql.org/cgi-bin/show_failures.pl. Looks like sanitizer failures. There were a

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-25 Thread Masahiko Sawada
On Mon, Mar 25, 2024 at 3:25 PM John Naylor wrote: > > On Fri, Mar 22, 2024 at 12:20 PM Masahiko Sawada > wrote: > > > > On Thu, Mar 21, 2024 at 7:48 PM John Naylor wrote: > > > > v77-0001 > > > > > > - dead_items = (VacDeadItems *) > > > palloc(vac_max_items_to_alloc_size(max_items)); > > >

Re: session username in default psql prompt?

2024-03-25 Thread Robert Haas
On Mon, Mar 25, 2024 at 4:30 AM Jelte Fennema-Nio wrote: > That problem seems easy to address by adding a newline into the > default prompt. Ugh. Please, no! -- Robert Haas EDB: http://www.enterprisedb.com

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Bertrand Drouvot
Hi, On Mon, Mar 25, 2024 at 02:07:21PM +0530, shveta malik wrote: > On Mon, Mar 25, 2024 at 1:37 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Mon, Mar 25, 2024 at 12:59:52PM +0530, Amit Kapila wrote: > > > On Mon, Mar 25, 2024 at 12:43 PM shveta malik > > > wrote: > > > > > > > > On

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread shveta malik
On Mon, Mar 25, 2024 at 1:37 PM Bertrand Drouvot wrote: > > Hi, > > Yeah, and I can see last_inactive_time is moving on the standby (while not the > case on the primary), probably due to the sync worker slot acquisition/release > which does not seem right. > Yes, you are right,

Re: session username in default psql prompt?

2024-03-25 Thread jian he
On Mon, Mar 25, 2024 at 6:32 PM Jelte Fennema-Nio wrote: > > Obviously I meant to put the \n before the %: > \set PROMPT1 '%n@%~%R\n%# ' > transaction related information lost. for example: jian@src6= # begin; BEGIN jian@src6= # select 1/0; 2024-03-25 18:37:59.313 CST [15252] ERROR: division

Re: session username in default psql prompt?

2024-03-25 Thread Jelte Fennema-Nio
On Mon, 25 Mar 2024 at 11:40, jian he wrote: > transaction related information lost. Ah yeah, it seems I somehow lost the %x How about: \set PROMPT1 '%n@%~\n%R%x%# ' Or maybe even this more verbose one, which closely matches the postgresql:// connection string format: \set PROMPT1

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Amit Kapila
On Mon, Mar 25, 2024 at 2:40 PM shveta malik wrote: > > On Mon, Mar 25, 2024 at 1:37 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > Yeah, and I can see last_inactive_time is moving on the standby (while not > > the > > case on the primary), probably due to the sync worker slot > >

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2024-03-25 Thread Amit Langote
On Fri, Mar 22, 2024 at 7:48 PM Ashutosh Bapat wrote: > With the later code, semi_rhs_expr is indeed free'able. It wasn't so when I > wrote the patches. Thanks for updating the comment. I wish we would have > freeObject(). Nothing that can be done for this patch. Yes. > Attached patches >

RE: speed up a logical replica setup

2024-03-25 Thread Hayato Kuroda (Fujitsu)
> IIUC, added options were inspired by Tomas. And he said the limitation > (pub/sub/slot > name cannot be specified) was acceptable for the first version. I agree with > him. > (To be honest, I feel that options should be fewer for the first release) Just to confirm - I don't think it is not

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Bertrand Drouvot
Hi, On Mon, Mar 25, 2024 at 02:39:50PM +0530, shveta malik wrote: > I am listing the concerns raised by me: > 3) alter replication slot to alter inactive_timout for synced slots on > standby, should this be allowed? I don't think it should be allowed. Regards, -- Bertrand Drouvot PostgreSQL

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2024-03-25 Thread Ashutosh Bapat
On Mon, Mar 25, 2024 at 4:01 PM Amit Langote wrote: > On Mon, Mar 25, 2024 at 7:25 PM Richard Guo > wrote: > > On Mon, Mar 25, 2024 at 5:17 PM Amit Langote > wrote: > >> I've pushed this now. > >> > >> When updating the commit message, I realized that you had made the > >> right call to divide

Re: speed up a logical replica setup

2024-03-25 Thread Peter Eisentraut
On 22.03.24 04:31, Euler Taveira wrote: On Thu, Mar 21, 2024, at 6:49 AM, Shlok Kyal wrote: There is a compilation error while building postgres with the patch due to a recent commit. I have attached a top-up patch v32-0003 to resolve this compilation error. I have not updated the version of

Re: Proposal for Resumable Vacuum (again ...)

2024-03-25 Thread Jay
Hi All, A revised proposal with few minor corrections (thanks to wolfgang20121...@yahoo.de for pointing the error): I am aware of few previous attempts and discussions on this topic (eventually shelved or didn't materialize): -

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Amit Kapila
On Mon, Mar 25, 2024 at 3:31 PM Bharath Rupireddy wrote: > > Right. Done that way i.e. not setting the last_inactive_time for slots > both while releasing the slot and restoring from the disk. > > Also, I've added a TAP function to check if the captured times are > sane per Bertrand's review

Re: Add bump memory context type and use it for tuplesorts

2024-03-25 Thread David Rowley
On Tue, 5 Mar 2024 at 15:42, David Rowley wrote: > The query I ran was: > > select chksz,mtype,pg_allocate_memory_test_reset(chksz, > 1024*1024,1024*1024*1024, mtype) from (values(8),(16),(32),(64)) > sizes(chksz),(values('aset'),('generation'),('slab'),('bump')) > cxt(mtype) order by

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread shveta malik
On Mon, Mar 25, 2024 at 1:37 PM Bertrand Drouvot wrote: > > Hi, > > On Mon, Mar 25, 2024 at 12:59:52PM +0530, Amit Kapila wrote: > > On Mon, Mar 25, 2024 at 12:43 PM shveta malik > > wrote: > > > > > > On Mon, Mar 25, 2024 at 11:53 AM shveta malik > > > wrote: > > > > > > > > On Mon, Mar 25,

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2024-03-25 Thread Ashutosh Bapat
On Mon, Mar 25, 2024 at 2:47 PM Amit Langote wrote: > > Attached patches > > Squashed your 0001 and 0002 into 0001. I have also reworded the commit > message to reflect the latest code changes. > > 0002 has some minor edits. Please feel free to include or reject when > committing the patch. > >

Re: Add new error_action COPY ON_ERROR "log"

2024-03-25 Thread Bharath Rupireddy
On Mon, Mar 25, 2024 at 10:42 AM Masahiko Sawada wrote: > > The current approach, eliminating the duplicated information in > CONTEXT, seems good to me. Thanks for looking into it. > One question about the latest (v8) patch: > > + else > + ereport(NOTICE,

Re: speed up a logical replica setup

2024-03-25 Thread Bharath Rupireddy
On Mon, Mar 25, 2024 at 5:25 PM Peter Eisentraut wrote: > > I have committed your version v33. Looks like BF animals aren't happy, please check - https://buildfarm.postgresql.org/cgi-bin/show_failures.pl. -- Bharath Rupireddy PostgreSQL Contributors Team RDS Open Source Databases Amazon Web

Re: Recording whether Heap2/PRUNE records are from VACUUM or from opportunistic pruning (Was: Show various offset arrays for heap WAL records)

2024-03-25 Thread Heikki Linnakangas
On 09/12/2023 23:48, Peter Geoghegan wrote: On Tue, Mar 21, 2023 at 3:37 PM Peter Geoghegan wrote: I think that we should do something like the attached, to completely avoid this ambiguity. This patch adds a new XLOG_HEAP2 bit that's similar to XLOG_HEAP_INIT_PAGE -- XLOG_HEAP2_BYVACUUM. This

Re: session username in default psql prompt?

2024-03-25 Thread Jelte Fennema-Nio
On Mon, 25 Mar 2024 at 09:30, Jelte Fennema-Nio wrote: > \set PROMPT1 '%n@%~%R%\n# ' Obviously I meant to put the \n before the %: \set PROMPT1 '%n@%~%R\n%# '

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread shveta malik
On Mon, Mar 25, 2024 at 3:31 PM Bharath Rupireddy wrote: > > Right. Done that way i.e. not setting the last_inactive_time for slots > both while releasing the slot and restoring from the disk. > > Also, I've added a TAP function to check if the captured times are > sane per Bertrand's review

Re: MIN/MAX functions for a record

2024-03-25 Thread Aleksander Alekseev
Hi, > I don't see any copying happening in, say, text_larger or > numeric_larger, so this shouldn't need to either. > > Personally I'd write "record_cmp(fcinfo) > 0" rather than indirecting > through record_gt. The way you have it is not strictly correct anyhow: > you're cheating by not using

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-25 Thread Heikki Linnakangas
On 24/03/2024 21:55, Melanie Plageman wrote: On Sat, Mar 23, 2024 at 01:09:30AM +0200, Heikki Linnakangas wrote: On 20/03/2024 21:17, Melanie Plageman wrote: There is another patch in the commitfest that touches this area: "Recording whether Heap2/PRUNE records are from VACUUM or from

Re: speed up a logical replica setup

2024-03-25 Thread Euler Taveira
On Mon, Mar 25, 2024, at 8:55 AM, Peter Eisentraut wrote: > On 22.03.24 04:31, Euler Taveira wrote: > > On Thu, Mar 21, 2024, at 6:49 AM, Shlok Kyal wrote: > >> There is a compilation error while building postgres with the patch > >> due to a recent commit. I have attached a top-up patch v32-0003

RE: Popcount optimization using AVX512

2024-03-25 Thread Amonson, Paul D
> -Original Message- > From: Amonson, Paul D > Sent: Monday, March 25, 2024 8:20 AM > To: Tom Lane > Cc: David Rowley ; Nathan Bossart > ; Andres Freund ; Alvaro > Herrera ; Shankaran, Akash > ; Noah Misch ; Matthias > van de Meent ; pgsql- > hack...@lists.postgresql.org > Subject: RE:

Re: pgsql: Clean up role created in new subscription test.

2024-03-25 Thread Andres Freund
Hi, On 2024-01-19 15:40:21 +0100, Peter Eisentraut wrote: > On 19.01.24 15:26, Daniel Gustafsson wrote: > > > On 18 Jan 2024, at 01:57, vignesh C wrote: > > > > > There are a lot of failures in CFBot at [1] with: > > > > > More details of the same are available at [2]. > > > Do we need to

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Nathan Bossart
On Mon, Mar 25, 2024 at 04:49:12PM +, Bertrand Drouvot wrote: > On Mon, Mar 25, 2024 at 12:25:37PM -0400, Robert Haas wrote: >> In the same vein, I think deactivated_at or inactive_since might be >> good names to consider. I think they get at the same thing as >> released_time, but they avoid

Re: Possibility to disable `ALTER SYSTEM`

2024-03-25 Thread Robert Haas
On Tue, Mar 19, 2024 at 9:13 AM Jelte Fennema-Nio wrote: > On Mon, 18 Mar 2024 at 18:27, Robert Haas wrote: > > I think for now we > > should just file this under "Other platforms and clients," which only > > has one existing setting. If the number of settings of this type > > grows, we can

Re: Built-in CTYPE provider

2024-03-25 Thread Jeff Davis
On Mon, 2024-03-25 at 08:29 +0100, Peter Eisentraut wrote: > Right.  I thought when you said there is an ICU configuration for it, > that it might be like collation options that you specify in the > locale > string.  But it appears it is only an internal API setting.  So that, > in > my mind,

Re: Possibility to disable `ALTER SYSTEM`

2024-03-25 Thread Robert Haas
On Mon, Mar 25, 2024 at 1:47 PM Tom Lane wrote: > FWIW, I never objected to the idea of being able to disable ALTER > SYSTEM. I felt that it ought to be part of a larger feature that > would provide a more bulletproof guarantee that a superuser can't > alter the system configuration; but I'm

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2024-03-25 Thread Noah Misch
On Mon, Mar 25, 2024 at 12:03:10PM -0400, Peter Geoghegan wrote: > On Sun, Mar 24, 2024 at 10:03 PM Noah Misch wrote: > Separately, I now see that the committed patch just reuses the code > that has long been used to check that things are in the correct order > across page boundaries: this is

Re: Large block sizes support in Linux

2024-03-25 Thread Bruce Momjian
On Mon, Mar 25, 2024 at 02:53:56PM +0100, Pankaj Raghav wrote: > This is an excellent question that needs a bit of community discussion to > expose a device agnostic value that userspace can trust. > > There might be a talk this year at LSFMM about untorn writes[1] in buffered IO > path. I will

Re: Possibility to disable `ALTER SYSTEM`

2024-03-25 Thread Jelte Fennema-Nio
On Mon, 25 Mar 2024 at 20:16, Bruce Momjian wrote: > I am wondering if the fact that you would be able to do: > > ALTER SYSTEM SET externally_managed_configuration = false > > and then be unable to use ALTER SYSTEM to revert the change is > significant. This is not possible, due to the

Re: Add bump memory context type and use it for tuplesorts

2024-03-25 Thread David Rowley
On Tue, 26 Mar 2024 at 03:53, Tom Lane wrote: > I agree with this completely. However, the current design for chunk > headers is mighty restrictive about how many kinds of contexts we can > have. We need to open that back up. Andres mentioned how we could do this in [1]. One possible issue

Re: Possibility to disable `ALTER SYSTEM`

2024-03-25 Thread Bruce Momjian
On Mon, Mar 25, 2024 at 09:40:55PM +0100, Jelte Fennema-Nio wrote: > On Mon, 25 Mar 2024 at 20:16, Bruce Momjian wrote: > > I am wondering if the fact that you would be able to do: > > > > ALTER SYSTEM SET externally_managed_configuration = false > > > > and then be unable to use ALTER

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-03-25 Thread Melanie Plageman
On Mon, Mar 25, 2024 at 2:29 AM Donghang Lin wrote: > > > > On Sat, Feb 17, 2024 at 2:31 PM Tomas Vondra > > wrote: > > 2) Leader vs. worker counters > > > > It seems to me this does nothing to add the per-worker values from "Heap > > Blocks" into the leader, which means we get stuff like this:

Re: Possibility to disable `ALTER SYSTEM`

2024-03-25 Thread Magnus Hagander
On Mon, Mar 25, 2024 at 7:27 PM Tom Lane wrote: > Robert Haas writes: > > OK, great. The latest patch doesn't specifically talk about backing it > > up with filesystem-level controls, but it does clearly say that this > > feature is not going to stop a determined superuser from bypassing the >

Re: Large block sizes support in Linux

2024-03-25 Thread Pankaj Raghav
On 23/03/2024 03:41, Bruce Momjian wrote: > On Fri, Mar 22, 2024 at 10:31:11PM +0100, Tomas Vondra wrote: >> Right, but things change over time - current storage devices support >> much larger sectors (LBA format), usually 4K. And if you do I/O with >> this size, it's usually atomic. >> >> AFAIK

Re: Large block sizes support in Linux

2024-03-25 Thread Pankaj Raghav
Hi Tomas and Bruce, >>> My knowledge of Postgres internals is limited, so I'm wondering if there >>> are any optimizations or potential optimizations that Postgres could >>> leverage once we have LBS support on Linux? >> >> We have discussed this in the past, and in fact in the early years we >>

Re: Large block sizes support in Linux

2024-03-25 Thread Pankaj Raghav
Hi Thomas, On 23/03/2024 05:53, Thomas Munro wrote: > On Fri, Mar 22, 2024 at 10:56 PM Pankaj Raghav (Samsung) > wrote: >> My team and I have been working on adding Large block size(LBS) >> support to XFS in Linux[1]. Once this feature lands upstream, we will be >> able to create XFS with FS

Re: Possibility to disable `ALTER SYSTEM`

2024-03-25 Thread Tom Lane
Robert Haas writes: > Since those are just minor points, that brings us to the question of > whether there is consensus to proceed with this. I believe that there > is a clear consensus that there should be some way to disable ALTER > SYSTEM. Sure, some people, particularly Tom, disagree, but I

Re: Possibility to disable `ALTER SYSTEM`

2024-03-25 Thread Tom Lane
Robert Haas writes: > OK, great. The latest patch doesn't specifically talk about backing it > up with filesystem-level controls, but it does clearly say that this > feature is not going to stop a determined superuser from bypassing the > feature, which I think is the appropriate level of detail.

Re: Popcount optimization using AVX512

2024-03-25 Thread Nathan Bossart
On Mon, Mar 25, 2024 at 06:42:36PM +, Amonson, Paul D wrote: > Ok, CI turned green after my re-post of the patches. Can this please get > merged? Thanks for the new patches. I intend to take another look soon. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Propagate pathkeys from CTEs up to the outer query

2024-03-25 Thread Tom Lane
Richard Guo writes: > This patch was initially posted in that same thread and has received > some comments from Tom in [2]. Due to the presence of multiple patches > in that thread, it has led to confusion. So fork a new thread here > specifically dedicated to discussing the patch about

Re: psql not responding to SIGINT upon db reconnection

2024-03-25 Thread Robert Haas
On Fri, Mar 22, 2024 at 4:58 PM Tristan Partin wrote: > I had a question about parameter naming. Right now I have a mix of > camel-case and snake-case in the function signature since that is what > I inherited. Should I change that to be consistent? If so, which case > would you like? Uh...

Re: Large block sizes support in Linux

2024-03-25 Thread Thomas Munro
On Tue, Mar 26, 2024 at 3:34 AM Pankaj Raghav wrote: > One question: Does ZFS do something like FUA request to force the device > to clear the cache before it can update the node to point to the new page? > > If it doesn't do it, there is no guarantee from device to update the data > atomically

Re: Catalog domain not-null constraints

2024-03-25 Thread Dean Rasheed
On Fri, 22 Mar 2024 at 08:28, jian he wrote: > > On Thu, Mar 21, 2024 at 7:23 PM Peter Eisentraut wrote: > > > > Hmm. CREATE DOMAIN uses column constraint syntax, but ALTER DOMAIN uses > > table constraint syntax. Attached is a patch to try to sort this out. > > also you should also change

Re: Possibility to disable `ALTER SYSTEM`

2024-03-25 Thread Robert Haas
On Mon, Mar 25, 2024 at 2:26 PM Tom Lane wrote: > I wonder whether this feature should include teaching the server > to ignore postgresql.auto.conf altogether, which would make it > relatively easy to get to a bulletproof configuration. This has been debated a few times on the thread already,

Re: Possibility to disable `ALTER SYSTEM`

2024-03-25 Thread Bruce Momjian
On Mon, Mar 25, 2024 at 01:29:46PM -0400, Robert Haas wrote: > What is less clear is whether there is a consensus in favor of this > particular method of disabling ALTER SYSTEM, namely, via a GUC. The > two alternate approaches that seem to enjoy some level of support are > (a) an extension or (b)

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-25 Thread Heikki Linnakangas
On 24/03/2024 18:32, Melanie Plageman wrote: On Thu, Mar 21, 2024 at 9:28 AM Heikki Linnakangas wrote: In heap_page_prune_and_freeze(), we now do some extra work on each live tuple, to set the all_visible_except_removable correctly. And also to update live_tuples, recently_dead_tuples and

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Nathan Bossart
I apologize that I haven't been able to keep up with this thread for a while, but I'm happy to see the continued interest in $SUBJECT. On Sun, Mar 24, 2024 at 03:05:44PM +0530, Bharath Rupireddy wrote: > This commit particularly lets one specify the inactive_timeout for > a slot via SQL functions

Re: New Table Access Methods for Multi and Single Inserts

2024-03-25 Thread Bharath Rupireddy
On Sat, Mar 23, 2024 at 5:47 AM Jeff Davis wrote: > > Comments: Thanks for looking into it. > * Do I understand correctly that CMV, RMV, and CTAS experience a > performance benefit, but COPY FROM does not? And is that because COPY > already used table_multi_insert, whereas CMV and RMV did not?

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Bharath Rupireddy
On Tue, Mar 26, 2024 at 1:30 AM Nathan Bossart wrote: > > On Mon, Mar 25, 2024 at 04:49:12PM +, Bertrand Drouvot wrote: > > On Mon, Mar 25, 2024 at 12:25:37PM -0400, Robert Haas wrote: > >> In the same vein, I think deactivated_at or inactive_since might be > >> good names to consider. I

Re: Why is parula failing?

2024-03-25 Thread David Rowley
On Thu, 21 Mar 2024 at 14:19, Tom Lane wrote: > > David Rowley writes: > > We could also do something like the attached just in case we're > > barking up the wrong tree. > > Yeah, checking indisvalid isn't a bad idea. I'd put another > one further down, just before the DROP of table ab, so we >

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Robert Haas
On Mon, Mar 25, 2024 at 10:02 AM Amit Kapila wrote: > We considered the other two names as last_inactive_at and > last_active_time. For the first (last_inactive_at), there was an > argument that most other fields that display time ends with _time. For > the second (last_active_time), there was an

Re: Popcount optimization using AVX512

2024-03-25 Thread Tom Lane
"Amonson, Paul D" writes: > I am re-posting the patches as CI for Mac failed (CI error not code/test > error). The patches are the same as last time. Just for a note --- the cfbot will re-test existing patches every so often without needing a bump. The current cycle period seems to be about

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Bertrand Drouvot
Hi, On Mon, Mar 25, 2024 at 08:38:16PM +0530, Amit Kapila wrote: > On Mon, Mar 25, 2024 at 7:51 PM Robert Haas wrote: > > > > On Mon, Mar 25, 2024 at 10:02 AM Amit Kapila > > wrote: > > > We considered the other two names as last_inactive_at and > > > last_active_time. For the first

Re: [PATCH] plpython function causes server panic

2024-03-25 Thread Robert Haas
On Sat, Mar 23, 2024 at 12:31 PM Tom Lane wrote: > However, the calling logic seems a bit shy of a load, in that it > trusts IsInParallelMode() completely to decide whether to check for > leaked parallel contexts. So we'd miss the case where somebody did > ExitParallelMode without having cleaned

Re: Slow GRANT ROLE on PostgreSQL 16 with thousands of ROLEs

2024-03-25 Thread Nathan Bossart
On Mon, Mar 25, 2024 at 11:08:39AM -0400, Tom Lane wrote: > * The magic constants (crossover list length and bloom filter size) > need some testing to see if there are better values. They should > probably be made into named #defines, too. I suspect, with little > proof, that the bloom filter

Re: documentation structure

2024-03-25 Thread Peter Eisentraut
On 22.03.24 14:59, Robert Haas wrote: And I don't believe that if someone were writing a physical book about PostgreSQL from scratch, they'd ever end up with a top-level chapter that looks anything like our GiST chapter. All of the index AM chapters are quite obviously clones of each other, and

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Bertrand Drouvot
Hi, On Mon, Mar 25, 2024 at 12:25:37PM -0400, Robert Haas wrote: > On Mon, Mar 25, 2024 at 12:12 PM Bertrand Drouvot > wrote: > > Would "released_time" sounds better? (at the end this is exactly what it > > does > > represent unless for the case where it is restored from disk for which the > >

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Robert Haas
On Mon, Mar 25, 2024 at 7:11 AM Amit Kapila wrote: > Track last_inactive_time in pg_replication_slots. > > This commit adds a new property called last_inactive_time for slots. It is > set to 0 whenever a slot is made active/acquired and set to the current > timestamp whenever the slot is

Re: add AVX2 support to simd.h

2024-03-25 Thread Nathan Bossart
On Mon, Mar 25, 2024 at 10:03:27AM +0700, John Naylor wrote: > Seems pretty good. It'd be good to see the results of 2- vs. > 4-register before committing, because that might lead to some > restructuring, but maybe it won't, and v8 is already an improvement > over HEAD. I tested this the other

Re: Popcount optimization using AVX512

2024-03-25 Thread Joe Conway
On 3/25/24 11:12, Tom Lane wrote: "Amonson, Paul D" writes: I am re-posting the patches as CI for Mac failed (CI error not code/test error). The patches are the same as last time. Just for a note --- the cfbot will re-test existing patches every so often without needing a bump. The current

Re: Potential issue in ecpg-informix decimal converting functions

2024-03-25 Thread Daniel Gustafsson
> On 6 Mar 2024, at 20:12, a.ima...@postgrespro.ru wrote: > I agree with the proposed changes in favor of backward compatibility. I went ahead to pushed this after another look. I'm a bit hesitant to backpatch this since there are no reports against it, and I don't have good sense for how ECPG

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Amit Kapila
On Mon, Mar 25, 2024 at 6:57 PM Robert Haas wrote: > > On Mon, Mar 25, 2024 at 7:11 AM Amit Kapila wrote: > > Track last_inactive_time in pg_replication_slots. > > > > This commit adds a new property called last_inactive_time for slots. It is > > set to 0 whenever a slot is made active/acquired

Re: pg_upgrade --copy-file-range

2024-03-25 Thread Robert Haas
On Sat, Mar 23, 2024 at 9:37 AM Tomas Vondra wrote: > OK, that makes sense. Here's a patch that should work like this - in > copy_file we check if we need to calculate checksums, and either use the > requested copy method, or fall back to the block-by-block copy. +Use efficient file

Re: pg_upgrade --copy-file-range

2024-03-25 Thread Robert Haas
On Sat, Mar 23, 2024 at 9:47 AM Tomas Vondra wrote: > BTW is there a reason why the code calls "write" and not "pg_pwrite"? I think it's mostly because I learned to code a really long time ago. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Add bump memory context type and use it for tuplesorts

2024-03-25 Thread Tomas Vondra
On 3/25/24 12:41, David Rowley wrote: > On Tue, 5 Mar 2024 at 15:42, David Rowley wrote: >> The query I ran was: >> >> select chksz,mtype,pg_allocate_memory_test_reset(chksz, >> 1024*1024,1024*1024*1024, mtype) from (values(8),(16),(32),(64)) >>

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Bertrand Drouvot
Hi, On Mon, Mar 25, 2024 at 08:59:55PM +0530, Amit Kapila wrote: > On Mon, Mar 25, 2024 at 8:46 PM Bertrand Drouvot > wrote: > > > > On Mon, Mar 25, 2024 at 08:38:16PM +0530, Amit Kapila wrote: > > > On Mon, Mar 25, 2024 at 7:51 PM Robert Haas wrote: > > > > > > > > On Mon, Mar 25, 2024 at

Re: [PATCH] plpython function causes server panic

2024-03-25 Thread Robert Haas
On Mon, Mar 25, 2024 at 11:36 AM Tom Lane wrote: > By that logic, we should rip out every Assert in the system, as well > as all of the (extensive) resource leak checking that already happens > during CommitTransaction. We've always felt that those leak checks > were worth the cost to help us

Re: Avoiding inadvertent debugging mode for pgbench

2024-03-25 Thread Nathan Bossart
Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Slow GRANT ROLE on PostgreSQL 16 with thousands of ROLEs

2024-03-25 Thread Nathan Bossart
On Fri, Mar 22, 2024 at 04:41:49PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> LGTM > > Thanks for looking, I'll push that shortly. Are there any changes you'd like to see for the Bloom patch [0]? I'd like to see about getting that committed for v17. One thing that crossed my mind is

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Amit Kapila
On Mon, Mar 25, 2024 at 7:51 PM Robert Haas wrote: > > On Mon, Mar 25, 2024 at 10:02 AM Amit Kapila wrote: > > We considered the other two names as last_inactive_at and > > last_active_time. For the first (last_inactive_at), there was an > > argument that most other fields that display time ends

  1   2   >