Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-20 Thread Bertrand Drouvot
Hi, On Thu, Mar 21, 2024 at 08:47:18AM +0530, Amit Kapila wrote: > On Wed, Mar 20, 2024 at 1:51 PM Bertrand Drouvot > wrote: > > > > On Wed, Mar 20, 2024 at 12:48:55AM +0530, Bharath Rupireddy wrote: > > > > > > 2. last_inactive_at and inactive_timeout are now tracked in on-disk > > >

Re: Trying to build x86 version on windows using meson

2024-03-20 Thread Andrew Dunstan
On Wed, Mar 20, 2024 at 6:21 PM Andres Freund wrote: > Hi, > > On 2024-03-21 11:02:27 +1300, David Rowley wrote: > > On Thu, 21 Mar 2024 at 11:00, Andres Freund wrote: > > > > > > On 2024-03-20 17:49:14 -0400, Dave Cramer wrote: > > > > First off this is on an ARM64 machine > > > > > > Uh,

Re: Proposal: Introduce row-level security templates

2024-03-20 Thread Stojan Dimitrovski
Hi Aadhav Vignesh, Interestingly, Alexander asked for ideas for GSoC projects at Supabase (I'm on the Auth team), and I proposed the RLS templates idea. As you already pointed out, the idea comes out of us seeing how RLS policies are used in real-life and the pain points associated with managing

Re: Table AM Interface Enhancements

2024-03-20 Thread Pavel Borisov
Hi, Alexander! On Wed, 20 Mar 2024 at 09:22, Pavel Borisov wrote: > Hi, Alexander! > > For 0007: > > Code inside > > +heapam_reloptions(char relkind, Datum reloptions, bool validate) > +{ > + if (relkind == RELKIND_RELATION || > + relkind == RELKIND_TOASTVALUE || > + relkind ==

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-20 Thread Bharath Rupireddy
On Thu, Mar 21, 2024 at 8:47 AM Amit Kapila wrote: > > On Wed, Mar 20, 2024 at 1:51 PM Bertrand Drouvot > wrote: > > > > On Wed, Mar 20, 2024 at 12:48:55AM +0530, Bharath Rupireddy wrote: > > > > > > 2. last_inactive_at and inactive_timeout are now tracked in on-disk > > > replication slot data

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-20 Thread Bharath Rupireddy
On Thu, Mar 21, 2024 at 9:07 AM Amit Kapila wrote: > > I also don't see any obvious problem with such an API. However, this > is not a good time to invent new APIs. Let's keep the feature simple > and then we can extend it in the next version after more discussion > and probably by that time we

Re: speed up a logical replica setup

2024-03-20 Thread Euler Taveira
On Wed, Mar 20, 2024, at 7:16 AM, Shubham Khanna wrote: > On Tue, Mar 19, 2024 at 8:54 PM vignesh C wrote: > > > > If you are not planning to have the checks for name length, this could > > alternatively be fixed by including database id also while querying > > pg_subscription like below in

Re: speed up a logical replica setup

2024-03-20 Thread Euler Taveira
On Tue, Mar 19, 2024, at 8:57 AM, Peter Eisentraut wrote: > On 19.03.24 12:26, Shlok Kyal wrote: > > > > I have added a top-up patch v30-0003. The issue in [1] still exists in > > the v30 patch. I was not able to come up with an approach to handle it > > in the code, so I have added it to the

Re: speed up a logical replica setup

2024-03-20 Thread Euler Taveira
On Mon, Mar 18, 2024, at 2:43 AM, Hayato Kuroda (Fujitsu) wrote: > Thanks for updating the patch. Here are my comments. > I used Grammarly to proofread sentences. > (The tool strongly recommends to use active voice, but I can ignore for now) Thanks for another review. I posted a new patch (v32)

Re: add AVX2 support to simd.h

2024-03-20 Thread John Naylor
On Thu, Mar 21, 2024 at 2:55 AM Nathan Bossart wrote: > > On Wed, Mar 20, 2024 at 09:31:16AM -0500, Nathan Bossart wrote: > > I don't mind removing the 2-register stuff if that's what you think we > > should do. I'm cautiously optimistic that it'd help more than the extra > > branch prediction

Re: speed up a logical replica setup

2024-03-20 Thread Euler Taveira
On Mon, Mar 18, 2024, at 10:52 AM, Peter Eisentraut wrote: > On 16.03.24 16:42, Euler Taveira wrote: > > I'm attaching a new version (v30) that adds: > > I have some review comments and attached a patch with some smaller > fixups (mainly message wording and avoid fixed-size string buffers).

Re: New Table Access Methods for Multi and Single Inserts

2024-03-20 Thread Bharath Rupireddy
On Tue, Mar 19, 2024 at 10:40 AM Masahiko Sawada wrote: > > I've not reviewed the patches in depth yet, but run performance tests > for CREATE MATERIALIZED VIEW. The test scenarios is: Thanks for looking into this. > Is there any reason why we copy a buffer-heap tuple to another > buffer-heap

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-20 Thread John Naylor
On Thu, Mar 21, 2024 at 9:37 AM Masahiko Sawada wrote: > > On Wed, Mar 20, 2024 at 11:19 PM John Naylor wrote: > > Are they (the blocks to be precise) really out of order? The VALUES > > statement is ordered, but after inserting it does not output that way. > > I wondered if this is platform

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-20 Thread Amit Kapila
On Thu, Mar 21, 2024 at 5:19 AM Bharath Rupireddy wrote: > > On Wed, Mar 20, 2024 at 7:08 PM Bertrand Drouvot > wrote: > > > > Regarding v12-0004: "Allow setting inactive_timeout in the replication > > command", > > shouldn't we also add an new SQL API say: pg_alter_replication_slot() that > >

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-20 Thread Amit Kapila
On Wed, Mar 20, 2024 at 1:51 PM Bertrand Drouvot wrote: > > On Wed, Mar 20, 2024 at 12:48:55AM +0530, Bharath Rupireddy wrote: > > > > 2. last_inactive_at and inactive_timeout are now tracked in on-disk > > replication slot data structure. > > Should last_inactive_at be tracked on disk? Say the

Re: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-03-20 Thread vignesh C
On Wed, 20 Mar 2024 at 17:09, Amit Kapila wrote: > > On Tue, Mar 19, 2024 at 5:18 PM Hayato Kuroda (Fujitsu) > wrote: > > > > Thanks for giving comments! > > > > > This behavior makes sense to me. But do we want to handle the case of > > > using environment variables too? > > > > Yeah, v5 does

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-03-20 Thread Noah Misch
On Mon, Mar 18, 2024 at 07:40:10PM +0100, Alvaro Herrera wrote: > I enabled the test again and also pushed the changes to dblink, > isolationtester and fe_utils (which AFAICS is used by pg_dump, I recommend adding a libpqsrv_cancel() function to libpq-be-fe-helpers.h, to use from dblink and

Re: Recent 027_streaming_regress.pl hangs

2024-03-20 Thread Andres Freund
Hi, On 2024-03-20 17:41:47 -0700, Andres Freund wrote: > There's a lot of other animals on the same machine, however it's rarely fuly > loaded (with either CPU or IO). > > I don't think the test just being slow is the issue here, e.g. in the last > failing iteration > > [...] > > I suspect we

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-20 Thread Masahiko Sawada
On Wed, Mar 20, 2024 at 11:19 PM John Naylor wrote: > > On Wed, Mar 20, 2024 at 8:30 PM Masahiko Sawada wrote: > > I forgot to report the results. Yes, I did some tests where I inserted > > many TIDs to make the tidstore use several GB memory. I did two cases: > > > > 1. insert 100M blocks of

Re: Add pg_basetype() function to obtain a DOMAIN base type

2024-03-20 Thread jian he
On Mon, Mar 18, 2024 at 11:43 PM Tom Lane wrote: > > Alexander Korotkov writes: > > On Mon, Mar 18, 2024 at 2:01 AM jian he wrote: > >> ` > >> Datum > >> pg_basetype(PG_FUNCTION_ARGS) > >> { > >> Oid oid; > >> > >> oid = get_fn_expr_argtype(fcinfo->flinfo, 0); > >>

Re: Possibility to disable `ALTER SYSTEM`

2024-03-20 Thread Magnus Hagander
On Wed, Mar 20, 2024 at 8:52 PM Robert Haas wrote: > On Wed, Mar 20, 2024 at 3:17 PM Magnus Hagander > wrote: > > Right, what I meant is that making it a packaging decision is the better > place. Wherever it goes, allowing the administrator to choose what fits > them should be made possible. >

Re: Why is parula failing?

2024-03-20 Thread Tom Lane
David Rowley writes: > We could also do something like the attached just in case we're > barking up the wrong tree. Yeah, checking indisvalid isn't a bad idea. I'd put another one further down, just before the DROP of table ab, so we can see the state both before and after the unstable tests.

Re: Why is parula failing?

2024-03-20 Thread David Rowley
On Thu, 21 Mar 2024 at 12:36, Tom Lane wrote: > So yeah, if we could have log_autovacuum_min_duration = 0 perhaps > that would yield a clue. FWIW, I agree with your earlier statement about it looking very much like auto-vacuum has run on that table, but equally, if something like the pg_index

Re: Flushing large data immediately in pqcomm

2024-03-20 Thread David Rowley
On Thu, 21 Mar 2024 at 13:24, Melih Mutlu wrote: > What if I do a simple comparison like PqSendStart == PqSendPointer instead of > calling pq_is_send_pending() as Heikki suggested, then this check should not > hurt that much. Right? Does that make sense? As I understand the code, there's no

Re: Recent 027_streaming_regress.pl hangs

2024-03-20 Thread Andres Freund
Hi, On 2024-03-14 16:56:39 -0400, Tom Lane wrote: > Thomas Munro writes: > > On Fri, Mar 15, 2024 at 7:00 AM Alexander Lakhin > > wrote: > >> Could it be that the timeout (360 sec?) is just not enough for the test > >> under the current (changed due to switch to meson) conditions? > > > But

Re: Popcount optimization using AVX512

2024-03-20 Thread David Rowley
On Wed, 20 Mar 2024 at 11:56, Amonson, Paul D wrote: > Changed in this patch set. > > * Rebased. > * Direct *slow* calls via macros as shown in example patch. > * Changed the choose filename to be platform specific as suggested. > * Falls back to intermediate "Fast" methods if AVX512 is not

Re: Flushing large data immediately in pqcomm

2024-03-20 Thread Melih Mutlu
David Rowley , 21 Mar 2024 Per, 00:54 tarihinde şunu yazdı: > On Fri, 15 Mar 2024 at 02:03, Jelte Fennema-Nio > wrote: > > > > On Thu, 14 Mar 2024 at 13:12, Robert Haas wrote: > > > > > > On Thu, Mar 14, 2024 at 7:22 AM Melih Mutlu > wrote: > > > > 1- Even though I expect both the patch and

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-20 Thread Bharath Rupireddy
On Wed, Mar 20, 2024 at 7:08 PM Bertrand Drouvot wrote: > > Regarding v12-0004: "Allow setting inactive_timeout in the replication > command", > shouldn't we also add an new SQL API say: pg_alter_replication_slot() that > would > allow to change the timeout property? > > That would allow users

Re: Patch: Add parse_type Function

2024-03-20 Thread David E. Wheeler
On Mar 20, 2024, at 17:23, Tom Lane wrote: > Pushed with some editorialization. Mostly, I whacked the > documentation around pretty heavily: we have a convention for what > examples in function descriptions should look like, and this wasn't > it. Not entirely your fault, since some nearby

Re: Why is parula failing?

2024-03-20 Thread Tom Lane
David Rowley writes: > Is it worth running that animal with log_autovacuum_min_duration = 0 > so we can see what's going on in terms of auto-vacuum auto-analyze in > the log? Maybe, but I'm not sure. I thought that if parula were somehow hitting an ill-timed autovac/autoanalyze, it should be

Re: [PATCH] Exponential backoff for auth_delay

2024-03-20 Thread Daniel Gustafsson
> On 20 Mar 2024, at 22:21, Jacob Champion > wrote: > > On Wed, Mar 20, 2024 at 2:15 PM Jacob Champion > wrote: >> I think solutions for case 1 and case 2 are necessarily at odds under >> the current design, if auth_delay relies on slot exhaustion to do its >> work effectively. Weakening that

Re: Trying to build x86 version on windows using meson

2024-03-20 Thread Andres Freund
Hi, On 2024-03-21 11:02:27 +1300, David Rowley wrote: > On Thu, 21 Mar 2024 at 11:00, Andres Freund wrote: > > > > On 2024-03-20 17:49:14 -0400, Dave Cramer wrote: > > > First off this is on an ARM64 machine > > > > Uh, that's a fairly crucial bit - you're actually trying to cross compile > >

Re: Trying to build x86 version on windows using meson

2024-03-20 Thread David Rowley
On Thu, 21 Mar 2024 at 11:00, Andres Freund wrote: > > On 2024-03-20 17:49:14 -0400, Dave Cramer wrote: > > First off this is on an ARM64 machine > > Uh, that's a fairly crucial bit - you're actually trying to cross compile > then. I don't know much about cross compiling on windows, so it's

Re: Trying to build x86 version on windows using meson

2024-03-20 Thread Andres Freund
Hi, On 2024-03-20 17:49:14 -0400, Dave Cramer wrote: > On Wed, 20 Mar 2024 at 17:11, Andres Freund wrote: > > On 2024-03-20 16:14:23 -0400, Dave Cramer wrote: > > > I am getting the following error > > > > > > meson.build:1479:17: ERROR: Can not run test applications in this cross > > >

Re: Flushing large data immediately in pqcomm

2024-03-20 Thread David Rowley
On Fri, 15 Mar 2024 at 01:46, Heikki Linnakangas wrote: > - the "(int *) )" cast is not ok, and will break visibly on > big-endian systems where sizeof(int) != sizeof(size_t). I think fixing this requires adjusting the signature of internal_flush_buffer() to use size_t instead of int. That

Re: Flushing large data immediately in pqcomm

2024-03-20 Thread David Rowley
On Fri, 15 Mar 2024 at 02:03, Jelte Fennema-Nio wrote: > > On Thu, 14 Mar 2024 at 13:12, Robert Haas wrote: > > > > On Thu, Mar 14, 2024 at 7:22 AM Melih Mutlu wrote: > > > 1- Even though I expect both the patch and HEAD behave similarly in case > > > of small data (case 1: 100 bytes), the

Re: Why is parula failing?

2024-03-20 Thread David Rowley
On Wed, 20 Mar 2024 at 08:58, Tom Lane wrote: > I suppose we could attach "autovacuum=off" settings to these tables, > but it doesn't seem to me that that should be necessary. These test > cases are several years old and haven't given trouble before. > Moreover, if that's necessary then there

Re: documentation structure

2024-03-20 Thread Tom Lane
Robert Haas writes: > On Wed, Mar 20, 2024 at 5:05 PM Alvaro Herrera > wrote: >> I think you can achieve this with a much smaller patch that just changes >> the outer tag in each file so that each file is a , then create a >> single file that includes all of these plus an additional outer tag

Re: Patch: Add parse_type Function

2024-03-20 Thread Tom Lane
"David E. Wheeler" writes: > Thanks, fixed in the attached patch. Pushed with some editorialization. Mostly, I whacked the documentation around pretty heavily: we have a convention for what examples in function descriptions should look like, and this wasn't it. Not entirely your fault, since

Re: documentation structure

2024-03-20 Thread Robert Haas
On Wed, Mar 20, 2024 at 5:05 PM Alvaro Herrera wrote: > I think you can achieve this with a much smaller patch that just changes > the outer tag in each file so that each file is a , then create a > single file that includes all of these plus an additional outer tag for > the (or maybe just add

Re: [PATCH] Exponential backoff for auth_delay

2024-03-20 Thread Jacob Champion
On Wed, Mar 20, 2024 at 2:15 PM Jacob Champion wrote: > I think solutions for case 1 and case 2 are necessarily at odds under > the current design, if auth_delay relies on slot exhaustion to do its > work effectively. Weakening that on purpose doesn't make much sense to > me; if a DBA is

Re: Possibility to disable `ALTER SYSTEM`

2024-03-20 Thread David Steele
On 3/20/24 22:30, Michael Banck wrote: On Tue, Mar 19, 2024 at 10:51:50AM -0400, Tom Lane wrote: Heikki Linnakangas writes: Perhaps we could make that even better with a GUC though. I propose a GUC called 'configuration_managed_externally = true / false". If you set it to true, we prevent

Re: Trying to build x86 version on windows using meson

2024-03-20 Thread Andres Freund
Hi, On 2024-03-20 16:14:23 -0400, Dave Cramer wrote: > I am getting the following error > > meson.build:1479:17: ERROR: Can not run test applications in this cross > environment. > > Have configured for amd64_x86 > > Running `meson setup --wipe build --prefix=c:\postgres86` This is not enough

Re: documentation structure

2024-03-20 Thread Alvaro Herrera
On 2024-Mar-20, Robert Haas wrote: > 0003 merges all of the "Internals" chapters whose names are the names > of built-in index access methods (Btree, Gin, etc.) into a single > chapter called "Built-In Index Access Methods". All of these chapters > have a very similar structure and none of them

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-20 Thread Melanie Plageman
On Wed, Mar 20, 2024 at 03:15:49PM +0200, Heikki Linnakangas wrote: > > 0009-: The rest of the v4 patches, rebased over the WAL format changes. I > also added a few small commits for little cleanups that caught my eye, let > me know if you disagree with those. This review is just of the patches

Trying to build x86 version on windows using meson

2024-03-20 Thread Dave Cramer
Greetings, I am getting the following error meson.build:1479:17: ERROR: Can not run test applications in this cross environment. Have configured for amd64_x86 Running `meson setup --wipe build --prefix=c:\postgres86` The docs say it is possible to build postgres for x86. Are there specific

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-20 Thread Peter Geoghegan
On Wed, Mar 20, 2024 at 4:06 PM Melanie Plageman wrote: > > What about the issue of cleanup locks, which aren't needed and aren't > > taken with the current heapam VACUUM record type? Will you preserve > > that aspect of the existing design? > > Yep, we have a flag to indicate whether or not a

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-20 Thread Melanie Plageman
On Wed, Mar 20, 2024 at 4:04 PM Peter Geoghegan wrote: > > On Wed, Mar 20, 2024 at 9:15 AM Heikki Linnakangas wrote: > > > I made it its own sub-record (xlhp_conflict_horizon) less to help with > > > alignment (though we can use all the help we can get there) and more to > > > keep it from

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-20 Thread Peter Geoghegan
On Wed, Mar 20, 2024 at 9:15 AM Heikki Linnakangas wrote: > > I made it its own sub-record (xlhp_conflict_horizon) less to help with > > alignment (though we can use all the help we can get there) and more to > > keep it from getting lost. When you look at heapam_xlog.h, you can see > > what a

Re: Possibility to disable `ALTER SYSTEM`

2024-03-20 Thread Michael Banck
Hi, On Wed, Mar 20, 2024 at 08:11:32PM +0100, Magnus Hagander wrote: > (And FWIW also already solved on debian-based platforms for example, > which but the main config files in /etc with postgres only having read > permissions on them JFTR - Debian/Ubuntu keep postgresql.conf under

Re: add AVX2 support to simd.h

2024-03-20 Thread Nathan Bossart
On Wed, Mar 20, 2024 at 09:31:16AM -0500, Nathan Bossart wrote: > On Wed, Mar 20, 2024 at 01:57:54PM +0700, John Naylor wrote: >> On Tue, Mar 19, 2024 at 11:30 PM Nathan Bossart >> wrote: >>> I tried to trim some of the branches, and came up with the attached patch. >>> I don't think this is

Re: Possibility to disable `ALTER SYSTEM`

2024-03-20 Thread Robert Haas
On Wed, Mar 20, 2024 at 3:17 PM Magnus Hagander wrote: > Right, what I meant is that making it a packaging decision is the better > place. Wherever it goes, allowing the administrator to choose what fits them > should be made possible. +1. Which is also the justification for this patch, when

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-03-20 Thread Matthias van de Meent
On Sat, 16 Mar 2024 at 01:12, Peter Geoghegan wrote: > > On Fri, Mar 8, 2024 at 9:00 AM Matthias van de Meent > wrote: > > I've attached v14, where 0001 is v13, 0002 is a patch with small > > changes + some large comment suggestions, and 0003 which contains > > sorted merge join code for

Re: Test 031_recovery_conflict.pl is not immune to autovacuum

2024-03-20 Thread Melanie Plageman
On Wed, Mar 20, 2024 at 10:00 AM Alexander Lakhin wrote: > > Hello Melanie, > > 20.03.2024 16:15, Melanie Plageman wrote: > > Seems like we could just add autovacuum_enabled=false to the table like > > this: > > diff --git a/src/test/recovery/t/031_recovery_conflict.pl > >

Re: cleanup patches for incremental backup

2024-03-20 Thread Nathan Bossart
On Wed, Mar 20, 2024 at 02:53:01PM -0400, Robert Haas wrote: > On Wed, Mar 20, 2024 at 2:35 PM Nathan Bossart > wrote: >> Committed. > > Thanks. Sorry you had to clean up after me. :-( No worries. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-20 Thread Melanie Plageman
On Wed, Mar 20, 2024 at 03:15:49PM +0200, Heikki Linnakangas wrote: > On 20/03/2024 03:36, Melanie Plageman wrote: > > On Mon, Mar 18, 2024 at 01:15:21AM +0200, Heikki Linnakangas wrote: > > > On 15/03/2024 02:56, Melanie Plageman wrote: > > > > Okay, so I was going to start using xl_heap_prune

Re: Possibility to disable `ALTER SYSTEM`

2024-03-20 Thread Magnus Hagander
On Wed, Mar 20, 2024 at 8:14 PM Robert Haas wrote: > On Wed, Mar 20, 2024 at 3:11 PM Magnus Hagander > wrote: > > I would argue that having the default permissions not allow postgres to > edit it's own config files *except* for postgresql.auto.conf would be a > better default than what we have

Re: Possibility to disable `ALTER SYSTEM`

2024-03-20 Thread Robert Haas
On Wed, Mar 20, 2024 at 3:11 PM Magnus Hagander wrote: > I would argue that having the default permissions not allow postgres to edit > it's own config files *except* for postgresql.auto.conf would be a better > default than what we have now, but that's completely independent of the patch >

Re: Possibility to disable `ALTER SYSTEM`

2024-03-20 Thread Magnus Hagander
On Wed, Mar 20, 2024 at 8:04 PM Robert Haas wrote: > On Wed, Mar 20, 2024 at 11:07 AM Jelte Fennema-Nio > wrote: > > > Ugh, please let's not do this. This was bouncing around in my head > last night, and this is really a quite radical change - especially just to > handle the given ask, which is

Re: WIP Incremental JSON Parser

2024-03-20 Thread Jacob Champion
This new return path... > + if (!tok_done) > + { > + if (lex->inc_state->is_last_chunk) > + return JSON_INVALID_TOKEN; ...also needs to set the token pointers. See one approach in the attached diff, which additionally asserts that we've consumed the entire

Re: Possibility to disable `ALTER SYSTEM`

2024-03-20 Thread Robert Haas
On Wed, Mar 20, 2024 at 11:07 AM Jelte Fennema-Nio wrote: > > Ugh, please let's not do this. This was bouncing around in my head last > > night, and this is really a quite radical change - especially just to > > handle the given ask, which is to prevent a specific command from running. > > Not

Re: Why is parula failing?

2024-03-20 Thread Matthias van de Meent
On Wed, 20 Mar 2024 at 11:50, Matthias van de Meent wrote: > > On Tue, 19 Mar 2024 at 20:58, Tom Lane wrote: > > > > For the last few days, buildfarm member parula has been intermittently > > failing the partition_prune regression test, due to unexpected plan > > changes [1][2][3][4]. The

Re: cleanup patches for incremental backup

2024-03-20 Thread Robert Haas
On Wed, Mar 20, 2024 at 2:35 PM Nathan Bossart wrote: > On Tue, Mar 19, 2024 at 01:15:02PM -0500, Nathan Bossart wrote: > > Assuming there are no objections or feedback, I plan to commit these two > > patches within the next couple of days. > > Committed. Thanks. Sorry you had to clean up after

Re: REVOKE FROM warning on grantor

2024-03-20 Thread Robert Haas
On Wed, Mar 20, 2024 at 1:26 PM Étienne BERSAC wrote: > The usual case is: a superuser grants writers role to alice. In > directory, alice is degraded to readers. ldap2pg is not superuser but > has CREATEROLE. ldap2pg applies the changes. In Postgres 15, revocation > is completed. In Postgres 16,

Re: What is a typical precision of gettimeofday()?

2024-03-20 Thread Jelte Fennema-Nio
On Wed, 20 Mar 2024 at 07:35, Peter Eisentraut wrote: > If we want to be robust without any guarantees from gettimeofday(), then > arguably gettimeofday() is not the right underlying function to use for > UUIDv7. There's also clock_gettime which exposes its resolution using clock_getres

Re: cleanup patches for incremental backup

2024-03-20 Thread Nathan Bossart
On Tue, Mar 19, 2024 at 01:15:02PM -0500, Nathan Bossart wrote: > Assuming there are no objections or feedback, I plan to commit these two > patches within the next couple of days. Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: documentation structure

2024-03-20 Thread Robert Haas
On Wed, Mar 20, 2024 at 1:35 PM Bruce Momjian wrote: > On Wed, Mar 20, 2024 at 12:43:08PM -0400, Robert Haas wrote: > > Overall, I think this achieves a minor but pleasant level of > > de-cluttering of the index. It's going to take a lot more than one > > morning's work to produce a major

Re: [DOCS] HOT - correct claim about indexes not referencing old line pointers

2024-03-20 Thread Robert Haas
On Thu, Mar 14, 2024 at 9:07 PM James Coleman wrote: > Obviously I have reasons for the other changes I made: for example, > "no longer visible" improves the correctness, since being an old > version isn't sufficient. I removed the "In summary" sentence because > it simply doesn't follow from the

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-20 Thread Tomas Vondra
On 3/18/24 16:19, Melanie Plageman wrote: > On Mon, Mar 18, 2024 at 02:10:28PM +0200, Heikki Linnakangas wrote: >> On 14/02/2024 21:42, Andres Freund wrote: >>> On 2024-02-13 18:11:25 -0500, Melanie Plageman wrote: patch 0004 is, I think, a bug fix. see [2]. >>> >>> I'd not quite call it a

Re: UUID v7

2024-03-20 Thread Andrey M. Borodin
> On 19 Mar 2024, at 13:55, Peter Eisentraut wrote: > > On 16.03.24 18:43, Andrey M. Borodin wrote: >>> On 15 Mar 2024, at 14:47, Aleksander Alekseev >>> wrote: >>> >>> +1 to the idea. I doubt that anyone will miss it. >> PFA v22. >> Changes: >> 1. Squashed all editorialisation by Jelte >>

Re: documentation structure

2024-03-20 Thread Bruce Momjian
On Wed, Mar 20, 2024 at 12:43:08PM -0400, Robert Haas wrote: > Overall, I think this achieves a minor but pleasant level of > de-cluttering of the index. It's going to take a lot more than one > morning's work to produce a major improvement, but at least this is > something. I think this kind of

Re: REVOKE FROM warning on grantor

2024-03-20 Thread Étienne BERSAC
Hi, > https://dev.mysql.com/doc/refman/8.0/en/revoke.html documents an "IF > EXISTS" option whose documentation reads, in relevant part, > "Otherwise, REVOKE executes normally; if the user does not exist, the > statement raises an error." > >

Re: Add Index-level REINDEX with multiple jobs

2024-03-20 Thread Alexander Korotkov
On Mon, Mar 11, 2024 at 3:44 PM Maxim Orlov wrote: > On Tue, 6 Feb 2024 at 09:22, Michael Paquier wrote: >> The problem may be actually trickier than that, no? Could there be >> other factors to take into account for their classification, like >> their sizes (typically, we'd want to process the

Re: WIP Incremental JSON Parser

2024-03-20 Thread Jacob Champion
On Tue, Mar 19, 2024 at 3:07 PM Andrew Dunstan wrote: > On Mon, Mar 18, 2024 at 3:35 PM Jacob Champion > wrote: >> With the incremental parser, I think prev_token_terminator is not >> likely to be safe to use except in very specific circumstances, since >> it could be pointing into a stale

Broken error detection in genbki.pl

2024-03-20 Thread Tom Lane
David Wheeler complained over in [1] that genbki.pl fails to produce a useful error message if there's anything wrong in a catalog data file. He's right about that, but the band-aid patch he proposed doesn't improve the situation much. The actual problem is that key error messages in genbki.pl

Re: sslinfo extension - add notbefore and notafter timestamps

2024-03-20 Thread Jacob Champion
On Wed, Mar 20, 2024 at 7:50 AM Daniel Gustafsson wrote: > We are subtracting 30 years from a calculation that we know didnt overflow, so > I guess if the certificate notBefore (the notAfter cannot be that early since > we wouldn't be able to connect with it) was set to early enough? It didn't >

Re: Change GUC hashtable to use simplehash?

2024-03-20 Thread Jeff Davis
On Wed, 2024-03-20 at 14:26 +0700, John Naylor wrote: > This was the culprit. The search path cache didn't trigger this when > it went in, but it seems for frontend a read past the end of malloc > fails -fsantize=address. By the same token, I'm guessing the only > reason this didn't fail for

Re: Avoiding inadvertent debugging mode for pgbench

2024-03-20 Thread Greg Sabino Mullane
My mistake. Attached please find version 3, which should hopefully make cfbot happy again. pgbench.dash.d.or.not.dash.d.v3.patch Description: Binary data

Re: Avoiding inadvertent debugging mode for pgbench

2024-03-20 Thread Nathan Bossart
On Tue, Mar 19, 2024 at 09:15:22PM -0400, Greg Sabino Mullane wrote: > Rebased version attached (v2), with another sentence in the sgml to explain > the optional use of -d cfbot seems quite unhappy with this: https://cirrus-ci.com/build/6429518263484416 -- Nathan Bossart Amazon Web

Re: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-03-20 Thread Masahiko Sawada
On Wed, Mar 20, 2024 at 2:24 PM vignesh C wrote: > > On Tue, 19 Mar 2024 at 17:18, Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Sawada-san, > > > > Thanks for giving comments! > > > > > This behavior makes sense to me. But do we want to handle the case of > > > using environment variables too?

Re: Possibility to disable `ALTER SYSTEM`

2024-03-20 Thread Jelte Fennema-Nio
On Wed, 20 Mar 2024 at 14:04, Greg Sabino Mullane wrote: >> >> As a bonus, if that GUC is set, we could even check at server startup that >> all the configuration files are not writable by the postgres user, >> and print a warning or refuse to start up if they are. > > > Ugh, please let's not do

Re: sslinfo extension - add notbefore and notafter timestamps

2024-03-20 Thread Daniel Gustafsson
> On 20 Mar 2024, at 15:28, Jacob Champion > wrote: >> + result -= ((POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * USECS_PER_DAY); >> + return TimestampTzGetDatum(result); > > Is that final bare subtraction able to wrap around for dates far in the past? We are subtracting 30 years from a

Re: documentation structure

2024-03-20 Thread Robert Haas
On Mon, Mar 18, 2024 at 5:40 PM Laurenz Albe wrote: > I also disagree that chapters 4 to 6 are a continuation of the tutorial. > Or at least, they shouldn't be. > When I am looking for a documentation reference on something like > security considerations of SECURITY DEFINER functions, my first >

Re: add AVX2 support to simd.h

2024-03-20 Thread Nathan Bossart
On Wed, Mar 20, 2024 at 01:57:54PM +0700, John Naylor wrote: > On Tue, Mar 19, 2024 at 11:30 PM Nathan Bossart > wrote: >> I tried to trim some of the branches, and came up with the attached patch. >> I don't think this is exactly what you were suggesting, but I think it's >> relatively close.

Re: minor tweak to catalogs.sgml pg_class.reltablespace

2024-03-20 Thread Alvaro Herrera
On 2024-Mar-19, Tom Lane wrote: > The bit about "(Not meaningful if the relation has no on-disk file.)" > is not correct, and now it's adjacent to text that contradicts it. > Maybe more like > >The tablespace in which this relation is stored. >If zero, the database's default

Re: sslinfo extension - add notbefore and notafter timestamps

2024-03-20 Thread Jacob Champion
On Wed, Mar 20, 2024 at 7:03 AM Daniel Gustafsson wrote: > The issue here is that postgres use a different epoch from the unix epoch, so > any dates calcuated based on the unix epoch need to be adjusted. Ah, thank you! I had just reproduced Cary's problem and was really confused... > I've

Re: Refactor SASL exchange in preparation for OAuth Bearer

2024-03-20 Thread Daniel Gustafsson
> On 29 Feb 2024, at 20:58, Jacob Champion > wrote: > > On Wed, Feb 28, 2024 at 2:54 PM Daniel Gustafsson wrote: >> I rank that as one of my better typos actually. Fixed though. > > LGTM! Thanks for review, and since Heikki marked it ready for committer I assume that counting as a +1 as

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-03-20 Thread Sutou Kouhei
Hi, Could someone review the v17 patch to proceed this? The v17 patch: https://www.postgresql.org/message-id/flat/20240305.171808.667980402249336456.kou%40clear-code.com#d2ee079b75ebcf00c410300ecc4a357a Some profiles by Michael:

Re: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-03-20 Thread Masahiko Sawada
On Tue, Mar 19, 2024 at 8:48 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Sawada-san, > > Thanks for giving comments! > > > This behavior makes sense to me. But do we want to handle the case of > > using environment variables too? > > Yeah, v5 does not consider which libpq parameters are specified

RE: Popcount optimization using AVX512

2024-03-20 Thread Amonson, Paul D
> -Original Message- > From: David Rowley > Sent: Tuesday, March 19, 2024 9:26 PM > To: Amonson, Paul D > > AMD's Zen4 also has AVX512, so it's misleading to indicate it's an Intel only > instruction. Also, writing the date isn't necessary as we have "git blame" Fixed. Thanks, Paul

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-20 Thread John Naylor
On Wed, Mar 20, 2024 at 8:30 PM Masahiko Sawada wrote: > I forgot to report the results. Yes, I did some tests where I inserted > many TIDs to make the tidstore use several GB memory. I did two cases: > > 1. insert 100M blocks of TIDs with an offset of 100. > 2. insert 10M blocks of TIDs with an

Re: pg_upgrade --copy-file-range

2024-03-20 Thread Jakub Wartak
Hi Tomas, > I took a quick look at the remaining part adding copy_file_range to > pg_combinebackup. The patch no longer applies, so I had to rebase it. > Most of the issues were trivial, but I had to fix a couple missing > prototypes - I added them to copy_file.h/c, mostly. > > 0001 is the

Re: Improve readability by using designated initializers when possible

2024-03-20 Thread Peter Eisentraut
On 18.03.24 11:01, jian he wrote: select relname from pg_class where relisshared and relkind = 'r'; relname --- pg_authid pg_subscription pg_database pg_db_role_setting pg_tablespace pg_auth_members pg_shdepend pg_shdescription pg_replication_origin

Re: sslinfo extension - add notbefore and notafter timestamps

2024-03-20 Thread Daniel Gustafsson
> On 20 Mar 2024, at 00:24, Cary Huang wrote: > but it seems to me that many of the timestamp related functions still consider > timestamp or timestampTz as "double values with units of seconds" though. The issue here is that postgres use a different epoch from the unix epoch, so any dates

Re: Test 031_recovery_conflict.pl is not immune to autovacuum

2024-03-20 Thread Alexander Lakhin
Hello Melanie, 20.03.2024 16:15, Melanie Plageman wrote: Seems like we could just add autovacuum_enabled=false to the table like this: diff --git a/src/test/recovery/t/031_recovery_conflict.pl b/src/test/recovery/t/031_recovery_conflict.pl index d87efa823fd..65bc858c02d 100644 ---

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-20 Thread Bertrand Drouvot
Hi, On Wed, Mar 20, 2024 at 12:48:55AM +0530, Bharath Rupireddy wrote: > On Mon, Mar 18, 2024 at 3:02 PM Bertrand Drouvot > wrote: > > > > > > Hm. Are you suggesting inactive_timeout to be a slot level parameter > > > > similar to 'failover' property added recently by > > > >

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-20 Thread Masahiko Sawada
On Wed, Mar 20, 2024 at 3:48 PM John Naylor wrote: > > On Thu, Mar 14, 2024 at 12:06 PM Masahiko Sawada > wrote: > > > > On Thu, Mar 14, 2024 at 1:29 PM John Naylor wrote: > > > Locally (not CI), we should try big inputs to make sure we can > > > actually go up to many GB -- it's easier and

Re: Test 031_recovery_conflict.pl is not immune to autovacuum

2024-03-20 Thread Melanie Plageman
On Wed, Mar 20, 2024 at 9:00 AM Alexander Lakhin wrote: > > Hello hackers, > > Among many recoveryCheck (more concretely, 027_stream_regress) failures > occurred on a number of buildfarm animals after switching to meson, which > can be explained by timeouts, I saw a different failure on adder: >

Re: Commitfest Manager for March

2024-03-20 Thread wenhui qiu
Hi Aleksander Alekseev Could you take a look at the patch ( https://commitfest.postgresql.org/47/4284/),How about your opinion Thanks On Tue, 12 Mar 2024 at 21:41, Aleksander Alekseev wrote: > Hi Andrey, > > > > If you need any help please let me know. > > > > Aleksander, I would

Re: query_id, pg_stat_activity, extended query protocol

2024-03-20 Thread Dave Cramer
> > >> >> FWIW, I'd like to think that we could improve the situation, requiring >> a mix of calling pgstat_report_query_id() while feeding on some query >> IDs retrieved from CachedPlanSource->query_list. I have not in >> details looked at how much could be achieved, TBH. >> > This just cropped

  1   2   >