Re: Provide a pg_truncate_freespacemap function

2024-03-06 Thread Ronan Dunklau
Le mercredi 6 mars 2024, 20:28:44 CET Stephen Frost a écrit : > I agree that this would generally be a useful thing to have. Thanks ! > > > Does that seem correct ? > > Definitely needs to have a 'REVOKE ALL ON FUNCTION' at the end of the > upgrade script, similar to what you'll find at the

Re: Shared detoast Datum proposal

2024-03-06 Thread Nikita Malakhov
Hi! Tomas, I thought about this issue - >What if you only need the first slice? In that case decoding everything >will be a clear regression. And completely agree with you, I'm currently working on it and will post a patch a bit later. Another issue we have here - if we have multiple slices

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-06 Thread Masahiko Sawada
On Thu, Mar 7, 2024 at 4:21 PM John Naylor wrote: > > On Thu, Mar 7, 2024 at 2:14 PM Masahiko Sawada wrote: > > > > On Thu, Mar 7, 2024 at 4:01 PM John Naylor wrote: > > > > > > On Thu, Mar 7, 2024 at 1:27 PM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, Mar 7, 2024 at 3:20 PM John

Re: Reducing the log spam

2024-03-06 Thread Laurenz Albe
On Wed, 2024-03-06 at 17:33 -0500, Isaac Morland wrote: > I have two questions about this: > > First, can it be done per role? If I have a particular application which is > constantly throwing some particular error, I might want to suppress it, but > not suppress the same error occasionally

Re: Combine headerscheck and cpluspluscheck scripts

2024-03-06 Thread Michael Paquier
On Thu, Mar 07, 2024 at 01:37:36PM +1300, Thomas Munro wrote: > +1 Looking at the patch, nice cleanup. -- Michael signature.asc Description: PGP signature

Re: Add new error_action COPY ON_ERROR "log"

2024-03-06 Thread Michael Paquier
On Thu, Mar 07, 2024 at 12:50:33PM +0530, Bharath Rupireddy wrote: > Nice catch. So, are you suggesting to log one NOTICE message per row > even if multiple columns in the single row fail to parse or are > malformed? One NOTICE per malformed value, I guess, which would be easier to parse

Re: Add new error_action COPY ON_ERROR "log"

2024-03-06 Thread Michael Paquier
On Thu, Mar 07, 2024 at 12:48:12PM +0530, Bharath Rupireddy wrote: > I'm okay with it. But, help me understand it better. We want the > 'log_verbosity' clause to have options 'default' and 'verbose', right? > And, later it can also be extended to contain all the LOG levels like > 'notice',

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-06 Thread John Naylor
On Thu, Mar 7, 2024 at 2:14 PM Masahiko Sawada wrote: > > On Thu, Mar 7, 2024 at 4:01 PM John Naylor wrote: > > > > On Thu, Mar 7, 2024 at 1:27 PM Masahiko Sawada > > wrote: > > > > > > On Thu, Mar 7, 2024 at 3:20 PM John Naylor > > > wrote: > > > > > > > > On Thu, Mar 7, 2024 at 12:59 PM

Re: Add new error_action COPY ON_ERROR "log"

2024-03-06 Thread Bharath Rupireddy
On Thu, Mar 7, 2024 at 12:37 PM Michael Paquier wrote: > > On Thu, Mar 07, 2024 at 03:52:41PM +0900, Masahiko Sawada wrote: > > One question I have is; do we want to write multiple NOTICE messages > > for one row if the row has malformed data on some columns? > > Good idea. We can do that as the

Re: Add new error_action COPY ON_ERROR "log"

2024-03-06 Thread Bharath Rupireddy
On Thu, Mar 7, 2024 at 9:30 AM Michael Paquier wrote: > > Is a boolean the best interface for the end-user, though? Maybe > something like a "mode" value would speak more than a yes/no from the > start, say a "default" mode to emit only the last LOG and a "verbose" > for the whole set in the

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-06 Thread Masahiko Sawada
On Thu, Mar 7, 2024 at 4:01 PM John Naylor wrote: > > On Thu, Mar 7, 2024 at 1:27 PM Masahiko Sawada wrote: > > > > On Thu, Mar 7, 2024 at 3:20 PM John Naylor wrote: > > > > > > On Thu, Mar 7, 2024 at 12:59 PM John Naylor > > > wrote: > > > > > ... cause "error: redefinition of typedef

Re: Get rid of the excess semicolon in planner.c

2024-03-06 Thread Richard Guo
On Wed, Mar 6, 2024 at 6:00 AM David Rowley wrote: > On Wed, 6 Mar 2024 at 00:43, Richard Guo wrote: > > > > I think this is a typo introduced in 0452b461b. > > > > +root->processed_groupClause = list_copy(parse->groupClause);; > > "git grep -E ";;$" -- *.c *.h" tell me it's the only one.

Re: Regarding the order of the header file includes

2024-03-06 Thread Richard Guo
On Wed, Mar 6, 2024 at 6:25 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Wed, Mar 6, 2024 at 3:02 PM Richard Guo wrote: > > > > I think we generally follow the rule that we include 'postgres.h' or > > 'postgres_fe.h' first, followed by system header files, and then

Re: Add new error_action COPY ON_ERROR "log"

2024-03-06 Thread Michael Paquier
On Thu, Mar 07, 2024 at 03:52:41PM +0900, Masahiko Sawada wrote: > One question I have is; do we want to write multiple NOTICE messages > for one row if the row has malformed data on some columns? Good idea. We can do that as the field strings are already parsed. -- Michael signature.asc

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-06 Thread John Naylor
On Thu, Mar 7, 2024 at 1:27 PM Masahiko Sawada wrote: > > On Thu, Mar 7, 2024 at 3:20 PM John Naylor wrote: > > > > On Thu, Mar 7, 2024 at 12:59 PM John Naylor wrote: > > > ... cause "error: redefinition of typedef 'rt_radix_tree' is a C11 > > > feature [-Werror,-Wtypedef-redefinition]" > > >

Re: Add new error_action COPY ON_ERROR "log"

2024-03-06 Thread Masahiko Sawada
On Thu, Mar 7, 2024 at 1:00 PM Michael Paquier wrote: > > On Wed, Mar 06, 2024 at 07:32:28PM +0530, Bharath Rupireddy wrote: > > Please see the attached v4 patch. If it looks good, I can pull > > LOG_VERBOSITY changes out into 0001 and with 0002 containing the > > detailed messages for discarded

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-06 Thread Masahiko Sawada
On Thu, Mar 7, 2024 at 3:27 PM Masahiko Sawada wrote: > > On Thu, Mar 7, 2024 at 3:20 PM John Naylor wrote: > > > > > > In addition, olingo and grassquit are showing different kinds of > > "AddressSanitizer: odr-violation" errors, which I'm not sure what to > > make of -- example: odr-violation

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-03-06 Thread Michael Paquier
On Wed, Mar 06, 2024 at 03:34:04PM +0900, Michael Paquier wrote: > I am not sure that my schedule is on track to allow that for this > release, unfortunately, especially with all the other items to review > and discuss to make this thread feature-complete. There should be > a bit more than four

RE: Synchronizing slots from primary to standby

2024-03-06 Thread Zhijie Hou (Fujitsu)
On Thursday, March 7, 2024 12:46 PM Amit Kapila wrote: > > On Thu, Mar 7, 2024 at 7:35 AM Peter Smith > wrote: > > > > Here are some review comments for v107-0001 > > > > == > > src/backend/replication/slot.c > > > > 1. > > +/* > > + * Struct for the configuration of standby_slot_names. > >

RE: Synchronizing slots from primary to standby

2024-03-06 Thread Zhijie Hou (Fujitsu)
On Thursday, March 7, 2024 10:05 AM Peter Smith wrote: > > Here are some review comments for v107-0001 Thanks for the comments. > > == > src/backend/replication/slot.c > > 1. > +/* > + * Struct for the configuration of standby_slot_names. > + * > + * Note: this must be a flat

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-06 Thread Masahiko Sawada
On Thu, Mar 7, 2024 at 3:20 PM John Naylor wrote: > > On Thu, Mar 7, 2024 at 12:59 PM John Naylor wrote: > > > > On Thu, Mar 7, 2024 at 12:55 PM John Naylor wrote: > > > > > > On Wed, Mar 6, 2024 at 6:59 PM Masahiko Sawada > > > wrote: > > > > > > > > > + /* Find the control object in shared

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-06 Thread John Naylor
On Thu, Mar 7, 2024 at 12:59 PM John Naylor wrote: > > On Thu, Mar 7, 2024 at 12:55 PM John Naylor wrote: > > > > On Wed, Mar 6, 2024 at 6:59 PM Masahiko Sawada > > wrote: > > > > > > > + /* Find the control object in shared memory */ > > > > + control = handle; > > > > > > I think it's mostly

Invalid query generated by postgres_fdw with UNION ALL and ORDER BY

2024-03-06 Thread Michał Kłeczek
The following query: SELECT * FROM ( SELECT 2023 AS year, * FROM remote_table_1 UNION ALL SELECT 2022 AS year, * FROM remote_table_2 ) ORDER BY year DESC; yields the following remote query: SELECT [columns] FROM remote_table_1 ORDER BY 2023 DESC and subsequently fails remote execution.

Re: Missing LWLock protection in pgstat_reset_replslot()

2024-03-06 Thread shveta malik
On Thu, Mar 7, 2024 at 11:12 AM Michael Paquier wrote: > > Yeah, it is possible that what you retrieve from > pgstat_fetch_replslot() does not refer exactly to the slot's content > under concurrent activity, but you cannot protect a single scan of > pg_stat_replication_slots as of an effect of

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-06 Thread John Naylor
On Thu, Mar 7, 2024 at 12:55 PM John Naylor wrote: > > On Wed, Mar 6, 2024 at 6:59 PM Masahiko Sawada wrote: > > > > > + /* Find the control object in shared memory */ > > > + control = handle; > > > > I think it's mostly because of readability; it makes clear that the > > handle should be

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-06 Thread John Naylor
On Wed, Mar 6, 2024 at 6:59 PM Masahiko Sawada wrote: > > > + /* Find the control object in shared memory */ > > + control = handle; > > I think it's mostly because of readability; it makes clear that the > handle should be castable to dsa_pointer and it's a control object. I > borrowed it from

Re: Synchronizing slots from primary to standby

2024-03-06 Thread Amit Kapila
On Thu, Mar 7, 2024 at 8:37 AM shveta malik wrote: > I thought about whether we can make standby_slot_names as USERSET instead of SIGHUP and it doesn't sound like a good idea as that can lead to inconsistent standby replicas even after configuring the correct value of standby_slot_names. One can

Re: Missing LWLock protection in pgstat_reset_replslot()

2024-03-06 Thread Michael Paquier
On Thu, Mar 07, 2024 at 10:57:28AM +0530, shveta malik wrote: > --It can happen in a small window in pg_stat_get_replication_slot() > when we are consuming the return values of pgstat_fetch_replslot > (using slotent). Yeah, it is possible that what you retrieve from pgstat_fetch_replslot() does

Re: Missing LWLock protection in pgstat_reset_replslot()

2024-03-06 Thread shveta malik
On Wed, Mar 6, 2024 at 2:36 PM Bertrand Drouvot wrote: > > Hi, > > On Wed, Mar 06, 2024 at 10:24:46AM +0530, shveta malik wrote: > > On Tue, Mar 5, 2024 at 6:52 PM Bertrand Drouvot > > wrote: > > Thanks. Can we try to get rid of multiple LwLockRelease in > > pgstat_reset_replslot(). Is this any

Re: Add new error_action COPY ON_ERROR "log"

2024-03-06 Thread torikoshia
On 2024-03-07 13:00, Michael Paquier wrote: On Wed, Mar 06, 2024 at 07:32:28PM +0530, Bharath Rupireddy wrote: Please see the attached v4 patch. If it looks good, I can pull LOG_VERBOSITY changes out into 0001 and with 0002 containing the detailed messages for discarded rows. The approach

Re: DOCS: Avoid using abbreviation "aka"

2024-03-06 Thread Michael Paquier
On Thu, Mar 07, 2024 at 10:02:36AM +0530, Amit Kapila wrote: > Pushed. Thanks. -- Michael signature.asc Description: PGP signature

Re: remaining sql/json patches

2024-03-06 Thread Himanshu Upadhyaya
On Wed, Mar 6, 2024 at 9:04 PM Tomas Vondra wrote: > > > I'm pretty sure this is the correct & expected behavior. The second > query treats the value as string (because that's what should happen for > values in double quotes). > > ok, Then why does the below query provide the correct

Re: Missing LWLock protection in pgstat_reset_replslot()

2024-03-06 Thread Michael Paquier
On Wed, Mar 06, 2024 at 09:05:59AM +, Bertrand Drouvot wrote: > Yeah, all of the above done in v3 attached. Interesting, so this relies on the slot index to ensure the unicity of the stat entries. And if the entry pointing to this ID is updated we may refer to just incorrect data. The

Re: remaining sql/json patches

2024-03-06 Thread jian he
two cosmetic minor issues. +/* + * JsonCoercion + * Information about coercing a SQL/JSON value to the specified + * type at runtime + * + * A node of this type is created if the parser cannot find a cast expression + * using coerce_type() or OMIT QUOTES is specified for JSON_QUERY. If the + *

Re: Synchronizing slots from primary to standby

2024-03-06 Thread Amit Kapila
On Thu, Mar 7, 2024 at 7:35 AM Peter Smith wrote: > > Here are some review comments for v107-0001 > > == > src/backend/replication/slot.c > > 1. > +/* > + * Struct for the configuration of standby_slot_names. > + * > + * Note: this must be a flat representation that can be held in a single >

Re: speed up a logical replica setup

2024-03-06 Thread Euler Taveira
On Wed, Mar 6, 2024, at 8:24 AM, vignesh C wrote: > Few comments: Thanks for your review. Some changes are included in v26. > 1) Can we use strdup here instead of atoi, as we do similarly in > case of pg_dump too, else we will do double conversion, convert using > atoi and again to string

Re: speed up a logical replica setup

2024-03-06 Thread Euler Taveira
On Wed, Mar 6, 2024, at 7:02 AM, Hayato Kuroda (Fujitsu) wrote: > Thanks for updating the patch! Thanks for the feedback. I'm attaching v26 that addresses most of your comments and some issues pointed by Vignesh [1]. > >* I also removed the check for standby is running. If the standby was >

Re: DOCS: Avoid using abbreviation "aka"

2024-03-06 Thread Amit Kapila
On Fri, Mar 1, 2024 at 11:29 AM Michael Paquier wrote: > > On Fri, Mar 01, 2024 at 11:08:21AM +0530, Amit Kapila wrote: > > I wanted to wait for two or three days to see if any other fixes in > > docs, typos, or cosmetic stuff are reported in this functionality then > > I can combine and push

Re: Add system identifier to backup manifest

2024-03-06 Thread Amul Sul
On Thu, Mar 7, 2024 at 9:37 AM Michael Paquier wrote: > On Wed, Mar 06, 2024 at 11:05:36AM -0500, Robert Haas wrote: > > So with that in mind, here's my proposal. This is an adjustment of > > Amit's previous refactoring patch. He renamed the existing > > get_controlfile() to

Re: remaining sql/json patches

2024-03-06 Thread jian he
On Tue, Mar 5, 2024 at 12:38 PM Andy Fan wrote: > > > In the commit message of 0001, we have: > > """ > Both JSON_VALUE() and JSON_QUERY() functions have options for > handling EMPTY and ERROR conditions, which can be used to specify > the behavior when no values are matched and when an error

Re: Add system identifier to backup manifest

2024-03-06 Thread Michael Paquier
On Wed, Mar 06, 2024 at 11:05:36AM -0500, Robert Haas wrote: > So with that in mind, here's my proposal. This is an adjustment of > Amit's previous refactoring patch. He renamed the existing > get_controlfile() to get_dir_controlfile() and made a new > get_controlfile() that accepted the path to

Re: Add new error_action COPY ON_ERROR "log"

2024-03-06 Thread Michael Paquier
On Wed, Mar 06, 2024 at 07:32:28PM +0530, Bharath Rupireddy wrote: > Please see the attached v4 patch. If it looks good, I can pull > LOG_VERBOSITY changes out into 0001 and with 0002 containing the > detailed messages for discarded rows. The approach looks sensible seen from here. +

Re: Synchronizing slots from primary to standby

2024-03-06 Thread Masahiko Sawada
On Wed, Mar 6, 2024 at 5:53 PM Amit Kapila wrote: > > On Wed, Mar 6, 2024 at 12:07 PM Masahiko Sawada wrote: > > > > On Fri, Mar 1, 2024 at 3:22 PM Peter Smith wrote: > > > > > > On Fri, Mar 1, 2024 at 5:11 PM Masahiko Sawada > > > wrote: > > > > > > > ... > > > > +/* > > > > +

Re: Improve eviction algorithm in ReorderBuffer

2024-03-06 Thread Masahiko Sawada
On Tue, Mar 5, 2024 at 3:28 PM Peter Smith wrote: > > Hi, here are some review comments for v7-0002 > > == > Commit Message > > 1. > This commit adds a hash table to binaryheap in order to track of > positions of each nodes in the binaryheap. That way, by using newly > added functions such as

Re: Improve eviction algorithm in ReorderBuffer

2024-03-06 Thread Masahiko Sawada
On Tue, Mar 5, 2024 at 12:20 PM vignesh C wrote: > > On Wed, 28 Feb 2024 at 11:40, Amit Kapila wrote: > > > > On Mon, Feb 26, 2024 at 7:54 PM Masahiko Sawada > > wrote: > > > > > > > A few comments on 0003: > > === > > 1. > > +/* > > + * Threshold of the total number of

Re: Synchronizing slots from primary to standby

2024-03-06 Thread shveta malik
On Wed, Mar 6, 2024 at 6:54 PM Zhijie Hou (Fujitsu) wrote: > > On Wednesday, March 6, 2024 9:13 PM Zhijie Hou (Fujitsu) > wrote: > > > > On Wednesday, March 6, 2024 11:04 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > On Wednesday, March 6, 2024 9:30 AM Masahiko Sawada > > > wrote: > > > > >

Re: Confine vacuum skip logic to lazy_scan_skip

2024-03-06 Thread Melanie Plageman
On Wed, Mar 06, 2024 at 09:55:21PM +0200, Heikki Linnakangas wrote: > On 27/02/2024 21:47, Melanie Plageman wrote: > > The attached v5 has some simplifications when compared to v4 but takes > > largely the same approach. > > > > 0001-0004 are refactoring > > I'm looking at just these 0001-0004

Re: remaining sql/json patches

2024-03-06 Thread jian he
On Wed, Mar 6, 2024 at 9:22 PM jian he wrote: > > Another case, I did test yet: more keys in a single json, but the > value is small. Another case attached. see the attached SQL file's comments. a single simple jsonb, with 33 keys, each key's value with fixed length: 256. total table size:

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-03-06 Thread Tender Wang
Andrei Lepikhov 于2024年3月6日周三 11:37写道: > I think, it is a bug. Should it be fixed (and back-patched) earlier? > Agreed. Need David to review it as he knows this area best. -- Tender Wang OpenPie: https://en.openpie.com/

Re: Synchronizing slots from primary to standby

2024-03-06 Thread Peter Smith
Here are some review comments for v107-0001 == src/backend/replication/slot.c 1. +/* + * Struct for the configuration of standby_slot_names. + * + * Note: this must be a flat representation that can be held in a single chunk + * of guc_malloc'd memory, so that it can be stored as the "extra"

Re: Fix race condition in InvalidatePossiblyObsoleteSlot()

2024-03-06 Thread Michael Paquier
On Wed, Mar 06, 2024 at 05:45:56PM +, Bertrand Drouvot wrote: > Thank you both for the report! I did a few test manually and can see the issue > from times to times. When the issue occurs, the logical decoding was able to > go through the place where LogicalConfirmReceivedLocation() updates

Re: Combine headerscheck and cpluspluscheck scripts

2024-03-06 Thread Thomas Munro
+1

Re: Improve eviction algorithm in ReorderBuffer

2024-03-06 Thread Masahiko Sawada
On Tue, Mar 5, 2024 at 11:25 AM Peter Smith wrote: > > Hi, Here are some review comments for v7-0001 Thank you for reviewing the patch. > > 1. > /* > * binaryheap_free > * > * Releases memory used by the given binaryheap. > */ > void > binaryheap_free(binaryheap *heap) > { > pfree(heap); >

improve ssl error code, 2147483650

2024-03-06 Thread David Zhang
Hi Hackers, When configuring SSL on the Postgres server side with the following information: ssl = on ssl_ca_file = 'root_ca.crt' ssl_cert_file = 'server-cn-only.crt' ssl_key_file = 'server-cn-only.key' If a user makes a mistake, for example, accidentally using 'root_ca.crl' instead of

Re: Stack overflow issue

2024-03-06 Thread Tom Lane
Alexander Korotkov writes: > Sorry for tediousness, but isn't pre-order a variation of depth-first order > [1]? To me, depth-first implies visiting children before parents. Do I have the terminology wrong? regards, tom lane

Re: Confine vacuum skip logic to lazy_scan_skip

2024-03-06 Thread Melanie Plageman
On Tue, Feb 27, 2024 at 02:47:03PM -0500, Melanie Plageman wrote: > On Mon, Jan 29, 2024 at 8:18 PM Melanie Plageman > wrote: > > > > On Fri, Jan 26, 2024 at 8:28 AM vignesh C wrote: > > > > > > CFBot shows that the patch does not apply anymore as in [1]: > > > === applying patch > > >

Re: Stack overflow issue

2024-03-06 Thread Alexander Korotkov
On Thu, Mar 7, 2024 at 12:52 AM Tom Lane wrote: > Alexander Korotkov writes: > > The revised set of remaining patches is attached. > > ... > > 0003 Avoid recursion in MemoryContext functions > > I've renamed MemoryContextTraverse() => MemoryContextTraverseNext(), > > which I think is a bit more

Re: [PATCH] Exponential backoff for auth_delay

2024-03-06 Thread Jacob Champion
On Wed, Mar 6, 2024 at 2:45 PM Michael Banck wrote: > In order to at least make case 2 not worse for exponential backoff, we > could maybe disable it (and just wait for auth_delay.milliseconds) once > MAX_CONN_RECORDS is full. In addition, maybe MAX_CONN_RECORDS should be > some fraction of

Re: Injection points: some tools to wait and wake

2024-03-06 Thread Michael Paquier
On Wed, Mar 06, 2024 at 10:19:41AM +0100, Jelte Fennema-Nio wrote: > What I mainly meant is that anything in src/test/modules is not even > close to somewhat useful for other people to use. They are really just > very specific tests that need to be written in C. Afaict all those > modules are not

Re: Stack overflow issue

2024-03-06 Thread Tom Lane
Alexander Korotkov writes: > The revised set of remaining patches is attached. > ... > 0003 Avoid recursion in MemoryContext functions > I've renamed MemoryContextTraverse() => MemoryContextTraverseNext(), > which I think is a bit more intuitive. Also I fixed > MemoryContextMemConsumed(), which

Re: [PATCH] Exponential backoff for auth_delay

2024-03-06 Thread Jacob Champion
On Wed, Mar 6, 2024 at 12:34 PM Tomas Vondra wrote: > Doesn't this mean this approach (as implemented) doesn't actually work > as a protection against this sort of DoS? > > If I'm an attacker, and I can just keep opening new connections for each > auth request, am I even subject to any auth

Re: Hooking into ExplainOneQuery() complicated by missing standard_ExplainOneQuery

2024-03-06 Thread Michael Paquier
On Wed, Mar 06, 2024 at 10:32:01AM +0100, Jelte Fennema-Nio wrote: > This patch would definitely be useful for Citus. We indeed currently > copy all of that code into our own explain hook. And it seems we > actually have some bug. Because the es->memory branches were not > copied (probably because

Re: [PATCH] Exponential backoff for auth_delay

2024-03-06 Thread Michael Banck
Hi, On Wed, Mar 06, 2024 at 09:34:37PM +0100, Tomas Vondra wrote: > On 3/6/24 19:24, Jacob Champion wrote: > > On Wed, Mar 6, 2024 at 8:10 AM Nathan Bossart > > wrote: > >> Assuming you are referring to the case where we run out of free slots in > >> acr_array, I'm not sure I see that as

Re: Reducing the log spam

2024-03-06 Thread Isaac Morland
On Tue, 5 Mar 2024 at 07:55, Laurenz Albe wrote: > Inspired by feedback to [1], I thought about how to reduce log spam. > > My experience from the field is that a lot of log spam looks like > > database/table/... "xy" does not exist > duplicate key value violates unique constraint "xy" > >

Re: vacuumdb/clusterdb/reindexdb: allow specifying objects to process in all databases

2024-03-06 Thread Nathan Bossart
On Tue, Mar 05, 2024 at 11:20:13PM +, Dean Rasheed wrote: > I'm not sure how useful these changes are, but I don't really object. > You need to update the synopsis section of the docs though. Thanks for taking a look. I updated the synopsis sections in v3. I also spent some more time on the

Re: Large files for relations

2024-03-06 Thread Thomas Munro
Rebased. I had intended to try to get this into v17, but a couple of unresolved problems came up while rebasing over the new incremental backup stuff. You snooze, you lose. Hopefully we can sort these out in time for the next commitfest: * should pg_combinebasebackup read the control file to

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-03-06 Thread Matthias van de Meent
On Wed, 6 Mar 2024 at 22:46, Matthias van de Meent wrote: > > On Wed, 6 Mar 2024 at 01:50, Peter Geoghegan wrote: > > At one point Heikki suggested that I just get rid of > > BTScanOpaqueData.arrayKeyData (which has been there for as long as > > nbtree had native support for SAOPs), and use > >

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-03-06 Thread Matthias van de Meent
On Wed, 6 Mar 2024 at 01:50, Peter Geoghegan wrote: > > On Mon, Mar 4, 2024 at 3:51 PM Matthias van de Meent > wrote: > > > +that searches for multiple values together. Queries that use certain > > > +SQL constructs to search for rows matching any > > > value > > > +out of a list

Re: Make attstattarget nullable

2024-03-06 Thread Tomas Vondra
Hi Peter, I took a look at this patch today. I think it makes sense to do this, and I haven't found any major issues with any of the three patches. A couple minor comments: 0001 1) I think this bit in ALTER STATISTICS docs is wrong: - new_target + SET STATISTICS { integer |

Re: Shared detoast Datum proposal

2024-03-06 Thread Tomas Vondra
On 3/5/24 10:03, Nikita Malakhov wrote: > Hi, > > Tomas, sorry for confusion, in my previous message I meant exactly > the same approach you've posted above, and came up with almost > the same implementation. > > Thank you very much for your attention to this thread! > > I've asked Andy

Re: Shared detoast Datum proposal

2024-03-06 Thread Tomas Vondra
On 3/6/24 07:09, Nikita Malakhov wrote: > Hi, > > In addition to the previous message - for the toast_fetch_datum_slice > the first (seems obvious) way is to detoast the whole value, save it > to cache and get slices from it on demand. I have another one on my > mind, but have to play with it

Re: [PATCH] Exponential backoff for auth_delay

2024-03-06 Thread Tomas Vondra
On 3/6/24 19:24, Jacob Champion wrote: > On Wed, Mar 6, 2024 at 8:10 AM Nathan Bossart > wrote: >> Assuming you are referring to the case where we run out of free slots in >> acr_array, I'm not sure I see that as desirable behavior. Once you run out >> of slots, all failed authentication

Re: Reducing the log spam

2024-03-06 Thread Laurenz Albe
On Wed, 2024-03-06 at 10:50 -0500, Greg Sabino Mullane wrote: > On Tue, Mar 5, 2024 at 7:55 AM Laurenz Albe wrote: > > My experience from the field is that a lot of log spam looks like > > > >   database/table/... "xy" does not exist > >   duplicate key value violates unique constraint "xy" > >

Re: MERGE ... RETURNING

2024-03-06 Thread Merlin Moncure
On Thu, Feb 29, 2024 at 1:49 PM Jeff Davis wrote: > > Can we get some input on whether the current MERGE ... RETURNING patch > is the right approach from a language standpoint? > MERGE_CLAUSE_NUMBER() seems really out of place to me, it feels out of place to identify output set by number rather

Re: Transaction timeout

2024-03-06 Thread Alexander Korotkov
On Wed, Mar 6, 2024 at 10:22 AM Andrey M. Borodin wrote: > > On 25 Feb 2024, at 21:50, Alexander Korotkov wrote: > > > > Thank you for the patches. I've pushed the 0001 patch to avoid > > further failures on buildfarm. Let 0004 wait till injections points > > by Mechael are committed. > >

Re: Confine vacuum skip logic to lazy_scan_skip

2024-03-06 Thread Heikki Linnakangas
On 27/02/2024 21:47, Melanie Plageman wrote: The attached v5 has some simplifications when compared to v4 but takes largely the same approach. 0001-0004 are refactoring I'm looking at just these 0001-0004 patches for now. I like those changes a lot for the sake of readablity even without any

Re: Provide a pg_truncate_freespacemap function

2024-03-06 Thread Stephen Frost
Greetings, * Ronan Dunklau (ronan.dunk...@aiven.io) wrote: > As we are currently experiencing a FSM corruption issue [1], we need to > rebuild FSM when we detect it. Ideally, we'd figure out a way to pick up on this and address it without the user needing to intervene, however ... > I noticed

Re: Potential issue in ecpg-informix decimal converting functions

2024-03-06 Thread a . imamov
Daniel Gustafsson писал(а) 2024-03-06 18:03: On 27 Feb 2024, at 06:08, Michael Paquier wrote: On Mon, Feb 26, 2024 at 12:28:51AM +0100, Daniel Gustafsson wrote: Yeah, I think this is for HEAD only, especially given the lack of complaints against backbranches. Daniel, are you planning to

Using the %m printf format more

2024-03-06 Thread Dagfinn Ilmari Mannsåker
Hi hackers, I just noticed that commit d93627bc added a bunch of pg_fatal() calls using %s and strerror(errno), which could be written more concisely as %m. I'm assuming this was done because the surrounding code also uses this pattern, and hadn't been changed to use %m when support for that was

Re: Proposal to add page headers to SLRU pages

2024-03-06 Thread Stephen Frost
Greetings, * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: > I suppose this is important to do if we ever want to move SLRUs into > shared buffers. However, I wonder about the extra time this adds to > pg_upgrade. Is this something we should be concerned about? Is there > any

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-03-06 Thread Jelte Fennema-Nio
On Wed, 6 Mar 2024 at 19:22, Alvaro Herrera wrote: > Docs: one bogus "that that". will fix > Did we consider having PQcancelConn() instead be called > PQcancelCreate()? Fine by me > Also, the comment still says > "Asynchronously cancel a query on the given connection. This requires > polling

Re: Statistics Import and Export

2024-03-06 Thread Stephen Frost
Greetings, * Matthias van de Meent (boekewurm+postg...@gmail.com) wrote: > On Wed, 6 Mar 2024 at 11:33, Stephen Frost wrote: > > On Wed, Mar 6, 2024 at 11:07 Matthias van de Meent > > wrote: > >> Or even just one VALUES for the whole statistics loading? > > I don’t think we’d want to go beyond

Re: [PATCH] Exponential backoff for auth_delay

2024-03-06 Thread Jacob Champion
On Wed, Mar 6, 2024 at 8:10 AM Nathan Bossart wrote: > Assuming you are referring to the case where we run out of free slots in > acr_array, I'm not sure I see that as desirable behavior. Once you run out > of slots, all failed authentication attempts are now subject to the maximum > delay,

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-03-06 Thread Alvaro Herrera
Docs: one bogus "that that". Did we consider having PQcancelConn() instead be called PQcancelCreate()? I think this better conveys that what we're doing is create an object that can be used to do something, and that nothing else is done with it by default. Also, the comment still says

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-03-06 Thread Jelte Fennema-Nio
On Wed, 6 Mar 2024 at 15:03, Denis Laxalde wrote: > > In patch 0004, I noticed a couple of typos in the documentation; please > find attached a fixup patch correcting these. Thanks, applied. > while not actually listing the "statuses". Should we list them? I listed the relevant statuses over

Re: Remove unnecessary code from psql's watch command

2024-03-06 Thread Tom Lane
Michael Paquier writes: > On Tue, Mar 05, 2024 at 10:05:52PM +0900, Yugo NAGATA wrote: >> In the current code of do_watch(), sigsetjmp is called if WIN32 >> is defined, but siglongjmp is not called in the signal handler >> in this condition. On Windows, currently, cancellation is checked >> only

Re: Fix race condition in InvalidatePossiblyObsoleteSlot()

2024-03-06 Thread Bertrand Drouvot
Hi, On Wed, Mar 06, 2024 at 05:45:56PM +0530, Bharath Rupireddy wrote: > On Wed, Mar 6, 2024 at 4:51 PM Michael Paquier wrote: > > > > On Wed, Mar 06, 2024 at 09:17:58AM +, Bertrand Drouvot wrote: > > > Right, somehow out of context here. > > > > We're not yet in the green yet, one of my

Re: logical decoding and replication of sequences, take 2

2024-03-06 Thread Tomas Vondra
Hi, Let me share a bit of an update regarding this patch and PG17. I have discussed this patch and how to move it forward with a couple hackers (both within EDB and outside), and my takeaway is that the patch is not quite baked yet, not enough to make it into PG17 :-( There are two main reasons

Re: recovery modules

2024-03-06 Thread Nathan Bossart
Here is another rebase. Given the size of this patch set and the lack of review, I am going to punt this one to v18. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From dd69e5987b477818f60b202af5fb9b2603dc8acb Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 15 Feb 2023

Re: MERGE ... RETURNING

2024-03-06 Thread Dean Rasheed
On Wed, 6 Mar 2024 at 08:51, Peter Eisentraut wrote: > > For comparison with standard SQL (see ): > > For an INSERT you could write > > SELECT whatever FROM NEW TABLE (INSERT statement here) > > or for an DELETE > > SELECT whatever FROM OLD TABLE (DELETE statement here) > > And for an UPDATE

Re: Switching XLog source from archive to streaming when primary available

2024-03-06 Thread Nathan Bossart
On Wed, Mar 06, 2024 at 10:02:43AM +0530, Bharath Rupireddy wrote: > On Wed, Mar 6, 2024 at 1:22 AM Nathan Bossart > wrote: >> I was thinking of something more like >> >> typedef enum >> { >> NO_FORCE_SWITCH_TO_STREAMING, /* no switch >> necessary */ >>

Re: [PATCH] Exponential backoff for auth_delay

2024-03-06 Thread Nathan Bossart
On Tue, Mar 05, 2024 at 05:14:46PM -0800, Jacob Champion wrote: > On Tue, Mar 5, 2024 at 1:51 PM Nathan Bossart > wrote: >> I don't have a strong opinion about making this configurable, but I do >> think we should consider making this a hash table so that we can set >> MAX_CONN_RECORDS much

Re: Add system identifier to backup manifest

2024-03-06 Thread Robert Haas
On Mon, Mar 4, 2024 at 2:47 PM Robert Haas wrote: > I don't have an enormously strong opinion on what the right thing to > do is here either, but I am not convinced that the change proposed by > Michael is an improvement. After all, that leaves us with the > situation where we know the path to

Re: Reducing the log spam

2024-03-06 Thread Laurenz Albe
On Wed, 2024-03-06 at 17:09 +0300, Aleksander Alekseev wrote: > I like the idea of suppressing certain log messages in general, but > the particular user interface doesn't strike me as an especially > convenient one. > > Firstly I don't think many people remember sqlstates and what 3F000 > stands

Logging which interface was connected to in log_line_prefix

2024-03-06 Thread Greg Sabino Mullane
Someone on -general was asking about this, as they are listening on multiple IPs and would like to know which exact one clients were hitting. I took a quick look and we already have that information, so I grabbed some stuff from inet_server_addr and added it as part of a "%L" (for 'local

Re: Reducing the log spam

2024-03-06 Thread Greg Sabino Mullane
On Tue, Mar 5, 2024 at 7:55 AM Laurenz Albe wrote: > My experience from the field is that a lot of log spam looks like > > database/table/... "xy" does not exist > duplicate key value violates unique constraint "xy" Forcibly hiding those at the Postgres level seems a heavy hammer for what

Re: remaining sql/json patches

2024-03-06 Thread Tomas Vondra
On 3/6/24 12:58, Himanshu Upadhyaya wrote: > On Tue, Mar 5, 2024 at 6:52 AM Amit Langote wrote: > > Hi, > > I am doing some random testing with the latest patch and found one scenario > that I wanted to share. > consider a below case. > > ‘postgres[102531]=#’SELECT * FROM JSON_TABLE(jsonb

Re: Change prefetch and read strategies to use range in pg_prewarm ... and raise a question about posix_fadvise WILLNEED

2024-03-06 Thread Cédric Villemain
Hi Nazir, thank you for your review. I comment below. On 05/03/2024 12:07, Nazir Bilal Yavuz wrote: 2. The second one does implement smgrprefetch with range and loops by default per segment to still have a check for interrupts. It looks good codewise but RELSEG_SIZE is too big to prefetch.

Re: Potential issue in ecpg-informix decimal converting functions

2024-03-06 Thread Daniel Gustafsson
> On 27 Feb 2024, at 06:08, Michael Paquier wrote: > > On Mon, Feb 26, 2024 at 12:28:51AM +0100, Daniel Gustafsson wrote: >> Yeah, I think this is for HEAD only, especially given the lack of complaints >> against backbranches. > > Daniel, are you planning to look at that? I haven't done any

Re: [PoC] Reducing planning time when tables have many partitions

2024-03-06 Thread Ashutosh Bapat
Hi Yuya On Wed, Feb 28, 2024 at 4:48 PM Yuya Watari wrote: > Hello, > > On Tue, Feb 13, 2024 at 6:19 AM Alena Rybakina > wrote: > > > > Yes, it is working correctly now with the assertion check. I suppose > > it's better to add this code with an additional comment and a > > recommendation for

  1   2   >