Re: POC, WIP: OR-clause support for indexes

2024-03-04 Thread Andrei Lepikhov
On 5/3/2024 12:30, Andrei Lepikhov wrote: On 4/3/2024 09:26, jian he wrote: ... and the new version of the patchset is attached. -- regards, Andrei Lepikhov Postgres Professional From 1c3ac3e006cd66ff40f1ddaaa09e3fc0f3a75ca5 Mon Sep 17 00:00:00 2001 From: Alena Rybakina Date: Fri, 2 Feb 2024

Re: Missing LWLock protection in pgstat_reset_replslot()

2024-03-04 Thread Heikki Linnakangas
On 01/03/2024 12:15, Bertrand Drouvot wrote: Hi hackers, I think that pgstat_reset_replslot() is missing LWLock protection. Indeed, we don't have any guarantee that the slot is active (then preventing it to be dropped/recreated) when this function is executed. Yes, so it seems at quick

Re: Some shared memory chunks are allocated even if related processes won't start

2024-03-04 Thread 'Alvaro Herrera'
On 2024-Mar-05, Hayato Kuroda (Fujitsu) wrote: > Basically sounds good. My concerns are: > > * GetNamedDSMSegment() does not returns a raw pointer to dsm_segment. This > means > that it may be difficult to do dsm_unpin_segment on the caller side. Maybe we don't need a "named" DSM segment at

Re: Hooking into ExplainOneQuery() complicated by missing standard_ExplainOneQuery

2024-03-04 Thread Mats Kindahl
On Tue, Mar 5, 2024 at 7:31 AM Michael Paquier wrote: > On Mon, Mar 04, 2024 at 03:41:16PM +0300, Aleksander Alekseev wrote: > >> I wanted to hook into the EXPLAIN output for queries and add some > >> extra information, but since there is no standard_ExplainOneQuery() I > >> had to copy the code

RE: Synchronizing slots from primary to standby

2024-03-04 Thread Zhijie Hou (Fujitsu)
On Tuesday, March 5, 2024 2:35 PM shveta malik wrote: > > On Tue, Mar 5, 2024 at 9:15 AM Amit Kapila wrote: > > > > On Tue, Mar 5, 2024 at 6:10 AM Peter Smith > wrote: > > > > > > == > > > src/backend/replication/walsender.c > > > > > > 5. NeedToWaitForWal > > > > > > + /* > > > + * Check

RE: Synchronizing slots from primary to standby

2024-03-04 Thread Zhijie Hou (Fujitsu)
On Tuesday, March 5, 2024 8:40 AM Peter Smith wrote: > > Here are some review comments for v105-0001 > > == > doc/src/sgml/config.sgml > > 1. > + > +The standbys corresponding to the physical replication slots in > +standby_slot_names must configure > +

RE: Synchronizing slots from primary to standby

2024-03-04 Thread Zhijie Hou (Fujitsu)
On Monday, March 4, 2024 11:44 PM Bertrand Drouvot wrote: > > Hi, > > On Mon, Mar 04, 2024 at 01:28:04PM +, Zhijie Hou (Fujitsu) wrote: > > Attach the V105 patch set > > Thanks! > > Sorry I missed those during the previous review: No problem, thanks for the comments! > > 1 === > >

Re: Synchronizing slots from primary to standby

2024-03-04 Thread Nisha Moond
I did performance tests for the v99 patch w.r.t. wait time analysis. As this patch is introducing a wait for standby before sending changes to a subscriber, at the primary node, logged time at the start and end of the XLogSendLogical() call (which eventually calls WalSndWaitForWal()) and

Re: Synchronizing slots from primary to standby

2024-03-04 Thread shveta malik
On Tue, Mar 5, 2024 at 9:15 AM Amit Kapila wrote: > > On Tue, Mar 5, 2024 at 6:10 AM Peter Smith wrote: > > > > == > > src/backend/replication/walsender.c > > > > 5. NeedToWaitForWal > > > > + /* > > + * Check if the standby slots have caught up to the flushed position. It > > + * is good to

Re: Hooking into ExplainOneQuery() complicated by missing standard_ExplainOneQuery

2024-03-04 Thread Michael Paquier
On Mon, Mar 04, 2024 at 03:41:16PM +0300, Aleksander Alekseev wrote: >> I wanted to hook into the EXPLAIN output for queries and add some >> extra information, but since there is no standard_ExplainOneQuery() I >> had to copy the code and create my own version. >> >> Since the pattern with other

Re: Improve eviction algorithm in ReorderBuffer

2024-03-04 Thread Peter Smith
Hi, here are some review comments for v7-0002 == Commit Message 1. This commit adds a hash table to binaryheap in order to track of positions of each nodes in the binaryheap. That way, by using newly added functions such as binaryheap_update_up() etc., both updating a key and removing a node

Re: Add comment to specify timeout unit in ConditionVariableTimedSleep()

2024-03-04 Thread Michael Paquier
On Tue, Mar 05, 2024 at 09:39:11AM +0530, shveta malik wrote: > ConditionVariableTimedSleep() accepts a timeout parameter, but it > doesn't explicitly state the unit for the timeout anywhere. To > determine this, one needs to look into the details of the function to > find it out from the comments

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-03-04 Thread Michael Paquier
On Mon, Mar 04, 2024 at 02:11:08PM +0900, Sutou Kouhei wrote: > If this is a blocker of making COPY format extendable, can > we defer moving the existing text/csv/binary format > implementations to Copy{From,To}Routine for now as Michael > suggested to proceed making COPY format extendable? (Can

Re: a wrong index choose when statistics is out of date

2024-03-04 Thread Andy Fan
David Rowley writes: > On Tue, 5 Mar 2024 at 00:37, Andy Fan wrote: >> >> David Rowley writes: >> > I don't think it would be right to fudge the costs in any way, but I >> > think the risk factor for IndexPaths could take into account the >> > number of unmatched index clauses and increment

Re: POC, WIP: OR-clause support for indexes

2024-03-04 Thread Andrei Lepikhov
On 4/3/2024 09:26, jian he wrote: On Thu, Feb 29, 2024 at 4:59 PM Andrei Lepikhov Feel free to add, change or totally rewrite these changes. On replacement of static ScalarArrayOpExpr dest with dynamic allocated one: After discussion [1] I agree with that replacement. Some style (and

Re: speed up a logical replica setup

2024-03-04 Thread Shlok Kyal
Hi, > I applied the v25 patch and did RUN the 'pg_createsubscriber' command. > I was unable to execute it and experienced the following error: > > $ ./pg_createsubscriber -D node2/ -P "host=localhost port=5432 > dbname=postgres" -d postgres -d db1 -p 9000 -r > ./pg_createsubscriber: invalid

Re: remaining sql/json patches

2024-03-04 Thread jian he
On Tue, Mar 5, 2024 at 9:22 AM Amit Langote wrote: > > Thanks for the heads up. Attaching rebased patches. > Walking through the v41-0001-Add-SQL-JSON-query-functions.patch documentation. I found some minor cosmetic issues. + +select json_query(jsonb '{"a": "[1, 2]"}', 'lax $.a'

Re: remaining sql/json patches

2024-03-04 Thread Andy Fan
Hi, > On Tue, Mar 5, 2024 at 12:03 AM Alvaro Herrera > wrote: >> On 2024-Mar-04, Erik Rijkers wrote: >> >> > In my hands (applying with patch), the patches, esp. 0001, do not apply. >> > But I see the cfbot builds without problem so maybe just ignore these >> > FAILED >> > lines. Better get

Add comment to specify timeout unit in ConditionVariableTimedSleep()

2024-03-04 Thread shveta malik
Hi hackers, ConditionVariableTimedSleep() accepts a timeout parameter, but it doesn't explicitly state the unit for the timeout anywhere. To determine this, one needs to look into the details of the function to find it out from the comments of the internally called function WaitLatch(). It would

Re: initdb's -c option behaves wrong way?

2024-03-04 Thread Kyotaro Horiguchi
At Mon, 4 Mar 2024 09:39:39 +0100, Daniel Gustafsson wrote in > > > > On 4 Mar 2024, at 02:01, Tom Lane wrote: > > > > Daniel Gustafsson writes: > >> I took the liberty to add this to the upcoming CF to make sure we don't > >> lose > >> track of it. > > > > Thanks for doing that, because

Re: MAINTAIN privilege -- what do we need to un-revert it?

2024-03-04 Thread Jeff Davis
On Wed, 2024-02-28 at 09:29 -0800, Jeff Davis wrote: > On Wed, 2024-02-28 at 10:55 -0600, Nathan Bossart wrote: > > I'm afraid I don't have a better idea than adding a short note in > > each > > affected commands's page. > > OK, that works for now. Committed. The only changes are documentation

Re: speed up a logical replica setup

2024-03-04 Thread Shubham Khanna
On Sat, Mar 2, 2024 at 2:19 AM Euler Taveira wrote: > > On Thu, Feb 22, 2024, at 12:45 PM, Hayato Kuroda (Fujitsu) wrote: > > Based on idea from Euler, I roughly implemented. Thought? > > 0001-0013 were not changed from the previous version. > > V24-0014: addressed your comment in the replied

Re: Synchronizing slots from primary to standby

2024-03-04 Thread Amit Kapila
On Tue, Mar 5, 2024 at 6:10 AM Peter Smith wrote: > > == > src/backend/replication/walsender.c > > 5. NeedToWaitForWal > > + /* > + * Check if the standby slots have caught up to the flushed position. It > + * is good to wait up to the flushed position and then let the WalSender > + * send

Re: Synchronizing slots from primary to standby

2024-03-04 Thread Amit Kapila
On Mon, Mar 4, 2024 at 2:27 PM Bertrand Drouvot wrote: > > On Thu, Feb 29, 2024 at 03:38:59PM +0530, Amit Kapila wrote: > > On Thu, Feb 29, 2024 at 9:13 AM Peter Smith wrote: > > > > > > On Tue, Feb 27, 2024 at 11:35 PM Amit Kapila > > > wrote: > > > > > > > > > > > Also, adding wait sounds >

Re: Improve eviction algorithm in ReorderBuffer

2024-03-04 Thread vignesh C
On Wed, 28 Feb 2024 at 11:40, Amit Kapila wrote: > > On Mon, Feb 26, 2024 at 7:54 PM Masahiko Sawada wrote: > > > > A few comments on 0003: > === > 1. > +/* > + * Threshold of the total number of top-level and sub transactions > that controls > + * whether we switch the memory

Re: Support "Right Semi Join" plan shapes

2024-03-04 Thread wenhui qiu
Hi Richard Agree +1 ,I think can push now. Richard On Tue, 5 Mar 2024 at 10:44, Richard Guo wrote: > > On Tue, Jan 30, 2024 at 2:51 PM Alena Rybakina > wrote: > >> I have reviewed your patch and I think it is better to add an Assert for >> JOIN_RIGHT_SEMI to the ExecMergeJoin and

Re: Shared detoast Datum proposal

2024-03-04 Thread Andy Fan
> >> 2. more likely to use up all the memory which is allowed. for example: >> if we set the limit to 1MB, then we kept more data which will be not >> used and then consuming all of the 1MB. >> >> My method is resolving this with some helps from other modules (kind of >> invasive) but can

Re: PostgreSQL Contributors Updates

2024-03-04 Thread vignesh C
On Mon, 4 Mar 2024 at 17:43, Aleksander Alekseev wrote: > > > > All, > > > > > > The PostgreSQL Contributor Page > > > (https://www.postgresql.org/community/contributors/) includes people who > > > have made substantial, long-term contributions of time and effort to the > > > PostgreSQL project.

Re: Support "Right Semi Join" plan shapes

2024-03-04 Thread Richard Guo
On Tue, Jan 30, 2024 at 2:51 PM Alena Rybakina wrote: > I have reviewed your patch and I think it is better to add an Assert for > JOIN_RIGHT_SEMI to the ExecMergeJoin and ExecNestLoop functions to > prevent the use of RIGHT_SEMI for these types of connections (NestedLoop > and MergeJoin).

Re: Support "Right Semi Join" plan shapes

2024-03-04 Thread Richard Guo
On Mon, Mar 4, 2024 at 10:33 AM wenhui qiu wrote: > HI Richard > Now it is starting the last commitfest for v17, can you respond to > Alena Rybakina points? > Thanks for reminding. Will do that soon. Thanks Richard

Re: Improve eviction algorithm in ReorderBuffer

2024-03-04 Thread Peter Smith
Hi, Here are some review comments for v7-0001 1. /* * binaryheap_free * * Releases memory used by the given binaryheap. */ void binaryheap_free(binaryheap *heap) { pfree(heap); } Shouldn't the above function (not modified by the patch) also firstly free the memory allocated for the

Re: vacuumdb/clusterdb/reindexdb: allow specifying objects to process in all databases

2024-03-04 Thread Nathan Bossart
On Mon, Oct 23, 2023 at 03:25:42PM -0500, Nathan Bossart wrote: > rebased I saw that this thread was referenced elsewhere [0], so I figured I'd take a fresh look. From a quick glance, I'd say 0001 and 0002 are pretty reasonable and could probably be committed for v17. 0003 probably requires

Re: Switching XLog source from archive to streaming when primary available

2024-03-04 Thread Nathan Bossart
cfbot claims that this one needs another rebase. I've spent some time thinking about this one. I'll admit I'm a bit worried about adding more complexity to this state machine, but I also haven't thought of any other viable approaches, and this still seems like a useful feature. So, for now, I

RE: Some shared memory chunks are allocated even if related processes won't start

2024-03-04 Thread Hayato Kuroda (Fujitsu)
Dear Alvaro, Thanks for giving comments! > > I agreed it sounds good, but I don't think it can be implemented by > > current interface. An interface for dynamically allocating memory is > > GetNamedDSMSegment(), and it returns the same shared memory region if > > input names are the same.

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-04 Thread Masahiko Sawada
On Mon, Mar 4, 2024 at 8:48 PM John Naylor wrote: > > On Mon, Mar 4, 2024 at 1:05 PM Masahiko Sawada wrote: > > > > On Sun, Mar 3, 2024 at 2:43 PM John Naylor wrote: > > > > Right, I should have said "reset". Resetting a context will delete > > > it's children as well, and seems like it should

Re: Preserve subscription OIDs during pg_upgrade

2024-03-04 Thread Bruce Momjian
On Mon, Feb 26, 2024 at 09:51:40AM +0530, Robert Haas wrote: > > I am not sure that it is a good idea to relax that for PG17 at this > > stage of the development cycle, though, as we have already done a lot > > in this area for pg_upgrade and it may require more tweaks during the > > beta period

Re: Fix race condition in InvalidatePossiblyObsoleteSlot()

2024-03-04 Thread Michael Paquier
On Mon, Feb 26, 2024 at 02:01:45PM +, Bertrand Drouvot wrote: > Though [1] mentioned up-thread is not pushed yet, I'm Sharing the POC patch > now > (see the attached). I have looked at what you have here. First, in a build where 818fefd8fd is included, this makes the test script a lot

Re: Synchronizing slots from primary to standby

2024-03-04 Thread Peter Smith
Here are some review comments for v105-0001 == doc/src/sgml/config.sgml 1. + +The standbys corresponding to the physical replication slots in +standby_slot_names must configure +sync_replication_slots = true so they can receive +logical failover

Re: [PATCH] updates to docs about HOT updates for BRIN

2024-03-04 Thread Jeff Davis
On Tue, 2024-02-27 at 09:48 -0500, Stephen Frost wrote: > Attached is an updated patch which drops the 'such as' and adds a > sentence mentioning that BRIN is the only in-core summarizing index. The original patch reads more clearly to me. In v4, summarizing (the exception) feels like it's

Re: Avoid is possible a expensive function call (src/backend/utils/adt/varlena.c)

2024-03-04 Thread Tom Lane
Michael Paquier writes: > On Mon, Mar 04, 2024 at 03:08:03PM -0300, Ranier Vilela wrote: >> I can't see any user validation at the function pg_newlocale_from_collation. > Matthias is right, look closer. I can see more than one check, > especially note the one related to the collation version

Re: Add new error_action COPY ON_ERROR "log"

2024-03-04 Thread Michael Paquier
On Mon, Mar 04, 2024 at 05:00:00AM +0530, Bharath Rupireddy wrote: > How about an extra option to error_action ignore-with-verbose which is > similar to ignore but when specified emits one NOTICE per malformed > row? With this, one can say COPY x FROM stdin (ON_ERROR > ignore-with-verbose);. > >

Re: a wrong index choose when statistics is out of date

2024-03-04 Thread David Rowley
On Tue, 5 Mar 2024 at 00:37, Andy Fan wrote: > > David Rowley writes: > > I don't think it would be right to fudge the costs in any way, but I > > think the risk factor for IndexPaths could take into account the > > number of unmatched index clauses and increment the risk factor, or > >

Re: Avoid is possible a expensive function call (src/backend/utils/adt/varlena.c)

2024-03-04 Thread Michael Paquier
On Mon, Mar 04, 2024 at 03:08:03PM -0300, Ranier Vilela wrote: > I can't see any user validation at the function pg_newlocale_from_collation. Matthias is right, look closer. I can see more than one check, especially note the one related to the collation version mismatch that should not be

Re: Adding deprecation notices to pgcrypto documentation

2024-03-04 Thread Nathan Bossart
On Mon, Mar 04, 2024 at 10:03:13PM +0100, Daniel Gustafsson wrote: > Cleaning out my inbox I came across this which I still think is worth doing, > any objections to going ahead with it? I think the general idea is reasonable, but I have two small comments: * Should this be a "Warning" and/or

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-03-04 Thread Tom Lane
I wrote: > It looks like SLRU_PAGE_EMPTY has (by chance, or deliberately) > the numeric value of zero, so I guess the majority of our BF > animals are understanding this as "address != NULL". But that > doesn't look like a useful test to be making. In hopes of noticing whether there are other

Re: Injection points: some tools to wait and wake

2024-03-04 Thread Michael Paquier
On Mon, Mar 04, 2024 at 10:51:41AM +0100, Jelte Fennema-Nio wrote: > Yeah, it makes sense that you'd want to backport fixes/changes to > this. As long as you put a disclaimer in the docs that you can do that > for this module, I think it would be fine. Our tests fairly regularly > break anyway

Re: Popcount optimization using AVX512

2024-03-04 Thread Nathan Bossart
(Please don't top-post on the Postgres lists.) On Mon, Mar 04, 2024 at 09:39:36PM +, Amonson, Paul D wrote: > First, apologies on the patch. Find re-attached updated version. Thanks for the new version of the patch. >> -#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62)) >>

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-03-04 Thread Tom Lane
Alvaro Herrera writes: > Pushed that way, but we can discuss further wording improvements/changes > if someone wants to propose any. I just noticed that drongo is complaining about two lines added by commit 53c2a97a9: drongo| 2024-03-04 14:34:52 |

Re: Improve the log message output of basic_archive when basic_archive.archive_directory parameter is not set

2024-03-04 Thread Tom Lane
Nathan Bossart writes: > On Mon, Mar 04, 2024 at 09:27:23PM +0100, Daniel Gustafsson wrote: >> Reviewed-by: Daniel Gustafsson, ¡lvaro Herrera >> Alvaro's name seems wrong. > Hm. It looks alright to me. I copied the name from his e-mail signature, > which has an accent over the first 'A'. I

Re: [PATCH] Exponential backoff for auth_delay

2024-03-04 Thread Michael Banck
Hi, On Mon, Mar 04, 2024 at 03:50:07PM -0500, Robert Haas wrote: > I agree that two GUCs here seems to be one more than necessary, but I > wonder whether we couldn't just say 0 means no exponential backoff and > any other value is the maximum time. Alright, I have changed it so that

Re: Improve the log message output of basic_archive when basic_archive.archive_directory parameter is not set

2024-03-04 Thread Nathan Bossart
On Mon, Mar 04, 2024 at 09:27:23PM +0100, Daniel Gustafsson wrote: > Looks good from a read-through, I like it. A few comments on the commit > message only: > > actionable details about the source of the miconfiguration. This > s/miconfiguration/misconfiguration/ I reworded the commit message

RE: Popcount optimization using AVX512

2024-03-04 Thread Amonson, Paul D
Hi, First, apologies on the patch. Find re-attached updated version. Now I have some questions #1 > -#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62)) > +#define LARGE_OFF_T off_t) 1 << 31) << 31) - 1 + (((off_t) 1 << 31) > +<< 31)) > > IME this means that the autoconf

Re: Adding deprecation notices to pgcrypto documentation

2024-03-04 Thread Daniel Gustafsson
> On 16 Nov 2023, at 21:49, Daniel Gustafsson wrote: > > In the "Allow tests to pass in OpenSSL FIPS mode" thread [0] it was discovered > that 3DES is joining the ranks of NIST disallowed algorithms. The attached > patch adds a small note to the pgcrypto documentation about deprecated uses of >

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-03-04 Thread Matthias van de Meent
On Sat, 2 Mar 2024 at 02:30, Peter Geoghegan wrote: > > On Thu, Feb 15, 2024 at 6:36 PM Peter Geoghegan wrote: > > Attached is v11, which now says something like that in the commit > > message. > > Attached is v12. Some initial comments on the documentation: > +that searches for multiple

Re: [PATCH] Exponential backoff for auth_delay

2024-03-04 Thread Robert Haas
On Mon, Mar 4, 2024 at 2:43 PM Michael Banck wrote: > > 3) Do we actually need the exponential_backoff GUC? Wouldn't it be > > sufficient to have the maximum value, and if it's -1 treat that as no > > backoff? > > That is a good question, I guess that makes sense. > > The next question then is:

Re: Improve the log message output of basic_archive when basic_archive.archive_directory parameter is not set

2024-03-04 Thread Daniel Gustafsson
> On 4 Mar 2024, at 18:22, Nathan Bossart wrote: > > On Mon, Mar 04, 2024 at 03:21:59PM +0100, Daniel Gustafsson wrote: >> Looking at this again I think this is about ready to go in. My only comment >> is >> that doc/src/sgml/archive-modules.sgml probably should be updated to refer to >>

Re: Shared detoast Datum proposal

2024-03-04 Thread Tomas Vondra
On 3/4/24 18:08, Andy Fan wrote: > ... >> >>> I assumed that releasing all of the memory at the end of executor once >>> is not an option since it may consumed too many memory. Then, when and >>> which entry to release becomes a trouble for me. For example: >>> >>> QUERY PLAN >>>

Re: Fix a typo in pg_rotate_logfile

2024-03-04 Thread Daniel Gustafsson
> On 14 Feb 2024, at 21:48, Daniel Gustafsson wrote: >> On 14 Feb 2024, at 19:51, Nathan Bossart wrote: >> On Wed, Feb 14, 2024 at 10:04:49AM -0500, Tom Lane wrote: >>> Daniel Gustafsson writes: Attached is a diff to show what it would look like to remove adminpack (catalog

Re: postgres_fdw fails to see that array type belongs to extension

2024-03-04 Thread Tom Lane
I wrote: > Now that the multirange issue is fixed (3e8235ba4), here's a > new version that includes the needed recursion in ALTER EXTENSION. > I spent some more effort on a proper test case, too. I looked this over again and pushed it. regards, tom lane

Re: Add system identifier to backup manifest

2024-03-04 Thread Robert Haas
On Mon, Mar 4, 2024 at 12:35 AM Amul Sul wrote: > Yes, you are correct. Both the current caller of get_controlfile() has > access to the root directory. > > I have dropped the 0002 patch -- I don't have a very strong opinion to > refactor > get_controlfile() apart from saying that it might be

Re: [PATCH] Exponential backoff for auth_delay

2024-03-04 Thread Michael Banck
Hi, On Wed, Feb 21, 2024 at 10:26:11PM +0100, Tomas Vondra wrote: > Thanks for the patch. I took a closer look at v3, so let me share some > review comments. Please push back if you happen to disagree with some of > it, some of this is going to be more a matter of personal preference. Thanks! As

Re: incremental backup mishandles XLOG_DBASE_CREATE_FILE_COPY

2024-03-04 Thread Robert Haas
On Sat, Feb 24, 2024 at 12:10 PM Noah Misch wrote: > Agreed, those don't touch relation data files. I think you've got all > relation data file mutations. XLOG_DBASE_CREATE_FILE_COPY and XLOG_DBASE_DROP > are the only record types that touch a relation data file without mentioning > it in

Re: Tidy fill hstv array (src/backend/access/heap/pruneheap.c)

2024-03-04 Thread Andres Freund
Hi, On 2024-03-04 08:47:11 -0300, Ranier Vilela wrote: > Does filling a memory area, one by one, with branches, need strong evidence > to prove that it is better than filling a memory area, all at once, without > branches? That's a bogus comparison: a) the memset version modifies the whole

Re: RFC: Logging plan of the running query

2024-03-04 Thread Robert Haas
On Sat, Mar 2, 2024 at 10:46 AM James Coleman wrote: > If I can rephrase this idea: it's basically "delay this interrupt > until inline to the next ExecProcNode execution". Yes, but it's not just that. It also means that the code which would handle the interrupt doesn't need to be called at

Re: Avoid is possible a expensive function call (src/backend/utils/adt/varlena.c)

2024-03-04 Thread Ranier Vilela
Em seg., 4 de mar. de 2024 às 14:54, Matthias van de Meent < boekewurm+postg...@gmail.com> escreveu: > On Mon, 4 Mar 2024 at 18:39, Ranier Vilela wrote: > > > > Hi, > > > > The function var_strcmp is a critical function. > > Inside the function, there is a shortcut condition, > > which allows

Re: Shared detoast Datum proposal

2024-03-04 Thread Andy Fan
> > I decided to whip up a PoC patch implementing this approach, to get a > better idea of how it compares to the original proposal, both in terms > of performance and code complexity. > > Attached are two patches that both add a simple cache in detoast.c, but > differ in when exactly the

Re: Avoid is possible a expensive function call (src/backend/utils/adt/varlena.c)

2024-03-04 Thread Matthias van de Meent
On Mon, 4 Mar 2024 at 18:39, Ranier Vilela wrote: > > Hi, > > The function var_strcmp is a critical function. > Inside the function, there is a shortcut condition, > which allows for a quick exit. > > Unfortunately, the current code calls a very expensive function beforehand, > which if the test

Avoid is possible a expensive function call (src/backend/utils/adt/varlena.c)

2024-03-04 Thread Ranier Vilela
Hi, The function var_strcmp is a critical function. Inside the function, there is a shortcut condition, which allows for a quick exit. Unfortunately, the current code calls a very expensive function beforehand, which if the test was true, all the call time is wasted. So, IMO, it's better to

Re: pgsql: Improve performance of subsystems on top of SLRU

2024-03-04 Thread Alvaro Herrera
FWIW there's a stupid bug in 0002, which is fixed here. I'm writing a simple test for it. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "Now I have my system running, not a byte was off the shelf; It rarely breaks and when it does I fix the code myself. It's

Re: Improve the log message output of basic_archive when basic_archive.archive_directory parameter is not set

2024-03-04 Thread Nathan Bossart
On Mon, Mar 04, 2024 at 03:21:59PM +0100, Daniel Gustafsson wrote: > Looking at this again I think this is about ready to go in. My only comment > is > that doc/src/sgml/archive-modules.sgml probably should be updated to refer to > setting the errdetail, especially since we document the

Re: Tidy fill hstv array (src/backend/access/heap/pruneheap.c)

2024-03-04 Thread Andrey M. Borodin
> On 4 Mar 2024, at 16:47, Ranier Vilela wrote: > > Does filling a memory area, one by one, with branches, need strong evidence > to prove that it is better than filling a memory area, all at once, without > branches? I apologise for being too quick to decide to mark the patch RwF. Wold

Re: a wrong index choose when statistics is out of date

2024-03-04 Thread Tomas Vondra
On 3/4/24 06:33, David Rowley wrote: > On Sun, 3 Mar 2024 at 20:08, Andy Fan wrote: >> The issue can be reproduced with the following steps: >> >> create table x_events (.., created_at timestamp, a int, b int); >> >> create index idx_1 on t(created_at, a); >> create index idx_2 on t(created_at,

Re: Extract numeric filed in JSONB more effectively

2024-03-04 Thread Andy Fan
Peter Eisentraut writes: > On 09.02.24 10:05, Andy Fan wrote: >> 2. Where is the current feature blocked for the past few months? >> It's error message compatible issue! Continue with above setup: >> master: >> select * from tb where (a->'b')::numeric > 3::numeric; >> ERROR: cannot cast jsonb

Re: Shared detoast Datum proposal

2024-03-04 Thread Tomas Vondra
On 3/4/24 02:23, Andy Fan wrote: > > Tomas Vondra writes: > >> On 2/26/24 16:29, Andy Fan wrote: >>> >>> ...> >>> I can understand the benefits of the TOAST cache, but the following >>> issues looks not good to me IIUC: >>> >>> 1). we can't put the result to tts_values[*] since without the

Re: LogwrtResult contended spinlock

2024-03-04 Thread Bharath Rupireddy
Thanks for looking into this. On Thu, Feb 22, 2024 at 1:54 AM Jeff Davis wrote: > > > 3. > > @@ -6371,7 +6373,9 @@ GetFlushRecPtr(TimeLineID *insertTLI) > > If at all, a read > > barrier is warranted here, we can use atomic read with full barrier > > I don't think we need a full barrier but I'm

Re: Synchronizing slots from primary to standby

2024-03-04 Thread Bertrand Drouvot
Hi, On Mon, Mar 04, 2024 at 01:28:04PM +, Zhijie Hou (Fujitsu) wrote: > Attach the V105 patch set Thanks! Sorry I missed those during the previous review: 1 === Commit message: "these functions will block until" s/block/wait/ ? 2 === +when used with logical failover slots, will

Re: Experiments with Postgres and SSL

2024-03-04 Thread Heikki Linnakangas
On 01/03/2024 23:49, Jacob Champion wrote: On Wed, Feb 28, 2024 at 4:10 AM Heikki Linnakangas wrote: I think we'd want to *avoid* changing the major protocol version in a way that would introduce a new roundtrip, though. I'm starting to get up to speed with this patchset. So far I'm mostly

Re: pgsql: Improve performance of subsystems on top of SLRU

2024-03-04 Thread Alvaro Herrera
On 2024-Mar-03, Tom Lane wrote: > This is certainly simpler, but I notice that it holds the current > LWLock across the line > > ptr = (MultiXactMember *) palloc(length * sizeof(MultiXactMember)); > > where the old code did not. Could the palloc take long enough that > holding the lock

Re: remaining sql/json patches

2024-03-04 Thread Alvaro Herrera
On 2024-Mar-04, Erik Rijkers wrote: > In my hands (applying with patch), the patches, esp. 0001, do not apply. > But I see the cfbot builds without problem so maybe just ignore these FAILED > lines. Better get them merged - so I can test there... It's because of dbbca2cf299b. It should apply

Re: remaining sql/json patches

2024-03-04 Thread Erik Rijkers
Op 3/4/24 om 10:40 schreef Amit Langote: Hi Jian, Thanks for the reviews and sorry for the late reply. Replying to all emails in one. > [v40-0001-Add-SQL-JSON-query-functions.patch] > [v40-0002-Show-function-name-in-TableFuncScan.patch] > [v40-0003-JSON_TABLE.patch] In my hands (applying

Re: Statistics Import and Export

2024-03-04 Thread Bertrand Drouvot
Hi, On Tue, Feb 20, 2024 at 02:24:52AM -0500, Corey Huinker wrote: > On Thu, Feb 15, 2024 at 4:09 AM Corey Huinker > wrote: > > > Posting v5 updates of pg_import_rel_stats() and pg_import_ext_stats(), > > which address many of the concerns listed earlier. > > > > Leaving the export/import

Re: Improve the log message output of basic_archive when basic_archive.archive_directory parameter is not set

2024-03-04 Thread Daniel Gustafsson
> On 6 Jan 2024, at 00:03, Nathan Bossart wrote: > I gave it a try. Looking at this again I think this is about ready to go in. My only comment is that doc/src/sgml/archive-modules.sgml probably should be updated to refer to setting the errdetail, especially since we document the errormessage

Provide a pg_truncate_freespacemap function

2024-03-04 Thread Ronan Dunklau
Hello, As we are currently experiencing a FSM corruption issue [1], we need to rebuild FSM when we detect it. I noticed we have something to truncate a visibility map, but nothing for the freespace map, so I propose the attached (liberally copied from the VM counterpart) to allow to truncate

Re: Some shared memory chunks are allocated even if related processes won't start

2024-03-04 Thread 'Alvaro Herrera'
Hello Hayato, On 2024-Mar-04, Hayato Kuroda (Fujitsu) wrote: > OK, I understood that my initial proposal is not so valuable, so I can > withdraw it. Yeah, that's what it seems to me. > About the suggetion, you imagined AutoVacuumRequestWork() and > brininsert(), right? Correct. > I agreed it

Re: Dump-restore loosing 'attnotnull' bit for DEFERRABLE PRIMARY KEY column(s).

2024-03-04 Thread Dean Rasheed
On Mon, 4 Mar 2024 at 12:34, Aleksander Alekseev wrote: > > > This was an experimental patch, I was looking for the comment on the > > proposed > > approach -- whether we could simply skip the throwaway NOT NULL constraint > > for > > deferred PK constraint. Moreover, skip that for any PK

Re: Failures in constraints regression test, "read only 0 of 8192 bytes"

2024-03-04 Thread Tomas Vondra
On 3/4/24 14:16, Ronan Dunklau wrote: > Le samedi 2 mars 2024, 23:29:52 CET Tomas Vondra a écrit : >> These are "my" animals (running at a local university). There's a couple >> interesting details: > > Hi Tomas, > do you still have the failing cluster data ? > > Noah pointed me to this

RE: Synchronizing slots from primary to standby

2024-03-04 Thread Zhijie Hou (Fujitsu)
On Monday, March 4, 2024 5:52 PM Amit Kapila wrote: > > On Mon, Mar 4, 2024 at 9:35 AM Peter Smith > wrote: > > > > OK, if the code will remain as-is wouldn't it be better to anyway > > change the function name to indicate what it really does? > > > > e.g. NeedToWaitForWal -->

RE: Synchronizing slots from primary to standby

2024-03-04 Thread Zhijie Hou (Fujitsu)
On Monday, March 4, 2024 9:55 AM Peter Smith wrote: > > On Sun, Mar 3, 2024 at 6:51 PM Zhijie Hou (Fujitsu) > wrote: > > > > On Sunday, March 3, 2024 7:47 AM Peter Smith > wrote: > > > > > > > 3. > > > + > > > + > > > + Value * is not accepted as it is > > >

RE: Synchronizing slots from primary to standby

2024-03-04 Thread Zhijie Hou (Fujitsu)
On Monday, March 4, 2024 7:22 PM Bertrand Drouvot wrote: > > Hi, > > On Sun, Mar 03, 2024 at 07:56:32AM +, Zhijie Hou (Fujitsu) wrote: > > Here is the V104 patch which addressed above and Peter's comments. > > Thanks! > > A few more random comments: Thanks for the comments! > > 1 ===

Re: Failures in constraints regression test, "read only 0 of 8192 bytes"

2024-03-04 Thread Ronan Dunklau
Le samedi 2 mars 2024, 23:29:52 CET Tomas Vondra a écrit : > These are "my" animals (running at a local university). There's a couple > interesting details: Hi Tomas, do you still have the failing cluster data ? Noah pointed me to this thread, and it looks a bit similar to the FSM corruption

Replication conflicts not processed in ClientWrite

2024-03-04 Thread Magnus Hagander
When a backend is blocked on writing data (such as with a network error or a very slow client), indicated with wait event ClientWrite, it appears to not properly notice that it's overrunning max_standby_streaming_delay, and therefore does not cancel the transaction on the backend. I've

RE: Some shared memory chunks are allocated even if related processes won't start

2024-03-04 Thread Hayato Kuroda (Fujitsu)
Dear Alvaro, Thanks for discussing! > > I think it would be worth allocating AutoVacuumShmem->av_workItems using > dynamic shmem allocation, particularly to prevent workitems from being > discarded just because the array is full¹; but other than that, the > struct is just 64 bytes long so I

Re: type cache cleanup improvements

2024-03-04 Thread Aleksander Alekseev
Hi Teodor, > I'd like to suggest two independent patches to improve performance of type > cache > cleanup. I found a case where type cache cleanup was a reason for low > performance. In short, customer makes 100 thousand temporary tables in one > transaction. > > 1 mapRelType.patch >It just

Re: abi-compliance-checker

2024-03-04 Thread Peter Eisentraut
On 27.02.24 14:25, Alvaro Herrera wrote: I like this idea and I think we should integrate it with the objective of it becoming the workhorse of ABI-stability testing. However, I do not think that the subsequent patches should be part of the tree at all; certainly not the produced .xml files in

Re: Hooking into ExplainOneQuery() complicated by missing standard_ExplainOneQuery

2024-03-04 Thread Aleksander Alekseev
Hi, > I wanted to hook into the EXPLAIN output for queries and add some extra > information, but since there is no standard_ExplainOneQuery() I had to copy > the code and create my own version. > > Since the pattern with other hooks for a function WhateverFunction() seems to > be that there is

Re: Dump-restore loosing 'attnotnull' bit for DEFERRABLE PRIMARY KEY column(s).

2024-03-04 Thread Aleksander Alekseev
Hi hackers, >> I did not see any test addition for this, can we add it? > > > Ok, added it in the attached version. > > This was an experimental patch, I was looking for the comment on the proposed > approach -- whether we could simply skip the throwaway NOT NULL constraint for > deferred PK

Re: Extract numeric filed in JSONB more effectively

2024-03-04 Thread Peter Eisentraut
On 09.02.24 10:05, Andy Fan wrote: 2. Where is the current feature blocked for the past few months? It's error message compatible issue! Continue with above setup: master: select * from tb where (a->'b')::numeric > 3::numeric; ERROR: cannot cast jsonb string to type numeric select * from tb

Re: Shared detoast Datum proposal

2024-03-04 Thread Andy Fan
Tomas Vondra writes: >>> But I'm a bit surprised the patch needs to pass a >>> MemoryContext to so many places as a function argument - that seems to >>> go against how we work with memory contexts. Doubly so because it seems >>> to only ever pass CurrentMemoryContext anyway. So what's that

Re: PostgreSQL Contributors Updates

2024-03-04 Thread Aleksander Alekseev
> > All, > > > > The PostgreSQL Contributor Page > > (https://www.postgresql.org/community/contributors/) includes people who > > have made substantial, long-term contributions of time and effort to the > > PostgreSQL project. The PostgreSQL Contributors Team recognizes the > > following people

Re: Commitfest Manager for March

2024-03-04 Thread Aleksander Alekseev
Hi, > >> The call for a CFM volunteer is still open. > > > > I always wanted to try. And most of the stuff I'm interested in is already > > committed. > > > > But given importance of last commitfest before feature freeze, we might be > > interested in more experienced CFM. > > If I can do

  1   2   >