Re: Removing unneeded self joins

2024-02-23 Thread Noah Misch
On Sat, Feb 24, 2024 at 12:36:59AM +0200, Alexander Korotkov wrote: > On Thu, Feb 22, 2024 at 10:51 AM Andrei Lepikhov > wrote: > > On 21/2/2024 14:26, Richard Guo wrote: > > > I think the right fix for these issues is to introduce a new element > > > 'sublevels_up' in ReplaceVarnoContext, and

Re: incremental backup mishandles XLOG_DBASE_CREATE_FILE_COPY

2024-02-23 Thread Noah Misch
On Fri, Feb 23, 2024 at 08:47:52PM +0530, Robert Haas wrote: > If XLOG_DBASE_CREATE_FILE_COPY occurs between an incremental backup > and its reference backup, every relation whose DB OID and tablespace > OID match the corresponding values in that record should be backed up > in full. Currently

Re: Why is subscription/t/031_column_list.pl failing so much?

2024-02-23 Thread Amit Kapila
On Sat, Feb 24, 2024 at 2:31 AM Tom Lane wrote: > > Amit Kapila writes: > > Seeing no objections, I have pushed the required test changes to silence > > the BF. > > The farm is still unhappy in the v16 branch. > As this feature and tests were introduced in 15, I think we should backpatch till

Re: Potential issue in ecpg-informix decimal converting functions

2024-02-23 Thread Michael Paquier
On Fri, Feb 23, 2024 at 06:03:41PM +0300, a.ima...@postgrespro.ru wrote: > Thank's for advice, the patch will be registered for the next commitfest. The risk looks really minimal to me, but playing with error codes while the logic of the function is unchanged does not strike me as something to

Re: ExecAppendAsyncEventWait() in REL_14_STABLE can corrupt PG_exception_stack

2024-02-23 Thread Michael Paquier
On Fri, Feb 23, 2024 at 01:21:14PM +0300, Alexander Pyhalov wrote: > Recent commit 555276f8594087ba15e0d58e38cd2186b9f39f6d introduced final > cleanup of node->as_eventset in ExecAppendAsyncEventWait(). > Unfortunately, now this function can return in the middle of TRY/FINALLY > block, without

Re: Proposal for implementing OCSP Stapling in PostgreSQL

2024-02-23 Thread David Zhang
Hi Hackers, This is the 2nd version patch with following updates: 1) Changed the frontend SSL parameter from `ssl_ocsp_stapling` to `sslocspstapling` to align with other SSL parameters. 2) Documented both the backend parameter `ssl_ocsp_file` and the frontend parameter `sslocspstapling`.

Re: RangeTblEntry jumble omissions

2024-02-23 Thread Tom Lane
Julien Rouhaud writes: > On Fri, Feb 23, 2024 at 04:26:53PM +0100, Peter Eisentraut wrote: >> - funcordinality >> This was probably just forgotten. It should be included because the WITH >> ORDINALITY clause changes the query result. > Agreed. Seems OK. >> - lateral >> Also probably

Re: Documentation: warn about two_phase when altering a subscription

2024-02-23 Thread Tristen Raab
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: tested, passed Spec compliant: not tested Documentation:tested, passed Hello, I've reviewed your patch, it applies correctly and the

Re: Add lookup table for replication slot invalidation causes

2024-02-23 Thread Michael Paquier
On Fri, Feb 23, 2024 at 09:04:04AM +1100, Peter Smith wrote: > I would've just removed every local variable instead of adding more of > them. I also felt the iteration starting from RS_INVAL_NONE instead of > 0 is asserting RS_INVAL_NONE must always be the first enum and can't > be rearranged.

Re: Printing backtrace of postgres processes

2024-02-23 Thread Michael Paquier
On Fri, Feb 23, 2024 at 04:39:47PM +0100, Christoph Berg wrote: > I'd be concerned about the cost of doing that as part of the startup > of every single backend process. Shouldn't this rather be done within > the postmaster so it's automatically inherited by forked backends? > (EXEC_BACKEND

Re: RangeTblEntry jumble omissions

2024-02-23 Thread Julien Rouhaud
Hi, On Fri, Feb 23, 2024 at 04:26:53PM +0100, Peter Eisentraut wrote: > > - alias > > Currently, two queries like > > SELECT * FROM t1 AS foo > SELECT * FROM t1 AS bar > > are counted together by pg_stat_statements -- that might be ok, but they > both get listed under whichever one is run first,

Re: Removing unneeded self joins

2024-02-23 Thread Alexander Korotkov
On Thu, Feb 22, 2024 at 10:51 AM Andrei Lepikhov wrote: > On 21/2/2024 14:26, Richard Guo wrote: > > I think the right fix for these issues is to introduce a new element > > 'sublevels_up' in ReplaceVarnoContext, and enhance replace_varno_walker > > to 1) recurse into subselects with sublevels_up

Re: cleanup patches for dshash

2024-02-23 Thread Nathan Bossart
On Sun, Jan 21, 2024 at 11:07:15PM -0600, Nathan Bossart wrote: > I attempted to fix this in v2 of the patch set. If there are no objections, I plan to commit these patches early next week. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: MAINTAIN privilege -- what do we need to un-revert it?

2024-02-23 Thread Nathan Bossart
(Apologies in advance for anything I'm bringing up that we've already covered somewhere else.) On Fri, Feb 16, 2024 at 04:03:55PM -0800, Jeff Davis wrote: > Note the changes in amcheck. It's creating functions and calling those > functions from the comparators, and so the comparators need to set

Re: Why is subscription/t/031_column_list.pl failing so much?

2024-02-23 Thread Tom Lane
Amit Kapila writes: > Seeing no objections, I have pushed the required test changes to silence the > BF. The farm is still unhappy in the v16 branch. regards, tom lane

Re: locked reads for atomics

2024-02-23 Thread Nathan Bossart
Here is a v3 of the patch set with the first draft of the commit messages. There are no code differences between v2 and v3. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From f1441aca96f157c5557d9a961fe85902b510f293 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Fri, 23

Re: locked reads for atomics

2024-02-23 Thread Nathan Bossart
On Fri, Feb 23, 2024 at 10:25:00AM -0800, Jeff Davis wrote: > To be clear: > > x = pg_atomic_[read|write]_membarrier_u64(); > > is semantically equivalent to: > > pg_memory_barrier(); > x = pg_atomic_[read|write]_u64(); > pg_memory_barrier(); > > ? > > If so, that does seem more

Re: locked reads for atomics

2024-02-23 Thread Jeff Davis
On Fri, 2024-02-23 at 10:17 -0600, Nathan Bossart wrote: > The idea is > to provide an easy way to remove spinlocks, etc. and use atomics for > less > performance-sensitive stuff.  The implementations are intended to be > relatively inexpensive and might continue to improve in the future, > but

Re: SQL Property Graph Queries (SQL/PGQ)

2024-02-23 Thread Tomas Vondra
On 2/23/24 17:15, Peter Eisentraut wrote: > On 16.02.24 20:23, Andres Freund wrote: >> One aspect that I m concerned with structurally is that the >> transformation, >> from property graph queries to something postgres understands, is done >> via the >> rewrite system. I doubt that that is a good

Re: Thoughts about NUM_BUFFER_PARTITIONS

2024-02-23 Thread Tomas Vondra
On 2/23/24 15:40, wenhui qiu wrote: > Hi Tomas Vondra > Thanks for the information! But I found postgres pro enterprise > version has been implemented ,However, it defaults to 16 and maxes out at > 128, and the maxes are the same as in PostgreSQL.I kindly hope that if the > developers can

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-23 Thread Давыдов Виталий
Hi Amit, Amit Kapila wrote: I don't see we do anything specific for 2PC transactions to make them behave differently than regular transactions with respect to synchronous_commit setting. What makes you think so? Can you pin point the code you are referring to?Yes, sure. The function

Re: Improve eviction algorithm in ReorderBuffer

2024-02-23 Thread Tomas Vondra
Hi, I did a basic review and testing of this patch today. Overall I think the patch is in very good shape - I agree with the tradeoffs it makes, and I like the approach in general. I do have a couple minor comments about the code, and then maybe a couple thoughts about the approach. First, some

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-23 Thread Давыдов Виталий
Hi Ajin, Thank you for your feedback. Could you please try to increase the number of clients (-c pgbench option) up to 20 or more? It seems, I forgot to specify it. With best regards, Vitaly Davydov On Fri, Feb 23, 2024 at 12:29 AM Давыдов Виталий wrote: Dear All, I'd like to present and

Re: locked reads for atomics

2024-02-23 Thread Nathan Bossart
On Thu, Feb 22, 2024 at 11:53:50AM -0800, Jeff Davis wrote: > On Thu, 2024-02-22 at 12:58 +0530, Bharath Rupireddy wrote: >> There's some immediate use for reads/writes with barrier semantics - > > Is this mainly a convenience for safety/readability? Or is it faster in > some cases than doing an

Re: SQL Property Graph Queries (SQL/PGQ)

2024-02-23 Thread Peter Eisentraut
On 16.02.24 20:23, Andres Freund wrote: One aspect that I m concerned with structurally is that the transformation, from property graph queries to something postgres understands, is done via the rewrite system. I doubt that that is a good idea. For one it bars the planner from making plans that

Re: Relation bulk write facility

2024-02-23 Thread Tom Lane
Heikki Linnakangas writes: > Thanks, the error message was clear enough: >> bulk_write.c:78:3: error: redefinition of typedef 'BulkWriteState' is a C11 >> feature [-Werror,-Wtypedef-redefinition] >> } BulkWriteState; > Fixed now, but I'm a bit surprised other buildfarm members nor cirrus CI >

Re: Relation bulk write facility

2024-02-23 Thread Heikki Linnakangas
On 23/02/2024 17:15, Tom Lane wrote: Heikki Linnakangas writes: Buildfarm animals 'sifaka' and 'longfin' are not happy, I will investigate.. Those are mine, let me know if you need local investigation. Thanks, the error message was clear enough: bulk_write.c:78:3: error: redefinition of

Re: Printing backtrace of postgres processes

2024-02-23 Thread Christoph Berg
Re: Michael Paquier > >>• backtrace() and backtrace_symbols_fd() don't call malloc() > >> explic‐ > >> itly, but they are part of libgcc, which gets loaded > >> dynamically > >> when first used. Dynamic loading usually triggers a call to > >> mal‐ > >>

RangeTblEntry jumble omissions

2024-02-23 Thread Peter Eisentraut
I think there are some fields from the RangeTblEntry struct missing in the jumble (function _jumbleRangeTblEntry()). Probably, some of these were really just forgotten, in other cases this might be an intentional decision, but then it might be good to document it. This has come up in thread

Re: RFC: Logging plan of the running query

2024-02-23 Thread Robert Haas
On Fri, Feb 23, 2024 at 7:50 PM Julien Rouhaud wrote: > On Fri, Feb 23, 2024 at 10:22:32AM +0530, Robert Haas wrote: > > On Thu, Feb 22, 2024 at 6:25 AM James Coleman wrote: > > > This is potentially a bit of a wild idea, but I wonder if having some > > > kind of argument to

Re: RangeTblEntry.inh vs. RTE_SUBQUERY

2024-02-23 Thread Tom Lane
Dean Rasheed writes: > On Fri, 23 Feb 2024 at 14:35, Peter Eisentraut wrote: >> Various code comments say that the RangeTblEntry field inh may only be >> set for entries of kind RTE_RELATION. > Yes, it's explained a bit more clearly/accurately in > expand_inherited_rtentry(): > * "inh" is

incremental backup mishandles XLOG_DBASE_CREATE_FILE_COPY

2024-02-23 Thread Robert Haas
If XLOG_DBASE_CREATE_FILE_COPY occurs between an incremental backup and its reference backup, every relation whose DB OID and tablespace OID match the corresponding values in that record should be backed up in full. Currently that's not happening, because the WAL summarizer doesn't see the

Re: Relation bulk write facility

2024-02-23 Thread Tom Lane
Heikki Linnakangas writes: > Buildfarm animals 'sifaka' and 'longfin' are not happy, I will investigate.. Those are mine, let me know if you need local investigation. regards, tom lane

Re: Relation bulk write facility

2024-02-23 Thread Heikki Linnakangas
On 23/02/2024 16:27, Heikki Linnakangas wrote: Committed this. Thanks everyone! Buildfarm animals 'sifaka' and 'longfin' are not happy, I will investigate.. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Potential issue in ecpg-informix decimal converting functions

2024-02-23 Thread a . imamov
Daniel Gustafsson писал(а) 2024-02-23 13:44: On 22 Feb 2024, at 17:54, a.ima...@postgrespro.ru wrote: PGTYPESnumeric_to_int() and PGTYPESnumeric_to_long() functions return only 0 or -1. So ECPG_INFORMIX_NUM_OVERFLOW can never be returned. Indeed, this looks like an oversight. I think

Re: RangeTblEntry.inh vs. RTE_SUBQUERY

2024-02-23 Thread Dean Rasheed
On Fri, 23 Feb 2024 at 14:35, Peter Eisentraut wrote: > > Various code comments say that the RangeTblEntry field inh may only be > set for entries of kind RTE_RELATION. > > The function pull_up_simple_union_all() in prepjointree.c sets ->inh to > true for RTE_SUBQUERY entries: > > /* >

Re: Thoughts about NUM_BUFFER_PARTITIONS

2024-02-23 Thread wenhui qiu
Hi Tomas Vondra Thanks for the information! But I found postgres pro enterprise version has been implemented ,However, it defaults to 16 and maxes out at 128, and the maxes are the same as in PostgreSQL.I kindly hope that if the developers can explain what the purpose of this is.May be 128

RangeTblEntry.inh vs. RTE_SUBQUERY

2024-02-23 Thread Peter Eisentraut
Various code comments say that the RangeTblEntry field inh may only be set for entries of kind RTE_RELATION. For example *inh is true for relation references that should be expanded to include *inheritance children, if the rel has any. This *must* be false for *RTEs other

Re: Relation bulk write facility

2024-02-23 Thread Heikki Linnakangas
Committed this. Thanks everyone! -- Heikki Linnakangas Neon (https://neon.tech)

Re: RFC: Logging plan of the running query

2024-02-23 Thread Julien Rouhaud
Hi, On Fri, Feb 23, 2024 at 10:22:32AM +0530, Robert Haas wrote: > On Thu, Feb 22, 2024 at 6:25 AM James Coleman wrote: > > This is potentially a bit of a wild idea, but I wonder if having some > > kind of argument to CHECK_FOR_INTERRUPTS() signifying we're in > > "normal" as opposed to

Re: Synchronizing slots from primary to standby

2024-02-23 Thread Bertrand Drouvot
Hi, On Fri, Feb 23, 2024 at 09:30:58AM +, Zhijie Hou (Fujitsu) wrote: > On Friday, February 23, 2024 5:07 PM Bertrand Drouvot > wrote: > > On Fri, Feb 23, 2024 at 02:15:11PM +0530, shveta malik wrote: > > > > > > Thanks for the details. I understand it now. We do not use '=' in our > > >

Re: table inheritance versus column compression and storage settings

2024-02-23 Thread Ashutosh Bapat
On Tue, Feb 20, 2024 at 3:51 PM Peter Eisentraut wrote: > > I have reverted the patch for now (and re-opened the commitfest entry). > We should continue to work on this and see if we can at least try to get > the pg_dump test coverage suitable. > I have started a separate thread for dump/restore

Re: Add missing error codes to PANIC/FATAL error reports in xlog.c and relcache.c

2024-02-23 Thread Daniel Gustafsson
> On 23 Feb 2024, at 13:09, Nazir Bilal Yavuz wrote: > Does errmsg_internal() need to be used all the time when turning elogs > into ereports? errmsg_internal()'s comment says that "This should be > used for "can't happen" cases that are probably not worth spending > translation effort on.". Is

Re: Add missing error codes to PANIC/FATAL error reports in xlog.c and relcache.c

2024-02-23 Thread Nazir Bilal Yavuz
Hi, Thanks for the review! On Thu, 22 Feb 2024 at 16:55, Daniel Gustafsson wrote: > > > On 6 Dec 2023, at 14:03, Nazir Bilal Yavuz wrote: > > > There is an ongoing thread [1] for adding missing SQL error codes to > > PANIC and FATAL error reports in xlogrecovery.c file. I did the same > > but

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-02-23 Thread Alvaro Herrera
On 2024-Feb-23, Andrey M. Borodin wrote: > I'm sure anyone with multiple CPUs should increase, not decrease > previous default of 128 buffers (with 512MB shared buffers). Having > more CPUs (the only way to benefit from more locks) implies bigger > transaction buffers. Sure. > IMO making bank

Re: Synchronizing slots from primary to standby

2024-02-23 Thread Bertrand Drouvot
Hi, On Fri, Feb 23, 2024 at 09:46:00AM +, Zhijie Hou (Fujitsu) wrote: > On Friday, February 23, 2024 1:22 PM shveta malik > wrote: > > > > > Thanks for the patches. Had a quick look at v95_2, here are some > > trivial comments: > > Thanks for the comments. > > > 6) streaming replication

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-02-23 Thread Dilip Kumar
On Fri, Feb 23, 2024 at 1:06 PM Dilip Kumar wrote: > > On Fri, Feb 23, 2024 at 1:48 AM Alvaro Herrera > wrote: > > > > On 2024-Feb-07, Dilip Kumar wrote: > > > > > On Wed, Feb 7, 2024 at 3:49 PM Alvaro Herrera > > > wrote: > > > > > > Sure, but is that really what we want? > > > > > > So your

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-02-23 Thread Andrey M. Borodin
> On 23 Feb 2024, at 12:36, Dilip Kumar wrote: > >> Another thing I've been thinking is that perhaps it would be useful to >> make the banks smaller, when the total number of buffers is small. For >> example, if you have 16 or 32 buffers, it's not really clear to me that >> it makes sense to

Re: Potential issue in ecpg-informix decimal converting functions

2024-02-23 Thread Daniel Gustafsson
> On 22 Feb 2024, at 17:54, a.ima...@postgrespro.ru wrote: > PGTYPESnumeric_to_int() and PGTYPESnumeric_to_long() > functions return only 0 or -1. So ECPG_INFORMIX_NUM_OVERFLOW can never > be returned. Indeed, this looks like an oversight. > I think dectoint(), dectolong() and

Refactor SASL exchange in preparation for OAuth Bearer

2024-02-23 Thread Daniel Gustafsson
The attached two patches are smaller refactorings to the SASL exchange and init codepaths which are required for the OAuthbearer work [0]. Regardless of the future of that patchset, these refactorings are nice cleanups and can be considered in isolation. Another goal is of course to reduce scope

ExecAppendAsyncEventWait() in REL_14_STABLE can corrupt PG_exception_stack

2024-02-23 Thread Alexander Pyhalov
Hi. Recent commit 555276f8594087ba15e0d58e38cd2186b9f39f6d introduced final cleanup of node->as_eventset in ExecAppendAsyncEventWait(). Unfortunately, now this function can return in the middle of TRY/FINALLY block, without restoring PG_exception_stack. We found this while working on our

Re: A failure in t/001_rep_changes.pl

2024-02-23 Thread vignesh C
On Wed, 14 Feb 2024 at 13:19, Bharath Rupireddy wrote: > > Hi, > > I recently observed an assertion failure twice in t/001_rep_changes.pl > on HEAD with the backtrace [1] on my dev EC2 c5.4xlarge instance [2]. > Unfortunately I'm not observing it again. I haven't got a chance to > dive deep into

Re: Synchronizing slots from primary to standby

2024-02-23 Thread Bertrand Drouvot
Hi, On Fri, Feb 23, 2024 at 04:36:44AM +, Zhijie Hou (Fujitsu) wrote: > On Friday, February 23, 2024 10:18 AM Zhijie Hou (Fujitsu) > wrote: > > > > > > Hi, > > > > > > Since the slotsync worker patch has been committed, I rebased the > > > remaining patches. > > > And here is the V95 patch

Re: Improve readability by using designated initializers when possible

2024-02-23 Thread Jelte Fennema-Nio
On Fri, 23 Feb 2024 at 02:57, Jeff Davis wrote: > Sorry, I was unclear. I was asking a question about the reason the > ObjectClass and the object_classes[] array exist in the current code, > it wasn't a direct question about your patch. I did a bit of git spelunking and the reason seems to be

RE: Synchronizing slots from primary to standby

2024-02-23 Thread Zhijie Hou (Fujitsu)
On Friday, February 23, 2024 1:22 PM shveta malik wrote: > > Thanks for the patches. Had a quick look at v95_2, here are some > trivial comments: Thanks for the comments. > 6) streaming replication standby server slot names that logical walsender > processes will wait for > > Is it better

Re: how to read table options during smgropen()

2024-02-23 Thread Heikki Linnakangas
On 22/02/2024 20:22, Dima Rybakov (Tlt) wrote: Dear pgsql hackers, I am developing custom storage for pgsql tables. I am using md* functions and smgrsw[] structure to switch between different magnetic disk access methods. I want to add some custom options while table created psql# create

Re: make BuiltinTrancheNames less ugly

2024-02-23 Thread Heikki Linnakangas
On 12/02/2024 19:01, Tristan Partin wrote: On Wed Jan 24, 2024 at 8:09 AM CST, Alvaro Herrera wrote: IMO it would be less ugly to have the origin file lwlocknames.txt be not a text file but a .h with a macro that can be defined by interested parties so that they can extract what they want from

RE: Synchronizing slots from primary to standby

2024-02-23 Thread Zhijie Hou (Fujitsu)
On Friday, February 23, 2024 5:07 PM Bertrand Drouvot wrote: > > On Fri, Feb 23, 2024 at 02:15:11PM +0530, shveta malik wrote: > > On Fri, Feb 23, 2024 at 1:28 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > Because one could create say the "=" OPERATOR in their own schema, > > >

Re: Improve eviction algorithm in ReorderBuffer

2024-02-23 Thread vignesh C
On Fri, 9 Feb 2024 at 20:51, Masahiko Sawada wrote: > > On Thu, Feb 8, 2024 at 6:33 PM Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Sawada-san, > > > > Thanks for making v3 patchset. I have also benchmarked the case [1]. > > Below results are the average of 5th, there are almost the same result

Re: Synchronizing slots from primary to standby

2024-02-23 Thread Bertrand Drouvot
Hi, On Fri, Feb 23, 2024 at 02:15:11PM +0530, shveta malik wrote: > On Fri, Feb 23, 2024 at 1:28 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > Because one could create say the "=" OPERATOR in their own schema, attach a > > function to it doing undesired stuff and change the search_path for

Re: Add publisher and subscriber to glossary documentation.

2024-02-23 Thread Shlok Kyal
> Here are some comments for patch v2. > > == > > 1. There are whitespace problems > > [postgres@CentOS7-x64 oss_postgres_misc]$ git apply > ../patches_misc/v2-0001-Add-publisher-and-subscriber-to-glossary-document.patch >

Re: Synchronizing slots from primary to standby

2024-02-23 Thread shveta malik
On Fri, Feb 23, 2024 at 1:28 PM Bertrand Drouvot wrote: > > Hi, > > Because one could create say the "=" OPERATOR in their own schema, attach a > function to it doing undesired stuff and change the search_path for the > database > the sync slot worker connects to. > > Then this new "=" operator