Re: Change GUC hashtable to use simplehash?

2023-12-11 Thread John Naylor
On Sun, Dec 10, 2023 at 2:18 AM Jeff Davis wrote: > > On Sat, 2023-12-09 at 18:52 +0700, John Naylor wrote: > > > I tested using the new hash function APIs for my search path cache, > > > and > > > there's a significant speedup for cases not benefiting from > > > a86c61c9ee. > > > It's enough that

Re: Adding facility for injection points (or probe points?) for more advanced tests

2023-12-11 Thread Dilip Kumar
On Mon, Dec 11, 2023 at 3:14 PM Michael Paquier wrote: > > On Mon, Dec 11, 2023 at 11:09:45AM +0530, Dilip Kumar wrote: > > I haven't specifically done a review or testing of this patch, but I > > have used this for testing the CLOG group update code with my > > SLRU-specific changes and I found i

Re: Improve eviction algorithm in ReorderBuffer

2023-12-11 Thread Dilip Kumar
On Tue, Dec 12, 2023 at 9:01 AM Masahiko Sawada wrote: > > Hi all, > > As the comment of ReorderBufferLargestTXN() says, it's very slow with > many subtransactions: > > /* > * Find the largest transaction (toplevel or subxact) to evict (spill to > disk). > * > * XXX With many subtransactions t

Improve eviction algorithm in ReorderBuffer

2023-12-11 Thread Masahiko Sawada
Hi all, As the comment of ReorderBufferLargestTXN() says, it's very slow with many subtransactions: /* * Find the largest transaction (toplevel or subxact) to evict (spill to disk). * * XXX With many subtransactions this might be quite slow, because we'll have * to walk through all of them. T

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-11 Thread John Naylor
On Mon, Dec 11, 2023 at 1:18 PM Masahiko Sawada wrote: > I've attached the updated patch set. From the previous patch set, I've > merged patches 0007 to 0010. The other changes such as adding RT_GET() > still are unmerged for now, for discussion. Probably we can make them > as follow-up patches a

RE: [Proposal] Add foreign-server health checks infrastructure

2023-12-11 Thread Hayato Kuroda (Fujitsu)
Dear Shubham, > > I am failing to apply the latest > Patch-"v39-0001-postgres_fdw-add-postgres_fdw_verify_connection-.patch" > for review. Please find the error I am facing: > D:\Project\Postgres>git am > D:\Project\Patch\v39-0001-postgres_fdw-add-postgres_fdw_verify_connection > -.patch > error

RE: Make COPY format extendable: Extract COPY TO format implementations

2023-12-11 Thread Hayato Kuroda (Fujitsu)
Dear Sutou-san, Junwang, Sorry for the delay reply. > > Can we discuss how to proceed this improvement? > > There are 2 approaches for it: > > 1. Do the followings concurrently: >a. Implementing small changes that got a consensus and > merge them step-by-step > (e.g. We got a c

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-11 Thread Junwang Zhao
On Mon, Dec 11, 2023 at 10:32 PM Masahiko Sawada wrote: > > On Mon, Dec 11, 2023 at 7:19 PM Michael Paquier wrote: > > > > On Mon, Dec 11, 2023 at 10:57:15AM +0900, Masahiko Sawada wrote: > > > IIUC we cannot create two same name functions with the same arguments > > > but a different return valu

Re: Add --check option to pgindent

2023-12-11 Thread Daniel Gustafsson
> On 12 Dec 2023, at 01:09, Tristan Partin wrote: > > Not sold on the name, but --check is a combination of --silent-diff and > --show-diff. I envision --check mostly being used in CI environments. I > recently came across a situation where this behavior would have been useful. > Without --ch

Add --check option to pgindent

2023-12-11 Thread Tristan Partin
Not sold on the name, but --check is a combination of --silent-diff and --show-diff. I envision --check mostly being used in CI environments. I recently came across a situation where this behavior would have been useful. Without --check, you're left to capture the output of --show-diff and exit

Adding comments to help understand psql hidden queries

2023-12-11 Thread Greg Sabino Mullane
The use of the --echo-hidden flag in psql is used to show people the way psql performs its magic for its backslash commands. None of them has more magic than "\d relation", but it suffers from needing a lot of separate queries to gather all of the information it needs. Unfortunately, those queries

Re: POC: Extension for adding distributed tracing - pg_tracing

2023-12-11 Thread Craig Ringer
Hi all Just saw this thread. I cooked up a PoC distributed tracing support in Pg years ago as part of the 2ndQuadrant BDR project. I used GUCs to set the `opentelemtery_trace_id` and `opentelemetry_span_id`. These can be sent as protocol-level messages by the client driver if the client driver h

Re: unconstify()/unvolatize() vs g++/clang++

2023-12-11 Thread Thomas Munro
On Mon, Dec 11, 2023 at 11:32 PM Thomas Munro wrote: > On Mon, Dec 11, 2023 at 10:17 PM Peter Eisentraut > wrote: > > If you are slightly more daring, you can write an alternative definition > > in C++ using const_cast? > > Oh, yeah, right, that works for my case. I can't think of any > reason

Re: Sorting regression of text function result since commit 586b98fdf1aae

2023-12-11 Thread Tom Lane
Jehan-Guillaume de Rorthais writes: > It looks like since 586b98fdf1aae, the result type collation of "convert_from" > is forced to "C", like the patch does for type "name", instead of the > "default" > collation for type "text". Well, convert_from() inherits its result collation from the input,

Sorting regression of text function result since commit 586b98fdf1aae

2023-12-11 Thread Jehan-Guillaume de Rorthais
Hi, A customer found what looks like a sort regression while testing his code from v11 on a higher version. We hunt this regression down to commit 586b98fdf1aae, introduced in v12. Consider the following test case: createdb -l fr_FR.utf8 -T template0 reg psql reg <<<" BEGIN; CREATE TABLE

Teach predtest about IS [NOT] proofs

2023-12-11 Thread James Coleman
Hello, I recently encountered a case where partial indexes were surprisingly not being used. The issue is that predtest doesn't understand how boolean values and IS expressions relate. For example if I have: create table foo(i int, bar boolean); create index on foo(i) where bar is true; then t

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

2023-12-11 Thread Alexander Korotkov
On Mon, Dec 11, 2023 at 5:56 PM Alexander Korotkov wrote: > > BTW, do we really need to keep around the BTScanOpaqueData.firstPage > > field? Why can't the call to _bt_readpage from _bt_first (and from > > _bt_endpoint) just pass "firstPage=true" as a simple argument? Note > > that the first call

Re: Synchronizing slots from primary to standby

2023-12-11 Thread Amit Kapila
On Mon, Dec 11, 2023 at 2:41 PM shveta malik wrote: > > > > > 5. > > +synchronize_slots(WalReceiverConn *wrconn) > > { > > ... > > ... > > + /* The syscache access needs a transaction env. */ > > + StartTransactionCommand(); > > + > > + /* > > + * Make result tuples live outside TopTransactionCont

Re: Report planning memory in EXPLAIN ANALYZE

2023-12-11 Thread Ashutosh Bapat
On Mon, Dec 11, 2023 at 2:06 PM jian he wrote: > > On Mon, Dec 4, 2023 at 3:24 PM Ashutosh Bapat > wrote: > > > > On Fri, Dec 1, 2023 at 8:27 AM Andrei Lepikhov > > wrote: > > > > > > On 30/11/2023 18:40, Alvaro Herrera wrote: > > > > Well, SUMMARY is enabled by default with ANALYZE, and I'd rat

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

2023-12-11 Thread Alexander Korotkov
Hi, Peter! On Sat, Dec 9, 2023 at 4:29 AM Peter Geoghegan wrote: > Does this really need to work at the scan key level, rather than at > the whole-scan level? Wouldn't it make more sense to just totally > disable it for the whole scan, since we'll barely ever need to do that > anyway? > > My Scal

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

2023-12-11 Thread Peter Geoghegan
Will you be in Prague this week? If not this might have to wait.

Re: brininsert optimization opportunity

2023-12-11 Thread Tomas Vondra
On 12/11/23 16:00, Alexander Lakhin wrote: > Hello Tomas and Soumyadeep, > > 25.11.2023 23:06, Tomas Vondra wrote: >> I've done a bit more cleanup on the last version of the patch (renamed >> the fields to start with bis_ as agreed, rephrased the comments / docs / >> commit message a bit) and push

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-11 Thread Tomas Vondra
On 12/11/23 07:12, Amit Kapila wrote: > On Sat, Dec 9, 2023 at 12:16 PM Amit Kapila wrote: >> >> Thanks, I could also reproduce the issue on back branches (tried till >> 12), and the fix works. I'll push this on Monday. >> > > Peter sent one minor suggestion (to write the check differently for >

Re: brininsert optimization opportunity

2023-12-11 Thread Alexander Lakhin
Hello Tomas and Soumyadeep, 25.11.2023 23:06, Tomas Vondra wrote: I've done a bit more cleanup on the last version of the patch (renamed the fields to start with bis_ as agreed, rephrased the comments / docs / commit message a bit) and pushed. Please look at a query, which produces warnings si

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-11 Thread Masahiko Sawada
On Mon, Dec 11, 2023 at 7:19 PM Michael Paquier wrote: > > On Mon, Dec 11, 2023 at 10:57:15AM +0900, Masahiko Sawada wrote: > > IIUC we cannot create two same name functions with the same arguments > > but a different return value type in the first place. It seems to me > > to be an overkill to ch

Re: [CAUTION!! freemail] Re: Partial aggregates pushdown

2023-12-11 Thread Robert Haas
On Thu, Dec 7, 2023 at 4:12 PM Bruce Momjian wrote: > Second, the patch already has a mechanism to check the remote server > version to see if it is the same or newer. Here is the version check > documentation patch: Right. This feature can certainly be implemented in a backward-compatible way.

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-12-11 Thread Alena Rybakina
Hi! Thank you for your work. Your patch looks better! On 10.12.2023 13:32, jian he wrote: On Fri, Dec 8, 2023 at 3:09 PM Alena Rybakina wrote: Thank you for your work. Unfortunately, your code contained errors during the make installation: 'SAVEPOINT' after 'SAVE_ERROR' in unreserved_keyword

Re: Assert failure on 'list_member_ptr(rel->joininfo, restrictinfo)'

2023-12-11 Thread Ashutosh Bapat
On Fri, Dec 8, 2023 at 11:24 PM Alexander Korotkov wrote: > > Hi, Ashutosh! > > On Fri, Dec 8, 2023 at 3:28 PM Ashutosh Bapat > wrote: > > I did some analysis of memory consumption by bitmapsets in such cases. > > [1] contains slides with the result of this analysis. The slides are > > crude and

Re: Memory consumption during partitionwise join planning

2023-12-11 Thread Ashutosh Bapat
On Thu, Jul 27, 2023 at 7:28 PM Ashutosh Bapat wrote: > The memory consumption is broken by the objects that consume memory > during planning. The second attached patch is used to measure breakup > by functionality . Here's a brief explanation of the rows in the > table. > > 1. Restrictlist trans

Re: ALTER COLUMN ... SET EXPRESSION to alter stored generated column's expression

2023-12-11 Thread Amul Sul
On Tue, Nov 28, 2023 at 5:40 PM Peter Eisentraut wrote: > On 23.11.23 15:13, Amul Sul wrote: > > The exact sequencing of this seems to be tricky. It's clear that we > > need to do it earlier than at the end. I also think it should be > > strictly after AT_PASS_ALTER_TYPE so that the

Re: Synchronizing slots from primary to standby

2023-12-11 Thread shveta malik
On Mon, Dec 11, 2023 at 1:22 PM Drouvot, Bertrand wrote: > > Hi, > > On 12/8/23 10:06 AM, Amit Kapila wrote: > > On Wed, Dec 6, 2023 at 4:53 PM shveta malik wrote: > >> > >> PFA v43, changes are: > >> > > > > I wanted to discuss 0003 patch about cascading standby's. It is not > > clear to me whet

Re: Synchronizing slots from primary to standby

2023-12-11 Thread Dilip Kumar
On Mon, Dec 11, 2023 at 2:21 PM shveta malik wrote: > > On Mon, Dec 11, 2023 at 1:47 PM Dilip Kumar wrote: > > > > On Fri, Dec 8, 2023 at 2:36 PM Amit Kapila wrote: > > > > > > On Wed, Dec 6, 2023 at 4:53 PM shveta malik > > > wrote: > > > > > > > > PFA v43, changes are: > > > > > > > > > > I

Re: Is WAL_DEBUG related code still relevant today?

2023-12-11 Thread Michael Paquier
On Fri, Dec 08, 2023 at 01:45:18PM +0900, Michael Paquier wrote: > Looks acceptable to me. Does somebody object to this removal? Hearing nothing, done that. -- Michael signature.asc Description: PGP signature

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-11 Thread Junwang Zhao
On Sat, Dec 9, 2023 at 7:38 PM Hannu Krosing wrote: > > Hi Junwang > > Please also see my presentation slides from last years PostgreSQL > Conference in Berlin (attached) I read through the slides, really promising ideas, it's will be great if we can get there at last. > > The main Idea is to ma

Re: unconstify()/unvolatize() vs g++/clang++

2023-12-11 Thread Thomas Munro
On Mon, Dec 11, 2023 at 10:17 PM Peter Eisentraut wrote: > If you are slightly more daring, you can write an alternative definition > in C++ using const_cast? Oh, yeah, right, that works for my case. I can't think of any reasons not to do this, but IANAC++L. 0001-Define-unconstify-and-unvolat

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-11 Thread Michael Paquier
On Mon, Dec 11, 2023 at 10:57:15AM +0900, Masahiko Sawada wrote: > IIUC we cannot create two same name functions with the same arguments > but a different return value type in the first place. It seems to me > to be an overkill to change such a design. Agreed to not touch the logictics of LookupFu

Re: GUC names in messages

2023-12-11 Thread Michael Paquier
On Mon, Dec 11, 2023 at 10:14:11AM +1100, Peter Smith wrote: > This v5* looks good to me, except it will need some further > modification if PeterE's suggestion [1] to keep quotes for the > MixedCase GUCs is adopted. -errdetail("The database cluster was initialized with CATALOG_VE

Re: Adding facility for injection points (or probe points?) for more advanced tests

2023-12-11 Thread Michael Paquier
On Mon, Dec 11, 2023 at 11:09:45AM +0530, Dilip Kumar wrote: > I haven't specifically done a review or testing of this patch, but I > have used this for testing the CLOG group update code with my > SLRU-specific changes and I found it quite helpful to test some of the > concurrent areas where you n

Re: Streaming I/O, vectored I/O (WIP)

2023-12-11 Thread Thomas Munro
On Mon, Dec 11, 2023 at 10:28 PM Heikki Linnakangas wrote: > On 11/12/2023 11:12, Thomas Munro wrote: > > 1. I eventually figured out how to generalise > > compute_remaining_iovec() (as I now call it) so that the existing > > pg_pwritev_with_retry() in file_utils.c could also use it, so that's >

Re: Synchronizing slots from primary to standby

2023-12-11 Thread shveta malik
On Thu, Dec 7, 2023 at 1:33 PM Peter Smith wrote: > > Hi. > > Here are my review comments for patch v43-0002. > Thanks for the feedback. I have addressed most of these in v45. Please find my response on a few which are pending or are not needed. > == > Commit message > > 1. > The nap time of

Re: backtrace_on_internal_error

2023-12-11 Thread Peter Eisentraut
On 08.12.23 19:14, Andres Freund wrote: FWIW, I did some analysis on aggregated logs on a larger number of machines, and it does look like that'd be a measurable increase in log volume. There are a few voluminous internal errors in core, but the bigger issue is extensions. They are typically much

Re: Streaming I/O, vectored I/O (WIP)

2023-12-11 Thread Heikki Linnakangas
On 11/12/2023 11:12, Thomas Munro wrote: 1. I eventually figured out how to generalise compute_remaining_iovec() (as I now call it) so that the existing pg_pwritev_with_retry() in file_utils.c could also use it, so that's now done in a patch of its own. In compute_remaining_iovec(): 'source'

Re: unconstify()/unvolatize() vs g++/clang++

2023-12-11 Thread Peter Eisentraut
On 11.12.23 01:42, Thomas Munro wrote: AFAICS you can't use unconstify()/unvolatize() in a static inline function in a .h file, or in a .cpp file, because __builtin_types_compatible_p is only available in C, not C++. Seems like a reasonable thing to want to be able to do, no? I'm not immediatel

Re: Streaming I/O, vectored I/O (WIP)

2023-12-11 Thread Thomas Munro
On Sat, Dec 9, 2023 at 10:23 PM Heikki Linnakangas wrote: > Ok, works for me. I finished up making a few more improvements: 1. I eventually figured out how to generalise compute_remaining_iovec() (as I now call it) so that the existing pg_pwritev_with_retry() in file_utils.c could also use it,

Re: Synchronizing slots from primary to standby

2023-12-11 Thread shveta malik
On Sun, Dec 10, 2023 at 4:33 PM Amit Kapila wrote: > > On Wed, Dec 6, 2023 at 4:53 PM shveta malik wrote: > > > > v43-002: > > > > Review comments on v43-0002: > = Thanks for the feedback Amit. Addressed these in v45. Please find my response on a few of these. > 1. > syn

Re: Synchronizing slots from primary to standby

2023-12-11 Thread shveta malik
On Mon, Dec 11, 2023 at 1:47 PM Dilip Kumar wrote: > > On Fri, Dec 8, 2023 at 2:36 PM Amit Kapila wrote: > > > > On Wed, Dec 6, 2023 at 4:53 PM shveta malik wrote: > > > > > > PFA v43, changes are: > > > > > > > I wanted to discuss 0003 patch about cascading standby's. It is not > > clear to me

Re: [Proposal] Add foreign-server health checks infrastructure

2023-12-11 Thread Shubham Khanna
On Mon, Dec 11, 2023 at 2:08 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Fujii-san, Tom, > > Thank you for giving a suggestion! PSA new version. > > > Regarding 0001 patch, on second thought, to me, it seems odd to expose > > a function that doesn't have anything to directly do with PostgreSQL > >

Re: Report planning memory in EXPLAIN ANALYZE

2023-12-11 Thread jian he
On Mon, Dec 4, 2023 at 3:24 PM Ashutosh Bapat wrote: > > On Fri, Dec 1, 2023 at 8:27 AM Andrei Lepikhov > wrote: > > > > On 30/11/2023 18:40, Alvaro Herrera wrote: > > > Well, SUMMARY is enabled by default with ANALYZE, and I'd rather not > > > have planner memory consumption displayed by default

Re: Synchronizing slots from primary to standby

2023-12-11 Thread Amit Kapila
On Mon, Dec 11, 2023 at 1:02 PM Peter Smith wrote: > > Here are some review comments for v44-0001 > > ~~~ > > 3. assign_standby_slot_names > > + if (!SplitIdentifierString(standby_slot_names_cpy, ',', &standby_slots)) > + { > + /* This should not happen if GUC checked check_standby_slot_names. */

Re: SQL:2011 application time

2023-12-11 Thread jian he
hi. some small issues diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c index e3ccf6c7f7..6781e55020 100644 --- a/src/backend/tcop/utility.c +++ b/src/backend/tcop/utility.c @@ -1560,7 +1560,7 @@ ProcessUtilitySlow(ParseState *pstate, true, /* check_rights */ true, /* ch

Re: Synchronizing slots from primary to standby

2023-12-11 Thread Dilip Kumar
On Fri, Dec 8, 2023 at 2:36 PM Amit Kapila wrote: > > On Wed, Dec 6, 2023 at 4:53 PM shveta malik wrote: > > > > PFA v43, changes are: > > > > I wanted to discuss 0003 patch about cascading standby's. It is not > clear to me whether we want to allow physical standbys to further wait > for cascadi