Re: Synchronizing slots from primary to standby

On Wed, Jul 26, 2023 at 10:31 AM shveta malik wrote: > > On Mon, Jul 24, 2023 at 9:00 AM Amit Kapila wrote: > > > > On Mon, Jul 24, 2023 at 8:03 AM Bharath Rupireddy > > wrote: > > > > > > Is having one (or a few more - not > > > necessarily one for each logical slot) worker for all logical

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

On Thu, Jul 27, 2023 at 6:46 AM Peter Smith wrote: > > Here are some review comments for v22-0003 > > == > > 1. ApplicationNameForTablesync > +/* > + * Determine the application_name for tablesync workers. > + * > + * Previously, the replication slot name was used as application_name. Since >

Re: PATCH: Add REINDEX tag to event triggers

I added a v2 patch for adding REINDEX to event triggers. The following has changed: 1. I fixed the docs to note that ddl_command_start is supported for REINDEX. Thanks, Michael! 2. I added Jian He's excellent partition table test and updated the expectations to include that regression test. What

Re: WAL Insertion Lock Improvements

On Tue, Jul 25, 2023 at 04:43:16PM +0900, Michael Paquier wrote: > We really need to do something in terms of documentation with > something like 0002, so I'll try to look at that next. I have applied a slightly-tweaked version of 0002 as of 66d86d4 to improve a bit the documentation of the area,

Re: PATCH: Add REINDEX tag to event triggers

Thank you for this, jian he! I will include it in the next patch version. P.S. Sorry for the double post. Sent from the wrong email address so I'm resending so the mailing list gets the email. My apologies! On Wed, Jul 26, 2023 at 4:30 AM jian he wrote: > On Wed, Jul 26, 2023 at 7:51 AM

Re: PATCH: Add REINDEX tag to event triggers

Thanks! I added the patch to the commitfest. TIL. > Hmm. What are the use cases you have in mind where you need to know the list of indexes listed by an event trigger like this? Direct use case is general index health and rebuild. We had a recent upgrade go poorly (our fault) that left a bunch

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

On Wed, Jul 26, 2023 at 12:07 PM Matthias van de Meent wrote: > We could cache the last accessed leaf page across amrescan operations > to reduce the number of index traversals needed when the join key of > the left side is highly (but not necessarily strictly) correllated. That sounds like

Re: Retiring is_pushed_down

On Tue, Jul 25, 2023 at 3:39 PM Richard Guo wrote: > * This patch calculates the outer join relids that are being formed > generally in this way: > > bms_difference(joinrelids, bms_union(outerrelids, innerrelids)) > > Of course this can only be used after the outer join relids has been >

Re: Performance degradation on concurrent COPY into a single relation in PG16.

On Wed, 26 Jul 2023 at 03:50, Andres Freund wrote: > On 2023-07-25 23:37:08 +1200, David Rowley wrote: > > On Tue, 25 Jul 2023 at 17:34, Andres Freund wrote: > > I've not really studied the fix_COPY_DEFAULT.patch patch. Is there a > > reason to delay committing that? It would be good to

Re: pg_usleep for multisecond delays

At Wed, 26 Jul 2023 16:41:25 -0700, Nathan Bossart wrote in > On Mon, Mar 13, 2023 at 02:16:31PM -0700, Nathan Bossart wrote: > I started on the former approach (work-in-progress patch attached), but I > figured I'd ask whether folks are alright with this before I spend more > time on it. Many

Re: [PATCH] Check more invariants during syscache initialization

On Wed, Jul 26, 2023 at 07:01:11PM +0300, Aleksander Alekseev wrote: > Hi, > > > Shouldn't these be calling `OidIsValid(…)`, not comparing directly to > > `InvalidOid`? > > They should, thanks. Here is the updated patch. I made sure there are > no others != InvalidOid checks in syscache.c and

Re: [PATCH] Check more invariants during syscache initialization

Hi, > On Jul 27, 2023, at 09:05, Michael Paquier wrote: > > One reason is that this increases the odds of > conflicts when backpatching on a stable branch. Agree. We could suggest to use OidIsValid() for new patches during review. Zhang Mingli https://www.hashdata.xyz

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

Here are some review comments for v22-0003 == 1. ApplicationNameForTablesync +/* + * Determine the application_name for tablesync workers. + * + * Previously, the replication slot name was used as application_name. Since + * it's possible to reuse tablesync workers now, a tablesync worker

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

Here are some review comments for v22-0002 == 1. General - errmsg AFAIK, the errmsg part does not need to be enclosed by extra parentheses. e.g. BEFORE ereport(LOG, (errmsg("logical replication table synchronization worker for subscription \"%s\" has finished", MySubscription->name)));

Re: [PATCH] Check more invariants during syscache initialization

On Wed, Jul 26, 2023 at 03:28:55PM +0100, Dagfinn Ilmari Mannsåker wrote: > Shouldn't these be calling `OidIsValid(…)`, not comparing directly to > `InvalidOid`? I think that they should use OidIsValid() on correctness ground, and that's the style I prefer. Now, I don't think that these are

Re: Improve pg_stat_statements by making jumble handle savepoint names better

On Wed, Jul 26, 2023 at 07:53:02AM +0900, Michael Paquier wrote: > I think that I'm OK with your proposal as savepoint names are in > defined places in these queries (contrary to some of the craziness > with BEGIN and the node structure of TransactionStmt, for example). > > Has somebody an

Re: Performance degradation on concurrent COPY into a single relation in PG16.

> On 2023-07-25 23:37:08 +1200, David Rowley wrote: > > On Tue, 25 Jul 2023 at 17:34, Andres Freund wrote: > > > HEAD: 812.690 > > > > > > your patch: 821.354 > > > > > > strtoint from 8692f6644e7: 824.543 > > > > > > strtoint from 6b423ec677d^:

Re: psql: Could we get "-- " prefixing on the **** QUERY **** outputs? (ECHO_HIDDEN)

On Wed, Jul 26, 2023 at 02:39:25PM -0700, Nathan Bossart wrote: > Great. I spent some time on the commit message in v4. I plan to commit > this shortly. Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: pg_usleep for multisecond delays

On Mon, Mar 13, 2023 at 02:16:31PM -0700, Nathan Bossart wrote: > At the moment, I'm thinking about either removing the check_interrupts > function pointer argument altogether or making it optional for code paths > where we might not want any interrupt handling to run. In the former > approach,

Re: There should be a way to use the force flag when restoring databases

On Sun, Jul 23, 2023 at 6:09 AM Ahmed Ibrahim wrote: > > Hi everyone, > > I have been working on this. This is a proposed patch for it so we have a > force option for DROPping the database. > > I'd appreciate it if anyone can review. Hi Ahmed, Thanks for working on this patch! + +int

Re: POC, WIP: OR-clause support for indexes

Hi! On 26.07.2023 02:47, Peter Geoghegan wrote: On Thu, Jun 29, 2023 at 2:32 AM Alena Rybakina wrote: Hi! I'm sorry I didn't answer you right away, I was too busy with work. Same for me, this time. I was busy working on my patch, which I finally posted yesterday. I'm glad to hear it, I've

Re: psql: Could we get "-- " prefixing on the **** QUERY **** outputs? (ECHO_HIDDEN)

On Wed, Jul 26, 2023 at 08:06:37AM +0200, Pavel Stehule wrote: > st 26. 7. 2023 v 6:22 odesílatel Nathan Bossart > napsal: >> Barring additional feedback, I think this is ready for commit. >> >> > +1 Great. I spent some time on the commit message in v4. I plan to commit this shortly. --

Re: Obsolete reference to pg_relation in comment

On Wed, Jul 26, 2023 at 05:14:08PM -0400, Tom Lane wrote: > I think we should reword this to just generically claim that holding > the Relation reference open for the whole transaction reduces overhead. WFM -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Obsolete reference to pg_relation in comment

Nathan Bossart writes: > On Wed, Jul 26, 2023 at 06:48:51PM +0100, Dagfinn Ilmari Mannsåker wrote: >> * All accesses to pg_largeobject and its index make use of a single >> Relation >> - * reference, so that we only need to open pg_relation once per transaction. >> + * reference, so that we

Re: incremental-checkopints

On Wed, Jul 26, 2023 at 9:54 PM Matthias van de Meent wrote: > > Then you ignore the max_wal_size GUC as PostgreSQL so often already > does. At least, it doesn't do what I expect it to do at face value - > limit the size of the WAL directory to the given size. That would require stopping any new

Re: Obsolete reference to pg_relation in comment

Okay, now looking at the patch... On Wed, Jul 26, 2023 at 06:48:51PM +0100, Dagfinn Ilmari Mannsåker wrote: > * All accesses to pg_largeobject and its index make use of a single Relation > - * reference, so that we only need to open pg_relation once per transaction. > + * reference, so that we

Re: Row pattern recognition

> I am going to add this thread to CommitFest and plan to add both of > you as reviewers. Thanks in advance. Done. https://commitfest.postgresql.org/44/4460/ Best reagards, -- Tatsuo Ishii SRA OSS LLC English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp

Re: incremental-checkopints

On Wed, 26 Jul 2023 at 20:58, Tomas Vondra wrote: > > > > On 7/26/23 15:16, Matthias van de Meent wrote: > > On Wed, 26 Jul 2023 at 14:41, Alvaro Herrera > > wrote: > >> > >> Hello > >> > >> On 2023-Jul-26, Thomas wen wrote: > >> > >>> Hi Hackes: I found this page : > >>>

Re: incremental-checkopints

Starting from increments checkpoint is approaching the problem from the wrong end. What you actually want is Atomic Disk Writes which will allow turning off full_page_writes . Without this you really can not do incremental checkpoints efficiently as checkpoints are currently what is used to

Re: Obsolete reference to pg_relation in comment

On Wed, Jul 26, 2023 at 11:53:06AM -0700, Nathan Bossart wrote: > On Wed, Jul 26, 2023 at 06:48:51PM +0100, Dagfinn Ilmari Mannsåker wrote: >> Triggered by a discussion on IRC, I noticed that there's a stray >> reference to pg_relation in a comment that was added long after it was >> renamed to

Re: incremental-checkopints

On 7/26/23 15:16, Matthias van de Meent wrote: > On Wed, 26 Jul 2023 at 14:41, Alvaro Herrera wrote: >> >> Hello >> >> On 2023-Jul-26, Thomas wen wrote: >> >>> Hi Hackes: I found this page : >>> https://pgsql-hackers.postgresql.narkive.com/cMxBwq65/incremental-checkopints,PostgreSQL >>> no

Re: Obsolete reference to pg_relation in comment

On Wed, Jul 26, 2023 at 06:48:51PM +0100, Dagfinn Ilmari Mannsåker wrote: > Triggered by a discussion on IRC, I noticed that there's a stray > reference to pg_relation in a comment that was added long after it was > renamed to pg_class. Here's a patch to bring that up to speed. > pg_relation was

Obsolete reference to pg_relation in comment

Hi hackers, Triggered by a discussion on IRC, I noticed that there's a stray reference to pg_relation in a comment that was added long after it was renamed to pg_class. Here's a patch to bring that up to speed. - ilmari >From e395f8cb293f674f45eb3847534de07c7124e738 Mon Sep 17 00:00:00 2001

Re: [PoC] Federated Authn/z with OAUTHBEARER

On Tue, Jul 18, 2023 at 4:04 PM Thomas Munro wrote: > On Tue, Jul 18, 2023 at 11:55 AM Jacob Champion > wrote: > +1 for EV_RECEIPT ("just tell me about errors, don't drain any > events"). Sounds good. > While comparing the cousin OSs' man pages just now, I noticed that > it's not only macOS

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

On Wed, 26 Jul 2023 at 15:42, Peter Geoghegan wrote: > > On Wed, Jul 26, 2023 at 5:29 AM Matthias van de Meent > wrote: > > Considering that it caches/reuses the page across SAOP operations, can > > (or does) this also improve performance for index scans on the outer > > side of a join if the

Re: [PATCH] Check more invariants during syscache initialization

Hi, > Shouldn't these be calling `OidIsValid(…)`, not comparing directly to > `InvalidOid`? They should, thanks. Here is the updated patch. I made sure there are no others != InvalidOid checks in syscache.c and catcache.c which are affected by my patch. I didn't change any other files since

Re: POC: Extension for adding distributed tracing - pg_tracing

I've initially thought of sending the spans from PostgreSQL since this is the usual behavior of tracing libraries. However, this created a lot potential issues: - Protocol support and differences between trace collectors. OpenTelemetry seems to use gRPC, others are using http and those will

Re: [PATCH] Check more invariants during syscache initialization

Aleksander Alekseev writes: > Hi Zhang, > >> That remind me to have a look other codes, and a grep search `oid != 0` show >> there are several files using old != 0. >> >> ``` >> .//src/bin/pg_resetwal/pg_resetwal.c: if (set_oid != 0) >> .//src/bin/pg_resetwal/pg_resetwal.c: if (set_oid != 0) >>

Re: Question about use_physical_tlist() which is applied on Scan path

I had the same question recently. In addition, I looked at the results of tpch which scale factor is 1 ran on postgres REL_15_STABLE and observed no performance improvement from physical tlist. To be specific, I run two versions of tpch, one with physical tlist enabled and one with physical tlist

[May be a bug] double free or corruption

Hi community, I use the PostgreSQL 17devel on x86_64-pc-linux-gnu, compiled by gcc (GCC) 13.1.1 20230614 (Red Hat 13.1.1-4), 64-bit.(Fedora Linux) And I use the gdb to debug the postgres, just test the pg_ctl. As you can see:

Re: POC: Extension for adding distributed tracing - pg_tracing

Nikita, > This patch looks very interesting, I'm working on the same subject too. But > I've used > another approach - I'm using C wrapper for C++ API library from > OpenTelemetry, and > handle span storage and output to this library. There are some nuances > though, but it > is possible. Have

Re: cataloguing NOT NULL constraints

On 2023-Jul-26, Alvaro Herrera wrote: > On 2023-Jul-26, Dean Rasheed wrote: > > > The new \d+ output certainly makes testing and reviewing easier, > > though I do understand people's concerns that this may make the output > > significantly longer in many real-world cases. > > Yeah, at this

Re: [PATCH] Check more invariants during syscache initialization

Hi Zhang, > That remind me to have a look other codes, and a grep search `oid != 0` show > there are several files using old != 0. > > ``` > .//src/bin/pg_resetwal/pg_resetwal.c: if (set_oid != 0) > .//src/bin/pg_resetwal/pg_resetwal.c: if (set_oid != 0) > .//src/bin/pg_dump/pg_backup_tar.c: if

Re: WaitForOlderSnapshots in DETACH PARTITION causes deadlocks

On 2023-Jul-25, Michael Paquier wrote: > On Mon, Jul 24, 2023 at 07:40:04PM +, Imseih (AWS), Sami wrote: > > WaitForOlderSnapshots is used here to ensure that snapshots older than > > the start of the ALTER TABLE DETACH CONCURRENTLY are completely removed > > to guarantee consistency, however

Re: Remove unused fields in ReorderBufferTupleBuf

Hi, > Here is the corrected patch. I added it to the nearest CF [1]. I played a bit more with the patch. There was an idea to make ReorderBufferTupleBufData an opaque structure known only within reorderbyffer.c but it turned out that replication/logical/decode.c accesses it directly so I

Re: cataloguing NOT NULL constraints

Thanks for spending so much time with this patch -- really appreciated. On 2023-Jul-26, Dean Rasheed wrote: > On Tue, 25 Jul 2023 at 13:36, Alvaro Herrera wrote: > > > > Okay then, I've made these show up in the footer of \d+. This is in > > patch 0003 here. Please let me know what do you

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

On Wed, Jul 26, 2023 at 5:29 AM Matthias van de Meent wrote: > Considering that it caches/reuses the page across SAOP operations, can > (or does) this also improve performance for index scans on the outer > side of a join if the order of join columns matches the order of the > index? It doesn't

Re: [PATCH] Check more invariants during syscache initialization

HI, > On Jul 26, 2023, at 20:50, Aleksander Alekseev > wrote: > > Hi Michael, > >> That was more a question. I was wondering if it was something you've >> noticed while working on a different patch because you somewhat >> assigned incorrect values in the syscache array, but it looks like you

Re: Remove unused fields in ReorderBufferTupleBuf

Hi, > I'm afraid it's a bit incomplete: > > ``` > ../src/backend/replication/logical/reorderbuffer.c: In function > ‘ReorderBufferGetTupleBuf’: > ../src/backend/replication/logical/reorderbuffer.c:565:14: error: > ‘ReorderBufferTupleBuf’ has no member named ‘alloc_tuple_size’ > 565 |

Re: incremental-checkopints

On Wed, 26 Jul 2023 at 14:41, Alvaro Herrera wrote: > > Hello > > On 2023-Jul-26, Thomas wen wrote: > > > Hi Hackes: I found this page : > > https://pgsql-hackers.postgresql.narkive.com/cMxBwq65/incremental-checkopints,PostgreSQL > > no incremental checkpoints have been implemented so far. When

Re: cataloguing NOT NULL constraints

On Tue, 25 Jul 2023 at 13:36, Alvaro Herrera wrote: > > Okay then, I've made these show up in the footer of \d+. This is in > patch 0003 here. Please let me know what do you think of the regression > changes. > The new \d+ output certainly makes testing and reviewing easier, though I do

Re: Remove unused fields in ReorderBufferTupleBuf

Hi, Thanks for the patch. > However, node and alloc_tuple_size are not used at all. It seems an > oversight in commit a4ccc1cef5a, which introduced the generation > context and used it in logical decoding. I think we can remove them > (only on HEAD). I've attached the patch. I'm afraid it's a

Re: [PATCH] Check more invariants during syscache initialization

Hi Michael, > That was more a question. I was wondering if it was something you've > noticed while working on a different patch because you somewhat > assigned incorrect values in the syscache array, but it looks like you > have noticed that while scanning the code. Oh, got it. That's correct.

Re: incremental-checkopints

Hello On 2023-Jul-26, Thomas wen wrote: > Hi Hackes: I found this page : > https://pgsql-hackers.postgresql.narkive.com/cMxBwq65/incremental-checkopints,PostgreSQL > no incremental checkpoints have been implemented so far. When a > checkpoint is triggered, the performance jitter of PostgreSQL

Re: incremental-checkopints

On 7/26/23 09:21, Thomas wen wrote: > Hi Hackes:  I found this page : > https://pgsql-hackers.postgresql.narkive.com/cMxBwq65/incremental-checkopints,PostgreSQL > > >  no incremental checkpoints have been

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

On Tue, 25 Jul 2023 at 03:34, Peter Geoghegan wrote: > > I've been working on a variety of improvements to nbtree's native > ScalarArrayOpExpr execution. This builds on Tom's work in commit > 9e8da0f7. Cool. > Attached patch is still at the prototype stage. I'm posting it as v1 a > little

Re: Synchronizing slots from primary to standby

On Mon, Jul 24, 2023 at 8:03 AM Bharath Rupireddy wrote: > > On Fri, Jul 21, 2023 at 5:16 PM shveta malik wrote: > > > > Thanks Bharat for letting us know. It is okay to split the patch, it > > may definitely help to understand the modules better but shall we take > > a step back and try to

Re: Row pattern recognition

Attached is the v3 patch. In this patch following changes are made. (1) I completely changed the pattern matching engine so that it performs backtracking. Now the engine evaluates all pattern elements defined in PATTER against each row, saving matched pattern variables in a string per row. For

Re: POC: Extension for adding distributed tracing - pg_tracing

Hi! This patch looks very interesting, I'm working on the same subject too. But I've used another approach - I'm using C wrapper for C++ API library from OpenTelemetry, and handle span storage and output to this library. There are some nuances though, but it is possible. Have you tried to use

Re: In Postgres 16 BETA, should the ParseNamespaceItem have the same index as it's RangeTableEntry?

Hello, On Fri, Jul 21, 2023 at 5:05 AM Farias de Oliveira wrote: > > Hello, > > Thank you for the help guys and I'm so sorry for my late response. Indeed, > the error relies on the ResultRelInfo. In GetResultRTEPermissionInfo() > function, it does a checking on the

incremental-checkopints

Hi Hackes: I found this page : https://pgsql-hackers.postgresql.narkive.com/cMxBwq65/incremental-checkopints,PostgreSQL no incremental checkpoints have been implemented so far. When a checkpoint is triggered, the performance jitter of PostgreSQL is very noticeable. I think incremental

Re: PATCH: Add REINDEX tag to event triggers

On Wed, Jul 26, 2023 at 7:51 AM Michael Paquier wrote: > > On Tue, Jul 25, 2023 at 04:34:47PM +0800, jian he wrote: > > so T_ReindexStmt should only be in ProcessUtilitySlow, if you want > > to create an event trigger on reindex? > > > > regression tests work fine. I even play with partitions.

Re: Question about use_physical_tlist() which is applied on Scan path

On 2023-Jul-26, Jian Guo wrote: > It looks the columns besides `ps_supplycost` and `ps_availqty` are not > necessary, but fetched from tuples all at once. For the row-based > storage such as heap, it looks fine, but for column-based storage, it > would result into unnecessary overhead and impact

Re: remaining sql/json patches

On Fri, Jul 21, 2023 at 7:33 PM Amit Langote wrote: > On Fri, Jul 21, 2023 at 1:02 AM Alvaro Herrera > wrote: > > On 2023-Jul-21, Amit Langote wrote: > > > > > I’m thinking of pushing 0001 and 0002 tomorrow barring objections. > > > > 0001 looks reasonable to me. I think you asked whether to

Re: [PATCH] Add loongarch native checksum implementation.

On Wed, Jul 26, 2023 at 8:25 AM YANG Xudong wrote: > > Many thanks to huchangqi. Now we have loongarch64 support for both old > world ABI and new world ABI on the buildfarm! Glad to hear it! On Wed, Jul 26, 2023 at 10:16 AM Michael Paquier wrote: > > The performance numbers presented upthread

Re: logical decoding and replication of sequences, take 2

On Wed, Jul 26, 2023 at 9:37 AM Amit Kapila wrote: > > On Tue, Jul 25, 2023 at 5:29 PM Tomas Vondra > wrote: > > > > On 7/25/23 08:28, Amit Kapila wrote: > > > On Mon, Jul 24, 2023 at 9:32 PM Tomas Vondra > > > wrote: > > >> > > >> On 7/24/23 12:40, Amit Kapila wrote: > > >>> On Wed, Jul 5,

Re: [feature]COPY FROM enable FORCE_NULL/FORCE_NOT_NULL on all columns

HI, > I've looked at this patch and it looks mostly fine, though I do not > intend to commit it myself; perhaps Andrew will. HI, Amit, thanks for review. > > A few minor things to improve: > > + If * is specified, it will be applied in all > columns. > ... > + If * is specified, it

Question about use_physical_tlist() which is applied on Scan path

Hi hackers, I have a question about `use_physical_tlist()` which is applied in `create_scan_plan()`: ``` if (flags == CP_IGNORE_TLIST) { tlist = NULL; } else if (use_physical_tlist(root, best_path, flags)) { if (best_path->pathtype == T_IndexOnlyScan) { /* For index-only scan, the

Re: [feature]COPY FROM enable FORCE_NULL/FORCE_NOT_NULL on all columns

Hello, On Thu, Jul 20, 2023 at 4:06 PM Zhang Mingli wrote: > > Hi, > > On Jul 7, 2023 at 18:00 +0800, Damir Belyalov , wrote: > > > V2 patch still have some errors when apply file doc/src/sgml/ref/copy.sgml, > rebased and fixed it in V3 path. > Thanks a lot for review. > > I have updated

Re: psql: Could we get "-- " prefixing on the **** QUERY **** outputs? (ECHO_HIDDEN)

st 26. 7. 2023 v 6:22 odesílatel Nathan Bossart napsal: > I took a look at this patch and changed a couple things: > > * I made a similar adjustment to a few lines that seem to have been >missed. > * I removed a couple of asterisks from the adjusted lines in order to >maintain the