Re: logicalrep_message_type throws an error

2023-07-21 Thread Amit Kapila
On Fri, Jul 21, 2023 at 6:28 AM Masahiko Sawada wrote: > > On Fri, Jul 21, 2023 at 1:39 AM Ashutosh Bapat > wrote: > > > > On Thu, Jul 20, 2023 at 9:11 AM Amit Kapila wrote: > > > > > > > > Okay, changed it accordingly. > > > > > > > > > > oops, forgot to attach the patch. > > > > > > > WFM > >

Re: Row pattern recognition

2023-07-21 Thread Vik Fearing
On 7/22/23 03:11, Tatsuo Ishii wrote: Maybe that can help clarify the design? It feels like it'll need to eventually be a "real" function that operates on the window state, even if it doesn't support all of the possible complexities in v1. Unfortunately an window function can not call other

Re: MERGE ... RETURNING

2023-07-21 Thread Dean Rasheed
On Mon, 17 Jul 2023 at 20:43, Jeff Davis wrote: > > > > Maybe instead of a function it could be a special table reference > > > like: > > > > > > MERGE ... RETURNING MERGE.action, MERGE.action_number, id, val? > > > > The benefits are: > > 1. It is naturally constrained to the right context. It

Re: Row pattern recognition

2023-07-21 Thread Tatsuo Ishii
>> One small concern is how to convert pattern variables to regex >> expression which our regex enginer understands. Suppose, >> >> PATTERN UP+ >> >> Currently I convert "UP+" to "U+" so that it can be fed to the regexp >> engine. In order to do that, we need to know which part of the pattern >>

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2023-07-21 Thread Thomas Munro
This is a frustrating thread, because despite the last patch solving most of the problems we discussed, it doesn't address the low-level-backup procedure in a nice way. We'd have to tell users they have to flock that file, or add a new step "pg_controldata --raw > pg_control", which seems weird

Re: Row pattern recognition

2023-07-21 Thread Vik Fearing
On 7/22/23 01:14, Jacob Champion wrote: On 7/20/23 17:07, Vik Fearing wrote: On 7/21/23 01:36, Jacob Champion wrote: (I've attached two failing tests against v2, to hopefully better illustrate, along with what I_think_ should be the correct results.) Almost. You are matching 07-01-2023 but

Re: Row pattern recognition

2023-07-21 Thread Jacob Champion
On 7/20/23 23:16, Tatsuo Ishii wrote: > I don't know at this point. I think context-free is not enough to be > repsented in Bison. The grammer also needs to be LALR(1). Moreover, > adding the grammer to existing parser may generate shift/reduce > errors. Ah. It's been too long since my compilers

Re: Row pattern recognition

2023-07-21 Thread Jacob Champion
On 7/20/23 17:07, Vik Fearing wrote: > On 7/21/23 01:36, Jacob Champion wrote: >> (I've attached two failing tests against v2, to hopefully better >> illustrate, along with what I_think_ should be the correct results.) > > Almost. You are matching 07-01-2023 but the condition is "price > 100".

Re: Fix search_path for all maintenance commands

2023-07-21 Thread Jeff Davis
On Mon, 2023-07-17 at 12:16 -0700, Jeff Davis wrote: > Based on feedback, I plan to commit soon. Attached is a new version. Changes: * Also switch the search_path during CREATE MATERIALIZED VIEW, so that it's consistent with REFRESH. As a part of this change, I slightly reordered things in

Re: Show WAL write and fsync stats in pg_stat_io

2023-07-21 Thread Melanie Plageman
On Wed, Jun 28, 2023 at 6:09 AM Nazir Bilal Yavuz wrote: > This is a WIP patch to add WAL write and fsync stats to pg_stat_io > view. Thanks for working on this! I have some feedback on the content of the patch as well as some items that I feel are missing. I think it would be good to count WAL

Re: Use of additional index columns in rows filtering

2023-07-21 Thread Peter Geoghegan
On Fri, Jul 21, 2023 at 4:52 AM Tomas Vondra wrote: > > (Actually, I'm glossing over a lot. The MDAM paper describes > > techniques that'd make even the really challenging cases safe, through > > a process of converting quals from conjunctive normal form into > > disjunctive normal form. This is

Re: MERGE ... RETURNING

2023-07-21 Thread Jeff Davis
On Thu, 2023-07-20 at 23:19 -0700, Gurjeet Singh wrote: > Plus, if we were able to make it work as SQL syntax, it's very likely > we can use the same technique to implement BEFORE and AFTER behaviour > in UPDATE ... RETURNING that the old thread could not accomplish a > decade ago. To clarify, I

Re: Issue in _bt_getrootheight

2023-07-21 Thread Tom Lane
Gurjeet Singh writes: > Please see attached the patch that does this. Let me know if this patch helps. I don't like this patch one bit, because it adds a lot of overhead (i.e., an extra index_open/close cycle for every btree index in every query) to support a tiny minority use-case. How come we

Re: Use COPY for populating all pgbench tables

2023-07-21 Thread Tristan Partin
On Thu Jul 20, 2023 at 9:14 PM CDT, Michael Paquier wrote: Attached is a v7, with these tests (should be a patch on its own but I'm lazy to split this morning) and some more adjustments that I have done while going through the patch. What do you think? v7 looks good from my perspective.

Re: Support worker_spi to execute the function dynamically.

2023-07-21 Thread Bharath Rupireddy
On Fri, Jul 21, 2023 at 4:05 PM Masahiro Ikeda wrote: > > > In SQL tests, I ensured worker_spi doesn't start static bg workers by > > setting worker_spi.total_workers = 0. Again, all of this is not > > necessary, but it will be a very good example for someone writing > > extensions and play

Re: Assert failure on bms_equal(child_joinrel->relids, child_joinrelids)

2023-07-21 Thread Tom Lane
Richard Guo writes: > This should be an oversight in 9df8f903. It seems that the new added > function add_outer_joins_to_relids() does not cope well with child > joins. The 'input_relids' for a child join is the relid sets of child > rels while 'othersj->min_xxxhand' refers to relids of parent

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-07-21 Thread Etsuro Fujita
Hi Richard, On Sun, Jun 25, 2023 at 3:05 PM Richard Guo wrote: > On Wed, Jun 14, 2023 at 2:49 PM Etsuro Fujita wrote: >> On Mon, Jun 5, 2023 at 10:19 PM Etsuro Fujita >> wrote: >> > To avoid this issue, I am wondering if we should modify >> > add_paths_to_joinrel() in back branches so that it

Re: Avoid unused value (src/fe_utils/print.c)

2023-07-21 Thread Karina Litskevich
On Wed, Jul 12, 2023 at 1:46 AM Ranier Vilela wrote: > As there is consensus to keep the no-op assignment, > I will go ahead and reject the patch. > In your patch you suggest removing two assignments, and we only have consensus to keep one of them. The other one is an obvious no-op. I

Re: Use of additional index columns in rows filtering

2023-07-21 Thread Tomas Vondra
On 7/21/23 05:32, Peter Geoghegan wrote: > On Thu, Jul 20, 2023 at 4:35 AM Tomas Vondra > wrote: >> I think the SAOP patch may need to be much more careful about this, but >> for this patch it's simpler because it doesn't really change any of the >> index internals, or the indexscan in general. >

Re: Synchronizing slots from primary to standby

2023-07-21 Thread shveta malik
On Fri, Jul 21, 2023 at 11:36 AM Bharath Rupireddy wrote: > > On Thu, Jul 20, 2023 at 5:05 PM shveta malik wrote: > > > > On Fri, Jun 16, 2023 at 3:26 PM Amit Kapila wrote: > > > > > > On Mon, Apr 17, 2023 at 7:37 PM Drouvot, Bertrand > > > wrote: > > > > > > > > > 3. As mentioned in the

Re: Support worker_spi to execute the function dynamically.

2023-07-21 Thread Masahiro Ikeda
Hi, On 2023-07-20 18:39, Bharath Rupireddy wrote: On Thu, Jul 20, 2023 at 2:59 PM Michael Paquier wrote: On Thu, Jul 20, 2023 at 05:54:55PM +0900, Masahiro Ikeda wrote: > Yes, you're right. When I tried using worker_spi to test wait event, > I found the behavior. And thanks a lot for your

Assert failure on bms_equal(child_joinrel->relids, child_joinrelids)

2023-07-21 Thread Richard Guo
This Assert failure can be reproduced with the query below. create table part_tbl (a integer) partition by range (a); create table part_tbl1 partition of part_tbl for values from (0) to (100); set enable_partitionwise_join to on; explain (costs off) select * from part_tbl t1 left join

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-21 Thread Melih Mutlu
Peter Smith , 21 Tem 2023 Cum, 12:48 tarihinde şunu yazdı: > On Fri, Jul 21, 2023 at 5:24 PM Amit Kapila > wrote: > > > > On Fri, Jul 21, 2023 at 12:05 PM Peter Smith > wrote: > > > > > > On Fri, Jul 21, 2023 at 3:39 PM Amit Kapila > wrote: > > > > > > > > > The other thing I noticed is that

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-21 Thread Peter Smith
On Fri, Jul 21, 2023 at 5:24 PM Amit Kapila wrote: > > On Fri, Jul 21, 2023 at 12:05 PM Peter Smith wrote: > > > > On Fri, Jul 21, 2023 at 3:39 PM Amit Kapila wrote: > > > > > > The other thing I noticed is that we > > > don't seem to be consistent in naming functions in these files. For > > >

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-21 Thread Melih Mutlu
Amit Kapila , 21 Tem 2023 Cum, 08:39 tarihinde şunu yazdı: > On Fri, Jul 21, 2023 at 7:30 AM Peter Smith wrote: > How about SetupLogRepWorker? The other thing I noticed is that we > don't seem to be consistent in naming functions in these files. For > example, shall we make all exposed functions

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-07-21 Thread Hayato Kuroda (Fujitsu)
Dear hackers, > Based on the above, we are considering that we delay the timing of shutdown > for > logical walsenders. The preliminary workflow is: > > 1. When logical walsenders receives siginal from checkpointer, it consumes all >of WAL records, change its state into

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-21 Thread Amit Kapila
On Fri, Jul 21, 2023 at 12:05 PM Peter Smith wrote: > > On Fri, Jul 21, 2023 at 3:39 PM Amit Kapila wrote: > > > > On Fri, Jul 21, 2023 at 7:30 AM Peter Smith wrote: > > > > > > ~~~ > > > > > > 2. StartLogRepWorker > > > > > > /* Common function to start the leader apply or tablesync worker. */

Re: Switching XLog source from archive to streaming when primary available

2023-07-21 Thread Bharath Rupireddy
On Tue, Apr 25, 2023 at 9:27 PM Bharath Rupireddy wrote: > > > Needed a rebase. I'm attaching the v11 patch for further review. > > Needed a rebase, so attaching the v12 patch. I word-smithed comments > and docs a bit. Needed a rebase. I'm attaching the v13 patch for further consideration. --

Re: Atomic ops for unlogged LSN

2023-07-21 Thread Bharath Rupireddy
On Fri, Jul 21, 2023 at 4:43 AM John Morris wrote: > > Based on your feedback, I’ve updated the patch with additional comments. > > Explain the two cases when writing to the control file, and > a bit more emphasis on unloggedLSNs not being valid after a crash. Given that the callers already have

Re: table_open/table_close with different lock mode

2023-07-21 Thread Junwang Zhao
On Fri, Jul 21, 2023 at 2:57 PM Gurjeet Singh wrote: > > On Thu, Jul 20, 2023 at 11:38 PM Junwang Zhao wrote: > > > > On Fri, Jul 21, 2023 at 2:26 PM Michael Paquier wrote: > > > > > > On Fri, Jul 21, 2023 at 02:05:56PM +0800, Junwang Zhao wrote: > > > > I noticed there are some places calling

Re: table_open/table_close with different lock mode

2023-07-21 Thread Gurjeet Singh
On Thu, Jul 20, 2023 at 11:38 PM Junwang Zhao wrote: > > On Fri, Jul 21, 2023 at 2:26 PM Michael Paquier wrote: > > > > On Fri, Jul 21, 2023 at 02:05:56PM +0800, Junwang Zhao wrote: > > > I noticed there are some places calling table_open with > > > RowExclusiveLock but table_close with NoLock,

Re: Support worker_spi to execute the function dynamically.

2023-07-21 Thread Bharath Rupireddy
On Fri, Jul 21, 2023 at 11:54 AM Michael Paquier wrote: > > On Fri, Jul 21, 2023 at 11:24:08AM +0530, Bharath Rupireddy wrote: > > Okay. Here's a quick patch for adding TAP tests to the worker_spi > > module. We can change it to taste. > > What do you think if we removed completely the sql/ test,

Re: table_open/table_close with different lock mode

2023-07-21 Thread Junwang Zhao
On Fri, Jul 21, 2023 at 2:26 PM Michael Paquier wrote: > > On Fri, Jul 21, 2023 at 02:05:56PM +0800, Junwang Zhao wrote: > > I noticed there are some places calling table_open with > > RowExclusiveLock but table_close with NoLock, like in function > > toast_save_datum. > > > > Can anybody explain

Re: MERGE ... RETURNING

2023-07-21 Thread Gurjeet Singh
On Mon, Jul 17, 2023 at 12:43 PM Jeff Davis wrote: > > On Fri, 2023-07-14 at 09:55 +0100, Dean Rasheed wrote: > > I found a 10-year-old thread discussing adding support for OLD/NEW to > > RETURNING [1], but it doesn't look like anything close to a > > committable solution was developed, or even a

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-21 Thread Peter Smith
On Fri, Jul 21, 2023 at 3:39 PM Amit Kapila wrote: > > On Fri, Jul 21, 2023 at 7:30 AM Peter Smith wrote: > > > > ~~~ > > > > 2. StartLogRepWorker > > > > /* Common function to start the leader apply or tablesync worker. */ > > void > > StartLogRepWorker(int worker_slot) > > { > > /* Attach to

Re: table_open/table_close with different lock mode

2023-07-21 Thread Michael Paquier
On Fri, Jul 21, 2023 at 02:05:56PM +0800, Junwang Zhao wrote: > I noticed there are some places calling table_open with > RowExclusiveLock but table_close with NoLock, like in function > toast_save_datum. > > Can anybody explain the underlying logic, thanks in advance. This rings a bell. This

Re: Support worker_spi to execute the function dynamically.

2023-07-21 Thread Michael Paquier
On Fri, Jul 21, 2023 at 11:24:08AM +0530, Bharath Rupireddy wrote: > Okay. Here's a quick patch for adding TAP tests to the worker_spi > module. We can change it to taste. What do you think if we removed completely the sql/ test, moving it to TAP so as we have only one cluster set up when running

Re: MERGE ... RETURNING

2023-07-21 Thread Gurjeet Singh
On Fri, Jul 14, 2023 at 1:55 AM Dean Rasheed wrote: > > On Thu, 13 Jul 2023 at 20:14, Jeff Davis wrote: > > > > On Thu, 2023-07-13 at 18:01 +0100, Dean Rasheed wrote: > > > For some use cases, I can imagine allowing OLD/NEW.colname would mean > > > you wouldn't need pg_merge_action() (if the

Re: Row pattern recognition

2023-07-21 Thread Tatsuo Ishii
Hi, > Hi Ishii-san, > > On 7/19/23 22:15, Tatsuo Ishii wrote: >> Currently my patch has a limitation for the sake of simple >> implementation: a pattern like "A+" is parsed and analyzed in the raw >> parser. This makes subsequent process much easier because the pattern >> element variable (in

Re: Synchronizing slots from primary to standby

2023-07-21 Thread Bharath Rupireddy
On Thu, Jul 20, 2023 at 5:05 PM shveta malik wrote: > > On Fri, Jun 16, 2023 at 3:26 PM Amit Kapila wrote: > > > > On Mon, Apr 17, 2023 at 7:37 PM Drouvot, Bertrand > > wrote: > > > > > > 3. As mentioned in the initial email, I think it would be better to > > replace LIST_SLOTS command with a

table_open/table_close with different lock mode

2023-07-21 Thread Junwang Zhao
Hey hackers, I noticed there are some places calling table_open with RowExclusiveLock but table_close with NoLock, like in function toast_save_datum. Can anybody explain the underlying logic, thanks in advance. -- Regards Junwang Zhao