RE: Data is copied twice when specifying both child and parent table in publication

On Fri, Mar 31, 2023 2:16 AM Jacob Champion wrote: > > On Wed, Mar 29, 2023 at 2:00 AM Amit Kapila > wrote: > > Pushed. > > While rebasing my logical-roots patch over the top of this, I ran into > another situation where mixed viaroot settings can duplicate data. The > key idea is to subscribe

RE: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

On Wed, Mar 22, 2023 2:53 PM Önder Kalacı wrote: > > We don't really need to, if you check the first patch, we don't have DROP for > generated case. I mostly > wanted to make the test a little more interesting, but it also seems to be a > little confusing. > > Now attaching v2 where we do not

RE: Allow logical replication to copy tables in binary format

On Wed Mar 22, 2023 7:29 AM Peter Smith wrote: > > Thanks for all the patch updates. Patch v19 LGTM. > +1 Regards, Shi Yu

RE: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

On Tuesday, March 21, 2023 8:03 PM Önder Kalacı wrote: > > Attached patches again. > Thanks for updating the patch. @@ -408,15 +412,18 @@ $node_subscriber->wait_for_subscription_sync; $node_publisher->safe_psql( 'postgres', qq( ALTER TABLE dropped_cols DROP COLUMN b_dr

RE: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

On Tue, Mar 21, 2023 4:51 PM Önder Kalacı wrote: > > Hi Amit, Shi Yu > > Now attaching the similar patches for generated columns. > Thanks for your patches. Here are some comments. 1. $node_publisher->safe_psql( 'postgres', qq( ALTER TABLE dropped_cols DROP COLUMN b_d

RE: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

On Fri, Mar 17, 2023 11:29 PM Önder Kalacı wrote: > > Thanks for sharing. Fixed > > > This time I was able to run all the tests with all the patches applied. > > Again, the generated column fix also has some minor differences > per version. So, overall we have 6 patches with very minor > diff

RE: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

On Friday, March 17, 2023 3:38 PM Önder Kalacı wrote: > > Hi Amit, all > > You can first submit the fix for dropped columns with patches till > v10. Once that is committed, then you can send the patches for > generated columns. > > Alright, attaching 2 patches for dropped columns, the names of

RE: Allow logical replication to copy tables in binary format

On Thu, Mar 16, 2023 9:20 PM Melih Mutlu wrote: > > Hi, > > Please see the attached v16. > Thanks for updating the patch. +# Cannot sync due to type mismatch +$node_subscriber->wait_for_log(qr/ERROR: ( [A-Z0-9]+:)? incorrect binary data format/); +# Ensure the COPY command is executed in t

RE: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

On Thu, Mar 16, 2023 5:23 PM Amit Kapila wrote: > > On Mon, Mar 13, 2023 at 6:26 PM Önder Kalacı > wrote: > > > > Attaching v2 > > > > Can we change the comment to: "Ignore dropped and generated columns as > the publisher doesn't send those."? After your change, att = > TupleDescAttr(slot1->tts

RE: Allow logical replication to copy tables in binary format

On Thu, Mar 16, 2023 2:26 AM Melih Mutlu wrote: > > Right, it needs to be ordered. Fixed. > Hi, Thanks for updating the patch. I tested some cases like toast data, combination of row filter and column lists, and it works well. Here is a comment: +# Ensure the COPY command is executed in bina

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

On Mon, Mar 13, 2023 10:16 PM Önder Kalacı wrote: > > Attaching v47. > Thanks for updating the patch. Here are some comments. 1. in RemoteRelContainsLeftMostColumnOnIdx(): + if (indexInfo->ii_NumIndexAttrs < 1) + return false; Did you see any cases that the condition is t

RE: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

On Sun, Mar 12, 2023 4:00 AM Önder Kalacı wrote: > > Attaching a patch that could possibly solve the problem. > Thanks for your patch. I tried it and it worked well. Here are some minor comments. 1. @@ -243,6 +243,17 @@ tuples_equal(TupleTableSlot *slot1, TupleTableSlot *slot2,

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

On Fri, Mar 10, 2023 8:17 PM Amit Kapila wrote: > > On Fri, Mar 10, 2023 at 5:16 PM Önder Kalacı wrote: > > > >> > >> wip_for_optimize_index_column_match > >> +static bool > >> +IndexContainsAnyRemoteColumn(IndexInfo *indexInfo, > >> + LogicalRepRelation *remoterel) > >> +{ > >> + for (int i =

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

On Tue, Mar 7, 2023 9:47 PM Önder Kalacı wrote: > > I'm attaching v35. > I noticed that if the index column only exists on the subscriber side, this index can also be chosen. This seems a bit odd because the index column isn't sent from publisher. e.g. -- pub CREATE TABLE test_replica_id_ful

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

On Monay, Mar 6, 2023 7:19 PM Önder Kalacı wrote: > > Yeah, seems easier to follow to me as well. Reflected it in the comment as > well. > Thanks for updating the patch. Here are some comments on v33-0001 patch. 1. + if (RelationReplicaIdentityFullIndexScanEnabled(localrel) && +

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

On Wed, Mar 1, 2023 9:22 PM Önder Kalacı wrote: > > Hi Andres, Amit, Shi Yu, all > > Andres Freund , 28 Şub 2023 Sal, 21:39 tarihinde > şunu yazdı: > Hi, > > On 2023-02-25 16:00:05 +0530, Amit Kapila wrote: > > On Tue, Feb 21, 2023 at 7:55 PM Önder Kalacı

RE: Allow logical replication to copy tables in binary format

On Thu, Feb 23, 2023 12:40 PM Kuroda, Hayato/黒田 隼人 wrote: > > > > I'm not sure the combination of "copy_format = binary" and "copy_data = > false" > > > should be accepted or not. How do you think? > > > > It seems quite useless indeed to specify the format of a copy that won't > happen. > > I

RE: Time delayed LR (WAS Re: logical replication restrictions)

On Wed, Feb 22, 2023 9:48 PM Kuroda, Hayato/黒田 隼人 wrote: > > Thank you for reviewing! PSA new version. > Thanks for your patch. Here is a comment. + elog(DEBUG2, "time-delayed replication for txid %u, delay_time = %d ms, remaining wait time: %ld ms", +ct

RE: "out of relcache_callback_list slots" after multiple calls to pg_logical_slot_get_binary_changes

On Wed, Feb 22, 2023 2:20 PM Michael Paquier wrote: > > On Wed, Feb 22, 2023 at 12:07:06PM +0900, Kyotaro Horiguchi wrote: > > At Wed, 22 Feb 2023 12:29:59 +1100, Peter Smith > wrote in > >> If you are going to do that, then won't just copying the > >> CacheRegisterSyscacheCallback(PUBLICATIONOI

RE: "out of relcache_callback_list slots" after multiple calls to pg_logical_slot_get_binary_changes

On Mon, Feb 20, 2023 11:31 PM Tom Lane wrote: > > Kyotaro Horiguchi writes: > > I'm pretty sure that everytime an output plugin is initialized on a > > process, it installs the same set of syscache/relcache callbacks each > > time. Do you think we could simply stop duplicate registration of > >

RE: Allow logical replication to copy tables in binary format

On Thu, Feb 16, 2023 8:48 PM Amit Kapila wrote: > > On Mon, Jan 30, 2023 at 4:19 PM Melih Mutlu > wrote: > > > > Logical replication between different types like int and smallint is > > already not > working properly on HEAD too. > > Yes, the scenario you shared looks like working. But you didn

"out of relcache_callback_list slots" after multiple calls to pg_logical_slot_get_binary_changes

Hi hackers, After multiple calls to the function pg_logical_slot_get_binary_changes() in single client backend (the output plugin of the slot is pgoutput), I got the following error: client backend FATAL: out of relcache_callback_list slots client backend CONTEXT: slot "testslot", output plugin

Missing default value of createrole_self_grant in document

Hi hackers, I noticed that the document of GUC createrole_self_grant doesn't mention its default value. The attached patch adds that. Regards, Shi Yu v1-0001-Add-default-value-of-createrole_self_grant-in-doc.patch Description: v1-0001-Add-default-value-of-createrole_self_grant-in-doc.patch

RE: run pgindent on a regular basis / scripted manner

On Thu, Feb 9, 2023 6:10 AM Andrew Dunstan wrote: > Thanks, I have committed this. Still looking at Robert's other request. > Hi, Commit #068a243b7 supported directories to be non-option arguments of pgindent. But the help text doesn't mention that. Should we update it? Attach a small patch whic

RE: Add LZ4 compression in pg_dump

On Fri, Jan 27, 2023 2:04 AM gkokola...@pm.me wrote: > > --- Original Message --- > On Thursday, January 26th, 2023 at 12:53 PM, Michael Paquier > wrote: > > > > > > > > On Thu, Jan 26, 2023 at 11:24:47AM +, gkokola...@pm.me wrote: > > > > > I gave this a little bit of thought. I t

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

On Sat, Feb 4, 2023 7:24 PM Amit Kapila wrote: > > On Thu, Feb 2, 2023 at 2:03 PM Önder Kalacı wrote: > > > >> > >> and if there's more > >> than one candidate index pick any one. Additionally, we can allow > >> disabling the use of an index scan for this particular case. If we are > >> too worr

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

On Mon, Feb 13, 2023 7:01 PM shiy.f...@fujitsu.com wrote: > > On Thu, Feb 2, 2023 4:34 PM Önder Kalacı wrote: > > > >> > >> and if there's more > >> than one candidate index pick any one. Additionally, we can allow > >> disabling the use

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

On Thu, Feb 2, 2023 4:34 PM Önder Kalacı wrote: > >> >> and if there's more >> than one candidate index pick any one. Additionally, we can allow >> disabling the use of an index scan for this particular case. If we are >> too worried about API change for allowing users to specify the index >> then

RE: run pgindent on a regular basis / scripted manner

On Thu, Feb 9, 2023 6:10 AM Andrew Dunstan wrote: > Thanks, I have committed this. Still looking at Robert's other request. > Hi, I tried the new option --commit and found that it seems to try to indent files which are deleted in the specified commit and reports an error. cannot open file "src

RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication

On Thu, Feb 2, 2023 11:48 AM shveta malik wrote: > > On Wed, Feb 1, 2023 at 5:42 PM Melih Mutlu > wrote: > > > > Hi Shveta, > > > > shveta malik , 1 Şub 2023 Çar, 15:01 tarihinde > şunu yazdı: > >> > >> On Wed, Feb 1, 2023 at 5:05 PM Melih Mutlu > wrote: > >> 2) I found a crash in the previous

RE: Time delayed LR (WAS Re: logical replication restrictions)

On Mon, Jan 30, 2023 6:05 PM Takamichi Osumi (Fujitsu) wrote: > > On Saturday, January 28, 2023 1:28 PM I wrote: > > Attached the updated patch v24. > Hi, > > > I've conducted the rebase affected by the commit(1e8b61735c) > by renaming the GUC to logical_replication_mode accordingly, > because

RE: Logical replication timeout problem

On Sun, Jan 29, 2023 3:41 PM wangw.f...@fujitsu.com wrote: > > I tested a mix transaction of transactional and non-transactional messages on > the current HEAD and reproduced the timeout problem. I think this result is > OK. > Because when decoding a transaction, non-transactional changes are p

RE: Update comments in multixact.c

On Wed, Jan 18, 2023 6:04 AM Peter Geoghegan wrote: > > On Tue, Jan 17, 2023 at 1:33 AM shiy.f...@fujitsu.com > wrote: > > I noticed that commit 5212d447fa updated some comments in multixact.c > because > > SLRU truncation for multixacts is performed during VACUUM, i

Update comments in multixact.c

Hi, I noticed that commit 5212d447fa updated some comments in multixact.c because SLRU truncation for multixacts is performed during VACUUM, instead of checkpoint. Should the following comments which mentioned checkpointer be changed, too? 1. * we compute it (using nextMXact if none are valid).

RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication

On Wed, Jan 11, 2023 4:31 PM Melih Mutlu wrote: > > Hi hackers, > > Rebased the patch to resolve conflicts. > Thanks for your patch. Here are some comments. 0001 patch 1. walsender.c + /* Create a tuple to send consisten WAL location */ "consisten" should be "consistent" I

RE: Fix pg_publication_tables to exclude generated columns

On Wed, Jan 11, 2023 2:40 PM Amit Kapila wrote: > > On Wed, Jan 11, 2023 at 10:07 AM Tom Lane wrote: > > > > Amit Kapila writes: > > >> On Mon, Jan 9, 2023 11:06 PM Tom Lane wrote: > > >>> We could just not fix it in the back branches. I'd argue that this is > > >>> as much a definition chang

RE: Allow logical replication to copy tables in binary format

On Mon, Nov 14, 2022 8:08 PM Melih Mutlu wrote: > > Attached patch with updated version of this patch. Thanks for your patch. I tried to do a performance test for this patch, the result looks good to me. (The steps are similar to what Melih shared [1].) The time to synchronize about 1GB data in

RE: Fix pg_publication_tables to exclude generated columns

On Mon, Jan 9, 2023 11:06 PM Tom Lane wrote: > > Amit Kapila writes: > > On Mon, Jan 9, 2023 at 5:29 PM shiy.f...@fujitsu.com > > wrote: > >> I think one way to fix it is to modify pg_publication_tables query to > >> exclude > >> generated colu

Fix pg_publication_tables to exclude generated columns

Hi hackers, I noticed that there is a problem about system view pg_publication_tables when looking into [1]. The column "attnames" contains generated columns when no column list is specified, but generated columns shouldn't be included because they are not replicated (see send_relation_and_attrs()

RE: Force streaming every change in logical decoding

On Thu, Dec 22, 2022 3:17 PM Masahiko Sawada wrote: > > I think > > instead of adding new tests with this patch, it may be better to > > change some of the existing tests related to streaming to use this > > parameter as that will clearly show one of the purposes of this patch. > > +1. I think t

Ignore dropped columns when checking the column list in logical replication

Hi hackers, I saw a problem related to column list. There's a restriction that different column lists for same table can't be used in the publications of single subscription. But we will get unexpected errors in some cases because the dropped columns are not ignored. For example: -- publisher cr

RE: Force streaming every change in logical decoding

On Fri, Dec 23, 2022 1:50 PM Amit Kapila > > On Thu, Dec 22, 2022 at 6:18 PM shiy.f...@fujitsu.com > wrote: > > > > > > Besides, I tried to reduce data size in streaming subscription tap tests by > > this > > new GUC (see 0002 patch). But I didn'

RE: Force streaming every change in logical decoding

On Thu, Dec 22, 2022 5:24 PM Amit Kapila wrote: > > On Thu, Dec 22, 2022 at 1:15 PM Kyotaro Horiguchi > wrote: > > > > At Thu, 22 Dec 2022 12:35:46 +0530, Amit Kapila > wrote in > > > I have addressed these comments in the attached. Additionally, I have > > > modified the docs and commit messag

RE: Force streaming every change in logical decoding

On Wed, Dec 21, 2022 4:05 PM Peter Smith wrote: > > Here are some review comments for patch v2. > > Since the GUC is still under design maybe these comments can be > ignored for now, but I guess similar comments will apply in future > anyhow (just with some name changes). > Thanks for your com

RE: Force streaming every change in logical decoding

On Wed, Dec 21, 2022 4:54 PM Amit Kapila wrote: > > On Wed, Dec 21, 2022 at 1:55 PM Peter Smith > wrote: > > > > On Wed, Dec 21, 2022 at 6:22 PM Masahiko Sawada > wrote: > > > > > > On Tue, Dec 20, 2022 at 7:49 PM Amit Kapila > wrote: > > > > > > > > On Tue, Dec 20, 2022 at 2:46 PM Hayato Kuro

RE: Force streaming every change in logical decoding

On Sat, Dec 10, 2022 2:03 PM Dilip Kumar wrote: > > On Tue, Dec 6, 2022 at 11:53 AM shiy.f...@fujitsu.com > wrote: > > > > Hi hackers, > > > > In logical decoding, when logical_decoding_work_mem is exceeded, the > changes are > > sent to outpu

RE: Avoid streaming the transaction which are skipped (in corner cases)

On Wed, Dec 7, 2022 12:01 PM Dilip Kumar wrote: > > On Wed, Dec 7, 2022 at 9:28 AM Amit Kapila > wrote: > > > > On Tue, Dec 6, 2022 at 11:55 AM shiy.f...@fujitsu.com > > wrote: > > > > > > On Mon, Dec 5, 2022 6:57 PM Amit Kapila > wrote: >

RE: Avoid streaming the transaction which are skipped (in corner cases)

On Mon, Dec 5, 2022 6:57 PM Amit Kapila wrote: > > On Mon, Dec 5, 2022 at 3:41 PM Dilip Kumar wrote: > > > > I think we need something like this[1] so that we can better control > > the streaming. > > > > +1. The additional advantage would be that we can generate parallel > apply and new stream

Force streaming every change in logical decoding

Hi hackers, In logical decoding, when logical_decoding_work_mem is exceeded, the changes are sent to output plugin in streaming mode. But there is a restriction that the minimum value of logical_decoding_work_mem is 64kB. I tried to add a GUC to allow sending every change to output plugin without

RE: Avoid streaming the transaction which are skipped (in corner cases)

On Sun, Nov 27, 2022 1:33 PM Dilip Kumar wrote: > > On Sat, Nov 26, 2022 at 12:15 PM Amit Kapila > wrote: > > > > On Fri, Nov 25, 2022 at 5:38 PM Amit Kapila > wrote: > > > > > > On Fri, Nov 25, 2022 at 1:35 PM Dilip Kumar > wrote: > > > > > > > > During DecodeCommit() for skipping a transacti

RE: Fix some newly modified tab-complete changes

On Thu, Nov 10, 2022 12:54 PM Michael Paquier wrote: > > On Tue, Oct 18, 2022 at 05:17:32PM +1100, Peter Smith wrote: > > I re-tested and confirm that the patch does indeed fix the quirk I'd > > previously reported. > > > > But, looking at the patch code, I don't know if it is the best way to > >

RE: Segfault on logical replication to partitioned table with foreign children

On Sun, Oct 30, 2022 9:39 PM Tom Lane wrote: > > What I'm wondering about is whether we can refactor this code > to avoid so many usually-useless catalog lookups. Pulling the > namespace name, in particular, is expensive and we generally > are not going to need the result. In the child-rel case

RE: Perform streaming logical transactions by background workers and parallel apply

On Tue, Oct 25, 2022 2:56 PM Wang, Wei/王 威 wrote: > > On Tues, Oct 25, 2022 at 14:28 PM Peter Smith > wrote: > > FYI - After a recent push, the v40-0001 patch no longer applies on the > > latest HEAD. > > > > [postgres@CentOS7-x64 oss_postgres_misc]$ git apply > > ../patches_misc/v40-0001-Perfor

RE: Perform streaming logical transactions by background workers and parallel apply

On Wed, Oct 26, 2022 7:19 PM Amit Kapila wrote: > > On Tue, Oct 25, 2022 at 8:38 AM Masahiko Sawada > wrote: > > > > On Fri, Oct 21, 2022 at 6:32 PM houzj.f...@fujitsu.com > > wrote: > > > > I've started to review this patch. I tested v40-0001 patch and have > > one question: > > > > IIUC even

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

On Wed, Oct 19, 2022 12:05 AM Önder Kalacı wrote: > > Attached v19. > Thanks for updating the patch. Here are some comments on v19. 1. In execReplication.c: + TypeCacheEntry **eq = NULL; /* only used when the index is not unique */ Maybe the comment here should be changed. Now it is u

RE: create subscription - improved warning message

On Tue, Oct 18, 2022 5:44 PM Peter Smith wrote: > > On Mon, Oct 17, 2022 at 7:11 PM shiy.f...@fujitsu.com > wrote: > > > ... > > > > Thanks for your patch. Here are some comments. > > > > In Example 2, the returned slot_name should be &

RE: create subscription - improved warning message

On Mon, Oct 17, 2022 9:47 AM Peter Smith wrote: > > On Sun, Oct 16, 2022 at 12:14 AM Amit Kapila > wrote: > > > > On Fri, Oct 14, 2022 at 8:22 AM Peter Smith > wrote: > > > > > > On Thu, Oct 13, 2022 at 9:07 AM Peter Smith > wrote: > > > > > ... > > > PSA a patch for adding examples of how to

RE: [RFC] building postgres with meson - v13

On Fri, Oct 14, 2022 12:40 AM Andres Freund wrote: > > Hi, > > On 2022-10-13 09:24:51 +, shiy.f...@fujitsu.com wrote: > > I noticed that `pg_config --configure` didn't show the options given when > > building with meson. > > Yes, that was noted somewhere

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

On Wed, Aug 24, 2022 12:25 AM Önder Kalacı wrote: > Hi, > > Thanks for the review! > Thanks for your reply. > > > > > 1. > > In FilterOutNotSuitablePathsForReplIdentFull(), is > > "nonPartialIndexPathList" a > > good name for the list? Indexes on only expressions are also be filtered. > > > >

RE: [RFC] building postgres with meson - v13

Hi, I noticed that `pg_config --configure` didn't show the options given when building with meson. For example, meson setup build -Dcache=gcc.cache -Ddtrace=enabled -Dicu=enabled -Dcassert=true -Dprefix=/home/postgres/install_meson/ meson compile -C build meson install -C build $ pg_config -

RE: Fix some newly modified tab-complete changes

On Mon, Oct 10, 2022 2:12 PM shiy.f...@fujitsu.com wrote: > > On Tue, Oct 4, 2022 4:17 PM Peter Smith wrote: > > > > But, while testing I noticed another different quirk > > > > It seems that neither the GRANT nor the REVOKE auto-complete > > recog

RE: Fix some newly modified tab-complete changes

On Tue, Oct 4, 2022 4:17 PM Peter Smith wrote: > > On Thu, Sep 29, 2022 at 12:50 PM shiy.f...@fujitsu.com > wrote: > > > > On Wed, Sep 28, 2022 1:49 PM Kyotaro Horiguchi > wrote: > > > > > > At Wed, 28 Sep 2022 14:14:01 +1000, Peter Smith > > &

RE: Fix some newly modified tab-complete changes

On Wed, Sep 28, 2022 1:49 PM Kyotaro Horiguchi wrote: > > At Wed, 28 Sep 2022 14:14:01 +1000, Peter Smith > wrote in > > On Tue, Sep 27, 2022 at 8:28 PM shiy.f...@fujitsu.com > > wrote: > > > > > > Hi hackers, > > > > > > I saw a problem

Fix some newly modified tab-complete changes

Hi hackers, I saw a problem when using tab-complete for "GRANT", "TABLES IN SCHEMA" should be "ALL TABLES IN SCHEMA" in the following case. postgres=# grant all on ALL FUNCTIONS IN SCHEMA DATABASE FUNCTION PARAMETER SCHEMATA

RE: Perform streaming logical transactions by background workers and parallel apply

On Mon, Sept 19, 2022 11:26 AM Wang, Wei/王 威 wrote: > > > Improved as suggested. > Thanks for updating the patch. Here are some comments on 0001 patch. 1. + case TRANS_LEADER_SERIALIZE: - oldctx = MemoryContextSwitchTo(ApplyContext); + /* +

RE: Handle infinite recursion in logical replication setup

On Wed, Sep 7, 2022 12:23 PM vignesh C wrote: > > > Thanks for the comments, the attached v47 patch has the changes for the > same. > Thanks for updating the patch. Here is a comment. + for (i = 0; i < subrel_count; i++) + { + Oid relid = subrel_

RE: Handle infinite recursion in logical replication setup

On Tue, Sep 6, 2022 11:14 AM vignesh C wrote: > > Thanks for the comments, the attached patch has the changes for the same. > Thanks for updating the patch. Here are some comments. 1. + if (subrel_count) + { + /* +* In case of ALTER SUBSCRIPTION ... RE

RE: Column Filtering in Logical Replication

On Mon, Sep 5, 2022 8:28 AM Peter Smith wrote: > > I have rebased the remaining patch (v6-0001 is the same as v5-0002) > Thanks for updating the patch. Here are some comments. 1. + the will be successful but later + the WalSender on the publisher, or the subscriber may throw an error.

RE: Handle infinite recursion in logical replication setup

On Wed, Aug 31, 2022 1:06 AM vignesh C wrote: > > The attached v43 patch has the changes for the same. > Thanks for updating the patch. Here is a comment on the 0001 patch. + if (!isnewtable) + { + pfree(nspname); + pfree

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

On Sat, Aug 20, 2022 7:02 PM Önder Kalacı wrote: > Hi, > > I'm a little late to catch up with your comments, but here are my replies: Thanks for your patch. Here are some comments. 1. In FilterOutNotSuitablePathsForReplIdentFull(), is "nonPartialIndexPathList" a good name for the list? Indexes

RE: Perform streaming logical transactions by background workers and parallel apply

On Wed, Aug 17, 2022 2:28 PM Wang, Wei/王 威 wrote: > > On Tues, August 16, 2022 15:33 PM I wrote: > > Attach the new patches. > > I found that cfbot has a failure. > After investigation, I think it is because the worker's exit state is not set > correctly. So I made some slight modifications. >

RE: Perform streaming logical transactions by background workers and parallel apply

On Thu, Aug 4, 2022 2:36 PM Wang, Wei/王 威 wrote: > > I also did some other improvements based on the suggestions posted in this > thread. Attach the new patches. > Thanks for updating the patch. Here are some comments on v20-0001 patch. 1. +typedef struct ApplyBgworkerShared +{ + slock_t

RE: Introduce wait_for_subscription_sync for TAP tests

On Thu, Aug 4, 2022 5:49 PM shiy.f...@fujitsu.com wrote: > > On Thu, Aug 4, 2022 2:28 PM Masahiko Sawada > wrote: > > > > On Thu, Aug 4, 2022 at 10:37 AM Amit Kapila > > wrote: > > > > > > On Wed, Aug 3, 2022 at 10:21 AM Amit Kapila > > &g

RE: Introduce wait_for_subscription_sync for TAP tests

On Thu, Aug 4, 2022 2:28 PM Masahiko Sawada wrote: > > On Thu, Aug 4, 2022 at 10:37 AM Amit Kapila > wrote: > > > > On Wed, Aug 3, 2022 at 10:21 AM Amit Kapila > wrote: > > > > > > Pushed this one and now I'll look at your other patch. > > > > > > > I have pushed the second patch as well after

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

On Wed, Aug 3, 2022 12:06 PM Masahiko Sawada wrote: > > I've attached updated patches that incorporated the above comments as > well as the comments from Shi yu. Please review them. > Thanks for updating the patch. I noticed that in SnapBuildXidSetCatalogChanges(), "i" is initialized in the if

RE: Handle infinite recursion in logical replication setup

On Fri, Jul 29, 2022 1:22 PM vignesh C wrote: > > On Fri, Jul 29, 2022 at 8:31 AM Peter Smith > wrote: > > > > Thanks for the comments, the attached v41 patch has the changes for the > same. > Thanks for updating the patch. A comment for 0002 patch. In the example in section 31.11.4 (Generi

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

On Mon, Aug 1, 2022 10:31 PM Amit Kapila wrote: > > On Mon, Aug 1, 2022 at 7:46 AM Masahiko Sawada > wrote: > > > > On Fri, Jul 29, 2022 at 3:45 PM Amit Kapila > wrote: > > > > > > > I've attached updated patches for all branches. Please review them. > > > > Thanks, the patches look mostly goo

RE: Handle infinite recursion in logical replication setup

On Fri, Jul 29, 2022 1:22 PM vignesh C wrote: > > > Thanks for the comments, the attached v41 patch has the changes for the > same. > Thanks for updating the patch. I wonder in the case that the publisher uses PG15 (or before), subscriber uses PG16, should we have this check (check if publica

RE: Introduce wait_for_subscription_sync for TAP tests

On Tue, Jul 26, 2022 3:42 PM Masahiko Sawada wrote: > > I've attached an updated patch as well as a patch to remove duplicated > waits in 007_ddl.pl. > Thanks for your patch. Here are some comments. 1. I think some comments need to be changed in the patch. For example: # Also wait for initial

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

On Tue, Jul 26, 2022 3:52 PM Masahiko Sawada wrote: > > On Tue, Jul 26, 2022 at 2:18 PM Amit Kapila > wrote: > > > > On Tue, Jul 26, 2022 at 7:00 AM Masahiko Sawada > wrote: > > > > > > On Mon, Jul 25, 2022 at 7:57 PM shiy.f...@fujitsu.com &g

RE: Handle infinite recursion in logical replication setup

On Sun, Jul 24, 2022 1:28 AM vignesh C wrote: > > Added a note for the same and referred it to the conflicts section. > > Thanks for the comments, the attached v38 patch has the changes for the > same. > Thanks for updating the patch. A comment on the test in 0001 patch. +# Alter subscriptio

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

Hi, I did some performance test for the master branch patch (based on v6 patch) to see if the bsearch() added by this patch will cause any overhead. I tested them three times and took the average. The results are as follows, and attach the bar chart. case 1 - No catalog modifying transa

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

On Fri, Jul 15, 2022 10:39 PM Masahiko Sawada wrote: > > This patch should have the fix for the issue that Shi yu reported. Shi > yu, could you please test it again with this patch? > Thanks for updating the patch! I have tested and confirmed that the problem I found has been fixed. Regards, S

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

On Mon, Jul 11, 2022 9:54 PM Masahiko Sawada wrote: > > I've attached an updated patch, please review it. > Thanks for your patch. Here are some comments for the REL14-v1 patch. 1. + Sizesz = sizeof(TransactionId) * nxacts;; There is a redundant semicolon at the end.

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

On Tue, Jul 12, 2022 5:23 PM Masahiko Sawada wrote: > > On Tue, Jul 12, 2022 at 5:58 PM shiy.f...@fujitsu.com > wrote: > > > > It happened when executing the following code because it tried to free a > NULL > > pointer (catchange_xip). > > > >

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

On Tue, Jul 12, 2022 8:49 AM Masahiko Sawada wrote: > > I've attached an updated patch. > Hi, I met a segmentation fault in test_decoding test after applying the patch for master branch. Attach the backtrace. It happened when executing the following code because it tried to free a NULL point

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

On Tue, Jul 12, 2022 8:49 AM Masahiko Sawada wrote: > > I've attached an updated patch. > > While trying this idea, I noticed there is no API to get the length of > dlist, as we discussed offlist. Alternative idea was to use List > (T_XidList) but I'm not sure it's a great idea since deleting an

RE: Perform streaming logical transactions by background workers and parallel apply

On Tue, Jun 28, 2022 11:22 AM Wang, Wei/王 威 wrote: > > I also improved patches as suggested by Peter-san in [1] and [2]. > Thanks for Shi Yu to improve the patches by addressing the comments in [2]. > > Attach the new patches. > Thanks for updating the patch. Here are some comments. 0001 pat

RE: Handle infinite recursion in logical replication setup

On Mon, Jul 4, 2022 4:17 PM shiy.f...@fujitsu.com wrote: > > On Sun, Jul 3, 2022 11:00 PM vignesh C wrote: > > > > Thanks for the comments, the attached v27 patch has the changes for the > > same. > > > > Thanks for updating the patch. > > A comment

RE: Handle infinite recursion in logical replication setup

On Sun, Jul 3, 2022 11:00 PM vignesh C wrote: > > Thanks for the comments, the attached v27 patch has the changes for the > same. > Thanks for updating the patch. A comment on 0003 patch: + /* +* No need to throw an error for the tables that are in ready state, +

RE: Handle infinite recursion in logical replication setup

On Tue, Jun 28, 2022 2:18 PM vignesh C wrote: > > Thanks for the comments, the attached v25 patch has the changes for the > same. > Thanks for updating the patch. Here are some comments. 0002 patch: == 1. +# Test the CREATE SUBSCRIPTION 'origin' parameter and its interaction with

RE: tablesync copy ignores publication actions

On Wed, Jun 22, 2022 4:49 PM Peter Smith wrote: > > On Wed, Jun 22, 2022 at 2:18 PM Amit Kapila > wrote: > > > > On Thu, Jun 16, 2022 at 6:07 AM Peter Smith > wrote: > > > > > > > > Thank you for your review comments. Those reported mistakes are fixed > > > in the attached patch v3. > > > > > >

RE: Handle infinite recursion in logical replication setup

On Mon, Jun 20, 2022 7:55 PM vignesh C wrote: > > Thanks for the comment, the v22 patch attached has the changes for the > same. Thanks for updating the patch, here are some comments on 0003 patch. 1. 032_origin.pl +###

RE: Replica Identity check of partition table on subscriber

On Tuesday, June 21, 2022 4:49 PM Amit Kapila wrote: > > On Tue, Jun 21, 2022 at 12:50 PM Amit Langote > wrote: > > > > On Tue, Jun 21, 2022 at 3:35 PM houzj.f...@fujitsu.com > > wrote: > > > > Attached a patch containing the above to consider as an alternative. > > > > Thanks, the patch looks

RE: Replica Identity check of partition table on subscriber

On Mon, Jun 20, 2022 1:33 PM Amit Kapila wrote: > > On Fri, Jun 17, 2022 at 11:22 AM shiy.f...@fujitsu.com > wrote: > > > > On Fri Jun 17, 2022 11:06 AM shiy.f...@fujitsu.com > wrote: > > > > > > Attached the new version of patch set. I also moved

RE: Handle infinite recursion in logical replication setup

On Thu, Jun 16, 2022 6:18 PM vignesh C wrote: > > Thanks for the comments, the attached v21 patch has the changes for the > same. > Thanks for updating the patch. Here are some comments. 0002 patch == 1. + publisher to only send changes that originated locally. Setting +

RE: Replica Identity check of partition table on subscriber

On Fri Jun 17, 2022 11:06 AM shiy.f...@fujitsu.com wrote: > > Attached the new version of patch set. I also moved the partitioned table > check > in logicalrep_rel_mark_updatable() to check_relation_updatable() as > discussed > [2]. > Attached back-branch patches of the

RE: Replica Identity check of partition table on subscriber

On Thu, Jun 16, 2022 2:13 PM Amit Langote wrote: > > Hi, > > On Thu, Jun 16, 2022 at 2:07 PM shiy.f...@fujitsu.com > wrote: > > On Wed, Jun 15, 2022 8:30 PM Amit Kapila > wrote: > > > I have pushed the first bug-fix patch today. > > > > Att

RE: Replica Identity check of partition table on subscriber

On Wed, Jun 15, 2022 8:30 PM Amit Kapila wrote: > > I have pushed the first bug-fix patch today. > Thanks. Attached the remaining patches which are rebased. Regards, Shi yu v9-0002-fix-memory-leak-about-attrmap.patch Description: v9-0002-fix-memory-leak-about-attrmap.patch v9-0001-Check-

RE: tablesync copy ignores publication actions

On Tue, Jun 14, 2022 3:36 PM Peter Smith wrote: > > PSA v2 of the patch, based on all feedback received. > > ~~~ > > Main differences from v1: > > * Rewording and more explanatory text. > > * The examples were moved to the "Subscription" [1] page and also > extended to show some normal replic

  1   2   >