Re: Data is copied twice when specifying both child and parent table in publication

2023-03-31 Thread Jacob Champion
On 3/31/23 03:04, shiy.f...@fujitsu.com wrote: > I noticed that a similar problem has been discussed in this thread, see [1] > [2] > [3] [4]. Ah, thank you. I didn't go far back enough in the thread... > It seems complicated to fix it if we want to automatically skip tables > that have been

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-31 Thread Jacob Champion
On 3/30/23 20:01, Peter Smith wrote: > For example, Just imagine if logic could be made smarter to recognize > that since there was already the 'part_def' being subscribed so it > should NOT use the default 'copy_data=true' when the REFRESH launches > the ancestor table 'part'... > > Even if that

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-31 Thread shiy.f...@fujitsu.com
On Fri, Mar 31, 2023 2:16 AM Jacob Champion wrote: > > On Wed, Mar 29, 2023 at 2:00 AM Amit Kapila > wrote: > > Pushed. > > While rebasing my logical-roots patch over the top of this, I ran into > another situation where mixed viaroot settings can duplicate data. The > key idea is to subscribe

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-30 Thread Peter Smith
On Fri, Mar 31, 2023 at 5:15 AM Jacob Champion wrote: > > On Wed, Mar 29, 2023 at 2:00 AM Amit Kapila wrote: > > Pushed. > > While rebasing my logical-roots patch over the top of this, I ran into > another situation where mixed viaroot settings can duplicate data. The > key idea is to subscribe

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-30 Thread Jacob Champion
On Wed, Mar 29, 2023 at 2:00 AM Amit Kapila wrote: > Pushed. While rebasing my logical-roots patch over the top of this, I ran into another situation where mixed viaroot settings can duplicate data. The key idea is to subscribe to two publications with mixed settings, as before, and add a

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-29 Thread Amit Kapila
On Tue, Mar 28, 2023 at 11:14 PM Jacob Champion wrote: > > On Tue, Mar 28, 2023 at 2:59 AM wangw.f...@fujitsu.com > > > > Would it > > > be enough to just replace that whole thing with gpt.attrs? > > > > Make sense. > > Changed as suggested. > > LGTM, by inspection. Thanks! > Pushed. -- With

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-28 Thread Amit Kapila
On Wed, Mar 29, 2023 at 7:44 AM Peter Smith wrote: > > A minor review comment for v25-0001. > > == > src/backend/commands/subscriptioncmds.c > > 1. > @@ -1936,21 +1936,56 @@ fetch_table_list(WalReceiverConn *wrconn, List > *publications) > WalRcvExecResult *res; > StringInfoData cmd; >

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-28 Thread Peter Smith
A minor review comment for v25-0001. == src/backend/commands/subscriptioncmds.c 1. @@ -1936,21 +1936,56 @@ fetch_table_list(WalReceiverConn *wrconn, List *publications) WalRcvExecResult *res; StringInfoData cmd; TupleTableSlot *slot; - Oid tableRow[3] = {TEXTOID, TEXTOID,

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-28 Thread Jacob Champion
On Tue, Mar 28, 2023 at 2:59 AM wangw.f...@fujitsu.com wrote: > The scenario of this bug is to subscribe to two publications at the same time, > and these two publications publish parent table and child table respectively. > And option via_root is specified in both publications or only in the >

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-28 Thread wangw.f...@fujitsu.com
On Tues, Mar 28, 2023 at 18:00 PM Wang, Wei/王 威 wrote: > Attach the new patch. Sorry, I attached the wrong patch. Here is the correct new version patch which addressed all comments so far. Regards, Wang Wei v25-0001-Avoid-syncing-data-twice-for-the-publish_via_par.patch Description:

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-28 Thread wangw.f...@fujitsu.com
On Tues, Mar 28, 2023 at 7:02 AM Jacob Champion wrote: > On Mon, Mar 20, 2023 at 11:22 PM Amit Kapila > wrote: > > If the tests you have in mind are only related to this patch set then > > feel free to propose them here if you feel the current ones are not > > sufficient. > > I think the new

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-27 Thread Jacob Champion
On Mon, Mar 20, 2023 at 11:22 PM Amit Kapila wrote: > If the tests you have in mind are only related to this patch set then > feel free to propose them here if you feel the current ones are not > sufficient. I think the new tests added by Wang cover my concerns (thanks!). I share Peter's comment

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-27 Thread Peter Smith
I looked at v24-0001. == src/test/subscription/t/028_row_filter.pl +# Check expected replicated rows for tap_pub_parent_sync and +# tap_pub_child_sync +# Since the option publish_via_partition_root of tap_pub_parent_sync is true, +# so the row filter of tap_pub_parent_sync will be used: +#

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-26 Thread wangw.f...@fujitsu.com
On Mon, Mar 27, 2023 at 11:32 AM Amit Kapila wrote: > On Mon, Mar 27, 2023 at 7:03 AM Peter Smith > wrote: > > > > > > 1. > > +# two publications, one publishing through ancestor and another one > > directly > > +# publsihing the partition, with different row filters > >

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-26 Thread Amit Kapila
On Mon, Mar 27, 2023 at 7:03 AM Peter Smith wrote: > > > 1. > +# two publications, one publishing through ancestor and another one directly > +# publsihing the partition, with different row filters > +$node_publisher->safe_psql('postgres', > + "CREATE PUBLICATION tap_pub_viaroot_sync_1 FOR TABLE

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-26 Thread Peter Smith
Here are some review comments for v23-0001. == src/test/subscription/t/028_row_filter.pl 1. +# two publications, one publishing through ancestor and another one directly +# publsihing the partition, with different row filters +$node_publisher->safe_psql('postgres', + "CREATE PUBLICATION

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-25 Thread Amit Kapila
On Fri, Mar 24, 2023 at 2:36 PM wangw.f...@fujitsu.com wrote: > > On Fri, Mar 24, 2023 at 14:17 PM Amit Kapila wrote: > > And I found there is a problem in the three back-branch patches > (HEAD_v21_0002*, > REL15_* and REL14_*): > In the function fetch_table_list, we use pg_partition_ancestors

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-24 Thread wangw.f...@fujitsu.com
On Fri, Mar 24, 2023 at 9:49 AM Peter Smith wrote: > On Thu, Mar 23, 2023 at 8:11 PM wangw.f...@fujitsu.com > wrote: > > > > On Thu, Mar 23, 2023 at 12:27 PM Peter Smith > wrote: > > > Here are some review comments for patch v20-0001. > > > ... > > > == > > >

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-24 Thread wangw.f...@fujitsu.com
On Fri, Mar 24, 2023 at 14:17 PM Amit Kapila wrote: > > > > == > > > > src/test/subscription/t/028_row_filter.pl > > > > > > > > 7. > > > > +# insert data into partitioned table. > > > > +$node_publisher->safe_psql('postgres', > > > > + "INSERT INTO tab_rowfilter_viaroot_part(a) VALUES(13),

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-24 Thread wangw.f...@fujitsu.com
On Fri, Mar 24, 2023 at 10:14 AM Peter Smith wrote: > Thanks for the information. > BTW, since this patch changes the signature of the API > pg_get_publication_tables, I assume the example in the CREATE > SUBSCRIPTION Notes [1] may not work anymore. The use case you mentioned is still work. >

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-24 Thread Amit Kapila
On Fri, Mar 24, 2023 at 7:19 AM Peter Smith wrote: > > Hi Wang-san. I looked at the v21-0001 patch. > > I don't have any new review comments -- only follow-ups for some of my > previous v20 comments that were rejected. > > On Thu, Mar 23, 2023 at 8:11 PM wangw.f...@fujitsu.com > wrote: > > > >

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-23 Thread Peter Smith
BTW, since this patch changes the signature of the API pg_get_publication_tables, I assume the example in the CREATE SUBSCRIPTION Notes [1] may not work anymore. Meanwhile, Tom Lane suggested [2] that the example could be re-written to avoid even mentioning pg_get_publication_tables at all.

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-23 Thread Peter Smith
Hi Wang-san. I looked at the v21-0001 patch. I don't have any new review comments -- only follow-ups for some of my previous v20 comments that were rejected. On Thu, Mar 23, 2023 at 8:11 PM wangw.f...@fujitsu.com wrote: > > On Thu, Mar 23, 2023 at 12:27 PM Peter Smith wrote: > > Here are some

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-23 Thread Peter Smith
On Thu, Mar 23, 2023 at 5:51 PM Amit Kapila wrote: > > On Thu, Mar 23, 2023 at 9:57 AM Peter Smith wrote: > > > > Here are some review comments for patch v20-0001. > > > > == > > General. > > > > 1. > > That function 'pg_get_publication_tables' does not seem to be > > described in the PG

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-23 Thread wangw.f...@fujitsu.com
On Thu, Mar 23, 2023 at 12:27 PM Peter Smith wrote: > Here are some review comments for patch v20-0001. Thanks for your comments. > == > src/backend/commands/subscriptioncmds.c > > 3. fetch_table_list > > + /* Get the list of tables from the publisher. */ > + if (server_version >= 16)

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-23 Thread Amit Kapila
On Thu, Mar 23, 2023 at 9:57 AM Peter Smith wrote: > > Here are some review comments for patch v20-0001. > > == > General. > > 1. > That function 'pg_get_publication_tables' does not seem to be > described in the PG documentation. Why isn't it in the "System Catalog > Information Functions"

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-22 Thread Peter Smith
Here are some review comments for patch v20-0001. == General. 1. That function 'pg_get_publication_tables' does not seem to be described in the PG documentation. Why isn't it in the "System Catalog Information Functions" table [1] ? I asked this same question a long time ago but then the

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-22 Thread wangw.f...@fujitsu.com
On Wed, Mar 22, 2023 at 14:32 PM Kuroda, Hayato/黒田 隼人 wrote: > Dear Wang, > > Thank you for updating patch! Following are comments form v19-0001. Thanks for your comments. > 01. logical-replication.sgml > > I found a following statement in logical-replication.sgml. I think this may > cause

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-22 Thread wangw.f...@fujitsu.com
On Wed, Mar 22, 2023 at 12:50 PM Peter Smith wrote: > Here are some review comments for patch code of HEAD_v19-0001 Thanks for your comments. > == > doc/src/sgml/ref/create_publication.sgml > > 1. > + > + There can be a case where a subscription combines multiple > +

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-22 Thread Hayato Kuroda (Fujitsu)
Dear Wang, Thank you for updating patch! Following are comments form v19-0001. 01. logical-replication.sgml I found a following statement in logical-replication.sgml. I think this may cause mis-reading because it's OK when publishers list partitions and publish_via_root is true. ``` A

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-21 Thread Peter Smith
Here are some review comments for patch code of HEAD_v19-0001 == doc/src/sgml/ref/create_publication.sgml 1. + + There can be a case where a subscription combines multiple + publications. If a root partitioned table is published by any + subscribed

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-21 Thread wangw.f...@fujitsu.com
On Mon, Mar 20, 2023 at 21:18 PM Kuroda, Hayato/黒田 隼人 wrote: > Dear Wang, > > I have tested about multilevel partitions, and it worked well. > Followings are my comments for v18-0001. Thanks for your comments and testing. > 01. pg_get_publication_tables > > ``` > + ListCell

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-21 Thread wangw.f...@fujitsu.com
On Mon, Mar 20, 2023 at 15:32 PM Peter Smith wrote: > Here are some review comments for v17-0001. Thanks for your comments. > == > src/backend/catalog/pg_publication.c > > 1. filter_partitions > > -static List * > -filter_partitions(List *relids) > +static void > +filter_partitions(List

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-21 Thread wangw.f...@fujitsu.com
On Sat, Mar 18, 2023 at 7:37 AM Jacob Champion wrote: > On Thu, Mar 16, 2023 at 11:28 PM wangw.f...@fujitsu.com > wrote: > > Attach the new patch set. Thanks for your comments and testing. > For example, the corner case mentioned in 0003, with multiple > publications having conflicting

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-21 Thread wangw.f...@fujitsu.com
On Mon, Mar 20, 2023 at 18:15 PM Amit Kapila wrote: > Thanks for your comments. > On Mon, Mar 20, 2023 at 1:02 PM Peter Smith > wrote: > > > > > > == > > src/include/catalog/pg_proc.dat > > > > 4. > > +{ oid => '6119', > > + descr => 'get information of the tables in the given publication

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-21 Thread Amit Kapila
On Mon, Mar 20, 2023 at 11:22 PM Jacob Champion wrote: > > On Fri, Mar 17, 2023 at 9:45 PM Amit Kapila wrote: > > > There are a bunch of moving parts and hidden subtleties here, and I fell > > > into a few traps when I was working on my patch, so it'd be nice to have > > > additional coverage.

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-20 Thread Jacob Champion
On Fri, Mar 17, 2023 at 9:45 PM Amit Kapila wrote: > > There are a bunch of moving parts and hidden subtleties here, and I fell > > into a few traps when I was working on my patch, so it'd be nice to have > > additional coverage. I'm happy to contribute effort in that area if it's > > helpful. >

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-20 Thread Hayato Kuroda (Fujitsu)
Dear Wang, I have tested about multilevel partitions, and it worked well. Followings are my comments for v18-0001. 01. pg_get_publication_tables ``` + ListCell *lc; ``` This definition can be inside of the "for (i = 0; i < nelems; i++)". 02. pg_get_publication_tables ``` -

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-20 Thread Amit Kapila
On Mon, Mar 20, 2023 at 1:02 PM Peter Smith wrote: > > > == > src/include/catalog/pg_proc.dat > > 4. > +{ oid => '6119', > + descr => 'get information of the tables in the given publication array', > > Should that be worded in a way to make it more clear that the > "publication array" is

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-20 Thread Amit Kapila
On Mon, Mar 20, 2023 at 1:02 PM Peter Smith wrote: > > > 2. pg_get_publication_tables > > + else > + { > + List*relids, > +*schemarelids; > + > + relids = GetPublicationRelations(pub_elem->oid, > + pub_elem->pubviaroot ? > + PUBLICATION_PART_ROOT : > + PUBLICATION_PART_LEAF); > +

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-20 Thread Peter Smith
Here are some review comments for v17-0001. == src/backend/catalog/pg_publication.c 1. filter_partitions -static List * -filter_partitions(List *relids) +static void +filter_partitions(List *table_infos) { - List*result = NIL; ListCell *lc; - ListCell *lc2; - foreach(lc, relids)

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-19 Thread wangw.f...@fujitsu.com
On Fri, Mar 17, 2023 at 20:07 PM Amit Kapila wrote: > On Fri, Mar 17, 2023 at 11:58 AM wangw.f...@fujitsu.com > wrote: > > > > On Thu, Mar 16, 2023 at 20:25 PM Amit Kapila > wrote: > > > > > > > Thanks for your comments. > > > > > + if (server_version >= 16) > > > + { > > > +

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-17 Thread Amit Kapila
On Sat, Mar 18, 2023 at 5:06 AM Jacob Champion wrote: > > On Thu, Mar 16, 2023 at 11:28 PM wangw.f...@fujitsu.com > wrote: > > Attach the new patch set. > > Hi, > > I ran into this problem while hacking on [1], so thank you for tackling > it! I have no strong opinions on the implementation

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-17 Thread Jacob Champion
On Thu, Mar 16, 2023 at 11:28 PM wangw.f...@fujitsu.com wrote: > Attach the new patch set. Hi, I ran into this problem while hacking on [1], so thank you for tackling it! I have no strong opinions on the implementation itself; I just want to register a concern that the tests have not kept up

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-17 Thread Amit Kapila
On Fri, Mar 17, 2023 at 11:58 AM wangw.f...@fujitsu.com wrote: > > On Thu, Mar 16, 2023 at 20:25 PM Amit Kapila wrote: > > > > Thanks for your comments. > > > + if (server_version >= 16) > > + { > > + appendStringInfo(, "SELECT DISTINCT N.nspname, C.relname,\n" > > + " ( SELECT

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-17 Thread wangw.f...@fujitsu.com
On Thu, Mar 16, 2023 at 20:25 PM Amit Kapila wrote: > Thanks for your comments. > + if (server_version >= 16) > + { > + appendStringInfo(, "SELECT DISTINCT N.nspname, C.relname,\n" > + " ( SELECT array_agg(a.attname ORDER BY a.attnum)\n" > + "FROM pg_attribute

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-16 Thread Amit Kapila
On Wed, Feb 8, 2023 at 9:21 AM wangw.f...@fujitsu.com wrote: > > I think this failure is caused by the recently commit (b7ae039) in the current > HEAD. Rebased the patch set and attach them. > + if (server_version >= 16) + { + appendStringInfo(, "SELECT DISTINCT N.nspname, C.relname,\n" + "

RE: Data is copied twice when specifying both child and parent table in publication

2023-02-07 Thread wangw.f...@fujitsu.com
On Wed, Feb 8, 2023 4:29 AM Andres Freund wrote: > Hi, > > On 2022-11-16 08:58:31 +, wangw.f...@fujitsu.com wrote: > > Attach the new patch set. > > This patch causes several of the tests to fail. See e.g.: > > https://cirrus-ci.com/task/6587624765259776 > > Most of the failures appear to

Re: Data is copied twice when specifying both child and parent table in publication

2023-02-07 Thread Andres Freund
Hi, On 2022-11-16 08:58:31 +, wangw.f...@fujitsu.com wrote: > Attach the new patch set. This patch causes several of the tests to fail. See e.g.: https://cirrus-ci.com/task/6587624765259776 Most of the failures appear to be due to the main regression tests failing:

RE: Data is copied twice when specifying both child and parent table in publication

2022-11-16 Thread wangw.f...@fujitsu.com
On Thurs, Nov 17, 2022 at 13:58 PM vignesh C wrote: > On Wed, 16 Nov 2022 at 14:28, wangw.f...@fujitsu.com > wrote: > > > > On Mon, Nov 14, 2022 at 0:56 AM vignesh C wrote: > > > > > > > > Attach new patches. > > > > > > > Thanks for your comments. > > > > > Here we are having tables list to

Re: Data is copied twice when specifying both child and parent table in publication

2022-11-16 Thread vignesh C
On Wed, 16 Nov 2022 at 14:28, wangw.f...@fujitsu.com wrote: > > On Mon, Nov 14, 2022 at 0:56 AM vignesh C wrote: > > > > > > Attach new patches. > > > > Thanks for your comments. > > > Here we are having tables list to store the relids and table_infos > > list which stores pubid along with

RE: Data is copied twice when specifying both child and parent table in publication

2022-11-16 Thread wangw.f...@fujitsu.com
On Mon, Nov 14, 2022 at 0:56 AM vignesh C wrote: > > > > Attach new patches. > Thanks for your comments. > Here we are having tables list to store the relids and table_infos > list which stores pubid along with relid. Here tables list acts as a > temporary list to get filter_partitions and

Re: Data is copied twice when specifying both child and parent table in publication

2022-11-13 Thread vignesh C
On Fri, 11 Nov 2022 at 11:13, wangw.f...@fujitsu.com wrote: > > On Fri, Oct 21, 2022 at 17:02 PM Peter Smith wrote: > > > > Thanks for your comments. Sorry for not replying in time. > > > On Mon, Oct 17, 2022 at 4:49 PM wangw.f...@fujitsu.com > > wrote: > > > > > > On Wed, Oct 5, 2022 at

RE: Data is copied twice when specifying both child and parent table in publication

2022-11-10 Thread wangw.f...@fujitsu.com
On Tues, Nov 8, 2022 at 12:12 PM Osumi, Takamichi/大墨 昂道 wrote: > On Monday, October 17, 2022 2:49 PM Wang, Wei/王 威 > wrote: > > Attach the new patch set. > Hi, thank you for posting the new patches. > > > Here are minor comments on the HEAD_v13-0002. Thanks for your comments. > (1)

RE: Data is copied twice when specifying both child and parent table in publication

2022-11-10 Thread wangw.f...@fujitsu.com
On Fri, Oct 21, 2022 at 17:02 PM Peter Smith wrote: > Here are my review comments for HEAD patches v13* Thanks for your comments. > Patch HEAD_v13-0002 > > 1. Commit message > > The following usage scenarios are not described in detail in the manual: > If one subscription subscribes multiple

RE: Data is copied twice when specifying both child and parent table in publication

2022-11-10 Thread wangw.f...@fujitsu.com
On Fri, Oct 21, 2022 at 17:02 PM Peter Smith wrote: > Thanks for your comments. Sorry for not replying in time. > On Mon, Oct 17, 2022 at 4:49 PM wangw.f...@fujitsu.com > wrote: > > > > On Wed, Oct 5, 2022 at 11:08 AM Peter Smith > wrote: > > > Hi Wang-san. Here are my review comments for

RE: Data is copied twice when specifying both child and parent table in publication

2022-11-07 Thread Takamichi Osumi (Fujitsu)
On Monday, October 17, 2022 2:49 PM Wang, Wei/王 威 wrote: > Attach the new patch set. Hi, thank you for posting the new patches. Here are minor comments on the HEAD_v13-0002. (1) Suggestion for the document description + + If a root partitioned table is published by any

Re: Data is copied twice when specifying both child and parent table in publication

2022-11-03 Thread Ian Lawrence Barwick
2022年10月17日(月) 14:49 wangw.f...@fujitsu.com : > > On Wed, Oct 5, 2022 at 11:08 AM Peter Smith wrote: > > Hi Wang-san. Here are my review comments for HEAD_v12-0001 patch. > > Thanks for your comments. > > > == > > > > 1. Missing documentation. > > > > In [1] you wrote: > > > I think the

Re: Data is copied twice when specifying both child and parent table in publication

2022-10-21 Thread Peter Smith
Here are my review comments for HEAD patches v13* // Patch HEAD_v13-0001 I already posted some follow-up questions. See [1] / Patch HEAD_v13-0002 1. Commit message The following usage scenarios are not described in detail in the manual: If one subscription subscribes multiple

Re: Data is copied twice when specifying both child and parent table in publication

2022-10-21 Thread Peter Smith
On Mon, Oct 17, 2022 at 4:49 PM wangw.f...@fujitsu.com wrote: > > On Wed, Oct 5, 2022 at 11:08 AM Peter Smith wrote: > > Hi Wang-san. Here are my review comments for HEAD_v12-0001 patch. > ... > > > > 3. QUESTION - pg_get_publication_tables / fetch_table_list > > > > When the same table is

RE: Data is copied twice when specifying both child and parent table in publication

2022-10-16 Thread wangw.f...@fujitsu.com
On Wed, Oct 5, 2022 at 23:05 PM Osumi, Takamichi/大墨 昂道 wrote: > Hi, thank you for the updated patches! > > > Here are my minor review comments for HEAD v12. Thanks for your comments. > (1) typo & suggestion to reword one comment > > > +* Publications support

RE: Data is copied twice when specifying both child and parent table in publication

2022-10-16 Thread wangw.f...@fujitsu.com
On Wed, Oct 5, 2022 at 11:08 AM Peter Smith wrote: > Hi Wang-san. Here are my review comments for HEAD_v12-0001 patch. Thanks for your comments. > == > > 1. Missing documentation. > > In [1] you wrote: > > I think the behaviour of multiple publications with parameter >

RE: Data is copied twice when specifying both child and parent table in publication

2022-10-05 Thread osumi.takami...@fujitsu.com
On Wednesday, September 28, 2022 5:36 PM Wang, Wei/王 威 wrote: > Also rebased the patch because the change in the HEAD (20b6847). > > Attach the new patches. Hi, thank you for the updated patches! Here are my minor review comments for HEAD v12. (1) typo & suggestion to reword one comment +

Re: Data is copied twice when specifying both child and parent table in publication

2022-10-04 Thread Peter Smith
Hi Wang-san. Here are my review comments for HEAD_v12-0001 patch. == 1. Missing documentation. In [1] you wrote: > I think the behaviour of multiple publications with parameter > publish_via_partition_root could be added to the pg-doc later in a separate > patch. ~ That doesn't seem

RE: Data is copied twice when specifying both child and parent table in publication

2022-09-28 Thread wangw.f...@fujitsu.com
On Tues, Sep 27, 2022 at 16:45 PM Peter Smith wrote: > Here are my review comments for the HEAD_v11-0001 patch: Thanks for your comments. > == > > 1. General - Another related bug? > > In [1] Hou-san wrote: > > For another case you mentioned (via_root used when publishing child) > CREATE

Re: Data is copied twice when specifying both child and parent table in publication

2022-09-27 Thread Peter Smith
Here are my review comments for the HEAD_v11-0001 patch: == 1. General - Another related bug? In [1] Hou-san wrote: For another case you mentioned (via_root used when publishing child) CREATE PUBLICATION pub1 for TABLE parent; CREATE PUBLICATION pub2 for TABLE child with

RE: Data is copied twice when specifying both child and parent table in publication

2022-09-25 Thread wangw.f...@fujitsu.com
On Mon, Sep 26, 2022 at 10:31 AM Osumi, Takamichi/大墨 昂道 wrote: > Hi, thank you for updating the patchset. > > > FYI, I noticed that the patch for head is no longer applicable. Thanks for your kindly reminder and comment. > $ git apply --check

RE: Data is copied twice when specifying both child and parent table in publication

2022-09-25 Thread osumi.takami...@fujitsu.com
On Tuesday, September 20, 2022 3:18 PM Wang, Wei/王 威 wrote: > Rebased the patch based on the changes in HEAD (20b6847). > Attach the new patches. Hi, thank you for updating the patchset. FYI, I noticed that the patch for head is no longer applicable. $ git apply --check

RE: Data is copied twice when specifying both child and parent table in publication

2022-09-20 Thread wangw.f...@fujitsu.com
On Mon, Sept 19, 2022 at 14:52 PM Peter Smith wrote: > FYI, I'm not sure why the cfbot hasn't reported this, but the apply v9 > patch failed for me on HEAD as below: > > [postgres@CentOS7-x64 oss_postgres_misc]$ git apply > ../patches_misc/HEAD_v9-0001-Fix-data-replicated-twice-when-specifying-

Re: Data is copied twice when specifying both child and parent table in publication

2022-09-19 Thread Peter Smith
FYI, I'm not sure why the cfbot hasn't reported this, but the apply v9 patch failed for me on HEAD as below: [postgres@CentOS7-x64 oss_postgres_misc]$ git apply ../patches_misc/HEAD_v9-0001-Fix-data-replicated-twice-when-specifying-publish.patch --verbose Checking patch

RE: Data is copied twice when specifying both child and parent table in publication

2022-08-30 Thread wangw.f...@fujitsu.com
On Tues, Aug 9, 2022 at 15:15 PM Peter Smith wrote: > Here are some review comment for the HEAD_v8 patch: Thanks for your comments. > 1. Commit message > > If there are two publications, one of them publish a parent table with > (publish_via_partition_root = true) and another publish child

RE: Data is copied twice when specifying both child and parent table in publication

2022-08-29 Thread houzj.f...@fujitsu.com
On Wednesday, August 10, 2022 7:45 AM Peter Smith wrote: > > Here are some more review comments for the HEAD_v8 patch: > > == > > 1. Commit message > > If there are two publications, one of them publish a parent table with > (publish_via_partition_root = true) and another publish child

Re: Data is copied twice when specifying both child and parent table in publication

2022-08-09 Thread Peter Smith
Here are some more review comments for the HEAD_v8 patch: == 1. Commit message If there are two publications, one of them publish a parent table with (publish_via_partition_root = true) and another publish child table, subscribing to both publications from one subscription results in two

Re: Data is copied twice when specifying both child and parent table in publication

2022-08-09 Thread Peter Smith
Here are some review comment for the HEAD_v8 patch: == 1. Commit message If there are two publications, one of them publish a parent table with (publish_via_partition_root = true) and another publish child table, subscribing to both publications from one subscription results in two initial

RE: Data is copied twice when specifying both child and parent table in publication

2022-08-05 Thread wangw.f...@fujitsu.com
On Thur, Jul 28, 2022 at 17:17 PM Peter Smith wrote: > Here are some review comments for the HEAD_v7-0001 patch: Thanks for your comments. > 2. Commit message. > > 2a. > > If there are two publications that publish the parent table and the child > table > separately, and both specify the

RE: Data is copied twice when specifying both child and parent table in publication

2022-08-04 Thread houzj.f...@fujitsu.com
On Thursday, July 28, 2022 5:17 PM Peter Smith wrote: > Here are some review comments for the HEAD_v7-0001 patch: > > == > > 1. > > I have a fundamental question about this patch. > > IIUC the purpose of this patch is to ensure that (when > publish_via_root = true) the copy of the

Re: Data is copied twice when specifying both child and parent table in publication

2022-07-28 Thread Peter Smith
Here are some review comments for the HEAD_v7-0001 patch: == 1. I have a fundamental question about this patch. IIUC the purpose of this patch is to ensure that (when publish_via_root = true) the copy of the partition data will happen only once (e.g. from one parent table on one of the

RE: Data is copied twice when specifying both child and parent table in publication

2022-07-21 Thread wangw.f...@fujitsu.com
On Thur, Jul 14, 2022 at 12:46 PM Peter Smith wrote: > Here are some review comments for the v6 patch (HEAD only): Thanks for your comments. > 1. Commit message > > If there are two publications that publish the parent table and the child > table > separately, and both specify the option

Re: Data is copied twice when specifying both child and parent table in publication

2022-07-13 Thread Peter Smith
Here are some review comments for the v6 patch (HEAD only): HEAD_v6-0001 1. Commit message If there are two publications that publish the parent table and the child table separately, and both specify the option PUBLISH_VIA_PARTITION_ROOT, subscribing to both

RE: Data is copied twice when specifying both child and parent table in publication

2022-07-01 Thread wangw.f...@fujitsu.com
On Wed, May 18, 2022 4:51 PM I wrote: > Attach the new patch. Since there are some new commits in HEAD (0ff20288, fd0b9dc and 52b5c53) that improve the functions pg_get_publication_tables and fetch_table_list, we cannot apply the patch cleanly. Therefore, I rebased the patch based on the changes

RE: Data is copied twice when specifying both child and parent table in publication

2022-05-18 Thread wangw.f...@fujitsu.com
On Wed, May 18, 2022 4:38 PM I wrote: > Attach the patches.(Only changed the patch for HEAD.) Sorry, I forgot to update commit message. Attach the new patch. 1. Only update the commit message for HEAD_v5. Regards, Wang wei HEAD_v5-0001-Fix-data-replicated-twice-when-specifying-PUBLISH.patch

RE: Data is copied twice when specifying both child and parent table in publication

2022-05-18 Thread wangw.f...@fujitsu.com
On Fri, May 13, 2022 10:57 PM Osumi, Takamichi/大墨 昂道 wrote: > On Friday, May 13, 2022 6:42 PM Wang, Wei/王 威 > wrote: > > Attach the patches.(Only changed the patch for HEAD.). > > 1. Optimize the code. Reduce calls to function filter_partitions. > > [suggestions by Amit-san] 2. Improve the

RE: Data is copied twice when specifying both child and parent table in publication

2022-05-18 Thread wangw.f...@fujitsu.com
On Tues, May 17, 2022 9:03 PM Amit Kapila wrote: > On Fri, May 13, 2022 at 3:11 PM wangw.f...@fujitsu.com > wrote: > > > > Attach the patches.(Only changed the patch for HEAD.). > > Thanks for your comments. > # publications > -{ oid => '6119', descr => 'get OIDs of tables in a publication', >

Re: Data is copied twice when specifying both child and parent table in publication

2022-05-17 Thread Amit Kapila
On Fri, May 13, 2022 at 3:11 PM wangw.f...@fujitsu.com wrote: > > Attach the patches.(Only changed the patch for HEAD.). > # publications -{ oid => '6119', descr => 'get OIDs of tables in a publication', +{ oid => '6119', descr => 'get OIDs of tables in one or more publications', proname =>

RE: Data is copied twice when specifying both child and parent table in publication

2022-05-13 Thread osumi.takami...@fujitsu.com
On Friday, May 13, 2022 6:42 PM Wang, Wei/王 威 wrote: > Attach the patches.(Only changed the patch for HEAD.). > 1. Optimize the code. Reduce calls to function filter_partitions. > [suggestions by > Amit-san] 2. Improve the alias name in SQL. [suggestions by Amit-san] 3. > Improve coding

RE: Data is copied twice when specifying both child and parent table in publication

2022-05-13 Thread wangw.f...@fujitsu.com
On Fri, May 13, 2022 1:59 PM Amit Kapila wrote: > On Fri, May 13, 2022 at 7:32 AM wangw.f...@fujitsu.com > wrote: > > > > Attach the patches.(Only changed the patch for HEAD.). > > > > Few comments: > = Thanks for your comments. > 1. > @@ -1135,6 +1172,15 @@

Re: Data is copied twice when specifying both child and parent table in publication

2022-05-12 Thread Amit Kapila
On Fri, May 13, 2022 at 7:32 AM wangw.f...@fujitsu.com wrote: > > Attach the patches.(Only changed the patch for HEAD.). > Few comments: = 1. @@ -1135,6 +1172,15 @@ pg_get_publication_tables(PG_FUNCTION_ARGS) if (publication->pubviaroot) tables = filter_partitions(tables); } +

RE: Data is copied twice when specifying both child and parent table in publication

2022-05-12 Thread wangw.f...@fujitsu.com
On Thur, May 12, 2022 9:48 AM osumi.takami...@fujitsu.com wrote: > On Wednesday, May 11, 2022 11:33 AM I wrote: > > On Monday, May 9, 2022 10:51 AM wangw.f...@fujitsu.com > > wrote: > > > Attach new patches. > > > The patch for HEAD: > > > 1. Modify the approach. Enhance the API of function > >

RE: Data is copied twice when specifying both child and parent table in publication

2022-05-11 Thread osumi.takami...@fujitsu.com
On Wednesday, May 11, 2022 11:33 AM I wrote: > On Monday, May 9, 2022 10:51 AM wangw.f...@fujitsu.com > wrote: > > Attach new patches. > > The patch for HEAD: > > 1. Modify the approach. Enhance the API of function > > pg_get_publication_tables to handle one publication or an array of > >

RE: Data is copied twice when specifying both child and parent table in publication

2022-05-10 Thread osumi.takami...@fujitsu.com
On Monday, May 9, 2022 10:51 AM wangw.f...@fujitsu.com wrote: > Attach new patches. > The patch for HEAD: > 1. Modify the approach. Enhance the API of function > pg_get_publication_tables to handle one publication or an array of > publications. > The patch for REL14: > 1. Improve the table sync

RE: Data is copied twice when specifying both child and parent table in publication

2022-05-08 Thread wangw.f...@fujitsu.com
On Tue, Apr 28, 2022 9:22 AM Shi, Yu/侍 雨 wrote: > Thanks for your patches. > > Here's a comment on the patch for REL14. Thanks for your comments. > + appendStringInfo(, "SELECT DISTINCT ns.nspname, c.relname\n" > + " FROM >

RE: Data is copied twice when specifying both child and parent table in publication

2022-04-27 Thread shiy.f...@fujitsu.com
On Sun, Apr 24, 2022 2:16 PM Wang, Wei/王 威 wrote: > > Attach the new patches.[suggestions by Amit-San] > The patch for HEAD: > 1. Add a new function to get tables info by a publications array. > The patch for REL14: > 1. Use an alias to make the statement understandable. BTW, I adjusted the >

RE: Data is copied twice when specifying both child and parent table in publication

2022-04-24 Thread wangw.f...@fujitsu.com
On Sun, Apr 24, 2022 at 2:16 PM I wrote: > On Thur, Apr 21, 2022 at 5:41 PM Amit Kapila wrote: > > IIRC, the column list and row filter also have some issues exactly due to > > this > > reason, so, I would like those cases to be also mentioned here and probably > > include the tests for them in

RE: Data is copied twice when specifying both child and parent table in publication

2022-04-24 Thread wangw.f...@fujitsu.com
On Thur, Apr 21, 2022 at 5:41 PM Amit Kapila wrote: > Thanks for your comments. > On Tue, Apr 19, 2022 at 2:23 PM shiy.f...@fujitsu.com > wrote: > > > > On Tue, Apr 19, 2022 3:05 PM houzj.f...@fujitsu.com > wrote: > > > > > > One suggestion is that can we simplify the code by moving the logic

Re: Data is copied twice when specifying both child and parent table in publication

2022-04-21 Thread Amit Kapila
On Tue, Apr 19, 2022 at 2:23 PM shiy.f...@fujitsu.com wrote: > > On Tue, Apr 19, 2022 3:05 PM houzj.f...@fujitsu.com > wrote: > > > > One suggestion is that can we simplify the code by moving the logic of > > checking > > the ancestor into the SQL ?. For example, we could filter the outpout of

RE: Data is copied twice when specifying both child and parent table in publication

2022-04-20 Thread wangw.f...@fujitsu.com
On Tue, Apr 19, 2022 4:53 PM Shi, Yu/侍 雨 wrote: > On Tue, Apr 19, 2022 3:05 PM houzj.f...@fujitsu.com > wrote: > > > > > -Original Message- > > > From: Wang, Wei/王 威 > > On Thursday, April 7, 2022 11:08 AM > > > > > > On Thur, Mar 10, 2021 at 10:08 AM houzj.f...@fujitsu.com wrote: > > >

RE: Data is copied twice when specifying both child and parent table in publication

2022-04-19 Thread shiy.f...@fujitsu.com
On Tue, Apr 19, 2022 3:05 PM houzj.f...@fujitsu.com wrote: > > > -Original Message- > > From: Wang, Wei/王 威 > On Thursday, April 7, 2022 11:08 AM > > > > On Thur, Mar 10, 2021 at 10:08 AM houzj.f...@fujitsu.com wrote: > > > Hi, > > > > > > When reviewing some logical replication related

RE: Data is copied twice when specifying both child and parent table in publication

2022-04-19 Thread houzj.f...@fujitsu.com
> -Original Message- > From: Wang, Wei/王 威 On Thursday, April 7, 2022 11:08 AM > > On Thur, Mar 10, 2021 at 10:08 AM houzj.f...@fujitsu.com wrote: > > Hi, > > > > When reviewing some logical replication related features. I noticed another > > possible problem if the subscriber

RE: Data is copied twice when specifying both child and parent table in publication

2022-04-06 Thread wangw.f...@fujitsu.com
On Thur, Mar 10, 2021 at 10:08 AM houzj.f...@fujitsu.com wrote: > Hi, > > When reviewing some logical replication related features. I noticed another > possible problem if the subscriber subscribes multiple publications which > publish parent and child table. > > For example: > > pub >

RE: Data is copied twice when specifying both child and parent table in publication

2022-03-09 Thread houzj.f...@fujitsu.com
Hi, When reviewing some logical replication related features. I noticed another possible problem if the subscriber subscribes multiple publications which publish parent and child table. For example: pub create table t (a int, b int, c int) partition by range (a); create table t_1 partition

  1   2   >