Re: Support logical replication of DDLs

2024-03-28 Thread Amit Kapila
On Thu, Mar 28, 2024 at 5:31 PM Andrey M. Borodin wrote: > > This thread is registered on CF [0] but is not active since 2023. Is anyone > working on this? I understand that this is a nice feature. Should we move it > to next CF or withdraw CF entry? > At this stage, we should close either

Re: Support logical replication of DDLs

2024-03-28 Thread Andrey M. Borodin
> On 18 Jul 2023, at 12:09, Zhijie Hou (Fujitsu) wrote: > > Here is the POC patch(0004) for the second approach Hi everyone! This thread is registered on CF [0] but is not active since 2023. Is anyone working on this? I understand that this is a nice feature. Should we move it to next CF

Re: Support logical replication of DDLs

2023-07-17 Thread Michael Paquier
On Tue, Jul 18, 2023 at 02:28:08PM +0900, Masahiko Sawada wrote: > I've considered some alternative approaches but I prefer the second > approach. A long test time could not be a big problem unless we run it > by default. We can prepare a buildfarm animal that is configured to > run the DDL

Re: Support logical replication of DDLs

2023-07-17 Thread Masahiko Sawada
On Tue, Jul 11, 2023 at 8:01 PM Zhijie Hou (Fujitsu) wrote: > > Hi, > > We have been researching how to create a test that detects failures resulting > from future syntax changes, where the deparser fails to update properly. > > The basic idea comes from what Robert Haas suggested in [1]: when

Re: Support logical replication of DDLs

2023-07-14 Thread Amit Kapila
On Tue, Jul 11, 2023 at 4:31 PM Zhijie Hou (Fujitsu) wrote: > > We have been researching how to create a test that detects failures resulting > from future syntax changes, where the deparser fails to update properly. > > The basic idea comes from what Robert Haas suggested in [1]: when running

RE: Support logical replication of DDLs

2023-07-10 Thread Zhijie Hou (Fujitsu)
On Monday, July 10, 2023 3:22 AM Zheng Li wrote: > > On Tue, Jun 27, 2023 at 6:16 AM vignesh C wrote: > > > While development, below are some of the challenges we faced: > > > 1. Almost all the members of the AlterTableType enum will have to be > annotated. > > 2. Complex functionalities

Re: Support logical replication of DDLs

2023-07-09 Thread Zheng Li
On Tue, Jun 27, 2023 at 6:16 AM vignesh C wrote: > While development, below are some of the challenges we faced: > 1. Almost all the members of the AlterTableType enum will have to be > annotated. > 2. Complex functionalities which require access to catalog tables > cannot be auto generated,

Re: Support logical replication of DDLs

2023-06-22 Thread Amit Kapila
On Tue, Jun 13, 2023 at 1:21 PM Michael Paquier wrote: > > The patch is made of a lot of one-one mapping between enum structures > and hardcoded text used in the JSON objects, making it something hard > to maintain if a node field is added, removed or even updated into > something else. I have

Re: Support logical replication of DDLs

2023-06-22 Thread shveta malik
On Wed, Jun 21, 2023 at 6:38 PM Jelte Fennema wrote: > > (to be clear I only skimmed the end of this thread and did not look at > all the previous messages) > > I took a quick look at the first patch (about deparsing table ddl) and > it seems like this would also be very useful for a SHOW CREATE

Re: Support logical replication of DDLs

2023-06-21 Thread shveta malik
On Mon, Jun 12, 2023 at 7:17 AM Wei Wang (Fujitsu) wrote: > > On Thur, Jun 8, 2023 20:02 PM shveta malik wrote: > > Thank You Vignesh for handling (a), Ajin for handling (b), Shi-san and > > Hou-san for contributing in (c). > > > > The new changes are in patch 0001, 0002, 0005 and 0008. > >

Re: Support logical replication of DDLs

2023-06-21 Thread Jelte Fennema
(to be clear I only skimmed the end of this thread and did not look at all the previous messages) I took a quick look at the first patch (about deparsing table ddl) and it seems like this would also be very useful for a SHOW CREATE TABLE, like command. Which was suggested in this thread:

Re: Support logical replication of DDLs

2023-06-19 Thread Amit Kapila
On Fri, Jun 16, 2023 at 4:01 PM shveta malik wrote: > > With these changes, I hope the patch-set is somewhat easier to review. > Few comments: = 1. +static Jsonb * +deparse_CreateStmt(Oid objectId, Node *parsetree) { ... + /* PERSISTENCE */ + appendStringInfoString(, "CREATE

Re: Support logical replication of DDLs

2023-06-18 Thread shveta malik
As per suggestion by Amit, reviewed two more formats to be used for DDL's WAL-logging purpose, analysis below: NodeToString: I do not think it is a good idea to use NodeToString in DDL Rep for reasons below: 1) It consists of too much internal and not-needed information. 2) Too large to be logged

Re: Support logical replication of DDLs

2023-06-13 Thread Michael Paquier
On Tue, Jun 13, 2023 at 06:49:42PM +0530, Amit Kapila wrote: > We have to choose one of the approaches between 0001 and 0008. I feel > we don't need an intermediate ObjTree representation as that adds > overhead and an additional layer which is not required. As mentioned > in my previous email I

Re: Support logical replication of DDLs

2023-06-13 Thread Amit Kapila
On Tue, Jun 13, 2023 at 1:21 PM Michael Paquier wrote: > > On Mon, Jun 12, 2023 at 01:47:02AM +, Wei Wang (Fujitsu) wrote: > > # load test cases from the regression tests > > -my @regress_tests = split /\s+/, $ENV{REGRESS}; > > +#my @regress_tests = split /\s+/, $ENV{REGRESS}; > > +my

Re: Support logical replication of DDLs

2023-06-13 Thread Michael Paquier
On Mon, Jun 12, 2023 at 01:47:02AM +, Wei Wang (Fujitsu) wrote: > # load test cases from the regression tests > -my @regress_tests = split /\s+/, $ENV{REGRESS}; > +#my @regress_tests = split /\s+/, $ENV{REGRESS}; > +my @regress_tests = ("create_type", "create_schema", "create_rule", >

Re: Support logical replication of DDLs

2023-06-12 Thread Amit Kapila
On Thu, Jun 8, 2023 at 5:32 PM shveta malik wrote: > > On Thu, Jun 8, 2023 at 10:36 AM Amit Kapila wrote: > > Please find new set of patches addressing below: > a) Issue mentioned by Wang-san in [1], > b) Comments from Peter given in [2] > c) Comments from Amit given in the last 2 emails. > As

RE: Support logical replication of DDLs

2023-06-11 Thread Wei Wang (Fujitsu)
On Thur, Jun 8, 2023 20:02 PM shveta malik wrote: > Thank You Vignesh for handling (a), Ajin for handling (b), Shi-san and > Hou-san for contributing in (c). > > The new changes are in patch 0001, 0002, 0005 and 0008. Thanks for updating the patch set. Here are some comments: === For 0002

Re: Support logical replication of DDLs

2023-06-09 Thread Ajin Cherian
On Thu, Jun 8, 2023 at 10:02 PM shveta malik wrote: > Please find new set of patches addressing below: > a) Issue mentioned by Wang-san in [1], > b) Comments from Peter given in [2] > c) Comments from Amit given in the last 2 emails. > > [1]: >

Re: Support logical replication of DDLs

2023-06-08 Thread Amit Kapila
On Fri, Jun 9, 2023 at 5:35 AM Masahiko Sawada wrote: > > On Thu, Jun 8, 2023 at 9:12 PM shveta malik wrote: > > > > > > Please find new set of patches addressing below: > > a) Issue mentioned by Wang-san in [1], > > b) Comments from Peter given in [2] > > c) Comments from Amit given in the last

Re: Support logical replication of DDLs

2023-06-08 Thread shveta malik
On Thu, Jun 8, 2023 at 5:31 PM shveta malik wrote: > > Please find new set of patches addressing below: > a) Issue mentioned by Wang-san in [1], > b) Comments from Peter given in [2] > c) Comments from Amit given in the last 2 emails. > > [1]: >

Re: Support logical replication of DDLs

2023-06-08 Thread Masahiko Sawada
Hi, On Thu, Jun 8, 2023 at 9:12 PM shveta malik wrote: > > > Please find new set of patches addressing below: > a) Issue mentioned by Wang-san in [1], > b) Comments from Peter given in [2] > c) Comments from Amit given in the last 2 emails. > > [1]: >

Re: Support logical replication of DDLs

2023-06-08 Thread shveta malik
On Tue, Jun 6, 2023 at 11:31 AM Wei Wang (Fujitsu) wrote: > > On Thur, June 1, 2023 at 23:42 vignesh C wrote: > > On Wed, 31 May 2023 at 14:32, Wei Wang (Fujitsu) > > wrote: > > > ~~~ > > > > > > 2. Deparsed results of the partition table. > > > When I run the following SQLs: > > > ``` > > >

Re: Support logical replication of DDLs

2023-06-08 Thread shveta malik
On Tue, Jun 6, 2023 at 4:26 PM Amit Kapila wrote: > > On Mon, Jun 5, 2023 at 3:00 PM shveta malik wrote: > > > > Few assorted comments: Hi Amit, thanks for the feedback. Addressed these in recent patch posted (*2023_06_08.patch) > === > 1. I see the following text in 0005

Re: Support logical replication of DDLs

2023-06-07 Thread Amit Kapila
On Tue, Jun 6, 2023 at 4:26 PM Amit Kapila wrote: > > > Few assorted comments: > === > Few comments on 0008* patch: == 1. After 0008*, deparse_CreateStmt(), forms dpcontext before forming identity whereas it is required only after it. It may not matter

Re: Support logical replication of DDLs

2023-06-06 Thread Amit Kapila
On Mon, Jun 5, 2023 at 3:00 PM shveta malik wrote: > Few assorted comments: === 1. I see the following text in 0005 patch: "It supports most of ALTER TABLE command except some commands(DDL related to PARTITIONED TABLE ...) that are recently introduced but are not yet supported by

Re: Support logical replication of DDLs

2023-06-06 Thread vignesh C
On Thu, 1 Jun 2023 at 07:42, Yu Shi (Fujitsu) wrote: > > On Wed, May 31, 2023 5:41 PM shveta malik wrote: > > > > PFA the set of patches consisting above changes. All the changes are > > made in 0008 patch. > > > > Apart from above changes, many partition attach/detach related tests > > are

RE: Support logical replication of DDLs

2023-06-06 Thread Wei Wang (Fujitsu)
On Thur, June 1, 2023 at 23:42 vignesh C wrote: > On Wed, 31 May 2023 at 14:32, Wei Wang (Fujitsu) > wrote: > > ~~~ > > > > 2. Deparsed results of the partition table. > > When I run the following SQLs: > > ``` > > create table parent (a int primary key) partition by range (a); > > create table

RE: Support logical replication of DDLs

2023-06-01 Thread Yu Shi (Fujitsu)
On Wed, May 31, 2023 5:41 PM shveta malik wrote: > > On Mon, May 29, 2023 at 11:45 AM Yu Shi (Fujitsu) > wrote: > > > > 0008 patch > > - > > 4. > > case AT_AddColumn: > > /* XXX need to set the "recurse" bit > > somewhere? */ > >

RE: Support logical replication of DDLs

2023-05-31 Thread Yu Shi (Fujitsu)
On Wed, May 31, 2023 5:41 PM shveta malik wrote: > > PFA the set of patches consisting above changes. All the changes are > made in 0008 patch. > > Apart from above changes, many partition attach/detach related tests > are uncommented in alter_table.sql in patch 0008. > Thanks for updating

RE: Support logical replication of DDLs

2023-05-31 Thread Wei Wang (Fujitsu)
On Tues, May 30, 2023 at 19:19 PM vignesh C wrote: > The attached patch has the changes for the above. Thanks for updating the new patch set. Here are some comments: === For patch 0001 1. In the function deparse_Seq_As. ``` + if (OidIsValid(seqdata->seqtypid)) +

Re: Support logical replication of DDLs

2023-05-29 Thread shveta malik
On Mon, May 29, 2023 at 6:16 PM vignesh C wrote: > > > > > I found few comments while making some changes to the patch: > > 1) Now that objtree is removed, these comments should be modified: > > * Deparse object tree is created by using: > > * a) new_objtree("know contents") where the complete

RE: Support logical replication of DDLs

2023-05-29 Thread Yu Shi (Fujitsu)
On Mon, May 22, 2023 1:57 PM shveta malik wrote: > > Please find the new set of patches for object-tree Removal. The new > changes are in patch 0008 only. The new changes address object tree > removal for below commands. > > create sequence > alter sequence > alter object owner to > alter

Re: Support logical replication of DDLs

2023-05-23 Thread vignesh C
On Mon, 22 May 2023 at 11:27, shveta malik wrote: > > On Wed, May 17, 2023 at 4:45 PM vignesh C wrote: > > > > On Wed, 17 May 2023 at 15:41, shveta malik wrote: > > > > > > On Fri, May 12, 2023 at 12:03 PM shveta malik > > > wrote: > > > > > > > > On Tue, May 9, 2023 at 4:23 PM shveta malik

Re: Support logical replication of DDLs

2023-05-08 Thread shveta malik
On Mon, May 8, 2023 at 3:58 PM shveta malik wrote: > > On Tue, May 2, 2023 at 8:30 AM shveta malik wrote: > > > > On Fri, Apr 28, 2023 at 5:11 PM Amit Kapila wrote: > > > > > > Now, I think we can try to eliminate this entire ObjTree machinery and > > > directly from the JSON blob during

Re: Support logical replication of DDLs

2023-05-05 Thread Peter Smith
I revisited the 0005 patch to verify all changes made to address my previous review comments [1][2][3][4] were OK. Not all changes were made quite as expected, and there were a few other things I noticed in passing. == 1. General I previously [1] wrote a comment: Use consistent uppercase

Re: Support logical replication of DDLs

2023-05-03 Thread Alvaro Herrera
Patch 0001 adds a new event trigger type that can be fired, but it's missing documentation and its own tests. (I think part of the docs are in 0002, but that seems to be only the changes to the supported operations table, without any other explanation for it in sect1 event-trigger-definition, and

Re: Support logical replication of DDLs

2023-05-01 Thread shveta malik
On Fri, Apr 28, 2023 at 5:11 PM Amit Kapila wrote: > > On Tue, Apr 25, 2023 at 9:28 AM Zhijie Hou (Fujitsu) > wrote: > > > > I have a few high-level comments on the deparsing approach used in the > patch. As per my understanding, we first build an ObjTree from the DDL > command, then convert the

Re: Support logical replication of DDLs

2023-04-28 Thread Amit Kapila
On Tue, Apr 25, 2023 at 9:28 AM Zhijie Hou (Fujitsu) wrote: > I have a few high-level comments on the deparsing approach used in the patch. As per my understanding, we first build an ObjTree from the DDL command, then convert the ObjTree to Jsonb which is then converted to a JSON string. Now,

RE: Support logical replication of DDLs

2023-04-26 Thread Zhijie Hou (Fujitsu)
On Wednesday, April 26, 2023 2:32 PM Masahiko Sawada > Subject: Re: Support logical replication of DDLs > > On Tue, Mar 28, 2023 at 3:22 PM houzj.f...@fujitsu.com > wrote: > > > > On Tuesday, March 28, 2023 1:41 PM Amit Kapila > wrote: > > > > > >

Re: Support logical replication of DDLs

2023-04-26 Thread vignesh C
On Wed, 26 Apr 2023 at 12:02, Masahiko Sawada wrote: > > On Tue, Mar 28, 2023 at 3:22 PM houzj.f...@fujitsu.com > wrote: > > > > On Tuesday, March 28, 2023 1:41 PM Amit Kapila > > wrote: > > > > > > On Mon, Mar 27, 2023 at 5:37 PM Amit Kapila > > > wrote: > > > > > > > > On Mon, Mar 27, 2023

Re: Support logical replication of DDLs

2023-04-26 Thread Amit Kapila
On Wed, Apr 26, 2023 at 12:01 PM Masahiko Sawada wrote: > > On Wed, Apr 26, 2023 at 2:56 PM Amit Kapila wrote: > > > > On Wed, Apr 26, 2023 at 10:01 AM Masahiko Sawada > > wrote: > > > > > > On Tue, Apr 25, 2023 at 12:58 PM Zhijie Hou (Fujitsu) > > > wrote: > > > > > > > > Aport from above

Re: Support logical replication of DDLs

2023-04-26 Thread Masahiko Sawada
On Tue, Mar 28, 2023 at 3:22 PM houzj.f...@fujitsu.com wrote: > > On Tuesday, March 28, 2023 1:41 PM Amit Kapila > wrote: > > > > On Mon, Mar 27, 2023 at 5:37 PM Amit Kapila > > wrote: > > > > > > On Mon, Mar 27, 2023 at 12:07 PM Amit Kapila > > wrote: > > > > > > > > On Mon, Mar 27, 2023 at

Re: Support logical replication of DDLs

2023-04-26 Thread Masahiko Sawada
On Wed, Apr 26, 2023 at 2:56 PM Amit Kapila wrote: > > On Wed, Apr 26, 2023 at 10:01 AM Masahiko Sawada > wrote: > > > > On Tue, Apr 25, 2023 at 12:58 PM Zhijie Hou (Fujitsu) > > wrote: > > > > > > Aport from above comments, I splitted the code related to verbose > > > mode to a separate

Re: Support logical replication of DDLs

2023-04-25 Thread Amit Kapila
On Wed, Apr 26, 2023 at 10:01 AM Masahiko Sawada wrote: > > On Tue, Apr 25, 2023 at 12:58 PM Zhijie Hou (Fujitsu) > wrote: > > > > Aport from above comments, I splitted the code related to verbose > > mode to a separate patch. And here is the new version patch set. > > > > As for DDL

Re: Support logical replication of DDLs

2023-04-25 Thread Masahiko Sawada
On Tue, Apr 25, 2023 at 12:58 PM Zhijie Hou (Fujitsu) wrote: > > Aport from above comments, I splitted the code related to verbose > mode to a separate patch. And here is the new version patch set. > As for DDL replication, we create event triggers to write deparsed DDL commands to WAL when

Re: Support logical replication of DDLs

2023-04-23 Thread Amit Kapila
On Mon, Apr 24, 2023 at 8:18 AM shveta malik wrote: > > On Thu, Apr 20, 2023 at 3:40 PM Amit Kapila wrote: > > > > On Thu, Apr 20, 2023 at 9:11 AM shveta malik wrote: > > > > > > > > > Few comments for ddl_deparse.c in patch dated April17: > > > > > > > > 6) There are plenty of places where we

Re: Support logical replication of DDLs

2023-04-23 Thread shveta malik
On Thu, Apr 20, 2023 at 3:40 PM Amit Kapila wrote: > > On Thu, Apr 20, 2023 at 9:11 AM shveta malik wrote: > > > > > > Few comments for ddl_deparse.c in patch dated April17: > > > > > 6) There are plenty of places where we use 'append_not_present' > > without using 'append_null_object'. > > Do

Re: Support logical replication of DDLs

2023-04-21 Thread Amit Kapila
On Thu, Apr 20, 2023 at 6:09 PM shveta malik wrote: > > > Few more comments, mainly for event_trigger.c in the patch dated April17: > > 1)EventTriggerCommonSetup() > +pub_funcoid = LookupFuncName(pubfuncname, 0, NULL, true); > > This is the code where we have special handling for

Re: Support logical replication of DDLs

2023-04-20 Thread Amit Kapila
On Thu, Apr 20, 2023 at 2:28 PM shveta malik wrote: > > Few comments for ddl_json.c in the patch dated April17: > ... > > 3) expand_jsonb_array() > arrayelem is allocated as below, but not freed. > initStringInfo() > > 4) expand_jsonb_array(), > we initialize iterator as below which internally

Re: Support logical replication of DDLs

2023-04-20 Thread Peter Smith
Here are some more review comments for the patch 0002-2023_04_07-2 This was a WIP review in parts because the patch was quite large: WIP part 1 [1] was posted 17/4. WIP part 2 [2] was posted 17/4. WIP part 3 [3] was posted 19/4. WIP part 4 is this post. (This is my final WIP part for this 0002

Re: Support logical replication of DDLs

2023-04-20 Thread shveta malik
On Thu, Apr 20, 2023 at 2:28 PM shveta malik wrote: > > On Thu, Apr 20, 2023 at 9:11 AM shveta malik wrote: >> >> On Mon, Apr 17, 2023 at 5:32 PM Zhijie Hou (Fujitsu) >> wrote: >> > >> > Attach the new version patch set which include the following changes: >> Few comments for ddl_deparse.c in

Re: Support logical replication of DDLs

2023-04-20 Thread Amit Kapila
On Thu, Apr 20, 2023 at 9:11 AM shveta malik wrote: > > > Few comments for ddl_deparse.c in patch dated April17: > > 1) append_format_string() > I think we need to have 'Assert(sub_fmt)' here like we have it in all > other similar functions (append_bool_object, append_object_object, > ...) > +1.

Re: Support logical replication of DDLs

2023-04-20 Thread shveta malik
On Thu, Apr 20, 2023 at 9:11 AM shveta malik wrote: > On Mon, Apr 17, 2023 at 5:32 PM Zhijie Hou (Fujitsu) > wrote: > > > > Attach the new version patch set which include the following changes: > > > > Few comments for ddl_deparse.c in patch dated April17: > > Few comments for ddl_json.c in

Re: Support logical replication of DDLs

2023-04-19 Thread shveta malik
On Mon, Apr 17, 2023 at 5:32 PM Zhijie Hou (Fujitsu) wrote: > > On Monday, April 10, 2023 7:20 PM Amit Kapila wrote: > > > > On Fri, Apr 7, 2023 at 8:52 AM houzj.f...@fujitsu.com > > wrote: > > > > > > Sorry, there was a miss when rebasing the patch which could cause the > > > CFbot to fail and

Re: Support logical replication of DDLs

2023-04-19 Thread Peter Smith
Here are some more WIP review comments for the patch 0002-2023_04_07-2 This is a WIP review in parts because the patch was quite large, so it is taking a while... WIP part 1 [1] was posted 17/4. WIP part 2 [2] was posted 17/4. This is WIP part 3 == doc/src/sgml/logical-replication.sgml

Re: Support logical replication of DDLs

2023-04-17 Thread Peter Smith
Here are some more review comments for the patch 0002-2023_04_07-2 Note: This is a WIP review (part 2); the comments in this post are only for the commit message and the PG docs == Commit message 1. It's not obvious that those "-" (like "- For DROP object:") represent major sections of this

RE: Support logical replication of DDLs

2023-04-17 Thread Zhijie Hou (Fujitsu)
On Wednesday, April 12, 2023 7:24 PM Amit Kapila wrote: > > On Fri, Apr 7, 2023 at 8:52 AM houzj.f...@fujitsu.com > wrote: > > > > Few comments on 0001 Thanks for the comments. > === > 1. > + ConstrObjDomain, > + ConstrObjForeignTable > +} ConstraintObjType; > > These both

Re: Support logical replication of DDLs

2023-04-17 Thread vignesh C
On Sat, 15 Apr 2023 at 06:38, vignesh C wrote: > > On Fri, 14 Apr 2023 at 13:06, vignesh C wrote: > > > > Few comments: Few more comments: 1) since missing_ok is passed as false, if there is an error the error will be handled in find_string_in_jsonbcontainer, "missing operator name" handling

Re: Support logical replication of DDLs

2023-04-17 Thread Peter Smith
Hi, here are some review comments for the patch 0002-2023_04_07-2 Note: This is a WIP review. The patch is quite large and I have managed to only look at ~50% of it. I will continue reviewing this same 0002-2023_04_07-2 and send more comments at a later time. Meanwhile, here are the review

Re: Support logical replication of DDLs

2023-04-14 Thread vignesh C
On Fri, 14 Apr 2023 at 13:06, vignesh C wrote: > > Few comments: Some more comments on 0001 patch: Few comments: 1) We could add a space after the 2nd parameter + * Note we don't have the luxury of sprintf-like compiler warnings for + * malformed argument lists. + */ +static ObjTree *

Re: Support logical replication of DDLs

2023-04-14 Thread vignesh C
On Fri, 7 Apr 2023 at 08:52, houzj.f...@fujitsu.com wrote: > > On Friday, April 7, 2023 11:13 AM houzj.f...@fujitsu.com > > > > > On Tuesday, April 4, 2023 7:35 PM shveta malik > > wrote: > > > > > > On Tue, Apr 4, 2023 at 8:43 AM houzj.f...@fujitsu.com > > > wrote: > > > > > > > Attach the

Re: Support logical replication of DDLs

2023-04-13 Thread Amit Kapila
On Wed, Apr 12, 2023 at 4:53 PM Amit Kapila wrote: > > > Few comments on 0001 > === > Some more comments on 0001 == 1. +/* + * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing. + * + * Given a table OID or domain OID, obtain its constraints and append

Re: Support logical replication of DDLs

2023-04-12 Thread Amit Kapila
On Fri, Apr 7, 2023 at 8:52 AM houzj.f...@fujitsu.com wrote: > Few comments on 0001 === 1. + ConstrObjDomain, + ConstrObjForeignTable +} ConstraintObjType; These both object types don't seem to be supported by the first patch. So, I don't see why these should be part of it. 2.

Re: Support logical replication of DDLs

2023-04-11 Thread Amit Kapila
On Mon, Apr 10, 2023 at 3:16 PM Zhijie Hou (Fujitsu) wrote: > > On Friday, April 7, 2023 11:23 amhouzj.f...@fujitsu.com > wrote: > > > > On Friday, April 7, 2023 11:13 AM houzj.f...@fujitsu.com > > > > > > > > On Tuesday, April 4, 2023 7:35 PM shveta malik > > > > > > wrote: > > > > > > > >

RE: Support logical replication of DDLs

2023-04-10 Thread Wei Wang (Fujitsu)
On Fri, Apr 7, 2023 11:23 AM Hou, Zhijie/侯 志杰 wrote: > Thanks for updating the patch set. Here are some comments: 1. The function deparse_drop_command in 0001 patch and the function publication_deparse_ddl_command_end in 0002 patch. ``` +/* + * Handle deparsing of DROP commands. + * + * Verbose

RE: Support logical replication of DDLs

2023-04-10 Thread Yu Shi (Fujitsu)
On Fri, Apr 7, 2023 11:23 AM houzj.f...@fujitsu.com wrote: > > On Friday, April 7, 2023 11:13 AM houzj.f...@fujitsu.com > > > > > On Tuesday, April 4, 2023 7:35 PM shveta malik > > wrote: > > > > > > On Tue, Apr 4, 2023 at 8:43 AM houzj.f...@fujitsu.com > > > wrote: > > > > > > > Attach the

Re: Support logical replication of DDLs

2023-04-10 Thread Amit Kapila
On Fri, Apr 7, 2023 at 8:52 AM houzj.f...@fujitsu.com wrote: > > Sorry, there was a miss when rebasing the patch which could cause the > CFbot to fail and here is the correct patch set. > I see the following note in the patch: "Note: For ATTACH/DETACH PARTITION, we haven't added extra logic on

RE: Support logical replication of DDLs

2023-04-10 Thread Zhijie Hou (Fujitsu)
On Friday, April 7, 2023 11:23 amhouzj.f...@fujitsu.com wrote: > > On Friday, April 7, 2023 11:13 AM houzj.f...@fujitsu.com > > > > > On Tuesday, April 4, 2023 7:35 PM shveta malik > > > > wrote: > > > > > > On Tue, Apr 4, 2023 at 8:43 AM houzj.f...@fujitsu.com > > > wrote: > > > > > > >

Re: Support logical replication of DDLs

2023-04-04 Thread shveta malik
On Tue, Apr 4, 2023 at 8:43 AM houzj.f...@fujitsu.com wrote: > Attach the new version patch set which did the following changes: > Hello, I tried below: pubnew=# ALTER PUBLICATION mypub2 SET (ddl = 'table'); ALTER PUBLICATION pubnew=# \dRp+ Publication

RE: Support logical replication of DDLs

2023-04-03 Thread houzj.f...@fujitsu.com
On Friday, March 31, 2023 6:31 AM Peter Smith wrote: Hi, > > It seems that lately, the patch attachments are lacking version numbers. It > causes unnecessary confusion. For example, I sometimes fetch patches from > this thread locally to "diff" them with previous patches to get a rough >

RE: Support logical replication of DDLs

2023-04-03 Thread Phil Florent
ian ; wangw.f...@fujitsu.com ; Runqi Tian ; Peter Smith ; Tom Lane ; li jie ; Dilip Kumar ; Alvaro Herrera ; Masahiko Sawada ; Japin Li ; rajesh singarapu ; Zheng Li ; PostgreSQL Hackers Objet : Re: Support logical replication of DDLs On Sun, Apr 2, 2023 at 3:25 PM Phil Florent wrote: >

Re: Support logical replication of DDLs

2023-04-02 Thread Amit Kapila
On Sun, Apr 2, 2023 at 3:25 PM Phil Florent wrote: > > As an end-user, I am highly interested in the patch > https://commitfest.postgresql.org/42/3595/ but I don't fully get its main > goal in its first version. > It's "for all tables" that will be implemented ? > If one needs a complete

Re: Support logical replication of DDLs

2023-03-30 Thread Peter Smith
It seems that lately, the patch attachments are lacking version numbers. It causes unnecessary confusion. For example, I sometimes fetch patches from this thread locally to "diff" them with previous patches to get a rough overview of the changes -- that has now become more difficult. Can you

Re: Support logical replication of DDLs

2023-03-30 Thread vignesh C
On Thu, 30 Mar 2023 at 13:29, houzj.f...@fujitsu.com wrote: > > > > > -Original Message- > > From: houzj.f...@fujitsu.com > > Sent: Thursday, March 30, 2023 2:37 PM > > > > On Tuesday, March 28, 2023 12:13 PM houzj.f...@fujitsu.com > > wrote: > > > > > > On Monday, March 27, 2023 8:08

Re: Support logical replication of DDLs

2023-03-30 Thread Amit Kapila
On Wed, Mar 29, 2023 at 10:23 PM Zheng Li wrote: > > On Wed, Mar 29, 2023 at 5:13 AM Amit Kapila wrote: > > > > On Wed, Mar 29, 2023 at 2:49 AM Zheng Li wrote: > > > > > > > > > I agree that a full fledged DDL deparser and DDL replication is too > > > big of a task for one patch. I think we may

Re: Support logical replication of DDLs

2023-03-30 Thread Amit Kapila
On Thu, Mar 30, 2023 at 3:16 PM vignesh C wrote: > > On Thu, 30 Mar 2023 at 13:29, houzj.f...@fujitsu.com > wrote: > > > > > > > > > -Original Message- > > > From: houzj.f...@fujitsu.com > > > Sent: Thursday, March 30, 2023 2:37 PM > > > > > > On Tuesday, March 28, 2023 12:13 PM

Re: Support logical replication of DDLs

2023-03-30 Thread vignesh C
On Thu, 30 Mar 2023 at 13:29, houzj.f...@fujitsu.com wrote: > > > > > -Original Message- > > From: houzj.f...@fujitsu.com > > Sent: Thursday, March 30, 2023 2:37 PM > > > > On Tuesday, March 28, 2023 12:13 PM houzj.f...@fujitsu.com > > wrote: > > > > > > On Monday, March 27, 2023 8:08

Re: Support logical replication of DDLs

2023-03-29 Thread Zheng Li
On Wed, Mar 29, 2023 at 5:13 AM Amit Kapila wrote: > > On Wed, Mar 29, 2023 at 2:49 AM Zheng Li wrote: > > > > > > I agree that a full fledged DDL deparser and DDL replication is too > > big of a task for one patch. I think we may consider approaching this > > feature in the following ways: > >

Re: Support logical replication of DDLs

2023-03-29 Thread Amit Kapila
On Wed, Mar 29, 2023 at 2:49 AM Zheng Li wrote: > > > I agree that a full fledged DDL deparser and DDL replication is too > big of a task for one patch. I think we may consider approaching this > feature in the following ways: > 1. Phased development and testing as discussed in other emails. >

Re: Support logical replication of DDLs

2023-03-28 Thread Zheng Li
On Sun, Mar 26, 2023 at 5:22 PM Tom Lane wrote: > I spent some time looking through this thread to try to get a sense > of the state of things, and I came away quite depressed. The patchset > has ballooned to over 2MB, which is a couple orders of magnitude > larger than anyone could hope to

Re: Support logical replication of DDLs

2023-03-28 Thread Jonathan S. Katz
On 3/27/23 2:37 AM, Amit Kapila wrote: On Mon, Mar 27, 2023 at 2:52 AM Tom Lane wrote: And TBH, I don't think that I quite believe the premise in the first place. The whole point of using logical rather than physical replication is that the subscriber installation(s) aren't exactly like the

RE: Support logical replication of DDLs

2023-03-28 Thread houzj.f...@fujitsu.com
On Tuesday, March 28, 2023 1:41 PM Amit Kapila wrote: > > On Mon, Mar 27, 2023 at 5:37 PM Amit Kapila > wrote: > > > > On Mon, Mar 27, 2023 at 12:07 PM Amit Kapila > wrote: > > > > > > On Mon, Mar 27, 2023 at 2:52 AM Tom Lane wrote: > > > > > > > > > > > I suggest taking a couple of steps

Re: Support logical replication of DDLs

2023-03-27 Thread Amit Kapila
On Mon, Mar 27, 2023 at 5:37 PM Amit Kapila wrote: > > On Mon, Mar 27, 2023 at 12:07 PM Amit Kapila wrote: > > > > On Mon, Mar 27, 2023 at 2:52 AM Tom Lane wrote: > > > > > > > > I suggest taking a couple of steps back from the minutiae of the > > > patch, and spending some hard effort thinking

RE: Support logical replication of DDLs

2023-03-27 Thread houzj.f...@fujitsu.com
On Monday, March 27, 2023 8:08 PM Amit Kapila wrote: > On Mon, Mar 27, 2023 at 12:07 PM Amit Kapila > wrote: > > > > On Mon, Mar 27, 2023 at 2:52 AM Tom Lane wrote: > > > > > > > > I suggest taking a couple of steps back from the minutiae of the > > > patch, and spending some hard effort

Re: Support logical replication of DDLs

2023-03-27 Thread Amit Kapila
On Mon, Mar 27, 2023 at 12:07 PM Amit Kapila wrote: > > On Mon, Mar 27, 2023 at 2:52 AM Tom Lane wrote: > > > > > I suggest taking a couple of steps back from the minutiae of the > > patch, and spending some hard effort thinking about how the thing > > would be controlled in a useful fashion

Re: Support logical replication of DDLs

2023-03-27 Thread Amit Kapila
On Mon, Mar 27, 2023 at 2:52 AM Tom Lane wrote: > > vignesh C writes: > > [ YA patch set ] > ... > > I'm also less than sold on the technical details, specifically > the notion of "let's translate utility parse trees into JSON and > send that down the wire". You can probably make that work for

Re: Support logical replication of DDLs

2023-03-26 Thread vignesh C
On Sun, 26 Mar 2023 at 18:08, vignesh C wrote: > > On Thu, 23 Mar 2023 at 09:22, Ajin Cherian wrote: > > > > On Mon, Mar 20, 2023 at 8:17 PM houzj.f...@fujitsu.com > > wrote: > > > > > > Attach the new patch set which addressed above comments. > > > 0002,0003,0004 patch has been updated in this

Re: Support logical replication of DDLs

2023-03-26 Thread Tom Lane
vignesh C writes: > [ YA patch set ] I spent some time looking through this thread to try to get a sense of the state of things, and I came away quite depressed. The patchset has ballooned to over 2MB, which is a couple orders of magnitude larger than anyone could hope to meaningfully review

Re: Support logical replication of DDLs

2023-03-26 Thread vignesh C
On Thu, 23 Mar 2023 at 09:22, Ajin Cherian wrote: > > On Mon, Mar 20, 2023 at 8:17 PM houzj.f...@fujitsu.com > wrote: > > > > Attach the new patch set which addressed above comments. > > 0002,0003,0004 patch has been updated in this version. > > > > Best Regards, > > Hou zj > > Attached a

Re: Support logical replication of DDLs

2023-03-23 Thread vignesh C
On Thu, 23 Mar 2023 at 09:22, Ajin Cherian wrote: > > On Mon, Mar 20, 2023 at 8:17 PM houzj.f...@fujitsu.com > wrote: > > > > Attach the new patch set which addressed above comments. > > 0002,0003,0004 patch has been updated in this version. > > > > Best Regards, > > Hou zj > > Attached a

RE: Support logical replication of DDLs

2023-03-17 Thread Takamichi Osumi (Fujitsu)
Hi On Tuesday, March 14, 2023 1:17 PM Ajin Cherian wrote: > I found out that the option ONLY was not parsed in the "CREATE INDEX" > command, for eg: CREATE UNIQUE INDEX ... ON ONLY table_name ... > > I've fixed this in patch 0002. FYI, cfbot reports a failure of v80 on linux [1]. Could you

RE: Support logical replication of DDLs

2023-03-15 Thread wangw.f...@fujitsu.com
On Tues, Mar 14, 2023 12:17 PM Ajin Cherian wrote: > On Mon, Mar 13, 2023 at 2:24 AM Zheng Li wrote: > > > > Thanks for working on the test coverage for CREATE and ALTER TABLE. > > I've made fixes for some of the failures in the v79 patch set (0002, > > 0003 and 0004 are updated). The changes

RE: Support logical replication of DDLs

2023-03-09 Thread houzj.f...@fujitsu.com
On Thur, Mar 9, 2023 10:27 AM Wang, Wei/王 威 > On Mon, Mar 6, 2023 18:17 PM Wang, Wei/王 威 > wrote: > > On Mon, Mar 6, 2023 14:34 AM Ajin Cherian wrote: > > > Changes are in patch 1 and patch 2 > > > > Thanks for updating the patch set. > > > > Here are some comments: > > Here are some more

RE: Support logical replication of DDLs

2023-03-08 Thread wangw.f...@fujitsu.com
On Mon, Mar 6, 2023 18:17 PM Wang, Wei/王 威 wrote: > On Mon, Mar 6, 2023 14:34 AM Ajin Cherian wrote: > > Changes are in patch 1 and patch 2 > > Thanks for updating the patch set. > > Here are some comments: Here are some more comments for v-75-0002* patch: 1. In the function

Re: Support logical replication of DDLs

2023-03-06 Thread Zheng Li
On Mon, Mar 6, 2023 at 5:17 AM wangw.f...@fujitsu.com wrote: > > For v-75-0003* patch. > 2. In the function deparse_CreateSeqStmt. > It seems that we are not deparsing the "AS data_type" clause (CREATE SEQUENCE > ... AS data_type). I think this causes all data_type to be default (bigint) > after

Re: Support logical replication of DDLs

2023-03-06 Thread vignesh C
On Mon, 6 Mar 2023 at 12:04, Ajin Cherian wrote: > > On Wed, Feb 15, 2023 at 3:33 PM Peter Smith wrote: > > > > > > > > > > 9. > > > > + > > > > +/* > > > > + * Append the parenthesized arguments of the given pg_proc row into > > > > the output > > > > + * buffer. force_qualify indicates

RE: Support logical replication of DDLs

2023-03-06 Thread wangw.f...@fujitsu.com
On Mon, Mar 6, 2023 14:34 AM Ajin Cherian wrote: > Changes are in patch 1 and patch 2 Thanks for updating the patch set. Here are some comments: For v-75-0002* patch. 1. In the function deparse_AlterRelation. + if ((sub->address.objectId != relId && +

Re: Support logical replication of DDLs

2023-02-24 Thread Masahiko Sawada
On Tue, Feb 21, 2023 at 11:09 AM Zheng Li wrote: > > On Mon, Feb 20, 2023 at 3:23 AM Masahiko Sawada wrote: > > > > On Fri, Feb 17, 2023 at 1:13 PM Zheng Li wrote: > > > > > > > > I've implemented a prototype to allow replicated objects to have the > > > > > same owner from the publisher in > >

Re: Support logical replication of DDLs

2023-02-20 Thread Zheng Li
On Mon, Feb 20, 2023 at 3:23 AM Masahiko Sawada wrote: > > On Fri, Feb 17, 2023 at 1:13 PM Zheng Li wrote: > > > > > > I've implemented a prototype to allow replicated objects to have the > > > > same owner from the publisher in > > > >

  1   2   3   4   >