Re: Reduce useless changes before reassembly during logical replication

2024-03-06 Thread li jie
Hi, Sorry I replied too late. > This is all true but note that in successful cases (where the table is > published) all the work done by FilterByTable(accessing caches, > transaction-related stuff) can add noticeable overhead as anyway we do > that later in pgoutput_change(). You are correct.

Reduce useless changes before reassembly during logical replication

2024-01-16 Thread li jie
Hi hackers, During logical replication, if there is a large write transaction, some spill files will be written to disk, depending on the setting of logical_decoding_work_mem. This behavior can effectively avoid OOM, but if the transaction generates a lot of change before commit, a large number

Reduce useless changes before reassembly during logical replication

2023-12-07 Thread li jie
Hi hackers, During logical replication, if there is a large write transaction, some spill files will be written to disk, depending on the setting of logical_decoding_work_mem. This behavior can effectively avoid OOM, but if the transaction generates a lot of change before commit, a large number

Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2023-12-03 Thread li jie
> On Fri, Dec 1, 2023 at 1:55 PM li jie wrote: > > > > > This is just an immature idea. I haven't started to implement it yet. > > > Maybe it was designed this way because there > > > are key factors that I didn't consider. So I want to hear everyo

Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2023-12-03 Thread li jie
> > This may be helpful for the case you have mentioned but how about > cases where there is nothing to filter by relation? You can see the complete antecedent in the email [1]. Relation that has not been published will also generate changes and put them into the entire transaction group, which

Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2023-12-01 Thread li jie
> This is just an immature idea. I haven't started to implement it yet. > Maybe it was designed this way because there > are key factors that I didn't consider. So I want to hear everyone's > opinions, especially the designers of logic decoding. Attached is the patch I used to implement this

Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2023-11-30 Thread li jie
Hi, During logical decoding, if there is a large write transaction, some spill files will be written to disk, depending on the setting of max_changes_in_memory. This behavior can effectively avoid OOM, but if the transaction generates a lot of change before commit, a large number of files may

Re: Support logical replication of DDLs

2022-12-19 Thread li jie
flat/CAH8n8_jMTunxxtP4L-3tc%3DGNamg%3Dmg1X%3DtgHr9CqqjjzFLwQng%40mail.gmail.com > The test patch is very useful. I see that the sql case in test_ddl_deparse_regress is similar to the one in test_ddl_deparse. Why don't we merge the test cases in test_ddl_deparse_regress into test_ddl_deparse, as the sql case can be completely reused with the sql files in test_ddl_deparse? I believe this will make the tests more comprehensive and reduce redundancy. Regards, li jie

Re: Support logical replication of DDLs

2022-12-12 Thread li jie
replication. I think the ownership relationship should not be lost, and we should perhaps add it back, like pg_dump "ALTER TABLE public.t1 OWNER TO a;", even though we do not currently support the replication of USER. Thought? li jie.

Re: Support logical replication of DDLs

2022-12-08 Thread li jie
> > Attached please find a new solution that skips the deparsing of ALTER TABLE > subcommands generated for TableLikeClause. The patch v42-0005 added a new > boolean field table_like to AlterTableStmt in order to identify an ALTER TABLE > subcommand generated internally for the TableLikeClause. >

Re: Support logical replication of DDLs

2022-12-08 Thread li jie
temp table t1(id int); ``` Thoughts? li jie vignesh C 于2022年12月8日周四 13:07写道: > > On Wed, 7 Dec 2022 at 17:50, Alvaro Herrera wrote: > > > > I think this patch is split badly. > > > > You have: > > > > 0001 an enormous patch including some required infra

Re: Support logical replication of DDLs

2022-12-01 Thread li jie
I applied patch 0005. I think this modification is a bit overdone. This design skips all subcommands, which results in many ddl replication failures. For example: ``` CREATE TABLE datatype_table (id SERIAL); ``` deparsed ddl is: CREATE TABLE public.datatype_table (id pg_catalog.int4 STORAGE

Re: Support logical replication of DDLs

2022-11-29 Thread li jie
I will continue to give feedback for this patch. 1. LIKE STORAGE ``` CREATE TABLE ctlt (a text, c text); ALTER TABLE ctlt ALTER COLUMN c SET STORAGE EXTERNAL; CREATE TABLE ctlt_storage (LIKE ctlt INCLUDING STORAGE); ``` postgres=# \d+ ctlt_storage Table

Re: Support logical replication of DDLs

2022-11-27 Thread li jie
> > > All three commands are captured by the event trigger. The first and > second command ends up getting deparsed, WAL-logged and > replayed on the subscriber. The replay of the ALTER TABLE command > causes a duplicate constraint error. The problem is that > while subcommands are captured by

Re: Support logical replication of DDLs

2022-11-25 Thread li jie
Hi Developer, I have been following this patch for a long time. Recently, I started to try to test it. I found several bugs here and want to give you feedback. 1. CREATE TABLE LIKE I found that this case may be repication incorrectly. You can run the following SQL statement: ```

Re: Is it worth pushing conditions to sublink/subplan?

2021-08-22 Thread li jie
Indeed, this may be useful for partition pruning. I am also curious about why this has not been achieved. Wenjing 于2021年8月23日周一 上午10:46写道: > Hi Hackers, > > Recently, a issue has been bothering me, This is about conditional > push-down in SQL. > I use cases from regression testing as an