Allow batched insert during cross-partition updates

2021-02-18 Thread Amit Langote
Based on the discussion at: https://www.postgresql.org/message-id/6929d485-2d2a-da46-3681-4a400a3d794f%40enterprisedb.com I'm posting the patch for $subject here in this new thread and I'll add it to the next CF per Tomas' advice. With 927f453a94106 committed earlier today, we limit insert batch

Re: Allow batched insert during cross-partition updates

2021-02-18 Thread Amit Langote
On Thu, Feb 18, 2021 at 6:52 PM Amit Langote wrote: > > Based on the discussion at: > > https://www.postgresql.org/message-id/6929d485-2d2a-da46-3681-4a400a3d794f%40enterprisedb.com > > I'm posting the patch for $subject here in this new thread and I'll > add it to the next CF per Tomas' advice.

Re: Allow batched insert during cross-partition updates

2021-03-09 Thread Georgios Kokolatos
Hi, thanks for the patch. I had a first look and played around with the code. The code seems clean, complete, and does what it says on the tin. I will need a bit more time to acclimatise with all the use cases for a more thorough review. I small question though is why expose PartitionTupleRoutin

Re: Allow batched insert during cross-partition updates

2021-03-10 Thread Georgios
Hi, ‐‐‐ Original Message ‐‐‐ On Thursday, February 18, 2021 10:54 AM, Amit Langote wrote: > On Thu, Feb 18, 2021 at 6:52 PM Amit Langote amitlangot...@gmail.com wrote: > > > Based on the discussion at: > > https://www.postgresql.org/message-id/6929d485-2d2a-da46-3681-4a400a3d794f%40en

Re: Allow batched insert during cross-partition updates

2021-03-10 Thread Georgios
‐‐‐ Original Message ‐‐‐ On Wednesday, March 10, 2021 1:23 PM, Georgios wrote: > > > Hi, > > ‐‐‐ Original Message ‐‐‐ > On Thursday, February 18, 2021 10:54 AM, Amit Langote amitlangot...@gmail.com > wrote: > > > On Thu, Feb 18, 2021 at 6:52 PM Amit Langote amitlangot...@gmail

Re: Allow batched insert during cross-partition updates

2021-03-11 Thread Amit Langote
Hi Georgios, On Wed, Mar 10, 2021 at 12:54 AM Georgios Kokolatos wrote: > > Hi, > > thanks for the patch. I had a first look and played around with the code. > > The code seems clean, complete, and does what it says on the tin. I will > need a bit more time to acclimatise with all the use cases f

Re: Allow batched insert during cross-partition updates

2021-03-11 Thread Amit Langote
Hi Georgios, On Wed, Mar 10, 2021 at 9:30 PM Georgios wrote: > I continued looking a bit at the patch, yet I am either failing to see fix or > I am > looking at the wrong thing. Please find attached a small repro of what my > expectetions > were. > > As you can see in the repro, I would expect

Re: Allow batched insert during cross-partition updates

2021-03-11 Thread gkokolatos
‐‐‐ Original Message ‐‐‐ On Thursday, March 11, 2021 9:42 AM, Amit Langote wrote: > Hi Georgios, > > On Wed, Mar 10, 2021 at 9:30 PM Georgios gkokola...@protonmail.com wrote: > > > I continued looking a bit at the patch, yet I am either failing to see fix > > or I am > > looking a

Re: Allow batched insert during cross-partition updates

2021-03-11 Thread Amit Langote
On Thu, Mar 11, 2021 at 8:36 PM wrote: > On Thursday, March 11, 2021 9:42 AM, Amit Langote > wrote: > > On Wed, Mar 10, 2021 at 9:30 PM Georgios gkokola...@protonmail.com wrote: > > > > > I continued looking a bit at the patch, yet I am either failing to see > > > fix or I am > > > looking at t

Re: Allow batched insert during cross-partition updates

2021-03-12 Thread gkokolatos
‐‐‐ Original Message ‐‐‐ On Friday, March 12, 2021 3:45 AM, Amit Langote wrote: > On Thu, Mar 11, 2021 at 8:36 PM gkokola...@pm.me wrote: > > > On Thursday, March 11, 2021 9:42 AM, Amit Langote amitlangot...@gmail.com > > wrote: > > > > > On Wed, Mar 10, 2021 at 9:30 PM Georgios gko

Re: Allow batched insert during cross-partition updates

2021-03-15 Thread Amit Langote
Hi Georgios, On Fri, Mar 12, 2021 at 7:59 PM wrote: > On Friday, March 12, 2021 3:45 AM, Amit Langote > wrote: > > On Thu, Mar 11, 2021 at 8:36 PM gkokola...@pm.me wrote: > > > On Thursday, March 11, 2021 9:42 AM, Amit Langote amitlangot...@gmail.com > > > wrote: > > > > What we do support how

Re: Allow batched insert during cross-partition updates

2021-03-16 Thread gkokolatos
‐‐‐ Original Message ‐‐‐ On Tuesday, March 16, 2021 6:13 AM, Amit Langote wrote: > Hi Georgios, > > On Fri, Mar 12, 2021 at 7:59 PM gkokola...@pm.me wrote: > > > On Friday, March 12, 2021 3:45 AM, Amit Langote amitlangot...@gmail.com > > wrote: > > > > > On Thu, Mar 11, 2021 at 8:

Re: Allow batched insert during cross-partition updates

2021-03-16 Thread Amit Langote
Hi Georgios, On Tue, Mar 16, 2021 at 5:12 PM wrote: > On Tuesday, March 16, 2021 6:13 AM, Amit Langote > wrote: > > On Fri, Mar 12, 2021 at 7:59 PM gkokola...@pm.me wrote: > > > On Friday, March 12, 2021 3:45 AM, Amit Langote amitlangot...@gmail.com > > > wrote: > > > > By the way, the test ca

Re: Allow batched insert during cross-partition updates

2021-03-16 Thread gkokolatos
‐‐‐ Original Message ‐‐‐ On Tuesday, March 16, 2021 9:59 AM, Amit Langote wrote: > Hi Georgios, > > On Tue, Mar 16, 2021 at 5:12 PM gkokola...@pm.me wrote: > > > On Tuesday, March 16, 2021 6:13 AM, Amit Langote amitlangot...@gmail.com > > wrote: > > > > > On Fri, Mar 12, 2021 at 7

Re: Allow batched insert during cross-partition updates

2021-04-04 Thread Amit Langote
On Tue, Mar 16, 2021 at 6:13 PM wrote: > Status updated to RfC in the commitfest app. Patch fails to apply per cfbot, so rebased. -- Amit Langote EDB: http://www.enterprisedb.com v3-0001-Allow-batching-of-inserts-during-cross-partition-.patch Description: Binary data

Re: Allow batched insert during cross-partition updates

2021-04-04 Thread Zhihong Yu
Hi, In the description: cross-partition update of partitioned tables can't use batching because ExecInitRoutingInfo() which initializes the insert target 'which' should be dropped since 'because' should start a sentence. +-- Check that batched inserts also works for inserts made during inserts

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Amit Langote
On Mon, Apr 5, 2021 at 1:16 AM Zhihong Yu wrote: > > Hi, > In the description: > > cross-partition update of partitioned tables can't use batching > because ExecInitRoutingInfo() which initializes the insert target > > 'which' should be dropped since 'because' should start a sentence. > > +-- Chec

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Bharath Rupireddy
On Tue, Apr 6, 2021 at 3:08 PM Amit Langote wrote: > > On Mon, Apr 5, 2021 at 1:16 AM Zhihong Yu wrote: > > > > Hi, > > In the description: > > > > cross-partition update of partitioned tables can't use batching > > because ExecInitRoutingInfo() which initializes the insert target > > > > 'which'

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Amit Langote
On Tue, Apr 6, 2021 at 6:49 PM Bharath Rupireddy wrote: > On Tue, Apr 6, 2021 at 3:08 PM Amit Langote wrote: > > Updated patch attached. I had to adjust the test case a little bit to > > account for the changes of 86dc90056d, something I failed to notice > > yesterday. Also, I expanded the test

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Bharath Rupireddy
On Tue, Apr 6, 2021 at 6:37 PM Amit Langote wrote: > > 3) will the cmin in the output always be the same? > > +SELECT cmin, * FROM batch_cp_upd_test3; > > TBH, I am not so sure. Maybe it's not a good idea to rely on cmin > after all. I've rewritten the tests to use a different method of > determ

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Amit Langote
On Tue, Apr 6, 2021 at 10:52 PM Bharath Rupireddy wrote: > > On Tue, Apr 6, 2021 at 6:37 PM Amit Langote wrote: > > > 3) will the cmin in the output always be the same? > > > +SELECT cmin, * FROM batch_cp_upd_test3; > > > > TBH, I am not so sure. Maybe it's not a good idea to rely on cmin > > af

RE: Allow batched insert during cross-partition updates

2021-05-07 Thread houzj.f...@fujitsu.com
> > Thanks! It looks good! > > Thanks for checking. I'll mark this as RfC. Hi, The patch cannot be applied to the latest head branch, it will be nice if you can rebase it. And when looking into the patch, I have some comments on it. 1) IIRC, After the commit c5b7ba4, the initialization of mt

Re: Allow batched insert during cross-partition updates

2021-05-09 Thread Amit Langote
On Fri, May 7, 2021 at 6:39 PM houzj.f...@fujitsu.com wrote: > > > > Thanks! It looks good! > > > > Thanks for checking. I'll mark this as RfC. > > Hi, > > The patch cannot be applied to the latest head branch, it will be nice if you > can rebase it. Thanks, done. > And when looking into the p

Re: Allow batched insert during cross-partition updates

2022-03-21 Thread Andres Freund
Hi, On 2021-08-24 12:03:59 +0900, Amit Langote wrote: > Tomas committed the bug-fix, so attaching a rebased version of the > patch for $subject. This patch is in the current CF, but doesn't apply: http://cfbot.cputube.org/patch_37_2992.log marked the entry as waiting-on-author. Greetings, Andr

Re: Allow batched insert during cross-partition updates

2022-03-21 Thread Amit Langote
On Tue, Mar 22, 2022 at 9:30 AM Andres Freund wrote: > On 2021-08-24 12:03:59 +0900, Amit Langote wrote: > > Tomas committed the bug-fix, so attaching a rebased version of the > > patch for $subject. > > This patch is in the current CF, but doesn't apply: > http://cfbot.cputube.org/patch_37_2992.

Re: Allow batched insert during cross-partition updates

2022-12-06 Thread Etsuro Fujita
Amit-san, On Tue, Mar 22, 2022 at 10:17 AM Amit Langote wrote: > Rebased to fix a minor conflict with some recently committed > nodeModifyTable.c changes. Apologies for not having reviewed the patch. Here are some review comments: * The patch conflicts with commit ffbb7e65a, so please update t

Re: Allow batched insert during cross-partition updates

2022-12-08 Thread Amit Langote
Hi Fujita-san, On Tue, Dec 6, 2022 at 6:47 PM Etsuro Fujita wrote: > On Tue, Mar 22, 2022 at 10:17 AM Amit Langote wrote: > > Rebased to fix a minor conflict with some recently committed > > nodeModifyTable.c changes. > > Apologies for not having reviewed the patch. Here are some review comment

Re: Allow batched insert during cross-partition updates

2022-12-08 Thread Etsuro Fujita
Amit-san, On Thu, Dec 8, 2022 at 5:00 PM Amit Langote wrote: > On Tue, Dec 6, 2022 at 6:47 PM Etsuro Fujita wrote: > > * In postgresGetForeignModifyBatchSize(): > > > > /* > > -* Should never get called when the insert is being performed as part > > of a > > -* row movement operatio

Re: Allow batched insert during cross-partition updates

2022-12-14 Thread Etsuro Fujita
On Thu, Dec 8, 2022 at 8:01 PM Etsuro Fujita wrote: > On Thu, Dec 8, 2022 at 5:00 PM Amit Langote wrote: > > Updated patch attached. > > I will review the patch a bit more, but I think > it would be committable. One thing I noticed is this bit: -- Clean up -DROP TABLE batch_table, batch_cp_upd

Re: Allow batched insert during cross-partition updates

2022-12-14 Thread Amit Langote
On Wed, Dec 14, 2022 at 6:45 PM Etsuro Fujita wrote: > On Thu, Dec 8, 2022 at 8:01 PM Etsuro Fujita wrote: > > On Thu, Dec 8, 2022 at 5:00 PM Amit Langote wrote: > > > Updated patch attached. > > > > I will review the patch a bit more, but I think > > it would be committable. > > One thing I not

Re: Allow batched insert during cross-partition updates

2022-12-20 Thread Etsuro Fujita
Hi Amit-san, On Wed, Dec 14, 2022 at 10:29 PM Amit Langote wrote: > On Wed, Dec 14, 2022 at 6:45 PM Etsuro Fujita wrote: > > One thing I noticed is this bit: > > > > -- Clean up > > -DROP TABLE batch_table, batch_cp_upd_test, batch_table_p0, > > batch_table_p1 CASCADE; > > +DROP TABLE batch_tab

Re: Allow batched insert during cross-partition updates

2022-12-20 Thread Amit Langote
On Tue, Dec 20, 2022 at 7:18 PM Etsuro Fujita wrote: > On Wed, Dec 14, 2022 at 10:29 PM Amit Langote wrote: > > On Wed, Dec 14, 2022 at 6:45 PM Etsuro Fujita > > wrote: > > > One thing I noticed is this bit: > > > > > > -- Clean up > > > -DROP TABLE batch_table, batch_cp_upd_test, batch_table_

Re: Allow batched insert during cross-partition updates

2021-07-01 Thread Tom Lane
Amit Langote writes: > [ v6-0001-Allow-batching-of-inserts-during-cross-partition-.patch ] Per the cfbot, this isn't applying anymore, so I'm setting it back to Waiting on Author. regards, tom lane

Re: Allow batched insert during cross-partition updates

2021-07-01 Thread Amit Langote
On Fri, Jul 2, 2021 at 1:39 AM Tom Lane wrote: > > Amit Langote writes: > > [ v6-0001-Allow-batching-of-inserts-during-cross-partition-.patch ] > > Per the cfbot, this isn't applying anymore, so I'm setting it back > to Waiting on Author. Rebased patch attached. Thanks for the reminder. -- Am

Re: Allow batched insert during cross-partition updates

2021-07-21 Thread vignesh C
On Fri, Jul 2, 2021 at 7:35 AM Amit Langote wrote: > > On Fri, Jul 2, 2021 at 1:39 AM Tom Lane wrote: > > > > Amit Langote writes: > > > [ v6-0001-Allow-batching-of-inserts-during-cross-partition-.patch ] > > > > Per the cfbot, this isn't applying anymore, so I'm setting it back > > to Waiting o

Re: Allow batched insert during cross-partition updates

2021-07-26 Thread Amit Langote
On Thu, Jul 22, 2021 at 2:18 PM vignesh C wrote: > On Fri, Jul 2, 2021 at 7:35 AM Amit Langote wrote: > > On Fri, Jul 2, 2021 at 1:39 AM Tom Lane wrote: > > > > > > Amit Langote writes: > > > > [ v6-0001-Allow-batching-of-inserts-during-cross-partition-.patch ] > > > > > > Per the cfbot, this i

Re: Allow batched insert during cross-partition updates

2021-08-23 Thread Amit Langote
On Tue, Jul 27, 2021 at 11:32 AM Amit Langote wrote: > On Thu, Jul 22, 2021 at 2:18 PM vignesh C wrote: > > One of the test postgres_fdw has failed, can you please post an > > updated patch for the fix: > > test postgres_fdw ... FAILED (test process exited with > > exit code 2)