Re: PG 14 release notes, first draft

2021-05-11 Thread Amit Langote
On Wed, May 12, 2021 at 6:23 AM Bruce Momjian wrote: > On Tue, May 11, 2021 at 11:57:10AM +0900, Amit Langote wrote: > > On Mon, May 10, 2021 at 11:40 PM Justin Pryzby wrote: > > > Same as the last couple years, I checked for missing items in the release > > > notes,

Re: Inherited UPDATE/DELETE vs async execution

2021-05-11 Thread Amit Langote
Fujita-san, On Tue, May 11, 2021 at 5:56 PM Etsuro Fujita wrote: > On Mon, May 10, 2021 at 9:21 PM Amit Langote wrote: > > On Sat, May 8, 2021 at 1:21 AM Etsuro Fujita > > wrote: > > > I noticed this while working on the > > > EXPLAIN-ANALYZE-for-async-capable

Re: update/delete and execution-time partition pruning

2021-05-10 Thread Amit Langote
On Tue, May 11, 2021 at 12:57 PM David Rowley wrote: > > On Tue, 11 May 2021 at 15:46, David Rowley wrote: > > I'll take care of this. > > Pushed. Thank you. -- Amit Langote EDB: http://www.enterprisedb.com

update/delete and execution-time partition pruning

2021-05-10 Thread Amit Langote
to be changed in a future release of PostgreSQL. Now because we include the Append/MergeAppend node even if the query is update/delete, this note is no longer useful or valid. Attached patch removes that note. -- Amit Langote EDB: http://www.enterprisedb.com update-delete-exec

Re: PG 14 release notes, first draft

2021-05-10 Thread Amit Langote
://www.postgresql.org/message-id/CA%2BHiwqEcawatEaUh1uTbZMEZTJeLzbroRTz9_X9Z5CFjTWJkhw%40mail.gmail.com Thank you. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Inherited UPDATE/DELETE vs async execution

2021-05-10 Thread Amit Langote
. Attached is a patch for > that. Maybe I am missing something, though. I see that your patch is to disable asynchronous execution in ForeignScan nodes responsible for direct update/delete, but why not do the same for other ForeignScan nodes too? Or the other way around -- is it because fixing th

Re: Allow batched insert during cross-partition updates

2021-05-09 Thread Amit Langote
mstate->aux_fmstate; > > It might be better to write like " if (fmstate != NULL && > fmstate->aux_fmstate != NULL)". Sure, done. Actually, there's a if (fmstate) statement right below the code being added, which I fixed to match the style used by the new code. -- Amit Langote EDB: http://www.enterprisedb.com v6-0001-Allow-batching-of-inserts-during-cross-partition-.patch Description: Binary data

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-05-05 Thread Amit Langote
> https://www.postgresql.org/docs/devel/ddl-partitioning.html#DDL-PARTITIONING-USING-INHERITANCE That makes sense, thanks for noticing. How about the attached? -- Amit Langote EDB: http://www.enterprisedb.com ddl-partitioning-note-detach-concurrenltly.patch Description: Binary data

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-30 Thread Amit Langote
(Thanks for committing the fix.) On Thu, Apr 29, 2021 at 1:11 AM Álvaro Herrera wrote: > On Wed, Apr 28, 2021, at 10:21 AM, Amit Langote wrote: > I noticed that rd_partdesc_nodetached_xmin can sometimes end up with > value 0. While you seem to be already aware of that, because otherw

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-28 Thread Amit Langote
e readable to just duplicate this stanza in the blocks that assign to rd_partdesc_nodetached and rd_partdesc, respectively? That's not much code to duplicate and it'd be easier to see which context is for which partdesc. + TransactionId rd_partdesc_nodetached_xmin; /* xmin for the above */ Could you please expand this description a bit? -- Amit Langote EDB: http://www.enterprisedb.com

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-27 Thread Amit Langote
On Tue, Apr 27, 2021 at 4:34 PM Amit Langote wrote: > Thanks for the updated patch. I've been reading it, but I noticed a > bug in 8aba9322511f, which I thought you'd want to know to make a note > of when committing this one. > > So we decided in 8aba9322511f that it i

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-27 Thread Amit Langote
etached is no longer transient, so the problem doesn't exist. I will write more about the updated patch in a bit... -- Amit Langote EDB: http://www.enterprisedb.com

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-26 Thread Amit Langote
ecessarily sane with asserts off, since you'll get an arbitrary > detach-Xmin assigned to the partdesc, depending on catalog scan order. Maybe this is an ignorant question but is the plan to add an elog() in this code path or a check (and an ereport()) somewhere in ATExecDetachPartition() to p

Re: Table refer leak in logical replication

2021-04-26 Thread Amit Langote
On Mon, Apr 26, 2021 at 3:27 PM Michael Paquier wrote: > On Fri, Apr 23, 2021 at 09:38:01PM +0900, Amit Langote wrote: > > On Thu, Apr 22, 2021 at 1:45 PM Michael Paquier wrote: > >> On the other hand, the tests for partitions have much more value IMO, > >> but look

Re: Does rewriteTargetListIU still need to add UPDATE tlist entries?

2021-04-25 Thread Amit Langote
0.00..55.20 rows=2260 width=42) Output: ((base.a + 1) - 44), ROW((base.a + 1), (base.b + 2)), base.ctid (7 rows) I didn't however study closely why REPLACEVARS_CHANGE_VARNO does the correct thing, so am not sure if there might be cases that would be broken. > So, as far as I can tell, this is an oversight in 86dc90056 and we > ought to clean it up as attached. Thanks for noticing this and the patch. If you are confident that REPLACEVARS_CHANGE_VARNO covers all imaginable cases, I suppose it makes sense to apply it. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Table refer leak in logical replication

2021-04-23 Thread Amit Langote
On Thu, Apr 22, 2021 at 1:45 PM Michael Paquier wrote: > On Wed, Apr 21, 2021 at 09:58:10PM +0900, Amit Langote wrote: > > Okay, done. > > So, I have been working on that today, and tried to apply the full set > before realizing when writing the commit message that this was

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-23 Thread Amit Langote
On Fri, Apr 23, 2021 at 4:26 AM Alvaro Herrera wrote: > On 2021-Apr-22, Amit Langote wrote: > > -* The reason for this check is that we want to avoid seeing the > > +* The reason for this hack is that we want to avoid seeing the > > * partition a

Re: problem with RETURNING and update row movement

2021-04-22 Thread Amit Langote
On Fri, Apr 23, 2021 at 12:49 AM Tom Lane wrote: > Amit Langote writes: > > I think we should also add slot != srcSlot to this condition. > > Good idea, should save useless comparisons of identical tupdescs. > Done. > > >> (I've not looked at porting this t

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-22 Thread Amit Langote
l find a way to write it more succinctly. BTW, I do feel a bit alarmed by the potential performance impact of this. If detached_exist of a cached partdesc is true, then RI queries invoked during a bulk DML operation would have to rebuild one for every tuple to be checked, right? I haven't tried an actual example yet though. -- Amit Langote EDB: http://www.enterprisedb.com

Re: problem with RETURNING and update row movement

2021-04-21 Thread Amit Langote
On Thu, Apr 22, 2021 at 9:37 AM Tom Lane wrote: > Amit Langote writes: > > FWIW, I think we should go ahead and apply the patches for the bug > > reported here. Anyone who tries to project an updated tuple's system > > columns using RETURNING are likely to face probl

Re: Table refer leak in logical replication

2021-04-21 Thread Amit Langote
On Wed, Apr 21, 2021 at 7:38 PM Michael Paquier wrote: > On Wed, Apr 21, 2021 at 04:21:52PM +0900, Amit Langote wrote: > > So I had started last night by adding some tests for this in > > 003_constraints.pl because there are already some replica BR trigger > > tests there. I

Re: Table refer leak in logical replication

2021-04-21 Thread Amit Langote
On Wed, Apr 21, 2021 at 11:13 AM Amit Langote wrote: > On Wed, Apr 21, 2021 at 9:31 AM Michael Paquier wrote: > > On Tue, Apr 20, 2021 at 06:20:03PM +0530, Amit Kapila wrote: > > > +1. I think it makes sense to add a test case especially because we > > > don't hav

Re: Table refer leak in logical replication

2021-04-20 Thread Amit Langote
h a table would need at least 4 columns. Agree about adding tests along these lines. Will post in a bit. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Partitioned table permission question

2021-04-20 Thread Amit Langote
d directly without an additional grant. In a similar way, the parent table's row security policies (see Section 5.8) are applied to rows coming from child tables during an inherited query. A child table's policies, if any, are applied only when it is the table explicitly named in the query; and in that case, any policies attached to its parent(s) are ignored." -- Amit Langote EDB: http://www.enterprisedb.com [1] https://www.postgresql.org/docs/current/ddl-inherit.html

Re: Table refer leak in logical replication

2021-04-20 Thread Amit Langote
On Tue, Apr 20, 2021 at 4:22 PM Michael Paquier wrote: > On Tue, Apr 20, 2021 at 02:48:35PM +0900, Amit Langote wrote: > > Manipulating the contents of es_opened_result_relations directly in > > worker.c is admittedly a "hack", which I am reluctant to have other > &g

Re: Table refer leak in logical replication

2021-04-19 Thread Amit Langote
Thanks for taking a look. On Tue, Apr 20, 2021 at 2:09 PM Michael Paquier wrote: > On Mon, Apr 19, 2021 at 09:44:05PM +0900, Amit Langote wrote: > > Okay, how about the attached then? > > create_estate_for_relation() returns an extra resultRelInfo that's

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-04-19 Thread Amit Langote
d/CA%2BHiwqEeU19iQgjN6HF1HTPU0L5%2BJxyS5CmxaOVGNXBAfUY06Q%40mail.gmail.com I had proposed to move the map creation from maybe_send_schema() to get_rel_sync_entry(), mainly because the latter is where I realized it belongs, though a bit too late. Attached is the part of the patch for this particular is

Re: Commit 86dc90056 - Rework planning and execution of UPDATE and DELETE

2021-04-19 Thread Amit Langote
pg_attribute.attstorage. That's because the new tuple would be computed from the subplan's targetlist and the TupleDesc for that targetlist is computed with no regard to where the computed tuple will go; IOW ignoring the target table's actual TupleDesc. After 86dc90056, the new tuple is c

Re: Table refer leak in logical replication

2021-04-19 Thread Amit Langote
On Mon, Apr 19, 2021 at 6:32 PM Michael Paquier wrote: > On Mon, Apr 19, 2021 at 02:33:10PM +0530, Amit Kapila wrote: > > On Mon, Apr 19, 2021 at 12:32 PM Amit Langote > > wrote: > >> FWIW, I agree with fixing this bug of 1375422c in as least scary > >> mann

Re: Table refer leak in logical replication

2021-04-19 Thread Amit Langote
at 02:33:10PM +0530, Amit Kapila wrote: > > > > > On Mon, Apr 19, 2021 at 12:32 PM Amit Langote > > > > > wrote: > > > > >> FWIW, I agree with fixing this bug of 1375422c in as least scary > > > > >> manner as possible. Hou-

Re: Table refer leak in logical replication

2021-04-19 Thread Amit Langote
On Mon, Apr 19, 2021 at 6:03 PM Amit Kapila wrote: > On Mon, Apr 19, 2021 at 12:32 PM Amit Langote wrote: > > On Sat, Apr 17, 2021 at 10:32 PM Amit Kapila > > wrote: > > > On Fri, Apr 16, 2021 at 11:55 AM Michael Paquier > > > wrote: > > > > A

Re: Doubt with [ RANGE partition with TEXT datatype ]

2021-04-19 Thread Amit Langote
o follow numerical rules, use a numeric data type. > If this behavior is expected, Kindly let me know, how to represent the range > from '5' to '10' with text datatype column? Don't know why you want to use the text type for the column and these particular values for the partitions bounds, but one workaround would be to use '05' instead of '5'. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Table refer leak in logical replication

2021-04-19 Thread Amit Langote
On Fri, Apr 16, 2021 at 3:24 PM Michael Paquier wrote: > On Tue, Apr 06, 2021 at 02:25:05PM +0900, Amit Langote wrote: > > While updating the patch to do so, it occurred to me that maybe we > > could move the ExecInitResultRelation() call into > > create_estate_for_relation

Re: Table refer leak in logical replication

2021-04-19 Thread Amit Langote
On Sat, Apr 17, 2021 at 10:32 PM Amit Kapila wrote: > On Fri, Apr 16, 2021 at 11:55 AM Michael Paquier wrote: > > On Tue, Apr 06, 2021 at 02:25:05PM +0900, Amit Langote wrote: > > > > Attached is v5 that I am finishing with. Much more could be done but > > I don

Re: ModifyTable overheads in generic plans

2021-04-13 Thread Amit Langote
On Wed, Apr 7, 2021 at 5:18 PM Amit Langote wrote: > On Wed, Apr 7, 2021 at 8:24 AM Tom Lane wrote: > > I also could not get excited about postponing initialization of RETURNING > > or WITH CHECK OPTIONS expressions. I grant that that can be helpful > > when those features

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-12 Thread Amit Langote
arlier query in the transaction, which does include the detach-pending partition, thus allowing the insert trying to insert a row referencing that partition to succeed. There is a provision in RelationGetPartitionDesc() to rebuild if any detach-pending partitions in the existing copy of PartitionDesc are not to be seen by the current query, but a bug mentioned in my earlier reply prevents that from kicking in. -- Amit Langote EDB: http://www.enterprisedb.com

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-12 Thread Amit Langote
On Mon, Apr 12, 2021 at 4:42 PM Amit Langote wrote: > On Mon, Apr 12, 2021 at 6:20 AM Alvaro Herrera > wrote: > > On 2021-Mar-31, Tom Lane wrote: > > > > > diff -U3 > > > /home/buildfarm/trilobite/buildroot/HEAD/pgsql.build/src/test/isolation/expected

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-12 Thread Amit Langote
key of d4_fk being violated. It seems problematic to me that the logic of setting includes_detached is oblivious of the special check in find_inheritance_children() to decide whether "force"-include a detach-pending child based on cross-checking its pg_inherit tuple's xmin against the active snapshot. Maybe fixing that would help, although I haven't tried that myself yet. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Table refer leak in logical replication

2021-04-11 Thread Amit Langote
On Sat, Apr 10, 2021 at 10:39 AM Justin Pryzby wrote: > I added this as an Open Item. Thanks, Justin. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Wired if-statement in gen_partprune_steps_internal

2021-04-08 Thread Amit Langote
On Thu, Apr 8, 2021 at 7:41 PM David Rowley wrote: > On Thu, 8 Apr 2021 at 21:04, Amit Langote wrote: > > Maybe, we should also updated the description of node struct as > > follows to consider that last point: >> > > * PartitionPruneStepOp - Information to prune us

Re: Wired if-statement in gen_partprune_steps_internal

2021-04-08 Thread Amit Langote
On Thu, Apr 8, 2021 at 5:34 PM David Rowley wrote: > On Thu, 8 Apr 2021 at 00:49, Amit Langote wrote: > > > > Thanks David. Actually, I was busy updating the patch to revert to > > gen_partprune_steps_internal() returning a list and was almost done > > with it when I

Re: ModifyTable overheads in generic plans

2021-04-07 Thread Amit Langote
On Thu, Apr 8, 2021 at 1:54 PM David Rowley wrote: > On Thu, 8 Apr 2021 at 15:32, Amit Langote wrote: > > There's 10-20% improvement in this case too for various partition > > counts, which really has more to do with 86dc90056 than the work done > > here. > >

Re: ModifyTable overheads in generic plans

2021-04-07 Thread Amit Langote
0 9020 v14dev HEAD 64 14925 14648 13361 128 14379 14333 13138 256 14478 14246 13316 102412744 12621 11579 There's 10-20% improvement in this case too for various partition counts, which really has more to do with 86dc90056 than the work done here. -- Amit Langote EDB: http://www.enterprisedb.com

Re: ModifyTable overheads in generic plans

2021-04-07 Thread Amit Langote
On Thu, Apr 8, 2021 at 1:34 AM Tom Lane wrote: > Amit Langote writes: > > Also, I think we should update the commentary around ri_projectNew a > > bit to make it clear that noplace beside ExecGet{Insert|Update}Tuple > > should be touching it and the associated slots. &

Re: Wired if-statement in gen_partprune_steps_internal

2021-04-07 Thread Amit Langote
On Wed, Apr 7, 2021 at 6:53 PM David Rowley wrote: > On Wed, 7 Apr 2021 at 21:04, Amit Langote wrote: > > > > On Wed, Apr 7, 2021 at 4:43 PM David Rowley wrote: > > > However, it does change the meaning of what PARTCLAUSE_MATCH_STEPS > > > does.

Re: Wired if-statement in gen_partprune_steps_internal

2021-04-07 Thread Amit Langote
o go ahead and commit the v3. I've attached the patch I had ended up with and was about to post as v3, just in case you wanted to glance. -- Amit Langote EDB: http://www.enterprisedb.com v3_amit.diff Description: Binary data

Re: Wired if-statement in gen_partprune_steps_internal

2021-04-07 Thread Amit Langote
On Wed, Apr 7, 2021 at 4:43 PM David Rowley wrote: > On Thu, 4 Mar 2021 at 19:03, Amit Langote wrote: > > On Tue, Oct 20, 2020 at 9:46 PM Amit Langote > > wrote: > > > I had updated the patch last week to address Horiguchi-san's comments > > > but d

Re: ModifyTable overheads in generic plans

2021-04-07 Thread Amit Langote
On Wed, Apr 7, 2021 at 8:24 AM Tom Lane wrote: > Amit Langote writes: > > On Mon, Apr 5, 2021 at 1:43 AM Tom Lane wrote: > >> OK. Do you want to pull out the bits of the patch that we can still > >> do without postponing BeginDirectModify? > > > I

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Amit Langote
On Tue, Apr 6, 2021 at 10:52 PM Bharath Rupireddy wrote: > > On Tue, Apr 6, 2021 at 6:37 PM Amit Langote wrote: > > > 3) will the cmin in the output always be the same? > > > +SELECT cmin, * FROM batch_cp_upd_test3; > > > > TBH, I am not so sure. Maybe

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Amit Langote
On Tue, Apr 6, 2021 at 6:49 PM Bharath Rupireddy wrote: > On Tue, Apr 6, 2021 at 3:08 PM Amit Langote wrote: > > Updated patch attached. I had to adjust the test case a little bit to > > account for the changes of 86dc90056d, something I failed to notice > > yesterday.

Re: Allow batched insert during cross-partition updates

2021-04-06 Thread Amit Langote
otally unrelated code. Updated patch attached. I had to adjust the test case a little bit to account for the changes of 86dc90056d, something I failed to notice yesterday. Also, I expanded the test case added in postgres_fdw.sql a bit to show the batching in action more explicitly. -- Amit Langote EDB: http://www.enterprisedb.com v4-0001-Allow-batching-of-inserts-during-cross-partition-.patch Description: Binary data

Re: A reloption for partitioned tables - parallel_workers

2021-04-06 Thread Amit Langote
On Fri, Apr 2, 2021 at 11:36 PM Laurenz Albe wrote: > On Wed, 2021-03-24 at 14:14 +1300, David Rowley wrote: > > On Fri, 19 Mar 2021 at 02:07, Amit Langote wrote: > > > Attached a new version rebased over c8f78b616, with the grouping > > > relation partitioning enhan

Re: postgres_fdw: IMPORT FOREIGN SCHEMA ... LIMIT TO (partition)

2021-04-06 Thread Amit Langote
archy, this approach should > +allow access to all the data without creating extra objects. > > Now "this approach" in the above is not clear? What about replacing it with > something like "importing only partitioned tables"? +1, that wording is better. -- Amit Langote EDB: http://www.enterprisedb.com

Re: postgres_fdw: IMPORT FOREIGN SCHEMA ... LIMIT TO (partition)

2021-04-06 Thread Amit Langote
s limitation. IMPORT FORAIGN > SCHEMA really is great when it works, but limitations like these are > crippling for some more specific use cases (e.g. allowing > long-duration read-only access to one partition in the partition tree > while also allowing the partition layout of the parents to be > modified). FWIW, I agree that it would be nice to have this. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Table refer leak in logical replication

2021-04-05 Thread Amit Langote
On Tue, Apr 6, 2021 at 1:57 PM Masahiko Sawada wrote: > On Tue, Apr 6, 2021 at 1:15 PM Amit Langote wrote: > > On Tue, Apr 6, 2021 at 1:01 PM houzj.f...@fujitsu.com > > > The commit introduced a > > > > new function ExecInitResultRelation() that sets both > >

Re: Table refer leak in logical replication

2021-04-05 Thread Amit Langote
t; We should call ExecCloseRangeTableRelations at the end of apply_handle_xxx to > close the rel in "Estate->es_relations". Right, ExecCloseRangeTableRelations() was missing. I think it may be better to create a sibling function to create_estate_for_relation(),

Re: ModifyTable overheads in generic plans

2021-04-05 Thread Amit Langote
On Mon, Apr 5, 2021 at 1:43 AM Tom Lane wrote: > Amit Langote writes: > > On Sun, Apr 4, 2021 at 10:20 AM Tom Lane wrote: > >> In some desultory performance testing here, it seemed like a > >> significant part of the cost is ExecOpenIndices, and I don't see &g

Re: Allow batched insert during cross-partition updates

2021-04-04 Thread Amit Langote
On Tue, Mar 16, 2021 at 6:13 PM wrote: > Status updated to RfC in the commitfest app. Patch fails to apply per cfbot, so rebased. -- Amit Langote EDB: http://www.enterprisedb.com v3-0001-Allow-batching-of-inserts-during-cross-partition-.patch Description: Binary data

Re: ModifyTable overheads in generic plans

2021-04-04 Thread Amit Langote
On Sun, Apr 4, 2021 at 10:20 AM Tom Lane wrote: > Amit Langote writes: > > On Thu, Apr 1, 2021 at 3:12 AM Tom Lane wrote: > >> Amit Langote writes: > > [ v14-0002-Initialize-result-relation-information-lazily.patch ] > >> Needs YA rebase over 86dc90056.

Re: simplifying foreign key/RI checks

2021-04-04 Thread Amit Langote
in one place, the two if statements can be > combined into one. Thanks for taking a look. I agree with most of your suggestions and have incorporated them in the v8 just posted. -- Amit Langote EDB: http://www.enterprisedb.com

Re: simplifying foreign key/RI checks

2021-04-04 Thread Amit Langote
Hi Alvaro, On Sat, Apr 3, 2021 at 12:01 AM Alvaro Herrera wrote: > On 2021-Apr-02, Amit Langote wrote: > > > On Sat, Mar 20, 2021 at 10:21 PM Amit Langote > > wrote: > > > Updated patches attached. Sorry about the delay. > > > > Rebased over the re

Re: a misbehavior of partition row movement (?)

2021-04-02 Thread Amit Langote
On Thu, Apr 1, 2021 at 10:56 AM Masahiko Sawada wrote: > On Tue, Mar 23, 2021 at 6:27 PM Amit Langote wrote: > > Actually, I found a big hole in my assumptions around deferrable > > foreign constraints, invalidating the approach I took in 0002 to use a > > query-lifetime

Re: simplifying foreign key/RI checks

2021-04-02 Thread Amit Langote
On Sat, Mar 20, 2021 at 10:21 PM Amit Langote wrote: > Updated patches attached. Sorry about the delay. Rebased over the recent DETACH PARTITION CONCURRENTLY work. Apparently, ri_ReferencedKeyExists() was using the wrong snapshot. -- Amit Langote EDB: http://www.enterprisedb.com v8-0

Re: making update/delete of inheritance trees scale better

2021-04-02 Thread Amit Langote
On Wed, Mar 31, 2021 at 9:54 PM Amit Langote wrote: > I reran some of the performance tests I did earlier (I've attached the > modified test running script for reference): For archives' sake, noticing a mistake in my benchmarking script, I repeated the tests. Apparently, all p

Re: ModifyTable overheads in generic plans

2021-04-01 Thread Amit Langote
On Thu, Apr 1, 2021 at 10:12 PM Amit Langote wrote: > > On Thu, Apr 1, 2021 at 3:12 AM Tom Lane wrote: > > Amit Langote writes: > > > [ v14-0002-Initialize-result-relation-information-lazily.patch ] > > Needs YA rebase over 86dc90056. > > Done. I will post t

Re: ModifyTable overheads in generic plans

2021-04-01 Thread Amit Langote
On Thu, Apr 1, 2021 at 3:12 AM Tom Lane wrote: > Amit Langote writes: > > [ v14-0002-Initialize-result-relation-information-lazily.patch ] > Needs YA rebase over 86dc90056. Done. I will post the updated results for -Mprepared benchmarks I did in the other thread shortly. -- Amit

Re: making update/delete of inheritance trees scale better

2021-03-31 Thread Amit Langote
On Thu, Apr 1, 2021 at 12:58 AM Tom Lane wrote: > Amit Langote writes: > > On Tue, Mar 30, 2021 at 1:51 PM Tom Lane wrote: > >> Here's a v13 patchset that I feel pretty good about. > > > Thanks. After staring at this for a day now, I do too. > > Thanks for

Re: making update/delete of inheritance trees scale better

2021-03-31 Thread Amit Langote
an be attributed to the fact that with a generic plan, there are a bunch of steps that must be done across all partitions, such as AcauireExecutorLocks(), ExecCheckRTPerms(), per-result-rel initializations in ExecInitModifyTable(), etc., even with the patched. As mentioned upthread, [1] can help with

Re: making update/delete of inheritance trees scale better

2021-03-30 Thread Amit Langote
st pushdown for the traditional inheritance cases as well. I agree that rethinking the whole apply_scanjoin_target_to_paths() approach might be a better use of our time. It has a looping-over-the-whole-partition-array bottleneck for simple lookup queries that I have long wanted to propose doing something about. -- Amit Langote EDB: http://www.enterprisedb.com

Re: making update/delete of inheritance trees scale better

2021-03-30 Thread Amit Langote
ree that that's not too bad for a worse-case test case, nor something we couldn't optimize. I have yet to look closely at where the problem lies though. -- Amit Langote EDB: http://www.enterprisedb.com

Re: making update/delete of inheritance trees scale better

2021-03-29 Thread Amit Langote
On Mon, Mar 29, 2021 at 11:41 PM Tom Lane wrote: > Amit Langote writes: > > On Sun, Mar 28, 2021 at 1:30 AM Tom Lane wrote: > >> I wanted to give a data dump of where I am. I've reviewed and > >> nontrivially modified 0001 and the executor parts of 0002, an

Re: making update/delete of inheritance trees scale better

2021-03-29 Thread Amit Langote
On Sun, Mar 28, 2021 at 1:30 AM Tom Lane wrote: > Amit Langote writes: > > Attached updated version of the patch. I have forgotten to mention in > > my recent posts on this thread one thing about 0001 that I had > > mentioned upthread back in June. That it curr

Re: making update/delete of inheritance trees scale better

2021-03-29 Thread Amit Langote
e where a remote partition chosen to insert a moved row into is also an UPDATE target partition that will be updated later. === I think we will need to take out the "...table will be updated later" part at the end of the sentence. -- Amit Langote EDB: http://www.enterprisedb.com

Re: making update/delete of inheritance trees scale better

2021-03-25 Thread Amit Langote
compute, which is what your approach sounds like. However, I had abandoned it due to the concern that it possibly hurt the prepared statements because we would build the targetlist on every execution, whereas only once if the planner does it. I'm just about done addressing Robert's comments, so will post an update shortly. -- Amit Langote EDB: http://www.enterprisedb.com

Re: making update/delete of inheritance trees scale better

2021-03-24 Thread Amit Langote
ucting the full updated tuple at the outset in > ModifyTable, and then proceeding pretty much as updates > have done in the past. But there's more we can do later. Agreed. I'm addressing Robert's comments and will post an updated patch by tomorrow. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Wired if-statement in gen_partprune_steps_internal

2021-03-23 Thread Amit Langote
Hi Ryan, On Tue, Mar 23, 2021 at 2:24 AM Ryan Lambert wrote: > Should the status of this patch be updated to ready for comitter to get in > line for Pg 14 deadline? Yes, I've done that. Thanks for the reminder. -- Amit Langote EDB: http://www.enterprisedb.com

Re: a misbehavior of partition row movement (?)

2021-03-23 Thread Amit Langote
ded to change the if condition as follows: - if (pset.sversion >= 110000) + if (pset.sversion >= 11 && pset.sversion < 14) Note that without this change, this code ends up revealing partitions' foreign key triggers, because we will now be marking them dependent on their parent trigger, which wasn't the case before this patch. > I'll look at 0002 patch. Actually, I found a big hole in my assumptions around deferrable foreign constraints, invalidating the approach I took in 0002 to use a query-lifetime tuplestore to record root parent tuples. I'm trying to find a way to make the tuplestore transaction-lifetime so that the patch still works. In the meantime, I'm attaching an updated set with 0001 changed per your comments. -- Amit Langote EDB: http://www.enterprisedb.com v6-0002-Enforce-foreign-key-correctly-during-cross-partit.patch Description: Binary data v6-0001-Create-foreign-key-triggers-in-partitioned-tables.patch Description: Binary data

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-03-22 Thread Amit Langote
d by operations that would invalidate the SMgrRelation that those smgr* operations would be called with. For example, the smgrnblocks() in gistBuildCallback() may get done too late than a corresponding RelationOpenSmgr() on the index relation. -- Amit Langote EDB: http://www.enterprisedb.com

Re: problem with RETURNING and update row movement

2021-03-22 Thread Amit Langote
On Sat, Mar 6, 2021 at 12:52 AM David Steele wrote: > On 10/30/20 6:00 AM, Amit Langote wrote: > > The question > > I guess is whether that thread must conclude before the fix here can > > be committed. > > Indeed. But it seems like there is a candidate patch on [1] th

Re: simplifying foreign key/RI checks

2021-03-20 Thread Amit Langote
On Mon, Mar 8, 2021 at 11:41 PM Amit Langote wrote: > On Thu, Mar 4, 2021 at 5:15 AM Tom Lane wrote: > > I guess I'm disturbed by the idea > > that we'd totally replace the implementation technology for only one > > variant of foreign key checks. That means that

Re: A reloption for partitioned tables - parallel_workers

2021-03-18 Thread Amit Langote
On Fri, Mar 5, 2021 at 11:06 PM Laurenz Albe wrote: > On Fri, 2021-03-05 at 22:55 +0900, Amit Langote wrote: > > On Fri, Mar 5, 2021 at 10:47 PM Laurenz Albe > > wrote: > > > On Wed, 2021-03-03 at 17:58 +0900, Amit Langote wrote: > > > > For example, with t

Re: crash during cascaded foreign key update

2021-03-18 Thread Amit Langote
On Wed, Mar 17, 2021 at 11:01 AM Amit Langote wrote: > On Tue, Mar 16, 2021 at 11:17 PM Tom Lane wrote: > > Amit Langote writes: > > > With HEAD (I think v12 and greater), I see $subject when trying out > > > the following scenario: Actually, the crash is reproducibl

Re: crash during cascaded foreign key update

2021-03-16 Thread Amit Langote
On Tue, Mar 16, 2021 at 11:17 PM Tom Lane wrote: > Amit Langote writes: > > With HEAD (I think v12 and greater), I see $subject when trying out > > the following scenario: > > I wonder if this is related to > > https://www.postgresql.org/message-id/flat/89429.1584443

crash during cascaded foreign key update

2021-03-16 Thread Amit Langote
) at postmaster.c:1736 #32 0x00891a4b in PostmasterMain (argc=3, argv=0x1b3cc20) at postmaster.c:1408 #33 0x0079360f in main (argc=3, argv=0x1b3cc20) at main.c:209 I haven't checked the failure in more detail yet other than that the failed Assert was added in 5db6df0c0117. -- Amit Langote EDB: http://www.enterprisedb.com

Re: simplifying foreign key/RI checks

2021-03-16 Thread Amit Langote
On Mon, Mar 8, 2021 at 11:41 PM Amit Langote wrote: > On Thu, Mar 4, 2021 at 5:15 AM Tom Lane wrote: > > Lastly, ri_PerformCheck is pretty careful about not only which > > snapshot it uses, but which *pair* of snapshots it uses, because > > sometimes it needs to worry abo

Re: Allow batched insert during cross-partition updates

2021-03-16 Thread Amit Langote
Hi Georgios, On Tue, Mar 16, 2021 at 5:12 PM wrote: > On Tuesday, March 16, 2021 6:13 AM, Amit Langote > wrote: > > On Fri, Mar 12, 2021 at 7:59 PM gkokola...@pm.me wrote: > > > On Friday, March 12, 2021 3:45 AM, Amit Langote amitlangot...@gmail.com > > > wrote

Re: Allow batched insert during cross-partition updates

2021-03-15 Thread Amit Langote
Hi Georgios, On Fri, Mar 12, 2021 at 7:59 PM wrote: > On Friday, March 12, 2021 3:45 AM, Amit Langote > wrote: > > On Thu, Mar 11, 2021 at 8:36 PM gkokola...@pm.me wrote: > > > On Thursday, March 11, 2021 9:42 AM, Amit Langote amitlangot...@gmail.com > > > wro

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-11 Thread Amit Langote
the patch for adding parallel_workers option to partitioned tables does? -- Amit Langote EDB: http://www.enterprisedb.com

Re: Allow batched insert during cross-partition updates

2021-03-11 Thread Amit Langote
On Thu, Mar 11, 2021 at 8:36 PM wrote: > On Thursday, March 11, 2021 9:42 AM, Amit Langote > wrote: > > On Wed, Mar 10, 2021 at 9:30 PM Georgios gkokola...@protonmail.com wrote: > > > > > I continued looking a bit at the patch, yet I am either failing to see > &g

Re: Allow batched insert during cross-partition updates

2021-03-11 Thread Amit Langote
is patch is for teaching those INSERTs to use batched mode if allowed, which is currently prohibited. So with this patch, if an UPDATE moves 10 rows from a local partition to a remote partition, then they will be inserted with a single INSERT command containing all 10 rows, instead of 10 separate IN

Re: Allow batched insert during cross-partition updates

2021-03-11 Thread Amit Langote
or functions would be better, I will change the patch that way. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-10 Thread Amit Langote
On Wed, Mar 10, 2021 at 6:18 PM Amit Kapila wrote: > On Mon, Mar 8, 2021 at 7:19 PM Amit Langote wrote: > > I just read through v25 and didn't find anything to complain about. > > Thanks a lot, pushed now! Amit L., your inputs are valuable for this work. Glad I could

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Amit Langote
On Thu, Mar 11, 2021 at 4:25 AM Tom Lane wrote: > Amit Langote writes: > > On Wed, Mar 10, 2021 at 8:37 AM Tom Lane wrote: > >> Hmm. So, the key point is that the values coming from the partitioned > >> child table are injected into the test query as parameters, no

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Amit Langote
On Wed, Mar 10, 2021 at 8:37 AM Tom Lane wrote: > Amit Langote writes: > > On Fri, Mar 5, 2021 at 6:00 AM Tom Lane wrote: > >> This claim seems false on its face: > >>> All child constraints of a given foreign key constraint can use the > >>> same RI

Re: TRUNCATE on foreign table

2021-03-08 Thread Amit Langote
) does. That leads to relids and relids_extra having different lengths, which trips the Assert in ExecuteTruncateGuts(). -- Amit Langote EDB: http://www.enterprisedb.com

Re: simplifying foreign key/RI checks

2021-03-08 Thread Amit Langote
s is a cute idea, and the speedup is pretty impressive, > but I don't think it's anywhere near committable. I also wonder > whether we really want ri_triggers.c having its own copy of > low-level stuff like the tuple-locking code you copied. Seems > like a likely maintenance hazard, so maybe some more refactoring > is needed. Okay, I will see if there's a way to avoid copying too much code. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-08 Thread Amit Langote
> 0001* patch or if you want to review it further? I just read through v25 and didn't find anything to complain about. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Huge memory consumption on partitioned table with FKs

2021-03-08 Thread Amit Langote
On Mon, Mar 8, 2021 at 9:53 PM Andy Fan wrote: > On Mon, Mar 8, 2021 at 8:42 PM Amit Langote wrote: >> On Mon, Mar 8, 2021 at 8:39 PM Andy Fan wrote: >> > On Mon, Mar 8, 2021 at 3:43 PM Andy Fan wrote: >> >> My point below is a bit off-topic, but I want to

Re: Huge memory consumption on partitioned table with FKs

2021-03-08 Thread Amit Langote
EATE TABLE p (a int) partitioned by list(a) RESTRICTED". We can add these > limitation to restricted partitioned relation only. I think you'd agree that the topics you want to discuss deserve a separate discussion thread. You may refer to this discussion in that new thread if you thi

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-07 Thread Amit Langote
ireExecutorLocksOnPartitions() will do in the generic plan case. -- Amit Langote EDB: http://www.enterprisedb.com

<    2   3   4   5   6   7   8   9   10   11   >