Re: a misbehavior of partition row movement (?)

2021-10-14 Thread Amit Langote
On Mon, Sep 20, 2021 at 3:32 PM Amit Langote wrote: > The problem was that the tuplestore > (afterTriggers.query_stack[query_level].tuplestore) that I decided to > use to store the AFTER trigger tuples of a partitioned table that is > the target of an cross-partition update lives

Re: [BUG] Unexpected action when publishing partition tables

2021-10-12 Thread Amit Langote
Hi Amit, On Fri, Oct 8, 2021 at 12:47 PM Amit Kapila wrote: > On Thu, Oct 7, 2021 at 12:39 PM Amit Langote wrote: > > Sorry that I didn't comment on this earlier, but I think either > > GetPubPartitionOptionRelations() or InvalidatePublicationRels() > > introduced

Re: [BUG] Unexpected action when publishing partition tables

2021-10-07 Thread Amit Langote
;t comment on this earlier, but I think either GetPubPartitionOptionRelations() or InvalidatePublicationRels() introduced in the commit 4548c76738b should lock the partitions, just like to the parent partitioned table would be, before invalidating them. There may be some hazards to invalidating a rel

Re: a comment in joinrel.c: compute_partition_bounds()

2021-10-06 Thread Amit Langote
Fujita-san, On Wed, Oct 6, 2021 at 5:41 PM Etsuro Fujita wrote: > On Fri, Sep 24, 2021 at 4:20 PM Etsuro Fujita wrote: > > On Fri, Sep 24, 2021 at 3:34 PM Amit Langote > > wrote: > > > I think there's a word missing in the following comment: > > > > &

Re: Multi-Column List Partitioning

2021-10-01 Thread Amit Langote
s. I thought about sending a v4 of the main patch with my proposed changes so far integrated, but decided to just post a delta_v2 for now. -- Amit Langote EDB: http://www.enterprisedb.com v3_delta_amit_v2.patch Description: Binary data

Re: Record a Bitmapset of non-pruned partitions

2021-09-30 Thread Amit Langote
On Fri, Oct 1, 2021 at 7:07 AM David Rowley wrote: > On Thu, 30 Sept 2021 at 20:25, Amit Langote wrote: > > Related to the above, I noticed while looking at > > build_merged_partition_bounds() that db632fbca3 missed adding a line > > to that function to set interleaved_pa

Re: Record a Bitmapset of non-pruned partitions

2021-09-30 Thread Amit Langote
Hi David, On Mon, Aug 2, 2021 at 11:59 AM Amit Langote wrote: > On Sun, Aug 1, 2021 at 9:31 PM David Rowley wrote: > > On Fri, 30 Jul 2021 at 19:10, Amit Langote wrote: > > > interleaved_parts idea looks clever. I wonder if you decided that > > > it's maybe not

a comment in joinrel.c: compute_partition_bounds()

2021-09-23 Thread Amit Langote
h the same * cardinal positions form the pairs. ": the join rel have the same..." seems to be missing a "will". Attached a patch to fix. -- Amit Langote EDB: http://www.enterprisedb.com comment-missing-word.patch Description: Binary data

Re: a misbehavior of partition row movement (?)

2021-09-19 Thread Amit Langote
On Fri, Sep 10, 2021 at 11:03 PM Amit Langote wrote: > On Fri, Sep 3, 2021 at 12:23 PM Amit Langote wrote: > > Hi Andrew, > > > > On Fri, Sep 3, 2021 at 6:19 AM Andrew Dunstan wrote: > > > On 7/13/21 8:09 AM, Amit Langote wrote: > > > > Unfortunately,

Re: resowner module README needs update?

2021-09-15 Thread Amit Langote
On Wed, Sep 15, 2021 at 4:09 PM Michael Paquier wrote: > On Tue, Sep 14, 2021 at 07:37:28PM +0900, Michael Paquier wrote: > > No objections here. > > Done as of cae6fc2. Thank you. -- Amit Langote EDB: http://www.enterprisedb.com

Re: resowner module README needs update?

2021-09-13 Thread Amit Langote
Thanks for looking. On Tue, Sep 14, 2021 at 9:17 AM Michael Paquier wrote: > On Mon, Sep 13, 2021 at 10:44:11PM +0900, Amit Langote wrote: > > It seems a bunch of other object/resource types have been integrated > > into the resowner mechanism since the list was las

resowner module README needs update?

2021-09-13 Thread Amit Langote
after updating becomes a bit too long. -- Amit Langote EDB: http://www.enterprisedb.com update-resowner-readme.patch Description: Binary data

Re: a misbehavior of partition row movement (?)

2021-09-10 Thread Amit Langote
On Fri, Sep 3, 2021 at 12:23 PM Amit Langote wrote: > Hi Andrew, > > On Fri, Sep 3, 2021 at 6:19 AM Andrew Dunstan wrote: > > On 7/13/21 8:09 AM, Amit Langote wrote: > > > Unfortunately, I don’t think I’ll have time in this CF to solve some > > > very fundam

Re: A reloption for partitioned tables - parallel_workers

2021-09-03 Thread Amit Langote
On Sat, Sep 4, 2021 at 3:10 Laurenz Albe wrote: > On Fri, 2021-09-03 at 18:24 +0200, Daniel Gustafsson wrote: > > > On 6 Apr 2021, at 09:46, Amit Langote wrote: > > > On Fri, Apr 2, 2021 at 11:36 PM Laurenz Albe > wrote: > > > > > > I don't know

Re: Multi-Column List Partitioning

2021-09-03 Thread Amit Langote
On Wed, Sep 1, 2021 at 2:31 PM Amit Langote wrote: > On Tue, Aug 31, 2021 at 8:02 PM Nitin Jadhav > wrote: > > The attached patch also fixes the above comments. > > I noticed that multi-column list partitions containing NULLs don't > work correctly with partition prunin

Re: a misbehavior of partition row movement (?)

2021-09-02 Thread Amit Langote
Hi Andrew, On Fri, Sep 3, 2021 at 6:19 AM Andrew Dunstan wrote: > On 7/13/21 8:09 AM, Amit Langote wrote: > > Unfortunately, I don’t think I’ll have time in this CF to solve some > > very fundamental issues I found in the patch during the last cycle. > > I’m fine with eithe

Re: Multi-Column List Partitioning

2021-08-31 Thread Amit Langote
-defined struct, I wonder if it makes sense to rename the struct while we're at it. Call it, say, PartitionListBound? Also, please keep part of the existing comment that says that the bound belongs to index'th partition. Will send more comments in a bit... -- Amit Langote EDB: http://www.enterprisedb.com

Re: Multi-Column List Partitioning

2021-08-30 Thread Amit Langote
traint of p01 should actually be: (a IS NOT NULL) AND (a = 1) AND (b IS NOT NULL) AND (b = 1) AND (c IS NOT NULL) AND (c = true) As per the patch's current implementation, tuple (1, NULL, true) passes p01's partition constraint, because only (b = 1) is not sufficient to reject a NULL value being assigned to b. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Partition Check not updated when insert into a partition

2021-08-27 Thread Amit Langote
On Thu, Aug 5, 2021 at 11:32 AM Amit Langote wrote: > On Wed, Jul 14, 2021 at 11:16 AM houzj.f...@fujitsu.com > > On Tuesday, July 13, 2021 2:52 AM Alvaro Herrera > > wrote: > > > Did you have a misbehaving test for the ATTACH case? > > > > Thanks for the res

Re: Allow batched insert during cross-partition updates

2021-08-23 Thread Amit Langote
On Tue, Jul 27, 2021 at 11:32 AM Amit Langote wrote: > On Thu, Jul 22, 2021 at 2:18 PM vignesh C wrote: > > One of the test postgres_fdw has failed, can you please post an > > updated patch for the fix: > > test postgres_fdw ... FAILED (test process exited

Re: Partition Check not updated when insert into a partition

2021-08-04 Thread Amit Langote
ntain) the root parent's partition constraint. * I think we should lock the partitions before sending the invalidation. The ATTACH code already locks the descendents for a different purpose, but DETACH doesn't, so the latter needs to be fixed to match. I've updated Alvaro's patch to address these points. Maybe, we should also add these cases to the regression and isolation suites? -- Amit Langote EDB: http://www.enterprisedb.com inval-partitions_v2.patch Description: Binary data

Re: Skip partition tuple routing with constant partition key

2021-08-01 Thread Amit Langote
I noticed that there is no CF entry for this, so created one in the next CF: https://commitfest.postgresql.org/34/3270/ Rebased patch attached. v8-0001-Teach-get_partition_for_tuple-to-cache-bound-offs.patch Description: Binary data

Re: Record a Bitmapset of non-pruned partitions

2021-08-01 Thread Amit Langote
On Sun, Aug 1, 2021 at 9:31 PM David Rowley wrote: > On Fri, 30 Jul 2021 at 19:10, Amit Langote wrote: > > interleaved_parts idea looks clever. I wonder if you decided that > > it's maybe not worth setting that field in the joinrel's > > PartitionBoundInfos? Fo

Re: Record a Bitmapset of non-pruned partitions

2021-07-30 Thread Amit Langote
s [ in terms of partitioning-defined ordering ]." Looks fine otherwise. -- Amit Langote EDB: http://www.enterprisedb.com

Re: a thinko in b676ac443b6

2021-07-27 Thread Amit Langote
On Wed, Jul 28, 2021 at 1:07 AM Tomas Vondra wrote: > On 7/27/21 4:28 AM, Amit Langote wrote: > > I think it can be incorrect to use the same TupleDesc for both the > > slots in ri_Slots (for ready-to-be-inserted tuples) and ri_PlanSlots > > (for subplan output tuples

Re: Allow batched insert during cross-partition updates

2021-07-26 Thread Amit Langote
On Thu, Jul 22, 2021 at 2:18 PM vignesh C wrote: > On Fri, Jul 2, 2021 at 7:35 AM Amit Langote wrote: > > On Fri, Jul 2, 2021 at 1:39 AM Tom Lane wrote: > > > > > > Amit Langote writes: > > > > [ v6-0001-Allow-batching-of-inserts-during-cross-partition-.

a thinko in b676ac443b6

2021-07-26 Thread Amit Langote
x27;s TupleDesc exactly (plus any junk columns), but now it produces only a partial tuple containing the values for the changed columns. So it's better to revert to using planSlot->tts_tupleDescriptor for the slots in ri_PlanSlots. Attached a patch to do so. -- Amit Lango

Re: a misbehavior of partition row movement (?)

2021-07-13 Thread Amit Langote
Hi Ibrar, Sawada-san, On Tue, Jul 13, 2021 at 20:25 Ibrar Ahmed wrote: > > > On Fri, Apr 2, 2021 at 6:09 PM Amit Langote > wrote: > >> On Thu, Apr 1, 2021 at 10:56 AM Masahiko Sawada >> wrote: >> > On Tue, Mar 23, 2021 at 6:27 PM Amit Langote >> wro

Re: ExecRTCheckPerms() and many prunable partitions

2021-07-07 Thread Amit Langote
On Wed, Jul 7, 2021 at 1:41 PM David Rowley wrote: > On Fri, 2 Jul 2021 at 12:41, Amit Langote wrote: > > I'll mark the CF entry as WoA, unless you'd rather I just mark it RwF. > > I've set it to waiting on author. It was still set to needs review. Sorry it slipped

Re: simplifying foreign key/RI checks

2021-07-05 Thread Amit Langote
d patches attached. -- Amit Langote EDB: http://www.enterprisedb.com v9-0001-Export-get_partition_for_tuple.patch Description: Binary data v9-0002-Avoid-using-SPI-for-some-RI-checks.patch Description: Binary data

Re: Allow batched insert during cross-partition updates

2021-07-01 Thread Amit Langote
On Fri, Jul 2, 2021 at 1:39 AM Tom Lane wrote: > > Amit Langote writes: > > [ v6-0001-Allow-batching-of-inserts-during-cross-partition-.patch ] > > Per the cfbot, this isn't applying anymore, so I'm setting it back > to Waiting on Author. Rebased patch att

Re: New committers: Daniel Gustafsson and John Naylor

2021-07-01 Thread Amit Langote
> > > Please join me in wishing them much success and few bugs. > > > > regards, tom lane > > Congrats to you both ! +1, congrats Daniel & John! -- Amit Langote EDB: http://www.enterprisedb.com

Re: ExecRTCheckPerms() and many prunable partitions

2021-07-01 Thread Amit Langote
On Fri, Jul 2, 2021 at 12:45 AM Tom Lane wrote: > Amit Langote writes: > > The problem is that it loops over the entire range table even though > > only one or handful of those entries actually need their permissions > > checked. Most entries, especially those of partition

Re: Record a Bitmapset of non-pruned partitions

2021-06-30 Thread Amit Langote
worthwhile if only for that purpose. He'd suggested that we fix things elsewhere such that that function is not needed in the first place [2], something I keep thinking about in between doing other things, but never sit down to actually write a patch. Given that you're proposing more uses for live_parts, maybe he'd be open to the idea. -- Amit Langote EDB: http://www.enterprisedb.com [1] https://www.postgresql.org/message-id/3f280722-46f2-c2a4-4c19-2cfa28c6c1cd%40lab.ntt.co.jp [2] https://www.postgresql.org/message-id/3529.1554051867%40sss.pgh.pa.us

Re: ExecRTCheckPerms() and many prunable partitions

2021-06-30 Thread Amit Langote
On Wed, Jun 30, 2021 at 23:34 David Rowley wrote: > On Thu, 1 Jul 2021 at 01:34, Amit Langote wrote: > > For now, I have implemented the idea 2 as the attached patch. > > I only just had a fleeting glance at the patch. Aren't you > accidentally missing the 0th RTE

ExecRTCheckPerms() and many prunable partitions

2021-06-30 Thread Amit Langote
ueries. So while still kind of PoC, will add this to July CF for keeping track. -- Amit Langote EDB: http://www.enterprisedb.com [1] https://www.postgresql.org/message-id/ca+hiwqg7zrubmmih3wpsbz4s0h2ehywrnxeducky5hr3fwz...@mail.gmail.com [2] https://w

Re: Deadlock risk while inserting directly into partition?

2021-06-27 Thread Amit Langote
On Mon, Jun 28, 2021 at 12:58 PM Amit Langote wrote: > On Fri, Jun 25, 2021 at 10:26 AM David Rowley wrote: > > On Thu, 24 Jun 2021 at 12:32, David Rowley wrote: > > > The overhead of taking these locks is pretty significant for > > > partitioned tables with lots of

Re: Deadlock risk while inserting directly into partition?

2021-06-27 Thread Amit Langote
/message-id/16db1458-67cf-4add-736e-31b053115e8e%40lab.ntt.co.jp Maybe a good idea to add a line or 2 in 5.11. Table Partitioning? -- Amit Langote EDB: http://www.enterprisedb.com

Re: Deadlock risk while inserting directly into partition?

2021-06-27 Thread Amit Langote
ich we'd have to do anyway, because perhaps the executor proper (mainly InitPlan) should also see the shrunken version of the range table. Not to mention the complexity of getting the "init" pruning itself to run outside a full-blown executor context. Anyway, do you agre

Re: Decoding speculative insert with toast leaks memory

2021-06-17 Thread Amit Langote
Hi Dilip, On Thu, Jun 17, 2021 at 4:45 PM Dilip Kumar wrote: > On Thu, Jun 17, 2021 at 12:52 PM Amit Langote wrote: > > > Oh I missed that the problem report is for the PG13 branch. > > > > How about the attached patch then? > > > Looks good, Thanks for ch

Re: Skip partition tuple routing with constant partition key

2021-06-17 Thread Amit Langote
On Thu, Jun 17, 2021 at 4:18 PM Zhihong Yu wrote: > On Wed, Jun 16, 2021 at 10:37 PM Amit Langote wrote: >> > + if (part_index < 0) >> > + { >> > + bound_offset = >> > partition_range_datum_bsea

Re: Decoding speculative insert with toast leaks memory

2021-06-17 Thread Amit Langote
On Thu, Jun 17, 2021 at 3:42 PM Amit Kapila wrote: > On Thu, Jun 17, 2021 at 10:39 AM Amit Langote wrote: > > > > On Thu, Jun 17, 2021 at 12:56 PM Amit Kapila > > wrote: > > > On Wed, Jun 16, 2021 at 8:18 PM Tom Lane wrote: > > > > >

Re: Skip partition tuple routing with constant partition key

2021-06-16 Thread Amit Langote
Hi, Thanks for reading the patch. On Thu, Jun 17, 2021 at 1:46 PM Zhihong Yu wrote: > On Wed, Jun 16, 2021 at 9:29 PM Amit Langote wrote: >> Attached a slightly revised version of that patch, with a commit >> message this time. > > + int n_tup

Re: Decoding speculative insert with toast leaks memory

2021-06-16 Thread Amit Langote
t;pubactions.pubtruncate = false; entry->publish_as_relid = InvalidOid; entry->map = NULL; /* will be set by maybe_send_schema() if needed */ } Do we need the same statement at the end of the following block? /* Validate the entry */ if (!entry->replicate_valid) { -- Amit Langote EDB: http://www.enterprisedb.com

Re: Skip partition tuple routing with constant partition key

2021-06-16 Thread Amit Langote
On Wed, Jun 16, 2021 at 4:27 PM Amit Langote wrote: > On Mon, Jun 7, 2021 at 8:38 PM houzj.f...@fujitsu.com > wrote: > > The rough idea is to check the average batch number every 1000 rows. > > If the average batch num is greater than 1, then we enable the cache check, > &g

Re: Skip partition tuple routing with constant partition key

2021-06-16 Thread Amit Langote
serted may be more adaptive. Please find attached a revised version of your patch, where I tried to make it a bit easier to follow, hopefully. While doing so, I realized that caching the bound offset across queries makes little sense now, so I decided to keep the changes confined to execPartitio

Re: Multi-Column List Partitioning

2021-06-10 Thread Amit Langote
On Fri, Jun 11, 2021 at 12:37 PM Amit Langote wrote: > I will look at other parts of the patch next week hopefully. For > now, attached is a delta patch that applies on top of your v1, which > does: > > * Simplify partition_list_bsearch() and partition_lbound_datum

Re: Multi-Column List Partitioning

2021-06-10 Thread Amit Langote
for a and b. I will look at other parts of the patch next week hopefully. For now, attached is a delta patch that applies on top of your v1, which does: * Simplify partition_list_bsearch() and partition_lbound_datum_cmp() * Make qsort_partition_list_value_cmp simply call partition_lbound_datum_cmp() instead of having its own logic to compare input bounds * Move partition_lbound_datum_cmp() into partbounds.c as a static function (export seems unnecessary) * Add a comment for PartitionBoundInfo.isnulls and remove that for null_index -- Amit Langote EDB: http://www.enterprisedb.com v1_delta_amit.patch Description: Binary data

Re: Skip partition tuple routing with constant partition key

2021-06-04 Thread Amit Langote
On Fri, Jun 4, 2021 at 6:05 PM Amit Langote wrote: > On Fri, Jun 4, 2021 at 4:38 PM Amit Langote wrote: > > On Thu, Jun 3, 2021 at 8:48 PM Amit Langote wrote: > > > On Tue, Jun 1, 2021 at 5:43 PM houzj.f...@fujitsu.com > > > wrote: > > > > So, If we wan

Re: Skip partition tuple routing with constant partition key

2021-06-04 Thread Amit Langote
On Fri, Jun 4, 2021 at 4:38 PM Amit Langote wrote: > On Thu, Jun 3, 2021 at 8:48 PM Amit Langote wrote: > > On Tue, Jun 1, 2021 at 5:43 PM houzj.f...@fujitsu.com > > wrote: > > > So, If we want to share the cached partition between statements, we seems > > > ca

Re: Skip partition tuple routing with constant partition key

2021-06-04 Thread Amit Langote
On Thu, Jun 3, 2021 at 8:48 PM Amit Langote wrote: > On Tue, Jun 1, 2021 at 5:43 PM houzj.f...@fujitsu.com > wrote: > > So, If we want to share the cached partition between statements, we seems > > cannot > > use ExecPartitionCheck. Instead, I tried directly invoke th

Re: parent foreign tables and row marks

2021-06-03 Thread Amit Langote
On Thu, Jun 3, 2021 at 10:07 AM Amit Langote wrote: > On Thu, Jun 3, 2021 at 3:39 AM Tom Lane wrote: > > > > Amit Langote writes: > > > I noticed that 428b260f87e8 (v12) broke the cases where a parent > > > foreign table has row marks assigned. > > >

Re: Skip partition tuple routing with constant partition key

2021-06-03 Thread Amit Langote
Hou-san, On Tue, Jun 1, 2021 at 5:43 PM houzj.f...@fujitsu.com wrote: > From: Amit Langote > > > > Where are you thinking to cache the partidx list? Inside > > > > PartitionDesc or some executor struct? > > > > > > I was thinking cache the

Re: parent foreign tables and row marks

2021-06-02 Thread Amit Langote
On Thu, Jun 3, 2021 at 3:39 AM Tom Lane wrote: > > Amit Langote writes: > > I noticed that 428b260f87e8 (v12) broke the cases where a parent > > foreign table has row marks assigned. > > Indeed :-(. Fix pushed. I tweaked the comments and test case slightly. Thank yo

Re: join pushdown and issue with foreign update

2021-06-02 Thread Amit Langote
ltype = get_rel_type_id(rte->relid); + if (!OidIsValid(reltype)) + continue; + att->atttypid = reltype; That may be a valid concern. I wonder if it would make sense to also check varattno == 0 here somewhere for good measure. -- Amit Langote EDB: http://www.enterprisedb.com

Re: join pushdown and issue with foreign update

2021-06-02 Thread Amit Langote
relations, I don't think we have other cases that are affected at this point. I have a feeling that your patch will have fixed things enough that the same problem will not occur when we have join pushdown under UPDATE occurring in more cases. -- Amit Langote EDB: http://www.enterprisedb.com

Re: join pushdown and issue with foreign update

2021-06-01 Thread Amit Langote
tables, as we add "wholerow" with type of one child relation > and then > try to use it with another child (of different table type). Perhaps, we can get away with adding the wholerow Var with the target relation's reltype when the target foreign table is not a "child" relation, but the root target relation itself. Maybe like the attached? -- Amit Langote EDB: http://www.enterprisedb.com wholerow-rowid-var-vartype-fix.patch Description: Binary data

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-06-01 Thread Amit Langote
On Tue, Jun 1, 2021 at 6:56 PM Amit Kapila wrote: > On Mon, May 31, 2021 at 8:51 AM Amit Langote wrote: > > On Thu, May 27, 2021 at 3:36 PM Amit Kapila wrote: > > > Why do we need to move send_relation_and_attrs() call? I think it > > > doesn't matter much eith

parent foreign tables and row marks

2021-06-01 Thread Amit Langote
) to add a TID row marking junk column if needed. Attached a patch for that, which also adds the test cases. Actually, I had to make a separate version of the patch to apply to the v12 branch, because EXPLAIN outputs relation aliases a bit differently starting in v13, which is attached too. -- Amit Langot

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-30 Thread Amit Langote
On Thu, May 27, 2021 at 3:36 PM Amit Kapila wrote: > On Fri, May 21, 2021 at 1:12 PM Amit Langote wrote: > > > > Hmm, maybe get_rel_syn_entry() should explicitly set map to NULL when > > first initializing an entry. It's possible that without doing so, the > > ma

Re: Skip partition tuple routing with constant partition key

2021-05-27 Thread Amit Langote
Hou-san, On Thu, May 27, 2021 at 3:56 PM houzj.f...@fujitsu.com wrote: > From: Amit Langote > Sent: Thursday, May 27, 2021 1:54 PM > > On Thu, May 27, 2021 at 11:47 AM houzj.f...@fujitsu.com > > wrote: > > > About teaching relcache about caching the target parti

Re: Skip partition tuple routing with constant partition key

2021-05-26 Thread Amit Langote
What do you think ? Where are you thinking to cache the partidx list? Inside PartitionDesc or some executor struct? -- Amit Langote EDB: http://www.enterprisedb.com

Re: Skip partition tuple routing with constant partition key

2021-05-26 Thread Amit Langote
On Thu, May 27, 2021 at 1:55 PM Zhihong Yu wrote: > For CanUseSavedPartitionForTuple, nit: you can check > !dispatch->savedPartResultInfo at the beginning and return early. > This would save some indentation. Sure, see the attached. -- Amit Langote EDB: http://www.enterprisedb.c

Re: Skip partition tuple routing with constant partition key

2021-05-26 Thread Amit Langote
tuple"). Maybe we don't need this parent-slot-clearing anymore due to code restructuring over the last 3 years, but that will have to be a separate patch. I hope the attached updated patch makes it a bit more clear what's going on. I refactored more of the code in ExecFin

Re: Multi-Column List Partitioning

2021-05-25 Thread Amit Langote
> thoughts about the approaches and please share if you have any better > solution than the above 2. Approach 1 sounds better. It sounds like approach 1 might help us implement support for allowing NULLs in range partition bounds in the future, if at all. For now, it might be

Re: Skip partition tuple routing with constant partition key

2021-05-25 Thread Amit Langote
Hou-san, On Wed, May 26, 2021 at 10:05 AM houzj.f...@fujitsu.com wrote: > From: Amit Langote > Sent: Tuesday, May 25, 2021 10:06 PM > > Though again, I think we can do this without changing the relcache > > interface, > > such as RelationGetPartitionQual(). > >

Re: Skip partition tuple routing with constant partition key

2021-05-25 Thread Amit Langote
Hou-san, On Mon, May 24, 2021 at 10:15 PM houzj.f...@fujitsu.com wrote: > From: Amit Langote > Sent: Monday, May 24, 2021 4:27 PM > > On Mon, May 24, 2021 at 10:31 AM houzj.f...@fujitsu.com > > wrote: > > > Currently for a normal RANGE partition key it will f

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-05-24 Thread Amit Langote
AGS = -DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE -DCLOBBER_CACHE_ALWAYS -D_GNU_SOURCE -- Amit Langote EDB: http://www.enterprisedb.com

Re: Skip partition tuple routing with constant partition key

2021-05-24 Thread Amit Langote
Hou-san, On Mon, May 24, 2021 at 10:31 AM houzj.f...@fujitsu.com wrote: > From: Amit Langote > Sent: Thursday, May 20, 2021 8:23 PM > > This one seems bit tough. ExecPartitionCheck() uses the generic expression > > evaluation machinery like a black box, which means exe

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-23 Thread Amit Langote
On Mon, May 24, 2021 at 12:16 PM osumi.takami...@fujitsu.com wrote: > On Saturday, May 22, 2021 11:58 AM Amit Langote > wrote: > > On Sat, May 22, 2021 at 11:00 AM osumi.takami...@fujitsu.com > > wrote: > > > I've checked the core file of v3's failure co

Re: Subscription tests fail under CLOBBER_CACHE_ALWAYS

2021-05-22 Thread Amit Langote
ecause > a04daa97a had changed a lot of the APIs here. So I left the APIs > for the xxx_internal() functions alone. Otherwise the patch > pretty much works as-is in v13. That looks reasonable. Thanks. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Subscription tests fail under CLOBBER_CACHE_ALWAYS

2021-05-21 Thread Amit Langote
On Sat, May 22, 2021 at 6:01 AM Tom Lane wrote: > Amit Langote writes: > > IMHO, it would be better to keep the lowest-level > > apply_handle_XXX_internal() out of this, because presumably we're only > > cleaning up the mess in higher-level callers. Somewhat related,

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-21 Thread Amit Langote
On Sat, May 22, 2021 at 11:00 AM osumi.takami...@fujitsu.com wrote: > On Friday, May 21, 2021 9:45 PM I worte: > > On Friday, May 21, 2021 4:43 PM Amit Langote > > wrote: > > > On Fri, May 21, 2021 at 3:55 PM osumi.takami...@fujitsu.com > > > wrote: > &

Re: Multi-Column List Partitioning

2021-05-21 Thread Amit Langote
On Fri, May 21, 2021 at 1:02 PM Amit Langote wrote: > I will now take a look at the patch itself. Some quick observations: * I get a lot of instances of the following 2 warnings when compiling the patched code: Warning #1: partprune.c: In function ‘get_matching_list_bounds’: partprune.c:2

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-21 Thread Amit Langote
On Fri, May 21, 2021 at 3:55 PM osumi.takami...@fujitsu.com wrote: > On Thursday, May 20, 2021 9:59 PM Amit Langote > wrote: > > Here are updated/divided patches. > Thanks for your updates. > > But, I've detected segmentation faults caused by the patch, > which can

Re: Multi-Column List Partitioning

2021-05-20 Thread Amit Langote
o your example would look like this: CREATE TABLE t2_1 PARTITION OF t2 FOR VALUES IN ((1, 2), (1, 5), (2, 2), (2, 10)); IMO, it is not such a bad syntax from a user's PoV. It's not hard to understand from this syntax that the partition constraint is something like (a, b) = (1, 2) OR (a, b) = (1, 5) OR ..., where the = performs row-wise comparison. I will now take a look at the patch itself. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-20 Thread Amit Langote
On Thu, May 20, 2021 at 5:59 PM osumi.takami...@fujitsu.com wrote: > On Tuesday, May 18, 2021 3:30 PM Amit Langote wrote: > > While doing so, it occurred to me (maybe not for the first time) that we are > > *unnecessarily* doing send_relation_and_attrs() for a relation if t

Re: Skip partition tuple routing with constant partition key

2021-05-20 Thread Amit Langote
Hou-san, On Thu, May 20, 2021 at 7:35 PM houzj.f...@fujitsu.com wrote: > From: Amit Langote > Sent: Wednesday, May 19, 2021 9:17 PM > > I guess it would be nice if we could fit in a solution for the use case > > that houjz > > mentioned as a special case. BTW, houjz, c

Re: Skip partition tuple routing with constant partition key

2021-05-20 Thread Amit Langote
On Thu, May 20, 2021 at 9:20 AM tsunakawa.ta...@fujitsu.com wrote: > From: Amit Langote > > On Tue, May 18, 2021 at 11:11 AM houzj.f...@fujitsu.com > > wrote: > > > For some big data scenario, we sometimes transfer data from one table(only > > store not expire

Re: Skip partition tuple routing with constant partition key

2021-05-20 Thread Amit Langote
On Thu, May 20, 2021 at 9:31 AM David Rowley wrote: > On Thu, 20 May 2021 at 01:17, Amit Langote wrote: > > I gave a shot to implementing your idea and ended up with the attached > > PoC patch, which does pass make check-world. > > I only had a quick look at this. >

Re: Subscription tests fail under CLOBBER_CACHE_ALWAYS

2021-05-19 Thread Amit Langote
On Thu, May 20, 2021 at 5:23 AM Tom Lane wrote: > Amit Langote writes: > > IOW, the patch you posted earlier seems like the way to go. > > I really dislike that patch. I think it's doubling down on the messy, > unstructured coding patterns that got us into this situatio

Re: Skip partition tuple routing with constant partition key

2021-05-19 Thread Amit Langote
ate partition not an acceptable solution to address this particular use case? Yeah, I know we should avoid encouraging users to perform DML directly on partitions, but... -- Amit Langote EDB: http://www.enterprisedb.com

Re: Skip partition tuple routing with constant partition key

2021-05-19 Thread Amit Langote
On Tue, May 18, 2021 at 10:28 AM David Rowley wrote: > On Tue, 18 May 2021 at 01:31, Amit Langote wrote: > > Hmm, does this seem common enough for the added complexity to be worthwhile? > > I'd also like to know if there's some genuine use case for this. For > testi

Re: Subscription tests fail under CLOBBER_CACHE_ALWAYS

2021-05-18 Thread Amit Langote
l(), after triggers of the subsequent INSERT can't be processed, instead causing: ERROR: AfterTriggerSaveEvent() called outside of query IOW, the patch you posted earlier seems like the way to go. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Subscription tests fail under CLOBBER_CACHE_ALWAYS

2021-05-18 Thread Amit Langote
re right that this is not a problem of 14~. I can reproduce the > problem on 13 as well, and we have no coverage for tuple routing with > triggers on this branch, so this would never have been stressed in the > buildfarm. There is a good argument to be made here in cherry-picking > 2ecfeda3 to REL_13_STABLE. +1 -- Amit Langote EDB: http://www.enterprisedb.com

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-17 Thread Amit Langote
On Mon, May 17, 2021 at 9:45 PM osumi.takami...@fujitsu.com wrote: > On Monday, May 17, 2021 6:52 PM Amit Langote wrote: > > Both patches are attached. > The patch for PG13 can be applied to it cleanly and the RT succeeded. > > I have few really minor comments on your com

Re: Skip partition tuple routing with constant partition key

2021-05-17 Thread Amit Langote
rthwhile? For an example of what's previously been considered worthwhile for a project like this, see what 0d5f05cde0 did. The cases it addressed are common enough -- a file being loaded into a (time range-) partitioned table using COPY FROM tends to have lines belonging to the same pa

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-17 Thread Amit Langote
On Mon, May 17, 2021 at 6:13 PM Amit Kapila wrote: > On Fri, May 14, 2021 at 12:44 PM Amit Langote wrote: > > On Thu, May 13, 2021 at 7:43 PM Amit Kapila wrote: > > > > > Also, don't we need to free the > > > entire map as suggested by me? > > >

Re: making update/delete of inheritance trees scale better

2021-05-16 Thread Amit Langote
partitions to share + * In an inherited UPDATE/DELETE it's important for child tables to share While at it, I also noticed that the comment refers to the row_identity_vars, but it can be unclear which variable it is referring to. So fixed that too. -- Amit Langote EDB: http://www.enterprisedb.com RowIdentityVarInfo-comment.patch Description: Binary data

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-14 Thread Amit Langote
Takamichi-san, On Fri, May 14, 2021 at 11:19 AM osumi.takami...@fujitsu.com wrote: > On Thursday, May 13, 2021 7:43 PM Amit Kapila wrote: > > On Tue, Apr 20, 2021 at 8:36 AM Amit Langote > > wrote: > > > Back in: > > https://www.postgresql.org/message-id/CA%2BHi

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-14 Thread Amit Langote
On Thu, May 13, 2021 at 7:43 PM Amit Kapila wrote: > On Tue, Apr 20, 2021 at 8:36 AM Amit Langote wrote: > > On Sat, Apr 17, 2021 at 1:30 PM Amit Kapila wrote: > > > On Fri, Apr 16, 2021 at 11:24 PM Andres Freund > > > wrote:> > This made me take a brie

Re: PG 14 release notes, first draft

2021-05-13 Thread Amit Langote
On Thu, May 13, 2021 at 11:59 PM Bruce Momjian wrote: > On Thu, May 13, 2021 at 02:46:58PM +0900, Amit Langote wrote: > > How about writing the 2nd line instead as: > > > > Updates/deletes on partitioned tables can now use execution-time > > partition pruning. >

Re: Inherited UPDATE/DELETE vs async execution

2021-05-13 Thread Amit Langote
On Thu, May 13, 2021 at 8:10 PM Etsuro Fujita wrote: > On Thu, May 13, 2021 at 5:00 PM Etsuro Fujita wrote: > > On Thu, May 13, 2021 at 3:32 PM Amit Langote > > wrote: > > > On Wed, May 12, 2021 at 6:45 PM Etsuro Fujita > > > wrote: > > > >

Re: Inherited UPDATE/DELETE vs async execution

2021-05-12 Thread Amit Langote
On Wed, May 12, 2021 at 6:45 PM Etsuro Fujita wrote: > On Tue, May 11, 2021 at 9:53 PM Amit Langote wrote: > > Okay, so I take it that making these ForeignScan nodes (that only > > fetch the data) asynchronous doesn't interfere with update/delete > > subsequently being

Re: PG 14 release notes, first draft

2021-05-12 Thread Amit Langote
On Thu, May 13, 2021 at 2:39 AM Bruce Momjian wrote: > On Tue, May 11, 2021 at 11:57:10AM +0900, Amit Langote wrote: > > On Mon, May 10, 2021 at 11:40 PM Justin Pryzby wrote: > > > 86dc90056d Rework planning and execution of UPDATE and DELETE. > > > a1115fa078 Post

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-05-12 Thread Amit Langote
On Fri, May 7, 2021 at 2:13 AM Álvaro Herrera wrote: > On 2021-Apr-30, Amit Langote wrote: > > > The case I was looking at is when a partition detach appears as > > in-progress to a serializable transaction. > > Yeah, I was exceedingly sloppy on my reasoning about this,

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-12 Thread Amit Langote
Takamichi-san, On Tue, Apr 27, 2021 at 9:37 PM osumi.takami...@fujitsu.com wrote: > On Tuesday, April 20, 2021 12:07 PM Amit Langote > wrote: > > On Sat, Apr 17, 2021 at 1:30 PM Amit Kapila > > wrote: > > > On Fri, Apr 16, 2021 at 11:24 PM Andres Freund > >

Re: Feedback on table expansion hook (including patch)

2021-05-12 Thread Amit Langote
o give partitioned > tables a special access method, and then shove most of the planner > and executor's hard-wired partitioning logic into access method > callbacks. That would make it a lot more feasible for extensions > to implement custom partitioning-like behavior ... or so I guess. Interesting proposition... -- Amit Langote EDB: http://www.enterprisedb.com

Re: Query regarding RANGE Partitioning

2021-05-12 Thread Amit Langote
ies insert any values for column 'a' between '0' > to '100', > either it should go to default partition if exists. Otherwise it should fail > saying, no partition > found. I feel there should be some way to create partitions in these > scenarios. Well, you simply need to come up with bound values for r3 that don't overlap with existing partitions' ranges; the following will work for example: create table r3 partition of r for values from (0,100) to (100,0); -- Amit Langote EDB: http://www.enterprisedb.com

Re: PG 14 release notes, first draft

2021-05-11 Thread Amit Langote
odifyTable on top to do the actual update/delete -- and because Append/MergeAppend nodes carry run-time pruning info, update/delete now get that ability. -- Amit Langote EDB: http://www.enterprisedb.com

<    1   2   3   4   5   6   7   8   9   10   >