Re: simplifying foreign key/RI checks

2021-11-11 Thread Amit Langote
On Fri, Nov 12, 2021 at 8:19 AM Tom Lane wrote: > Amit Langote writes: > > Rebased patches attached. > > I've spent some more time digging into the snapshot-management angle. Thanks for looking at this. > I think you are right that the crosscheck_snapshot isn't really

Re: Multi-Column List Partitioning

2021-10-29 Thread Amit Langote
Hi Nitin, On Fri, Oct 22, 2021 at 6:48 PM Nitin Jadhav wrote: > Thanks for sharing. I have fixed the issue in the attached patch. I noticed that there's no commitfest entry for this. Will you please add this to the next one? -- Amit Langote EDB: http://www.enterprisedb.com

Re: Data is copied twice when specifying both child and parent table in publication

2021-10-28 Thread Amit Langote
tables view exposes way less information to be able to determine how the tables listed are related to each other. So I think we'd need to fix the view to return leaf partitions' owning root table (if they're published through a pubviaroot publication) or rewrite the query that fetch_table_list() uses to access the underlying catalogs directly in the back-ported version. That is roughly how I'd go about this. Thoughts? -- Amit Langote EDB: http://www.enterprisedb.com

Re: pgstat_assert_is_up() can fail in walsender

2021-10-21 Thread Amit Langote
On Fri, Oct 22, 2021 at 2:14 AM Fujii Masao wrote: > On 2021/10/19 22:14, Amit Langote wrote: > > Hi, > > > > I can (almost) consistently reproduce $subject by executing the > > attached shell script, which I was using while constructing a test > > case for anothe

pgstat_assert_is_up() can fail in walsender

2021-10-19 Thread Amit Langote
n.c:198 cc'ing Andres and Horiguchi-san as pgstat_assert_is_up() is added in the recent commit ee3f8d3d3ae, though not sure if the problem is that commit's fault. I wonder if it may be of the adjacent commit fb2c5028e635. -- Amit Langote EDB: http://www.enterprisedb.com pubnode_dir=/tmp/pubnode pu

Re: Partition Check not updated when insert into a partition

2021-10-18 Thread Amit Langote
On Tue, Oct 19, 2021 at 7:14 AM Alvaro Herrera wrote: > Pushed now to all branches. Thanks much! Thanks Álavro. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Data is copied twice when specifying both child and parent table in publication

2021-10-18 Thread Amit Langote
tion explicitly, so it is up to the user to make sure that the tables are not added duplicatively. One may however argue that the way we've decided to handle FOR ALL TABLES does assume something about partitions where it skips advertising them to subscribers when publish_via_partition_root flag is set to true, but that is exactly to avoid the duplication of data that goes to a subscriber. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Partition Check not updated when insert into a partition

2021-10-18 Thread Amit Langote
1C4609A25186D1FBF194089%40OS3PR01MB5718.jpnprd01.prod.outlook.com Yeah, maybe the link is unnecessary in the test comment, so removed. Though, I do occasionally see one of those in the test files (try `git grep https src/test`). Thanks again. -- Amit Langote EDB: http://www.enterprisedb.com v6-0001-Invalidate-partitions-of-table-being-attached-det.patch Description: Binary data

Re: Multi-Column List Partitioning

2021-10-15 Thread Amit Langote
both of the partition keys (c, c), but it is not given the way have_partkey_equi_join() is currently coded. I suspect you'd get the same behavior if you'd used a RANGE partitioned table with keys (c, c). Not sure though if it'd be worthwhile to fix that coding to cater to this odd partition key setting. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Partition Check not updated when insert into a partition

2021-10-15 Thread Amit Langote
ren. I've updated the patch that way. (Also, reintroduced the slightly longer commit message that I had added in v3. :)) -- Amit Langote EDB: http://www.enterprisedb.com v5-0001-Invalidate-partitions-of-table-being-attached-det.patch Description: Binary data

Re: a misbehavior of partition row movement (?)

2021-10-14 Thread Amit Langote
On Mon, Sep 20, 2021 at 3:32 PM Amit Langote wrote: > The problem was that the tuplestore > (afterTriggers.query_stack[query_level].tuplestore) that I decided to > use to store the AFTER trigger tuples of a partitioned table that is > the target of an cross-partition updat

Re: [BUG] Unexpected action when publishing partition tables

2021-10-12 Thread Amit Langote
Hi Amit, On Fri, Oct 8, 2021 at 12:47 PM Amit Kapila wrote: > On Thu, Oct 7, 2021 at 12:39 PM Amit Langote wrote: > > Sorry that I didn't comment on this earlier, but I think either > > GetPubPartitionOptionRelations() or InvalidatePublicationRels() > > introduced in t

Re: [BUG] Unexpected action when publishing partition tables

2021-10-07 Thread Amit Langote
on this earlier, but I think either GetPubPartitionOptionRelations() or InvalidatePublicationRels() introduced in the commit 4548c76738b should lock the partitions, just like to the parent partitioned table would be, before invalidating them. There may be some hazards to invalidating a relation with

Re: a comment in joinrel.c: compute_partition_bounds()

2021-10-06 Thread Amit Langote
Fujita-san, On Wed, Oct 6, 2021 at 5:41 PM Etsuro Fujita wrote: > On Fri, Sep 24, 2021 at 4:20 PM Etsuro Fujita wrote: > > On Fri, Sep 24, 2021 at 3:34 PM Amit Langote > > wrote: > > > I think there's a word missing in the following comment: > > > >

Re: Multi-Column List Partitioning

2021-10-01 Thread Amit Langote
with my proposed changes so far integrated, but decided to just post a delta_v2 for now. -- Amit Langote EDB: http://www.enterprisedb.com v3_delta_amit_v2.patch Description: Binary data

Re: Record a Bitmapset of non-pruned partitions

2021-09-30 Thread Amit Langote
On Fri, Oct 1, 2021 at 7:07 AM David Rowley wrote: > On Thu, 30 Sept 2021 at 20:25, Amit Langote wrote: > > Related to the above, I noticed while looking at > > build_merged_partition_bounds() that db632fbca3 missed adding a line > > to that function to set interleaved_pa

Re: Record a Bitmapset of non-pruned partitions

2021-09-30 Thread Amit Langote
Hi David, On Mon, Aug 2, 2021 at 11:59 AM Amit Langote wrote: > On Sun, Aug 1, 2021 at 9:31 PM David Rowley wrote: > > On Fri, 30 Jul 2021 at 19:10, Amit Langote wrote: > > > interleaved_parts idea looks clever. I wonder if you decided that > > > it's maybe n

a comment in joinrel.c: compute_partition_bounds()

2021-09-24 Thread Amit Langote
* cardinal positions form the pairs. ": the join rel have the same..." seems to be missing a "will". Attached a patch to fix. -- Amit Langote EDB: http://www.enterprisedb.com comment-missing-word.patch Description: Binary data

Re: a misbehavior of partition row movement (?)

2021-09-20 Thread Amit Langote
On Fri, Sep 10, 2021 at 11:03 PM Amit Langote wrote: > On Fri, Sep 3, 2021 at 12:23 PM Amit Langote wrote: > > Hi Andrew, > > > > On Fri, Sep 3, 2021 at 6:19 AM Andrew Dunstan wrote: > > > On 7/13/21 8:09 AM, Amit Langote wrote: > > > > Unfortunatel

Re: resowner module README needs update?

2021-09-15 Thread Amit Langote
On Wed, Sep 15, 2021 at 4:09 PM Michael Paquier wrote: > On Tue, Sep 14, 2021 at 07:37:28PM +0900, Michael Paquier wrote: > > No objections here. > > Done as of cae6fc2. Thank you. -- Amit Langote EDB: http://www.enterprisedb.com

Re: resowner module README needs update?

2021-09-13 Thread Amit Langote
Thanks for looking. On Tue, Sep 14, 2021 at 9:17 AM Michael Paquier wrote: > On Mon, Sep 13, 2021 at 10:44:11PM +0900, Amit Langote wrote: > > It seems a bunch of other object/resource types have been integrated > > into the resowner mechanism since the list was las

resowner module README needs update?

2021-09-13 Thread Amit Langote
after updating becomes a bit too long. -- Amit Langote EDB: http://www.enterprisedb.com update-resowner-readme.patch Description: Binary data

Re: a misbehavior of partition row movement (?)

2021-09-10 Thread Amit Langote
On Fri, Sep 3, 2021 at 12:23 PM Amit Langote wrote: > Hi Andrew, > > On Fri, Sep 3, 2021 at 6:19 AM Andrew Dunstan wrote: > > On 7/13/21 8:09 AM, Amit Langote wrote: > > > Unfortunately, I don’t think I’ll have time in this CF to solve some > > > very fundam

Re: A reloption for partitioned tables - parallel_workers

2021-09-03 Thread Amit Langote
On Sat, Sep 4, 2021 at 3:10 Laurenz Albe wrote: > On Fri, 2021-09-03 at 18:24 +0200, Daniel Gustafsson wrote: > > > On 6 Apr 2021, at 09:46, Amit Langote wrote: > > > On Fri, Apr 2, 2021 at 11:36 PM Laurenz Albe > wrote: > > > > > > I don't know if S

Re: Multi-Column List Partitioning

2021-09-03 Thread Amit Langote
On Wed, Sep 1, 2021 at 2:31 PM Amit Langote wrote: > On Tue, Aug 31, 2021 at 8:02 PM Nitin Jadhav > wrote: > > The attached patch also fixes the above comments. > > I noticed that multi-column list partitions containing NULLs don't > work correctly with partition pruning yet

Re: a misbehavior of partition row movement (?)

2021-09-02 Thread Amit Langote
Hi Andrew, On Fri, Sep 3, 2021 at 6:19 AM Andrew Dunstan wrote: > On 7/13/21 8:09 AM, Amit Langote wrote: > > Unfortunately, I don’t think I’ll have time in this CF to solve some > > very fundamental issues I found in the patch during the last cycle. > > I’m fine with eithe

Re: Multi-Column List Partitioning

2021-08-31 Thread Amit Langote
nse to rename the struct while we're at it. Call it, say, PartitionListBound? Also, please keep part of the existing comment that says that the bound belongs to index'th partition. Will send more comments in a bit... -- Amit Langote EDB: http://www.enterprisedb.com

Re: Multi-Column List Partitioning

2021-08-30 Thread Amit Langote
t of p01 should actually be: (a IS NOT NULL) AND (a = 1) AND (b IS NOT NULL) AND (b = 1) AND (c IS NOT NULL) AND (c = true) As per the patch's current implementation, tuple (1, NULL, true) passes p01's partition constraint, because only (b = 1) is not sufficient to reject a NULL value being assigned to b. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Partition Check not updated when insert into a partition

2021-08-27 Thread Amit Langote
On Thu, Aug 5, 2021 at 11:32 AM Amit Langote wrote: > On Wed, Jul 14, 2021 at 11:16 AM houzj.f...@fujitsu.com > > On Tuesday, July 13, 2021 2:52 AM Alvaro Herrera > > wrote: > > > Did you have a misbehaving test for the ATTACH case? > > > > Thanks for the res

Re: Allow batched insert during cross-partition updates

2021-08-23 Thread Amit Langote
On Tue, Jul 27, 2021 at 11:32 AM Amit Langote wrote: > On Thu, Jul 22, 2021 at 2:18 PM vignesh C wrote: > > One of the test postgres_fdw has failed, can you please post an > > updated patch for the fix: > > test postgres_fdw ... FAILED (test process exited

Re: Partition Check not updated when insert into a partition

2021-08-04 Thread Amit Langote
de already locks the descendents for a different purpose, but DETACH doesn't, so the latter needs to be fixed to match. I've updated Alvaro's patch to address these points. Maybe, we should also add these cases to the regression and isolation suites? -- Amit Langote EDB: http://www.enterprisedb.com inval-partitions_v2.patch Description: Binary data

Re: Skip partition tuple routing with constant partition key

2021-08-02 Thread Amit Langote
I noticed that there is no CF entry for this, so created one in the next CF: https://commitfest.postgresql.org/34/3270/ Rebased patch attached. v8-0001-Teach-get_partition_for_tuple-to-cache-bound-offs.patch Description: Binary data

Re: Record a Bitmapset of non-pruned partitions

2021-08-01 Thread Amit Langote
On Sun, Aug 1, 2021 at 9:31 PM David Rowley wrote: > On Fri, 30 Jul 2021 at 19:10, Amit Langote wrote: > > interleaved_parts idea looks clever. I wonder if you decided that > > it's maybe not worth setting that field in the joinrel's > > PartitionBoundInfos? For exa

Re: Record a Bitmapset of non-pruned partitions

2021-07-30 Thread Amit Langote
ms of partitioning-defined ordering ]." Looks fine otherwise. -- Amit Langote EDB: http://www.enterprisedb.com

Re: a thinko in b676ac443b6

2021-07-27 Thread Amit Langote
On Wed, Jul 28, 2021 at 1:07 AM Tomas Vondra wrote: > On 7/27/21 4:28 AM, Amit Langote wrote: > > I think it can be incorrect to use the same TupleDesc for both the > > slots in ri_Slots (for ready-to-be-inserted tuples) and ri_PlanSlots > > (for subplan output tuples

Re: Allow batched insert during cross-partition updates

2021-07-26 Thread Amit Langote
On Thu, Jul 22, 2021 at 2:18 PM vignesh C wrote: > On Fri, Jul 2, 2021 at 7:35 AM Amit Langote wrote: > > On Fri, Jul 2, 2021 at 1:39 AM Tom Lane wrote: > > > > > > Amit Langote writes: > > > > [ v6-0001-Allow-batching-of-inserts-during-cross-parti

a thinko in b676ac443b6

2021-07-26 Thread Amit Langote
TupleDesc exactly (plus any junk columns), but now it produces only a partial tuple containing the values for the changed columns. So it's better to revert to using planSlot->tts_tupleDescriptor for the slots in ri_PlanSlots. Attached a patch to do so. -- Amit Langote EDB: ht

Re: a misbehavior of partition row movement (?)

2021-07-13 Thread Amit Langote
Hi Ibrar, Sawada-san, On Tue, Jul 13, 2021 at 20:25 Ibrar Ahmed wrote: > > > On Fri, Apr 2, 2021 at 6:09 PM Amit Langote > wrote: > >> On Thu, Apr 1, 2021 at 10:56 AM Masahiko Sawada >> wrote: >> > On Tue, Mar 23, 2021 at 6:27 PM Amit Langote >>

Re: ExecRTCheckPerms() and many prunable partitions

2021-07-07 Thread Amit Langote
On Wed, Jul 7, 2021 at 1:41 PM David Rowley wrote: > On Fri, 2 Jul 2021 at 12:41, Amit Langote wrote: > > I'll mark the CF entry as WoA, unless you'd rather I just mark it RwF. > > I've set it to waiting on author. It was still set to needs review. Sorry it slipped my mind to do

Re: simplifying foreign key/RI checks

2021-07-05 Thread Amit Langote
d patches attached. -- Amit Langote EDB: http://www.enterprisedb.com v9-0001-Export-get_partition_for_tuple.patch Description: Binary data v9-0002-Avoid-using-SPI-for-some-RI-checks.patch Description: Binary data

Re: Allow batched insert during cross-partition updates

2021-07-01 Thread Amit Langote
On Fri, Jul 2, 2021 at 1:39 AM Tom Lane wrote: > > Amit Langote writes: > > [ v6-0001-Allow-batching-of-inserts-during-cross-partition-.patch ] > > Per the cfbot, this isn't applying anymore, so I'm setting it back > to Waiting on Author. Rebased patch attached. Th

Re: New committers: Daniel Gustafsson and John Naylor

2021-07-01 Thread Amit Langote
gt; > > > Please join me in wishing them much success and few bugs. > > > > regards, tom lane > > Congrats to you both ! +1, congrats Daniel & John! -- Amit Langote EDB: http://www.enterprisedb.com

Re: ExecRTCheckPerms() and many prunable partitions

2021-07-01 Thread Amit Langote
On Fri, Jul 2, 2021 at 12:45 AM Tom Lane wrote: > Amit Langote writes: > > The problem is that it loops over the entire range table even though > > only one or handful of those entries actually need their permissions > > checked. Most entries, especially those of partition

Re: Record a Bitmapset of non-pruned partitions

2021-06-30 Thread Amit Langote
gs elsewhere such that that function is not needed in the first place [2], something I keep thinking about in between doing other things, but never sit down to actually write a patch. Given that you're proposing more uses for live_parts, maybe he'd be open to the idea. -- Amit Langote EDB: http://www.enterprisedb.com [1] https://www.postgresql.org/message-id/3f280722-46f2-c2a4-4c19-2cfa28c6c1cd%40lab.ntt.co.jp [2] https://www.postgresql.org/message-id/3529.1554051867%40sss.pgh.pa.us

Re: ExecRTCheckPerms() and many prunable partitions

2021-06-30 Thread Amit Langote
On Wed, Jun 30, 2021 at 23:34 David Rowley wrote: > On Thu, 1 Jul 2021 at 01:34, Amit Langote wrote: > > For now, I have implemented the idea 2 as the attached patch. > > I only just had a fleeting glance at the patch. Aren't you > accidentally missing the 0th RTE here?

ExecRTCheckPerms() and many prunable partitions

2021-06-30 Thread Amit Langote
o while still kind of PoC, will add this to July CF for keeping track. -- Amit Langote EDB: http://www.enterprisedb.com [1] https://www.postgresql.org/message-id/ca+hiwqg7zrubmmih3wpsbz4s0h2ehywrnxeducky5hr3fwz...@mail.gmail.com [2] https://www.postgre

Re: Deadlock risk while inserting directly into partition?

2021-06-28 Thread Amit Langote
On Mon, Jun 28, 2021 at 12:58 PM Amit Langote wrote: > On Fri, Jun 25, 2021 at 10:26 AM David Rowley wrote: > > On Thu, 24 Jun 2021 at 12:32, David Rowley wrote: > > > The overhead of taking these locks is pretty significant for > > > partitioned tables with lots

Re: Deadlock risk while inserting directly into partition?

2021-06-27 Thread Amit Langote
d/16db1458-67cf-4add-736e-31b053115e8e%40lab.ntt.co.jp Maybe a good idea to add a line or 2 in 5.11. Table Partitioning? -- Amit Langote EDB: http://www.enterprisedb.com

Re: Deadlock risk while inserting directly into partition?

2021-06-27 Thread Amit Langote
r proper (mainly InitPlan) should also see the shrunken version of the range table. Not to mention the complexity of getting the "init" pruning itself to run outside a full-blown executor context. Anyway, do you agree with starting a thread to discuss possible approaches to a

Re: Decoding speculative insert with toast leaks memory

2021-06-17 Thread Amit Langote
Hi Dilip, On Thu, Jun 17, 2021 at 4:45 PM Dilip Kumar wrote: > On Thu, Jun 17, 2021 at 12:52 PM Amit Langote wrote: > > > Oh I missed that the problem report is for the PG13 branch. > > > > How about the attached patch then? > > > Looks good, Thanks for ch

Re: Skip partition tuple routing with constant partition key

2021-06-17 Thread Amit Langote
On Thu, Jun 17, 2021 at 4:18 PM Zhihong Yu wrote: > On Wed, Jun 16, 2021 at 10:37 PM Amit Langote wrote: >> > + if (part_index < 0) >> > + { >> > + bound_offset = >> > partition_range_datum_bsea

Re: Decoding speculative insert with toast leaks memory

2021-06-17 Thread Amit Langote
On Thu, Jun 17, 2021 at 3:42 PM Amit Kapila wrote: > On Thu, Jun 17, 2021 at 10:39 AM Amit Langote wrote: > > > > On Thu, Jun 17, 2021 at 12:56 PM Amit Kapila > > wrote: > > > On Wed, Jun 16, 2021 at 8:18 PM Tom Lane wrote: > > > > >

Re: Skip partition tuple routing with constant partition key

2021-06-16 Thread Amit Langote
Hi, Thanks for reading the patch. On Thu, Jun 17, 2021 at 1:46 PM Zhihong Yu wrote: > On Wed, Jun 16, 2021 at 9:29 PM Amit Langote wrote: >> Attached a slightly revised version of that patch, with a commit >> message this time. > > + int n_tup

Re: Decoding speculative insert with toast leaks memory

2021-06-16 Thread Amit Langote
blish_as_relid = InvalidOid; entry->map = NULL; /* will be set by maybe_send_schema() if needed */ } Do we need the same statement at the end of the following block? /* Validate the entry */ if (!entry->replicate_valid) { -- Amit Langote EDB: http://www.enterprisedb.com

Re: Skip partition tuple routing with constant partition key

2021-06-16 Thread Amit Langote
On Wed, Jun 16, 2021 at 4:27 PM Amit Langote wrote: > On Mon, Jun 7, 2021 at 8:38 PM houzj.f...@fujitsu.com > wrote: > > The rough idea is to check the average batch number every 1000 rows. > > If the average batch num is greater than 1, then we enable the cache check, > &g

Re: Skip partition tuple routing with constant partition key

2021-06-16 Thread Amit Langote
rted may be more adaptive. Please find attached a revised version of your patch, where I tried to make it a bit easier to follow, hopefully. While doing so, I realized that caching the bound offset across queries makes little sense now, so I decided to keep the changes confined to execPartition.c.

Re: Multi-Column List Partitioning

2021-06-11 Thread Amit Langote
On Fri, Jun 11, 2021 at 12:37 PM Amit Langote wrote: > I will look at other parts of the patch next week hopefully. For > now, attached is a delta patch that applies on top of your v1, which > does: > > * Simplify partition_list_bsearch() and partition_lbound_datum

Re: Multi-Column List Partitioning

2021-06-10 Thread Amit Langote
* Make qsort_partition_list_value_cmp simply call partition_lbound_datum_cmp() instead of having its own logic to compare input bounds * Move partition_lbound_datum_cmp() into partbounds.c as a static function (export seems unnecessary) * Add a comment for PartitionBoundInfo.isnulls and remove that for null_index -- Amit Langote EDB: http://www.enterprisedb.com v1_delta_amit.patch Description: Binary data

Re: Skip partition tuple routing with constant partition key

2021-06-04 Thread Amit Langote
On Fri, Jun 4, 2021 at 6:05 PM Amit Langote wrote: > On Fri, Jun 4, 2021 at 4:38 PM Amit Langote wrote: > > On Thu, Jun 3, 2021 at 8:48 PM Amit Langote wrote: > > > On Tue, Jun 1, 2021 at 5:43 PM houzj.f...@fujitsu.com > > > wrote: > > > > So, If we wan

Re: Skip partition tuple routing with constant partition key

2021-06-04 Thread Amit Langote
On Fri, Jun 4, 2021 at 4:38 PM Amit Langote wrote: > On Thu, Jun 3, 2021 at 8:48 PM Amit Langote wrote: > > On Tue, Jun 1, 2021 at 5:43 PM houzj.f...@fujitsu.com > > wrote: > > > So, If we want to share the cached partition between statements, we seems > > > ca

Re: Skip partition tuple routing with constant partition key

2021-06-04 Thread Amit Langote
On Thu, Jun 3, 2021 at 8:48 PM Amit Langote wrote: > On Tue, Jun 1, 2021 at 5:43 PM houzj.f...@fujitsu.com > wrote: > > So, If we want to share the cached partition between statements, we seems > > cannot > > use ExecPartitionCheck. Instead, I tried directly invoke th

Re: parent foreign tables and row marks

2021-06-03 Thread Amit Langote
On Thu, Jun 3, 2021 at 10:07 AM Amit Langote wrote: > On Thu, Jun 3, 2021 at 3:39 AM Tom Lane wrote: > > > > Amit Langote writes: > > > I noticed that 428b260f87e8 (v12) broke the cases where a parent > > > foreign table has row marks assigned. > >

Re: Skip partition tuple routing with constant partition key

2021-06-03 Thread Amit Langote
Hou-san, On Tue, Jun 1, 2021 at 5:43 PM houzj.f...@fujitsu.com wrote: > From: Amit Langote > > > > Where are you thinking to cache the partidx list? Inside > > > > PartitionDesc or some executor struct? > > > > > > I was thinking cach

Re: parent foreign tables and row marks

2021-06-02 Thread Amit Langote
On Thu, Jun 3, 2021 at 3:39 AM Tom Lane wrote: > > Amit Langote writes: > > I noticed that 428b260f87e8 (v12) broke the cases where a parent > > foreign table has row marks assigned. > > Indeed :-(. Fix pushed. I tweaked the comments and test case slightly. Thank yo

Re: join pushdown and issue with foreign update

2021-06-02 Thread Amit Langote
->relid); + if (!OidIsValid(reltype)) + continue; + att->atttypid = reltype; That may be a valid concern. I wonder if it would make sense to also check varattno == 0 here somewhere for good measure. -- Amit Langote EDB: http://www.enterprisedb.com

Re: join pushdown and issue with foreign update

2021-06-02 Thread Amit Langote
t think we have other cases that are affected at this point. I have a feeling that your patch will have fixed things enough that the same problem will not occur when we have join pushdown under UPDATE occurring in more cases. -- Amit Langote EDB: http://www.enterprisedb.com

Re: join pushdown and issue with foreign update

2021-06-01 Thread Amit Langote
add "wholerow" with type of one child relation > and then > try to use it with another child (of different table type). Perhaps, we can get away with adding the wholerow Var with the target relation's reltype when the target foreign table is not a "child" relation, but the root target relation itself. Maybe like the attached? -- Amit Langote EDB: http://www.enterprisedb.com wholerow-rowid-var-vartype-fix.patch Description: Binary data

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-06-01 Thread Amit Langote
On Tue, Jun 1, 2021 at 6:56 PM Amit Kapila wrote: > On Mon, May 31, 2021 at 8:51 AM Amit Langote wrote: > > On Thu, May 27, 2021 at 3:36 PM Amit Kapila wrote: > > > Why do we need to move send_relation_and_attrs() call? I think it > > > doesn't matter much either wa

parent foreign tables and row marks

2021-06-01 Thread Amit Langote
umn if needed. Attached a patch for that, which also adds the test cases. Actually, I had to make a separate version of the patch to apply to the v12 branch, because EXPLAIN outputs relation aliases a bit differently starting in v13, which is attached too. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-30 Thread Amit Langote
On Thu, May 27, 2021 at 3:36 PM Amit Kapila wrote: > On Fri, May 21, 2021 at 1:12 PM Amit Langote wrote: > > > > Hmm, maybe get_rel_syn_entry() should explicitly set map to NULL when > > first initializing an entry. It's possible that without doing so, the > > map rem

Re: Skip partition tuple routing with constant partition key

2021-05-27 Thread Amit Langote
Hou-san, On Thu, May 27, 2021 at 3:56 PM houzj.f...@fujitsu.com wrote: > From: Amit Langote > Sent: Thursday, May 27, 2021 1:54 PM > > On Thu, May 27, 2021 at 11:47 AM houzj.f...@fujitsu.com > > wrote: > > > About teaching relcache about caching the target parti

Re: Skip partition tuple routing with constant partition key

2021-05-26 Thread Amit Langote
gt; What do you think ? Where are you thinking to cache the partidx list? Inside PartitionDesc or some executor struct? -- Amit Langote EDB: http://www.enterprisedb.com

Re: Skip partition tuple routing with constant partition key

2021-05-26 Thread Amit Langote
On Thu, May 27, 2021 at 1:55 PM Zhihong Yu wrote: > For CanUseSavedPartitionForTuple, nit: you can check > !dispatch->savedPartResultInfo at the beginning and return early. > This would save some indentation. Sure, see the attached. -- Amit Langote EDB: http://www.enterprisedb.c

Re: Skip partition tuple routing with constant partition key

2021-05-26 Thread Amit Langote
ybe we don't need this parent-slot-clearing anymore due to code restructuring over the last 3 years, but that will have to be a separate patch. I hope the attached updated patch makes it a bit more clear what's going on. I refactored more of the code in ExecFindPartition() to make this patc

Re: Multi-Column List Partitioning

2021-05-25 Thread Amit Langote
e above 2. Approach 1 sounds better. It sounds like approach 1 might help us implement support for allowing NULLs in range partition bounds in the future, if at all. For now, it might be better to not allocate the isnull array except for list partitioning. I'll wait for you to post a

Re: Skip partition tuple routing with constant partition key

2021-05-25 Thread Amit Langote
Hou-san, On Wed, May 26, 2021 at 10:05 AM houzj.f...@fujitsu.com wrote: > From: Amit Langote > Sent: Tuesday, May 25, 2021 10:06 PM > > Though again, I think we can do this without changing the relcache > > interface, > > such as RelationGetPartitionQual(). > >

Re: Skip partition tuple routing with constant partition key

2021-05-25 Thread Amit Langote
Hou-san, On Mon, May 24, 2021 at 10:15 PM houzj.f...@fujitsu.com wrote: > From: Amit Langote > Sent: Monday, May 24, 2021 4:27 PM > > On Mon, May 24, 2021 at 10:31 AM houzj.f...@fujitsu.com > > wrote: > > > Currently for a normal RANGE partition key it will f

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-05-24 Thread Amit Langote
AGS = -DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE -DCLOBBER_CACHE_ALWAYS -D_GNU_SOURCE -- Amit Langote EDB: http://www.enterprisedb.com

Re: Skip partition tuple routing with constant partition key

2021-05-24 Thread Amit Langote
Hou-san, On Mon, May 24, 2021 at 10:31 AM houzj.f...@fujitsu.com wrote: > From: Amit Langote > Sent: Thursday, May 20, 2021 8:23 PM > > This one seems bit tough. ExecPartitionCheck() uses the generic expression > > evaluation machinery like a black box, which means exe

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-23 Thread Amit Langote
On Mon, May 24, 2021 at 12:16 PM osumi.takami...@fujitsu.com wrote: > On Saturday, May 22, 2021 11:58 AM Amit Langote > wrote: > > On Sat, May 22, 2021 at 11:00 AM osumi.takami...@fujitsu.com > > wrote: > > > I've checked the core file of v3's failure core and pri

Re: Subscription tests fail under CLOBBER_CACHE_ALWAYS

2021-05-22 Thread Amit Langote
a had changed a lot of the APIs here. So I left the APIs > for the xxx_internal() functions alone. Otherwise the patch > pretty much works as-is in v13. That looks reasonable. Thanks. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Subscription tests fail under CLOBBER_CACHE_ALWAYS

2021-05-21 Thread Amit Langote
On Sat, May 22, 2021 at 6:01 AM Tom Lane wrote: > Amit Langote writes: > > IMHO, it would be better to keep the lowest-level > > apply_handle_XXX_internal() out of this, because presumably we're only > > cleaning up the mess in higher-level callers. Somewhat related, one

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-21 Thread Amit Langote
On Sat, May 22, 2021 at 11:00 AM osumi.takami...@fujitsu.com wrote: > On Friday, May 21, 2021 9:45 PM I worte: > > On Friday, May 21, 2021 4:43 PM Amit Langote > > wrote: > > > On Fri, May 21, 2021 at 3:55 PM osumi.takami...@fujitsu.com > > > wrote: > &

Re: Multi-Column List Partitioning

2021-05-21 Thread Amit Langote
On Fri, May 21, 2021 at 1:02 PM Amit Langote wrote: > I will now take a look at the patch itself. Some quick observations: * I get a lot of instances of the following 2 warnings when compiling the patched code: Warning #1: partprune.c: In function ‘get_matching_list_bounds’: partprune.c:2

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-21 Thread Amit Langote
On Fri, May 21, 2021 at 3:55 PM osumi.takami...@fujitsu.com wrote: > On Thursday, May 20, 2021 9:59 PM Amit Langote > wrote: > > Here are updated/divided patches. > Thanks for your updates. > > But, I've detected segmentation faults caused by the patch, > which can h

Re: Multi-Column List Partitioning

2021-05-20 Thread Amit Langote
EATE TABLE t2_1 PARTITION OF t2 FOR VALUES IN ((1, 2), (1, 5), (2, 2), (2, 10)); IMO, it is not such a bad syntax from a user's PoV. It's not hard to understand from this syntax that the partition constraint is something like (a, b) = (1, 2) OR (a, b) = (1, 5) OR ..., where the = performs row-wise comparison. I will now take a look at the patch itself. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-20 Thread Amit Langote
On Thu, May 20, 2021 at 5:59 PM osumi.takami...@fujitsu.com wrote: > On Tuesday, May 18, 2021 3:30 PM Amit Langote wrote: > > While doing so, it occurred to me (maybe not for the first time) that we are > > *unnecessarily* doing send_relation_and_attrs() for a relation if t

Re: Skip partition tuple routing with constant partition key

2021-05-20 Thread Amit Langote
Hou-san, On Thu, May 20, 2021 at 7:35 PM houzj.f...@fujitsu.com wrote: > From: Amit Langote > Sent: Wednesday, May 19, 2021 9:17 PM > > I guess it would be nice if we could fit in a solution for the use case > > that houjz > > mentioned as a special case. BTW, houjz

Re: Skip partition tuple routing with constant partition key

2021-05-20 Thread Amit Langote
On Thu, May 20, 2021 at 9:20 AM tsunakawa.ta...@fujitsu.com wrote: > From: Amit Langote > > On Tue, May 18, 2021 at 11:11 AM houzj.f...@fujitsu.com > > wrote: > > > For some big data scenario, we sometimes transfer data from one table(only > > store not expire

Re: Skip partition tuple routing with constant partition key

2021-05-20 Thread Amit Langote
On Thu, May 20, 2021 at 9:31 AM David Rowley wrote: > On Thu, 20 May 2021 at 01:17, Amit Langote wrote: > > I gave a shot to implementing your idea and ended up with the attached > > PoC patch, which does pass make check-world. > > I only had a quick look at this. >

Re: Subscription tests fail under CLOBBER_CACHE_ALWAYS

2021-05-19 Thread Amit Langote
On Thu, May 20, 2021 at 5:23 AM Tom Lane wrote: > Amit Langote writes: > > IOW, the patch you posted earlier seems like the way to go. > > I really dislike that patch. I think it's doubling down on the messy, > unstructured coding patterns that got us into this situation to

Re: Skip partition tuple routing with constant partition key

2021-05-19 Thread Amit Langote
ion not an acceptable solution to address this particular use case? Yeah, I know we should avoid encouraging users to perform DML directly on partitions, but... -- Amit Langote EDB: http://www.enterprisedb.com

Re: Skip partition tuple routing with constant partition key

2021-05-19 Thread Amit Langote
On Tue, May 18, 2021 at 10:28 AM David Rowley wrote: > On Tue, 18 May 2021 at 01:31, Amit Langote wrote: > > Hmm, does this seem common enough for the added complexity to be worthwhile? > > I'd also like to know if there's some genuine use case for this. For > testing purp

Re: Subscription tests fail under CLOBBER_CACHE_ALWAYS

2021-05-19 Thread Amit Langote
triggers of the subsequent INSERT can't be processed, instead causing: ERROR: AfterTriggerSaveEvent() called outside of query IOW, the patch you posted earlier seems like the way to go. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Subscription tests fail under CLOBBER_CACHE_ALWAYS

2021-05-18 Thread Amit Langote
a problem of 14~. I can reproduce the > problem on 13 as well, and we have no coverage for tuple routing with > triggers on this branch, so this would never have been stressed in the > buildfarm. There is a good argument to be made here in cherry-picking > 2ecfeda3 to REL_13_STABLE. +1 -- Amit Langote EDB: http://www.enterprisedb.com

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-18 Thread Amit Langote
On Mon, May 17, 2021 at 9:45 PM osumi.takami...@fujitsu.com wrote: > On Monday, May 17, 2021 6:52 PM Amit Langote wrote: > > Both patches are attached. > The patch for PG13 can be applied to it cleanly and the RT succeeded. > > I have few really minor comments on your com

Re: Skip partition tuple routing with constant partition key

2021-05-17 Thread Amit Langote
rthwhile? For an example of what's previously been considered worthwhile for a project like this, see what 0d5f05cde0 did. The cases it addressed are common enough -- a file being loaded into a (time range-) partitioned table using COPY FROM tends to have lines belonging to the same partiti

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-17 Thread Amit Langote
On Mon, May 17, 2021 at 6:13 PM Amit Kapila wrote: > On Fri, May 14, 2021 at 12:44 PM Amit Langote wrote: > > On Thu, May 13, 2021 at 7:43 PM Amit Kapila wrote: > > > > > Also, don't we need to free the > > > entire map as suggested by me? > > >

Re: making update/delete of inheritance trees scale better

2021-05-17 Thread Amit Langote
artitions to share + * In an inherited UPDATE/DELETE it's important for child tables to share While at it, I also noticed that the comment refers to the row_identity_vars, but it can be unclear which variable it is referring to. So fixed that too. -- Amit Langote EDB: http://www.enterprisedb.com RowIdentityVarInfo-comment.patch Description: Binary data

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-14 Thread Amit Langote
Takamichi-san, On Fri, May 14, 2021 at 11:19 AM osumi.takami...@fujitsu.com wrote: > On Thursday, May 13, 2021 7:43 PM Amit Kapila wrote: > > On Tue, Apr 20, 2021 at 8:36 AM Amit Langote > > wrote: > > > Back in: > > https://www.postgresql.org/message-id/CA%2BHi

<    1   2   3   4   5   6   7   8   9   10   >