Re: speeding up planning with partitions

2018-11-14 Thread Amit Langote
On 2018/11/15 10:19, Imai, Yoshikazu wrote: > On Tue, Nov 13, 2018 at 10:29 PM, Amit Langote wrote: >> On 2018/11/12 13:35, Imai, Yoshikazu wrote: >>> How amount of memory is used with above tests is... >>> >>> without v5 patches, Set1: 242MB >>> withou

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-11-14 Thread Amit Langote
On 2018/11/15 8:58, Alvaro Herrera wrote: > On 2018-Nov-15, David Rowley wrote: > >> On 15 November 2018 at 07:10, Alvaro Herrera >> wrote: >>> What's with this comment? >>> >>> * Initially we must only set up 1 PartitionDispatch object; the >>> one for >>> * the partitioned

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-14 Thread Amit Langote
On 2018/11/15 4:27, Robert Haas wrote: > RelationBuildPartitionDesc doesn't lock the children > whose relpartbounds it is fetching (!), so unless we're guaranteed to > have already locked them children earlier for some other reason, we > could grab the partition bound at this point and then it

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-14 Thread Amit Langote
On 2018/11/15 14:38, Michael Paquier wrote: > On Thu, Nov 15, 2018 at 01:38:55PM +0900, Amit Langote wrote: >> I've fixed 0001 again to re-order the code so that allocations happen the >> correct context and now tests pass with the rebased patches. > > I have been looking

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-14 Thread Amit Langote
On 2018/11/15 11:03, Amit Langote wrote: > As Michael pointed out, the first cleanup patch needs to be rebased due to > a recent commit [1]. I did that to see if something we did in that commit > made things worse for your patch, but seems fine. I had to go and change > th

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-14 Thread Amit Langote
On 2018/11/15 15:22, Michael Paquier wrote: >> If there are no partitions, nparts is 0, and other fields are NULL, though >> rd_partdesc itself is never NULL. > > I find a bit confusing that both concepts have the same meaning, aka > that a relation has no partition, and that it is actually

Re: using expression syntax for partition bounds

2018-11-08 Thread Amit Langote
ors: Kyotaro Horiguchi, Tom Lane, Amit Langote --- doc/src/sgml/ref/alter_table.sgml | 6 +- doc/src/sgml/ref/create_table.sgml | 16 +-- src/backend/commands/tablecmds.c | 9 ++ src/backend/optimizer/util/clauses.c | 4 +- src/backend/parser/gra

Re: speeding up planning with partitions

2018-11-09 Thread Amit Langote
Imai-san, On 2018/11/07 10:00, Imai, Yoshikazu wrote: > About inheritance_make_rel_from_joinlist(), I considered how it processes > joins for sub-partitioned-table. > > sub-partitioned-table image: > part > sub1 > leaf1 > leaf2 > > inheritance_make_rel_from_joinlist() translates

Re: speeding up planning with partitions

2018-11-09 Thread Amit Langote
Imai-san, Thank you for reviewing. On 2018/11/05 17:28, Imai, Yoshikazu wrote: > Since it takes me a lot of time to see all of the patches, I will post > comments > little by little from easy parts like typo check. I've fixed the typos you pointed out. > 4. > 0004: > -static void

Re: Performance improvements of INSERTs to a partitioned table

2018-11-08 Thread Amit Langote
On 2018/11/09 11:19, David Rowley wrote: > On 7 November 2018 at 21:31, Kato, Sho wrote: >> AFAIK, When CREATE INDEX on a partition and INSERT to a parent table are >> executed at the same time, this patch causes deadlock. >> >> * partitions information >> >> Partition key: RANGE (a) >>

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-11-08 Thread Amit Langote
On 2018/11/08 20:28, David Rowley wrote: > On 8 November 2018 at 20:15, Amit Langote > wrote: >> Actually, as I also proposed upthread, we should move root_tuple_slot from >> PartitionTupleRouting to ModifyTableState as mt_root_tuple_slot, because >> it's part of the fi

Re: BUG #15212: Default values in partition tables don't work as expected and allow NOT NULL violation

2018-11-08 Thread Amit Langote
On Fri, Nov 9, 2018 at 1:03 AM Alvaro Herrera wrote: > On 2018-Nov-07, Amit Langote wrote: > > > I think the result in this case should be an error, just as it would in > > the regular inheritance case. > > > > create table parent (a text); > > create table chi

Re: Getting ERROR: could not open file "base/13164/t3_16388" with partition table with ON COMMIT

2018-11-04 Thread Amit Langote
On 2018/11/05 9:19, Michael Paquier wrote: > On Fri, Nov 02, 2018 at 04:39:07PM +0900, Amit Langote wrote: >> Agreed that they're two independent issues, although it wouldn't be such a >> bad idea to fix them in one go, as they're both issues related to the >> handling

Re: Hooks to Modify Execution Flow and Query Planner

2018-11-04 Thread Amit Langote
Hi, On 2018/11/03 17:28, Vincent Mirian wrote: > Hi Amit, > > Thank you for your response. Chapters 51, 57 and 59 (Overview of PostgreSQL > Internals, Writing A Foreign Data Wrapper and Writing A Custom Scan > Provider) seem to be relevant. Aside from the source code snippets in the > document,

ON COMMIT actions and inheritance

2018-11-04 Thread Amit Langote
Hi, Michael pointed out a problem with specifying different ON COMMIT actions on a temporary inheritance parent and its children: https://www.postgresql.org/message-id/20181102051804.GV1727%40paquier.xyz The problem is that when PreCommit_on_commit_actions() executes an ON COMMIT DROP action on

Re: executor relation handling

2018-10-04 Thread Amit Langote
On 2018/10/05 5:59, Tom Lane wrote: > Amit Langote writes: >> I've rebased the remaining patches. I broke down one of the patches into >> 2 and re-ordered the patches as follows: > >> 0001: introduces a function that opens range table relations and maintains >>

Re: partition tree inspection functions

2018-10-05 Thread Amit Langote
On 2018/10/05 14:56, Michael Paquier wrote: > On Thu, Oct 04, 2018 at 05:18:07PM +0900, Michael Paquier wrote: >> So it seems that I am clearly outvoted here ;) >> >> Okay, let's do as you folks propose. > > And attached is a newer version with this isleaf stuff and the previous > feedback from

Re: pg_upgrade failed with ERROR: null relpartbound for relation 18159 error.

2018-10-05 Thread Amit Langote
On 2018/10/05 16:05, Michael Paquier wrote: >> As of commit 2fbdf1b38bc [1], which has been applied in 11 and HEAD >> branches, RelationBuildPartitionDesc emits an error if we don't find >> relpartbound set for a child found by scanning pg_inherits, instead of >> skipping such children. While

Re: pg_upgrade failed with ERROR: null relpartbound for relation 18159 error.

2018-10-05 Thread Amit Langote
Thanks for the report. On 2018/10/04 3:58, Rajkumar Raghuwanshi wrote: > Hi, > > I am getting ERROR: null relpartbound for relation 18159 while doing > pg_upgrade from v11 to v11/master. > > -- user-defined operator class in partition key > CREATE FUNCTION my_int4_sort(int4,int4) RETURNS int

Re: Why we allow CHECK constraint contradiction?

2018-10-10 Thread Amit Langote
On 2018/10/10 14:25, Imai, Yoshikazu wrote: > Hi, all. > > I have a wonder about the behaviour of creating table which has a constraint > contradiction. > > I created below table. > > bugtest=# create table ct (a int, CHECK(a is not null and a >= 0 and a < 100 > and a >= 200 and a < 300)); >

Re: partition tree inspection functions

2018-10-09 Thread Amit Langote
On 2018/10/09 20:17, Michael Paquier wrote: > On Tue, Oct 09, 2018 at 07:20:40PM +0900, Amit Langote wrote: >> Sorry if I'm misunderstanding something, but why would we need a new >> clone? If we don't change pg_partition_tree() to only accept tables >> (regular/parti

Re: partition tree inspection functions

2018-10-09 Thread Amit Langote
On 2018/10/06 15:26, Michael Paquier wrote: > On Fri, Oct 05, 2018 at 08:22:49AM -0400, Jesper Pedersen wrote: >> Looks good. > > Actually, after sleeping on it, there could be potentially two problems: > 1) We don't check the relkind of the relation. For example it is > possible to get a tree

Re: pg_upgrade failed with ERROR: null relpartbound for relation 18159 error.

2018-10-09 Thread Amit Langote
On 2018/10/08 0:09, Michael Paquier wrote: > On Sun, Oct 07, 2018 at 11:13:19AM -0300, Alvaro Herrera wrote: >> Good point. I cannot do that today, but if you want to, please go >> ahead. > > Okay, done. Thank you both. Regards, Amit

Re: executor relation handling

2018-10-09 Thread Amit Langote
On 2018/10/09 0:38, Tom Lane wrote: > I wrote: >> Keeping that comparison in mind, I'm inclined to think that 0001 >> is the best thing to do for now. The incremental win from 0002 >> is not big enough to justify the API break it creates, while your >> 0005 is not really attacking the problem the

Re: executor relation handling

2018-10-09 Thread Amit Langote
On 2018/10/08 9:29, David Rowley wrote: > On 8 October 2018 at 13:13, Tom Lane wrote: >> The idea I had in mind was to allow hard pruning of any leaf that's >> been excluded *at plan time* based on partition constraints seen in >> its parent rel(s). That should be safe enough as long as we take

Re: executor relation handling

2018-10-09 Thread Amit Langote
On 2018/10/08 8:18, Tom Lane wrote: > I wrote: >> Still need to think a bit more about whether we want 0005 in >> anything like its current form. > > So I poked at that for a bit, and soon realized that the *main* problem > there is that ExecFindRowMark() eats O(N^2) time due to repeated searches

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-10-08 Thread Amit Langote
Hi David, On 2018/10/05 21:55, David Rowley wrote: > On 17 September 2018 at 21:15, David Rowley > wrote: >> v9 patch attached. Fixes conflict with 6b78231d. > > v10 patch attached. Fixes conflict with cc2905e9. Thanks for rebasing. > I'm not so sure we need to zero the

Re: executor relation handling

2018-10-09 Thread Amit Langote
On 2018/10/07 3:59, Tom Lane wrote: > Amit Langote writes: >> On 2018/10/05 5:59, Tom Lane wrote: >>> So I'm inclined to just omit 0003. AFAICS this would only mean that >>> we couldn't drop the global PlanRowMarks list from PlannedStmt, which >>> does not b

Re: executor relation handling

2018-10-09 Thread Amit Langote
On 2018/10/08 3:55, Tom Lane wrote: > I didn't like the idea of unifying ModifyTable.nominalRelation with > the partition root info. Those fields serve different masters --- > nominalRelation, at least in its original intent, is only meant for > use of EXPLAIN and might have nothing to do with

Re: partition tree inspection functions

2018-10-09 Thread Amit Langote
On 2018/10/09 19:05, Michael Paquier wrote: > On Tue, Oct 09, 2018 at 06:41:59PM +0900, Amit Langote wrote: >> Partitioned indexes, just like partitioned tables, don't have their own >> storage, so pg_relation_size() cannot be used to obtain their size. We >> decided that t

Re: partition tree inspection functions

2018-10-09 Thread Amit Langote
On 2018/10/09 18:10, Michael Paquier wrote: > On Tue, Oct 09, 2018 at 05:11:59PM +0900, Amit Langote wrote: >> Hmm, how would one find the size of a partitioned index tree if we don't >> allow indexes to be passed? > > pg_total_relation_size() and pg_indexes_size() are design

Re: executor relation handling

2018-10-09 Thread Amit Langote
On Tue, Oct 9, 2018 at 11:07 PM Tom Lane wrote: > > Amit Langote writes: > > On 2018/10/08 3:55, Tom Lane wrote: > >> I didn't like the idea of unifying ModifyTable.nominalRelation with > >> the partition root info. Those fields serve different masters --- > &g

Re: Why we allow CHECK constraint contradiction?

2018-10-10 Thread Amit Langote
On 2018/10/10 16:28, Imai, Yoshikazu wrote: > On Tue, Oct 9, 2018 at 6:01 PM, Amit Langote wrote: >> I had wondered about it when developing the partitioning feature about >> a couple of years ago and this is the response I'd gotten: >> >> https://www.p

Re: partition tree inspection functions

2018-10-01 Thread Amit Langote
On 2018/10/01 15:27, Michael Paquier wrote: > On Mon, Oct 01, 2018 at 03:16:32PM +0900, Amit Langote wrote: >> I wasn't able to respond to some of issues that Jesper brought up with the >> approach taken by the latest patch whereby there is no separate >> pg_partition_lev

Re: Relations being opened without any lock whatever

2018-10-01 Thread Amit Langote
On 2018/10/01 4:20, Tom Lane wrote: > Running the regression tests with the patch I showed in > https://www.postgresql.org/message-id/16565.1538327...@sss.pgh.pa.us > exposes two places where HEAD is opening relations without having > any lock at all on them: Maybe you've noticed but the

Re: executor relation handling

2018-10-01 Thread Amit Langote
On 2018/09/30 5:04, Tom Lane wrote: > David Rowley writes: >> I've attached v10 which fixes this and aligns the WARNING text in >> ExecInitRangeTable() and addRangeTableEntryForRelation(). > > I started poking at this. Thanks a lot for looking at this. > I thought that it would be a good

Re: partition tree inspection functions

2018-10-01 Thread Amit Langote
Hi, On 2018/10/01 15:03, Michael Paquier wrote: > On Thu, Aug 09, 2018 at 01:05:56PM +0900, Amit Langote wrote: >> Attached updated patch. > > So, except if I am missing something, what we have here is a patch which > has been debatted quite a bit and has semantics which

Re: speeding up planning with partitions

2018-10-02 Thread Amit Langote
On 2018/10/03 8:31, Thomas Munro wrote: > On Fri, Sep 14, 2018 at 10:29 PM Amit Langote > wrote: >> Attached is what I have at the moment. > > I realise this is a WIP but FYI the docs don't build (you removed a > element that is still needed, when removing a parag

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-27 Thread Amit Langote
Hi, On 2018/09/28 10:23, David Rowley wrote: > On Thu, 20 Sep 2018 at 11:31, Andres Freund wrote: >> >> On 2018-09-19 12:06:47 +0900, Michael Paquier wrote: >>> On Wed, Sep 19, 2018 at 01:14:10PM +1200, David Rowley wrote: Wouldn't it be better to modify copy.c to just perform the heap_sync

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-27 Thread Amit Langote
On 2018/09/28 12:12, Michael Paquier wrote: > On Fri, Sep 28, 2018 at 02:46:30PM +1200, David Rowley wrote: >> I don't agree that we can skip explaining why one of the optimisations >> can't be applied just because we've explained why a similar >> optimisation cannot be applied somewhere close by.

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-09-27 Thread Amit Langote
On 2018/09/28 11:46, David Rowley wrote: > On 28 September 2018 at 14:25, Amit Langote > wrote: >> Looking at the patch itself, does it seem like both the newly added >> comments repeat the same point (that we'll need per-partition hi_options >> to enable these optimizatio

Re: Slotification of partition tuple conversion

2018-09-28 Thread Amit Langote
On 2018/09/28 19:06, Amit Khandekar wrote: > On Wed, 26 Sep 2018 at 03:33, Andres Freund wrote: >> >> Hi Amit, >> >> Could you rebase this patch, it doesn't apply anymore. > > Thanks for informing. Attached are both mine and Amit Langote's patch > rebased and attached ... Thanks Amit for also

Re: Calculate total_table_pages after set_base_rel_sizes()

2018-10-10 Thread Amit Langote
umber is > subtlely broken as it counts all partitions, even ones that will later > be pruned, thus decreasing the likelihood of an index being used as > random_page_cost would be applied to a higher number of index pages. > > Amit Langote has since posted a patch to delay the Range

Re: speeding up planning with partitions

2018-10-01 Thread Amit Langote
On 2018/10/02 10:20, Michael Paquier wrote: > On Sat, Sep 29, 2018 at 07:00:02PM +0530, Dilip Kumar wrote: >> I think we can delay allocating memory for rel->part_rels? And we can >> allocate in add_rel_partitions_to_query only >> for those partitions which survive pruning. > > This last review

Re: executor relation handling

2018-10-01 Thread Amit Langote
On 2018/10/01 2:18, Tom Lane wrote: > I wrote: >> 1. You set up transformRuleStmt to insert AccessExclusiveLock into >> the "OLD" and "NEW" RTEs for a view. This is surely wrong; we do >> not want to take exclusive lock on a view just to run a query using >> the view. It should (usually, anyway)

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-10-02 Thread Amit Langote
On 2018/08/31 21:40, Etsuro Fujita wrote: > (2018/08/31 21:30), Jonathan S. Katz wrote: >> Thank you for taking care of that and for committing the patch. I have >> now closed this issues on the open items list. > > Thanks! I noticed that the CF entry for this was not closed. As of this

Re: Tuple conversion naming

2018-10-02 Thread Amit Langote
Hi, I agree that some clean up might be in order, but want to clarify a few points. On 2018/10/02 15:11, Andres Freund wrote: > Hi, > > The naming around partition related tuple conversions is imo worthy of > improvement. Note that tuple conversion functionality in tupconvert.c has existed

Re: Tuple conversion naming

2018-10-02 Thread Amit Langote
On 2018/10/02 16:40, Andres Freund wrote: >>> For executing them we have: >>> - do_convert_tuple >>> - ConvertPartitionTupleSlot >>> >>> which is two randomly differing spellings of related functionality, >>> without the name indicating that they, for reasons, don't both use >>>

Re: Slotification of partition tuple conversion

2018-10-02 Thread Amit Langote
Hi, I looked at the patch. Some comments. On 2018/10/02 16:35, Andres Freund wrote: > I wasn't quite happy yet with that patch. > > - ConvertTupleSlot seems like a too generic name, it's very unclear it's > related to tuple mapping, rather than something internal to slots. I > went for

Re: speeding up planning with partitions

2018-10-04 Thread Amit Langote
Imai-san, On 2018/10/04 17:11, Imai, Yoshikazu wrote: > Hi, Amit! > > On Thu, Sept 13, 2018 at 10:29 PM, Amit Langote wrote: >> Attached is what I have at the moment. > > I also do the code review of the patch. Thanks a lot for your review. I'm working on updating the

Re: partition tree inspection functions

2018-10-04 Thread Amit Langote
On 2018/10/03 12:37, Michael Paquier wrote: > On Mon, Oct 01, 2018 at 04:27:57PM +0900, Amit Langote wrote: >> Yeah, maybe there is no reason to delay proceeding with >> pg_partition_children which provides a useful functionality. > > So, I have been looking at your patch, a

Re: partition tree inspection functions

2018-10-04 Thread Amit Langote
On 2018/10/04 9:27, Michael Paquier wrote: > On Wed, Oct 03, 2018 at 08:12:59AM -0400, Jesper Pedersen wrote: >> Removing isleaf would require extra round trips to the server to get >> that information. So, I think we should keep it. > > I don't really get your point about extra round trips with

Re: executor relation handling

2018-10-04 Thread Amit Langote
On 2018/10/04 5:16, Tom Lane wrote: > I wrote: >> Amit Langote writes: >>> Should this check that we're not in a parallel worker process? > >> Hmm. I've not seen any failures in the parallel parts of the regular >> regression tests, but maybe I'd better do a

Re: Postgres 11 release notes

2018-09-29 Thread Amit Langote
On Fri, Sep 28, 2018 at 7:24 PM Bruce Momjian wrote: > On Fri, Sep 28, 2018 at 10:21:16AM +0900, Amit Langote wrote: > > On 2018/09/27 23:24, Alvaro Herrera wrote: > > > On 2018-Sep-27, Amit Langote wrote: > > > > > >> Sorry I couldn't reply sooner,

Re: speeding up planning with partitions

2018-09-02 Thread Amit Langote
Thank you Kato-san for testing. On 2018/08/31 19:48, Kato, Sho wrote: > Hi, Amit > > Great! > With the total number of records being 6400, I benchmarked while increasing > the number of partitions from 100 to 6400. > Applying three all patches, 20% performance improved for 100 partitions. > >

Re: pointless check in RelationBuildPartitionDesc

2018-09-03 Thread Amit Langote
On 2018/09/04 6:39, Alvaro Herrera wrote: > I noticed this strange hack in RelationBuildPartitionDesc: > >/* > * It is possible that the pg_class tuple of a partition has not been > * updated yet to set its relpartbound field. The only case where > * this happens

Re: pointless check in RelationBuildPartitionDesc

2018-09-03 Thread Amit Langote
On 2018/09/04 10:19, Michael Paquier wrote: > On Tue, Sep 04, 2018 at 09:47:07AM +0900, Amit Langote wrote: >> On 2018/09/04 6:39, Alvaro Herrera wrote: >>> After looking, it seems that this is just self-inflicted pain: for some >>> reason, we store the pg_i

Re: Slotification of partition tuple conversion

2018-09-03 Thread Amit Langote
Hi Amit, Thanks for the updated patch and sorry I couldn't reply sooner. On 2018/08/21 16:18, Amit Khandekar wrote: > On 21 August 2018 at 08:12, Amit Langote > wrote: >> Here are some comments on the patch: > > Thanks for the review. > >> >> +ConvertTup

Re: speeding up planning with partitions

2018-09-04 Thread Amit Langote
Hi Dilip, Thanks for taking a look. On 2018/09/03 20:57, Dilip Kumar wrote: > The idea looks interesting while going through the patch I observed > this comment. > > /* > * inheritance_planner > * Generate Paths in the case where the result relation is an > * inheritance set. > * > *

Re: Make executor's Range Table an array instead of a List

2018-09-04 Thread Amit Langote
On 2018/08/24 7:22, David Rowley wrote: > On 24 August 2018 at 02:26, Amit Langote wrote: >> One of the patches I sent last week does the same thing, among a >> couple of other things with regard to handling relations in the >> executor. On a cursory look at the patch,

Re: Make executor's Range Table an array instead of a List

2018-09-04 Thread Amit Langote
On 2018/09/04 17:14, David Rowley wrote: > On 4 September 2018 at 19:31, Amit Langote > wrote: >> On 2018/08/24 7:22, David Rowley wrote: >>> On 24 August 2018 at 02:26, Amit Langote wrote: >>>> One of the patches I sent last week does the same thing, am

Re: pointless check in RelationBuildPartitionDesc

2018-09-04 Thread Amit Langote
On 2018/09/04 13:08, Alvaro Herrera wrote: > On 2018-Sep-04, Amit Langote wrote: > >> On 2018/09/04 10:19, Michael Paquier wrote: >>> On Tue, Sep 04, 2018 at 09:47:07AM +0900, Amit Langote wrote: >>>> On 2018/09/04 6:39, Alvaro Herrera wrote: >>>>>

Re: executor relation handling

2018-09-04 Thread Amit Langote
On 2018/08/16 17:22, Amit Langote wrote: > 0004-Revise-executor-range-table-relation-opening-closing.patch > > This adds two arrays to EState indexed by RT indexes, one for > RangeTblEntry's and another for Relation pointers. The former reduces the > cost of fetching RangeTblEn

Re: Postgres 11 release notes

2018-09-27 Thread Amit Langote
Hi, On 2018/09/17 1:07, Jonathan S. Katz wrote: > On 9/16/18 11:36 AM, Bruce Momjian wrote: >> On Sat, Sep 15, 2018 at 11:06:04AM -0400, Jonathan Katz wrote: >>> @@ -2414,12 +2408,8 @@ same commits as above >>> The option --create-slot creates >>> the named replication slot

Re: executor relation handling

2018-09-27 Thread Amit Langote
On 2018/09/27 18:15, David Rowley wrote: > I've just completed a review of the v5 patch set. I ended up just > making the changes myself since Amit mentioned he was on leave for a > few weeks. Thanks David. I'm back today and will look at the updated patches tomorrow. Regards, Amit

Re: Delay locking partitions during query execution

2019-01-16 Thread Amit Langote
On 2019/01/04 9:53, David Rowley wrote: > Without PREPAREd statements, if the planner itself was unable to prune > the partitions it would already have obtained the lock during > planning, so AcquireExecutorLocks(), in this case, would bump into the > local lock hash table entry and forego trying

Re: speeding up planning with partitions

2019-01-17 Thread Amit Langote
Thank you Imai-san for testing. Sorry it totally slipped my mind to reply to this email. On 2019/01/09 11:08, Imai, Yoshikazu wrote: > I wonder why force_custom_plan is faster than auto after applied the patch. > > When we use PREPARE-EXECUTE, a generic plan is created and used if its cost is >

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2019-01-19 Thread Amit Langote
On Fri, Jan 18, 2019 at 9:58 PM Etsuro Fujita wrote: > I updated the patch as such and rebased it to the latest HEAD. I also > added the commit message. Attached is an updated patch. Does that make > sense? If there are no objections, I'll push that patch early next week. Thank you. Looks

Re: problems with foreign keys on partitioned tables

2019-01-19 Thread Amit Langote
On Sat, Jan 19, 2019 at 7:16 AM Alvaro Herrera wrote: > Thanks, this is better. There were a few other things I didn't like, so > I updated it. Mostly, two things: > > 1. I didn't like a seqscan on pg_trigger, so I turned that into an > indexed scan on the constraint OID, and then the other two

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2019-01-14 Thread Amit Langote
Fujita-san, On 2019/01/11 20:04, Etsuro Fujita wrote: > (2019/01/11 13:46), Amit Langote wrote: >> On 2019/01/10 15:07, Etsuro Fujita wrote: >>> (The name of the flag isn't >>> good?  If so, that would be my fault because I named that flag.) >> >>

Re: [Sender Address Forgery]Re: error message when subscription target is a partitioned table

2019-01-14 Thread Amit Langote
On 2019/01/13 16:56, Michael Paquier wrote: > On Fri, Jan 11, 2019 at 04:04:41PM +0900, Michael Paquier wrote: >> Thanks. I can commit this version if there are no objections then. > > And done. Thank you! Regards, Amit

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2019-01-14 Thread Amit Langote
Fujita-san, On 2019/01/11 21:50, Etsuro Fujita wrote: >>> (2019/01/10 10:41), Amit Langote wrote: >>>> That's a loaded meaning and abusing it to mean something else can be >>>> challenged, but we can live with that if properly documented.  >>>> Sp

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2019-01-14 Thread Amit Langote
On 2019/01/15 10:46, Amit Langote wrote: > On 2019/01/11 20:04, Etsuro Fujita wrote: >> One thing I intended in that commit was to set the flag to false for >> partitioned tables contained in inheritance trees where the top parent is >> a UNION ALL subquery, because we don't c

Re: using expression syntax for partition bounds

2019-01-15 Thread Amit Langote
Thanks for the review. On 2019/01/15 22:24, Peter Eisentraut wrote: > On 15/01/2019 07:31, Amit Langote wrote: >>> Is "partition bound" the right term? For list partitioning, it's not >>> really a bound. Maybe it's OK. >> >> Hmm, maybe &q

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2019-01-21 Thread Amit Langote
On 2019/01/21 17:17, Etsuro Fujita wrote: > (2019/01/19 21:17), Amit Langote wrote: >> On Fri, Jan 18, 2019 at 9:58 PM Etsuro Fujita >>   wrote: >>> I updated the patch as such and rebased it to the latest HEAD.  I also >>> added the commit message.  Attached is

Re: partitioned tables referenced by FKs

2019-01-21 Thread Amit Langote
Hi Alvaro, On 2018/12/01 4:12, Alvaro Herrera wrote: > Here's a more credible version of this patch series. Are you going to post rebased patches soon? Thanks, Amit

Typo: llvm*.cpp files identified as llvm*.c

2019-01-21 Thread Amit Langote
Hi, Attached find a patch to fix $subject. Thanks, Amit diff --git a/src/backend/jit/llvm/llvmjit_error.cpp b/src/backend/jit/llvm/llvmjit_error.cpp index ba3907c452..9c6e8026e7 100644 --- a/src/backend/jit/llvm/llvmjit_error.cpp +++ b/src/backend/jit/llvm/llvmjit_error.cpp @@ -9,7 +9,7 @@ *

Re: problems with foreign keys on partitioned tables

2019-01-21 Thread Amit Langote
On 2019/01/22 8:30, Alvaro Herrera wrote: > Hi Amit, > > Will you please rebase 0002? Please add your proposed tests cases to > it, too. Done. See the attached patches for HEAD and PG 11. Thanks, Amit From 432c4551990d0da1c77b6b9523296b0a2a0a5119 Mon Sep 17 00:00:00 2001 From: amit Date:

Re: speeding up planning with partitions

2019-01-20 Thread Amit Langote
Imai-san, On 2019/01/21 9:45, Imai, Yoshikazu wrote: > I'll check if this fact is really correct by majoring the time of the > first 5 queries before generic plan is created and the other queries > after generic plan is created. That would really help. If you are able to recreate that behavior

Re: Typo: llvm*.cpp files identified as llvm*.c

2019-01-22 Thread Amit Langote
On 2019/01/23 4:51, Andres Freund wrote: > Hi, > > On 2019-01-22 13:43:32 +0900, Amit Langote wrote: >> Attached find a patch to fix $subject. > > Thanks, pushed to master and 11. Thank you. Regards, Amit

Re: using expression syntax for partition bounds

2019-01-14 Thread Amit Langote
Thanks for the review and sorry it took me a while to reply. On 2019/01/02 22:58, Peter Eisentraut wrote: > On 26/11/2018 05:58, Amit Langote wrote: >> On 2018/11/09 14:38, Amit Langote wrote: >>> Rebased due to change in addRangeTableEntryForRelation's API. >> >>

Re: problems with foreign keys on partitioned tables

2019-01-17 Thread Amit Langote
On 2019/01/18 7:54, Alvaro Herrera wrote: > On 2019-Jan-09, Amit Langote wrote: > >> 1. Foreign keys of partitions stop working correctly after being detached >> from the parent table > >> This happens because the action triggers defined on the PK relation (pk) >&g

Re: speeding up planning with partitions

2019-01-17 Thread Amit Langote
Tsunakawa-san, On 2019/01/18 14:12, Tsunakawa, Takayuki wrote: > From: Amit Langote [mailto:langote_amit...@lab.ntt.co.jp] >> Are you saying that, when using auto mode, all executions of the query >> starting from 7th are slower than the first 5 executions? That is, the >>

Re: using expression syntax for partition bounds

2019-01-18 Thread Amit Langote
Amit >From f68a0c72c68dd096182be751b238621b3a8c6741 Mon Sep 17 00:00:00 2001 From: Kyotaro Horiguchi Date: Fri, 6 Jul 2018 14:05:22 +0900 Subject: [PATCH v9] Allow generalized expression syntax for partition bounds Authors: Kyotaro Horiguchi, Tom Lane, Amit Langote --- doc/src/sgml/ref/alter_ta

inherited primary key misbehavior

2019-01-23 Thread Amit Langote
Hi, It seems that ATExecDetachPartition misses to mark a child's primary key constraint entry (if any) as local (conislocal=true, coninhcount=0), which causes this: create table p (a int primary key) partition by list (a); create table p2 partition of p for values in (2); alter table p detach

Re: Typo: llvm*.cpp files identified as llvm*.c

2019-01-22 Thread Amit Langote
Hi Thomas, On 2019/01/23 9:37, Thomas Munro wrote: > On Wed, Jan 23, 2019 at 1:16 PM Amit Langote > wrote: >> On 2019/01/23 4:51, Andres Freund wrote: >>> On 2019-01-22 13:43:32 +0900, Amit Langote wrote: >>>> Attached find a patch to fix $subject. >&g

Re: speeding up planning with partitions

2019-01-22 Thread Amit Langote
On 2019/01/22 18:47, David Rowley wrote: > On Tue, 22 Jan 2019 at 20:01, Imai, Yoshikazu >> What I understand so far is about 10,000 while loops at total >> (4098+4098+some extra) is needed in hash_seq_search() in EXECUTE query after >> the creation of the generic plan. >> 10,000 while loops

Re: pg_partition_tree crashes for a non-defined relation

2018-12-12 Thread Amit Langote
On 2018/12/12 9:52, Michael Paquier wrote: > On Mon, Dec 10, 2018 at 10:52:37PM +0900, Michael Paquier wrote: >> OK. Sure, let's do as you suggest then. I'll wait a couple of days >> before committing the patch so as if someone has extra comments they >> have the time to post. So please feel

Re: Fixing typos in tests of partition_info.sql

2018-12-17 Thread Amit Langote
On 2018/12/18 10:57, Michael Paquier wrote: > On Mon, Dec 17, 2018 at 07:49:42PM +0900, Michael Paquier wrote: >> Okay, this suggestion sounds fine to me. Thanks! > > And committed with all your suggestions included. Thanks for the > discussion. Thank you! Regards, Amit

Re: ToDo: show size of partitioned table

2018-12-17 Thread Amit Langote
Hi, Thank you for updating the patch. On 2018/12/17 17:48, Pavel Stehule wrote: > new update of this patch Documentation portion of this patch still contains some typos that I mentioned before here: https://www.postgresql.org/message-id/1c83bb5c-47cd-d796-226c-e95795b05551%40lab.ntt.co.jp +

Re: don't create storage when unnecessary

2018-12-17 Thread Amit Langote
Hi, On 2018/12/18 14:56, Kyotaro HORIGUCHI wrote: > Hello. > > At Sun, 16 Dec 2018 17:47:16 -0300, Alvaro Herrera > wrote in <20181216204716.jddzijk3fb2dpdk4@alvherre.pgsql> >> On 2018-Dec-07, Michael Paquier wrote: >> >>> A macro makes sense to control that. >> >> I added Ashutosh's

Re: lock level for DETACH PARTITION looks sketchy

2018-12-20 Thread Amit Langote
On 2018/12/21 6:07, Alvaro Herrera wrote: > On 2018-Dec-20, Robert Haas wrote: > >> On Thu, Dec 20, 2018 at 3:58 PM Alvaro Herrera >> wrote: >>> I think what prompted the lock to be AccessShareLock for the child rel >>> in the first place is the fact that ATExecDropInherit() (ALTER TABLE NO >>>

Re: Speeding up creating UPDATE/DELETE generic plan for partitioned table into a lot

2018-12-21 Thread Amit Langote
Kato-san, On 2018/12/21 15:36, Kato, Sho wrote: > Hi, > I want to speed up the creation of UPDATE/DELETE generic plan for tables > partitioned into a lot. > > Currently, creating a generic plan of UPDATE/DELTE for such table, planner > creates a plan to scan all partitions. > So it takes a

Re: speeding up planning with partitions

2018-12-25 Thread Amit Langote
Thank you Imai-san. On 2018/12/25 16:47, Imai, Yoshikazu wrote: > Here's the continuation of the review. Almost all of below comments are > little fixes. > > --- > 0001: line 76-77 > In commit message: > exclusion for target child relation, which is no longer > is no longer needed.

Re: Speeding up creating UPDATE/DELETE generic plan for partitioned table into a lot

2018-12-27 Thread Amit Langote
Kato-san, On 2018/12/27 16:53, Kato, Sho wrote: >> What do you mean by "since the partitions to access are partial"? > > I mean planner create scan nodes based on the parameters specified for > EXECUTE and backend keep them in CachedPlan. > > Before: > > postgres=# explain execute

Re: PostgreSQL partition tables use more private memory

2018-12-27 Thread Amit Langote
Hi, On 2018/12/27 15:44, 大松 wrote: > # PostgreSQL partition tables use more private memory > > Hi, there is a process private memory issue about partition tables in our > production environment. We're not sure if it's a bug or Pg just works in this > way. > > - when dml operated on partition

Re: pg_dump emits ALTER TABLE ONLY partitioned_table

2018-12-10 Thread Amit Langote
On Tue, Dec 11, 2018 at 3:20 AM Stephen Frost wrote: > * Stephen Frost (sfr...@snowman.net) wrote: > > * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > > > Do you intend to push the other patch to add regression tests for the > > > non-inherited constraints?

Re: Fixing typos in tests of partition_info.sql

2018-12-16 Thread Amit Langote
Hi, On 2018/12/17 15:40, Michael Paquier wrote: > Hi Amit, > (CC: -hackers) > > I was just going through some of the tests, when I noticed that the > tests of partition_info.sql have two typos and that the last set of > tests is imprecise about the expected behavior of the functions. > > Do you

Re: Fixing typos in tests of partition_info.sql

2018-12-17 Thread Amit Langote
On 2018/12/17 17:25, Michael Paquier wrote: > On Mon, Dec 17, 2018 at 04:41:01PM +0900, Amit Langote wrote: >> Okay, let's use "Functions" then, although I wonder if you shouldn't tweak >> it later when you commit the pg_partition_root patch? > > There are already

Re: Fixing typos in tests of partition_info.sql

2018-12-16 Thread Amit Langote
On 2018/12/17 16:38, Michael Paquier wrote: > On Mon, Dec 17, 2018 at 04:14:07PM +0900, Amit Langote wrote: >> --- A table not part of a partition tree works is the only member listed. >> +-- A table not part of a partition tree is the only member listed. >> >>

<    2   3   4   5   6   7   8   9   10   11   >