Re: pg_partition_tree crashes for a non-defined relation

2019-02-27 Thread Amit Langote
Hi, On 2019/02/28 10:45, Michael Paquier wrote: > On Wed, Feb 27, 2019 at 03:48:08PM -0300, Alvaro Herrera wrote: >> I just happened to come across the result of this rationale in >> pg_partition_tree() (an SRF) while developing a new related function, >> pg_partition_ancestors(), and find the res

Re: partitioned tables referenced by FKs

2019-02-27 Thread Amit Langote
Hi Alvaro, I looked at the latest patch and most of the issues/bugs that I was going to report based on the late January version of the patch seem to have been taken care of; sorry that I couldn't post sooner which would've saved you some time. The patch needs to be rebased on top of ff11e7f4b9

Re: BUG #15623: Inconsistent use of default for updatable view

2019-02-27 Thread Amit Langote
On 2019/02/27 18:37, Dean Rasheed wrote: > On Tue, 12 Feb 2019 at 10:33, Dean Rasheed wrote: >> Here's an updated patch ... > > So I pushed that. However, ... > > Playing around with it some more, I realised that whilst this appeared > to fix the reported problem, it exposes another issue which

Re: pg_partition_tree crashes for a non-defined relation

2019-02-28 Thread Amit Langote
Hi, On 2019/03/01 9:22, Michael Paquier wrote: > On Thu, Feb 28, 2019 at 04:32:03PM -0300, Alvaro Herrera wrote: >> Yeah, looks good, please push. > > Done for this part. > >> I would opt for returning the empty set for legacy inheritance too. >> >> More generally, I think we should return empty

Re: speeding up planning with partitions

2019-03-01 Thread Amit Langote
Imai-san, Thanks for testing and sorry it took me a while to reply. On 2019/02/25 15:24, Imai, Yoshikazu wrote: > [update_pt_with_joining_another_pt.sql] > update rt set c = jrt.c + 100 from jrt where rt.b = jrt.b; > > [pgbench] > pgbench -n -f update_pt_with_joining_another_pt_for_ptkey.sql -T

Re: speeding up planning with partitions

2019-03-01 Thread Amit Langote
On 2019/03/01 22:01, Amit Langote wrote: > Of course, I had to make sure that query_planner (which is not > in the charge of adding source inheritance child objects) can notice that. Oops, I meant to write "query_planner (which *is* in the charge of adding source inheritance ch

Re: pg_partition_tree crashes for a non-defined relation

2019-03-03 Thread Amit Langote
On 2019/03/02 18:21, Michael Paquier wrote: > On Fri, Mar 01, 2019 at 11:38:20AM -0500, Tom Lane wrote: >> Right, while you'd get zero rows out for a non-partitioned table. >> WFM. > > Exactly. I have committed a patch doing exactly that, and I have > added test cases with a partitioned table and

Re: speeding up planning with partitions

2019-03-04 Thread Amit Langote
Hi, On 2019/03/01 22:01, Amit Langote wrote: > Please find attached updated patches. Will update source code comments, > commit message and perform other fine-tuning over the weekend if possible. I realized when "fine-tuning" that the patch 0001 contained too many changes tha

Re: speeding up planning with partitions

2019-03-04 Thread Amit Langote
Imai-san, Thanks for the review. On 2019/03/04 18:14, Imai, Yoshikazu wrote: > I've taken at glance the codes and there are some minor comments about the > patch. > > * You changed a usage/arguments of get_relation_info, but why you did it? I > made those codes back to the original code and ch

Re: speeding up planning with partitions

2019-03-04 Thread Amit Langote
Hi Jesper, Thanks for the review. I've made all of the changes per your comments in my local repository. I'll post the updated patches after diagnosing what I'm suspecting a memory over-allocation bug in one of the patches. If you configure build with --enable-cassert, you'll see that throughpu

Re: psql display of foreign keys

2019-03-04 Thread Amit Langote
On 2019/03/05 4:41, Alvaro Herrera wrote: > Here's the patch I'm really interested about :-) Thanks for the updated patch. I applied it and rebased the foreign-keys-referencing-partitioned-tables patch on top. Here's something I think you may have missed: -- partitioned primary key table create

Re: Inheriting table AMs for partitioned tables

2019-03-04 Thread Amit Langote
On 2019/03/05 8:47, Andres Freund wrote: > Hi, > > In the pluggable storage patch [1], one thing that I'm wondering about > is how exactly to inherit the storage AM across partitions. I think > that's potentially worthy of a discussion with a wider audience than I'd > get in that thread. It seems

Re: Inheriting table AMs for partitioned tables

2019-03-04 Thread Amit Langote
On 2019/03/05 11:59, Amit Langote wrote: > On 2019/03/05 8:47, Andres Freund wrote: >> Hi, >> >> In the pluggable storage patch [1], one thing that I'm wondering about >> is how exactly to inherit the storage AM across partitions. I think >> that's pote

Re: speeding up planning with partitions

2019-03-04 Thread Amit Langote
Imai-san, Thanks for checking. On 2019/03/05 15:03, Imai, Yoshikazu wrote: > I've also done code review of 0001 and 0002 patch so far. > > [0001] > 1. Do we need to open/close a relation in add_appendrel_other_rels()? > > + if (rel->part_scheme) > { > - ListCell *l; >

Re: speeding up planning with partitions

2019-03-05 Thread Amit Langote
On 2019/03/05 9:50, Amit Langote wrote: > I'll post the updated patches after diagnosing what > I'm suspecting a memory over-allocation bug in one of the patches. If you > configure build with --enable-cassert, you'll see that throughput > decreases as number of partiti

Re: speeding up planning with partitions

2019-03-05 Thread Amit Langote
On 2019/03/04 19:38, Amit Langote wrote: > 2. Defer inheritance expansion to add_other_rels_to_query(). Although the > purpose of doing this is to perform partition pruning before adding the > children, this patch doesn't change when the pruning occurs. It deals > with other is

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-05 Thread Amit Langote
Hi, (cc'ing -hackers and Peter E) On Tue, Mar 5, 2019 at 8:02 PM PG Bug reporting form wrote: > > The following bug has been logged on the website: > > Bug reference: 15668 > Logged by: Alexander Lakhin > Email address: exclus...@gmail.com > PostgreSQL version: Unsupported/Unk

Re: speeding up planning with partitions

2019-03-05 Thread Amit Langote
On 2019/03/06 0:57, Jesper Pedersen wrote: > On 3/5/19 5:24 AM, Amit Langote wrote: >> Attached an updated version.  This incorporates fixes for both Jesper's >> and Imai-san's review.  I haven't been able to pin down the bug (or >> whatever) that makes throu

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
Hi David, On 2019/03/06 11:06, David Rowley wrote: > On Tue, 5 Mar 2019 at 03:01, Derek Hans wrote: >> Based on a reply to reporting this as a bug, moving rows out of foreign >> partitions is not yet implemented so this is behaving as expected. There's a >> mention of this limitation in the Not

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
On 2019/03/06 11:29, David Rowley wrote: > On Wed, 6 Mar 2019 at 15:26, Amit Langote > wrote: >> >>> I've tried to put all this right again in the attached. However, I was >>> a bit unsure of what "but they can be moved into a foreign table if >&

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
Fujita-san, On 2019/03/06 13:04, Etsuro Fujita wrote: > (2019/03/06 11:34), Amit Langote wrote: >> Ah, indeed.  In the documentation fix patch I'd posted, I also made >> changes to release-11.sgml to link to the limitations section.  (I'm >> attaching it here fo

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
On 2019/03/06 12:47, David Rowley wrote: > On Wed, 6 Mar 2019 at 16:29, Etsuro Fujita > wrote: >> That means that rows can be moved from a local partition to a foreign >> partition if the FDW supports it. > > It seems a bit light on detail to me. If I was a user I'd want to know > what exactly t

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
On 2019/03/06 13:30, David Rowley wrote: > On Wed, 6 Mar 2019 at 17:20, Amit Langote > wrote: >> >> On 2019/03/06 12:47, David Rowley wrote: >>> It seems a bit light on detail to me. If I was a user I'd want to know >>> what exactly the FDW neede

Re: speeding up planning with partitions

2019-03-05 Thread Amit Langote
Imai-san, Thanks for the review. On 2019/03/06 11:09, Imai, Yoshikazu wrote: > Here is the code review for previous v26 patches. > > [0002] > In expand_inherited_rtentry(): > > expand_inherited_rtentry() > { > ... > + RelOptInfo *rel = NULL; > > can be declared at more later: > > if (ol

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
Fujita-san, On 2019/03/06 15:10, Etsuro Fujita wrote: > --- a/doc/src/sgml/ddl.sgml > +++ b/doc/src/sgml/ddl.sgml > @@ -3376,6 +3376,13 @@ ALTER TABLE measurement ATTACH PARTITION > measurement_y2008m02 >    >   > > + > +  > +   UPDATE row movement is not supported in the

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-05 Thread Amit Langote
Hi, On 2019/03/06 15:48, Michael Paquier wrote: > On Tue, Mar 05, 2019 at 11:04:17PM +0900, Amit Langote wrote: >> Maybe we should error out as follows in >> transformPartitionRangeBounds(), although that means we'll get >> different error message than when using list pa

bug in update tuple routing with foreign partitions

2019-03-06 Thread Amit Langote
Hi, (added Fujita-san) I noticed a bug with how UPDATE tuple routing initializes ResultRelInfos to use for partition routing targets. Specifically, the bug occurs when UPDATE targets include a foreign partition that is locally modified (as opposed to being modified directly on the remove server)

Re: Update does not move row across foreign partitions in v11

2019-03-07 Thread Amit Langote
On 2019/03/07 22:54, Robert Haas wrote: > On Thu, Mar 7, 2019 at 7:35 AM Etsuro Fujita > wrote: >> Thanks for the patch! >> >> The patch looks good to me, but one thing I'm wondering is: as suggested >> by David, it would be better to rephrase this mention in the UPDATE >> reference page, in a sin

Re: Update does not move row across foreign partitions in v11

2019-03-07 Thread Amit Langote
Thanks for the review. On 2019/03/07 21:35, Etsuro Fujita wrote: > The patch looks good to me, but one thing I'm wondering is: as suggested > by David, it would be better to rephrase this mention in the UPDATE > reference page, in a single commit: > > "Currently, rows cannot be moved from a parti

Re: Is it too soon for a PG12 open items wiki page?

2019-03-08 Thread Amit Langote
On Fri, Mar 8, 2019 at 7:52 PM Julien Rouhaud wrote: > > On Fri, Mar 8, 2019 at 11:49 AM David Rowley > wrote: > > > > Is it too soon for a PG12 open items wiki page? > > It seems like a good timing to me. > > > I've got a few things I need to keep track of. Normally writing a > > patch and putt

Re: Update does not move row across foreign partitions in v11

2019-03-08 Thread Amit Langote
On Fri, Mar 8, 2019 at 8:21 PM David Rowley wrote: > > On Sat, 9 Mar 2019 at 00:09, Etsuro Fujita > wrote: > > IMO I think it's better that we also mention that the UPDATE can move > > rows into a foreign partition if the FDW supports it. No? > > In my opinion, there's not much need to talk abo

Re: Update does not move row across foreign partitions in v11

2019-03-08 Thread Amit Langote
On Fri, Mar 8, 2019 at 11:09 PM Alvaro Herrera wrote: > > On 2019-Mar-08, Amit Langote wrote: > > > diff --git a/doc/src/sgml/ref/update.sgml b/doc/src/sgml/ref/update.sgml > > index 77430a586c..f5cf8eab85 100644 > > --- a/doc/src/sgml/ref/update.sgml > > +

Re: Update does not move row across foreign partitions in v11

2019-03-08 Thread Amit Langote
On Sat, Mar 9, 2019 at 12:03 AM Alvaro Herrera wrote: > > On 2019-Mar-08, Amit Langote wrote: > > > On Fri, Mar 8, 2019 at 11:09 PM Alvaro Herrera > > wrote: > > > > I'm not sure about copying the same to ddl.sgml. Why is that needed? > > > Update

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-03-10 Thread Amit Langote
On 2019/03/11 0:25, Justin Pryzby wrote: > On Sun, Mar 10, 2019 at 10:53:02PM +1300, David Rowley wrote: >> On Fri, 11 May 2018 at 17:37, Amit Langote >> wrote: >>> 5. The last sentence in caveats, that is, >>> >>> "Partitioning using these

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-03-10 Thread Amit Langote
On 2019/03/11 11:00, David Rowley wrote: > On Mon, 11 Mar 2019 at 14:33, Amit Langote > wrote: >> PG 11 moved the needle a bit for SELECT queries: >> >> Excluding unnecessary partitions is slow for UPDATE and DELETE queries, > > With those words I expect the use

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-03-10 Thread Amit Langote
On 2019/03/11 11:13, David Rowley wrote: > On Mon, 11 Mar 2019 at 15:00, David Rowley > wrote: >> >> On Mon, 11 Mar 2019 at 14:33, Amit Langote >> wrote: >>> PG 11 moved the needle a bit for SELECT queries: >>> >>> Excluding unnecessa

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-03-10 Thread Amit Langote
On 2019/03/11 13:22, Justin Pryzby wrote: > On Mon, Mar 11, 2019 at 01:06:08PM +0900, Amit Langote wrote: >> On 2019/03/11 11:13, David Rowley wrote: >>> On Mon, 11 Mar 2019 at 15:00, David Rowley >>> wrote: >>>> On Mon, 11 Mar 2019 at 14:33, Amit Lang

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-10 Thread Amit Langote
Hi, On 2019/03/06 17:27, Michael Paquier wrote: > On Wed, Mar 06, 2019 at 04:00:42PM +0900, Amit Langote wrote: >> Thanks for looking at this. Your patch seems better, because it allows us >> to keep the error message consistent with the message one would get with >> li

Re: Update does not move row across foreign partitions in v11

2019-03-12 Thread Amit Langote
On 2019/03/13 1:04, Alvaro Herrera wrote: > On 2019-Mar-09, Amit Langote wrote: > >> Attached find 3 patches -- for PG 10, 11, and HEAD. I also realizes >> that a description of PARTITION OF clause was also missing in the >> Parameters section of CREATE FOREIGN T

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-03-12 Thread Amit Langote
On 2019/03/13 8:28, David Rowley wrote: > On Wed, 13 Mar 2019 at 04:07, Robert Haas wrote: >> I think it should be added to one of the existing sub-headings. I >> suggest adding it to the end of 5.10.1 and rephrasing it so that it >> makes clearer the distinction between what will happen with >>

Re: Inadequate executor locking of indexes

2019-03-12 Thread Amit Langote
(this is not a reply to your full proposal, just something I thought to point out) On 2019/03/13 10:38, David Rowley wrote: > i.e don't open the indexes for DELETEs. I had ideas that maybe this > could be changed to check the idxlockmode and open the indexes if it's > above AccessSharedLock. The

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-12 Thread Amit Langote
On 2019/03/13 1:35, Tom Lane wrote: > Robert Haas writes: >> On Mon, Mar 11, 2019 at 2:45 AM Amit Langote >> wrote: >>> I noticed another issue with the code -- it's using strcmp() to compare >>> specified string against "minvalue" and "

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-12 Thread Amit Langote
On 2019/03/11 16:21, Michael Paquier wrote: > On Mon, Mar 11, 2019 at 03:44:39PM +0900, Amit Langote wrote: >> We could make the error message more meaningful depending on the context, >> but maybe it'd better be pursue it as a separate project. > > Yeah, I noticed tha

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-12 Thread Amit Langote
On 2019/03/13 14:15, Amit Langote wrote: > On 2019/03/11 16:21, Michael Paquier wrote: >> On Mon, Mar 11, 2019 at 03:44:39PM +0900, Amit Langote wrote: >>> We could make the error message more meaningful depending on the context, >>> but maybe it'd better be

Re: ToDo: show size of partitioned table

2019-03-13 Thread Amit Langote
On 2019/02/22 1:41, Pavel Stehule wrote: > čt 21. 2. 2019 v 0:56 odesílatel Justin Pryzby > napsal: > >> On Sat, Feb 16, 2019 at 10:52:35PM +0100, Pavel Stehule wrote: >>> I like your changes. I merged all - updated patch is attached >> >> I applied and tested your v10 patch. >> >> Find attached

Re: Inadequate executor locking of indexes

2019-03-13 Thread Amit Langote
Hi David, On 2019/03/13 10:38, David Rowley wrote: > I had another go at this patch and fixed the problem by just setting > the idxlockmode inside the planner just before the call to > expand_inherited_tables(). This allows the lockmode to be copied into > child RTEs. I have one question about t

Re: hyrax vs. RelationBuildPartitionDesc

2019-03-13 Thread Amit Langote
On 2019/03/14 5:18, Tom Lane wrote: > Robert Haas writes: >> On Wed, Mar 13, 2019 at 3:14 PM Tom Lane wrote: >>> Meanwhile, who's going to take point on cleaning up rd_partcheck? >>> I don't really understand this code well enough to know whether that >>> can share one of the existing partitionin

Re: ToDo: show size of partitioned table

2019-03-13 Thread Amit Langote
On 2019/03/14 2:11, Pavel Stehule wrote: > st 13. 3. 2019 v 8:02 odesílatel Amit Langote > napsal: >> Not a native speaker either, but I like Justin's changes. Although I >> noticed that he missed changing one sentence to look like other similar >> sen

Re: Fix handling of unlogged tables in FOR ALL TABLES publications

2019-03-13 Thread Amit Langote
On 2019/03/13 21:03, Peter Eisentraut wrote: > If a FOR ALL TABLES publication exists, unlogged tables are ignored > for publishing changes. But CheckCmdReplicaIdentity() would still > check in that case that such a table has a replica identity set before > accepting updates. That is useless, so

Re: Fix handling of unlogged tables in FOR ALL TABLES publications

2019-03-13 Thread Amit Langote
On 2019/03/14 15:03, Kyotaro HORIGUCHI wrote: > At Thu, 14 Mar 2019 11:30:12 +0900, Amit Langote > wrote in > <59e5a734-9e06-1035-385b-626717581...@lab.ntt.co.jp> >> On 2019/03/13 21:03, Peter Eisentraut wrote: >>> If a FOR ALL TABLES publication exists, unlog

why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Amit Langote
Hi, I'm curious why DestroyPartitionDirectory doesn't do hash_destroy(pdir->pdir_hash)? Thanks, Amit

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Amit Langote
On 2019/03/14 16:32, Kyotaro HORIGUCHI wrote: > At Thu, 14 Mar 2019 16:13:23 +0900, Amit Langote > wrote in > <3ad792cd-0805-858e-595c-c09e9d1ce...@lab.ntt.co.jp> >> Hi, >> >> I'm curious why DestroyPartitionDirectory doesn't do >> hash_d

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Amit Langote
On 2019/03/14 16:46, Amit Langote wrote: > On 2019/03/14 16:32, Kyotaro HORIGUCHI wrote: >> At Thu, 14 Mar 2019 16:13:23 +0900, Amit Langote >> wrote in >> <3ad792cd-0805-858e-595c-c09e9d1ce...@lab.ntt.co.jp> >>> Hi, >>> >>> I'm curiou

Re: Inadequate executor locking of indexes

2019-03-14 Thread Amit Langote
On 2019/03/14 7:12, David Rowley wrote: > Thanks for having a look at this. > > On Wed, 13 Mar 2019 at 22:45, Amit Langote > wrote: >> I have one question about the relation between idxlockmode and >> rellockmode? From skimming the patch, it appears that they're alm

Re: speeding up planning with partitions

2019-03-14 Thread Amit Langote
Imai-san, On 2019/03/13 19:34, Imai, Yoshikazu wrote: > I have done code review of v31 patches from 0001 to 0004. > I described below what I confirmed or thoughts. Thanks for checking. > 0001: This seems ok. > > 0002: > * I don't really know that delaying adding resjunk output columns to the pl

Re: [sqlsmith] Failed assertion at relnode.c

2019-03-14 Thread Amit Langote
On Thu, Mar 14, 2019 at 11:53 PM tushar wrote: > > Hi, > > While running sqlsmith against 12devel, got the the following > assertion- (issue is reproducible on v10/v11 as well) > > TRAP: FailedAssertion("!(bms_is_subset(appendrel->lateral_relids, > required_outer))", File: "relnode.c", Line: 152

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-17 Thread Amit Langote
On 2019/03/15 1:02, Robert Haas wrote: > On Thu, Mar 14, 2019 at 3:13 AM Amit Langote > wrote: >> I'm curious why DestroyPartitionDirectory doesn't do >> hash_destroy(pdir->pdir_hash)? > > What would be the point? It's more efficient to let context

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-17 Thread Amit Langote
On 2019/03/15 2:56, Robert Haas wrote: > On Thu, Mar 14, 2019 at 1:16 PM Tom Lane wrote: >> Actually, now that I've absorbed a bit more about 898e5e329, >> I don't like very much about it at all. I think having it >> try to hang onto pointers into the relcache is a completely >> wrongheaded desig

Re: partitioned tables referenced by FKs

2019-03-18 Thread Amit Langote
Hi, On 2019/03/15 2:31, Alvaro Herrera wrote: > Once I was finished, fixed bugs and tested it, I realized that that was > a stupid thing to have done -- because THOSE ARE DIFFERENT CONSTRAINTS. > When you say "fk (a) references pk1" you're saying that all the values > in fk(a) must appear in pk1.

Re: Problem with default partition pruning

2019-03-18 Thread Amit Langote
Hosoya-san, On 2019/03/15 15:05, Yuzuko Hosoya wrote: > Indeed, it's problematic. I also did test and I found that > this problem was occurred when any partition didn't match > WHERE clauses. So following query didn't work correctly. > > # explain select * from test1_3 where (id > 0 and id <

Re: speeding up planning with partitions

2019-03-18 Thread Amit Langote
Imai-san, On 2019/03/15 14:40, Imai, Yoshikazu wrote: > Amit-san, > > I have another little comments about v31-patches. > > * We don't need is_first_child in inheritance_planner(). > > On Fri, Mar 8, 2019 at 9:18 AM, Amit Langote wrote: >> On 2019/03/08 16:

Re: partitioned tables referenced by FKs

2019-03-19 Thread Amit Langote
Hi, Thanks for updating the patch. I'll reply to other parts separately. On 2019/03/19 7:02, Alvaro Herrera wrote: > A pretty silly bug remains here. Watch: > > create table pk (a int primary key) partition by list (a); > create table pk1 partition of pk for values in (1); > create table fk (a

Re: partitioned tables referenced by FKs

2019-03-19 Thread Amit Langote
On Tue, Mar 19, 2019 at 8:49 PM Alvaro Herrera wrote: > On 2019-Mar-19, Amit Langote wrote: > > > Will it suffice or be OK if we skipped invoking the pre-drop callback for > > objects that are being "indirectly" dropped? I came up with the attached > > patch to

Re: speeding up planning with partitions

2019-03-19 Thread Amit Langote
On 2019/03/20 9:49, Robert Haas wrote: > On Fri, Mar 8, 2019 at 4:18 AM Amit Langote > wrote: >> Maybe you know that range_table_mutator() spends quite a long time if >> there are many target children, but I realized there's no need for >> range_table_mutator() to

Re: speeding up planning with partitions

2019-03-19 Thread Amit Langote
Imai-san, On 2019/03/20 11:21, Imai, Yoshikazu wrote: > (4) > We expect the performance does not depend on the number of partitions after > applying all patches, if possible. > > num of partTPS > --- - > 1024 7,257 (7274, 7246, 7252) > 2048 6,718 (6627, 6780, 674

Re: speeding up planning with partitions

2019-03-19 Thread Amit Langote
On 2019/03/20 11:51, Imai, Yoshikazu wrote: > Amit-san, > > On Wed, Mar 20, 2019 at 2:34 AM, Amit Langote wrote: >> On 2019/03/20 11:21, Imai, Yoshikazu wrote: >>> (4) >>> We expect the performance does not depend on the number of partitions >> after applyin

selecting from partitions and constraint exclusion

2019-03-19 Thread Amit Langote
Hi, While looking at a partition pruning bug [1], I noticed something that started to feel like a regression: Setup: create table p (a int) partition by list (a); create table p1 partition of p for values in (1); In PG 10: set constraint_exclusion to on; explain select * from p1 where a = 2;

Re: speeding up planning with partitions

2019-03-20 Thread Amit Langote
Imai-san, On 2019/03/20 12:15, Imai, Yoshikazu wrote: > Here the details. > > [creating partitioned tables (with 1024 partitions)] > drop table if exists rt; > create table rt (a int, b int, c int) partition by range (a); > \o /dev/null > select 'create table rt' || x::text || ' partition of rt f

Re: Problem with default partition pruning

2019-03-20 Thread Amit Langote
Hi Thibaut, On 2019/03/19 23:58, Thibaut Madelaine wrote: > I kept on testing with sub-partitioning. Thanks. > I found a case, using 2 default partitions, where a default partition is > not pruned: > > -- > > create table test2(id int, val text) partition by range (id); > create tab

Re: speeding up planning with partitions

2019-03-20 Thread Amit Langote
Imai-san, On 2019/03/20 17:36, Imai, Yoshikazu wrote: > On Wed, Mar 20, 2019 at 8:21 AM, Amit Langote wrote: >> On 2019/03/20 12:15, Imai, Yoshikazu wrote: >>> [select1024.sql] >>> \set a random (1, 1024) >>> select * from rt where a = :a; >>> >>

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-20 Thread Amit Langote
Hi, On 2019/03/20 11:07, Michael Paquier wrote: > On Thu, Mar 14, 2019 at 01:23:08PM +0900, Michael Paquier wrote: >> I actually think that what you propose here makes more sense than what >> HEAD does because the most inner expression gets evaluated first. >> This for example generates the same e

Re: ToDo: show size of partitioned table

2019-03-21 Thread Amit Langote
On 2019/03/22 2:23, David Steele wrote: > On 3/14/19 6:19 AM, Amit Langote wrote: >> On 2019/03/14 2:11, Pavel Stehule wrote: >>> >>>> I've attached v11 of the patch, which merges most of Justin's changes and >>>> some of my own on top -- docume

Re: speeding up planning with partitions

2019-03-21 Thread Amit Langote
Jesper, Imai-san, Thanks for testing and reporting your findings. On 2019/03/21 23:10, Imai Yoshikazu wrote: > On 2019/03/20 23:25, Jesper Pedersen wrote:> Hi, > > My tests - using hash partitions - shows that the extra time is spent in > > make_partition_pruneinfo() for the relid_subplan_map v

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-21 Thread Amit Langote
Hi, Thanks for splitting. It makes sense, because, as you know, the bug that causes the crash is a separate problem from unintuitive error messages which result from the way in which we parse expressions. On 2019/03/22 14:09, Michael Paquier wrote: > On Wed, Mar 20, 2019 at 06:17:27PM +0900, Mic

Re: Problem with default partition pruning

2019-03-21 Thread Amit Langote
Hosoya-san, On 2019/03/22 15:02, Yuzuko Hosoya wrote: > I understood Amit's proposal. But I think the issue Thibaut reported would > occur regardless of whether clauses have OR clauses or not as follows. > I tested a query which should output "One-Time Filter: false". > > # explain select * fro

Re: selecting from partitions and constraint exclusion

2019-03-22 Thread Amit Langote
Hi David, Thanks for checking. On 2019/03/20 19:41, David Rowley wrote: > On Wed, 20 Mar 2019 at 17:37, Amit Langote > wrote: >> That's because get_relation_constraints() no longer (as of PG 11) includes >> the partition constraint for SELECT queries. But that's ba

Re: Problem with default partition pruning

2019-03-24 Thread Amit Langote
Hi, On 2019/03/23 2:36, Thibaut Madelaine wrote: > I tested your last patch and if I didn't mix up patches on the end of a > too long week, I get a problem when querying the sub-sub partition: > > test=# explain select * from test2_0_10 where id = 25; > QUERY PLAN

Re: selecting from partitions and constraint exclusion

2019-03-24 Thread Amit Langote
On 2019/03/22 17:17, Amit Langote wrote: > I'll add this to July fest to avoid forgetting about this. I'd forgotten to do this, but done today. :) Thanks, Amit

Re: selecting from partitions and constraint exclusion

2019-03-25 Thread Amit Langote
On 2019/03/26 0:21, Robert Haas wrote: > On Wed, Mar 20, 2019 at 12:37 AM Amit Langote > wrote: >> That's because get_relation_constraints() no longer (as of PG 11) includes >> the partition constraint for SELECT queries. > > What commit made that change? That

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-25 Thread Amit Langote
Hi, On 2019/03/26 10:15, Michael Paquier wrote: > Done as you suggested, with a minimal set enough to trigger the crash, > still the error message is rather misleading as you would expect :) Thanks for committing. >> A separate thread will definitely attract more attention, at least in due >> ti

Re: psql display of foreign keys

2019-03-25 Thread Amit Langote
Hi, On 2019/03/26 7:38, Alvaro Herrera wrote: > v9 attached; this one's final AFAICT. Agreed. Thanks, Amit

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-25 Thread Amit Langote
On 2019/03/26 1:53, Alvaro Herrera wrote: > Here's v6 of this patch. I have rebased on top of today's CLUSTER > monitoring, as well as on table AM commits. The latter caused a bit of > trouble, as now the number of blocks processed by a scan is not as easy > to get as before; I added a new entry

Re: partitioned tables referenced by FKs

2019-03-25 Thread Amit Langote
Hi Jesper, On 2019/03/22 22:01, Jesper Pedersen wrote: > Hi Alvaro, > > On 3/21/19 6:18 PM, Alvaro Herrera wrote: >> On 2019-Mar-21, Jesper Pedersen wrote: >>> pgbench -M prepared -f select.sql >>> >>> I'm seeing 82.64% spent in GetCachedPlan(). plan_cache_mode is auto. >> >> Hmm, I can't re

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-03-26 Thread Amit Langote
Tsunakawa-san, On 2019/03/26 17:21, Tsunakawa, Takayuki wrote: > From: David Rowley [mailto:david.row...@2ndquadrant.com] >> On Mon, 25 Mar 2019 at 23:44, Peter Eisentraut >> wrote: >>> Perhaps "speeding up planning with partitions" needs to be accepted first? >> >> Yeah, I think it likely will r

Re: partitioned tables referenced by FKs

2019-03-26 Thread Amit Langote
Hi Alvaro, On 2019/03/22 6:54, Alvaro Herrera wrote: > Here's v7; Needs rebasing on top of 940311e4bb3. 0001: + Oid objectClass = getObjectClass(thisobj); I guess you meant to use ObjectClass, not Oid here. Tested 0002 a bit more and found some problems. cre

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 1:06, Tom Lane wrote: > Amit Langote writes: >> 0002 is a new patch to get rid of the duplicate RTE and PlanRowMark that's >> created for partitioned parent table, as it's pointless. I was planning >> to propose it later, but decided to post it now i

Re: BUG #15672: PostgreSQL 11.1/11.2 crashed after dropping a partition table

2019-03-26 Thread Amit Langote
On 2019/03/08 19:22, Amit Langote wrote: > On 2019/03/07 20:36, Amit Langote wrote: >> On Thu, Mar 7, 2019 at 11:17 AM Amit Langote >> wrote: >>> The problem start when ALTER TABLE users ALTER COLUMN is executed. >>> create table users(user_id int, name v

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 12:06, Amit Langote wrote: > I wonder if I should rework inherit.c so that its internal interfaces > don't pass around parent Relation, but make do with the RelOptInfo? I'll > need to add tupdesc and reltype fields to RelOptInfo to go ahead with that > though.

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 13:50, Amit Langote wrote: > On 2019/03/27 12:06, Amit Langote wrote: >> I wonder if I should rework inherit.c so that its internal interfaces >> don't pass around parent Relation, but make do with the RelOptInfo? I'll >> need to add tupdesc and rel

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 14:26, David Rowley wrote: > On Wed, 27 Mar 2019 at 18:13, Amit Langote > wrote: >> >> On 2019/03/27 13:50, Amit Langote wrote: >>> On 2019/03/27 12:06, Amit Langote wrote: >>>> I wonder if I should rework inherit.c so that its internal i

Re: Ordered Partitioned Table Scans

2019-03-26 Thread Amit Langote
Hi David, Sorry if this was discussed before, but why does this patch add any new code to partprune.c? AFAICT, there's no functionality changes to the pruning code. Both +bool +partkey_is_bool_constant_for_query(RelOptInfo *partrel, int partkeycol) and +static bool +matches_boolean_partition_

Re: Ordered Partitioned Table Scans

2019-03-27 Thread Amit Langote
On 2019/03/27 15:48, Amit Langote wrote: > Hi David, > > Sorry if this was discussed before, but why does this patch add any new > code to partprune.c? AFAICT, there's no functionality changes to the > pruning code. > > Both > > +bool > +partkey_is_bool_con

Re: Ordered Partitioned Table Scans

2019-03-27 Thread Amit Langote
Hi, On 2019/03/28 7:29, David Rowley wrote: > On Wed, 27 Mar 2019 at 19:48, Amit Langote > wrote: >> Sorry if this was discussed before, but why does this patch add any new >> code to partprune.c? AFAICT, there's no functionality changes to the >> pruning code. &

Re: partitioned tables referenced by FKs

2019-03-27 Thread Amit Langote
Hi, On 2019/03/27 8:27, Alvaro Herrera wrote: > On 2019-Mar-26, Alvaro Herrera wrote: > >> Thanks for the thorough testing and bug analysis! It was spot-on. I've >> applied your two proposed fixes, as well as added a new test setup that >> covers both these bugs. The attached set is rebased on

Re: speeding up planning with partitions

2019-03-27 Thread Amit Langote
On 2019/03/27 23:57, Tom Lane wrote: > David Rowley writes: >> On Wed, 27 Mar 2019 at 18:39, Amit Langote >> wrote: >>> On 2019/03/27 14:26, David Rowley wrote: >>>> Perhaps the way to make this work, at least in the long term is to do >>>&g

Re: ToDo: show size of partitioned table

2019-03-28 Thread Amit Langote
Hi, On 2019/03/28 18:12, Peter Eisentraut wrote: > On 2019-03-22 01:21, Amit Langote wrote: >> On 2019/03/22 2:23, David Steele wrote: >>> On 3/14/19 6:19 AM, Amit Langote wrote: >>>> On 2019/03/14 2:11, Pavel Stehule wrote: >>>>> >>>&g

Re: ToDo: show size of partitioned table

2019-03-28 Thread Amit Langote
On 2019/03/28 18:31, Amit Langote wrote: > Hi, > > On 2019/03/28 18:12, Peter Eisentraut wrote: >> On 2019-03-22 01:21, Amit Langote wrote: >>> On 2019/03/22 2:23, David Steele wrote: >>>> On 3/14/19 6:19 AM, Amit Langote wrote: >>>>> On 2019/0

Re: speeding up planning with partitions

2019-03-28 Thread Amit Langote
On 2019/03/28 14:03, Tom Lane wrote: > Amit Langote writes: >> On 2019/03/27 23:57, Tom Lane wrote: >>> Yeah, there's something to be said for having plancat.c open each table >>> *and store the Relation pointer in the RelOptInfo*, and then close that >>>

Re: Ordered Partitioned Table Scans

2019-03-28 Thread Amit Langote
Hi David, On 2019/03/28 8:04, David Rowley wrote: > On Wed, 27 Mar 2019 at 21:24, Amit Langote > wrote: >> Noticed a typo. >> >> + * multiple subpaths then we can't make guarantees about the >> + * order tuples in those subpath

Re: Misleading errors with column references in default expressions and partition bounds

2019-03-28 Thread Amit Langote
On 2019/03/28 21:14, Michael Paquier wrote: > On Wed, Mar 27, 2019 at 12:13:16PM +0900, Michael Paquier wrote: >> ParseExprKind is an enum, so listing all the options without the >> default has the advantage to generate a warning if somebody adds a >> value. This way anybody changing this code wil

  1   2   3   4   5   6   7   8   9   10   >