Re: Ordered Partitioned Table Scans

2019-04-01 Thread Amit Langote
Hi, On 2019/03/29 7:59, David Rowley wrote: > On Fri, 29 Mar 2019 at 00:00, Amit Langote > wrote: >> >> On 2019/03/28 8:04, David Rowley wrote: >>> If it's *always* scanned last then it's fine for ORDER BY partkey >>> NULLS LAST. If they have ORDER BY partkey

Re: speeding up planning with partitions

2019-04-04 Thread Amit Langote
On 2019/04/05 6:59, David Rowley wrote: > On Fri, 5 Apr 2019 at 07:33, Floris Van Nee wrote: >> I had a question about the performance of pruning of functions like now() >> and current_date. I know these are handled differently, as they cannot be >> excluded during the first phases of planning.

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-04 Thread Amit Langote
On 2019/04/05 5:42, Peter Eisentraut wrote: > On 2019-04-04 06:58, Amit Langote wrote: >> Also, since the "speed up partition planning" patch went in (428b260f8), >> it might be possible to see the performance boost even with the >> partitioning example you cited up

Re: Why does "toast_tuple_target" allow values below TOAST_TUPLE_TARGET?

2019-04-04 Thread Amit Langote
On 2019/04/05 14:09, David Rowley wrote: > Or rather, why does the reloption allow values below the compile-time > constant? Maybe there is already a discussion in progress on the topic? * Caveats from reloption toast_tuple_target *

Re: speeding up planning with partitions

2019-04-04 Thread Amit Langote
On 2019/04/05 12:18, David Rowley wrote: > On Fri, 5 Apr 2019 at 16:09, Amit Langote > wrote: >> Although, we still have ways >> to go in terms of scaling generic plan execution to larger partition >> counts, solution(s) for which have been proposed by David but haven'

Re: speeding up planning with partitions

2019-04-05 Thread Amit Langote
On 2019/04/02 14:50, Amit Langote wrote: > Attached patch is only for HEAD this time. I'll post one for PG 11 (if > you'd like) once we reach consensus on the best thing to do here is. While we're on the topic of the relation between constraint exclusion and partition pruning, I'd like

Re: Problem with default partition pruning

2019-04-05 Thread Amit Langote
Hosoya-san, On 2019/04/04 13:00, Yuzuko Hosoya wrote: > I added some test cases to each patch according to tests > discussed in this thread. Thanks a lot. > However, I found another problem as follows. This query should > output "One-Time Filter: false" because rlp3's constraints >

Re: speeding up planning with partitions

2019-04-05 Thread Amit Langote
Hi, On 2019/04/05 18:13, Floris Van Nee wrote: > One unrelated thing I noticed (but I'm not sure if it's worth a separate > email thread) is that the changed default of jit=on in v12 doesn't work very > well with a large number of partitions at run-time, for which a large number > get excluded

Re: speeding up planning with partitions

2019-03-29 Thread Amit Langote
Thanks for the new patches. On Sat, Mar 30, 2019 at 9:17 AM Tom Lane wrote: > > Amit Langote writes: > > On 2019/03/29 7:38, Tom Lane wrote: > >> 2. I seriously dislike what's been done in joinrels.c, too. That > >> really seems like a kluge (and I haven't had

Re: Ordered Partitioned Table Scans

2019-03-28 Thread Amit Langote
Hi David, On 2019/03/28 8:04, David Rowley wrote: > On Wed, 27 Mar 2019 at 21:24, Amit Langote > wrote: >> Noticed a typo. >> >> + * multiple subpaths then we can't make guarantees about the >> + * order tuples in those subpaths, so

Re: speeding up planning with partitions

2019-03-28 Thread Amit Langote
On 2019/03/28 14:03, Tom Lane wrote: > Amit Langote writes: >> On 2019/03/27 23:57, Tom Lane wrote: >>> Yeah, there's something to be said for having plancat.c open each table >>> *and store the Relation pointer in the RelOptInfo*, and then close that >>> agai

Re: speeding up planning with partitions

2019-03-30 Thread Amit Langote
On Sun, Mar 31, 2019 at 12:11 AM Tom Lane wrote: > Amit Langote writes: > > I think the performance results did prove that degradation due to > > those loops over part_rels becomes significant for very large > > partition counts. Is there a better solution than the bitmap

Re: Berserk Autovacuum (let's save next Mandrill)

2019-03-30 Thread Amit Langote
On Sun, Mar 31, 2019 at 1:11 AM Robert Haas wrote: > > On Wed, Mar 27, 2019 at 5:32 PM Alvaro Herrera > wrote: > > * certain tables would have some sort of partial scan that sets the > > visibility map. There's no reason to invoke the whole vacuuming > > machinery. I don't think this is

Re: speeding up planning with partitions

2019-03-30 Thread Amit Langote
On Sun, Mar 31, 2019 at 12:59 AM Robert Haas wrote: > > On Sat, Mar 30, 2019 at 11:46 AM Tom Lane wrote: > > > The only problem with PartitionPruneInfo structures of which I am > > > aware is that they rely on PartitionDesc offsets not changing. But I > > > added code in that commit in

Re: speeding up planning with partitions

2019-03-30 Thread Amit Langote
On Sun, Mar 31, 2019 at 11:45 AM Imai Yoshikazu wrote: > On 2019/03/31 1:06, Amit Langote wrote: > > On Sun, Mar 31, 2019 at 12:11 AM Tom Lane wrote: > >> I am curious as to why there seems to be more degradation > >> for hash cases, as

Re: ToDo: show size of partitioned table

2019-03-28 Thread Amit Langote
Hi, On 2019/03/28 18:12, Peter Eisentraut wrote: > On 2019-03-22 01:21, Amit Langote wrote: >> On 2019/03/22 2:23, David Steele wrote: >>> On 3/14/19 6:19 AM, Amit Langote wrote: >>>> On 2019/03/14 2:11, Pavel Stehule wrote: >>>>> >>>

Re: ToDo: show size of partitioned table

2019-03-28 Thread Amit Langote
On 2019/03/28 18:31, Amit Langote wrote: > Hi, > > On 2019/03/28 18:12, Peter Eisentraut wrote: >> On 2019-03-22 01:21, Amit Langote wrote: >>> On 2019/03/22 2:23, David Steele wrote: >>>> On 3/14/19 6:19 AM, Amit Langote wrote: >>>>> On 2019/0

Re: speeding up planning with partitions

2019-03-28 Thread Amit Langote
Thanks a lot for hacking on the patch. I'm really happy with the direction you took for inheritance_planner, as it allows UPDATE/DELETE to use partition pruning. On 2019/03/29 7:38, Tom Lane wrote: > I've been hacking on this pretty hard over the last couple days, > because I really didn't like

Re: Misleading errors with column references in default expressions and partition bounds

2019-03-28 Thread Amit Langote
On 2019/03/28 21:14, Michael Paquier wrote: > On Wed, Mar 27, 2019 at 12:13:16PM +0900, Michael Paquier wrote: >> ParseExprKind is an enum, so listing all the options without the >> default has the advantage to generate a warning if somebody adds a >> value. This way anybody changing this code

Re: speeding up planning with partitions

2019-03-29 Thread Amit Langote
Here are some comments on v38. On 2019/03/29 12:44, Amit Langote wrote: > Thanks again for the new patch. I'm reading it now and will send comments > later today if I find something. -Assert(rte->rtekind == RTE_RELATION || - rte->rtekind ==

Re: speeding up planning with partitions

2019-03-27 Thread Amit Langote
On 2019/03/27 23:57, Tom Lane wrote: > David Rowley writes: >> On Wed, 27 Mar 2019 at 18:39, Amit Langote >> wrote: >>> On 2019/03/27 14:26, David Rowley wrote: >>>> Perhaps the way to make this work, at least in the long term is to do >>>

Re: Ordered Partitioned Table Scans

2019-03-27 Thread Amit Langote
Hi, On 2019/03/28 7:29, David Rowley wrote: > On Wed, 27 Mar 2019 at 19:48, Amit Langote > wrote: >> Sorry if this was discussed before, but why does this patch add any new >> code to partprune.c? AFAICT, there's no functionality changes to the >> pruning code. >

Re: partitioned tables referenced by FKs

2019-03-27 Thread Amit Langote
Hi, On 2019/03/27 8:27, Alvaro Herrera wrote: > On 2019-Mar-26, Alvaro Herrera wrote: > >> Thanks for the thorough testing and bug analysis! It was spot-on. I've >> applied your two proposed fixes, as well as added a new test setup that >> covers both these bugs. The attached set is rebased

Re: speeding up planning with partitions

2019-04-01 Thread Amit Langote
On 2019/04/01 3:46, Tom Lane wrote: > One thing that I intentionally left out of the committed patch was changes > to stop short of scanning the whole simple_rel_array when looking only for > baserels. I thought that had been done in a rather piecemeal fashion > and it'd be better to address it

Re: speeding up planning with partitions

2019-04-01 Thread Amit Langote
On 2019/03/30 0:29, Tom Lane wrote: > Amit Langote writes: >> Finally, it's not in the patch, but how about visiting >> get_relation_constraints() for revising this block of code: > > That seems like probably an independent patch --- do you want to write it? Here is th

Re: ToDo: show size of partitioned table

2019-04-08 Thread Amit Langote
Hi, On 2019/04/08 12:59, Pavel Stehule wrote: > ne 7. 4. 2019 v 21:13 odesílatel Alvaro Herrera > napsal: >> Added to the docs, and pushed. >> > > Thank you very much Thank you Alvaro for pushing this to completion. Also, thank you Justin and Pavel for reviewing it till the last minute.

Re: Back-branch bugs with fully-prunable UPDATEs

2019-04-07 Thread Amit Langote
On 2019/04/08 1:57, Tom Lane wrote: > Amit Langote writes: >> On Sun, Apr 7, 2019 at 5:28 AM Tom Lane wrote: >>> This test script works fine in HEAD: >>> In v11, it suffers an assertion failure in ExecSetupPartitionTupleRouting. >>> In v10, it doesn't crash,

Re: selecting from partitions and constraint exclusion

2019-04-07 Thread Amit Langote
Hi Thibaut, On 2019/04/06 1:12, Thibaut wrote: > Le 25/03/2019 à 01:31, Amit Langote a écrit : >> On 2019/03/22 17:17, Amit Langote wrote: >>> I'll add this to July fest to avoid forgetting about this. >> I'd forgotten to do this, but done today. :) >> >

Re: Ordered Partitioned Table Scans

2019-04-07 Thread Amit Langote
On 2019/04/06 18:06, Julien Rouhaud wrote: > On Sat, Apr 6, 2019 at 2:45 AM David Rowley > wrote: >> >> On Sat, 6 Apr 2019 at 12:26, Tom Lane wrote: >>> Pushed with some hacking, mostly trying to improve the comments. >> >> Great! Many thanks for improving those and pushing it. >> >> Many thanks

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-08 Thread Amit Langote
On 2019/03/14 10:40, Amit Langote wrote: > On 2019/03/14 5:18, Tom Lane wrote: >> Robert Haas writes: >>> On Wed, Mar 13, 2019 at 3:14 PM Tom Lane wrote: >>>> Meanwhile, who's going to take point on cleaning up rd_partcheck? >>>> I don't really unders

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 1:06, Tom Lane wrote: > Amit Langote writes: >> 0002 is a new patch to get rid of the duplicate RTE and PlanRowMark that's >> created for partitioned parent table, as it's pointless. I was planning >> to propose it later, but decided to post it now if we're mo

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 12:06, Amit Langote wrote: > I wonder if I should rework inherit.c so that its internal interfaces > don't pass around parent Relation, but make do with the RelOptInfo? I'll > need to add tupdesc and reltype fields to RelOptInfo to go ahead with that > though. To give

Re: BUG #15672: PostgreSQL 11.1/11.2 crashed after dropping a partition table

2019-03-26 Thread Amit Langote
On 2019/03/08 19:22, Amit Langote wrote: > On 2019/03/07 20:36, Amit Langote wrote: >> On Thu, Mar 7, 2019 at 11:17 AM Amit Langote >> wrote: >>> The problem start when ALTER TABLE users ALTER COLUMN is executed. >>> create table users(user_id int, name v

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 13:50, Amit Langote wrote: > On 2019/03/27 12:06, Amit Langote wrote: >> I wonder if I should rework inherit.c so that its internal interfaces >> don't pass around parent Relation, but make do with the RelOptInfo? I'll >> need to add tupdesc and reltype field

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 14:26, David Rowley wrote: > On Wed, 27 Mar 2019 at 18:13, Amit Langote > wrote: >> >> On 2019/03/27 13:50, Amit Langote wrote: >>> On 2019/03/27 12:06, Amit Langote wrote: >>>> I wonder if I should rework inherit.c so that its internal i

Re: speeding up planning with partitions

2019-02-18 Thread Amit Langote
Thanks for looking. On 2019/02/19 2:27, Tom Lane wrote: > Amit Langote writes: >> [ v22 patch set ] > > I started to look at this, and immediately choked on the 0001 patch: > > if (childpruned || > !apply_child_basequals(root, rel, chi

Re: pg_partition_tree crashes for a non-defined relation

2019-02-28 Thread Amit Langote
Hi, On 2019/03/01 9:22, Michael Paquier wrote: > On Thu, Feb 28, 2019 at 04:32:03PM -0300, Alvaro Herrera wrote: >> Yeah, looks good, please push. > > Done for this part. > >> I would opt for returning the empty set for legacy inheritance too. >> >> More generally, I think we should return

Re: pg_partition_tree crashes for a non-defined relation

2019-03-03 Thread Amit Langote
On 2019/03/02 18:21, Michael Paquier wrote: > On Fri, Mar 01, 2019 at 11:38:20AM -0500, Tom Lane wrote: >> Right, while you'd get zero rows out for a non-partitioned table. >> WFM. > > Exactly. I have committed a patch doing exactly that, and I have > added test cases with a partitioned table

Re: speeding up planning with partitions

2019-03-04 Thread Amit Langote
Imai-san, Thanks for the review. On 2019/03/04 18:14, Imai, Yoshikazu wrote: > I've taken at glance the codes and there are some minor comments about the > patch. > > * You changed a usage/arguments of get_relation_info, but why you did it? I > made those codes back to the original code and

Re: speeding up planning with partitions

2019-03-04 Thread Amit Langote
Hi, On 2019/03/01 22:01, Amit Langote wrote: > Please find attached updated patches. Will update source code comments, > commit message and perform other fine-tuning over the weekend if possible. I realized when "fine-tuning" that the patch 0001 contained too many changes tha

Re: speeding up planning with partitions

2019-02-20 Thread Amit Langote
On 2019/02/21 0:50, Tom Lane wrote: > However, there are other reasons why I'm not really happy with the > approach proposed in this patch. > > The main problem is that cloning the PlannerInfo while still sharing a lot > of infrastructure between the clones is a horrid hack that I think will be >

Re: Problem with default partition pruning

2019-02-26 Thread Amit Langote
Hosoya-san, On 2019/02/22 17:14, Yuzuko Hosoya wrote: > Hi, > > I found the bug of default partition pruning when executing a range query. > > - > postgres=# create table test1(id int, val text) partition by range (id); > postgres=# create table test1_1 partition of test1 for values from

Re: speeding up planning with partitions

2019-03-04 Thread Amit Langote
Imai-san, Thanks for checking. On 2019/03/05 15:03, Imai, Yoshikazu wrote: > I've also done code review of 0001 and 0002 patch so far. > > [0001] > 1. Do we need to open/close a relation in add_appendrel_other_rels()? > > + if (rel->part_scheme) > { > - ListCell *l; >

Re: Inheriting table AMs for partitioned tables

2019-03-04 Thread Amit Langote
On 2019/03/05 11:59, Amit Langote wrote: > On 2019/03/05 8:47, Andres Freund wrote: >> Hi, >> >> In the pluggable storage patch [1], one thing that I'm wondering about >> is how exactly to inherit the storage AM across partitions. I think >> that's potentially wo

Re: Inheriting table AMs for partitioned tables

2019-03-04 Thread Amit Langote
On 2019/03/05 8:47, Andres Freund wrote: > Hi, > > In the pluggable storage patch [1], one thing that I'm wondering about > is how exactly to inherit the storage AM across partitions. I think > that's potentially worthy of a discussion with a wider audience than I'd > get in that thread. It

Re: speeding up planning with partitions

2019-03-04 Thread Amit Langote
Hi Jesper, Thanks for the review. I've made all of the changes per your comments in my local repository. I'll post the updated patches after diagnosing what I'm suspecting a memory over-allocation bug in one of the patches. If you configure build with --enable-cassert, you'll see that

Re: psql display of foreign keys

2019-03-04 Thread Amit Langote
On 2019/03/05 4:41, Alvaro Herrera wrote: > Here's the patch I'm really interested about :-) Thanks for the updated patch. I applied it and rebased the foreign-keys-referencing-partitioned-tables patch on top. Here's something I think you may have missed: -- partitioned primary key table

Re: speeding up planning with partitions

2019-03-05 Thread Amit Langote
Imai-san, Thanks for the review. On 2019/03/06 11:09, Imai, Yoshikazu wrote: > Here is the code review for previous v26 patches. > > [0002] > In expand_inherited_rtentry(): > > expand_inherited_rtentry() > { > ... > + RelOptInfo *rel = NULL; > > can be declared at more later: > > if

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
Fujita-san, On 2019/03/06 15:10, Etsuro Fujita wrote: > --- a/doc/src/sgml/ddl.sgml > +++ b/doc/src/sgml/ddl.sgml > @@ -3376,6 +3376,13 @@ ALTER TABLE measurement ATTACH PARTITION > measurement_y2008m02 >    >   > > + > +  > +   UPDATE row movement is not supported in the

bug in update tuple routing with foreign partitions

2019-03-06 Thread Amit Langote
Hi, (added Fujita-san) I noticed a bug with how UPDATE tuple routing initializes ResultRelInfos to use for partition routing targets. Specifically, the bug occurs when UPDATE targets include a foreign partition that is locally modified (as opposed to being modified directly on the remove

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-05 Thread Amit Langote
Hi, On 2019/03/06 15:48, Michael Paquier wrote: > On Tue, Mar 05, 2019 at 11:04:17PM +0900, Amit Langote wrote: >> Maybe we should error out as follows in >> transformPartitionRangeBounds(), although that means we'll get >> different error message than when using list partiti

Re: speeding up planning with partitions

2019-02-22 Thread Amit Langote
Hi Justin, Thanks for checking. On Sat, Feb 23, 2019 at 1:59 AM Justin Pryzby wrote: > > On Fri, Feb 22, 2019 at 09:45:38PM +0900, Amit Langote wrote: > > I have updated the inheritance expansion patch. > > > > Patch 0001 rewrites optimizer/utils/inherit.c, so that it al

Re: speeding up planning with partitions

2019-03-05 Thread Amit Langote
On 2019/03/05 9:50, Amit Langote wrote: > I'll post the updated patches after diagnosing what > I'm suspecting a memory over-allocation bug in one of the patches. If you > configure build with --enable-cassert, you'll see that throughput > decreases as number of partitions run into ma

Re: speeding up planning with partitions

2019-03-05 Thread Amit Langote
On 2019/03/04 19:38, Amit Langote wrote: > 2. Defer inheritance expansion to add_other_rels_to_query(). Although the > purpose of doing this is to perform partition pruning before adding the > children, this patch doesn't change when the pruning occurs. It deals > with other issu

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-05 Thread Amit Langote
Hi, (cc'ing -hackers and Peter E) On Tue, Mar 5, 2019 at 8:02 PM PG Bug reporting form wrote: > > The following bug has been logged on the website: > > Bug reference: 15668 > Logged by: Alexander Lakhin > Email address: exclus...@gmail.com > PostgreSQL version:

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
On 2019/03/06 12:47, David Rowley wrote: > On Wed, 6 Mar 2019 at 16:29, Etsuro Fujita > wrote: >> That means that rows can be moved from a local partition to a foreign >> partition if the FDW supports it. > > It seems a bit light on detail to me. If I was a user I'd want to know > what exactly

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
Fujita-san, On 2019/03/06 13:04, Etsuro Fujita wrote: > (2019/03/06 11:34), Amit Langote wrote: >> Ah, indeed.  In the documentation fix patch I'd posted, I also made >> changes to release-11.sgml to link to the limitations section.  (I'm >> attaching it here for your

Re: speeding up planning with partitions

2019-03-05 Thread Amit Langote
On 2019/03/06 0:57, Jesper Pedersen wrote: > On 3/5/19 5:24 AM, Amit Langote wrote: >> Attached an updated version.  This incorporates fixes for both Jesper's >> and Imai-san's review.  I haven't been able to pin down the bug (or >> whatever) that makes throughput go down a

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
On 2019/03/06 11:29, David Rowley wrote: > On Wed, 6 Mar 2019 at 15:26, Amit Langote > wrote: >> >>> I've tried to put all this right again in the attached. However, I was >>> a bit unsure of what "but they can be moved into a foreign table if &g

Re: speeding up planning with partitions

2019-03-01 Thread Amit Langote
On 2019/03/01 22:01, Amit Langote wrote: > Of course, I had to make sure that query_planner (which is not > in the charge of adding source inheritance child objects) can notice that. Oops, I meant to write "query_planner (which *is* in the charge of adding source inheritance ch

Re: speeding up planning with partitions

2019-03-01 Thread Amit Langote
Imai-san, Thanks for testing and sorry it took me a while to reply. On 2019/02/25 15:24, Imai, Yoshikazu wrote: > [update_pt_with_joining_another_pt.sql] > update rt set c = jrt.c + 100 from jrt where rt.b = jrt.b; > > [pgbench] > pgbench -n -f update_pt_with_joining_another_pt_for_ptkey.sql -T

Re: pg_partition_tree crashes for a non-defined relation

2019-02-27 Thread Amit Langote
Hi, On 2019/02/28 10:45, Michael Paquier wrote: > On Wed, Feb 27, 2019 at 03:48:08PM -0300, Alvaro Herrera wrote: >> I just happened to come across the result of this rationale in >> pg_partition_tree() (an SRF) while developing a new related function, >> pg_partition_ancestors(), and find the

Re: partitioned tables referenced by FKs

2019-02-27 Thread Amit Langote
Hi Alvaro, I looked at the latest patch and most of the issues/bugs that I was going to report based on the late January version of the patch seem to have been taken care of; sorry that I couldn't post sooner which would've saved you some time. The patch needs to be rebased on top of ff11e7f4b9

Re: BUG #15623: Inconsistent use of default for updatable view

2019-02-27 Thread Amit Langote
On 2019/02/27 18:37, Dean Rasheed wrote: > On Tue, 12 Feb 2019 at 10:33, Dean Rasheed wrote: >> Here's an updated patch ... > > So I pushed that. However, ... > > Playing around with it some more, I realised that whilst this appeared > to fix the reported problem, it exposes another issue which

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Amit Langote
On 2019/03/14 16:46, Amit Langote wrote: > On 2019/03/14 16:32, Kyotaro HORIGUCHI wrote: >> At Thu, 14 Mar 2019 16:13:23 +0900, Amit Langote >> wrote in >> <3ad792cd-0805-858e-595c-c09e9d1ce...@lab.ntt.co.jp> >>> Hi, >>> >>> I'm curious why

Re: speeding up planning with partitions

2019-03-14 Thread Amit Langote
Imai-san, On 2019/03/13 19:34, Imai, Yoshikazu wrote: > I have done code review of v31 patches from 0001 to 0004. > I described below what I confirmed or thoughts. Thanks for checking. > 0001: This seems ok. > > 0002: > * I don't really know that delaying adding resjunk output columns to the

why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Amit Langote
Hi, I'm curious why DestroyPartitionDirectory doesn't do hash_destroy(pdir->pdir_hash)? Thanks, Amit

Re: Fix handling of unlogged tables in FOR ALL TABLES publications

2019-03-14 Thread Amit Langote
On 2019/03/14 15:03, Kyotaro HORIGUCHI wrote: > At Thu, 14 Mar 2019 11:30:12 +0900, Amit Langote > wrote in > <59e5a734-9e06-1035-385b-626717581...@lab.ntt.co.jp> >> On 2019/03/13 21:03, Peter Eisentraut wrote: >>> If a FOR ALL TABLES publication exist

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Amit Langote
On 2019/03/14 16:32, Kyotaro HORIGUCHI wrote: > At Thu, 14 Mar 2019 16:13:23 +0900, Amit Langote > wrote in > <3ad792cd-0805-858e-595c-c09e9d1ce...@lab.ntt.co.jp> >> Hi, >> >> I'm curious why DestroyPartitionDirectory doesn't do >> hash_destroy(pdi

Re: Inadequate executor locking of indexes

2019-03-14 Thread Amit Langote
On 2019/03/14 7:12, David Rowley wrote: > Thanks for having a look at this. > > On Wed, 13 Mar 2019 at 22:45, Amit Langote > wrote: >> I have one question about the relation between idxlockmode and >> rellockmode? From skimming the patch, it appears that they're almost &

Re: [sqlsmith] Failed assertion at relnode.c

2019-03-14 Thread Amit Langote
On Thu, Mar 14, 2019 at 11:53 PM tushar wrote: > > Hi, > > While running sqlsmith against 12devel, got the the following > assertion- (issue is reproducible on v10/v11 as well) > > TRAP: FailedAssertion("!(bms_is_subset(appendrel->lateral_relids, > required_outer))", File: "relnode.c", Line:

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-17 Thread Amit Langote
On 2019/03/15 1:02, Robert Haas wrote: > On Thu, Mar 14, 2019 at 3:13 AM Amit Langote > wrote: >> I'm curious why DestroyPartitionDirectory doesn't do >> hash_destroy(pdir->pdir_hash)? > > What would be the point? It's more efficient to let context teardown >

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-17 Thread Amit Langote
On 2019/03/15 2:56, Robert Haas wrote: > On Thu, Mar 14, 2019 at 1:16 PM Tom Lane wrote: >> Actually, now that I've absorbed a bit more about 898e5e329, >> I don't like very much about it at all. I think having it >> try to hang onto pointers into the relcache is a completely >> wrongheaded

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-03-12 Thread Amit Langote
On 2019/03/13 8:28, David Rowley wrote: > On Wed, 13 Mar 2019 at 04:07, Robert Haas wrote: >> I think it should be added to one of the existing sub-headings. I >> suggest adding it to the end of 5.10.1 and rephrasing it so that it >> makes clearer the distinction between what will happen with >>

Re: Update does not move row across foreign partitions in v11

2019-03-12 Thread Amit Langote
On 2019/03/13 1:04, Alvaro Herrera wrote: > On 2019-Mar-09, Amit Langote wrote: > >> Attached find 3 patches -- for PG 10, 11, and HEAD. I also realizes >> that a description of PARTITION OF clause was also missing in the >> Parameters section of CREATE FOREIGN T

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-12 Thread Amit Langote
On 2019/03/13 1:35, Tom Lane wrote: > Robert Haas writes: >> On Mon, Mar 11, 2019 at 2:45 AM Amit Langote >> wrote: >>> I noticed another issue with the code -- it's using strcmp() to compare >>> specified string against "minvalue" and "maxva

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-12 Thread Amit Langote
On 2019/03/11 16:21, Michael Paquier wrote: > On Mon, Mar 11, 2019 at 03:44:39PM +0900, Amit Langote wrote: >> We could make the error message more meaningful depending on the context, >> but maybe it'd better be pursue it as a separate project. > > Yeah, I noticed th

Re: Inadequate executor locking of indexes

2019-03-12 Thread Amit Langote
(this is not a reply to your full proposal, just something I thought to point out) On 2019/03/13 10:38, David Rowley wrote: > i.e don't open the indexes for DELETEs. I had ideas that maybe this > could be changed to check the idxlockmode and open the indexes if it's > above AccessSharedLock.

Re: ToDo: show size of partitioned table

2019-03-13 Thread Amit Langote
On 2019/02/22 1:41, Pavel Stehule wrote: > čt 21. 2. 2019 v 0:56 odesílatel Justin Pryzby > napsal: > >> On Sat, Feb 16, 2019 at 10:52:35PM +0100, Pavel Stehule wrote: >>> I like your changes. I merged all - updated patch is attached >> >> I applied and tested your v10 patch. >> >> Find attached

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-13 Thread Amit Langote
On 2019/03/13 14:15, Amit Langote wrote: > On 2019/03/11 16:21, Michael Paquier wrote: >> On Mon, Mar 11, 2019 at 03:44:39PM +0900, Amit Langote wrote: >>> We could make the error message more meaningful depending on the context, >>> but maybe it'd better be purs

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-03-10 Thread Amit Langote
On 2019/03/11 11:13, David Rowley wrote: > On Mon, 11 Mar 2019 at 15:00, David Rowley > wrote: >> >> On Mon, 11 Mar 2019 at 14:33, Amit Langote >> wrote: >>> PG 11 moved the needle a bit for SELECT queries: >>> >>> Excluding unnecessa

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-03-10 Thread Amit Langote
On 2019/03/11 11:00, David Rowley wrote: > On Mon, 11 Mar 2019 at 14:33, Amit Langote > wrote: >> PG 11 moved the needle a bit for SELECT queries: >> >> Excluding unnecessary partitions is slow for UPDATE and DELETE queries, > > With those words I expect the use

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-03-10 Thread Amit Langote
On 2019/03/11 0:25, Justin Pryzby wrote: > On Sun, Mar 10, 2019 at 10:53:02PM +1300, David Rowley wrote: >> On Fri, 11 May 2018 at 17:37, Amit Langote >> wrote: >>> 5. The last sentence in caveats, that is, >>> >>> "Partitioning using the

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-03-10 Thread Amit Langote
On 2019/03/11 13:22, Justin Pryzby wrote: > On Mon, Mar 11, 2019 at 01:06:08PM +0900, Amit Langote wrote: >> On 2019/03/11 11:13, David Rowley wrote: >>> On Mon, 11 Mar 2019 at 15:00, David Rowley >>> wrote: >>>> On Mon, 11 Mar 2019 at 14:33, Amit

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-11 Thread Amit Langote
Hi, On 2019/03/06 17:27, Michael Paquier wrote: > On Wed, Mar 06, 2019 at 04:00:42PM +0900, Amit Langote wrote: >> Thanks for looking at this. Your patch seems better, because it allows us >> to keep the error message consistent with the message one would get with >> li

Re: Inadequate executor locking of indexes

2019-03-13 Thread Amit Langote
Hi David, On 2019/03/13 10:38, David Rowley wrote: > I had another go at this patch and fixed the problem by just setting > the idxlockmode inside the planner just before the call to > expand_inherited_tables(). This allows the lockmode to be copied into > child RTEs. I have one question about

Re: Is it too soon for a PG12 open items wiki page?

2019-03-08 Thread Amit Langote
On Fri, Mar 8, 2019 at 7:52 PM Julien Rouhaud wrote: > > On Fri, Mar 8, 2019 at 11:49 AM David Rowley > wrote: > > > > Is it too soon for a PG12 open items wiki page? > > It seems like a good timing to me. > > > I've got a few things I need to keep track of. Normally writing a > > patch and

Re: Update does not move row across foreign partitions in v11

2019-03-08 Thread Amit Langote
On Sat, Mar 9, 2019 at 12:03 AM Alvaro Herrera wrote: > > On 2019-Mar-08, Amit Langote wrote: > > > On Fri, Mar 8, 2019 at 11:09 PM Alvaro Herrera > > wrote: > > > > I'm not sure about copying the same to ddl.sgml. Why is that needed? > > > Update is n

Re: Update does not move row across foreign partitions in v11

2019-03-08 Thread Amit Langote
On Fri, Mar 8, 2019 at 11:09 PM Alvaro Herrera wrote: > > On 2019-Mar-08, Amit Langote wrote: > > > diff --git a/doc/src/sgml/ref/update.sgml b/doc/src/sgml/ref/update.sgml > > index 77430a586c..f5cf8eab85 100644 > > --- a/doc/src/sgml/ref/update.sgml > > +

Re: Update does not move row across foreign partitions in v11

2019-03-08 Thread Amit Langote
On Fri, Mar 8, 2019 at 8:21 PM David Rowley wrote: > > On Sat, 9 Mar 2019 at 00:09, Etsuro Fujita > wrote: > > IMO I think it's better that we also mention that the UPDATE can move > > rows into a foreign partition if the FDW supports it. No? > > In my opinion, there's not much need to talk

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
On 2019/03/06 13:30, David Rowley wrote: > On Wed, 6 Mar 2019 at 17:20, Amit Langote > wrote: >> >> On 2019/03/06 12:47, David Rowley wrote: >>> It seems a bit light on detail to me. If I was a user I'd want to know >>> what exactly the FDW needed to

Re: speeding up planning with partitions

2019-03-19 Thread Amit Langote
On 2019/03/20 9:49, Robert Haas wrote: > On Fri, Mar 8, 2019 at 4:18 AM Amit Langote > wrote: >> Maybe you know that range_table_mutator() spends quite a long time if >> there are many target children, but I realized there's no need for >> range_table_mutator() to copy/

Re: speeding up planning with partitions

2019-03-19 Thread Amit Langote
Imai-san, On 2019/03/20 11:21, Imai, Yoshikazu wrote: > (4) > We expect the performance does not depend on the number of partitions after > applying all patches, if possible. > > num of partTPS > --- - > 1024 7,257 (7274, 7246, 7252) > 2048 6,718 (6627, 6780,

Re: speeding up planning with partitions

2019-03-19 Thread Amit Langote
On 2019/03/20 11:51, Imai, Yoshikazu wrote: > Amit-san, > > On Wed, Mar 20, 2019 at 2:34 AM, Amit Langote wrote: >> On 2019/03/20 11:21, Imai, Yoshikazu wrote: >>> (4) >>> We expect the performance does not depend on the number of partitions >> after applyin

Re: speeding up planning with partitions

2019-03-20 Thread Amit Langote
Imai-san, On 2019/03/20 12:15, Imai, Yoshikazu wrote: > Here the details. > > [creating partitioned tables (with 1024 partitions)] > drop table if exists rt; > create table rt (a int, b int, c int) partition by range (a); > \o /dev/null > select 'create table rt' || x::text || ' partition of rt

Re: Problem with default partition pruning

2019-03-20 Thread Amit Langote
Hi Thibaut, On 2019/03/19 23:58, Thibaut Madelaine wrote: > I kept on testing with sub-partitioning. Thanks. > I found a case, using 2 default partitions, where a default partition is > not pruned: > > -- > > create table test2(id int, val text) partition by range (id); > create

selecting from partitions and constraint exclusion

2019-03-19 Thread Amit Langote
Hi, While looking at a partition pruning bug [1], I noticed something that started to feel like a regression: Setup: create table p (a int) partition by list (a); create table p1 partition of p for values in (1); In PG 10: set constraint_exclusion to on; explain select * from p1 where a = 2;

Re: speeding up planning with partitions

2019-03-20 Thread Amit Langote
Imai-san, On 2019/03/20 17:36, Imai, Yoshikazu wrote: > On Wed, Mar 20, 2019 at 8:21 AM, Amit Langote wrote: >> On 2019/03/20 12:15, Imai, Yoshikazu wrote: >>> [select1024.sql] >>> \set a random (1, 1024) >>> select * from rt where a = :a; >>> >>

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-20 Thread Amit Langote
Hi, On 2019/03/20 11:07, Michael Paquier wrote: > On Thu, Mar 14, 2019 at 01:23:08PM +0900, Michael Paquier wrote: >> I actually think that what you propose here makes more sense than what >> HEAD does because the most inner expression gets evaluated first. >> This for example generates the same

Re: partitioned tables referenced by FKs

2019-03-19 Thread Amit Langote
On Tue, Mar 19, 2019 at 8:49 PM Alvaro Herrera wrote: > On 2019-Mar-19, Amit Langote wrote: > > > Will it suffice or be OK if we skipped invoking the pre-drop callback for > > objects that are being "indirectly" dropped? I came up with the attached > > patch to

<    4   5   6   7   8   9   10   11   12   13   >