Re: Getting ERROR: could not open file "base/13164/t3_16388" with partition table with ON COMMIT

2018-11-05 Thread Amit Langote
On 2018/11/06 4:37, Alvaro Herrera wrote: > On 2018-Nov-02, Amit Langote wrote: > >> Well, performDeletion *does* drop the child, because when the parent is >> dropped due to its ON COMMIT DROP action, it's done using: >> >> /* >>

Re: First-draft release notes for back-branch releases

2018-11-05 Thread Amit Langote
On 2018/11/06 11:25, Noah Misch wrote: > On Mon, Nov 05, 2018 at 04:01:59PM -0500, Jonathan S. Katz wrote: >> Attached is a draft of the press release for review. Please let me know >> if there are any corrections/suggestions. >> * Disallows the creation of a new partition from a trigger that is at

Re: First-draft release notes for back-branch releases

2018-11-05 Thread Amit Langote
Hi, On 2018/11/06 12:49, Jonathan S. Katz wrote: > On 11/5/18 9:58 PM, Amit Langote wrote: >> On 2018/11/06 11:25, Noah Misch wrote: >>> On Mon, Nov 05, 2018 at 04:01:59PM -0500, Jonathan S. Katz wrote: >>>> Attached is a draft of the press release for review. Please

Re: BUG #15212: Default values in partition tables don't work as expected and allow NOT NULL violation

2018-11-06 Thread Amit Langote
Hi, Thank you for looking at this. On 2018/11/06 7:25, Alvaro Herrera wrote: > On 2018-Aug-07, Amit Langote wrote: > >>> But in >>> case of partitioning there is only one parent and hence >>> coldef->constraints is NULL and hence just overwriting it

Re: ON COMMIT actions and inheritance

2018-11-06 Thread Amit Langote
Hi, On 2018/11/06 12:03, Michael Paquier wrote: > On Mon, Nov 05, 2018 at 02:37:05PM +0900, Amit Langote wrote: >> Michael pointed out a problem with specifying different ON COMMIT actions >> on a temporary inheritance parent and its children: >> >> https://ww

Re: BUG #15212: Default values in partition tables don't work as expected and allow NOT NULL violation

2018-11-06 Thread Amit Langote
Hi, On 2018/11/07 0:10, Alvaro Herrera wrote: > Looking over the stuff in gram.y (to make sure there's nothing that > could be lost), I noticed that we're losing the COLLATE clause when they > are added to columns in partitions. I would expect part1 to end up with > collation es_CL, or alternativ

Re: First-draft release notes for back-branch releases

2018-11-06 Thread Amit Langote
On 2018/11/07 11:28, Noah Misch wrote: > On Mon, Nov 05, 2018 at 10:49:30PM -0500, Jonathan S. Katz wrote: >> On 11/5/18 9:58 PM, Amit Langote wrote: >>> On 2018/11/06 11:25, Noah Misch wrote: >>>> On Mon, Nov 05, 2018 at 04:01:59PM -0500, Jonathan S. Katz wrote: &g

Re: speeding up planning with partitions

2018-11-06 Thread Amit Langote
Thanks for looking. On 2018/11/07 12:32, Alvaro Herrera wrote: > ISTM patch 0004 is impossible to review just because of size -- I > suppose the bulk of it is just code that moves from one file to another, > but that there are also code changes in it. Maybe it would be better to > split it so tha

Re: move PartitionBoundInfo creation code

2018-11-06 Thread Amit Langote
Hi, Thank your for taking a look. On 2018/11/05 16:21, Michael Paquier wrote: > On Thu, Nov 01, 2018 at 01:03:00PM +0900, Amit Langote wrote: >> Done a few moments ago. :) > > From the file size this move is actually negative. From what I can see > partcache decreases t

Re: First-draft release notes for back-branch releases

2018-11-06 Thread Amit Langote
Hi, On 2018/11/07 13:08, Jonathan S. Katz wrote: > Thanks. I left it as "Fix several crashes with triggers" - please see > attached. Thank you. Just one more comment on the top sentence. * Ensure that if a parent partition has an index created in a tablespace, then all child indexes will be cre

Re: move PartitionBoundInfo creation code

2018-11-07 Thread Amit Langote
On 2018/11/08 11:48, Michael Paquier wrote: >>> Thinking crazy, we could also create a subfolder partitioning/bounds/ >>> which includes three files with this refactoring:x >>> - range.c >>> - values.c >>> - hash.c >>> Then we keep partbounds.c which is the entry point used by partcache.c, >>> and

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-07 Thread Amit Langote
them. There are good reasons to do > that independently of reducing lock levels, and we certainly couldn't > do it without having some efficient way to check whether it had > already been done. So then the mechanism wouldn't feel like so much > like a special-purpose hack just for concurrent ATTACH/DETACH. (Was > Amit Langote already working on this, or was that some other kind of > on-demand locking?) I think the patch mentioned above gets us closer to that goal. Thanks, Amit

Re: move PartitionBoundInfo creation code

2018-11-07 Thread Amit Langote
On 2018/11/08 0:04, Alvaro Herrera wrote: >> On Wed, Nov 07, 2018 at 03:34:38PM +0900, Amit Langote wrote: >>> I think we can design the interface of partition_bounds_create such that >>> it returns information needed to re-arrange OIDs to be in the canonical >>&

Re: move PartitionBoundInfo creation code

2018-11-07 Thread Amit Langote
On 2018/11/08 12:59, Amit Langote wrote: > It might be okay to split the big switch in partition_bounds_create() into > different functions for different partitioning methods for clarity as you > say, but let's avoid creating new files for range, list, and hash. > > I will p

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-11-07 Thread Amit Langote
On 2018/11/07 20:46, David Rowley wrote: > On 5 November 2018 at 20:17, Amit Langote > wrote: >> On 2018/11/04 19:07, David Rowley wrote: >>> Perhaps a better design would be to instead of having random special >>> partitioned-table-only fields in ResultRelInfo,

Re: ON COMMIT actions and inheritance

2018-11-07 Thread Amit Langote
Hi, Thank you updating the patch and adding notes to the documentation about the points I raised. On 2018/11/07 9:53, Michael Paquier wrote: > On Tue, Nov 06, 2018 at 07:04:17PM +0900, Amit Langote wrote: >> Agree with keeping it simple. Maybe, we could (should?) document that the &

Re: ON COMMIT actions and inheritance

2018-11-08 Thread Amit Langote
On 2018/11/08 18:03, Michael Paquier wrote: > On Thu, Nov 08, 2018 at 04:46:46PM +0900, Amit Langote wrote: >> How about: >> When used on tables with inheritance children (including partitioned >> tables), this also drops the children (partitions). > > Even if the styl

Re: BUG #15212: Default values in partition tables don't work as expected and allow NOT NULL violation

2018-11-08 Thread Amit Langote
On Fri, Nov 9, 2018 at 1:03 AM Alvaro Herrera wrote: > On 2018-Nov-07, Amit Langote wrote: > > > I think the result in this case should be an error, just as it would in > > the regular inheritance case. > > > > create table parent (a text); > > create table chi

Re: BUG #15212: Default values in partition tables don't work as expected and allow NOT NULL violation

2018-11-08 Thread Amit Langote
On 2018/11/09 1:38, Alvaro Herrera wrote: > On 2018-Nov-09, Amit Langote wrote: > >> On Fri, Nov 9, 2018 at 1:03 AM Alvaro Herrera >> wrote: >>> On 2018-Nov-07, Amit Langote wrote: > >>> Hmm, I'm thinking perhaps we shouldn't backpatch this pa

Re: BUG #15212: Default values in partition tables don't work as expected and allow NOT NULL violation

2018-11-08 Thread Amit Langote
On 2018/11/09 4:39, Alvaro Herrera wrote: > Pushed. Thank you for committing. I've closed the CF entry. > I included the test case for collations to the three branches, but no > code changes. We can patch master for the handling of collations per > your patch, Okay, but should we back-patch it

Re: BUG #15212: Default values in partition tables don't work as expected and allow NOT NULL violation

2018-11-08 Thread Amit Langote
On 2018/11/09 14:04, Amit Langote wrote: > On 2018/11/09 4:39, Alvaro Herrera wrote: >> and if somebody has it, a change to how defaults are applied >> when routing tuples. > > I haven't written such a patch yet. Do we want such a feature? Or is it a *bug* of tup

Re: using expression syntax for partition bounds

2018-11-08 Thread Amit Langote
Authors: Kyotaro Horiguchi, Tom Lane, Amit Langote --- doc/src/sgml/ref/alter_table.sgml | 6 +- doc/src/sgml/ref/create_table.sgml | 16 +-- src/backend/commands/tablecmds.c | 9 ++ src/backend/optimizer/util/clauses.c | 4 +- src/backend/parse

Re: Performance improvements of INSERTs to a partitioned table

2018-11-08 Thread Amit Langote
On 2018/11/09 11:19, David Rowley wrote: > On 7 November 2018 at 21:31, Kato, Sho wrote: >> AFAIK, When CREATE INDEX on a partition and INSERT to a parent table are >> executed at the same time, this patch causes deadlock. >> >> * partitions information >> >> Partition key: RANGE (a) >> Partitions

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-11-08 Thread Amit Langote
On 2018/11/08 20:28, David Rowley wrote: > On 8 November 2018 at 20:15, Amit Langote > wrote: >> Actually, as I also proposed upthread, we should move root_tuple_slot from >> PartitionTupleRouting to ModifyTableState as mt_root_tuple_slot, because >> it's part of t

Re: speeding up planning with partitions

2018-11-09 Thread Amit Langote
Imai-san, Thank you for reviewing. On 2018/11/05 17:28, Imai, Yoshikazu wrote: > Since it takes me a lot of time to see all of the patches, I will post > comments > little by little from easy parts like typo check. I've fixed the typos you pointed out. > 4. > 0004: > -static void expand_inheri

Re: speeding up planning with partitions

2018-11-09 Thread Amit Langote
Imai-san, On 2018/11/07 10:00, Imai, Yoshikazu wrote: > About inheritance_make_rel_from_joinlist(), I considered how it processes > joins for sub-partitioned-table. > > sub-partitioned-table image: > part > sub1 > leaf1 > leaf2 > > inheritance_make_rel_from_joinlist() translates join_l

Re: BUG #15212: Default values in partition tables don't work as expected and allow NOT NULL violation

2018-11-11 Thread Amit Langote
On 2018/11/10 7:33, Tom Lane wrote: > Alvaro Herrera writes: >> On 2018-Nov-09, Jürgen Strobel wrote: >>> Regarding your example, what I expected is that *both* inserts would >>> consistently result in a tuple of (1, 42) since p should route the >>> insert to p1 and use p1's defaults. The current

Re: BUG #15212: Default values in partition tables don't work as expected and allow NOT NULL violation

2018-11-11 Thread Amit Langote
On 2018/11/12 12:59, Tom Lane wrote: > Amit Langote writes: >> On 2018/11/10 7:33, Tom Lane wrote: >>> I'd argue not, actually. I think there is plausible precedent in >>> updatable views, where what we use is the defaults associated with the >>> view, n

Re: move PartitionBoundInfo creation code

2018-11-12 Thread Amit Langote
Hi, On 2018/11/12 22:55, Michael Paquier wrote: > On Thu, Nov 08, 2018 at 03:11:35PM +0900, Amit Langote wrote: >> And here is the new version. The break down into smaller local >> functions for different partitioning methods is in patch 0002. > > Okay, here we go. Th

Re: move PartitionBoundInfo creation code

2018-11-12 Thread Amit Langote
On 2018/11/13 11:34, Michael Paquier wrote: > Attached is an updated patch. Perhaps you are spotting something else? Looks good to me. Thanks, Amit

Re: move PartitionBoundInfo creation code

2018-11-12 Thread Amit Langote
On 2018/11/13 12:40, Amit Langote wrote: > On 2018/11/13 11:34, Michael Paquier wrote: >> Attached is an updated patch. Perhaps you are spotting something else? > > Looks good to me. Marked the CF entry as Ready for Committer. Thanks, Amit

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-11-13 Thread Amit Langote
On 2018/11/14 0:32, Jesper Pedersen wrote: > Hi, > > On 11/12/18 6:17 PM, David Rowley wrote: >> On 9 November 2018 at 19:18, Amit Langote >> wrote: >>> I have a comment regarding how you chose to make >>> PartitionTupleRouting private. >>> >>

Re: move PartitionBoundInfo creation code

2018-11-13 Thread Amit Langote
Hi, On 2018/11/13 22:59, Alvaro Herrera wrote: > On 2018-Nov-13, Michael Paquier wrote: > >> On Tue, Nov 13, 2018 at 09:58:08AM -0300, Alvaro Herrera wrote: >>> "the context that was active when the function was called" is typically >>> expressed simply as "the current memory context". Perhaps t

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-11-13 Thread Amit Langote
Thanks for updating the patch. On 2018/11/14 13:16, David Rowley wrote: > Thanks for looking at this again. > > On 14 November 2018 at 13:47, Amit Langote > wrote: >> +if (dispatchidx >= proute->dispatch_allocsize) >> +{ >> +/* Expand al

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-11-14 Thread Amit Langote
On 2018/11/15 8:58, Alvaro Herrera wrote: > On 2018-Nov-15, David Rowley wrote: > >> On 15 November 2018 at 07:10, Alvaro Herrera >> wrote: >>> What's with this comment? >>> >>> * Initially we must only set up 1 PartitionDispatch object; the >>> one for >>> * the partitioned t

Re: speeding up planning with partitions

2018-11-14 Thread Amit Langote
On 2018/11/15 10:19, Imai, Yoshikazu wrote: > On Tue, Nov 13, 2018 at 10:29 PM, Amit Langote wrote: >> On 2018/11/12 13:35, Imai, Yoshikazu wrote: >>> How amount of memory is used with above tests is... >>> >>> without v5 patches, Set1: 242MB >>> withou

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-14 Thread Amit Langote
On 2018/11/15 4:27, Robert Haas wrote: > RelationBuildPartitionDesc doesn't lock the children > whose relpartbounds it is fetching (!), so unless we're guaranteed to > have already locked them children earlier for some other reason, we > could grab the partition bound at this point and then it coul

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-14 Thread Amit Langote
On 2018/11/15 11:03, Amit Langote wrote: > As Michael pointed out, the first cleanup patch needs to be rebased due to > a recent commit [1]. I did that to see if something we did in that commit > made things worse for your patch, but seems fine. I had to go and change > th

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-14 Thread Amit Langote
On 2018/11/15 14:38, Michael Paquier wrote: > On Thu, Nov 15, 2018 at 01:38:55PM +0900, Amit Langote wrote: >> I've fixed 0001 again to re-order the code so that allocations happen the >> correct context and now tests pass with the rebased patches. > > I have been lookin

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-14 Thread Amit Langote
On 2018/11/15 15:22, Michael Paquier wrote: >> If there are no partitions, nparts is 0, and other fields are NULL, though >> rd_partdesc itself is never NULL. > > I find a bit confusing that both concepts have the same meaning, aka > that a relation has no partition, and that it is actually relkin

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-15 Thread Amit Langote
On 2018/11/15 22:57, Robert Haas wrote: > On Thu, Nov 15, 2018 at 12:38 AM Michael Paquier wrote: >> On Thu, Nov 15, 2018 at 01:38:55PM +0900, Amit Langote wrote: >>> I've fixed 0001 again to re-order the code so that allocations happen the >>> correct context and

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-15 Thread Amit Langote
On Fri, Nov 16, 2018 at 1:00 PM Michael Paquier wrote: > On Fri, Nov 16, 2018 at 10:57:57AM +0900, Amit Langote wrote: > > Maybe partition_bounds_create() should've had a MemoryContext argument to > > pass it the context we want it to create the PartitionBoundInfo in. That &g

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-11-15 Thread Amit Langote
On Fri, Nov 16, 2018 at 11:40 AM Alvaro Herrera wrote: > On 2018-Nov-15, Amit Langote wrote: > > > Maybe name it PARTITION_INIT_ALLOCSIZE (dropping the ROUTING from it), or > > PROUTE_INIT_ALLOCSIZE, to make it clear that it's only allocation size? > > Here's a

Re: ToDo: show size of partitioned table

2018-11-18 Thread Amit Langote
On 2018/11/19 11:17, Alvaro Herrera wrote: > On 2018-Jul-23, Pavel Stehule wrote: > >> p.s. Another patch can be replacement of relation type from "table" to >> "partitioned table" >> >> postgres=# \dt+ >> List of relations >> +++---

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-18 Thread Amit Langote
On 2018/11/17 9:06, Michael Paquier wrote: > On Fri, Nov 16, 2018 at 09:38:40AM -0500, Robert Haas wrote: >> OK, but it seems to me that your version of my patch rearranges the >> code more than necessary. >> >> How about the attached? > > What you are proposing here looks good to me. Thanks! Me

typo fix

2018-11-19 Thread Amit Langote
Hi, It seems to me that EquivalenceClass, the struct/type name, has been misspelled as 'EquivalenceClasses' a couple of times in the comment above its definition. Attached fixes that. Thanks, Amit diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h index 6fd24203dd..a695012c

Re: typo fix

2018-11-19 Thread Amit Langote
Thank you for looking. On 2018/11/20 14:13, Michael Paquier wrote: > On Tue, Nov 20, 2018 at 02:00:39PM +0900, Amit Langote wrote: >> It seems to me that EquivalenceClass, the struct/type name, has been >> misspelled as 'EquivalenceClasses' a couple of times in the commen

Re: typo fix

2018-11-19 Thread Amit Langote
On 2018/11/20 15:58, Tom Lane wrote: > Michael Paquier writes: >> On Tue, Nov 20, 2018 at 02:00:39PM +0900, Amit Langote wrote: >>> - * We also use EquivalenceClasses as the base structure for PathKeys, >>> letting >>> + * We also use EquivalenceClass as the

Re: Regarding performance regression on specific query

2018-11-19 Thread Amit Langote
Hi, On 2018/11/20 2:49, Jung, Jinho wrote: > Execution time > = > 1.sql > 10.6 : 469 ms > 9.4.20: 10 ms > > 4.sql > 10.6 : 34019 ms > 9.4.20: 0.4 ms I noticed that these two are fixed by running ANALYZE in the database in which these queries are run. > 20.sql > 10.6 : 2791 ms > 9

Re: ToDo: show size of partitioned table

2018-11-20 Thread Amit Langote
On 2018/11/20 16:50, Michael Paquier wrote: > Testing the feature, \dP shows all partitioned relations, still does not > show the relationship when multiple levels are used. Could it make > sense to also show the direct parent of a partitioned table when > verbose mode is used? Yeah. I think it

Re: Regarding performance regression on specific query

2018-11-20 Thread Amit Langote
On Tue, Nov 20, 2018 at 10:08 PM Jung, Jinho wrote: > We are wondering how ANALYZE mitigated regression from query "1.sql" and > "4.sql". > > We followed this procedure but still observe performance regression: > 1) run ANALYZE on used table_name > analyze pg_catalog.pg_ts_parser; > analy

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-11-21 Thread Amit Langote
Hi, On Thu, Nov 22, 2018 at 7:25 AM David Rowley wrote: > > On Thu, 22 Nov 2018 at 07:06, Alvaro Herrera wrote: > > On 2018-Nov-21, David Rowley wrote: > > > If I wasn't on leave late last week and early this week then the only > > > thing I'd have mentioned was the lack of empty comment line in

Re: Inadequate executor locking of indexes

2018-11-25 Thread Amit Langote
Sorry for jumping in late on this. On 2018/11/24 1:25, Tom Lane wrote: > David Rowley writes: > Thinking more about this, the problem I noted previously about two of > these solutions not working if the index scan node is not physically > underneath the ModifyTable node actually applies to all th

Re: using expression syntax for partition bounds

2018-11-25 Thread Amit Langote
On 2018/11/09 14:38, Amit Langote wrote: > Rebased due to change in addRangeTableEntryForRelation's API. Rebased again due to changes in psql's describe output for partitioned tables. Thanks, Amit >From 999aa53b459a6fc0fe899e613406f0e0035aca94 Mon Sep 17 00:00:00 2001 From: K

Re: Inadequate executor locking of indexes

2018-11-25 Thread Amit Langote
On 2018/11/26 14:25, David Rowley wrote: > On Mon, 26 Nov 2018 at 17:37, Amit Langote > wrote: >> On 2018/11/24 1:25, Tom Lane wrote: >>> I'm beginning to think that postponing target-index locking to >>> ExecInitModifyTable was a damfool idea and we should undo

Re: Pluggable Storage - Andres's take

2018-11-26 Thread Amit Langote
Hi, On 2018/11/02 9:17, Haribabu Kommi wrote: > Here I attached the cumulative fixes of the patches, new API additions for > zheap and > basic outline of the documentation. I've read the documentation patch while also looking at the code and here are some comments. + Each table is stored as it

Re: Inadequate executor locking of indexes

2018-11-26 Thread Amit Langote
On 2018/11/27 6:19, David Rowley wrote: > On Mon, 26 Nov 2018 at 18:57, Amit Langote > wrote: >> On 2018/11/26 14:25, David Rowley wrote: >>> I'm making efforts to delay per-partition work even further in the >>> executor, for example locking of the per-partiti

Re: MERGE SQL statement for PG12

2018-11-27 Thread Amit Langote
Hi Pavan, Thanks for continuing to work on this. On 2018/11/27 20:18, Pavan Deolasee wrote: > Ok. I will try that approach again. In the meanwhile, I am posting a > rebased version. There had been quite a lot changes on partitioning side > and that caused non-trivial conflicts. I noticed a couple

Re: Planning time of Generic plan for a table partitioned into a lot

2018-11-27 Thread Amit Langote
Hi Kato-san, On 2018/11/27 19:05, Kato, Sho wrote: > Of course, in case of plan_cache_mode = force_custom_plan, it is not problem > because unnecessary paths are pruned by speeding up planning with partitions > patch[1]. > > However, if plan_cachemode is force_generic_plan, generic plan is made

Re: Planning time of Generic plan for a table partitioned into a lot

2018-11-27 Thread Amit Langote
On 2018/11/28 13:46, Amit Langote wrote: > It's cheaper than using a cached generic plan (without re-planning), > because the latter has to pay the cost of AcquireExecutorLocks which takes > longer as the number of partitions increases. Perhaps something to try > fix fixing t

Re: Add pg_partition_tree to doc index

2018-11-28 Thread Amit Langote
Hi Michael, On 2018/11/29 16:27, Michael Paquier wrote: > Hi all, > > d5eec4ee has added pg_partition_tree, unfortunately it is not showing in > the documentation index. Please see attached to fix the mistake. Oh, I wasn't aware of this index and that the tag is meant for that. > Any thoughts

Re: Planning time of Generic plan for a table partitioned into a lot

2018-11-29 Thread Amit Langote
On 2018/11/29 19:54, David Rowley wrote: > The problem is only made worse in PG11 from PG10 > because generating the custom plan has become faster than it > previously was due to the new partition pruning code which might make > it appear we can handle more partitions than we could previously, Actu

Re: Planning time of Generic plan for a table partitioned into a lot

2018-11-29 Thread Amit Langote
On 2018/11/30 14:58, David Rowley wrote: > On Fri, 30 Nov 2018 at 15:04, Amit Langote > wrote: >> >> On 2018/11/29 19:54, David Rowley wrote: >>> The problem is only made worse in PG11 from PG10 >>> because generating the custom plan has become faster than

Re: non-bulk inserts and tuple routing

2018-02-15 Thread Amit Langote
Fujita-san, Thanks for the review. On 2018/02/15 21:10, Etsuro Fujita wrote: > (2018/02/13 10:12), Amit Langote wrote: >> Updated patch is attached. > > Thanks, here are some minor comments: > > o On changes to ExecCleanupTupleRouting: > > -   Exe

Re: FOR EACH ROW triggers on partitioned tables

2018-02-15 Thread Amit Langote
On 2018/02/16 6:55, Alvaro Herrera wrote: > Amit Langote wrote: >> On 2018/02/15 6:26, Alvaro Herrera wrote: >>> Another option is to rethink this feature from the ground up: instead of >>> cloning catalog rows for each children, maybe we should have the trigger >&g

Re: non-bulk inserts and tuple routing

2018-02-15 Thread Amit Langote
On 2018/02/16 12:41, Etsuro Fujita wrote: > (2018/02/16 10:49), Amit Langote wrote: >> I think you're right.  If node->returningLists is non-NULL at all, >> ExecInitModifyTable() would've initialized the needed slot and expression >> context.  I added Assert()

Re: non-bulk inserts and tuple routing

2018-02-16 Thread Amit Langote
Fujita-san, Thanks for the review. On 2018/02/16 18:12, Etsuro Fujita wrote: > (2018/02/16 13:42), Amit Langote wrote: >> Attached v9.  Thanks a for the review! > > Thanks for the updated patch!  In the patch you added the comments: > > +   wcoList = linitial(node-

Re: non-bulk inserts and tuple routing

2018-02-18 Thread Amit Langote
Fujita-san, On 2018/02/16 19:50, Etsuro Fujita wrote: > (2018/02/16 18:23), Amit Langote wrote: >> Good idea. Done. > Thanks.  I fixed a typo (s/Converti/Convert/) and adjusted these comments > a bit further to match the preceding code/comments.  Attached is the > updated versi

Re: [HACKERS] path toward faster partition pruning

2018-02-18 Thread Amit Langote
Hi David. On 2018/02/17 18:24, David Rowley wrote: > On 2 February 2018 at 23:03, Amit Langote > wrote: >> I started wondering if it's not such a good idea to make >> PartitionClauseInfo a Node at all? I went back to your earlier message >> [1] where you said that

Re: unique indexes on partitioned tables

2018-02-20 Thread Amit Langote
Hi. On 2018/02/20 5:45, Alvaro Herrera wrote: > I pushed this now, with fixes for the last few comments there were. I noticed with the commit that, while ON CONFLICT (conflict_target) DO UPDATE gives a less surprising error message by catching it in the parser, ON CONFLICT (conflict_target) DO NO

Re: non-bulk inserts and tuple routing

2018-02-20 Thread Amit Langote
Fujita-san, On 2018/02/20 19:40, Etsuro Fujita wrote: > (2018/02/19 13:19), Amit Langote wrote: >> Attached rebased patch. > > Thanks for the rebased patch! > > One thing I noticed while updating the > tuple-routing-for-foreign-partitions patch on top of this is: we shou

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-02-21 Thread Amit Langote
Fujita-san, On 2018/02/21 20:54, Etsuro Fujita wrote: > (2018/02/02 19:33), Etsuro Fujita wrote: >> (2018/01/25 23:33), Stephen Frost wrote: >>> I'm afraid a good bit of this patch is now failing to apply. I don't >>> have much else to say except to e

Re: [HACKERS] Runtime Partition Pruning

2018-02-22 Thread Amit Langote
Hi David. On 2018/02/21 18:06, David Rowley wrote: > Other things I don't particularly like about the current patch are how > I had to construct the partition key expressions in > set_valid_runtime_subplans_recurse(). This pretty much uses code > borrowed from set_baserel_partition_key_exprs(). On

Re: [HACKERS] path toward faster partition pruning

2018-02-22 Thread Amit Langote
On 2018/02/22 17:41, David Rowley wrote: > On 21 February 2018 at 23:44, Amit Langote > wrote: >> Please find attached updated patches. > > Thanks for updating the code. > > The question I have now is around NULL handling in > partkey_datum_from_expr(). I've no

Re: non-bulk inserts and tuple routing

2018-02-22 Thread Amit Langote
On 2018/02/23 1:53, Alvaro Herrera wrote: > Robert Haas wrote: >> On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote >> wrote: >>>> Attached is an updated version for that. >>> >>> Thanks for updating the patch. >> >> Committed with

Re: non-bulk inserts and tuple routing

2018-02-22 Thread Amit Langote
On 2018/02/23 1:10, Robert Haas wrote: > On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote > wrote: >>> Attached is an updated version for that. >> >> Thanks for updating the patch. > > Committed with a few changes. The big one was that I got rid of

Re: [HACKERS] path toward faster partition pruning

2018-02-22 Thread Amit Langote
On 2018/02/22 20:28, David Rowley wrote: > On 22 February 2018 at 22:48, Amit Langote > wrote: >>> I'm having to add some NULL handling there for the run-time pruning >>> patch but wondered if it was also required for your patch. >> >> Hmm, not sure w

Re: FOR EACH ROW triggers on partitioned tables

2018-02-22 Thread Amit Langote
On 2018/02/23 8:52, Alvaro Herrera wrote: > We could mitigate the performance loss to some extent by adding more to > RelationData. For example, a "is_partition" boolean would help: skip > searching pg_inherits for a relation that is not a partition. Unless I'm missing something, doesn't rd_rel->r

Re: [HACKERS] Runtime Partition Pruning

2018-02-22 Thread Amit Langote
Hi David. On 2018/02/23 0:11, David Rowley wrote: > On 23 February 2018 at 01:15, David Rowley > wrote: >> One problem that I'm facing now is down to the way I'm gathering the >> ParamIds that match the partkeys. As you'll see from the patch I've >> added a 'paramids' field to PartitionPruneCont

Re: FOR EACH ROW triggers on partitioned tables

2018-02-22 Thread Amit Langote
On Fri, Feb 23, 2018 at 11:32 AM, Alvaro Herrera wrote: > Amit Langote wrote: >> On 2018/02/23 8:52, Alvaro Herrera wrote: >> > We could mitigate the performance loss to some extent by adding more to >> > RelationData. For example, a "is_partition" bo

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-02-22 Thread Amit Langote
Fujita-san, On Thu, Feb 22, 2018 at 8:49 PM, Etsuro Fujita wrote: > (2018/02/22 11:52), Amit Langote wrote: >> I wonder why partition_index needs to be made part of this API? > > The reason for that is because I think the FDW might want to look at the > partition info

Re: [HACKERS] Runtime Partition Pruning

2018-02-25 Thread Amit Langote
(Sorry, I had mistakenly replied only to Simon on Friday) On Fri, Feb 23, 2018 at 9:04 PM, Simon Riggs wrote: > On 23 February 2018 at 11:40, David Rowley > wrote: >> On 23 February 2018 at 04:11, Jesper Pedersen >> wrote: >>> Are UPDATE and DELETE suppose to be supported ? >> >> To be honest,

Re: [HACKERS] Runtime Partition Pruning

2018-02-25 Thread Amit Langote
On 2018/02/23 20:40, David Rowley wrote: > On 23 February 2018 at 04:11, Jesper Pedersen > wrote: >> Are UPDATE and DELETE suppose to be supported ? > > To be honest, I had not even considered those. Without looking in > detail I imagine it may be possible to allow this simply by setting > the Ap

Re: [HACKERS] path toward faster partition pruning

2018-02-26 Thread Amit Langote
On 2018/02/27 3:27, Robert Haas wrote: > On Sun, Feb 25, 2018 at 11:10 PM, Amit Langote > wrote: >> I think I'm convinced that partopcintype OIDs can be used where I thought >> parttypid ones were necessary. The pruning patch uses the respective OID >> from this ar

Re: [HACKERS] path toward faster partition pruning

2018-02-27 Thread Amit Langote
On 2018/02/28 1:05, Robert Haas wrote: > On Mon, Feb 26, 2018 at 10:59 PM, Amit Langote > wrote: >> You may say that partition bounds might have to be different too in this >> case and hence partition-wise join won't occur anyway, but I'm wondering >> if the mism

inserts into partitioned table may cause crash

2018-02-28 Thread Amit Langote
I've run into what seems to be a bug in ExecInsert() that causes a crash when inserting multiple rows into a partitioned table that each go into different partitions with different tuple descriptors. Crash occurs if ExecInsert() returns without resetting estate->es_result_relation_info back to the

Re: inserts into partitioned table may cause crash

2018-02-28 Thread Amit Langote
On 2018/02/28 17:36, Amit Langote wrote: > I've run into what seems to be a bug in ExecInsert() that causes a crash > when inserting multiple rows into a partitioned table that each go into > different partitions with different tuple descriptors. Crash occurs if > ExecInsert(

Re: ON CONFLICT DO UPDATE for partitioned tables

2018-02-28 Thread Amit Langote
On 2018/02/28 9:46, Alvaro Herrera wrote: > I updated Amit Langote's patch for INSERT ON CONFLICT DO UPDATE[1]. > Following the lead of edd44738bc88 ("Be lazier about partition tuple > routing.") this incarnation only does the necessary push-ups for the > specific partition that needs it, at execut

Re: ON CONFLICT DO UPDATE for partitioned tables

2018-02-28 Thread Amit Langote
On 2018/03/01 1:03, Robert Haas wrote: > On Tue, Feb 27, 2018 at 7:46 PM, Alvaro Herrera > wrote: >> I updated Amit Langote's patch for INSERT ON CONFLICT DO UPDATE[1]. >> Following the lead of edd44738bc88 ("Be lazier about partition tuple >> routing.") this incarnation only does the necessary pu

Re: [HACKERS] path toward faster partition pruning

2018-02-28 Thread Amit Langote
On 2018/02/28 19:14, Ashutosh Bapat wrote: > On Wed, Feb 28, 2018 at 6:42 AM, Amit Langote wrote: >> BTW, should there be a relevant test in partition_join.sql? If yes, >> attached a patch (partitionwise-join-collation-test-1.patch) to add one. > > A partition-wise join pat

Re: [HACKERS] path toward faster partition pruning

2018-02-28 Thread Amit Langote
On 2018/03/01 2:23, Robert Haas wrote: > On Tue, Feb 27, 2018 at 8:12 PM, Amit Langote > wrote: >> Ah, OK. I was missing that there is no need to have both parttypcoll and >> partcollation in PartitionSchemeData, as the Vars in rel->partexprs are >> built fr

Re: no partition pruning when partitioning using array type

2018-03-01 Thread Amit Langote
On 2018/02/02 0:20, Robert Haas wrote: > On Thu, Feb 1, 2018 at 4:42 AM, Amit Langote > wrote: >>> I hope someone else chimes in as well. :) >> >> Bug #15042 [1] seems to be caused by this same problem. There, a >> RelabelType node is being slapped (by the parti

Re: 2018-03 Commitfest Summary (Andres #3)

2018-03-01 Thread Amit Langote
Hi Andres. On 2018/03/02 13:34, Andres Freund wrote: > - reorganize partitioning code > > NR. Created recently, but split off an older patch. > > Seems like a generally reasonable idea. Wonder if it conflicts with > some other partition related patches? It actually does. There are at lea

Re: [HACKERS] path toward faster partition pruning

2018-03-01 Thread Amit Langote
On 2018/03/01 21:56, Robert Haas wrote: > On Tue, Feb 27, 2018 at 4:33 AM, Amit Langote > wrote: >> Attached an updated version in which I incorporated some of the revisions >> that David Rowley suggested to OR clauses handling (in partprune.c) that >> he posted as a s

Re: pg_dump outputs invalid syntax for partitions with bool partkeys in pg10

2018-03-01 Thread Amit Langote
Hi David. On 2018/03/02 12:41, David Rowley wrote: > Quite simply: > > d1=# create table bp (b bool) partition by list(b); > CREATE TABLE > d1=# create table bp_f partition of bp for values in('f'); > CREATE TABLE > d1=# \q > $ createdb d2 > $ pg_dump d1 | psql d2 > > ... > > ERROR: syntax err

Re: Boolean partitions syntax

2018-03-01 Thread Amit Langote
On 2018/03/02 15:58, Andres Freund wrote: > On 2018-02-02 17:00:24 -0500, Tom Lane wrote: >> Peter Eisentraut writes: >>> There might be other options, but one way to solve this would be to >>> treat partition bounds as a general expression in the grammar and then >>> check in post-parse analysis

Re: Boolean partitions syntax

2018-03-01 Thread Amit Langote
Horiguchi-san, On 2018/02/05 18:17, Kyotaro HORIGUCHI wrote: > At Mon, 29 Jan 2018 13:21:54 +0900, Amit Langote wrote: >> Partition bound literals as captured gram.y don't have any type >> information attached. They're carried over in a A_Const to >> transformParti

Re: [HACKERS] path toward faster partition pruning

2018-03-02 Thread Amit Langote
Thanks for your comments. On 2018/03/02 4:13, Robert Haas wrote: > On Wed, Feb 28, 2018 at 11:53 PM, Amit Langote > wrote: >> Attached updated patches. > > + memcpy(part_scheme->partsupfunc, partkey->partsupfunc, > + sizeof(FmgrInfo) * partnat

Re: [HACKERS] path toward faster partition pruning

2018-03-02 Thread Amit Langote
On 2018/03/02 11:12, David Rowley wrote: > On 2 March 2018 at 08:13, Robert Haas wrote: >> I don't like the comments at the top of partprune.c very much. It >> seems strange to document individual functions here; those functions >> can (and should) be documented in their individual header comment

Re: Re: reorganizing partitioning code

2018-03-02 Thread Amit Langote
Hi David. On Fri, Mar 2, 2018 at 11:53 PM, David Steele wrote: > Hi Amit, > > On 2/16/18 3:36 AM, Amit Langote wrote: >> >> Attached updated version. > > This patch no longer applies and the conflicts do not appear to be trivial. > > I'm a bit confused abo

<    6   7   8   9   10   11   12   13   14   15   >