Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-06 Thread John Papandriopoulos
On 12/6/10 10:03 AM, Tom Lane wrote: > John Papandriopoulos writes: >> I am still wondering whether the inheritance_planner(...) can be avoided if >> the rowtypes of children are the same as the parent? > > Possibly, but it's far from a trivial change. The difficul

Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-05 Thread John Papandriopoulos
On 12/5/10 12:14 PM, Tom Lane wrote: > I wrote: >> You could get rid of the memory growth, at the cost of a lot of >> tree-copying, by doing each child plan step in a discardable memory >> context. I'm not sure that'd be a win for normal sizes of inheritance >> trees though --- you'd need to copy

Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-05 Thread John Papandriopoulos
On 12/4/10 3:19 PM, Mladen Gogala wrote: > Tom Lane wrote: >> Partitioning using these techniques will work well with up to >> perhaps a hundred partitions; don't try to use many thousands of >> partitions. >> regards, tom lane > Hmmm, what happens if I need 10 years of data, in monthly partitions?

Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-05 Thread John Papandriopoulos
On 12/4/10 2:40 PM, Tom Lane wrote: > [ pokes at that for a bit ... ] Ah, I had forgotten that UPDATE/DELETE > go through inheritance_planner() while SELECT doesn't. And > inheritance_planner() makes a copy of the querytree, including the > already-expanded range table, for each target relation.

Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-05 Thread John Papandriopoulos
On 12/4/10 3:38 PM, Jochen Erwied wrote: > Sunday, December 5, 2010, 12:19:29 AM you wrote: > >> Hmmm, what happens if I need 10 years of data, in monthly partitions? It >> would be 120 partitions. Can you please elaborate on that limitation? >> Any plans on lifting that restriction? > > I'm runn

Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-04 Thread John Papandriopoulos
On 12/4/10 8:42 AM, Tom Lane wrote: John Papandriopoulos writes: I've recreated the same example with just one parent table, and 4096 child tables. SELECT query planning is lightning fast as before; DELETE and UPDATE cause my machine to swap. What's different about DELETE

Re: [PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-04 Thread John Papandriopoulos
On 12/3/10 10:20 PM, Tom Lane wrote: > John Papandriopoulos writes: >> I've found that a k-ary table inheritance tree works quite well to >> reduce the O(n) CHECK constraint overhead [1] in the query planner >> when enabling partition constraint exclusion. > >

[PERFORM] Query-plan for partitioned UPDATE/DELETE slow and swaps vmem compared to SELECT

2010-12-03 Thread John Papandriopoulos
Hi everyone, I've been trialling different inheritance schemes for partitioning to a large number of tables. I am looking at ~1e9 records, totaling ~200GB. I've found that a k-ary table inheritance tree works quite well to reduce the O(n) CHECK constraint overhead [1] in the query planner when