Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-18 Thread Peter Geoghegan
On Mon, Jul 18, 2016 at 7:56 AM, Robert Haas wrote: > The test case I used previously was an external sort, which does lots > of retail pfrees. Now that we've mostly abandoned replacement > selection, there will be many fewer pfrees while building runs, I > think, but

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-18 Thread Robert Haas
On Mon, Jul 18, 2016 at 10:19 AM, Greg Stark wrote: > On Sun, Jul 17, 2016 at 1:55 PM, Robert Haas wrote: >>On Wed, Jul 13, 2016 at 4:39 PM, Tom Lane wrote: >>> I wonder whether we could compromise by reducing the minimum "standard >>>

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-18 Thread Tom Lane
Greg Stark writes: > I wonder if we could go further. If we don't imagine having a very > large number of allocators then we could just ask each one in turn if > this block is one of theirs and which context it came from. That would > allow an allocator that just allocated

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-18 Thread Greg Stark
On Sun, Jul 17, 2016 at 1:55 PM, Robert Haas wrote: > >On Wed, Jul 13, 2016 at 4:39 PM, Tom Lane wrote: >> >> I wonder whether we could compromise by reducing the minimum "standard >> chunk header" to be just a pointer to owning context, with the other

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-17 Thread Robert Haas
On Wed, Jul 13, 2016 at 4:39 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Jul 13, 2016 at 1:10 PM, Tomas Vondra >> wrote: >> What's not clear to me is to what extent slowing down pfree is an >> acceptable price for

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-13 Thread Tom Lane
Andres Freund writes: > On 2016-07-13 16:39:58 -0400, Tom Lane wrote: >> I think there's a lot, but I'm afraid most of them are in contexts >> (pun intended) where aset.c already works pretty well, ie it's a >> short-lived context anyway. > FWIW, hacking up the aset/mcxt.c to

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-13 Thread Andres Freund
On 2016-07-13 16:39:58 -0400, Tom Lane wrote: > Robert Haas writes: > > On Wed, Jul 13, 2016 at 1:10 PM, Tomas Vondra > > wrote: > > What's not clear to me is to what extent slowing down pfree is an > > acceptable price for improving the

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-13 Thread Tom Lane
Robert Haas writes: > On Wed, Jul 13, 2016 at 1:10 PM, Tomas Vondra > wrote: > What's not clear to me is to what extent slowing down pfree is an > acceptable price for improving the behavior in other ways. I wonder > how many of the pfree

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-13 Thread Robert Haas
On Wed, Jul 13, 2016 at 1:10 PM, Tomas Vondra wrote: > I think the MemoryContext API may not be right abstraction for this. +1. The MemoryContext API is little short of an absolute bar to implementing an allocator that works significantly differently than aset.c,

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-13 Thread Tomas Vondra
On 07/13/2016 07:37 PM, Tom Lane wrote: Peter Geoghegan writes: On Wed, Jul 13, 2016 at 7:53 AM, Tomas Vondra wrote: In the thread [1] dealing with hashjoin bug introduced in 9.5, Tom voiced his dislike of dense_alloc. I kinda agree with him

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-13 Thread Tom Lane
Andres Freund writes: > On 2016-07-13 13:37:55 -0400, Tom Lane wrote: >> We already know that >> that code has performance issues, cf bug #14231, so I suspect there's >> a redesign in its future anyway. > Note that it's not the slab allocators that is having problems, it's >

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-13 Thread Andres Freund
On 2016-07-13 13:37:55 -0400, Tom Lane wrote: > I wonder though if we don't already have another similar use-case in > the ad-hoc "slab allocators" in reorderbuffer.c. That seems to call more for a general slab allocator design, than for something like here. After all, there's plenty of freeing

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-13 Thread Tom Lane
Peter Geoghegan writes: > On Wed, Jul 13, 2016 at 7:53 AM, Tomas Vondra > wrote: >> In the thread [1] dealing with hashjoin bug introduced in 9.5, Tom voiced >> his dislike of dense_alloc. I kinda agree with him that introducing "local >>

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-13 Thread Peter Geoghegan
On Wed, Jul 13, 2016 at 7:53 AM, Tomas Vondra wrote: > In the thread [1] dealing with hashjoin bug introduced in 9.5, Tom voiced > his dislike of dense_alloc. I kinda agree with him that introducing "local > allocators" may not be the best idea, and as dense_alloc

[HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-13 Thread Tomas Vondra
Hi, In the thread [1] dealing with hashjoin bug introduced in 9.5, Tom voiced his dislike of dense_alloc. I kinda agree with him that introducing "local allocators" may not be the best idea, and as dense_alloc was introduced by me I was playing with the idea to wrap this into a regular