Use generation memory context for tuplestore.c

2024-05-03 Thread David Rowley
(40af10b57 did this for tuplesort.c, this is the same, but for tuplestore.c) I was looking at the tuplestore.c code a few days ago and noticed that it allocates tuples in the memory context that tuplestore_begin_heap() is called in, which for nodeMaterial.c, is ExecutorState. I didn't think this

Re: Use generation memory context for tuplestore.c

2024-05-03 Thread Matthias van de Meent
On Fri, 3 May 2024 at 15:55, David Rowley wrote: > > (40af10b57 did this for tuplesort.c, this is the same, but for tuplestore.c) > > I was looking at the tuplestore.c code a few days ago and noticed that > it allocates tuples in the memory context that tuplestore_begin_heap() > is called in, whic

Re: Use generation memory context for tuplestore.c

2024-05-03 Thread David Rowley
On Sat, 4 May 2024 at 03:51, Matthias van de Meent wrote: > Was a bump context considered? If so, why didn't it make the cut? > If tuplestore_trim is the only reason why the type of context in patch > 2 is a generation context, then couldn't we make the type of context > conditional on state->efla

Re: Use generation memory context for tuplestore.c

2024-05-10 Thread Dmitry Dolgov
> On Sat, May 04, 2024 at 01:55:22AM +1200, David Rowley wrote: > (40af10b57 did this for tuplesort.c, this is the same, but for tuplestore.c) An interesting idea, thanks. I was able to reproduce the results of your benchmark and get similar conclusions from the results. > Using generation has th

Re: Use generation memory context for tuplestore.c

2024-05-10 Thread Matthias van de Meent
On Sat, 4 May 2024 at 04:02, David Rowley wrote: > > On Sat, 4 May 2024 at 03:51, Matthias van de Meent > wrote: > > Was a bump context considered? If so, why didn't it make the cut? > > If tuplestore_trim is the only reason why the type of context in patch > > 2 is a generation context, then cou

Re: Use generation memory context for tuplestore.c

2024-05-10 Thread David Rowley
Thanks for having a look at this. On Sat, 11 May 2024 at 04:34, Dmitry Dolgov <9erthali...@gmail.com> wrote: > Do I understand correctly, that the efficiency of generation memory > context could be measured directly via counting number of malloc/free > calls? In those experiments I've conducted th

Re: Use generation memory context for tuplestore.c

2024-05-30 Thread David Rowley
On Sat, 4 May 2024 at 03:51, Matthias van de Meent wrote: > > On Fri, 3 May 2024 at 15:55, David Rowley wrote: > > master @ 8f0a97dff > > Storage: Memory Maximum Storage: 16577kB > > > > patched: > > Storage: Memory Maximum Storage: 8577kB > > Those are some impressive numbers. This patch need

Re: Use generation memory context for tuplestore.c

2024-07-02 Thread Matthias van de Meent
On Fri, 31 May 2024 at 05:26, David Rowley wrote: > > On Sat, 4 May 2024 at 03:51, Matthias van de Meent > wrote: > > > > On Fri, 3 May 2024 at 15:55, David Rowley wrote: > > > master @ 8f0a97dff > > > Storage: Memory Maximum Storage: 16577kB > > > > > > patched: > > > Storage: Memory Maximum

Re: Use generation memory context for tuplestore.c

2024-07-03 Thread David Rowley
Thank you for having another look at this. On Wed, 3 Jul 2024 at 00:20, Matthias van de Meent wrote: > I noticed this change to buffile.c shows up in both v1 and v2 of the > patchset, but I can't trace the reasoning why it changed with this > patch (rather than a separate bugfix): I didn't expl

Re: Use generation memory context for tuplestore.c

2024-07-04 Thread Alexander Lakhin
Hello David, 03.07.2024 13:41, David Rowley wrote: Lastly, I think this would benefit from a test in regress/sql/explain.sql, as the test changes that were included removed the only occurrance of a Materialize node from the regression tests' EXPLAIN outputs. I've modified the tests where the

Re: Use generation memory context for tuplestore.c

2024-07-04 Thread David Rowley
On Fri, 5 Jul 2024 at 16:00, Alexander Lakhin wrote: > Please look at a segfault crash introduced with 1eff8279d: > CREATE TABLE t1(i int); > CREATE TABLE t2(i int) PARTITION BY RANGE (i); > CREATE TABLE t2p PARTITION OF t2 FOR VALUES FROM (1) TO (2); > > EXPLAIN ANALYZE SELECT * FROM t1 JOIN t2 O

Re: Use generation memory context for tuplestore.c

2024-07-04 Thread David Rowley
On Wed, 3 Jul 2024 at 22:41, David Rowley wrote: > > On Wed, 3 Jul 2024 at 00:20, Matthias van de Meent > > Lastly, I think this would benefit from a test in > > regress/sql/explain.sql, as the test changes that were included > > removed the only occurrance of a Materialize node from the regressio

Re: Use generation memory context for tuplestore.c

2024-07-05 Thread Alexander Lakhin
05.07.2024 07:57, David Rowley wrote: Thanks for the report. I've just pushed a fix in 53abb1e0e. Thank you, David! Please look at another anomaly introduced with 590b045c3: echo " CREATE TABLE t(f int, t int); INSERT INTO t VALUES (1, 1); WITH RECURSIVE sg(f, t) AS ( SELECT * FROM t t1 UNION

Re: Use generation memory context for tuplestore.c

2024-07-05 Thread David Rowley
On Sat, 6 Jul 2024 at 02:00, Alexander Lakhin wrote: > CREATE TABLE t(f int, t int); > INSERT INTO t VALUES (1, 1); > > WITH RECURSIVE sg(f, t) AS ( > SELECT * FROM t t1 > UNION ALL > SELECT t2.* FROM t t2, sg WHERE t2.f = sg.t > ) SEARCH DEPTH FIRST BY f, t SET seq > SELECT * FROM sg; > " | timeo

Re: Use generation memory context for tuplestore.c

2024-07-05 Thread David Rowley
On Sat, 6 Jul 2024 at 12:08, David Rowley wrote: > The attached fixes the issue. I'll stare at it a bit more and try to > decide if that's the best way to fix it. I did that staring work and thought about arranging the code to reduce the number of #ifdefs. In the end, I decided it was better to k

Re: Use generation memory context for tuplestore.c

2024-07-05 Thread David Rowley
On Sat, 6 Jul 2024 at 12:08, David Rowley wrote: > I also had not intended that the buffile.c stuff would use the > generation context. I'll need to fix that too, but I think I'll fix > the GenerationRealloc() first. I've pushed a fix for that now too. David