Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Ashutosh Bapat
On Tue, Feb 20, 2024 at 8:19 AM Andrei Lepikhov wrote: > > On 19/2/2024 19:25, Ashutosh Bapat wrote: > > On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov > > wrote: > >> Live example: right now, I am working on the code like MSSQL has - a > >> combination of NestLoop and HashJoin paths and

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Andrei Lepikhov
On 19/2/2024 19:25, Ashutosh Bapat wrote: On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov wrote: Live example: right now, I am working on the code like MSSQL has - a combination of NestLoop and HashJoin paths and switching between them in real-time. It requires both paths in the path list at

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Ashutosh Bapat
On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov wrote: > Live example: right now, I am working on the code like MSSQL has - a > combination of NestLoop and HashJoin paths and switching between them in > real-time. It requires both paths in the path list at the moment when > extensions are coming.

Re: Memory consumed by paths during partitionwise join planning

2024-02-15 Thread Andrei Lepikhov
On 15/2/2024 19:06, Ashutosh Bapat wrote: On Thu, Feb 15, 2024 at 9:41 AM Andrei Lepikhov But I'm not sure about freeing unreferenced paths. I would have to see alternatives in the pathlist. I didn't understand this. Can you please elaborate? A path in any pathlist is referenced. An

Re: Memory consumed by paths during partitionwise join planning

2024-02-15 Thread Ashutosh Bapat
On Thu, Feb 15, 2024 at 9:41 AM Andrei Lepikhov wrote: > > On 6/2/2024 19:51, Ashutosh Bapat wrote: > > On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat > > The patches are raw. make check has some crashes that I need to fix. I > > am waiting to hear whether this is useful and whether the design is

Re: Memory consumed by paths during partitionwise join planning

2024-02-14 Thread Andrei Lepikhov
On 6/2/2024 19:51, Ashutosh Bapat wrote: On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat The patches are raw. make check has some crashes that I need to fix. I am waiting to hear whether this is useful and whether the design is on the right track. Let me write words of opinion on that feature. I

Re: Memory consumed by paths during partitionwise join planning

2024-02-06 Thread Ashutosh Bapat
On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat wrote: > > > > That looks pretty small considering the benefits. What do you think? > > > > [1] > > https://www.postgresql.org/message-id/caexhw5stmouobe55pmt83r8uxvfcph+pvo5dnpdrvcsbgxe...@mail.gmail.com > > If you want to experiment, please use

Re: Memory consumed by paths during partitionwise join planning

2023-12-14 Thread Ashutosh Bapat
Forgot to mention, On Thu, Dec 14, 2023 at 5:34 PM Ashutosh Bapat wrote: > > On Thu, Dec 7, 2023 at 6:19 PM David Rowley wrote: > > > > Maybe we can try to move forward with your refcount idea and see how > > the performance looks. If that's intolerable then that might help us > > decide on

Re: Memory consumed by paths during partitionwise join planning

2023-12-14 Thread Ashutosh Bapat
On Thu, Dec 7, 2023 at 6:19 PM David Rowley wrote: > > Maybe we can try to move forward with your refcount idea and see how > the performance looks. If that's intolerable then that might help us > decide on the next best alternative solution. > Here are performance numbers setup create table

Re: Memory consumed by paths during partitionwise join planning

2023-12-08 Thread Ashutosh Bapat
On Fri, Dec 8, 2023 at 1:02 PM David Rowley wrote: > > On Fri, 8 Dec 2023 at 18:02, Ashutosh Bapat > wrote: > > given path. E.g. we have three path chains as follows > > 1. joinpath_1->joinpath_2->seqpath_1, > > 2. joinpath_3->joinpath_4->seqpath_1, > > 3. joinpath_5->joinpath_2->seqpath_1 > > >

Re: Memory consumed by paths during partitionwise join planning

2023-12-07 Thread David Rowley
On Fri, 8 Dec 2023 at 18:02, Ashutosh Bapat wrote: > given path. E.g. we have three path chains as follows > 1. joinpath_1->joinpath_2->seqpath_1, > 2. joinpath_3->joinpath_4->seqpath_1, > 3. joinpath_5->joinpath_2->seqpath_1 > > Please note that this is not full path tree/forest. It is difficult

Re: Memory consumed by paths during partitionwise join planning

2023-12-07 Thread Ashutosh Bapat
On Thu, Dec 7, 2023 at 6:19 PM David Rowley wrote: > > On Fri, 1 Dec 2023 at 19:59, Ashutosh Bapat > wrote: > > I am fine to work on this further if the community thinks it is > > acceptable. It seems from your point of view the worth of this work is > > as follows > > a. memory savings - not

Re: Memory consumed by paths during partitionwise join planning

2023-12-07 Thread David Rowley
On Fri, 1 Dec 2023 at 19:59, Ashutosh Bapat wrote: > I am fine to work on this further if the community thinks it is > acceptable. It seems from your point of view the worth of this work is > as follows > a. memory savings - not worth it > b. getting rid of !IsA(old_path, IndexPath) - worth it >

Re: Memory consumed by paths during partitionwise join planning

2023-11-30 Thread Ashutosh Bapat
On Wed, Nov 29, 2023 at 1:10 AM David Rowley wrote: > > On Fri, 28 Jul 2023 at 02:06, Ashutosh Bapat > wrote: > > Table 1: Join between unpartitioned tables > > Number of tables | without patch | with patch | % reduction | > > being joined ||| | > >

Re: Memory consumed by paths during partitionwise join planning

2023-11-28 Thread David Rowley
On Fri, 28 Jul 2023 at 02:06, Ashutosh Bapat wrote: > Table 1: Join between unpartitioned tables > Number of tables | without patch | with patch | % reduction | > being joined ||| | > -- >

Memory consumed by paths during partitionwise join planning

2023-07-27 Thread Ashutosh Bapat
Hi All, If add_path() and add_partial_path() do not find a new path to be superior to any existing paths, they free the new path. They free an existing path if it is found to be inferior to the new path. But not all paths surviving in a RelOptInfo are used to create paths for relations which use