Re: plan shape work

2025-09-17 Thread Richard Guo
On Sat, Sep 6, 2025 at 1:00 AM Tom Lane wrote: > Robert Haas writes: > > I was not aware of outer_join_rels, so thank you for pointing it out. > > However, consider this query: > > > select 1 from pg_class a inner join pg_class b on a.relfilenode = > > b.relfilenode; > > > Here, we end up with a

Re: plan shape work

2025-09-17 Thread Robert Haas
On Fri, Sep 12, 2025 at 1:44 PM Tom Lane wrote: > Robert Haas writes: > > But, I also can't commit either v4-0003 or v5-0003 or any variant > > thereof until we agree on what to do about 0001, and you're the > > holdout there. > > Yeah, I owe you a review, hope to get to it over the weekend. Tha

Re: plan shape work

2025-09-17 Thread Robert Haas
On Fri, Sep 12, 2025 at 12:18 PM Tom Lane wrote: > Not the RelOptInfo, but the Path. I have not looked at details, but > it might be necessary to identify these labels at Path construction > time rather than reconstructing them during createplan.c. I'd rather > not, because bloating Path nodes w

Re: plan shape work

2025-09-16 Thread Robert Haas
CI is unhappy with v6 because: [14:37:37.742] In function ‘show_result_replacement_info’, [14:37:37.742] inlined from ‘ExplainNode’ at explain.c:2240:4: [14:37:37.742] explain.c:4849:33: error: ‘replacement_type’ may be used uninitialized [-Werror=maybe-uninitialized] [14:37:37.742] 4849 |

Re: plan shape work

2025-09-16 Thread Robert Haas
On Mon, Sep 15, 2025 at 3:43 PM Tom Lane wrote: > anything to the RTE's eref. While that makes your argument > nominally true, I'd be inclined to argue that this was an oversight > and it should have changed the alias/eref fields to look like other > RTE_RESULTs. (I've not investigated, but I wo

Re: plan shape work

2025-09-15 Thread Tom Lane
Robert Haas writes: > Well, I was just bringing it up because you seemed to think that the > Replaces: line was not necessarily where we would have put it in a > green field. I am not sure about where we would have put it, but I > think in a green field we wouldn't have it at all, and I actually >

Re: plan shape work

2025-09-15 Thread Robert Haas
On Mon, Sep 15, 2025 at 3:43 PM Tom Lane wrote: > > Yes, I wondered if it should actually look more like this: > > Degenerate Scan on blah > > Degenerate Join on blah, blah, blah > > Degenerate Aggregate > > MinMaxAggregate Result > > So getting rid of Replaces: altogether. > > Yeah, that would be

Re: plan shape work

2025-09-15 Thread Tom Lane
Robert Haas writes: > On Sun, Sep 14, 2025 at 7:42 PM Tom Lane wrote: >> ... But I'd really argue that from a user's standpoint this >> information is part of the fundamental plan structure and so it >> deserves more prominence. I'd lean to putting it first in the >> T_Result case, before the "O

Re: plan shape work

2025-09-15 Thread Robert Haas
Thanks for the review. Comments to which I don't respond below are duly noted. On Sun, Sep 14, 2025 at 7:42 PM Tom Lane wrote: > * I think your placement of the show_result_replacement_info call > site suffers from add-at-the-end syndrome. It should certainly go > before the show_instrumentation

Re: plan shape work

2025-09-14 Thread Tom Lane
I wrote: > Robert Haas writes: >> But, I also can't commit either v4-0003 or v5-0003 or any variant >> thereof until we agree on what to do about 0001, and you're the >> holdout there. > Yeah, I owe you a review, hope to get to it over the weekend. I'm running out of weekend, but I found time to

Re: plan shape work

2025-09-13 Thread Tom Lane
Robert Haas writes: > But, I also can't commit either v4-0003 or v5-0003 or any variant > thereof until we agree on what to do about 0001, and you're the > holdout there. Yeah, I owe you a review, hope to get to it over the weekend. regards, tom lane

Re: plan shape work

2025-09-13 Thread Richard Guo
On Sat, Sep 13, 2025 at 2:32 AM Robert Haas wrote: > We sort of got started down this path because, reviewing v4-0003, > Richard commented that I might be able to sanity-check > something-or-other about RTE_JOIN RTIs instead of just focusing on > baserels. From there, this sub-thread has turned in

Re: plan shape work

2025-09-13 Thread Richard Guo
On Sat, Sep 13, 2025 at 12:08 AM Tom Lane wrote: > After thinking about this for awhile, I believe that Richard and I > each had half of the right solution ;-). Let me propose some new > terminology in hopes of clarifying matters: > > * A join plan node "starts" an outer join if it performs the >

Re: plan shape work

2025-09-12 Thread Tom Lane
Robert Haas writes: > So, it looks to me like the way this works today is that > join_is_legal() figures out the relevant SpecialJoinInfo and then > add_outer_joins_to_relids() decides what to add to the joinrel's relid > set. So I think, though I am not quite sure, that if somewhere around > that

Re: plan shape work

2025-09-12 Thread Robert Haas
On Fri, Sep 12, 2025 at 11:08 AM Tom Lane wrote: > I was arguing for labeling plan nodes according to which OJ they > start (always a unique relid). Richard was arguing for labeling > according to which OJ(s) they complete (zero, one, or more relids). I agree with everything in your reply up to

Re: plan shape work

2025-09-11 Thread Robert Haas
On Tue, Sep 9, 2025 at 4:37 PM Tom Lane wrote: > Maybe. I kinda feel that actually redesigning plan trees is outside > the scope of this project, but maybe we should consider it. Yeah, I would rather not go there, all things being equal. We could also consider displaying things differently even

Re: plan shape work

2025-09-11 Thread Robert Haas
On Thu, Sep 11, 2025 at 2:19 PM Tom Lane wrote: > I do not think this is correct, or at least it's not the most useful > way to think about it. As I stated earlier, each join plan node that > is doing a non-inner join has a unique corresponding outer-join RTE > that describes what it's doing. Th

Re: plan shape work

2025-09-11 Thread Robert Haas
Here's a likely-doomed new version of just the first three patches. 0002 is unchanged. 0001 has been reworked so that a Result node contains a result_type. This is as per Tom's suggestion, but it's unclear to me that he will like the details. 0003 has been reworked so that when we build a Join plan

Re: plan shape work

2025-09-11 Thread Tom Lane
Richard Guo writes: > As you can see, the set of outer joins calculated at the same join can > vary depending on the join order. What I suggested is to record this > information in JoinPaths (or maybe also in Join plan nodes so that > get_scanned_rtindexes can collect it) for the assertion. I do

Re: plan shape work

2025-09-10 Thread Richard Guo
On Tue, Sep 9, 2025 at 10:18 PM Robert Haas wrote: > On Mon, Sep 8, 2025 at 10:22 PM Richard Guo wrote: > > One idea (not fully thought through) is that we record the calculated > > outerjoin_relids for each outer join in its JoinPaths. (We cannot > > store this in the joinrel's RelOptInfo becau

Re: plan shape work

2025-09-09 Thread Tom Lane
Robert Haas writes: > Just a random thought, but another idea that crossed my mind here at > one point was to actually split the Result node up into Result nodes > with subplans and Result nodes without subplans. We could call the > version with a subplan "Project" and the version without a subpla

Re: plan shape work

2025-09-09 Thread Robert Haas
On Tue, Sep 9, 2025 at 12:00 PM Robert Haas wrote: > On Tue, Sep 9, 2025 at 11:12 AM Tom Lane wrote: > > I think what would make a ton more sense is to add > > an enum field to Result that explicitly identifies why it's there. > > We've got at least "apply one-time filter to subplan", "apply per-

Re: plan shape work

2025-09-09 Thread Robert Haas
On Tue, Sep 9, 2025 at 11:12 AM Tom Lane wrote: > I think what would make a ton more sense is to add > an enum field to Result that explicitly identifies why it's there. > We've got at least "apply one-time filter to subplan", "apply per-row > gating filter to subplan", "represent a relation prove

Re: plan shape work

2025-09-09 Thread Tom Lane
Robert Haas writes: > First of all, as an administrative note, since both you and Alex seem > to like 0001 and 0002 and no suggestions for improvement have been > offered, I plan to commit those soon unless there are objections or > additional review comments. FWIW, I don't love the details of 00

Re: plan shape work

2025-09-09 Thread Robert Haas
First of all, as an administrative note, since both you and Alex seem to like 0001 and 0002 and no suggestions for improvement have been offered, I plan to commit those soon unless there are objections or additional review comments. I will likely do the same for 0003 as well, pending the results of

Re: plan shape work

2025-09-08 Thread Richard Guo
On Mon, Sep 8, 2025 at 10:56 PM Robert Haas wrote: > On Mon, Sep 8, 2025 at 5:51 AM Richard Guo wrote: > > BTW, I'm wondering if we can take outer join relids into account in > > assert_join_preserves_scan_rtis(), which could make the check more > > useful. A joinrel's relids consists of three p

Re: plan shape work

2025-09-08 Thread Robert Haas
On Mon, Sep 8, 2025 at 5:51 AM Richard Guo wrote: > > Plain (not-outer) joins will never be included in a relid set in the > > first place. > > Exactly. Non-outer joins wouldn't cause Vars to become null, so we > never include them in the joinrel's relids. OK. I didn't understand what the rules

Re: plan shape work

2025-09-07 Thread Dilip Kumar
On Wed, May 21, 2025 at 7:29 PM Robert Haas wrote: > > On Tue, May 20, 2025 at 2:45 PM Tomas Vondra wrote: > I have a sense - possibly an incorrect one - that the core of the > problem here is that the planner considers lots of very similar > alternatives. A hypothetical feature that showed the

Re: plan shape work

2025-09-07 Thread Robert Haas
On Thu, Sep 4, 2025 at 4:21 AM Richard Guo wrote: > I found some issues with 0003 though. It seems get_scanned_rtindexes > is intended to return RTI sets with outer join relids excluded. For > some node types, such as Append and MergeAppend, it fails to do so, > which can cause the assertion in

Re: plan shape work

2025-09-06 Thread Robert Haas
On Fri, Sep 5, 2025 at 12:00 PM Tom Lane wrote: > Plain (not-outer) joins will never be included in a relid set in the > first place. Ah, well then Richard's idea might work! Let me try it and see what happens... Thanks, -- Robert Haas EDB: http://www.enterprisedb.com

Re: plan shape work

2025-09-05 Thread Tom Lane
Robert Haas writes: > I was not aware of outer_join_rels, so thank you for pointing it out. > However, consider this query: > select 1 from pg_class a inner join pg_class b on a.relfilenode = > b.relfilenode; > Here, we end up with a three-item range table: one for a, one for b, > and one for t

Re: plan shape work

2025-09-04 Thread Richard Guo
On Wed, Sep 3, 2025 at 5:07 AM Robert Haas wrote: > Thanks for the review. Responding just briefly to avoid quoting too much text: > > - I'm glad to hear that you like 0002 and consider it an improvement > independent of what follows. > - I'm glad to hear that you're OK with the current split betw

Re: plan shape work

2025-08-28 Thread Alexandra Wang
Hi Robert, On Tue, Aug 26, 2025 at 7:59 AM Robert Haas wrote: > On Mon, May 19, 2025 at 2:01 PM Robert Haas wrote: > > A couple of people at pgconf.dev seemed to want to know more about my > > ongoing plan shape work, so here are the patches I have currently. > > Here's an updated patch set. My

Re: plan shape work

2025-08-27 Thread Andrei Lepikhov
On 19/5/2025 20:01, Robert Haas wrote: Hope you find this interesting. If you do, let me know what you think.Thanks for looking in this direction! Since 2017, we designed features that should 'memorise' the experience of previous executions, checking the PlanState tree and instrumentation af

Re: plan shape work

2025-08-26 Thread Bruce Momjian
On Tue, Aug 26, 2025 at 10:58:33AM -0400, Robert Haas wrote: > During planning, there is one range table per subquery; at the end if > planning, those separate range tables are flattened into a single > range table. Prior to this change, it was impractical for code > examining the final plan to und

Re: plan shape work

2025-05-22 Thread Andy Fan
Andy Fan writes: > >> This list of elided nodes is stored in the PlannedStmt > > I did a quick check on the attached patches and I can see some more > information is added into PlannedStmt. then my question are the > PlannedStmt is not avaiable during the future planning cycle, then how > does th

Re: plan shape work

2025-05-21 Thread Andy Fan
Robert Haas writes: > Hi, > > ... As a recap, my overall goal here > is to make it so that you can examine a finished plan, figure out what > decisions the planner made, and then somehow get the planner to make > those same decisions over again in a future planning cycle. I am feeling that this

Re: plan shape work

2025-05-21 Thread Maciek Sakrejda
On Wed, May 21, 2025 at 7:29 AM Robert Haas wrote: > > On Tue, May 20, 2025 at 3:09 PM Maciek Sakrejda wrote: > > +1, this seems like it could be very useful. A somewhat related issue > > is being able to tie plan nodes back to the query text: it can be hard > > to understand the planner's decisi

Re: plan shape work

2025-05-21 Thread Robert Haas
On Wed, May 21, 2025 at 12:03 PM Maciek Sakrejda wrote: > That may be due to your extensive experience with Postgres and EXPLAIN plans. Yes, that is very possible. All things being equal, it helps to have done something a lot of times. > Fair enough, although the people trying to make sense of E

Re: plan shape work

2025-05-21 Thread Robert Haas
On Tue, May 20, 2025 at 3:09 PM Maciek Sakrejda wrote: > +1, this seems like it could be very useful. A somewhat related issue > is being able to tie plan nodes back to the query text: it can be hard > to understand the planner's decisions if it's not even clear what part > of the query it's makin

Re: plan shape work

2025-05-21 Thread Robert Haas
On Tue, May 20, 2025 at 2:45 PM Tomas Vondra wrote: > Thanks for the overview. I don't have any immediate feedback, but it > sounds like it might be related to the "making planner decisions clear" > session from the unconference ... > > The basic premise of that session was about how to give users

Re: plan shape work

2025-05-20 Thread Maciek Sakrejda
+1, this seems like it could be very useful. A somewhat related issue is being able to tie plan nodes back to the query text: it can be hard to understand the planner's decisions if it's not even clear what part of the query it's making decisions about. I'm sure this is not an easy problem in gener

Re: plan shape work

2025-05-20 Thread Tomas Vondra
On 5/19/25 20:01, Robert Haas wrote: > Hi, > > A couple of people at pgconf.dev seemed to want to know more about my > ongoing plan shape work, so here are the patches I have currently. > This is a long way from something that actually looks like a usable > feature, but these are bits of infrastru