Re: a path towards replacing GEQO with something better

2021-06-24 Thread John Naylor
On Mon, Jun 14, 2021 at 12:10 PM Tomas Vondra wrote: > >> I haven't read the [Kossmann & Stocker, 2000] paper yet, but the > >> [Neumann, 2018] paper seems to build on it, and it seems to work with > >> much larger subtrees of the join tree than k=5. > > > > Right, in particular it builds on "IDP

Re: a path towards replacing GEQO with something better

2021-06-20 Thread AJG
Hi, I stumbled across this which may be of interest to this topic and GEQO alternative. The main creator/author of Neo and Bao (ML for Query Optimizer) Ryan Marcus (finishing Postdoc and looking for job) recently posted [1] about Bao for distributed systems. But what was interesting was

Re: a path towards replacing GEQO with something better

2021-06-16 Thread John Naylor
On Wed, Jun 16, 2021 at 12:01 PM Robert Haas wrote: > > I feel like these are completely equivalent. Either way, the planner > is going to deduce that all the ".col" columns are equal to each other > via the equivalence class machinery, and then the subsequent planning > will be absolutely identic

Re: a path towards replacing GEQO with something better

2021-06-16 Thread Tom Lane
John Naylor writes: > On Wed, Jun 16, 2021 at 12:01 PM Robert Haas wrote: >> I feel like these are completely equivalent. Either way, the planner >> is going to deduce that all the ".col" columns are equal to each other >> via the equivalence class machinery, and then the subsequent planning >> w

Re: a path towards replacing GEQO with something better

2021-06-16 Thread John Naylor
On Wed, Jun 16, 2021 at 12:01 PM Robert Haas wrote: > > On Tue, Jun 15, 2021 at 2:16 PM John Naylor > wrote: > > > I don't quite understand the difference between the "chain" case and > > > the "star" case. Can you show sample queries for each one? e.g. SELECT > > > ... FROM a_1, a_2, ..., a_n WH

Re: a path towards replacing GEQO with something better

2021-06-16 Thread Robert Haas
On Tue, Jun 15, 2021 at 2:16 PM John Naylor wrote: > > I don't quite understand the difference between the "chain" case and > > the "star" case. Can you show sample queries for each one? e.g. SELECT > > ... FROM a_1, a_2, ..., a_n WHERE ? > > SELECT * > FROMtab1, tab2, tab3, tab4 > WHERE ta

Re: a path towards replacing GEQO with something better

2021-06-15 Thread John Naylor
On Tue, Jun 15, 2021 at 12:15 PM Robert Haas wrote: > > On Wed, Jun 9, 2021 at 9:24 PM John Naylor wrote: > > 3) It actually improves the existing exhaustive search, because the complexity of the join order problem depends on the query shape: a "chain" shape (linear) vs. a "star" shape (as in sta

Re: a path towards replacing GEQO with something better

2021-06-15 Thread Robert Haas
On Tue, Jun 15, 2021 at 1:00 PM Tom Lane wrote: > Still, I take your point that maybe we could ratchet down the cost of > exhaustive search by skimping on this part. Maybe it'd work to skip > bushy so long as we'd found at least one left-deep or right-deep path > for the current rel. Yes, that s

Re: a path towards replacing GEQO with something better

2021-06-15 Thread Tom Lane
Robert Haas writes: > One idea I just ran across in > https://15721.courses.cs.cmu.edu/spring2020/papers/22-costmodels/p204-leis.pdf > is to try to economize by skipping consideration of bushy plans. We > could start doing that when some budget is exceeded, similar to what > you are proposing here

Re: a path towards replacing GEQO with something better

2021-06-15 Thread Robert Haas
On Wed, Jun 9, 2021 at 9:24 PM John Naylor wrote: > 3) It actually improves the existing exhaustive search, because the > complexity of the join order problem depends on the query shape: a "chain" > shape (linear) vs. a "star" shape (as in star schema), for the most common > examples. The size

Re: a path towards replacing GEQO with something better

2021-06-14 Thread Bruce Momjian
On Mon, Jun 14, 2021 at 06:10:28PM +0200, Tomas Vondra wrote: > On 6/14/21 1:16 PM, John Naylor wrote: > > On Sun, Jun 13, 2021 at 9:50 AM Tomas Vondra > > mailto:tomas.von...@enterprisedb.com>> > > wrote: > > > >> > 2) We can still keep GEQO around (with some huge limit by default) for a > >> > f

Re: a path towards replacing GEQO with something better

2021-06-14 Thread Tomas Vondra
On 6/14/21 1:16 PM, John Naylor wrote: > On Sun, Jun 13, 2021 at 9:50 AM Tomas Vondra > mailto:tomas.von...@enterprisedb.com>> > wrote: > >> > 2) We can still keep GEQO around (with some huge limit by default) for a >> > few years as an escape hatch, while we refine the replacement. If there >>

Re: a path towards replacing GEQO with something better

2021-06-14 Thread John Naylor
On Sun, Jun 13, 2021 at 9:50 AM Tomas Vondra wrote: > > 2) We can still keep GEQO around (with some huge limit by default) for a > > few years as an escape hatch, while we refine the replacement. If there > > is some bug that prevents finding a plan, we can emit a WARNING and fall > > back to GEQ

Re: a path towards replacing GEQO with something better

2021-06-13 Thread Tomas Vondra
Hi, On 6/10/21 3:21 AM, John Naylor wrote: > Hi, > > On occasion it comes up that the genetic query optimizer (GEQO) doesn't > produce particularly great plans, and is slow ([1] for example). The > only alternative that has gotten as far as a prototype patch (as far as > I know) is simulated anne

a path towards replacing GEQO with something better

2021-06-09 Thread John Naylor
Hi, On occasion it comes up that the genetic query optimizer (GEQO) doesn't produce particularly great plans, and is slow ([1] for example). The only alternative that has gotten as far as a prototype patch (as far as I know) is simulated annealing some years ago, which didn't seem to get far. The