Re: Calcite's CBO

2019-02-12 Thread Lekshmi
Hi Stamatis, Thanks a lot for the quick response and support. I think, now I can give an overview of the cost model used in calcite to my other group members. , Thanks and Regards Lekshmi B.G Email: lekshmib...@gmail.com On Tue, Feb 12, 2019 at 1:58 PM Stamatis Zampetakis wrote: > Hi

Re: Calcite's CBO

2019-02-12 Thread Stamatis Zampetakis
Hi Lekshmi, There is really no need to apologise. Hope you and your kid are better now. I don't think there is right or wrong regarding the cost model since it depends mostly on the use-case. If you have a look at the RelOptCost interface you will notice that it is an implementation detail when

Re: Calcite's CBO

2019-02-12 Thread Lekshmi
Hi, Thanks a lot all for your support. I'm really sorry for the late response because my kid and myself were affected with flu since the last few days. I have one more doubt, > records are a partial order: (1, 3) is less than (2, 4), but (1, 3) is neither less than nor greater than (2, 2). If

Re: Calcite's CBO

2019-02-08 Thread Michael Mior
Cost is always a record in Calcite, so yes. Costs from different conventions are compared directly. It's assumed that all the costs are expressed in a way they can be correctly compared. Again, this is not ideal, but it's what we have now and it works relatively well. The planner finds the

Re: Calcite's CBO

2019-02-08 Thread Vladimir Sitnikov
>And how does calcite > compare the costs of different nodes in different convention? Lekshmi, please check RelOptCost interface. https://github.com/apache/calcite/blob/9721283bd0ce46a337f51a3691585cca8003e399/core/src/main/java/org/apache/calcite/plan/RelOptCost.java#L68-L91 Adaptor

Re: Calcite's CBO

2019-02-08 Thread Lekshmi
Hi, I understand... If we are overriding the calcite's cost function in adapter, then should we also generate cost as record? And how does calcite compare the costs of different nodes in different convention? And how calcite generate the cheapest plan? Is there any algorithm dedicated for that?

Re: Calcite's CBO

2019-02-07 Thread Michael Mior
I/O cost is always zero because Calcite itself doesn't try to estimate I/O cost. As I mentioned. my previous reply, this can be overridden by adapters. Row count is an estimate which starts off as the estimated number of rows in the table but then is impacted by the estimated filter factors of

Re: Calcite's CBO

2019-02-07 Thread Lekshmi
Dear Michael Mior, Thank you so much for the reply. But still, I'm confused with following When I connected PostgreSQL with Calcite and execute TPCH queries, provided data residing in the Postgres database, I got the cheapest plan like below for TPCH query 1.

Re: Calcite's CBO

2019-02-06 Thread Michael Mior
You're correct that it's not a single value. That said, in practice, not much is actually done with. CPU or I/O cost. You'll see in TableScan that computeSelfCost returns a cost which really only makes use of the number of rows in the table. Individual adapters may override this, but few do. --

Re: Calcite's CBO

2019-02-06 Thread Lekshmi
Hi, In [0], they suggest, "the optimizer implementer can choose the cost to be a number or record". Which does one Apache Calcite use? I found, it as a record, when I run queries in debug mode. Is that correct? Then can you please define (rows, CPU, I/O) in Calcite? Also, when, we are connecting

Re: Calcite's CBO

2019-02-05 Thread Stamatis Zampetakis
I had exactly the same pointers with Michael in mind. @Lekshmi: Note also that traits used by Calcite are, I think, equivalent to the notion of "physical properties" which appears in the Volcano paper (so in Volcano terminology a "convention" trait == a physical property). Στις Τρί, 5 Φεβ

Re: Calcite's CBO

2019-02-05 Thread Michael Mior
Calcite's optimizer is based off the Volcano optimizer[0]. In that paper you'll find an outline of the algorithm which is basically equivalent to what Calcite uses. Adding multiple systems doesn't complicate things very much. The main addition used by Calcite is what we call a "convention" trait

Calcite's CBO

2019-02-05 Thread Lekshmi
Hi, I would like to know about the Calcite CBO in detail, including how it deals with global optimization when multiple processing systems are associated with it. Any documentation, pointers are much appreciated. Thanks and Regards Lekshmi B.G Email: lekshmib...@gmail.com