[
https://issues.apache.org/jira/browse/CALCITE-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972701#comment-14972701
]
Maryann Xue commented on CALCITE-830:
-------------------------------------
Yes, just ran through the Calcite test suite. Got 1 OOM error
(JdbcTest.testJoinManyWay), 1 stack overflow (JdbcTest.testRunMisc), and 3
query plan diffs in JdbcTest (these might not be real failures), and it took 4
times the time to finish running the whole suite as usual. Just from looking at
the logic, the patch is apparently adding a lot more overhead, for it computes
best cost every time regardless of if the current cost is less than the current
best cost. And in most cases (where stats are monotonic) these operations can
well be avoided.
I apologize for not having made it clear that this "patch" was not ready yet
and was meant for demonstration of the problem and for discussion. I'll see if
I come up with a better solution. But anyway this is a rare case, and
correcting it might more or less have some performance impact. Think we can
just move it out of this release.
> Keep VolcanoPlanner state consistent when stats are non-monotonic
> -----------------------------------------------------------------
>
> Key: CALCITE-830
> URL: https://issues.apache.org/jira/browse/CALCITE-830
> Project: Calcite
> Issue Type: Bug
> Reporter: Maryann Xue
> Assignee: Julian Hyde
> Fix For: 1.5.0
>
> Attachments: CALCITE-830.patch, calcite.log.2
>
>
> VolcanoPlanner may have inconsistent state and may not pick the cheapest plan
> if stats (e.g. row count) is not monotonic increasing with cost.
> An example can be:
> select i.item_id, o.quantity from orderTable o join (select distinct item_id
> from itemTable) i on o.item_id = i.item_id
> where item_id is the primary key of itemTable and thus is unique (distinct).
> The AggregateRemoveRule will be applied for the subquery "select distinct
> item_id from itemTable", and generates a cheaper sub-tree equivalent to
> "select item_id from itemTable". However the original Rel sub-tree gives a
> smaller rowCount (it shouldn't have, but let's say it's an independent issue)
> than the new Rel sub-tree (after applying AggregateRemoveRule). By picking
> the new Rel sub-tree as the new best of the child subset, the parent RelNode
> "Join" now has a higher cost, but this change is not reflected in its own
> enclosing subset, which then causes inconsistency.
> Please see the attached log:
> line 27099 ~ 27100:
> {code}
> rel#221:Subset#5.PHOENIX_CLIENT.[0], best=rel#220,
> importance=0.4984472052723946
>
> rel#220:PhoenixToClientConverter.PHOENIX_CLIENT.[0](input=rel#50:Subset#5.PHOENIX_SERVERJOIN.[0]),
> rowcount=525.0, cumulative cost={153.3897966465855 rows, 6 2.0 cpu, 0.0
> io}
> {code}
> line 27092 ~ 27093:
> {code}
> rel#50:Subset#5.PHOENIX_SERVERJOIN.[0], best=rel#49,
> importance=0.4984472052723946
>
> rel#49:PhoenixServerJoin.PHOENIX_SERVERJOIN.[[0]](left=rel#47:Subset#1.PHOENIX_SERVER.[0],right=rel#48:Subset#4.PHOENIX_CLIENT.[],condition==($1,
> $3),joinTyp e=inner), rowcount=525.0, cumulative
> cost={193.63272822430963 rows, 62.0 cpu, 0.0 io}
> {code}
> rel#220 has a lower cumulative cost than its input rel#50.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
