Vladimir Sitnikov created CALCITE-7876:
------------------------------------------
Summary: VolcanoPlanner fires logical transformation rules such as
ProjectMergeRule on JDBC rels
Key: CALCITE-7876
URL: https://issues.apache.org/jira/browse/CALCITE-7876
Project: Calcite
Issue Type: Bug
Components: core, jdbc-adapter
Reporter: Vladimir Sitnikov
When I plan a query against a JDBC schema, {{VolcanoPlanner}} fires logical
transformation rules on JDBC rels. {{ProjectMergeRule}} matches two
{{JdbcProject}} rels and produces a {{LogicalProject}};
{{FilterProjectTransposeRule}} matches {{JdbcFilter}} over {{JdbcProject}} and
produces a {{JdbcProject}}. The same rules do not fire on Enumerable rels.
*Expected*
The javadoc of {{TransformationRule}} says: "only logical operator can be rule
operand, and only generate logical alternatives". So in {{VolcanoPlanner}},
{{ProjectMergeRule}}, {{FilterProjectTransposeRule}}, and
{{SortRemoveConstantKeysRule}} (via {{SubstitutionRule}}) should have no
attempts whose first operand is a {{JdbcRel}}, which is what happens for
{{EnumerableRel}} today.
*Actual*
Run on main at 6ff13eaccd (1.43.0-SNAPSHOT), with the {{scott}} JDBC model from
the test suite. Rule attempts whose first operand is a {{JdbcRel}}, with the
operand classes:
{noformat}
JDBC_ATTEMPTS={EnumerableLimitRule on JdbcSort,=2, FilterProjectTransposeRule
on JdbcFilter,JdbcProject,=1,
JdbcToEnumerableConverterRule(in:JDBC.SCOTT,out:ENUMERABLE) on JdbcFilter,=2,
JdbcToEnumerableConverterRule(in:JDBC.SCOTT,out:ENUMERABLE) on JdbcProject,=4,
JdbcToEnumerableConverterRule(in:JDBC.SCOTT,out:ENUMERABLE) on JdbcSort,=2,
JdbcToEnumerableConverterRule(in:JDBC.SCOTT,out:ENUMERABLE) on
JdbcTableScan,=1, ProjectMergeRule on JdbcProject,JdbcProject,=1,
SortRemoveConstantKeysRule on JdbcSort,=2, SortRemoveRule on JdbcSort,=2}
JDBC_PRODUCED={EnumerableLimitRule -> EnumerableLimit=1,
FilterProjectTransposeRule -> JdbcProject=1,
JdbcToEnumerableConverterRule(in:JDBC.SCOTT,out:ENUMERABLE) ->
JdbcToEnumerableConverter=9, ProjectMergeRule -> LogicalProject=1,
SortRemoveRule -> RelSubset=1}
{noformat}
The same probe on the {{hr}} reflective schema (Enumerable only) recorded one
attempt on an {{EnumerableRel}}: {{EnumerableLimitRule}} on {{EnumerableSort}},
whose {{onMatch}} returns at once because the sort has no fetch.
I did not measure planning time, so I do not know how much these attempts cost
on large queries.
*Reproducer* (a JUnit 5 test in {{core/src/test/java}})
{code:java}
@Test void transformationRulesOnJdbcRels() {
Map<String, Integer> attempts = new TreeMap<>();
Map<String, Integer> produced = new TreeMap<>();
CalciteAssert.model(JdbcTest.SCOTT_MODEL)
.query("select x + 1 as y from (select empno + deptno as x, ename"
+ " from scott.emp where deptno > 10) order by y limit 2")
.withHook(Hook.PLANNER, (Consumer<RelOptPlanner>) planner ->
planner.addListener(new RelOptListener() {
@Override public void ruleAttempted(RuleAttemptedEvent e) {
if (e.isBefore() && e.getRuleCall().rels[0] instanceof JdbcRel) {
StringBuilder sb = new StringBuilder();
for (RelNode r : e.getRuleCall().rels) {
sb.append(r.getClass().getSimpleName()).append(',');
}
attempts.merge(e.getRuleCall().getRule() + " on " + sb, 1,
Integer::sum);
}
}
@Override public void ruleProductionSucceeded(RuleProductionEvent
e) {
if (!e.isBefore() && e.getRuleCall().rels[0] instanceof JdbcRel) {
produced.merge(e.getRuleCall().getRule() + " -> "
+ e.getRel().getClass().getSimpleName(), 1, Integer::sum);
}
}
@Override public void relEquivalenceFound(RelEquivalenceEvent e) { }
@Override public void relDiscarded(RelDiscardedEvent e) { }
@Override public void relChosen(RelChosenEvent e) { }
}))
.returnsCount(2);
System.out.println("JDBC_ATTEMPTS=" + attempts);
System.out.println("JDBC_PRODUCED=" + produced);
}
{code}
*Analysis (a guess; the report above stands without it)*
{{VolcanoPlanner}} skips {{TransformationRule}} operands only for classes that
implement {{PhysicalNode}} ({{VolcanoPlanner#addRule}} and
{{VolcanoPlanner#onNewClass}}, added by CALCITE-3997). {{EnumerableRel}}
extends {{PhysicalNode}}; {{JdbcRel}} and {{BindableRel}} extend {{RelNode}}
only, and so do the rel interfaces of the other adapters. One possible fix is
to make those interfaces extend {{PhysicalNode}}; I have not checked what else
that changes, for example how {{passThrough}} and {{derive}} behave in top-down
mode.
CALCITE-3455 reported this for Enumerable rels, and CALCITE-3997 fixed it
there; this issue covers the conventions that CALCITE-3997 left out.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
