[
https://issues.apache.org/jira/browse/IMPALA-15284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18108001#comment-18108001
]
ASF subversion and git services commented on IMPALA-15284:
----------------------------------------------------------
Commit 13e8b9618b00f3d3712e8d1767d1b3f98fbb489d in impala's branch
refs/heads/master from Aleksandr Efimov
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=13e8b9618 ]
IMPALA-15284: Cover the pre-conversion seam programs
The direct Calcite-only harness pins the client-visible labels, the
exact output types, the PERCENT_RANK rewrite and seven query shapes.
Everything the seam runs in between is exercised only indirectly, so a
change there surfaces as a golden-file diff elsewhere, or not at all.
Widen the corpus to twenty-four shapes: conditional expressions, string
and decimal functions, DISTINCT, HAVING, LIMIT with OFFSET, INTERSECT,
EXCEPT, outer and cross joins, semi and anti joins from IN and NOT IN,
EXISTS, a scalar subquery, an inline view, a common table expression
and LEAD/LAG. The synthetic fixture gains the columns and the second
table these need.
Pin what the programs at the seam do to the plan: CUME_DIST and NTILE
expansion next to the existing PERCENT_RANK case, nested projects
merging into one, unused columns trimmed, multi-row VALUES arriving as
a single Values, and the Impala decimal typing of FLOOR and CEIL
including the cap at the maximum precision. Check the unsupported
complex-type case on unnesting as well as on projection.
Pin cte_threshold too: off by default, and above the threshold the
repeated subtree becomes an ImpalaSequence over a producer and one
consumer per reference. With the option set the seam stops being a
plan of standard Calcite relations, which is worth stating in a test.
Testing:
- CalciteOptimizerTest: 11 tests, 0 failures, 0 errors on Linux
- git diff --check
Change-Id: I8a4403b5fbdd59e2eea79139891d8bb7c4a0413b
Assisted-by: Claude Opus 5 (Anthropic)
Reviewed-on: http://gerrit.cloudera.org:8080/24724
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Steve Carlin <[email protected]>
Reviewed-by: Michael Smith <[email protected]>
> Calcite planner: Cover the pre-conversion seam programs in the direct harness
> -----------------------------------------------------------------------------
>
> Key: IMPALA-15284
> URL: https://issues.apache.org/jira/browse/IMPALA-15284
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Aleksandr Efimov
> Assignee: Aleksandr Efimov
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> IMPALA-15237 added a direct Calcite-only harness around the logical plan that
> createPreImpalaConvertPlan() returns. It pins the client-visible labels and
> the
> exact output types, the PERCENT_RANK rewrite, and seven query shapes that have
> to keep reaching an Impala plan. The phases in between - node expansion, type
> coercion, field trimming, CTE planning - are exercised only indirectly, so a
> change in any of them shows up as a golden-file diff somewhere else, or not at
> all.
>
> Extend the harness:
>
> - widen the query corpus that has to keep reaching an Impala plan with the
> same client-visible labels: conditional expressions, string and decimal
> functions, DISTINCT, HAVING, LIMIT with OFFSET, INTERSECT, EXCEPT, outer and
> cross joins, semi and anti joins from IN and NOT IN, EXISTS, a scalar
> subquery, an inline view, a common table expression, and LEAD/LAG;
> - pin the rules that rewrite the plan at the seam: CUME_DIST and NTILE
> expansion next to the existing PERCENT_RANK case, nested projects merging
> into one, unused columns being trimmed, and multi-row VALUES arriving as a
> single Values relation;
> - pin the Impala-specific decimal typing of FLOOR and CEIL, including the
> cap at the maximum precision;
> - pin what cte_threshold does at the seam: off by default, and above the
> threshold the repeated subtree becomes an ImpalaSequence over a producer and
> one consumer per reference. That is worth stating explicitly, because with
> the option set the seam is no longer a plan of standard Calcite relations;
> - check the unsupported complex-type case on unnesting as well as projection.
>
> Test-only change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
