[
https://issues.apache.org/jira/browse/IMPALA-15297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Efimov resolved IMPALA-15297.
---------------------------------------
Resolution: Duplicate
Folded into IMPALA-15296 before review. The work was first cut one sub-task per
finding, which came to nine on top of the two the series started with;
regrouped by subject so that a reviewer reads the mechanism once and the
applications after it. Nothing described here is dropped: the code and the
reasoning both live on in IMPALA-15296, and the Gerrit change was abandoned
with the same note.
> Calcite planner: map Impala operators onto Substrait functions
> --------------------------------------------------------------
>
> Key: IMPALA-15297
> URL: https://issues.apache.org/jira/browse/IMPALA-15297
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Aleksandr Efimov
> Assignee: Aleksandr Efimov
> Priority: Major
> Labels: substrait
>
> IMPALA-15296 added a test-scoped exporter for the pre-conversion logical
> plan. Twenty of the twenty-four query shapes the direct harness covers export
> end to end. The rest stop on an operator Isthmus cannot resolve.
> Isthmus resolves a call by looking the operator up in a map keyed on the
> operator itself, and Calcite compares operators by class, name and kind.
> Impala substitutes its own classes for a good number of functions, so an
> operator that means exactly what Substrait's does still misses. Isthmus takes
> additional signatures for this: they say which Substrait function an Impala
> operator stands for and leave the operand matching alone.
> Add them for the operators that have a counterpart. For scalars that is
> coalesce, concat, length, and floor and ceil with their d-prefixed spellings;
> Impala reaches those by three routes, the USE_IMPALA_OPERATOR set in
> ImpalaOperatorTable, builtins Calcite has no operator for, and the
> replacements in ImpalaCustomOperatorTable, and all three resolve by name. For
> windows it is lag, first_value and last_value, resolved through a converter
> of their own that first tries to normalise the aggregate by SqlKind; Impala's
> window operators carry SqlKind.OTHER, so normalisation passes them through
> and the lookup lands on the operator itself. lead needs nothing, since
> USE_IMPALA_OPERATOR lists lag but not lead.
> Together that takes the corpus from twenty to twenty-three.
> What is left needs a different mechanism rather than another name. Substrait
> models a conditional as the IfThen expression rather than as a function, so
> Impala's IF has no name to map to, and it is the one corpus shape still not
> exporting. Substrait's round takes its digit count as i32 where Impala types
> the literal as i8, so the name lines up but the operands do not, and
> Substrait has no truncate at all. NDV is left out on other grounds:
> approx_count_distinct is the obvious counterpart, but that is a rename
> carrying a claim about the estimator rather than a matter of operator class.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
