[
https://issues.apache.org/jira/browse/IMPALA-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Efimov resolved IMPALA-15304.
---------------------------------------
Resolution: Duplicate
Folded into IMPALA-15303 before review. The work was first cut one sub-task per
finding, which came to nine on top of the two the series started with;
regrouped by subject so that a reviewer reads the mechanism once and the
applications after it. Nothing described here is dropped: the code and the
reasoning both live on in IMPALA-15303, and the Gerrit change was abandoned
with the same note.
> Calcite planner: check the Substrait export reads back unchanged
> ----------------------------------------------------------------
>
> Key: IMPALA-15304
> URL: https://issues.apache.org/jira/browse/IMPALA-15304
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Aleksandr Efimov
> Assignee: Aleksandr Efimov
> Priority: Major
>
> The tests so far check what the exporter writes. None of them check what a
> consumer reads back, and a plan can serialize cleanly and still mean
> something else on the way through. substrait-java#1159 is exactly that: a
> multi-set aggregate whose columns came back in another order, found from the
> reading side rather than the writing one.
> Read the exported plan back into Calcite with Isthmus and check it still says
> the same: the same relations in the same order, the same column types, and
> the same output names.
> Field names on the relations are not compared, and should not be. Substrait
> carries output names on the plan root rather than on the relations, so
> Calcite invents its own on the way back. The root names are what a consumer
> reads, and those are checked.
> One difference is pinned rather than worked around. Impala's string is a
> varchar of maximum width and the exported plan says so, but reading it back
> gives a varchar of 65536: SubstraitTypeSystem leaves
> RelDataTypeSystemImpl.getMaxPrecision alone, so Calcite caps it. What we
> write is not the lossy part, the reading is, so this belongs in a test that
> names it rather than in a workaround here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
