Aleksandr Efimov created IMPALA-15304:
-----------------------------------------
Summary: Calcite planner: check the Substrait export reads back
unchanged
Key: IMPALA-15304
URL: https://issues.apache.org/jira/browse/IMPALA-15304
Project: IMPALA
Issue Type: Sub-task
Components: Frontend
Reporter: Aleksandr Efimov
The tests so far check what the exporter writes. None of them check what a
consumer reads back, and a plan can serialize cleanly and still mean something
else on the way through. substrait-java#1159 is exactly that: a multi-set
aggregate whose columns came back in another order, found from the reading side
rather than the writing one.
Read the exported plan back into Calcite with Isthmus and check it still says
the same: the same relations in the same order, the same column types, and the
same output names.
Field names on the relations are not compared, and should not be. Substrait
carries output names on the plan root rather than on the relations, so Calcite
invents its own on the way back. The root names are what a consumer reads, and
those are checked.
One difference is pinned rather than worked around. Impala's string is a
varchar of maximum width and the exported plan says so, but reading it back
gives a varchar of 65536: SubstraitTypeSystem leaves
RelDataTypeSystemImpl.getMaxPrecision alone, so Calcite caps it. What we write
is not the lossy part, the reading is, so this belongs in a test that names it
rather than in a workaround here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
