Aleksandr Efimov has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24747
Change subject: IMPALA-15301: Export Impala's explicit cast as a Substrait cast ...................................................................... IMPALA-15301: Export Impala's explicit cast as a Substrait cast ImpalaOperatorTable rewrites cast into an operator of its own, named EXPLICIT_CAST and carrying SqlKind.OTHER, to keep Calcite from folding it. The converter Isthmus already has for SqlKind.CAST therefore never sees it, and every explicit cast stops the export. Over the TPC-DS queries that carry Calcite goldens this is what blocks seven of them. Add a call converter that emits a Substrait Cast, the same way IMPALA-15298 handles IF. The failure behaviour is the closest of the two Substrait offers rather than an exact match, and that is worth stating plainly. Measured against a running Impala: casting an unparseable string returns NULL, and an integer that does not fit wraps round silently, so 99999999999 cast to int comes back as 1215752191. Impala never raises. RETURN_NULL states the first half. There is no Substrait failure behaviour for the second, so a consumer would produce NULL where Impala produces the wrapped value. Almost every EXPLICIT_CAST in a plan is one the planner inserted to line types up, where nothing overflows. A narrowing cast a user wrote is a real difference, and the alternative is not exporting any plan that contains a cast at all. Two of the seven queries stop again further along, on a blocker the cast had been hiding: one on the multiply that a datetime interval expands into, one on SINGLE_VALUE. The corpus goes from 96 of 110 to 101 rather than to 103. Testing, in a local cluster on Linux: - SubstraitPlanExporterTest: 17 tests, 0 failures, 0 errors. The added one checks that the result is a CAST expression rather than a call, that it carries RETURN_NULL, and that it targets the right type - CalciteOptimizerTest and CalciteSubstraitCompatibilityTest, neither of which this patch touches: 11 and 4 tests, 0 failures - Withholding the converter fails the added test, so it is not vacuous - A probe over the TPC-DS queries carrying Calcite goldens reports 101 of 110 with this patch and 96 without it - git diff --check Change-Id: I7523bc5ebd6239e92dc817bbe521920121ec8b2b Assisted-by: Claude Opus 5 (Anthropic) --- M java/calcite-planner/src/test/java/org/apache/impala/calcite/service/SubstraitOperatorMappings.java M java/calcite-planner/src/test/java/org/apache/impala/calcite/service/SubstraitPlanExporterTest.java 2 files changed, 50 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/24747/1 -- To view, visit http://gerrit.cloudera.org:8080/24747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I7523bc5ebd6239e92dc817bbe521920121ec8b2b Gerrit-Change-Number: 24747 Gerrit-PatchSet: 1 Gerrit-Owner: Aleksandr Efimov <[email protected]>
