[
https://issues.apache.org/jira/browse/IMPALA-15303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Efimov reassigned IMPALA-15303:
-----------------------------------------
Assignee: Aleksandr Efimov
> Calcite planner: cover the TPC-DS corpus in the Substrait export tests
> ----------------------------------------------------------------------
>
> Key: IMPALA-15303
> URL: https://issues.apache.org/jira/browse/IMPALA-15303
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Aleksandr Efimov
> Assignee: Aleksandr Efimov
> Priority: Major
>
> The exporter is covered by query shapes written by hand, one construct at a
> time. Every gap it turned out to have was found by running a real workload
> instead, and the numbers those runs produced have been quoted in IMPALA-15297
> and after without anything in the tree that reproduces them.
> Run the exporter over the TPC-DS queries that carry Calcite goldens, as a
> test.
> The tables come from the schema template rather than from the catalogue, so
> this runs wherever the frontend tests run and needs no loaded warehouse. Only
> column types matter for an export, so no data or statistics are involved,
> which also means the plans are not the ones a loaded warehouse would produce.
> The expectation is a named list rather than a count: which queries do not
> export and what stops each. A count would only say that the number moved.
> Removing the abs mapping, for instance, fails the test with tpcds-q47, q53,
> q57, q63 and q89 named against ABS.
> Today 102 of 110 export. The eight that do not are outside this exporter:
> four stop on SINGLE_VALUE, which Calcite uses to guard a scalar subquery and
> Impala turns into a runtime cardinality check that Substrait has no way to
> express, and four on GROUPING, which is substrait-java#993. A query leaving
> that list matters as much as one joining it, so the test reports both
> directions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
