[
https://issues.apache.org/jira/browse/IMPALA-15306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Efimov reassigned IMPALA-15306:
-----------------------------------------
Assignee: Aleksandr Efimov
> Calcite planner: carry Calcite's estimates into the Substrait plan
> ------------------------------------------------------------------
>
> Key: IMPALA-15306
> URL: https://issues.apache.org/jira/browse/IMPALA-15306
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Aleksandr Efimov
> Assignee: Aleksandr Efimov
> Priority: Major
>
> A plan leaves the exporter saying nothing about how much data any relation is
> expected to produce. Isthmus derives no statistics of its own, so a consumer
> planning or costing the query has to guess. Substrait has a place for this on
> every relation, and Calcite has already worked the numbers out by the time
> the plan reaches the seam.
> Carry them out. SubstraitRelVisitor.apply is public and every relation goes
> through it, so a subclass can attach RelMetadataQuery's row count and average
> row size to what it returns.
> Nothing is asserted that Substrait does not already treat as approximate: its
> own words for the field are "estimated statistics" and "the estimated row
> count". That matters, because Impala's row count is a real one from COMPUTE
> STATS when the table has been analysed and an estimate derived from file
> sizes when it has not, and CalciteTable hands both over through the same
> getRowCount. Writing either into a field declared as an estimate says nothing
> untrue; writing them into a field that promised a fact would.
> The estimates are attached to what reaches apply rather than to the Calcite
> node they came from. Isthmus drops a projection that only renames, returning
> its input instead, and rewrites an aggregate before the visitor sees it, so
> binding to the returned relation keeps the numbers on the relation a consumer
> will read.
> Requires substrait-java 0.101.0: Rel.withHint arrived there, and 0.100.0 can
> read a hint but has no type-agnostic way to set one.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
