[
https://issues.apache.org/jira/browse/IMPALA-15305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Efimov resolved IMPALA-15305.
---------------------------------------
Resolution: Duplicate
Folded into IMPALA-15296 before review. The work was first cut one sub-task per
finding, which came to nine on top of the two the series started with;
regrouped by subject so that a reviewer reads the mechanism once and the
applications after it. Nothing described here is dropped: the code and the
reasoning both live on in IMPALA-15296, and the Gerrit change was abandoned
with the same note.
> Calcite planner: export NDV as a Substrait approx_count_distinct
> ----------------------------------------------------------------
>
> Key: IMPALA-15305
> URL: https://issues.apache.org/jira/browse/IMPALA-15305
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Aleksandr Efimov
> Assignee: Aleksandr Efimov
> Priority: Major
>
> NDV was left out of the mapping table added by IMPALA-15297 on the grounds
> that approx_count_distinct is the obvious counterpart but the rename would
> assert something about the estimator that a name alone does not back.
> Checking rather than assuming settles it. Substrait says of
> approx_count_distinct that it calculates the approximate number of distinct
> values "using HyperLogLog". Impala's ndv is HyperLogLog too, sized by
> ComputeHllLengthFromScale. Both name the same algorithm, so the mapping
> asserts nothing that is not already true of either side.
> Map it, and wire an aggregate converter to carry it: the provider had scalar
> and window converters until now.
> The two-argument ndv, which sets the precision, needs nothing to stay
> refused. Substrait has one variant of approx_count_distinct and it takes one
> operand, so a call carrying a scale matches nothing and the export says so.
> There is no way to drop the precision without noticing, and nothing in the
> mapping arranges that.
> No TPC-DS query is affected: ndv is an Impala function rather than standard
> SQL, so the corpus stays at 102 of 110. It is common in real Impala queries,
> which is the reason to carry it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
