[ https://issues.apache.org/jira/browse/IMPALA-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Rogers resolved IMPALA-8058. --------------------------------- Resolution: Fixed Assignee: Paul Rogers Fix Version/s: Impala 3.2.0 > HBase scan cardinality division-by-zero leads to bogus cardinality > ------------------------------------------------------------------ > > Key: IMPALA-8058 > URL: https://issues.apache.org/jira/browse/IMPALA-8058 > Project: IMPALA > Issue Type: Bug > Components: Frontend > Affects Versions: Impala 3.1.0 > Reporter: Paul Rogers > Assignee: Paul Rogers > Priority: Major > Fix For: Impala 3.2.0 > > > A particular HBase query has highly selective key filters and runs into code > bugs that produce a bogus, huge cardinality value. > {{HbaseScanNode.computeStats()}} attempts to compute table cardinality by > calling {{HBaseTable.getEstimatedRowStats()}}. This then calls into (in the > latest versions) {{FeHBaseTable.getEstimatedRowStats()}}. > This code tries to estimate cardinality by: > * Scanning a set of regions. > * For each getting the size. > * Averaging a bunch of rows to estimate row width. > Once we know the size of the regions we need to scan, and the average row > width, we can compute the scan cardinality. > The problem in this particular query is that the predicates are so selective > that no regions match. As a result, the average row width is zero. We divide > (as a double) the region size by 0 and get INF. We cast that to a long and > get Long.MAX_VALUE. We then use that as our (highly bogus) cardinality > estimate. > The code must: > * Detect the division-by-zero (now sample rows) case. > * Use an alternative estimate (such as multiplying total table row count from > HMS by the filter selectivity.) -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org