[ https://issues.apache.org/jira/browse/DRILL-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17015586#comment-17015586 ]
ASF GitHub Bot commented on DRILL-7491: --------------------------------------- paul-rogers commented on pull request #1955: DRILL-7491: Incorrect count() returned for complex types in parquet URL: https://github.com/apache/drill/pull/1955#discussion_r366671949 ########## File path: exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScanWithMetadata.java ########## @@ -167,29 +167,46 @@ public boolean isMatchAllMetadata() { */ @Override public long getColumnValueCount(SchemaPath column) { - long tableRowCount, colNulls; - Long nulls; ColumnStatistics<?> columnStats = getTableMetadata().getColumnStatistics(column); - ColumnStatistics<?> nonInterestingColStats = null; - if (columnStats == null) { - nonInterestingColStats = getNonInterestingColumnsMetadata().getColumnStatistics(column); - } + ColumnStatistics<?> nonInterestingColStats = columnStats == null + ? getNonInterestingColumnsMetadata().getColumnStatistics(column) : null; + long tableRowCount; if (columnStats != null) { tableRowCount = TableStatisticsKind.ROW_COUNT.getValue(getTableMetadata()); } else if (nonInterestingColStats != null) { tableRowCount = TableStatisticsKind.ROW_COUNT.getValue(getNonInterestingColumnsMetadata()); Review comment: Having a hard time understanding this. If a column is uninteresing, we get the row count from the non-interesting columns metadata. Seems round-about. Should we get the row count from the table itself? That is, the indirection through non-interesting columns to get table metadata seems awkward. Also, if we do have column stats, we get the row count from the table metadata. This raises the question: by column value count, do we mean NDV (number of distinct values)? Otherwise, the column value count for top-level columns is defined as the same as the row count whether the column is interesting or not. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Incorrect count() returned for complex types in parquet > ------------------------------------------------------- > > Key: DRILL-7491 > URL: https://issues.apache.org/jira/browse/DRILL-7491 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill, Functions - Hive, Storage - Parquet > Affects Versions: 1.16.0, 1.17.0 > Reporter: Igor Guzenko > Assignee: Igor Guzenko > Priority: Major > Labels: ready-to-commit > Fix For: 1.18.0 > > Attachments: hive_alltypes.parquet > > > To reproduce use the attached file for {{hive_alltypes.parquet}} (this is > parquet file generated by Hive) and try count on columns *c13 - c15.* For > example, > {code:sql} > SELECT count(c13) FROM dfs.tmp.`hive_alltypes.parquet` > {code} > *Expected result:* {color:green}3 {color} > *Actual result:* {color:red}0{color} -- This message was sent by Atlassian Jira (v8.3.4#803005)