Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/8480 )
Change subject: IMPALA-4985: use parquet stats of nested types for dynamic pruning ...................................................................... Patch Set 3: (4 comments) http://gerrit.cloudera.org:8080/#/c/8480/3/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java: http://gerrit.cloudera.org:8080/#/c/8480/3/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@179 PS3, Line 179: it is checked to be not empty. It could help to explain how that check is generated, e.g. by mentioning that the analyzer adds a IsNotEmptyPredicate for some queries. http://gerrit.cloudera.org:8080/#/c/8480/3/testdata/workloads/functional-query/queries/QueryTest/nested-types-parquet-stats.test File testdata/workloads/functional-query/queries/QueryTest/nested-types-parquet-stats.test: http://gerrit.cloudera.org:8080/#/c/8480/3/testdata/workloads/functional-query/queries/QueryTest/nested-types-parquet-stats.test@98 PS3, Line 98: !empty Do we introduce the "!empty" guard somewhere and explain what it means? http://gerrit.cloudera.org:8080/#/c/8480/3/testdata/workloads/functional-query/queries/QueryTest/nested-types-parquet-stats.test@145 PS3, Line 145: # Nested access in left-outer-join that would be incorrect if scan was pruned. Impressive find! Can you reword this to emphasize that this does not allow pruning despite having a predicate on one of the columns that does not match any rows? http://gerrit.cloudera.org:8080/#/c/8480/2/tests/query_test/test_nested_types.py File tests/query_test/test_nested_types.py: http://gerrit.cloudera.org:8080/#/c/8480/2/tests/query_test/test_nested_types.py@85 PS2, Line 85: @SkipIfIsilon.hive Thank you :) -- To view, visit http://gerrit.cloudera.org:8080/8480 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0c99e20cb080b504442cd5376ea3e046016158fe Gerrit-Change-Number: 8480 Gerrit-PatchSet: 3 Gerrit-Owner: Vuk Ercegovac <vercego...@cloudera.com> Gerrit-Reviewer: Lars Volker <l...@cloudera.com> Gerrit-Reviewer: Vuk Ercegovac <vercego...@cloudera.com> Gerrit-Comment-Date: Thu, 16 Nov 2017 21:28:26 +0000 Gerrit-HasComments: Yes
