Qifan Chen has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/17568 )
Change subject: IMPALA-10738: Min/max filters should be enabled for partition columns ...................................................................... IMPALA-10738: Min/max filters should be enabled for partition columns This patch enables min/max filters for partitoned columns to take advantage of the min/max filter infrastructure already built by default. To turn off the feature, set the new query option minmax_filter_partition_column to false. In the patch, the existing query option enabled_runtime_filter_types is made to play a role in the types of the filters generated. The default value ALL generates both the bloom and min/max filters. The alternative value BLOOM generates only the bloom filters and another alternative value MIN_MAX generates only the min/max filters. The normal control knobs minmax_filter_threshold (for threshold) and minmax_filtering_level (for filtering level) still work. When the threshold is 0, the patch automatically assigns a reasonable value for the threshhold. Testing: 1). Added a new test in overlap_min_max_filters.test to verify that a min/max filter is generated for a partition column and is able to filter out all partitions. 2). Core tests [TBD] Change-Id: I89e135ef48b4bb36d70075287b03d1c12496b042 --- M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaService.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java M fe/src/test/java/org/apache/impala/planner/TpcdsPlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters-hdfs-num-rows-est-enabled.test M testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters.test M testdata/workloads/functional-planner/queries/PlannerTest/runtime-filter-query-options.test M testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters.test 11 files changed, 112 insertions(+), 63 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/17568/4 -- To view, visit http://gerrit.cloudera.org:8080/17568 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I89e135ef48b4bb36d70075287b03d1c12496b042 Gerrit-Change-Number: 17568 Gerrit-PatchSet: 4 Gerrit-Owner: Qifan Chen <qc...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>