Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/20366 )
Change subject: IMPALA-12357: Skip scheduling bloom filter from full-build scan ...................................................................... Patch Set 6: (3 comments) http://gerrit.cloudera.org:8080/#/c/20366/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20366/4//COMMIT_MSG@29 PS4, Line 29: thus reducing the bloom filter building, : aggregation, and evalua > This applies to the remote bloom filters right ? For local filters that are This patch does not differentiate between the locality of the filter target. I look again and found that it is still beneficial to prune LOCAL filter as well. For example, all pruned filter from new testcases in bloom-filter-assignment.test are LOCAL filter. The pruning is also correct since the JOIN node has BROADCAST build. http://gerrit.cloudera.org:8080/#/c/20366/4/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java File fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java: http://gerrit.cloudera.org:8080/#/c/20366/4/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java@252 PS4, Line 252: private int level_ = 1; > Is 'rank' the right term here ? Normally, rank implies some type of compar Replaced with level. http://gerrit.cloudera.org:8080/#/c/20366/4/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java@844 PS4, Line 844: = true; > For testing purposes or for a backdoor in a real deployment, would it be us Added max_filter_error_rate_from_full_scan, mainly to allow disabling this feature. There are also existing max_filter_error_rate flag and RUNTIME_FILTER_ERROR_RATE query option to tune runtime filter accuracy. The difference is those configs are preliminary used to calculate ideal bloom filter size before RUNTIME_FILTER_MAX_SIZE and RUNTIME_FILTER_MIN_SIZE limits are applied. -- To view, visit http://gerrit.cloudera.org:8080/20366 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I494533bc06da84e606cbd1ae1619083333089a5e Gerrit-Change-Number: 20366 Gerrit-PatchSet: 6 Gerrit-Owner: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Comment-Date: Mon, 28 Aug 2023 19:22:50 +0000 Gerrit-HasComments: Yes
