Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13727 )
Change subject: IMPALA-8698: Disable row count estimate to avoid a flaky test ...................................................................... Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/13727/2/testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test File testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test: http://gerrit.cloudera.org:8080/#/c/13727/2/testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test@131 PS2, Line 131: SET DISABLE_HDFS_NUM_ROWS_ESTIMATE=1; > instead of disabling this feature, I would recommend changing the query to Hi Bikram, thank you very much for your suggestion! According to your suggestion, I tried to run the following two SQL statements on my local dev box. Notice that the table "functional_seq_gzip.alltypes" is an hdfs table w/o stats. 1. The original statement in bloom_filters.test. $IMPALA_HOME/bin/impala-shell.sh -q \ "SET RUNTIME_FILTER_MODE=GLOBAL; SET RUNTIME_FILTER_WAIT_TIME_MS=30000; SET RUNTIME_FILTER_MIN_SIZE=4KB; SET RUNTIME_BLOOM_FILTER_SIZE=4KB; select STRAIGHT_JOIN count(*) from functional_seq_gzip.alltypes a join [SHUFFLE] functional_seq_gzip.alltypes b on a.id = b.id; PROFILE;" -p | less 2. The revised statement according to your suggestion (notice that I changed that 7,300 to 3 to force the returned size of the sub-query to be at most 3 instead of 7,300). $IMPALA_HOME/bin/impala-shell.sh -q \ "SET RUNTIME_FILTER_MODE=GLOBAL; SET RUNTIME_FILTER_WAIT_TIME_MS=30000; SET RUNTIME_FILTER_MIN_SIZE=4KB; SET RUNTIME_BLOOM_FILTER_SIZE=4KB; select STRAIGHT_JOIN count(*) from (select * from functional_seq_gzip.alltypes limit 3) a join [SHUFFLE] (select * from functional_seq_gzip.alltypes limit 3) b on a.id = b.id; PROFILE;" -p | less However, on my local dev box, the sizes of "Filter 0" in both cases are 8.00 KB and thus it seems that restricting the sizes of the sub-queries may not be useful in this case. But I might be wrong. Any more suggestions or ideas would be greatly welcomed. :-) -- To view, visit http://gerrit.cloudera.org:8080/13727 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8342bc20a6b7935823d2a8bac2b42afaa1a8aae0 Gerrit-Change-Number: 13727 Gerrit-PatchSet: 3 Gerrit-Owner: Fang-Yu Rao <fangyu....@cloudera.com> Gerrit-Reviewer: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Fang-Yu Rao <fangyu....@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Comment-Date: Tue, 25 Jun 2019 23:21:44 +0000 Gerrit-HasComments: Yes
