[Impala-ASF-CR] [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17295 ) Change subject: [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8602/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 Gerrit-Change-Number: 17295 Gerrit-PatchSet: 7 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 19 Apr 2021 23:35:23 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early
Qifan Chen has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/17295 ) Change subject: [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early .. [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early This change set addresses the weakness in population min/max filters in the hash join builder by periodically measuring the usefulness of each such filter and set the 'always_true_' flag to true. For each insert into a filter with always_true_ flag being true, the steps from the evaluation of the value from the row to the verification the value in the the min/max range are completely skipped. The above optimization is also LLVM-codeded. Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/filter-context.cc M be/src/exec/filter-context.h M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/partitioned-hash-join-builder.h M be/src/runtime/runtime-filter-ir.cc M be/src/util/min-max-filter-ir.cc M be/src/util/min-max-filter.cc M be/src/util/min-max-filter.h 9 files changed, 178 insertions(+), 38 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/17295/7 -- To view, visit http://gerrit.cloudera.org:8080/17295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 Gerrit-Change-Number: 17295 Gerrit-PatchSet: 7 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17295 ) Change subject: [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8601/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 Gerrit-Change-Number: 17295 Gerrit-PatchSet: 6 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 19 Apr 2021 23:05:34 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early
Qifan Chen has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/17295 ) Change subject: [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early .. [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early This change set addresses the weakness in population min/max filters in the hash join builder by periodically measuring the usefulness of each such filter and set the AlwaysTrue flag to true. For each insert into a not useful filter, this reduces the amount of work from at least two comparisons and two conditional assignments to one Boolean test. Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 --- M be/src/exec/filter-context.cc M be/src/exec/filter-context.h M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/partitioned-hash-join-builder.h M be/src/runtime/runtime-filter-ir.cc M be/src/util/min-max-filter-ir.cc M be/src/util/min-max-filter.cc M be/src/util/min-max-filter.h 8 files changed, 63 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/17295/6 -- To view, visit http://gerrit.cloudera.org:8080/17295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 Gerrit-Change-Number: 17295 Gerrit-PatchSet: 6 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/17295 ) Change subject: [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/17295/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17295/5//COMMIT_MSG@10 PS5, Line 10: periodically measuring > What's the average sampling rate for typical load? It is per every batch of rows (1024) for now. The performance test is pending. -- To view, visit http://gerrit.cloudera.org:8080/17295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 Gerrit-Change-Number: 17295 Gerrit-PatchSet: 5 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 12 Apr 2021 17:53:38 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17295 ) Change subject: [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/17295/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17295/5//COMMIT_MSG@10 PS5, Line 10: periodically measuring What's the average sampling rate for typical load? -- To view, visit http://gerrit.cloudera.org:8080/17295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 Gerrit-Change-Number: 17295 Gerrit-PatchSet: 5 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Sat, 10 Apr 2021 03:20:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17295 ) Change subject: [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early .. Patch Set 5: Do you have performance test results to show the performance gain? -- To view, visit http://gerrit.cloudera.org:8080/17295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 Gerrit-Change-Number: 17295 Gerrit-PatchSet: 5 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 09 Apr 2021 22:57:27 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17295 ) Change subject: [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8532/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 Gerrit-Change-Number: 17295 Gerrit-PatchSet: 5 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 09 Apr 2021 17:22:11 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17295 ) Change subject: [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8531/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 Gerrit-Change-Number: 17295 Gerrit-PatchSet: 3 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 09 Apr 2021 17:21:08 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early
Qifan Chen has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/17295 ) Change subject: [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early .. [WIP] IMPALA-10650: Bailout min/max filters in hash join builder early This change set addresses the weakness in population min/max filters in the hash join builder by periodically measuring the usefulness of each such filter and set the AlwaysTrue flag to true. For each insert into a not useful filter, this reduces the amount of work from at least two comparisons and two conditional assignments to one Boolean test. Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 --- M be/src/exec/filter-context.cc M be/src/exec/filter-context.h M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/partitioned-hash-join-builder.h M be/src/util/min-max-filter-ir.cc M be/src/util/min-max-filter.cc 6 files changed, 60 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/17295/5 -- To view, visit http://gerrit.cloudera.org:8080/17295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I193646e7acfdd3023f7c947d8107da58a1f41183 Gerrit-Change-Number: 17295 Gerrit-PatchSet: 5 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins
