[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. IMPALA-3816: Codegen perf critical loops in Sorter This fix added the functionality to codegen recursive method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit test to order 7300 rows from table functional.alltypes, the speedup of the code-gen version over non-code-gen version of the method is around 65%. In another unit test to partially order 2880404 rows, the speedup is around 61%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Reviewed-on: http://gerrit.cloudera.org:8080/16621 Reviewed-by: Csaba Ringhofer Tested-by: Impala Public Jenkins --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 136 insertions(+), 41 deletions(-) Approvals: Csaba Ringhofer: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 15 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 14: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 14 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 09 Nov 2020 21:28:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 14: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6633/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 14 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 09 Nov 2020 16:05:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 14: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 14 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 09 Nov 2020 16:04:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 14: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7622/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 14 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 09 Nov 2020 14:53:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#14). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. IMPALA-3816: Codegen perf critical loops in Sorter This fix added the functionality to codegen recursive method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit test to order 7300 rows from table functional.alltypes, the speedup of the code-gen version over non-code-gen version of the method is around 65%. In another unit test to partially order 2880404 rows, the speedup is around 61%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 136 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/14 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 14 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 13: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/16621/13/be/src/exec/sort-node.cc File be/src/exec/sort-node.cc: http://gerrit.cloudera.org:8080/#/c/16621/13/be/src/exec/sort-node.cc@114 PS13, Line 114: AddCodegenStatus The pattern seems to be to call this function only once per plan node - it isn't useful to add "ok" twice. I would prefer set 'status' here, and call AddCodegenStatus only at the end. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 13 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 07 Nov 2020 15:47:14 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 13: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7618/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 13 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 05 Nov 2020 14:14:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 13: One change was made in SortPlanNode::Codegen() to check the status of Codegen on compare_fn before code-gen for the SorterHelper method. This is necessary to prevent compare_fn being a nullptr. With the change, core tests pass. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 13 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 05 Nov 2020 13:56:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#13). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. IMPALA-3816: Codegen perf critical loops in Sorter This fix added the functionality to codegen recursive method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit test to order 7300 rows from table functional.alltypes, the speedup of the code-gen version over non-code-gen version of the method is around 65%. In another unit test to partially order 2880404 rows, the speedup is around 61%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 136 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/13 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 13 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7615/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 12 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 04 Nov 2020 20:20:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#12). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. IMPALA-3816: Codegen perf critical loops in Sorter This fix added the functionality to codegen recursive method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit test to order 7300 rows from table functional.alltypes, the speedup of the code-gen version over non-code-gen version of the method is around 65%. In another unit test to partially order 2880404 rows, the speedup is around 61%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 133 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/12 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 12 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 11: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6630/ -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 11 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 04 Nov 2020 01:20:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 11: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 11 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 04 Nov 2020 00:16:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 11: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 11 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 21:34:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 11: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6630/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 11 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 21:34:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 10: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 10 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 20:32:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7607/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 10 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 14:45:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. IMPALA-3816: Codegen perf critical loops in Sorter This fix added the functionality to codegen recursive method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit test to order 7300 rows from table functional.alltypes, the speedup of the code-gen version over non-code-gen version of the method is around 65%. In another unit test to partially order 2880404 rows, the speedup is around 61%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 133 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/10 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 10 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 10: (1 comment) http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc File be/src/runtime/sorter.cc: http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc@1215 PS8, Line 1215: LlvmCodeGen* codegen = state->codegen(); > Did some additional testing with instrumentation and found the return addre Added the logic to replace the call sites (two recursive calls) within the LLVMed SorterHelp with good improvement (~65% and ~61% over non-LLVM version, respectively). -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 10 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 14:25:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7606/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 9 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 01:39:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has posted comments on this change. (
http://gerrit.cloudera.org:8080/16621 )
Change subject: IMPALA-3816: Codegen perf critical loops in Sorter
..
Patch Set 9:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc
File be/src/runtime/sorter.cc:
http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc@1215
PS8, Line 1215: llvm::Function* fn =
codegen->GetFunction(IRFunction::TUPLE_SORTER_SORT_HELPER, true);
> Looks like there is only on call to SortHelper() which is from Sorter::Tupl
Did some additional testing with instrumentation and found the return addresses
reported via __builtin_return_address(0) in the LLVM and non-LLVM version of
SortHelper() are quite different: In the range of 0x7f7f12135exx for LLVM and
0x18c2yyy in non-LLVM. Note that __builtin_return_address(0) returns the
address of the instruction after a function call.
167 Status Sorter::TupleSorter::SortHelper(TupleIterator begin, TupleIterator
end) {
168 // Use insertion sort for smaller sequences.
169
170 printf("Enter SortHelper(): return_address(0)=%p\n",
__builtin_return_address(0));
171
172 while (end.index() - begin.index() > INSERTION_THRESHOLD) {
173 // Select a pivot and call Partition() to split the tuples in [begin,
end) into two
174 // groups (<= pivot and >= pivot) in-place. 'cut' is the index of the
first tuple in
175 // the second group.
176 Tuple* pivot = SelectPivot(begin, end);
177 TupleIterator cut;
178 RETURN_IF_ERROR(Partition(begin, end, pivot, ));
179
180 // Recurse on the smaller partition. This limits stack size to log(n)
stack frames.
181 if (cut.index() - begin.index() < end.index() - cut.index()) {
182 // Left partition is smaller.
183 printf("call SortHelper() for [begin, cut]\n");
184 RETURN_IF_ERROR(SortHelper(begin, cut));
185 begin = cut;
186 } else {
187 // Right partition is equal or smaller.
188 printf("call SortHelper() for [cut, end]\n");
189 RETURN_IF_ERROR(SortHelper(cut, end));
190 end = cut;
191 }
192 }
193
194 if (begin.index() < end.index()) RETURN_IF_ERROR(InsertionSort(begin,
end));
195
196 printf("Exit SortHelper(): return_address(0)=%p\n",
__builtin_return_address(0));
197 return Status::OK();
198 }
--
To view, visit http://gerrit.cloudera.org:8080/16621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b
Gerrit-Change-Number: 16621
Gerrit-PatchSet: 9
Gerrit-Owner: Qifan Chen
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Qifan Chen
Gerrit-Reviewer: Sahil Takiar
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Tue, 03 Nov 2020 01:21:09 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. IMPALA-3816: Codegen perf critical loops in Sorter This fix added the functionality to codegen method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit test to order 7300 rows from table functional.alltypes, the speedup of the code-gen version over non-code-gen version of the method is around 26%. In another unit test to partially order 2880404 rows, the speedup is around 20%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 123 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/9 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 9 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has posted comments on this change. (
http://gerrit.cloudera.org:8080/16621 )
Change subject: IMPALA-3816: Codegen perf critical loops in Sorter
..
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc
File be/src/runtime/sorter.cc:
http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc@1215
PS8, Line 1215: llvm::Function* fn =
codegen->GetFunction(IRFunction::TUPLE_SORTER_SORT_HELPER, true);
> We don't anywhere else. https://gerrit.cloudera.org/#/c/12828/ have a compl
Looks like there is only on call to SortHelper() which is from
Sorter::TupleSorter::Sort() shown below.
752 Status Sorter::TupleSorter::Sort(Run* run) {
753 DCHECK(run->is_finalized());
754 DCHECK(!run->is_sorted());
755 run_ = run;
756 const SortHelperFn sort_helper_fn =
parent_->codegend_sort_helper_fn_.load();
757 if (sort_helper_fn != nullptr) {
758 RETURN_IF_ERROR(
759 sort_helper_fn(this, TupleIterator::Begin(run_),
TupleIterator::End(run_)));
760 } else {
761 RETURN_IF_ERROR(SortHelper(TupleIterator::Begin(run_),
TupleIterator::End(run_)));
762 }
763 run_->set_sorted();
764 return Status::OK();
765 }
The speedup may also come from the inlining of several small functions before
LLVM optimizing "the big chunk of code" :-).
I have collected the llvm code and uploaded it to the case here IMPALA-3816"
target="_blank"
rel="nofollow">https://issues.apache.org/jira/browse/IMPALA-3816.
Interestingly, there are 2 definitions (one fast cc and one regular) for the
SortHelp method and 4 calls to the fast cc version.
803 define internal fastcc void
@_ZN6impala6Sorter11TupleSorter10SortHelperENS0_13TupleIteratorES2_(%"class.impala::Status"*
noalias sr et, %"class.impala::Sorter::TupleSorter"*,
%"class.impala::Sorter::TupleIterator"* byval nocapture align 8,
%"class.impala::Sorter: :TupleIterator"* byval nocapture align 8)
unnamed_addr #3 align 2 personality i8* bitcast (i32 (...)*
@__gxx_personality_v0 to i8*)
1724 call fastcc void
@_ZN6impala6Sorter11TupleSorter10SortHelperENS0_13TupleIteratorES2_(%"class.impala::Status"*
nonnull sret %19, % "class.impala::Sorter::TupleSorter"* nonnull %1,
%"class.impala::Sorter::TupleIterator"* byval nonnull align 8 %2,
%"class.impala:: Sorter::TupleIterator"* byval nonnull align 8 %17)
1753 call fastcc void
@_ZN6impala6Sorter11TupleSorter10SortHelperENS0_13TupleIteratorES2_(%"class.impala::Status"*
nonnull sret %20, % "class.impala::Sorter::TupleSorter"* nonnull %1,
%"class.impala::Sorter::TupleIterator"* byval nonnull align 8 %17,
%"class.impala: :Sorter::TupleIterator"* byval nonnull align 8 %3)
2756 define void
@_ZN6impala6Sorter11TupleSorter10SortHelperENS0_13TupleIteratorES2_.3(%"class.impala::Status"*
noalias sret, %"class.im pala::Sorter::TupleSorter"*,
%"class.impala::Sorter::TupleIterator"* byval nocapture align 8,
%"class.impala::Sorter::TupleIterator "* byval nocapture align 8)
local_unnamed_addr #3 align 2 personality i8* bitcast (i32 (...)*
@__gxx_personality_v0 to i8*) {
3877 call fastcc void
@_ZN6impala6Sorter11TupleSorter10SortHelperENS0_13TupleIteratorES2_(%"class.impala::Status"*
nonnull sret %11, % "class.impala::Sorter::TupleSorter"* nonnull %1,
%"class.impala::Sorter::TupleIterator"* byval nonnull align 8 %2,
%"class.impala:: Sorter::TupleIterator"* byval nonnull align 8 %9)
3905 call fastcc void
@_ZN6impala6Sorter11TupleSorter10SortHelperENS0_13TupleIteratorES2_(%"class.impala::Status"*
nonnull sret %12, % "class.impala::Sorter::TupleSorter"* nonnull %1,
%"class.impala::Sorter::TupleIterator"* byval nonnull align 8 %9,
%"class.impala:: Sorter::TupleIterator"* byval nonnull align 8 %3)
Edit
--
To view, visit http://gerrit.cloudera.org:8080/16621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b
Gerrit-Change-Number: 16621
Gerrit-PatchSet: 8
Gerrit-Owner: Qifan Chen
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Qifan
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc File be/src/runtime/sorter.cc: http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc@1215 PS8, Line 1215: llvm::Function* fn = codegen->GetFunction(IRFunction::TUPLE_SORTER_SORT_HELPER, true); > Hmm, the 29% improvement sounds a bit too much if this only optimized the f We don't anywhere else. https://gerrit.cloudera.org/#/c/12828/ have a complicated but complete solution do doing this. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 8 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 27 Oct 2020 17:32:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc File be/src/runtime/sorter.cc: http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc@1215 PS8, Line 1215: llvm::Function* fn = codegen->GetFunction(IRFunction::TUPLE_SORTER_SORT_HELPER, true); > I think there's an issue here - we don't replace recursive calls to SortHel Hmm, the 29% improvement sounds a bit too much if this only optimized the first pass over the runs. Tim, do you know another example where we cross-compile/codegen a recursive function? -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 8 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 27 Oct 2020 16:38:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc File be/src/runtime/sorter.cc: http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc@1215 PS8, Line 1215: llvm::Function* fn = codegen->GetFunction(IRFunction::TUPLE_SORTER_SORT_HELPER, true); I think there's an issue here - we don't replace recursive calls to SortHelper() as far as I can tell. So the first specialized SortHelper() call will end up calling a non-specalized SortHelper() version recursively and you'll only get the codegen benefit for the first one. I think this might be easy to fix by finding all the calls to the original TUPLE_SORTER_SORT_HELPER and replacing them with calls to fn. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 8 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 26 Oct 2020 17:58:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7556/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 8 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 26 Oct 2020 15:57:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. IMPALA-3816: Codegen perf critical loops in Sorter This fix added the functionality to codegen method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit test to order 7300 rows from table functional.alltypes, the speedup of the code-gen version over non-code-gen version of the method is around 29%. In another unit test to partially order 2880404 rows, the speedup is around 20%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 123 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/8 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 8 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Qifan Chen has posted comments on this change. (
http://gerrit.cloudera.org:8080/16621 )
Change subject: IMPALA-3816 Codegen perf critical loops in Sorter
..
Patch Set 7:
(8 comments)
Thanks a lot Csaba and Daniel!
On the comment "if 'CodegenFnPtr
codegend_sort_helper_fn_' could be owned by Sorter::TupleSorter instead of
SortPlanNode and PartialSortPlanNode. It seems that Sorter::TupleSorter is
responsible for both codegening and calling it."
There is a timing between the codegen and the use of the codegened method,
where the former is done for the plan node in general throughout the code base,
originated from FragmentInstanceState::Open() via InvokeCodegen().
http://gerrit.cloudera.org:8080/#/c/16621/7//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/16621/7//COMMIT_MSG@7
PS7, Line 7: IMPALA-3816 Codegen perf critical loops in Sorter
> nit: missing :
Done
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/exec/partial-sort-node.h
File be/src/exec/partial-sort-node.h:
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/exec/partial-sort-node.h@45
PS7, Line 45: TupelSorter
> Typo: TupleSorter.
Done
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/exec/partial-sort-node.cc
File be/src/exec/partial-sort-node.cc:
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/exec/partial-sort-node.cc@26
PS7, Line 26: #include "runtime/sorter-internal.h"
> Nit: This include should come before "util/runtime-profile-counters.h" beca
Done
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/exec/sort-node.cc
File be/src/exec/sort-node.cc:
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/exec/sort-node.cc@27
PS7, Line 27: #include "codegen/llvm-codegen.h"
: #include "runtime/sorter-internal.h"
> Should be sorted alphabetically.
Done
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter-internal.h
File be/src/runtime/sorter-internal.h:
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter-internal.h@440
PS7, Line 440: attemp
> typo: attempt
Done
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter-internal.h@442
PS7, Line 442: GENERAL
> The implementation doesn't return Status("GENERAL") but Status("Sorter::Tup
Done
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter.h
File be/src/runtime/sorter.h:
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter.h@106
PS7, Line 106: code-gen
> Nit: we usually write it as 'codegen'.
Done
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter.cc
File be/src/runtime/sorter.cc:
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter.cc@35
PS7, Line 35: #include "runtime/fragment-state.h"
> The included headers are generally in the block above, sorted alphabeticall
Good to know! Done.
--
To view, visit http://gerrit.cloudera.org:8080/16621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b
Gerrit-Change-Number: 16621
Gerrit-PatchSet: 7
Gerrit-Owner: Qifan Chen
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Qifan Chen
Gerrit-Reviewer: Sahil Takiar
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Mon, 26 Oct 2020 14:02:00 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Daniel Becker has posted comments on this change. (
http://gerrit.cloudera.org:8080/16621 )
Change subject: IMPALA-3816 Codegen perf critical loops in Sorter
..
Patch Set 7:
(5 comments)
Great work! Aside from the nits, I wonder if
'CodegenFnPtr codegend_sort_helper_fn_' could be owned by
Sorter::TupleSorter instead of SortPlanNode and PartialSortPlanNode. It seems
that Sorter::TupleSorter is responsible for both codegening and calling it.
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/exec/partial-sort-node.h
File be/src/exec/partial-sort-node.h:
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/exec/partial-sort-node.h@45
PS7, Line 45: TupelSorter
Typo: TupleSorter.
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/exec/partial-sort-node.cc
File be/src/exec/partial-sort-node.cc:
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/exec/partial-sort-node.cc@26
PS7, Line 26: #include "runtime/sorter-internal.h"
Nit: This include should come before "util/runtime-profile-counters.h" because
of the alphabetical order.
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/exec/sort-node.cc
File be/src/exec/sort-node.cc:
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/exec/sort-node.cc@27
PS7, Line 27: #include "codegen/llvm-codegen.h"
: #include "runtime/sorter-internal.h"
Should be sorted alphabetically.
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter-internal.h
File be/src/runtime/sorter-internal.h:
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter-internal.h@442
PS7, Line 442: GENERAL
The implementation doesn't return Status("GENERAL") but
Status("Sorter::TupleSorter::Codegen(): failed to finalize function"). The
specific error message is better, so this doc comment should be updated.
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter.h
File be/src/runtime/sorter.h:
http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter.h@106
PS7, Line 106: code-gen
Nit: we usually write it as 'codegen'.
--
To view, visit http://gerrit.cloudera.org:8080/16621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b
Gerrit-Change-Number: 16621
Gerrit-PatchSet: 7
Gerrit-Owner: Qifan Chen
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Sahil Takiar
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Mon, 26 Oct 2020 10:53:09 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816 Codegen perf critical loops in Sorter .. Patch Set 7: Code-Review+1 (3 comments) lgtm, only nits http://gerrit.cloudera.org:8080/#/c/16621/7//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16621/7//COMMIT_MSG@7 PS7, Line 7: IMPALA-3816 Codegen perf critical loops in Sorter nit: missing : http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter-internal.h File be/src/runtime/sorter-internal.h: http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter-internal.h@440 PS7, Line 440: attemp typo: attempt http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter.cc File be/src/runtime/sorter.cc: http://gerrit.cloudera.org:8080/#/c/16621/7/be/src/runtime/sorter.cc@35 PS7, Line 35: #include "runtime/fragment-state.h" The included headers are generally in the block above, sorted alphabetically. #include "common/names.h" is a special one that comes last, I am not sure why though :) -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 7 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 26 Oct 2020 09:04:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816 Codegen perf critical loops in Sorter .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7531/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 7 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 22 Oct 2020 20:55:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816 Codegen perf critical loops in Sorter .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7530/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 6 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 22 Oct 2020 20:42:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816 Codegen perf critical loops in Sorter .. IMPALA-3816 Codegen perf critical loops in Sorter This fix added the functionality to codegen method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit test to order 7300 rows from table functional.alltypes, the speedup of the code-gen version over non-code-gen version of the method is around 29%. In another unit test to partially order 2880404 rows, the speedup is around 20%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 122 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/7 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 7 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816 Codegen perf critical loops in Sorter .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/16621/6/be/src/runtime/sorter.h File be/src/runtime/sorter.h: http://gerrit.cloudera.org:8080/#/c/16621/6/be/src/runtime/sorter.h@106 PS6, Line 106: /// codegend_sort_helper_fn is a reference to the code-gen version of line has trailing whitespace -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 6 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 22 Oct 2020 20:25:15 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816 Codegen perf critical loops in Sorter .. IMPALA-3816 Codegen perf critical loops in Sorter This fix added the functionality to codegen method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit test to order 7300 rows from table functional.alltypes, the speedup of the code-gen version over non-code-gen version of the method is around 29%. In another unit test to partially order 2880404 rows, the speedup is around 20%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 122 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/6 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 6 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816 Codegen perf critical loops in Sorter .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7522/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 4 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 22 Oct 2020 19:12:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816 Codegen perf critical loops in Sorter .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7521/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 3 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 22 Oct 2020 18:56:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/16621 )
Change subject: IMPALA-3816 Codegen perf critical loops in Sorter
..
Patch Set 4:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/16621/4/be/src/runtime/sorter-internal.h
File be/src/runtime/sorter-internal.h:
http://gerrit.cloudera.org:8080/#/c/16621/4/be/src/runtime/sorter-internal.h@487
PS4, Line 487: Status IR_ALWAYS_INLINE InsertionSort(const TupleIterator&
begin, const TupleIterator& end);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16621/4/be/src/runtime/sorter-ir.cc
File be/src/runtime/sorter-ir.cc:
http://gerrit.cloudera.org:8080/#/c/16621/4/be/src/runtime/sorter-ir.cc@68
PS4, Line 68: bool IR_ALWAYS_INLINE Sorter::TupleSorter::Less(const TupleRow*
lhs, const TupleRow* rhs) {
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/16621/4/be/src/runtime/sorter-ir.cc@188
PS4, Line 188: Tuple* IR_ALWAYS_INLINE
Sorter::TupleSorter::SelectPivot(TupleIterator begin, TupleIterator end) {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16621/4/be/src/runtime/sorter-ir.cc@212
PS4, Line 212: Tuple* IR_ALWAYS_INLINE
Sorter::TupleSorter::MedianOfThree(Tuple* t1, Tuple* t2, Tuple* t3) {
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16621/4/be/src/runtime/sorter-ir.cc@249
PS4, Line 249: void IR_ALWAYS_INLINE Sorter::TupleSorter::Swap(Tuple* RESTRICT
left, Tuple* RESTRICT right,
line too long (92 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/16621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b
Gerrit-Change-Number: 16621
Gerrit-PatchSet: 4
Gerrit-Owner: Qifan Chen
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Thu, 22 Oct 2020 18:50:59 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816 Codegen perf critical loops in Sorter .. IMPALA-3816 Codegen perf critical loops in Sorter This fix added the functionality to codegen method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit testing to order 7300 rows from functional.alltypes, the speedup of the code-gen version of sort node is 29%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 112 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/4 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 4 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816 Codegen perf critical loops in Sorter .. IMPALA-3816 Codegen perf critical loops in Sorter This fix added the functionality to codegen method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 112 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/3 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 3 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-3816 Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/16621 )
Change subject: IMPALA-3816 Codegen perf critical loops in Sorter
..
Patch Set 3:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/16621/3/be/src/runtime/sorter-internal.h
File be/src/runtime/sorter-internal.h:
http://gerrit.cloudera.org:8080/#/c/16621/3/be/src/runtime/sorter-internal.h@487
PS3, Line 487: Status IR_ALWAYS_INLINE InsertionSort(const TupleIterator&
begin, const TupleIterator& end);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16621/3/be/src/runtime/sorter-ir.cc
File be/src/runtime/sorter-ir.cc:
http://gerrit.cloudera.org:8080/#/c/16621/3/be/src/runtime/sorter-ir.cc@68
PS3, Line 68: bool IR_ALWAYS_INLINE Sorter::TupleSorter::Less(const TupleRow*
lhs, const TupleRow* rhs) {
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/16621/3/be/src/runtime/sorter-ir.cc@188
PS3, Line 188: Tuple* IR_ALWAYS_INLINE
Sorter::TupleSorter::SelectPivot(TupleIterator begin, TupleIterator end) {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16621/3/be/src/runtime/sorter-ir.cc@212
PS3, Line 212: Tuple* IR_ALWAYS_INLINE
Sorter::TupleSorter::MedianOfThree(Tuple* t1, Tuple* t2, Tuple* t3) {
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16621/3/be/src/runtime/sorter-ir.cc@249
PS3, Line 249: void IR_ALWAYS_INLINE Sorter::TupleSorter::Swap(Tuple* RESTRICT
left, Tuple* RESTRICT right,
line too long (92 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/16621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b
Gerrit-Change-Number: 16621
Gerrit-PatchSet: 3
Gerrit-Owner: Qifan Chen
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Thu, 22 Oct 2020 18:41:58 +
Gerrit-HasComments: Yes
