[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. IMPALA-7231: group plan nodes into pipelines This adds some informational output to explain plans and sends the information to the backend. The idea is that this will make it easier to explain how Impala's pipelined execution works and also enable future work on profile analysis that can more intelligently group plan nodes. Tests: * Updated planner tests to include new output. Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Reviewed-on: http://gerrit.cloudera.org:8080/10848 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/planner/JoinNode.java A fe/src/main/java/org/apache/impala/planner/PipelineMembership.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SubplanNode.java M fe/src/main/java/org/apache/impala/planner/UnionNode.java M testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test M testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters.test M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test M testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering-disabled.test M testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test M testdata/workloads/functional-planner/queries/PlannerTest/partition-pruning.test M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test M testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test M testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test M testdata/workloads/functional-planner/queries/PlannerTest/union.test M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test M testdata/workloads/functional-query/queries/QueryTest/explain-level3.test M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test 25 files changed, 1,174 insertions(+), 21 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 19:31:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/284/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 16:41:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2976/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 16:10:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 5: Code-Review+2 Fix newly-added planner tests -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 16:10:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 5: (4 comments) http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/JoinNode.java File fe/src/main/java/org/apache/impala/planner/JoinNode.java: http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/JoinNode.java@702 PS5, Line 702: probePipeline.getId(), probePipeline.getHeight() + 1, TExecNodePhase.GETNEXT)); line too long (93 > 90) http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/PlanNode.java File fe/src/main/java/org/apache/impala/planner/PlanNode.java: http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/PlanNode.java@675 PS5, Line 675:childPipeline.getId(), childPipeline.getHeight() + 1, TExecNodePhase.GETNEXT)); line too long (94 > 90) http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/SubplanNode.java File fe/src/main/java/org/apache/impala/planner/SubplanNode.java: http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/SubplanNode.java@127 PS5, Line 127: leftPipeline.getId(), leftPipeline.getHeight() + 1, TExecNodePhase.GETNEXT)); line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/UnionNode.java File fe/src/main/java/org/apache/impala/planner/UnionNode.java: http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/UnionNode.java@336 PS5, Line 336: childPipeline.getId(), childPipeline.getHeight() + 1, TExecNodePhase.GETNEXT)); line too long (93 > 90) -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 16:10:09 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 16:10:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Hello Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10848 to look at the new patch set (#5). Change subject: IMPALA-7231: group plan nodes into pipelines .. IMPALA-7231: group plan nodes into pipelines This adds some informational output to explain plans and sends the information to the backend. The idea is that this will make it easier to explain how Impala's pipelined execution works and also enable future work on profile analysis that can more intelligently group plan nodes. Tests: * Updated planner tests to include new output. Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 --- M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/planner/JoinNode.java A fe/src/main/java/org/apache/impala/planner/PipelineMembership.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SubplanNode.java M fe/src/main/java/org/apache/impala/planner/UnionNode.java M testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test M testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters.test M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test M testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering-disabled.test M testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test M testdata/workloads/functional-planner/queries/PlannerTest/partition-pruning.test M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test M testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test M testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test M testdata/workloads/functional-planner/queries/PlannerTest/union.test M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test M testdata/workloads/functional-query/queries/QueryTest/explain-level3.test M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test 25 files changed, 1,174 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/10848/5 -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 4: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2972/ -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 03:15:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/278/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 00:42:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 4: Code-Review+2 Fix missing license header -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 00:14:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2972/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 00:14:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2970/ -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 00:14:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 4: (4 comments) http://gerrit.cloudera.org:8080/#/c/10848/4/fe/src/main/java/org/apache/impala/planner/JoinNode.java File fe/src/main/java/org/apache/impala/planner/JoinNode.java: http://gerrit.cloudera.org:8080/#/c/10848/4/fe/src/main/java/org/apache/impala/planner/JoinNode.java@702 PS4, Line 702: probePipeline.getId(), probePipeline.getHeight() + 1, TExecNodePhase.GETNEXT)); line too long (93 > 90) http://gerrit.cloudera.org:8080/#/c/10848/4/fe/src/main/java/org/apache/impala/planner/PlanNode.java File fe/src/main/java/org/apache/impala/planner/PlanNode.java: http://gerrit.cloudera.org:8080/#/c/10848/4/fe/src/main/java/org/apache/impala/planner/PlanNode.java@675 PS4, Line 675:childPipeline.getId(), childPipeline.getHeight() + 1, TExecNodePhase.GETNEXT)); line too long (94 > 90) http://gerrit.cloudera.org:8080/#/c/10848/4/fe/src/main/java/org/apache/impala/planner/SubplanNode.java File fe/src/main/java/org/apache/impala/planner/SubplanNode.java: http://gerrit.cloudera.org:8080/#/c/10848/4/fe/src/main/java/org/apache/impala/planner/SubplanNode.java@127 PS4, Line 127: leftPipeline.getId(), leftPipeline.getHeight() + 1, TExecNodePhase.GETNEXT)); line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/10848/4/fe/src/main/java/org/apache/impala/planner/UnionNode.java File fe/src/main/java/org/apache/impala/planner/UnionNode.java: http://gerrit.cloudera.org:8080/#/c/10848/4/fe/src/main/java/org/apache/impala/planner/UnionNode.java@336 PS4, Line 336: childPipeline.getId(), childPipeline.getHeight() + 1, TExecNodePhase.GETNEXT)); line too long (93 > 90) -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 00:14:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Hello Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10848 to look at the new patch set (#4). Change subject: IMPALA-7231: group plan nodes into pipelines .. IMPALA-7231: group plan nodes into pipelines This adds some informational output to explain plans and sends the information to the backend. The idea is that this will make it easier to explain how Impala's pipelined execution works and also enable future work on profile analysis that can more intelligently group plan nodes. Tests: * Updated planner tests to include new output. Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 --- M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/planner/JoinNode.java A fe/src/main/java/org/apache/impala/planner/PipelineMembership.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SubplanNode.java M fe/src/main/java/org/apache/impala/planner/UnionNode.java M testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test M testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters.test M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test M testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test M testdata/workloads/functional-planner/queries/PlannerTest/partition-pruning.test M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test M testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test M testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test M testdata/workloads/functional-planner/queries/PlannerTest/union.test M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test M testdata/workloads/functional-query/queries/QueryTest/explain-level3.test M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test 24 files changed, 1,131 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/10848/4 -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 3: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/277/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 10 Aug 2018 00:09:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 3: (4 comments) http://gerrit.cloudera.org:8080/#/c/10848/3/fe/src/main/java/org/apache/impala/planner/JoinNode.java File fe/src/main/java/org/apache/impala/planner/JoinNode.java: http://gerrit.cloudera.org:8080/#/c/10848/3/fe/src/main/java/org/apache/impala/planner/JoinNode.java@702 PS3, Line 702: probePipeline.getId(), probePipeline.getHeight() + 1, TExecNodePhase.GETNEXT)); line too long (93 > 90) http://gerrit.cloudera.org:8080/#/c/10848/3/fe/src/main/java/org/apache/impala/planner/PlanNode.java File fe/src/main/java/org/apache/impala/planner/PlanNode.java: http://gerrit.cloudera.org:8080/#/c/10848/3/fe/src/main/java/org/apache/impala/planner/PlanNode.java@675 PS3, Line 675:childPipeline.getId(), childPipeline.getHeight() + 1, TExecNodePhase.GETNEXT)); line too long (94 > 90) http://gerrit.cloudera.org:8080/#/c/10848/3/fe/src/main/java/org/apache/impala/planner/SubplanNode.java File fe/src/main/java/org/apache/impala/planner/SubplanNode.java: http://gerrit.cloudera.org:8080/#/c/10848/3/fe/src/main/java/org/apache/impala/planner/SubplanNode.java@127 PS3, Line 127: leftPipeline.getId(), leftPipeline.getHeight() + 1, TExecNodePhase.GETNEXT)); line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/10848/3/fe/src/main/java/org/apache/impala/planner/UnionNode.java File fe/src/main/java/org/apache/impala/planner/UnionNode.java: http://gerrit.cloudera.org:8080/#/c/10848/3/fe/src/main/java/org/apache/impala/planner/UnionNode.java@336 PS3, Line 336: childPipeline.getId(), childPipeline.getHeight() + 1, TExecNodePhase.GETNEXT)); line too long (93 > 90) -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 09 Aug 2018 23:40:54 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2970/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 09 Aug 2018 23:40:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Hello Joe McDonnell, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10848 to look at the new patch set (#3). Change subject: IMPALA-7231: group plan nodes into pipelines .. IMPALA-7231: group plan nodes into pipelines This adds some informational output to explain plans and sends the information to the backend. The idea is that this will make it easier to explain how Impala's pipelined execution works and also enable future work on profile analysis that can more intelligently group plan nodes. Tests: * Updated planner tests to include new output. Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 --- M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/planner/JoinNode.java A fe/src/main/java/org/apache/impala/planner/PipelineMembership.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SubplanNode.java M fe/src/main/java/org/apache/impala/planner/UnionNode.java M testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test M testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters.test M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test M testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test M testdata/workloads/functional-planner/queries/PlannerTest/partition-pruning.test M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test M testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test M testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test M testdata/workloads/functional-planner/queries/PlannerTest/union.test M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test M testdata/workloads/functional-query/queries/QueryTest/explain-level3.test M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test 24 files changed, 1,114 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/10848/3 -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 3: Code-Review+2 Clean rebase -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 09 Aug 2018 23:40:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 2: Code-Review+2 I think this makes sense. It makes sense to get it in. -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 08 Aug 2018 23:38:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 2: Any more thoughts? -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 08 Aug 2018 21:43:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. Patch Set 2: I took a first pass. I think this is mostly straight forward. The one exception is the union case, which I need to think about a bit more. -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 09 Jul 2018 17:54:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines
Tim Armstrong has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/10848 ) Change subject: IMPALA-7231: group plan nodes into pipelines .. IMPALA-7231: group plan nodes into pipelines This adds some informational output to explain plans and sends the information to the backend. The idea is that this will make it easier to explain how Impala's pipelined execution works and also enable future work on profile analysis that can more intelligently group plan nodes. Tests: * Updated planner tests to include new output. Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 --- M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/planner/JoinNode.java A fe/src/main/java/org/apache/impala/planner/PipelineMembership.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SubplanNode.java M fe/src/main/java/org/apache/impala/planner/UnionNode.java M testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test M testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters.test M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test M testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test M testdata/workloads/functional-planner/queries/PlannerTest/partition-pruning.test M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test M testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test M testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test M testdata/workloads/functional-planner/queries/PlannerTest/union.test M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test M testdata/workloads/functional-query/queries/QueryTest/explain-level3.test M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test 24 files changed, 1,114 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/10848/2 -- To view, visit http://gerrit.cloudera.org:8080/10848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721 Gerrit-Change-Number: 10848 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong
