[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. IMPALA-10766: Better selectivity for =,not distinct For = : If the right side is null, then selectivity is 0. If the left side is null, null should be excluded when calculating selectivity. For is not distinct from : If the right side is null, non null should be excluded when calculating selectivity, and only null should be included. If the left side is null and the right side is not null, null should be excluded when calculating selectivity, including part of non-null. Testing : Change the UT, modify the selectivity calculation error, add two new cases column != null and column = null Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Reviewed-on: http://gerrit.cloudera.org:8080/17637 Tested-by: Impala Public Jenkins Reviewed-by: Qifan Chen --- M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M testdata/workloads/functional-planner/queries/PlannerTest/card-inner-join.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/joins.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q08.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q13.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q16.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q19.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q24a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q24b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q30.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q33.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q42.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q44.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q48.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q52.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q55.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q56.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q60.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q61.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q66.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q71.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q75.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q80.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q81.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q84.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q85.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q91.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q94.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q95.test 32 files changed, 1,203 insertions(+), 1,186 deletions(-) Approvals: Impala Public Jenkins: Verified Qifan Chen: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 8 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7294/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 7 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Tue, 13 Jul 2021 13:35:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 7: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/17637/6/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java File fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java: http://gerrit.cloudera.org:8080/#/c/17637/6/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@288 PS6, Line 288: selectivity_ = selectivity_ * (double) (numRows - numNulls) / numRows + : numNulls / numRows; > This calculation formula may be (1- 1/ndv)* (numRows - numNulls) / numRows Done -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 7 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Tue, 13 Jul 2021 13:26:51 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 7 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Tue, 13 Jul 2021 08:43:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7292/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 7 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Tue, 13 Jul 2021 02:37:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
liuyao has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/17637/6/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java File fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java: http://gerrit.cloudera.org:8080/#/c/17637/6/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@288 PS6, Line 288: selectivity_ = selectivity_ * (double) (numRows - numNulls) / numRows + : numNulls / numRows; > nit. In this case, can we use selectivity = #nulls / #rows directly? This calculation formula may be (1- 1/ndv)* (numRows - numNulls) / numRows + numNulls / numRows. Some non-null values satisfy "is distinct from not-null". ie. 'aa' is distinct from 'bb' -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 7 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Tue, 13 Jul 2021 02:36:56 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/17637/6/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java File fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java: http://gerrit.cloudera.org:8080/#/c/17637/6/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@288 PS6, Line 288: selectivity_ = selectivity_ * (double) (numRows - numNulls) / numRows + : numNulls / numRows; nit. In this case, can we use selectivity = #nulls / #rows directly? -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 6 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Mon, 12 Jul 2021 17:28:16 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 6: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7285/ -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 6 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Mon, 12 Jul 2021 14:46:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9072/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 6 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Mon, 12 Jul 2021 09:01:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7285/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 6 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Mon, 12 Jul 2021 08:43:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
liuyao has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 6: (3 comments) http://gerrit.cloudera.org:8080/#/c/17637/5/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java File fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java: http://gerrit.cloudera.org:8080/#/c/17637/5/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@250 PS5, Line 250: rChildIsNull > nit. Do we need to worry about left child is null here, or not since the le Considering this special situation, the selectivity will be more accurate. http://gerrit.cloudera.org:8080/#/c/17637/5/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@280 PS5, Line 280: // For =, !=, "is distinct from null" and "is not distinct from non-null", : // all null values are false. : selectivity_ *= (double) (numRows - numNulls) / numRows; > nit. may combine them into a single sentence: Done http://gerrit.cloudera.org:8080/#/c/17637/5/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@284 PS5, Line 284: stinct from null, only null values are > Another case here is "is distinct from not-null". Done -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 6 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Mon, 12 Jul 2021 08:40:21 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Hello Aman Sinha, Qifan Chen, Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17637 to look at the new patch set (#6). Change subject: IMPALA-10766: Better selectivity for =,not distinct .. IMPALA-10766: Better selectivity for =,not distinct For = : If the right side is null, then selectivity is 0. If the left side is null, null should be excluded when calculating selectivity. For is not distinct from : If the right side is null, non null should be excluded when calculating selectivity, and only null should be included. If the left side is null and the right side is not null, null should be excluded when calculating selectivity, including part of non-null. Testing : Change the UT, modify the selectivity calculation error, add two new cases column != null and column = null Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 --- M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M testdata/workloads/functional-planner/queries/PlannerTest/card-inner-join.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/joins.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q08.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q13.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q16.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q19.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q24a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q24b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q30.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q33.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q42.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q44.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q48.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q52.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q55.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q56.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q60.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q61.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q66.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q71.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q75.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q80.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q81.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q84.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q85.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q91.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q94.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q95.test 32 files changed, 1,203 insertions(+), 1,186 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/17637/6 -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 6 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 5: (3 comments) Looks very good. Thanks a lot for the follow-up work! http://gerrit.cloudera.org:8080/#/c/17637/5/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java File fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java: http://gerrit.cloudera.org:8080/#/c/17637/5/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@250 PS5, Line 250: rChildIsNull nit. Do we need to worry about left child is null here, or not since the left child is always a column reference? http://gerrit.cloudera.org:8080/#/c/17637/5/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@280 PS5, Line 280: // For = and !=, all null values are false : // For is distinct from null, all null values are false : // For is not distinct from non-null, all null values are false nit. may combine them into a single sentence: For =, !=, "is distinct from null" and "is not distinct from non-null", all null values must be excluded. http://gerrit.cloudera.org:8080/#/c/17637/5/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@284 PS5, Line 284: Operator.NOT_DISTINCT && rChildIsNull) Another case here is "is distinct from not-null". -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 5 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Fri, 09 Jul 2021 17:55:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 5: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 5 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Fri, 09 Jul 2021 12:33:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9061/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 5 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Fri, 09 Jul 2021 10:10:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
liuyao has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 5: (3 comments) http://gerrit.cloudera.org:8080/#/c/17637/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17637/4//COMMIT_MSG@20 PS4, Line 20: Testin > Testing Done http://gerrit.cloudera.org:8080/#/c/17637/4/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java File fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java: http://gerrit.cloudera.org:8080/#/c/17637/4/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@253 PS4, Line 253: ! > != Done http://gerrit.cloudera.org:8080/#/c/17637/4/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@283 PS4, Line 283: (numRows - numNulls) / numRows; > nit: I think it's worth to extract it to a variable just like numRows Done -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 5 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: liuyao Gerrit-Comment-Date: Fri, 09 Jul 2021 09:52:41 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Hello Aman Sinha, Qifan Chen, Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17637 to look at the new patch set (#5). Change subject: IMPALA-10766: Better selectivity for =,not distinct .. IMPALA-10766: Better selectivity for =,not distinct For = : If the right side is null, then selectivity is 0. If the left side is null, null should be excluded when calculating selectivity. For is not distinct from : If the right side is null, non null should be excluded when calculating selectivity, and only null should be included. If the left side is null and the right side is not null, null should be excluded when calculating selectivity, including part of non-null. Testing : Change the UT, modify the selectivity calculation error, add two new cases column != null and column = null Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 --- M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M testdata/workloads/functional-planner/queries/PlannerTest/card-inner-join.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/joins.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q08.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q13.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q16.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q19.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q24a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q24b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q30.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q33.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q42.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q44.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q48.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q52.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q55.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q56.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q60.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q61.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q66.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q71.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q75.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q80.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q81.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q84.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q85.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q91.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q94.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q95.test 32 files changed, 1,198 insertions(+), 1,185 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/17637/5 -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 5 Gerrit-Owner: liuyao Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 4: Code-Review+1 (3 comments) Found a few nits, but otherwise LGTM. http://gerrit.cloudera.org:8080/#/c/17637/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17637/4//COMMIT_MSG@20 PS4, Line 20: Tesing Testing http://gerrit.cloudera.org:8080/#/c/17637/4/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java File fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java: http://gerrit.cloudera.org:8080/#/c/17637/4/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@253 PS4, Line 253: = != http://gerrit.cloudera.org:8080/#/c/17637/4/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@283 PS4, Line 283: slotDesc.getStats().getNumNulls() nit: I think it's worth to extract it to a variable just like numRows -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 4 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 09 Jul 2021 08:23:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 4 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 29 Jun 2021 09:04:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9021/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 4 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 29 Jun 2021 03:24:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7272/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 4 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 29 Jun 2021 03:07:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17637 to look at the new patch set (#4). Change subject: IMPALA-10766: Better selectivity for =,not distinct .. IMPALA-10766: Better selectivity for =,not distinct For = : If the right side is null, then selectivity is 0. If the left side is null, null should be excluded when calculating selectivity. For is not distinct from : If the right side is null, non null should be excluded when calculating selectivity, and only null should be included. If the left side is null and the right side is not null, null should be excluded when calculating selectivity, including part of non-null. Tesing : Change the UT, modify the selectivity calculation error, add two new cases column != null and column = null Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 --- M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M testdata/workloads/functional-planner/queries/PlannerTest/card-inner-join.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/joins.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q08.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q13.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q16.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q19.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q24a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q24b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q30.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q33.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q42.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q44.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q48.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q52.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q55.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q56.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q60.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q61.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q66.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q71.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q75.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q80.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q81.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q84.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q85.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q91.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q94.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q95.test 32 files changed, 1,198 insertions(+), 1,185 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/17637/4 -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 4 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7271/ -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 3 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 28 Jun 2021 13:42:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9018/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 3 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 28 Jun 2021 08:02:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7271/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 3 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 28 Jun 2021 07:40:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17637 to look at the new patch set (#3). Change subject: IMPALA-10766: Better selectivity for =,not distinct .. IMPALA-10766: Better selectivity for =,not distinct For = : If the right side is null, then selectivity is 0. If the left side is null, null should be excluded when calculating selectivity. For is not distinct from : If the right side is null, non null should be excluded when calculating selectivity, and only null should be included. If the left side is null and the right side is not null, null should be excluded when calculating selectivity, including part of non-null. Tesing : Change the UT, modify the selectivity calculation error, add two new cases column != null and column = null Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 --- M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M testdata/workloads/functional-planner/queries/PlannerTest/card-inner-join.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/joins.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q08.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q13.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q16.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q19.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q24a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q24b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q30.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q33.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q42.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q44.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q48.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q52.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q55.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q56.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q60.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q61.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q66.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q71.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q75.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q80.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q81.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q84.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q85.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q91.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q94.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q95.test 32 files changed, 1,188 insertions(+), 1,173 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/17637/3 -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 3 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7267/ -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 2 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 25 Jun 2021 16:02:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9007/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 2 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 25 Jun 2021 10:04:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7267/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 2 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 25 Jun 2021 09:59:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/9006/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 1 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 25 Jun 2021 09:58:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17637 to look at the new patch set (#2). Change subject: IMPALA-10766: Better selectivity for =,not distinct .. IMPALA-10766: Better selectivity for =,not distinct For = : If the right side is null, then selectivity is 0. If the left side is null, null should be excluded when calculating selectivity. For is not distinct from : If the right side is null, non null should be excluded when calculating selectivity, and only null should be included. If the left side is null and the right side is not null, null should be excluded when calculating selectivity, including part of non-null. Tesing : Change the UT, modify the selectivity calculation error, add two new cases column != null and column = null Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 --- M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M testdata/workloads/functional-planner/queries/PlannerTest/card-inner-join.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/joins.test 6 files changed, 70 insertions(+), 55 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/17637/2 -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 2 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17637 ) Change subject: IMPALA-10766: Better selectivity for =,not distinct .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/17637/1/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java File fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java: http://gerrit.cloudera.org:8080/#/c/17637/1/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@270 PS1, Line 270: line has trailing whitespace http://gerrit.cloudera.org:8080/#/c/17637/1/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@276 PS1, Line 276: if (op_ == Operator.EQ || op_ == Operator.NE line has trailing whitespace http://gerrit.cloudera.org:8080/#/c/17637/1/fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java File fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java: http://gerrit.cloudera.org:8080/#/c/17637/1/fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java@268 PS1, Line 268: line has trailing whitespace -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 1 Gerrit-Owner: liuyao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 25 Jun 2021 09:36:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10766: Better selectivity for =,not distinct
liuyao has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17637 Change subject: IMPALA-10766: Better selectivity for =,not distinct .. IMPALA-10766: Better selectivity for =,not distinct For = : If the right side is null, then selectivity is 0. If the left side is null, null should be excluded when calculating selectivity. For is not distinct from : If the right side is null, non null should be excluded when calculating selectivity, and only null should be included. If the left side is null and the right side is not null, null should be excluded when calculating selectivity, including part of non-null. Tesing : Change the UT, modify the selectivity calculation error, add two new cases column != null and column = null Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 --- M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M testdata/workloads/functional-planner/queries/PlannerTest/card-inner-join.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/joins.test 6 files changed, 70 insertions(+), 55 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/17637/1 -- To view, visit http://gerrit.cloudera.org:8080/17637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib8ec62f2355a7036125cc0d261b790644b9f4b60 Gerrit-Change-Number: 17637 Gerrit-PatchSet: 1 Gerrit-Owner: liuyao