[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 5: Thanks, Piotr for getting back. I'll take this one forward and submit a new PS that addresses the comments. -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 5 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 20 Nov 2018 00:26:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
piotr.findei...@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 5: Hi Bharath, sorry for being stucked on this. It will be probably better if you take it from here. -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 5 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Sun, 18 Nov 2018 21:58:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 5: Hi Piotr, Any updates? If you are busy and don't have time to look into this, please let us know and one of us can pick it up. Thanks. -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 5 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Sun, 11 Nov 2018 03:00:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Paul Rogers has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 5: (2 comments) A few comments. http://gerrit.cloudera.org:8080/#/c/11565/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11565/5//COMMIT_MSG@10 PS5, Line 10: IMPALA-7655. This fix would be helpful for IMPALA-7310: if NDV=0, we need to know if nullCount > 0 to know if we should adjust NDV upward by one. (The planner includes nulls in NDV, statistics don't.) http://gerrit.cloudera.org:8080/#/c/11565/5/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java File fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java: http://gerrit.cloudera.org:8080/#/c/11565/5/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@251 PS5, Line 251: " IS NULL THEN 1 ELSE NULL END)"); If this expression is faster than an if() function (IMPALA-7655), then does it makes sense to optimize the if() function in the planner so it benefits all queries rather than just changing this one? -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 5 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 18 Oct 2018 22:25:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 5: Looks like a bunch of tests failed because of the difference in stats: https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/3310/#showFailuresLink -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 5 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Reviewer: piotr.findei...@gmail.com Gerrit-Comment-Date: Mon, 08 Oct 2018 21:26:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 5: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3271/ -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 5 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Reviewer: piotr.findei...@gmail.com Gerrit-Comment-Date: Fri, 05 Oct 2018 02:29:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 5: I'll start a dry run just to see what tests fail. -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 5 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: piotr.findei...@gmail.com Gerrit-Comment-Date: Thu, 04 Oct 2018 22:46:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3271/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 5 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: piotr.findei...@gmail.com Gerrit-Comment-Date: Thu, 04 Oct 2018 22:46:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 5: The change in expression makes sense to me. I think we need someone who knows the planner better to evaluate whether we need more testing here. I suspect that tests will need modification to reflect the behaviour change (otherwise we have a testing gap). -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 5 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: piotr.findei...@gmail.com Gerrit-Comment-Date: Thu, 04 Oct 2018 22:46:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/943/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 5 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: piotr.findei...@gmail.com Gerrit-Comment-Date: Thu, 04 Oct 2018 20:26:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/942/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 4 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: piotr.findei...@gmail.com Gerrit-Comment-Date: Thu, 04 Oct 2018 20:18:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/941/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 3 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: piotr.findei...@gmail.com Gerrit-Comment-Date: Thu, 04 Oct 2018 20:03:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/11565/4/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java File fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java: http://gerrit.cloudera.org:8080/#/c/11565/4/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@250 PS4, Line 250: columnStatsSelectList.add("COUNT(CASE WHEN " + colRefSql + " IS NULL THEN 1 ELSE NULL END)"); line too long (99 > 90) -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 4 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: piotr.findei...@gmail.com Gerrit-Comment-Date: Thu, 04 Oct 2018 19:47:51 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
piotr.findei...@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 5: Thanks Tim for your help -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 5 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: piotr.findei...@gmail.com Gerrit-Comment-Date: Thu, 04 Oct 2018 19:51:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Hello Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11565 to look at the new patch set (#5). Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. IMPALA-7659: Simplify expression to collect NULLs count For reasoning behind replacing COUNT(IF(...)) with COUNT(CASE...) see IMPALA-7655. Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a --- M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java 1 file changed, 3 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/11565/5 -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 5 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: piotr.findei...@gmail.com
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Hello Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11565 to look at the new patch set (#4). Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. IMPALA-7659: Simplify expression to collect NULLs count For reasoning behind replacing COUNT(IF(...)) with COUNT(CASE...) see IMPALA-7655. Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a --- M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java 1 file changed, 2 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/11565/4 -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 4 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: piotr.findei...@gmail.com
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
piotr.findei...@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. Patch Set 3: > If you rewrite with a case expression instead of if you get > something thats much faster again. You something like: COUNT(CASE WHEN symbol IS NULL THEN NULL ELSE 1 END) ? -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 3 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: piotr.findei...@gmail.com Gerrit-Comment-Date: Thu, 04 Oct 2018 19:30:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Simplify expression to collect NULLs count
Hello Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11565 to look at the new patch set (#3). Change subject: IMPALA-7659: Simplify expression to collect NULLs count .. IMPALA-7659: Simplify expression to collect NULLs count Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a --- M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java 1 file changed, 2 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/11565/3 -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 3 Gerrit-Owner: piotr.findei...@gmail.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: piotr.findei...@gmail.com