[Impala-ASF-CR] (WIP) IMPALA-10262: RPM/DEB Packaging Support
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/16612 ) Change subject: (WIP) IMPALA-10262: RPM/DEB Packaging Support .. Patch Set 2: (1 comment) Thanks for uploading the patch! I'll support this. > Installing to /opt/impala seems reasonable or is /usr better? I think /usr/bin is good for executable binaries. So they occur in PATH automatically. http://gerrit.cloudera.org:8080/#/c/16612/2/CMakeLists.txt File CMakeLists.txt: http://gerrit.cloudera.org:8080/#/c/16612/2/CMakeLists.txt@436 PS2, Line 436: impala-frontend-0.1-SNAPSHOT.jar This should be impala-frontend-4.0.0-SNAPSHOT.jar now. -- To view, visit http://gerrit.cloudera.org:8080/16612 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I535e24e151371f5f594c7a8a6d2d585d18cb694f Gerrit-Change-Number: 16612 Gerrit-PatchSet: 2 Gerrit-Owner: Shant Hovsepian Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 27 Oct 2020 10:22:51 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10132 Implement ds hll estimate bounds as string() function.
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16626 ) Change subject: IMPALA-10132 Implement ds_hll_estimate_bounds_as_string() function. .. Patch Set 2: (9 comments) Thanks for taking care of this implementation! The patch is in quite a good shape I just had some minor comments. http://gerrit.cloudera.org:8080/#/c/16626/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16626/2//COMMIT_MSG@7 PS2, Line 7: IMPALA-10132 Implement ds_hll_estimate_bounds_as_string() function. Could you mention that there are 2 versions of this function. One that accepts a kappa parameter and one that doesn't. Also it would help to understand this feature to explain this kappa parameter here and it's purpose. http://gerrit.cloudera.org:8080/#/c/16626/2//COMMIT_MSG@15 PS2, Line 15: butwith typo http://gerrit.cloudera.org:8080/#/c/16626/2//COMMIT_MSG@16 PS2, Line 16: ds_kll_cdf_as_string wrong function name, guess copy-pasted from the CDF change :) http://gerrit.cloudera.org:8080/#/c/16626/2/be/src/exprs/datasketches-common.h File be/src/exprs/datasketches-common.h: http://gerrit.cloudera.org:8080/#/c/16626/2/be/src/exprs/datasketches-common.h@43 PS2, Line 43: DS_DEFAULR_KAPPA typo http://gerrit.cloudera.org:8080/#/c/16626/2/be/src/exprs/datasketches-functions.h File be/src/exprs/datasketches-functions.h: http://gerrit.cloudera.org:8080/#/c/16626/2/be/src/exprs/datasketches-functions.h@50 PS2, Line 50: disinct typo http://gerrit.cloudera.org:8080/#/c/16626/2/be/src/exprs/datasketches-functions.h@55 PS2, Line 55: static StringVal DsHllEstimateBoundsAsString(FunctionContext* ctx, I wouldn't write this whole comment for this function. I'd rather mention that this is similar to the function above with the exception of this kappa param (and would explain this param.) No strong feelings here though, it's up to you. http://gerrit.cloudera.org:8080/#/c/16626/2/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test File testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test: http://gerrit.cloudera.org:8080/#/c/16626/2/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test@291 PS2, Line 291: select ds_hll_estimate_bounds_as_string(ds_kll_sketch(cast(f2 as float))) from functional_parquet.emptytable; Could you wrap these lines into 90 chars. If the output of a query is too long then there is no need to wrap but the comments and the queries shouldn't be longer than 90 chars for more readability. http://gerrit.cloudera.org:8080/#/c/16626/2/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test@313 PS2, Line 313: 4 Please add a test for fraction inputs as well. http://gerrit.cloudera.org:8080/#/c/16626/2/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test@315 PS2, Line 315: UDF ERROR: Kappa may not be less than 1 or greater than 3. I'd change this error msg to explicitly say these 3 values that are allowed. With the current msg one might have the impression that e.g. 1.5 is also a valid input. -- To view, visit http://gerrit.cloudera.org:8080/16626 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I46bf8263e8fd3877a087b9cb6f0d1a2392bb9153 Gerrit-Change-Number: 16626 Gerrit-PatchSet: 2 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 27 Oct 2020 11:43:10 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10234: Add support for cookie authentication to impala-shell
Attila Jeges has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16660 Change subject: IMPALA-10234: Add support for cookie authentication to impala-shell .. IMPALA-10234: Add support for cookie authentication to impala-shell IMPALA-8584 added support for cookie authentication to Impala. This change adds cookie authentication support to impala-shell as well when using 'hs2-http' protocol. Testing: - Unit tests were added to test cookie handling methods. - Tested e2e manually. Change-Id: Icb0bc6e0f58f236866ca9913a2e63d97d5148f51 --- M shell/ImpalaHttpClient.py A shell/cookie_util.py M shell/impala_client.py M shell/impala_shell.py A tests/shell/test_cookie_util.py 5 files changed, 307 insertions(+), 56 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/16660/1 -- To view, visit http://gerrit.cloudera.org:8080/16660 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Icb0bc6e0f58f236866ca9913a2e63d97d5148f51 Gerrit-Change-Number: 16660 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges
[Impala-ASF-CR] IMPALA-10234: Add support for cookie authentication to impala-shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16660 ) Change subject: IMPALA-10234: Add support for cookie authentication to impala-shell .. Patch Set 1: (11 comments) http://gerrit.cloudera.org:8080/#/c/16660/1/shell/ImpalaHttpClient.py File shell/ImpalaHttpClient.py: http://gerrit.cloudera.org:8080/#/c/16660/1/shell/ImpalaHttpClient.py@214 PS1, Line 214: o flake8: E501 line too long (92 > 90 characters) http://gerrit.cloudera.org:8080/#/c/16660/1/shell/cookie_util.py File shell/cookie_util.py: http://gerrit.cloudera.org:8080/#/c/16660/1/shell/cookie_util.py@25 PS1, Line 25: def cookie_matches_path(c, path): flake8: E302 expected 2 blank lines, found 1 http://gerrit.cloudera.org:8080/#/c/16660/1/shell/cookie_util.py@45 PS1, Line 45: e flake8: E722 do not use bare except' http://gerrit.cloudera.org:8080/#/c/16660/1/shell/cookie_util.py@58 PS1, Line 58: e flake8: E722 do not use bare except' http://gerrit.cloudera.org:8080/#/c/16660/1/shell/impala_client.py File shell/impala_client.py: http://gerrit.cloudera.org:8080/#/c/16660/1/shell/impala_client.py@400 PS1, Line 400: d flake8: E306 expected 1 blank line before a nested definition, found 0 http://gerrit.cloudera.org:8080/#/c/16660/1/shell/impala_client.py@402 PS1, Line 402: flake8: E202 whitespace before ')' http://gerrit.cloudera.org:8080/#/c/16660/1/tests/shell/test_cookie_util.py File tests/shell/test_cookie_util.py: http://gerrit.cloudera.org:8080/#/c/16660/1/tests/shell/test_cookie_util.py@24 PS1, Line 24: o flake8: E501 line too long (95 > 90 characters) http://gerrit.cloudera.org:8080/#/c/16660/1/tests/shell/test_cookie_util.py@74 PS1, Line 74: k flake8: E501 line too long (97 > 90 characters) http://gerrit.cloudera.org:8080/#/c/16660/1/tests/shell/test_cookie_util.py@76 PS1, Line 76: flake8: E501 line too long (96 > 90 characters) http://gerrit.cloudera.org:8080/#/c/16660/1/tests/shell/test_cookie_util.py@79 PS1, Line 79: d flake8: E303 too many blank lines (2) http://gerrit.cloudera.org:8080/#/c/16660/1/tests/shell/test_cookie_util.py@123 PS1, Line 123: r flake8: E501 line too long (93 > 90 characters) -- To view, visit http://gerrit.cloudera.org:8080/16660 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icb0bc6e0f58f236866ca9913a2e63d97d5148f51 Gerrit-Change-Number: 16660 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 27 Oct 2020 14:56:16 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10234: Add support for cookie authentication to impala-shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16660 ) Change subject: IMPALA-10234: Add support for cookie authentication to impala-shell .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7563/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16660 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icb0bc6e0f58f236866ca9913a2e63d97d5148f51 Gerrit-Change-Number: 16660 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 27 Oct 2020 15:15:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10234: Add support for cookie authentication to impala-shell
Attila Jeges has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/16660 ) Change subject: IMPALA-10234: Add support for cookie authentication to impala-shell .. IMPALA-10234: Add support for cookie authentication to impala-shell IMPALA-8584 added support for cookie authentication to Impala. This change adds cookie authentication support to impala-shell as well when using 'hs2-http' protocol. Testing: - Unit tests were added to test cookie handling methods. - Tested e2e manually. Change-Id: Icb0bc6e0f58f236866ca9913a2e63d97d5148f51 --- M shell/ImpalaHttpClient.py A shell/cookie_util.py M shell/impala_client.py M shell/impala_shell.py A tests/shell/test_cookie_util.py 5 files changed, 314 insertions(+), 56 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/16660/2 -- To view, visit http://gerrit.cloudera.org:8080/16660 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icb0bc6e0f58f236866ca9913a2e63d97d5148f51 Gerrit-Change-Number: 16660 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-10234: Add support for cookie authentication to impala-shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16660 ) Change subject: IMPALA-10234: Add support for cookie authentication to impala-shell .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/16660/2/tests/shell/test_cookie_util.py File tests/shell/test_cookie_util.py: http://gerrit.cloudera.org:8080/#/c/16660/2/tests/shell/test_cookie_util.py@25 PS2, Line 25: g flake8: E126 continuation line over-indented for hanging indent -- To view, visit http://gerrit.cloudera.org:8080/16660 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icb0bc6e0f58f236866ca9913a2e63d97d5148f51 Gerrit-Change-Number: 16660 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 27 Oct 2020 15:37:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10252: fix invalid runtime filters for outer joins
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16622 ) Change subject: IMPALA-10252: fix invalid runtime filters for outer joins .. Patch Set 5: Code-Review+1 (2 comments) http://gerrit.cloudera.org:8080/#/c/16622/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16622/4//COMMIT_MSG@17 PS4, Line 17: x = isnull(y, 1) can return true even if y is NULL. > > I wonder if the root cause is that the null rows from the inner do not pa Okay. Thanks a lot for trying the NULL row (to the filter) method. Yes, my question was motivated from the current fix by evaluating the expression with all nulls. Can you please explain the non-nullable slots/tuples in the inner? I thought for outer joins, the inner row produced must be nullable. Done. http://gerrit.cloudera.org:8080/#/c/16622/4/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java File fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java: http://gerrit.cloudera.org:8080/#/c/16622/4/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java@373 PS4, Line 373: filterSrcNode.getJoinOp().isLeftOuterJoin() || : filterSrcNode.getJoinOp().isFullOuterJoin()) > This is done towards the end of distributed planning when the join order an Good point. Thanks. Done. -- To view, visit http://gerrit.cloudera.org:8080/16622 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I507af1cc8df15bca21e0d8555019997812087261 Gerrit-Change-Number: 16622 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 27 Oct 2020 15:42:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10234: Add support for cookie authentication to impala-shell
Attila Jeges has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16660 ) Change subject: IMPALA-10234: Add support for cookie authentication to impala-shell .. IMPALA-10234: Add support for cookie authentication to impala-shell IMPALA-8584 added support for cookie authentication to Impala. This change adds cookie authentication support to impala-shell as well when using 'hs2-http' protocol. Testing: - Unit tests were added to test cookie handling methods. - Tested e2e manually. Change-Id: Icb0bc6e0f58f236866ca9913a2e63d97d5148f51 --- M shell/ImpalaHttpClient.py A shell/cookie_util.py M shell/impala_client.py M shell/impala_shell.py A tests/shell/test_cookie_util.py 5 files changed, 314 insertions(+), 56 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/16660/3 -- To view, visit http://gerrit.cloudera.org:8080/16660 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icb0bc6e0f58f236866ca9913a2e63d97d5148f51 Gerrit-Change-Number: 16660 Gerrit-PatchSet: 3 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-10234: Add support for cookie authentication to impala-shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16660 ) Change subject: IMPALA-10234: Add support for cookie authentication to impala-shell .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7564/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16660 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icb0bc6e0f58f236866ca9913a2e63d97d5148f51 Gerrit-Change-Number: 16660 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 27 Oct 2020 15:56:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10234: Add support for cookie authentication to impala-shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16660 ) Change subject: IMPALA-10234: Add support for cookie authentication to impala-shell .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7565/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16660 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icb0bc6e0f58f236866ca9913a2e63d97d5148f51 Gerrit-Change-Number: 16660 Gerrit-PatchSet: 3 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 27 Oct 2020 16:14:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10252: fix invalid runtime filters for outer joins
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16622 ) Change subject: IMPALA-10252: fix invalid runtime filters for outer joins .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/16622/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16622/4//COMMIT_MSG@17 PS4, Line 17: x = isnull(y, 1) can return true even if y is NULL. > Okay. Thanks a lot for trying the NULL row (to the filter) method. Yes, my https://cwiki.apache.org/confluence/display/IMPALA/Impala+Row+Batches describes the physical layout of the row batches. So each row is a composition of pointers to tuples. A join produces rows with a new layout which is the concatenation of the tuples of the input rows (except semi-joins, which retain the row layout from the outer). You can see that in the plan I included below, where a left join produces a row composed of tuples 1 and 2 from the inputs: tuple-ids=0,1N An unmatched row from an outer join is represented by a NULL tuple pointer. Nullability is represented in the row descriptor as an extra flag (that's the N in 1N above). Most tuples are non-nullable - that's only introduced by outer joins and aggregations with multiple agg classes IIRC. The runtime filter expressions in this example would be evaluated over the input row produced by operator 04, which has a single non-nullable tuple 1. The problem is that each expression tree in the planner is specific to a particular row layout, so to evaluate a runtime filter expression over a row with layout 1N instead of layout 1, we'd need to generate a new RowDescriptor, then clone the runtime filter expression and fixed it up by replacing all the SlotRefs with SlotRefs referencing the new row descriptor. I think we would also need to add some TupleIsNull() predicates in order to correctly handle the nullability (since SlotRef only handles the slot-level nullability, I think). Then we'd have to plumb the expression through to the backend so it can be evaluated. > explain select count(*) from functional.alltypes t1 left join functional.alltypestiny t2 on t1.id = t2.id; Query: explain select count(*) from functional.alltypes t1 left join functional.alltypestiny t2 on t1.id = t2.id +-+ | Explain String | +-+ | Max Per-Host Resource Reservation: Memory=1.98MB Threads=5 | | Per-Host Resource Estimates: Memory=214MB | | Codegen disabled by planner | | Analyzed query: SELECT count(*) FROM functional.alltypes t1 LEFT OUTER JOIN | | functional.alltypestiny t2 ON t1.id = t2.id | | | | F02:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | | | Per-Host Resources: mem-estimate=10.02MB mem-reservation=0B thread-reservation=1 | | PLAN-ROOT SINK | | | output exprs: count(*) | | | mem-estimate=0B mem-reservation=0B thread-reservation=0 | | | | | 06:AGGREGATE [FINALIZE] | | | output: count:merge(*) | | | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0 | | | tuple-ids=2 row-size=8B cardinality=1 | | | in pipelines: 06(GETNEXT), 03(OPEN) | | | | | 05:EXCHANGE [UNPARTITIONED] | | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0 | | | tuple-ids=2 row-size=8B cardinality=1 | | | in pipelines: 03(GETNEXT) | | | | | F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 | | Per-Host Resources: mem-estimate=171.94MB mem-reservation=1.97MB thread-reservation=2 | | 03
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc File be/src/runtime/sorter.cc: http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc@1215 PS8, Line 1215: llvm::Function* fn = codegen->GetFunction(IRFunction::TUPLE_SORTER_SORT_HELPER, true); > I think there's an issue here - we don't replace recursive calls to SortHel Hmm, the 29% improvement sounds a bit too much if this only optimized the first pass over the runs. Tim, do you know another example where we cross-compile/codegen a recursive function? -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 8 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 27 Oct 2020 16:38:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10294: Improvement to test skew reporting in runtime profile
Qifan Chen has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16662 Change subject: IMPALA-10294: Improvement to test_skew_reporting_in_runtime_profile .. IMPALA-10294: Improvement to test_skew_reporting_in_runtime_profile This fix improved the skew reporting test by lowering the threshold to 0 and by taking care of the extreme case of no skews. Together, they made the test more robust. Testing: 1. Unit testing. Change-Id: I7a36551f2507d724891707d26b7394fbe3a5657b --- M tests/query_test/test_observability.py 1 file changed, 10 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/16662/1 -- To view, visit http://gerrit.cloudera.org:8080/16662 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I7a36551f2507d724891707d26b7394fbe3a5657b Gerrit-Change-Number: 16662 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen
[Impala-ASF-CR] IMPALA-10294: Improvement to test skew reporting in runtime profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16662 ) Change subject: IMPALA-10294: Improvement to test_skew_reporting_in_runtime_profile .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/16662/1/tests/query_test/test_observability.py File tests/query_test/test_observability.py: http://gerrit.cloudera.org:8080/#/c/16662/1/tests/query_test/test_observability.py@753 PS1, Line 753: flake8: E203 whitespace before ':' -- To view, visit http://gerrit.cloudera.org:8080/16662 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7a36551f2507d724891707d26b7394fbe3a5657b Gerrit-Change-Number: 16662 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 27 Oct 2020 16:39:33 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10295: fix analytic limit pushdown with no predicates
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16663 Change subject: IMPALA-10295: fix analytic limit pushdown with no predicates .. IMPALA-10295: fix analytic limit pushdown with no predicates This handles the first case where analytic limit pushdown could be applied incorrectly: when there are no predicates applied to the output of the analytic. If no rows are filtered out between the pre-analytic sort and the place where the top-N will be inserted, and the order matches exactly, we can push down the limit safely because the limit below the analytic will filter exactly the same rows as the limit above the analytic would. We add a helper to check if the sort order matches exactly and then handle the case with no select node correctly. We leave the other cases where there is a special predicate to be handled in the next patch of the series, as the logic there is a bit more subtle. Tests: Added regression planner and query tests that demonstrate the problem. Ran core tests. Change-Id: I254e85edd5ea6b6e76d20cbdf27fd88059a98a21 --- M fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java M testdata/workloads/functional-planner/queries/PlannerTest/limit-pushdown-analytic.test M testdata/workloads/tpch/queries/limit-pushdown-analytic.test 3 files changed, 232 insertions(+), 38 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/16663/1 -- To view, visit http://gerrit.cloudera.org:8080/16663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I254e85edd5ea6b6e76d20cbdf27fd88059a98a21 Gerrit-Change-Number: 16663 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] IMPALA-10295: fix analytic limit pushdown with no predicates
Hello Aman Sinha, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16663 to look at the new patch set (#2). Change subject: IMPALA-10295: fix analytic limit pushdown with no predicates .. IMPALA-10295: fix analytic limit pushdown with no predicates This handles the first case where analytic limit pushdown could be applied incorrectly: when there are no predicates applied to the output of the analytic. If no rows are filtered out between the pre-analytic sort and the place where the top-N will be inserted, and the order matches exactly, we can push down the limit safely because the limit below the analytic will filter exactly the same rows as the limit above the analytic would. We add a helper to check if the sort order matches exactly and then handle the case with no select node correctly. We leave the other cases where there is a special predicate to be handled in the next patch of the series, as the logic there is a bit more subtle. Tests: Added regression planner and query tests that demonstrate the problem. Ran core tests. Change-Id: I254e85edd5ea6b6e76d20cbdf27fd88059a98a21 --- M fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java M testdata/workloads/functional-planner/queries/PlannerTest/limit-pushdown-analytic.test M testdata/workloads/tpch/queries/limit-pushdown-analytic.test 3 files changed, 234 insertions(+), 38 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/16663/2 -- To view, visit http://gerrit.cloudera.org:8080/16663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I254e85edd5ea6b6e76d20cbdf27fd88059a98a21 Gerrit-Change-Number: 16663 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10295: fix analytic limit pushdown with no predicates
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/16663 )
Change subject: IMPALA-10295: fix analytic limit pushdown with no predicates
..
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/16663/1/fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java
File fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java:
http://gerrit.cloudera.org:8080/#/c/16663/1/fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java@478
PS1, Line 478: if
(!sortInfo.getIsAscOrder().get(i).equals(analyticSortInfo.getIsAscOrder().get(i)))
{
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16663/1/fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java@481
PS1, Line 481: if
(!sortInfo.getNullsFirst().get(i).equals(analyticSortInfo.getNullsFirst().get(i)))
{
line too long (93 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/16663
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I254e85edd5ea6b6e76d20cbdf27fd88059a98a21
Gerrit-Change-Number: 16663
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Aman Sinha
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Tue, 27 Oct 2020 16:44:41 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10295: fix analytic limit pushdown with no predicates
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/16663 )
Change subject: IMPALA-10295: fix analytic limit pushdown with no predicates
..
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/16663/2/fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java
File fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java:
http://gerrit.cloudera.org:8080/#/c/16663/2/fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java@480
PS2, Line 480: if
(!sortInfo.getIsAscOrder().get(i).equals(analyticSortInfo.getIsAscOrder().get(i)))
{
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16663/2/fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java@483
PS2, Line 483: if
(!sortInfo.getNullsFirst().get(i).equals(analyticSortInfo.getNullsFirst().get(i)))
{
line too long (93 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/16663
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I254e85edd5ea6b6e76d20cbdf27fd88059a98a21
Gerrit-Change-Number: 16663
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Aman Sinha
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Tue, 27 Oct 2020 16:45:25 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10294: Improvement to test skew reporting in runtime profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16662 ) Change subject: IMPALA-10294: Improvement to test_skew_reporting_in_runtime_profile .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7566/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16662 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7a36551f2507d724891707d26b7394fbe3a5657b Gerrit-Change-Number: 16662 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 27 Oct 2020 16:59:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10295: fix analytic limit pushdown with no predicates
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16663 ) Change subject: IMPALA-10295: fix analytic limit pushdown with no predicates .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7567/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I254e85edd5ea6b6e76d20cbdf27fd88059a98a21 Gerrit-Change-Number: 16663 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 27 Oct 2020 17:05:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10295: fix analytic limit pushdown with no predicates
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16663 ) Change subject: IMPALA-10295: fix analytic limit pushdown with no predicates .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7568/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I254e85edd5ea6b6e76d20cbdf27fd88059a98a21 Gerrit-Change-Number: 16663 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 27 Oct 2020 17:05:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10279: Import CPC functionality from DataSketches
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16645 ) Change subject: IMPALA-10279: Import CPC functionality from DataSketches .. Patch Set 6: Hey, I'm just letting you know that I'm planning to take a look most probably tomorrow. -- To view, visit http://gerrit.cloudera.org:8080/16645 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1e8d0c2e80df95fa84af82c64d493df9bbb34a8c Gerrit-Change-Number: 16645 Gerrit-PatchSet: 6 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 27 Oct 2020 17:10:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc File be/src/runtime/sorter.cc: http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc@1215 PS8, Line 1215: llvm::Function* fn = codegen->GetFunction(IRFunction::TUPLE_SORTER_SORT_HELPER, true); > Hmm, the 29% improvement sounds a bit too much if this only optimized the f We don't anywhere else. https://gerrit.cloudera.org/#/c/12828/ have a complicated but complete solution do doing this. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 8 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 27 Oct 2020 17:32:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10252: fix invalid runtime filters for outer joins
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16622 ) Change subject: IMPALA-10252: fix invalid runtime filters for outer joins .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/16622/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16622/4//COMMIT_MSG@17 PS4, Line 17: x = isnull(y, 1) can return true even if y is NULL. > https://cwiki.apache.org/confluence/display/IMPALA/Impala+Row+Batches descr Okay. The tuple level nullability indicator in the plan is very helpful. I like it a lot. For this special case when prepare the filter for a join predicate non null-rejecting, I guess we need an extra step to compute the filter value for a null row from the inner due to the outer join. This is almost like we need to an expression on top of 4 to expand tuple 1 to 1N, where N is for the null row described above. The expression probably can take care of the descriptor clone. Anyway, I am very fine with the current fix. -- To view, visit http://gerrit.cloudera.org:8080/16622 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I507af1cc8df15bca21e0d8555019997812087261 Gerrit-Change-Number: 16622 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 27 Oct 2020 17:46:43 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10277: Fix test catalogd debug actions on S3 builds
Vihang Karajgaonkar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16664 Change subject: IMPALA-10277: Fix test_catalogd_debug_actions on S3 builds .. IMPALA-10277: Fix test_catalogd_debug_actions on S3 builds test_catalogd_debug_actions fails on S3 builds because the time taken to load a table on S3 is faster than on HDFS. The fix changes the test to reduce the expected delay when the debug actions are set so that test works on S3 builds. Testing: 1. Ran the test on the s3 build jenkins job. 2. [WIP] Running the test on HDFS builds. Change-Id: I8348a33ff8e9c7812540e014f4de2c65636da64f --- M tests/metadata/test_catalogd_debug_actions.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/64/16664/1 -- To view, visit http://gerrit.cloudera.org:8080/16664 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8348a33ff8e9c7812540e014f4de2c65636da64f Gerrit-Change-Number: 16664 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-10277: Fix test catalogd debug actions on S3 builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16664 ) Change subject: IMPALA-10277: Fix test_catalogd_debug_actions on S3 builds .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7569/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16664 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8348a33ff8e9c7812540e014f4de2c65636da64f Gerrit-Change-Number: 16664 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 27 Oct 2020 18:32:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has posted comments on this change. (
http://gerrit.cloudera.org:8080/16621 )
Change subject: IMPALA-3816: Codegen perf critical loops in Sorter
..
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc
File be/src/runtime/sorter.cc:
http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc@1215
PS8, Line 1215: llvm::Function* fn =
codegen->GetFunction(IRFunction::TUPLE_SORTER_SORT_HELPER, true);
> We don't anywhere else. https://gerrit.cloudera.org/#/c/12828/ have a compl
Looks like there is only on call to SortHelper() which is from
Sorter::TupleSorter::Sort() shown below.
752 Status Sorter::TupleSorter::Sort(Run* run) {
753 DCHECK(run->is_finalized());
754 DCHECK(!run->is_sorted());
755 run_ = run;
756 const SortHelperFn sort_helper_fn =
parent_->codegend_sort_helper_fn_.load();
757 if (sort_helper_fn != nullptr) {
758 RETURN_IF_ERROR(
759 sort_helper_fn(this, TupleIterator::Begin(run_),
TupleIterator::End(run_)));
760 } else {
761 RETURN_IF_ERROR(SortHelper(TupleIterator::Begin(run_),
TupleIterator::End(run_)));
762 }
763 run_->set_sorted();
764 return Status::OK();
765 }
The speedup may also come from the inlining of several small functions before
LLVM optimizing "the big chunk of code" :-).
I have collected the llvm code and uploaded it to the case here IMPALA-3816"
target="_blank"
rel="nofollow">https://issues.apache.org/jira/browse/IMPALA-3816.
Interestingly, there are 2 definitions (one fast cc and one regular) for the
SortHelp method and 4 calls to the fast cc version.
803 define internal fastcc void
@_ZN6impala6Sorter11TupleSorter10SortHelperENS0_13TupleIteratorES2_(%"class.impala::Status"*
noalias sr et, %"class.impala::Sorter::TupleSorter"*,
%"class.impala::Sorter::TupleIterator"* byval nocapture align 8,
%"class.impala::Sorter: :TupleIterator"* byval nocapture align 8)
unnamed_addr #3 align 2 personality i8* bitcast (i32 (...)*
@__gxx_personality_v0 to i8*)
1724 call fastcc void
@_ZN6impala6Sorter11TupleSorter10SortHelperENS0_13TupleIteratorES2_(%"class.impala::Status"*
nonnull sret %19, % "class.impala::Sorter::TupleSorter"* nonnull %1,
%"class.impala::Sorter::TupleIterator"* byval nonnull align 8 %2,
%"class.impala:: Sorter::TupleIterator"* byval nonnull align 8 %17)
1753 call fastcc void
@_ZN6impala6Sorter11TupleSorter10SortHelperENS0_13TupleIteratorES2_(%"class.impala::Status"*
nonnull sret %20, % "class.impala::Sorter::TupleSorter"* nonnull %1,
%"class.impala::Sorter::TupleIterator"* byval nonnull align 8 %17,
%"class.impala: :Sorter::TupleIterator"* byval nonnull align 8 %3)
2756 define void
@_ZN6impala6Sorter11TupleSorter10SortHelperENS0_13TupleIteratorES2_.3(%"class.impala::Status"*
noalias sret, %"class.im pala::Sorter::TupleSorter"*,
%"class.impala::Sorter::TupleIterator"* byval nocapture align 8,
%"class.impala::Sorter::TupleIterator "* byval nocapture align 8)
local_unnamed_addr #3 align 2 personality i8* bitcast (i32 (...)*
@__gxx_personality_v0 to i8*) {
3877 call fastcc void
@_ZN6impala6Sorter11TupleSorter10SortHelperENS0_13TupleIteratorES2_(%"class.impala::Status"*
nonnull sret %11, % "class.impala::Sorter::TupleSorter"* nonnull %1,
%"class.impala::Sorter::TupleIterator"* byval nonnull align 8 %2,
%"class.impala:: Sorter::TupleIterator"* byval nonnull align 8 %9)
3905 call fastcc void
@_ZN6impala6Sorter11TupleSorter10SortHelperENS0_13TupleIteratorES2_(%"class.impala::Status"*
nonnull sret %12, % "class.impala::Sorter::TupleSorter"* nonnull %1,
%"class.impala::Sorter::TupleIterator"* byval nonnull align 8 %9,
%"class.impala:: Sorter::TupleIterator"* byval nonnull align 8 %3)
Edit
--
To view, visit http://gerrit.cloudera.org:8080/16621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b
Gerrit-Change-Number: 16621
Gerrit-PatchSet: 8
Gerrit-Owner: Qifan Chen
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Qifan Ch
[Impala-ASF-CR] IMPALA-10206: Avoid MD5 Digest Authorization in FIPS mode
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/16630 ) Change subject: IMPALA-10206: Avoid MD5 Digest Authorization in FIPS mode .. Patch Set 4: (3 comments) http://gerrit.cloudera.org:8080/#/c/16630/4/be/src/thirdparty/squeasel/squeasel.c File be/src/thirdparty/squeasel/squeasel.c: http://gerrit.cloudera.org:8080/#/c/16630/4/be/src/thirdparty/squeasel/squeasel.c@1527 PS4, Line 1527: #ifndef USE_SQ_OWN_HASH_FUNCTIONS Does FIPS have a requirement that non-openssl crypto functions have to be compiled out, or is it good enough that they don't get used? Seems like given your change in webserver.cc that prevents us from using passwords files in FIPS that these functions won't ever actually get used. http://gerrit.cloudera.org:8080/#/c/16630/4/be/src/thirdparty/squeasel/squeasel.c@3247 PS4, Line 3247: #ifndef USE_SQ_OWN_HASH_FUNCTIONS Like above, seems like these functions aren't actually getting used. And in fact, I think these are already getting compiled out due to the USE_WEBSOCKET above, which I don't think we set. If we really do still want to make this change, I might suggest just completely deleting the squeasel functions and leave openssl as the only option. http://gerrit.cloudera.org:8080/#/c/16630/4/be/src/util/webserver.cc File be/src/util/webserver.cc: http://gerrit.cloudera.org:8080/#/c/16630/4/be/src/util/webserver.cc@401 PS4, Line 401: Don't support HTTP Digest Authorization in FIPS mode. nit: this is worded kind of strangely. Maybe "HTTP digest authorization is not supported in FIPS mode" -- To view, visit http://gerrit.cloudera.org:8080/16630 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie075389b3ab65c612d64ba58e16a10b19bdf4d6f Gerrit-Change-Number: 16630 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 27 Oct 2020 20:28:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10292 Improvement to test misaligned parquet row groups
Qifan Chen has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16668 Change subject: IMPALA-10292 Improvement to test_misaligned_parquet_row_groups .. IMPALA-10292 Improvement to test_misaligned_parquet_row_groups This fix improved the function _misaligned_parquet_row_groups_helper in test_scanners.py by removing the hardcoded three backends running a query. Instead, the number is fetched from the counter NumBackends in the profile. Testing: 1. Unit testing. Change-Id: Icd190bca840ffdad0f615d8bde059431cb13b984 --- M tests/query_test/test_scanners.py 1 file changed, 6 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/16668/2 -- To view, visit http://gerrit.cloudera.org:8080/16668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Icd190bca840ffdad0f615d8bde059431cb13b984 Gerrit-Change-Number: 16668 Gerrit-PatchSet: 2 Gerrit-Owner: Qifan Chen
[Impala-ASF-CR] IMPALA-10292 Improvement to test misaligned parquet row groups
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16668 ) Change subject: IMPALA-10292 Improvement to test_misaligned_parquet_row_groups .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7570/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icd190bca840ffdad0f615d8bde059431cb13b984 Gerrit-Change-Number: 16668 Gerrit-PatchSet: 2 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 27 Oct 2020 21:15:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10292 Improvement to test misaligned parquet row groups
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16668 ) Change subject: IMPALA-10292 Improvement to test_misaligned_parquet_row_groups .. Patch Set 2: The underlying reason the test fails in this case is that an impalad crashed, so there is the wrong number of impalads. Typically, when an impalad crashes, hundreds of tests fail. I see that as ok, and I don't think we should put much effort into making the tests resilient to the wrong number of impalads. The crash is the important part, and without the crash, this test will consistently pass. -- To view, visit http://gerrit.cloudera.org:8080/16668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icd190bca840ffdad0f615d8bde059431cb13b984 Gerrit-Change-Number: 16668 Gerrit-PatchSet: 2 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 27 Oct 2020 22:53:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10206: Avoid MD5 Digest Authorization in FIPS mode
Wenzhe Zhou has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/16630 ) Change subject: IMPALA-10206: Avoid MD5 Digest Authorization in FIPS mode .. IMPALA-10206: Avoid MD5 Digest Authorization in FIPS mode To compliant with FIPS requirement, we should use OpenSSL libraries for cryptographic hash functions, instead of own hash functions. This patch replace MD5 and SHA1 functions in Squeasel Web server with OpenSSL APIs. It also force to turn off Digest Authorization for Web server in FIPS mode since Digest Authorization use MD5 hash. Testing: - Passed exhaustive tests. - Manually verified HTTP Digest Authorization could not be enabled by setting webserver_password_file on a FIPS enabled cluster. Change-Id: Ie075389b3ab65c612d64ba58e16a10b19bdf4d6f --- M be/src/thirdparty/squeasel/squeasel.c M be/src/util/webserver.cc 2 files changed, 28 insertions(+), 333 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/16630/5 -- To view, visit http://gerrit.cloudera.org:8080/16630 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie075389b3ab65c612d64ba58e16a10b19bdf4d6f Gerrit-Change-Number: 16630 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-10206: Avoid MD5 Digest Authorization in FIPS mode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16630 ) Change subject: IMPALA-10206: Avoid MD5 Digest Authorization in FIPS mode .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7571/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16630 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie075389b3ab65c612d64ba58e16a10b19bdf4d6f Gerrit-Change-Number: 16630 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 27 Oct 2020 23:14:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10277: Fix test catalogd debug actions on S3 builds
Vihang Karajgaonkar has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/16664 ) Change subject: IMPALA-10277: Fix test_catalogd_debug_actions on S3 builds .. IMPALA-10277: Fix test_catalogd_debug_actions on S3 builds test_catalogd_debug_actions fails on S3 builds because the time taken to load a table on S3 is faster than on HDFS. The fix changes the test to reduce the expected delay when the debug actions are set so that test works on S3 builds. Testing: 1. Ran the test on the s3 build jenkins job. 2. Ran the test on HDFS build jenkins job. Change-Id: I8348a33ff8e9c7812540e014f4de2c65636da64f --- M tests/metadata/test_catalogd_debug_actions.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/64/16664/2 -- To view, visit http://gerrit.cloudera.org:8080/16664 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8348a33ff8e9c7812540e014f4de2c65636da64f Gerrit-Change-Number: 16664 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10277: Fix test catalogd debug actions on S3 builds
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/16664 ) Change subject: IMPALA-10277: Fix test_catalogd_debug_actions on S3 builds .. Patch Set 2: Code-Review+2 Since this is a minor test only change, I am confident enough to do a self review. -- To view, visit http://gerrit.cloudera.org:8080/16664 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8348a33ff8e9c7812540e014f4de2c65636da64f Gerrit-Change-Number: 16664 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Oct 2020 00:08:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10277: Fix test catalogd debug actions on S3 builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16664 ) Change subject: IMPALA-10277: Fix test_catalogd_debug_actions on S3 builds .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6612/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16664 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8348a33ff8e9c7812540e014f4de2c65636da64f Gerrit-Change-Number: 16664 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Oct 2020 00:09:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10277: Fix test catalogd debug actions on S3 builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16664 ) Change subject: IMPALA-10277: Fix test_catalogd_debug_actions on S3 builds .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16664 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8348a33ff8e9c7812540e014f4de2c65636da64f Gerrit-Change-Number: 16664 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Oct 2020 00:09:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10277: Fix test catalogd debug actions on S3 builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16664 ) Change subject: IMPALA-10277: Fix test_catalogd_debug_actions on S3 builds .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7572/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16664 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8348a33ff8e9c7812540e014f4de2c65636da64f Gerrit-Change-Number: 16664 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Oct 2020 00:27:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10132: Implement ds hll estimate bounds as string() function.
Hello Gabor Kaszab, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16626 to look at the new patch set (#4). Change subject: IMPALA-10132: Implement ds_hll_estimate_bounds_as_string() function. .. IMPALA-10132: Implement ds_hll_estimate_bounds_as_string() function. This function receives a string that is a serialized Apache DataSketches HLL sketch and optional kappa that is a number of standard deviations from the mean: 1, 2 or 3 (default 2). returns estimate and bounds with the values separated with commas. The result is three values: estimate, lower bound and upper bound. ds_hll_estimate_bounds_as_string(sketch [, kappa]) Kappa: 1 represent the 68.3% confidence bounds 2 represent the 95.4% confidence bounds 3 represent the 99.7% confidence bounds Note, ds_hll_estimate_bounds() should return an Array of doubles as the result but with that we have to wait for the complex type support. Until, we provide ds_hll_estimate_bounds_as_string() that can be deprecated once we have array support. Tracking Jira for returning complex types from functions is IMPALA-9520. Example: select ds_hll_estimate_bounds_as_string(ds_hll_sketch(int_col)) from functional_parquet.alltypestiny; +--+ | ds_hll_estimate_bounds_as_string(ds_hll_sketch(int_col)) | +--+ | 2,2,2.0002 | +--+ Change-Id: I46bf8263e8fd3877a087b9cb6f0d1a2392bb9153 --- M be/src/exprs/datasketches-common.h M be/src/exprs/datasketches-functions-ir.cc M be/src/exprs/datasketches-functions.h M common/function-registry/impala_functions.py M testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test 5 files changed, 103 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/16626/4 -- To view, visit http://gerrit.cloudera.org:8080/16626 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I46bf8263e8fd3877a087b9cb6f0d1a2392bb9153 Gerrit-Change-Number: 16626 Gerrit-PatchSet: 4 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10298: Change column mask hash as SHA512 in FIPS mode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16671 ) Change subject: IMPALA-10298: Change column mask hash as SHA512 in FIPS mode .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7573/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16671 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I422d4b11b31c3e6eb7963260a1da730579c4ca74 Gerrit-Change-Number: 16671 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 28 Oct 2020 04:13:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10298: Change column mask hash as SHA512 in FIPS mode
Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16671 Change subject: IMPALA-10298: Change column mask hash as SHA512 in FIPS mode .. IMPALA-10298: Change column mask hash as SHA512 in FIPS mode Column masking API is called by Ranger during policy evaluation. Ranger team requires to change the column mask hash as SHA-512 in FIPS mode without changing API. This patch changes the MaskFunctions::MaskHash() for string type to use SHA-512 in FIPS mode. Testing: - Passed exhaustive tests. - Manually test the API. Change-Id: I422d4b11b31c3e6eb7963260a1da730579c4ca74 --- M be/src/exprs/expr-test.cc M be/src/exprs/mask-functions-ir.cc 2 files changed, 34 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/16671/1 -- To view, visit http://gerrit.cloudera.org:8080/16671 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I422d4b11b31c3e6eb7963260a1da730579c4ca74 Gerrit-Change-Number: 16671 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-10132: Implement ds hll estimate bounds as string() function.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16626 ) Change subject: IMPALA-10132: Implement ds_hll_estimate_bounds_as_string() function. .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7574/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16626 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I46bf8263e8fd3877a087b9cb6f0d1a2392bb9153 Gerrit-Change-Number: 16626 Gerrit-PatchSet: 4 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 28 Oct 2020 04:31:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10132: Implement ds hll estimate bounds as string() function.
Fucun Chu has posted comments on this change. ( http://gerrit.cloudera.org:8080/16626 ) Change subject: IMPALA-10132: Implement ds_hll_estimate_bounds_as_string() function. .. Patch Set 2: (9 comments) Thanks for the review! Addressed the comments. http://gerrit.cloudera.org:8080/#/c/16626/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16626/2//COMMIT_MSG@7 PS2, Line 7: IMPALA-10132 Implement ds_hll_estimate_bounds_as_string() function. > Could you mention that there are 2 versions of this function. One that acce Done http://gerrit.cloudera.org:8080/#/c/16626/2//COMMIT_MSG@15 PS2, Line 15: butwith > typo Done http://gerrit.cloudera.org:8080/#/c/16626/2//COMMIT_MSG@16 PS2, Line 16: ds_kll_cdf_as_string > wrong function name, guess copy-pasted from the CDF change :) Done http://gerrit.cloudera.org:8080/#/c/16626/2/be/src/exprs/datasketches-common.h File be/src/exprs/datasketches-common.h: http://gerrit.cloudera.org:8080/#/c/16626/2/be/src/exprs/datasketches-common.h@43 PS2, Line 43: DS_DEFAULR_KAPPA > typo Done http://gerrit.cloudera.org:8080/#/c/16626/2/be/src/exprs/datasketches-functions.h File be/src/exprs/datasketches-functions.h: http://gerrit.cloudera.org:8080/#/c/16626/2/be/src/exprs/datasketches-functions.h@50 PS2, Line 50: disinct > typo Done http://gerrit.cloudera.org:8080/#/c/16626/2/be/src/exprs/datasketches-functions.h@55 PS2, Line 55: static StringVal DsHllEstimateBoundsAsString(FunctionContext* ctx, > I wouldn't write this whole comment for this function. I'd rather mention t Only the kappa param specification is retained http://gerrit.cloudera.org:8080/#/c/16626/2/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test File testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test: http://gerrit.cloudera.org:8080/#/c/16626/2/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test@291 PS2, Line 291: select ds_hll_estimate_bounds_as_string(ds_kll_sketch(cast(f2 as float))) from functional_parquet.emptytable; > Could you wrap these lines into 90 chars. If the output of a query is too l Done http://gerrit.cloudera.org:8080/#/c/16626/2/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test@313 PS2, Line 313: 4 > Please add a test for fraction inputs as well. Done http://gerrit.cloudera.org:8080/#/c/16626/2/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test@315 PS2, Line 315: UDF ERROR: Kappa may not be less than 1 or greater than 3. > I'd change this error msg to explicitly say these 3 values that are allowed Done -- To view, visit http://gerrit.cloudera.org:8080/16626 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I46bf8263e8fd3877a087b9cb6f0d1a2392bb9153 Gerrit-Change-Number: 16626 Gerrit-PatchSet: 2 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 28 Oct 2020 05:19:38 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10277: Fix test catalogd debug actions on S3 builds
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16664 ) Change subject: IMPALA-10277: Fix test_catalogd_debug_actions on S3 builds .. IMPALA-10277: Fix test_catalogd_debug_actions on S3 builds test_catalogd_debug_actions fails on S3 builds because the time taken to load a table on S3 is faster than on HDFS. The fix changes the test to reduce the expected delay when the debug actions are set so that test works on S3 builds. Testing: 1. Ran the test on the s3 build jenkins job. 2. Ran the test on HDFS build jenkins job. Change-Id: I8348a33ff8e9c7812540e014f4de2c65636da64f Reviewed-on: http://gerrit.cloudera.org:8080/16664 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M tests/metadata/test_catalogd_debug_actions.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16664 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8348a33ff8e9c7812540e014f4de2c65636da64f Gerrit-Change-Number: 16664 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-10277: Fix test catalogd debug actions on S3 builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16664 ) Change subject: IMPALA-10277: Fix test_catalogd_debug_actions on S3 builds .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16664 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8348a33ff8e9c7812540e014f4de2c65636da64f Gerrit-Change-Number: 16664 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Oct 2020 05:40:50 + Gerrit-HasComments: No
