[Impala-ASF-CR] IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11625 ) Change subject: IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver" .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11625 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I679469ed7f27944f75004ec4b16d513e6ea6b544 Gerrit-Change-Number: 11625 Gerrit-PatchSet: 7 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Wed, 17 Oct 2018 05:39:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver"
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11625 ) Change subject: IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver" .. IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver" This patch reverses the revert of IMPALA-7660. The problem with IMPALA-7660 was that urllib.urlopen added the 'context' parameter in 2.7.9, so it isn't present on rhel7, which uses 2.7.5 The fix is to switch to using the 'requests' library, which supports ssl connections on all the platforms Impala is supported on. This patch also adds more info to the error message printed by start-impala-cluster.py when the debug webserver cannot be reached yet to help with debugging these issues in the future. Testing: - Ran full builds on rhel7, rhel6, and ubuntu16. Change-Id: I679469ed7f27944f75004ec4b16d513e6ea6b544 Reviewed-on: http://gerrit.cloudera.org:8080/11625 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/thirdparty/squeasel/squeasel.c M tests/common/impala_cluster.py M tests/common/impala_service.py M tests/custom_cluster/test_client_ssl.py M tests/custom_cluster/test_redaction.py M tests/run-tests.py 6 files changed, 110 insertions(+), 52 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11625 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I679469ed7f27944f75004ec4b16d513e6ea6b544 Gerrit-Change-Number: 11625 Gerrit-PatchSet: 8 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-7555: Set socket timeout in impala-shell
Hello Michael Ho, Thomas Marshall, Lars Volker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11540 to look at the new patch set (#9). Change subject: IMPALA-7555: Set socket timeout in impala-shell .. IMPALA-7555: Set socket timeout in impala-shell impala-shell does not set any socket timeout while connecting to the impala server. This change sets a timeout on the socket before connecting and unsets it back after successfully connecting. The default timeout on this socket is 5 sec. Usage: impala-shell --client_connect_timeout= Testing: 1. Created a kerberized impala cluster with ssl enabled and connected to the impalad using an openssl client (block the beeswax server thread to accept new connection) - E.g. - openssl s_client -connect :21000 Used impala-shell to connect to the same impalad later. impala-shell timed out after the default of 5 sec.I verified it manually. Change-Id: I130fc47f7a83f591918d6842634b4e5787d00813 --- M shell/impala_client.py M shell/impala_shell.py M shell/impala_shell_config_defaults.py M shell/option_parser.py M tests/shell/test_shell_commandline.py 5 files changed, 52 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/11540/9 -- To view, visit http://gerrit.cloudera.org:8080/11540 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I130fc47f7a83f591918d6842634b4e5787d00813 Gerrit-Change-Number: 11540 Gerrit-PatchSet: 9 Gerrit-Owner: Anuj Phadke Gerrit-Reviewer: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11696 ) Change subject: IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11696 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Gerrit-Change-Number: 11696 Gerrit-PatchSet: 6 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 17 Oct 2018 05:33:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11696 ) Change subject: IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled .. IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled This patch adds a test coverage for catalogd restart when authorization is enabled to ensure all privileges in the impalad's catalogs get reset after the catalogd restart to avoid stale privileges in the impalad's catalogs, which can pose a security issue. Testing: - Ran all E2E authorization tests - Added a new test Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Reviewed-on: http://gerrit.cloudera.org:8080/11696 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M tests/authorization/test_authorization.py M tests/conftest.py 2 files changed, 67 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11696 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Gerrit-Change-Number: 11696 Gerrit-PatchSet: 7 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7555: Set socket timeout in impala-shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11540 ) Change subject: IMPALA-7555: Set socket timeout in impala-shell .. Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/11540/8/shell/impala_client.py File shell/impala_client.py: http://gerrit.cloudera.org:8080/#/c/11540/8/shell/impala_client.py@61 PS8, Line 61: def print_to_stderr(message): flake8: E302 expected 2 blank lines, found 1 -- To view, visit http://gerrit.cloudera.org:8080/11540 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I130fc47f7a83f591918d6842634b4e5787d00813 Gerrit-Change-Number: 11540 Gerrit-PatchSet: 8 Gerrit-Owner: Anuj Phadke Gerrit-Reviewer: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Wed, 17 Oct 2018 04:28:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11638 ) Change subject: IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11638 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 Gerrit-Change-Number: 11638 Gerrit-PatchSet: 4 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 17 Oct 2018 04:03:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11638 ) Change subject: IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs .. IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs The bug here was that any partial RPC on an IncompleteTable was throwing an NPE. Ideally, we attempt to load the table (if we find that it is not loaded) before making the partial info request, but a concurrent invalidate could reset the table state and move it back to an uninitialized state. This patch handles this case better by propagating a meaningful error to the caller. Testing: --- - Added a test that fails consistently with an NPE without this patch. Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 Reviewed-on: http://gerrit.cloudera.org:8080/11638 Reviewed-by: Bharath Vissapragada Tested-by: Impala Public Jenkins --- M common/thrift/CatalogService.thrift M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java M fe/src/test/java/org/apache/impala/catalog/PartialCatalogInfoTest.java M tests/custom_cluster/test_local_catalog.py 6 files changed, 60 insertions(+), 3 deletions(-) Approvals: Bharath Vissapragada: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/11638 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 Gerrit-Change-Number: 11638 Gerrit-PatchSet: 5 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7689: Reduce per column per partition stats estimate size
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11706 ) Change subject: IMPALA-7689: Reduce per column per partition stats estimate size .. Patch Set 1: Code-Review+2 (1 comment) looks good. main thing for this one that would be helpful is to explain where the 400 came from (and from there, the 200). http://gerrit.cloudera.org:8080/#/c/11706/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/11706/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@199 PS1, Line 199: 200 the 400 byte estimate was pretty accurate from the examples I saw. is there a pointer you can put here to explain how that was derived, then explain the 50% of that from compression? -- To view, visit http://gerrit.cloudera.org:8080/11706 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I347b41d9b298d7cd73ec812692172e0511415eee Gerrit-Change-Number: 11706 Gerrit-PatchSet: 1 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 17 Oct 2018 03:42:53 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7689: Reduce per column per partition stats estimate size
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11706 ) Change subject: IMPALA-7689: Reduce per column per partition stats estimate size .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1072/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11706 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I347b41d9b298d7cd73ec812692172e0511415eee Gerrit-Change-Number: 11706 Gerrit-PatchSet: 1 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 17 Oct 2018 02:24:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7715: [DOCS] Better descriptions for conditional functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11704 ) Change subject: IMPALA-7715: [DOCS] Better descriptions for conditional functions .. Patch Set 3: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/108/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cc90d62645730d2674bcb3af614863aa92b92f6 Gerrit-Change-Number: 11704 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Wed, 17 Oct 2018 02:20:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7715: [DOCS] Better descriptions for conditional functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11704 ) Change subject: IMPALA-7715: [DOCS] Better descriptions for conditional functions .. Patch Set 3: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/108/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cc90d62645730d2674bcb3af614863aa92b92f6 Gerrit-Change-Number: 11704 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Wed, 17 Oct 2018 02:17:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7715: [DOCS] Better descriptions for conditional functions
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/11704 ) Change subject: IMPALA-7715: [DOCS] Better descriptions for conditional functions .. Patch Set 2: (4 comments) http://gerrit.cloudera.org:8080/#/c/11704/2/docs/topics/impala_conditional_functions.xml File docs/topics/impala_conditional_functions.xml: http://gerrit.cloudera.org:8080/#/c/11704/2/docs/topics/impala_conditional_functions.xml@341 PS2, Line 341: FALSE if the expression is true or NULL. > Should the other refs to boolean constants be called out? "is false" --> "i WIP. In this file, I changed them to upper case. I just started converting the keywords and function names to be uppercase. It will take a while to convert them all in the function docs. http://gerrit.cloudera.org:8080/#/c/11704/2/docs/topics/impala_conditional_functions.xml@345 PS2, Line 345: > After reading your other change, it appears we support IS FALSE. So, we mig Done http://gerrit.cloudera.org:8080/#/c/11704/2/docs/topics/impala_conditional_functions.xml@445 PS2, Line 445: FALSE if the expression is false or NULL. > "an expression" --> "the expression". Code font for true and false. Done http://gerrit.cloudera.org:8080/#/c/11704/2/docs/topics/impala_conditional_functions.xml@449 PS2, Line 449: > Add comment "Is same as the IS TRUE operator." Done -- To view, visit http://gerrit.cloudera.org:8080/11704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cc90d62645730d2674bcb3af614863aa92b92f6 Gerrit-Change-Number: 11704 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Wed, 17 Oct 2018 02:17:33 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7715: [DOCS] Better descriptions for conditional functions
Hello Paul Rogers, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11704 to look at the new patch set (#3). Change subject: IMPALA-7715: [DOCS] Better descriptions for conditional functions .. IMPALA-7715: [DOCS] Better descriptions for conditional functions - Updated the descriptions for ISTRUE, ISFALSE, NONULVALUE, NULLVALUE. - Updated several function names to use caps. Change-Id: I5cc90d62645730d2674bcb3af614863aa92b92f6 --- M docs/shared/impala_common.xml M docs/topics/impala_conditional_functions.xml 2 files changed, 115 insertions(+), 89 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/11704/3 -- To view, visit http://gerrit.cloudera.org:8080/11704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5cc90d62645730d2674bcb3af614863aa92b92f6 Gerrit-Change-Number: 11704 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11696 ) Change subject: IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1071/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11696 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Gerrit-Change-Number: 11696 Gerrit-PatchSet: 5 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 17 Oct 2018 02:08:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6661 Make NaN values equal for grouping purposes.
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/11535 ) Change subject: IMPALA-6661 Make NaN values equal for grouping purposes. .. Patch Set 15: (1 comment) http://gerrit.cloudera.org:8080/#/c/11535/15/testdata/workloads/functional-query/queries/QueryTest/exprs.test File testdata/workloads/functional-query/queries/QueryTest/exprs.test: http://gerrit.cloudera.org:8080/#/c/11535/15/testdata/workloads/functional-query/queries/QueryTest/exprs.test@3075 PS15, Line 3075: # IMPALA-6661 - NaN should not evaluate as the same as any other NaN via <=> : WITH W AS (SELECT T.*, CAST(SQRT(X) AS FLOAT) P, CAST(SQRT(Y) AS FLOAT) Q : FROM (VALUES((-1.0 X, -1.0 Y), (-1.0, 0), (0, -1.0), (0, 0))) T ) : SELECT * FROM W WHERE W.Q<=>W.P : RESULTS : 0,0,0,0 : TYPES : FLOAT, FLOAT, FLOAT, FLOAT > This test case is here in response to your comment: "It may warrant a test Sorry, I wasn't clear in my comment. I meant to say a test case which exercise the join condition on Nan <=> Nan out of caution. hash-table.cc logic is used both by join and aggregation. -- To view, visit http://gerrit.cloudera.org:8080/11535 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I996c4a2e1934fd887046ed0c55457b7285375086 Gerrit-Change-Number: 11535 Gerrit-PatchSet: 15 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 17 Oct 2018 01:51:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7689: Reduce per column per partition stats estimate size
Bharath Vissapragada has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11706 Change subject: IMPALA-7689: Reduce per column per partition stats estimate size .. IMPALA-7689: Reduce per column per partition stats estimate size With the improvements in the incremental stats memory representation (IMPALA-7424), the per column per partition stats estimate should be reduced to account for the compressed memory footprint. Doing some experiments on various test tables, I see the size is down by 50-70%. This patch reduces the size estimate by 50% (conservative). Ideally we don't need to estimate on the Catalog server during serialization since we can compute the byte sizes by looping through all the partitions. However this patch retains the current logic to keep it consistent with "compute incremental stats" analysis. Change-Id: I347b41d9b298d7cd73ec812692172e0511415eee --- M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java 1 file changed, 1 insertion(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/11706/1 -- To view, visit http://gerrit.cloudera.org:8080/11706 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I347b41d9b298d7cd73ec812692172e0511415eee Gerrit-Change-Number: 11706 Gerrit-PatchSet: 1 Gerrit-Owner: Bharath Vissapragada
[Impala-ASF-CR] [DOCS] Additional note in impala alter database
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11703 ) Change subject: [DOCS] Additional note in impala_alter_database .. [DOCS] Additional note in impala_alter_database - Added a note about the effective ALL with GRANT and ALL without GRANT privileges. Change-Id: Idf2cb58993adadb7fb32ec188f5e4dff216c5897 Reviewed-on: http://gerrit.cloudera.org:8080/11703 Tested-by: Impala Public Jenkins Reviewed-by: Fredy Wijaya --- M docs/topics/impala_alter_database.xml 1 file changed, 7 insertions(+), 5 deletions(-) Approvals: Impala Public Jenkins: Verified Fredy Wijaya: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/11703 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Idf2cb58993adadb7fb32ec188f5e4dff216c5897 Gerrit-Change-Number: 11703 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-6661 Make NaN values equal for grouping purposes.
Michal Ostrowski has posted comments on this change. ( http://gerrit.cloudera.org:8080/11535 ) Change subject: IMPALA-6661 Make NaN values equal for grouping purposes. .. Patch Set 15: (1 comment) http://gerrit.cloudera.org:8080/#/c/11535/15/testdata/workloads/functional-query/queries/QueryTest/exprs.test File testdata/workloads/functional-query/queries/QueryTest/exprs.test: http://gerrit.cloudera.org:8080/#/c/11535/15/testdata/workloads/functional-query/queries/QueryTest/exprs.test@3075 PS15, Line 3075: # IMPALA-6661 - NaN should not evaluate as the same as any other NaN via <=> : WITH W AS (SELECT T.*, CAST(SQRT(X) AS FLOAT) P, CAST(SQRT(Y) AS FLOAT) Q : FROM (VALUES((-1.0 X, -1.0 Y), (-1.0, 0), (0, -1.0), (0, 0))) T ) : SELECT * FROM W WHERE W.Q<=>W.P : RESULTS : 0,0,0,0 : TYPES : FLOAT, FLOAT, FLOAT, FLOAT > I wasn't trying to test join code... I was trying to test the <=> operator. This test case is here in response to your comment: "It may warrant a test case to verify the behavior of Nan <=> Nan " I don't see how joins relate to this. -- To view, visit http://gerrit.cloudera.org:8080/11535 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I996c4a2e1934fd887046ed0c55457b7285375086 Gerrit-Change-Number: 11535 Gerrit-PatchSet: 15 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 17 Oct 2018 01:38:54 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11625 ) Change subject: IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver" .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11625 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I679469ed7f27944f75004ec4b16d513e6ea6b544 Gerrit-Change-Number: 11625 Gerrit-PatchSet: 7 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Wed, 17 Oct 2018 01:37:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11696 ) Change subject: IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11696 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Gerrit-Change-Number: 11696 Gerrit-PatchSet: 6 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 17 Oct 2018 01:36:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6661 Make NaN values equal for grouping purposes.
Michal Ostrowski has posted comments on this change. ( http://gerrit.cloudera.org:8080/11535 ) Change subject: IMPALA-6661 Make NaN values equal for grouping purposes. .. Patch Set 15: (1 comment) http://gerrit.cloudera.org:8080/#/c/11535/15/testdata/workloads/functional-query/queries/QueryTest/exprs.test File testdata/workloads/functional-query/queries/QueryTest/exprs.test: http://gerrit.cloudera.org:8080/#/c/11535/15/testdata/workloads/functional-query/queries/QueryTest/exprs.test@3075 PS15, Line 3075: # IMPALA-6661 - NaN should not evaluate as the same as any other NaN via <=> : WITH W AS (SELECT T.*, CAST(SQRT(X) AS FLOAT) P, CAST(SQRT(Y) AS FLOAT) Q : FROM (VALUES((-1.0 X, -1.0 Y), (-1.0, 0), (0, -1.0), (0, 0))) T ) : SELECT * FROM W WHERE W.Q<=>W.P : RESULTS : 0,0,0,0 : TYPES : FLOAT, FLOAT, FLOAT, FLOAT > This is not the right test to exercise join code. You can check the query p I wasn't trying to test join code... I was trying to test the <=> operator. -- To view, visit http://gerrit.cloudera.org:8080/11535 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I996c4a2e1934fd887046ed0c55457b7285375086 Gerrit-Change-Number: 11535 Gerrit-PatchSet: 15 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 17 Oct 2018 01:36:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11696 ) Change subject: IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3322/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11696 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Gerrit-Change-Number: 11696 Gerrit-PatchSet: 6 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 17 Oct 2018 01:36:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11625 ) Change subject: IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver" .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3323/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11625 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I679469ed7f27944f75004ec4b16d513e6ea6b544 Gerrit-Change-Number: 11625 Gerrit-PatchSet: 7 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Wed, 17 Oct 2018 01:37:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11673 ) Change subject: IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement .. IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement Change-Id: I203a800855a413069a40c728dfa157939ea15caf Reviewed-on: http://gerrit.cloudera.org:8080/11673 Tested-by: Impala Public Jenkins Reviewed-by: Fredy Wijaya --- M docs/topics/impala_alter_table.xml M docs/topics/impala_alter_view.xml 2 files changed, 443 insertions(+), 316 deletions(-) Approvals: Impala Public Jenkins: Verified Fredy Wijaya: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/11673 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I203a800855a413069a40c728dfa157939ea15caf Gerrit-Change-Number: 11673 Gerrit-PatchSet: 6 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Fredy Wijaya has posted comments on this change. (
http://gerrit.cloudera.org:8080/11696 )
Change subject: IMPALA-7713: Add test coverage for catalogd restart when
authorization is enabled
..
Patch Set 5: Code-Review+2
(2 comments)
Carry Phil's +2.
http://gerrit.cloudera.org:8080/#/c/11696/4/tests/authorization/test_authorization.py
File tests/authorization/test_authorization.py:
http://gerrit.cloudera.org:8080/#/c/11696/4/tests/authorization/test_authorization.py@399
PS4, Line 399: TestAuthorization._check_privileges(result, [["database",
"functional",
: "", "", "",
"all", "false"]])
:
: result = self.client.execute("show grant role %s_bar"
> If you really wanted to, you could make 399 part of _check_privileges(). An
I like the conciseness. Done.
http://gerrit.cloudera.org:8080/#/c/11696/4/tests/authorization/test_authorization.py@443
PS4, Line 443:
> Where is create_time here? I don't quite see what this is skipping.
I forgot to remove this. It's no longer needed. Done.
--
To view, visit http://gerrit.cloudera.org:8080/11696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369
Gerrit-Change-Number: 11696
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Wed, 17 Oct 2018 01:34:59 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Fredy Wijaya has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/11696 ) Change subject: IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled .. IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled This patch adds a test coverage for catalogd restart when authorization is enabled to ensure all privileges in the impalad's catalogs get reset after the catalogd restart to avoid stale privileges in the impalad's catalogs, which can pose a security issue. Testing: - Ran all E2E authorization tests - Added a new test Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 --- M tests/authorization/test_authorization.py M tests/conftest.py 2 files changed, 67 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/11696/5 -- To view, visit http://gerrit.cloudera.org:8080/11696 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Gerrit-Change-Number: 11696 Gerrit-PatchSet: 5 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7715: [DOCS] Better descriptions for conditional functions
Paul Rogers has posted comments on this change. ( http://gerrit.cloudera.org:8080/11704 ) Change subject: IMPALA-7715: [DOCS] Better descriptions for conditional functions .. Patch Set 2: (4 comments) Thanks for making these changes so quickly! A few minor suggestions. http://gerrit.cloudera.org:8080/#/c/11704/2/docs/topics/impala_conditional_functions.xml File docs/topics/impala_conditional_functions.xml: http://gerrit.cloudera.org:8080/#/c/11704/2/docs/topics/impala_conditional_functions.xml@341 PS2, Line 341: FALSE if the expression is true or NULL. Should the other refs to boolean constants be called out? "is false" --> "is FALSE"? Also for true? http://gerrit.cloudera.org:8080/#/c/11704/2/docs/topics/impala_conditional_functions.xml@345 PS2, Line 345: After reading your other change, it appears we support IS FALSE. So, we might add: "Same as the IS FALSE operator." http://gerrit.cloudera.org:8080/#/c/11704/2/docs/topics/impala_conditional_functions.xml@445 PS2, Line 445: FALSE if the expression is false or NULL. "an expression" --> "the expression". Code font for true and false. http://gerrit.cloudera.org:8080/#/c/11704/2/docs/topics/impala_conditional_functions.xml@449 PS2, Line 449: Add comment "Is same as the IS TRUE operator." Maybe also do this for "isnottrue() - IS NOT TRUE" and "isnotfalse() - IS NOT FALSE" -- To view, visit http://gerrit.cloudera.org:8080/11704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cc90d62645730d2674bcb3af614863aa92b92f6 Gerrit-Change-Number: 11704 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Wed, 17 Oct 2018 01:28:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [DOCS] Additional note in impala alter database
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/11703 ) Change subject: [DOCS] Additional note in impala_alter_database .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11703 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf2cb58993adadb7fb32ec188f5e4dff216c5897 Gerrit-Change-Number: 11703 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 17 Oct 2018 01:00:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/11673 ) Change subject: IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11673 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I203a800855a413069a40c728dfa157939ea15caf Gerrit-Change-Number: 11673 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 17 Oct 2018 00:59:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11625 ) Change subject: IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver" .. Patch Set 6: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3320/ -- To view, visit http://gerrit.cloudera.org:8080/11625 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I679469ed7f27944f75004ec4b16d513e6ea6b544 Gerrit-Change-Number: 11625 Gerrit-PatchSet: 6 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Wed, 17 Oct 2018 00:55:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11638 ) Change subject: IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1070/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11638 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 Gerrit-Change-Number: 11638 Gerrit-PatchSet: 4 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 17 Oct 2018 00:39:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4063: Merge report of query fragment instances per executor
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/11615 ) Change subject: IMPALA-4063: Merge report of query fragment instances per executor .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/11615/5/be/src/runtime/query-state.cc File be/src/runtime/query-state.cc: http://gerrit.cloudera.org:8080/#/c/11615/5/be/src/runtime/query-state.cc@495 PS5, Line 495: error: : // This point is reached if there were general errors to start query fragment instances. : // Wait for all running fragment instances to finish preparing and report status to the : // coordinator to start query cancellation. > Since we are changing some error handling for thread creation, a way to tes Thanks for pointing that out. We already have a test to exercise the thread creation failure here (https://github.com/apache/impala/blob/master/tests/failure/test_failpoints.py#L159-L173) but I think it's also a good idea to exercise the paths stress option you mentioned. Thanks for the suggestion. -- To view, visit http://gerrit.cloudera.org:8080/11615 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5f95e026ba05631f33f48ce32da6db39c6f421fa Gerrit-Change-Number: 11615 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 17 Oct 2018 00:31:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6661 Make NaN values equal for grouping purposes.
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/11535 ) Change subject: IMPALA-6661 Make NaN values equal for grouping purposes. .. Patch Set 15: (1 comment) http://gerrit.cloudera.org:8080/#/c/11535/15/testdata/workloads/functional-query/queries/QueryTest/exprs.test File testdata/workloads/functional-query/queries/QueryTest/exprs.test: http://gerrit.cloudera.org:8080/#/c/11535/15/testdata/workloads/functional-query/queries/QueryTest/exprs.test@3075 PS15, Line 3075: # IMPALA-6661 - NaN should not evaluate as the same as any other NaN via <=> : WITH W AS (SELECT T.*, CAST(SQRT(X) AS FLOAT) P, CAST(SQRT(Y) AS FLOAT) Q : FROM (VALUES((-1.0 X, -1.0 Y), (-1.0, 0), (0, -1.0), (0, 0))) T ) : SELECT * FROM W WHERE W.Q<=>W.P : RESULTS : 0,0,0,0 : TYPES : FLOAT, FLOAT, FLOAT, FLOAT This is not the right test to exercise join code. You can check the query plan but it doesn't have a join in it. You can check the query plan by doing explain of the query in Impala shell and see if it matches your expectation. You can do a self-join like the following. Also, this test belongs to joins.test. with y as (select t1.float_col as v from functional.alltypessmall t1, functional.alltypessmall t2 where sqrt(3.5-t1.float_col) <=> sqrt(3.5-t2.float_col)) select count(*) from y where is_nan(v); -- To view, visit http://gerrit.cloudera.org:8080/11535 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I996c4a2e1934fd887046ed0c55457b7285375086 Gerrit-Change-Number: 11535 Gerrit-PatchSet: 15 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 17 Oct 2018 00:21:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11638 ) Change subject: IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3321/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11638 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 Gerrit-Change-Number: 11638 Gerrit-PatchSet: 4 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 17 Oct 2018 00:06:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs
Hello Todd Lipcon, Impala Public Jenkins, Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11638 to look at the new patch set (#4). Change subject: IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs .. IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs The bug here was that any partial RPC on an IncompleteTable was throwing an NPE. Ideally, we attempt to load the table (if we find that it is not loaded) before making the partial info request, but a concurrent invalidate could reset the table state and move it back to an uninitialized state. This patch handles this case better by propagating a meaningful error to the caller. Testing: --- - Added a test that fails consistently with an NPE without this patch. Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 --- M common/thrift/CatalogService.thrift M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java M fe/src/test/java/org/apache/impala/catalog/PartialCatalogInfoTest.java M tests/custom_cluster/test_local_catalog.py 6 files changed, 60 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/38/11638/4 -- To view, visit http://gerrit.cloudera.org:8080/11638 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 Gerrit-Change-Number: 11638 Gerrit-PatchSet: 4 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11638 ) Change subject: IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs .. Patch Set 4: Code-Review+2 (1 comment) carrying +2. http://gerrit.cloudera.org:8080/#/c/11638/3/tests/custom_cluster/test_local_catalog.py File tests/custom_cluster/test_local_catalog.py: http://gerrit.cloudera.org:8080/#/c/11638/3/tests/custom_cluster/test_local_catalog.py@274 PS3, Line 274: ert failed_queries.empty(),\ : "Failed > can simplify to: assert not failed_queries.empty(), "..." Done -- To view, visit http://gerrit.cloudera.org:8080/11638 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 Gerrit-Change-Number: 11638 Gerrit-PatchSet: 4 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 17 Oct 2018 00:06:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11638 ) Change subject: IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs .. Patch Set 3: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/11638/3/tests/custom_cluster/test_local_catalog.py File tests/custom_cluster/test_local_catalog.py: http://gerrit.cloudera.org:8080/#/c/11638/3/tests/custom_cluster/test_local_catalog.py@274 PS3, Line 274: not failed_queries.empty(): : assert Fal can simplify to: assert not failed_queries.empty(), "..." -- To view, visit http://gerrit.cloudera.org:8080/11638 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 Gerrit-Change-Number: 11638 Gerrit-PatchSet: 3 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 16 Oct 2018 23:58:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7639: Move concurrent UDF tests to a custom cluster test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11701 ) Change subject: IMPALA-7639: Move concurrent UDF tests to a custom cluster test .. Patch Set 1: (12 comments) http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py File tests/custom_cluster/test_udf_concurrency.py: http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py@23 PS1, Line 23: from subprocess import call, check_call flake8: F401 'subprocess.call' imported but unused http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py@27 PS1, Line 27: from tests.common.test_dimensions import ( flake8: F401 'tests.common.test_dimensions.create_exec_option_dimension_from_dict' imported but unused http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py@27 PS1, Line 27: from tests.common.test_dimensions import ( flake8: F401 'tests.common.test_dimensions.create_exec_option_dimension' imported but unused http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py@27 PS1, Line 27: from tests.common.test_dimensions import ( flake8: F401 'tests.common.test_dimensions.create_uncompressed_text_dimension' imported but unused http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py@31 PS1, Line 31: from tests.common.test_vector import ImpalaTestDimension flake8: F401 'tests.common.test_vector.ImpalaTestDimension' imported but unused http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py@37 PS1, Line 37: class TestUdfConcurrency(CustomClusterTestSuite): flake8: E302 expected 2 blank lines, found 1 http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py@56 PS1, Line 56: a flake8: E501 line too long (97 > 90 characters) http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py@84 PS1, Line 84: d flake8: E306 expected 1 blank line before a nested definition, found 0 http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py@121 PS1, Line 121: ; flake8: E703 statement ends with a semicolon http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py@174 PS1, Line 174: d flake8: E306 expected 1 blank line before a nested definition, found 0 http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py@194 PS1, Line 194: d flake8: F841 local variable 'drop_threads' is assigned to but never used http://gerrit.cloudera.org:8080/#/c/11701/1/tests/custom_cluster/test_udf_concurrency.py@202 PS1, Line 202: ; flake8: E703 statement ends with a semicolon -- To view, visit http://gerrit.cloudera.org:8080/11701 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3f255823167a4dd807a07276f630ef02435900a3 Gerrit-Change-Number: 11701 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 16 Oct 2018 23:55:33 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code"
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/11699 ) Change subject: Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code" .. Patch Set 2: Lars, what's the status of this patch ? Should it be abandoned ? -- To view, visit http://gerrit.cloudera.org:8080/11699 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5ab7f2f6317f1281928af5ea6cf3fd7d0c6e0a09 Gerrit-Change-Number: 11699 Gerrit-PatchSet: 2 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Tue, 16 Oct 2018 23:41:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7272: Fix crash in StringMinMaxFilter
Michael Ho has posted comments on this change. (
http://gerrit.cloudera.org:8080/11650 )
Change subject: IMPALA-7272: Fix crash in StringMinMaxFilter
..
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/11650/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/11650/2//COMMIT_MSG@26
PS2, Line 26: I have been unable to repro the actual crash despite trying a
large
: variety of different things. However, with additional logging
added
: its clear that the MemPool is being used concurrently, which is
: incorrect.
Can you reproduce the problem now with DFAKE_MUTEX ?
http://gerrit.cloudera.org:8080/#/c/11650/2/be/CMakeLists.txt
File be/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/11650/2/be/CMakeLists.txt@234
PS2, Line 234: SET(CLANG_IR_CXX_FLAGS "${CLANG_IR_CXX_FLAGS}" "-DNDEBUG")
I think this makes sense. I wonder what the historical context for not doing it
in the first place. I checked the code on some older releases from more than 4
years ago and it was there all along. May be the codegen time was too slow back
then to include all the debug code ?
How much slow down (if any) did you notice in debug builds with this change ?
http://gerrit.cloudera.org:8080/#/c/11650/2/be/src/runtime/mem-pool.h
File be/src/runtime/mem-pool.h:
http://gerrit.cloudera.org:8080/#/c/11650/2/be/src/runtime/mem-pool.h@135
PS2, Line 135: DFAKE_SCOPED_LOCK(mutex_);
nit: Feel free to ignore but it seems easier to just add DFAKE_SCOPED_LOCK()
directly in Allocate(). That said, the current code also looks fine as-is.
--
To view, visit http://gerrit.cloudera.org:8080/11650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I751cad7e6b75c9d95d7ad029bbd1e52fe09e8a29
Gerrit-Change-Number: 11650
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Tue, 16 Oct 2018 23:40:22 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11638 ) Change subject: IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1069/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11638 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 Gerrit-Change-Number: 11638 Gerrit-PatchSet: 3 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 16 Oct 2018 23:10:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7715: [DOCS] Better descriptions for conditional functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11704 ) Change subject: IMPALA-7715: [DOCS] Better descriptions for conditional functions .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/107/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cc90d62645730d2674bcb3af614863aa92b92f6 Gerrit-Change-Number: 11704 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Tue, 16 Oct 2018 22:57:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11673 ) Change subject: IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement .. Patch Set 5: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/106/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11673 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I203a800855a413069a40c728dfa157939ea15caf Gerrit-Change-Number: 11673 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Oct 2018 22:54:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7715: [DOCS] Better descriptions for conditional functions
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11704 to look at the new patch set (#2). Change subject: IMPALA-7715: [DOCS] Better descriptions for conditional functions .. IMPALA-7715: [DOCS] Better descriptions for conditional functions - Updated the descriptions for ISTRUE, ISFALSE, NONULVALUE, NULLVALUE. - Updated several function names to use caps. Change-Id: I5cc90d62645730d2674bcb3af614863aa92b92f6 --- M docs/shared/impala_common.xml M docs/topics/impala_conditional_functions.xml 2 files changed, 74 insertions(+), 72 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/11704/2 -- To view, visit http://gerrit.cloudera.org:8080/11704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5cc90d62645730d2674bcb3af614863aa92b92f6 Gerrit-Change-Number: 11704 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7555: Set socket timeout in impala-shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11540 ) Change subject: IMPALA-7555: Set socket timeout in impala-shell .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1068/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11540 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I130fc47f7a83f591918d6842634b4e5787d00813 Gerrit-Change-Number: 11540 Gerrit-PatchSet: 8 Gerrit-Owner: Anuj Phadke Gerrit-Reviewer: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 16 Oct 2018 22:48:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7715: [DOCS] Better descriptions for conditional functions
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11704 Change subject: IMPALA-7715: [DOCS] Better descriptions for conditional functions .. IMPALA-7715: [DOCS] Better descriptions for conditional functions - Updated the descriptions for ISTRUE, ISFALSE, NONULVALUE, NULLVALUE. Change-Id: I5cc90d62645730d2674bcb3af614863aa92b92f6 --- M docs/shared/impala_common.xml M docs/topics/impala_conditional_functions.xml 2 files changed, 74 insertions(+), 72 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/11704/1 -- To view, visit http://gerrit.cloudera.org:8080/11704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5cc90d62645730d2674bcb3af614863aa92b92f6 Gerrit-Change-Number: 11704 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7715: [DOCS] Better descriptions for conditional functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11704 ) Change subject: IMPALA-7715: [DOCS] Better descriptions for conditional functions .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/107/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cc90d62645730d2674bcb3af614863aa92b92f6 Gerrit-Change-Number: 11704 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Oct 2018 22:48:07 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code"
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/11699 ) Change subject: Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code" .. Patch Set 2: Code-Review-2 Looks like we might still need this -- To view, visit http://gerrit.cloudera.org:8080/11699 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5ab7f2f6317f1281928af5ea6cf3fd7d0c6e0a09 Gerrit-Change-Number: 11699 Gerrit-PatchSet: 2 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Tue, 16 Oct 2018 22:32:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7555: Set socket timeout in impala-shell
Lars Volker has posted comments on this change. (
http://gerrit.cloudera.org:8080/11540 )
Change subject: IMPALA-7555: Set socket timeout in impala-shell
..
Patch Set 8:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/11540/8/shell/impala_client.py
File shell/impala_client.py:
http://gerrit.cloudera.org:8080/#/c/11540/8/shell/impala_client.py@265
PS8, Line 265: print_to_stderr('Connected to %s:%s' % (self.impalad_host,
self.impalad_port))
This message seems misleading, given that the client has its own notion of
being connected (see is_connected() above).
We can change it to "Opened TCP connection to %s:%s" and only print it when the
log level is set to debug. If you want to tell both hangs apart, we should use
two separate log messages.
http://gerrit.cloudera.org:8080/#/c/11540/8/shell/impala_client.py@266
PS8, Line 266: # Setting a timeout of None disables timeouts on socket
operations
"on sockets."
http://gerrit.cloudera.org:8080/#/c/11540/6/shell/impala_shell.py
File shell/impala_shell.py:
http://gerrit.cloudera.org:8080/#/c/11540/6/shell/impala_shell.py@816
PS6, Line 816: except ImportError:
I think we should keep this, see my comment elsewhere
http://gerrit.cloudera.org:8080/#/c/11540/8/shell/impala_shell.py
File shell/impala_shell.py:
http://gerrit.cloudera.org:8080/#/c/11540/8/shell/impala_shell.py@1510
PS8, Line 1510: given set_variables."""
nit: indent
http://gerrit.cloudera.org:8080/#/c/11540/8/shell/impala_shell.py@1540
PS8, Line 1540: """Look for a namespace:var_name pattern in an option name.
While you're here, "Looks", "Returns"
http://gerrit.cloudera.org:8080/#/c/11540/6/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:
http://gerrit.cloudera.org:8080/#/c/11540/6/tests/shell/test_shell_commandline.py@737
PS6, Line 737: """
Where did this go?
--
To view, visit http://gerrit.cloudera.org:8080/11540
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I130fc47f7a83f591918d6842634b4e5787d00813
Gerrit-Change-Number: 11540
Gerrit-PatchSet: 8
Gerrit-Owner: Anuj Phadke
Gerrit-Reviewer: Anuj Phadke
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Tue, 16 Oct 2018 22:31:00 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs
Hello Todd Lipcon, Impala Public Jenkins, Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11638 to look at the new patch set (#3). Change subject: IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs .. IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs The bug here was that any partial RPC on an IncompleteTable was throwing an NPE. Ideally, we attempt to load the table (if we find that it is not loaded) before making the partial info request, but a concurrent invalidate could reset the table state and move it back to an uninitialized state. This patch handles this case better by propagating a meaningful error to the caller. Testing: --- - Added a test that fails consistently with an NPE without this patch. Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 --- M common/thrift/CatalogService.thrift M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java M fe/src/test/java/org/apache/impala/catalog/PartialCatalogInfoTest.java M tests/custom_cluster/test_local_catalog.py 6 files changed, 60 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/38/11638/3 -- To view, visit http://gerrit.cloudera.org:8080/11638 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 Gerrit-Change-Number: 11638 Gerrit-PatchSet: 3 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-4063: Merge report of query fragment instances per executor
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/11615 ) Change subject: IMPALA-4063: Merge report of query fragment instances per executor .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/11615/5/be/src/runtime/query-state.cc File be/src/runtime/query-state.cc: http://gerrit.cloudera.org:8080/#/c/11615/5/be/src/runtime/query-state.cc@495 PS5, Line 495: error: : // This point is reached if there were general errors to start query fragment instances. : // Wait for all running fragment instances to finish preparing and report status to the : // coordinator to start query cancellation. Since we are changing some error handling for thread creation, a way to test that is to run some tests with thread_creation_fault_injection=true and verify that nothing hangs/crashes. -- To view, visit http://gerrit.cloudera.org:8080/11615 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5f95e026ba05631f33f48ce32da6db39c6f421fa Gerrit-Change-Number: 11615 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 16 Oct 2018 22:26:39 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7555: Set socket timeout in impala-shell
Anuj Phadke has posted comments on this change. (
http://gerrit.cloudera.org:8080/11540 )
Change subject: IMPALA-7555: Set socket timeout in impala-shell
..
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11540/8/shell/impala_client.py
File shell/impala_client.py:
http://gerrit.cloudera.org:8080/#/c/11540/8/shell/impala_client.py@265
PS8, Line 265: print_to_stderr('Connected to %s:%s' % (self.impalad_host,
self.impalad_port))
I am moving this here. Logging this before ping_impala_service()
should help us narrow down a hang in open() vs ping_impala_service()
potentially.
--
To view, visit http://gerrit.cloudera.org:8080/11540
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I130fc47f7a83f591918d6842634b4e5787d00813
Gerrit-Change-Number: 11540
Gerrit-PatchSet: 8
Gerrit-Owner: Anuj Phadke
Gerrit-Reviewer: Anuj Phadke
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Tue, 16 Oct 2018 22:19:20 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7644: Hide Parquet page index writing with feature flag
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/11694 ) Change subject: IMPALA-7644: Hide Parquet page index writing with feature flag .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/11694/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11694/2//COMMIT_MSG@18 PS2, Line 18: custom_cluset nit: typo http://gerrit.cloudera.org:8080/#/c/11694/2/be/src/exec/hdfs-parquet-table-writer.cc File be/src/exec/hdfs-parquet-table-writer.cc: http://gerrit.cloudera.org:8080/#/c/11694/2/be/src/exec/hdfs-parquet-table-writer.cc@705 PS2, Line 705: num_data_pages_ > Maybe we could just drop 'num_data_pages_' and call std::vector::resize(0) I agree, that would simplify things substantially. When you do this, can you either do a perf run or inspect the assembly to make sure that the additional dtor ctor calls for the pages are constant time, i.e. the DataPage structs are all PODs? It might be better to do all of that in a subsequent change, but given we can into this issue, we should improve the current code. If you don't have time to push the additional change soon, please add a comment here. The SO discussion linked in the cppreference reads as if clear() should be safe. If you're unsure, you can add a DCHECK on the capacity right after calling clear(). -- To view, visit http://gerrit.cloudera.org:8080/11694 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib4a9098a2085a385351477c715ae245d83bf1c72 Gerrit-Change-Number: 11694 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 16 Oct 2018 22:16:38 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7555: Set socket timeout in impala-shell
Hello Michael Ho, Thomas Marshall, Lars Volker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11540 to look at the new patch set (#8). Change subject: IMPALA-7555: Set socket timeout in impala-shell .. IMPALA-7555: Set socket timeout in impala-shell impala-shell does not set any socket timeout while connecting to the impala server. This change sets a timeout on the socket before connecting and unsets it back after successfully connecting. The default timeout on this socket is 5 sec. Usage: impala-shell --client_connect_timeout= Testing: 1. Created a kerberized impala cluster with ssl enabled and connected to the impalad using an openssl client (block the beeswax server thread to accept new connection) - E.g. - openssl s_client -connect :21000 Used impala-shell to connect to the same impalad later. impala-shell timed out after the default of 5 sec.I verified it manually. Change-Id: I130fc47f7a83f591918d6842634b4e5787d00813 --- M shell/impala_client.py M shell/impala_shell.py M shell/impala_shell_config_defaults.py M shell/option_parser.py 4 files changed, 32 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/11540/8 -- To view, visit http://gerrit.cloudera.org:8080/11540 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I130fc47f7a83f591918d6842634b4e5787d00813 Gerrit-Change-Number: 11540 Gerrit-PatchSet: 8 Gerrit-Owner: Anuj Phadke Gerrit-Reviewer: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11673 ) Change subject: IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement .. Patch Set 5: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/106/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11673 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I203a800855a413069a40c728dfa157939ea15caf Gerrit-Change-Number: 11673 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Oct 2018 22:10:57 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Additional note in impala alter database
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11703 ) Change subject: [DOCS] Additional note in impala_alter_database .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/105/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11703 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf2cb58993adadb7fb32ec188f5e4dff216c5897 Gerrit-Change-Number: 11703 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Oct 2018 22:10:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7639: Move concurrent UDF tests to a custom cluster test
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11701 ) Change subject: IMPALA-7639: Move concurrent UDF tests to a custom cluster test .. Patch Set 1: Code-Review+2 thanks! -- To view, visit http://gerrit.cloudera.org:8080/11701 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3f255823167a4dd807a07276f630ef02435900a3 Gerrit-Change-Number: 11701 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 16 Oct 2018 22:08:24 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Additional note in impala alter database
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11703 ) Change subject: [DOCS] Additional note in impala_alter_database .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/105/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11703 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf2cb58993adadb7fb32ec188f5e4dff216c5897 Gerrit-Change-Number: 11703 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Oct 2018 22:07:27 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Additional note in impala alter database
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11703 Change subject: [DOCS] Additional note in impala_alter_database .. [DOCS] Additional note in impala_alter_database - Added a note about the effective ALL with GRANT and ALL without GRANT privileges. Change-Id: Idf2cb58993adadb7fb32ec188f5e4dff216c5897 --- M docs/topics/impala_alter_database.xml 1 file changed, 7 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/11703/1 -- To view, visit http://gerrit.cloudera.org:8080/11703 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Idf2cb58993adadb7fb32ec188f5e4dff216c5897 Gerrit-Change-Number: 11703 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7272: Fix crash in StringMinMaxFilter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11650 ) Change subject: IMPALA-7272: Fix crash in StringMinMaxFilter .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1067/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11650 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I751cad7e6b75c9d95d7ad029bbd1e52fe09e8a29 Gerrit-Change-Number: 11650 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 16 Oct 2018 22:06:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7639: Move concurrent UDF tests to a custom cluster test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11701 ) Change subject: IMPALA-7639: Move concurrent UDF tests to a custom cluster test .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1066/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11701 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3f255823167a4dd807a07276f630ef02435900a3 Gerrit-Change-Number: 11701 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Oct 2018 21:56:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11673 ) Change subject: IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement .. Patch Set 4: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/103/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11673 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I203a800855a413069a40c728dfa157939ea15caf Gerrit-Change-Number: 11673 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Oct 2018 21:47:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7644: Hide Parquet page index writing with feature flag
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/11694 ) Change subject: IMPALA-7644: Hide Parquet page index writing with feature flag .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11694/2/be/src/exec/hdfs-parquet-table-writer.cc File be/src/exec/hdfs-parquet-table-writer.cc: http://gerrit.cloudera.org:8080/#/c/11694/2/be/src/exec/hdfs-parquet-table-writer.cc@705 PS2, Line 705: num_data_pages_ > Can you add a comment to this loop to explain why we don't use a range-base Maybe we could just drop 'num_data_pages_' and call std::vector::resize(0) on 'pages_' in Reset(). This should not affect the capacity of the vector, hence we would still save the cost of unnecessary deallocation/allocation. std::vector::clear() should also work, but I'm a bit unsure about its effect on capacity: http://www.cplusplus.com/reference/vector/vector/clear/ https://en.cppreference.com/w/cpp/container/vector/clear -- To view, visit http://gerrit.cloudera.org:8080/11694 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib4a9098a2085a385351477c715ae245d83bf1c72 Gerrit-Change-Number: 11694 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 16 Oct 2018 21:44:14 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Merge branch 'master' of https://github.com/apache/impala into impala-7076
Alex Rodoni has abandoned this change. ( http://gerrit.cloudera.org:8080/11702 ) Change subject: Merge branch 'master' of https://github.com/apache/impala into impala-7076 .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/11702 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I0165b28f020a17cadb8758883cfa591ff0f53281 Gerrit-Change-Number: 11702 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] Merge branch 'master' of https://github.com/apache/impala into impala-7076
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11702 Change subject: Merge branch 'master' of https://github.com/apache/impala into impala-7076 .. Merge branch 'master' of https://github.com/apache/impala into impala-7076 Change-Id: I0165b28f020a17cadb8758883cfa591ff0f53281 --- M docs/topics/impala_alter_database.xml 2 files changed, 7 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/11702/1 -- To view, visit http://gerrit.cloudera.org:8080/11702 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I0165b28f020a17cadb8758883cfa591ff0f53281 Gerrit-Change-Number: 11702 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/11673 ) Change subject: IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/11673/1/docs/topics/impala_alter_table.xml File docs/topics/impala_alter_table.xml: http://gerrit.cloudera.org:8080/#/c/11673/1/docs/topics/impala_alter_table.xml@269 PS1, Line 269: An : owner of a table has the ALL privilege > Also update this similar to the one in ALTER VIEW SET OWNER. We should also Done http://gerrit.cloudera.org:8080/#/c/11673/1/docs/topics/impala_alter_view.xml File docs/topics/impala_alter_view.xml: http://gerrit.cloudera.org:8080/#/c/11673/1/docs/topics/impala_alter_view.xml@105 PS1, Line 105: An owner of a view has the ALL privilege > We can rewrite it like this "When object ownership is enabled in Sentry, an Done -- To view, visit http://gerrit.cloudera.org:8080/11673 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I203a800855a413069a40c728dfa157939ea15caf Gerrit-Change-Number: 11673 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Oct 2018 21:36:49 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11673 ) Change subject: IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement .. Patch Set 4: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/103/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11673 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I203a800855a413069a40c728dfa157939ea15caf Gerrit-Change-Number: 11673 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Oct 2018 21:36:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement
Hello Fredy Wijaya, Impala Public Jenkins, Adam Holley, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11673 to look at the new patch set (#4). Change subject: IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement .. IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement Change-Id: I203a800855a413069a40c728dfa157939ea15caf --- M docs/topics/impala_alter_table.xml M docs/topics/impala_alter_view.xml 2 files changed, 444 insertions(+), 316 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/73/11673/4 -- To view, visit http://gerrit.cloudera.org:8080/11673 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I203a800855a413069a40c728dfa157939ea15caf Gerrit-Change-Number: 11673 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-4063: Merge report of query fragment instances per executor
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/11615 ) Change subject: IMPALA-4063: Merge report of query fragment instances per executor .. Patch Set 5: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/11615 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5f95e026ba05631f33f48ce32da6db39c6f421fa Gerrit-Change-Number: 11615 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 16 Oct 2018 21:34:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4063: Merge report of query fragment instances per executor
Bikramjeet Vig has posted comments on this change. (
http://gerrit.cloudera.org:8080/11615 )
Change subject: IMPALA-4063: Merge report of query fragment instances per
executor
..
Patch Set 5:
(3 comments)
as a followup to this jira, do we have any task that will look at how this will
impact scaling up cluster sizes?
http://gerrit.cloudera.org:8080/#/c/11615/5/be/src/runtime/coordinator-backend-state.cc
File be/src/runtime/coordinator-backend-state.cc:
http://gerrit.cloudera.org:8080/#/c/11615/5/be/src/runtime/coordinator-backend-state.cc@283
PS5, Line 283:
nit: can you briefly mention that every instance's instanceExecStatus in
backend_exec_status.instance_exec_status() and its thrift profile tree in the
'TRuntimeProfileForest' are at the same index. A reader will not have to trace
it back to QueryState::ConstructReport to make sure its true.
On that note, does it make sense to add the instance id to the
TRuntimeProfileTree, so that we can add a dcheck here and catch any bug that
might be caused due to any future changes to that code? Or will that be an
overkill?
http://gerrit.cloudera.org:8080/#/c/11615/3/be/src/runtime/fragment-instance-state.cc
File be/src/runtime/fragment-instance-state.cc:
http://gerrit.cloudera.org:8080/#/c/11615/3/be/src/runtime/fragment-instance-state.cc@444
PS3, Line 444: if (fragment_ctx_.fragment.output_sink.type !=
TDataSinkType::PLAN_ROOT_SINK) {
: // if we haven't already release this thread token in
Prepare(), release it now
: ReleaseThreadToken();
: }
> This still seems like a conceptual step in the fragment life cycle so it ma
wfm
http://gerrit.cloudera.org:8080/#/c/11615/3/be/src/runtime/query-state.cc
File be/src/runtime/query-state.cc:
http://gerrit.cloudera.org:8080/#/c/11615/3/be/src/runtime/query-state.cc@397
PS3, Line 397: void QueryState::ErrorDuringExecute(const Status& status, const
TUniqueId& finst_id) {
> It's okay to skip the racy call for now until evidence suggests otherwise.
wfm
--
To view, visit http://gerrit.cloudera.org:8080/11615
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5f95e026ba05631f33f48ce32da6db39c6f421fa
Gerrit-Change-Number: 11615
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Ho
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Thomas Marshall
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Tue, 16 Oct 2018 21:34:29 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7272: Fix crash in StringMinMaxFilter
Hello Michael Ho, Philip Zeyliger, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11650 to look at the new patch set (#2). Change subject: IMPALA-7272: Fix crash in StringMinMaxFilter .. IMPALA-7272: Fix crash in StringMinMaxFilter StringMinMaxFilter uses a MemPool to allocate space for StringBuffers. Previously, the MemPool was created by RuntimeFilterBank and passed to each StringMinMaxFilter. In queries with multiple StringMinMaxFilters being generated by the same fragment instance, this leads to overlapping use of the MemPool by different threads, which is incorrect as MemPools are not thread-safe. The solution is to have each StringMinMaxFilter create its own MemPool. This patch also documents MemPool as not thread-safe and introduces a DFAKE_MUTEX to help enforce correct usage. Doing this requires modifying our CMakeLists.txt to pass '-DNDEBUG' to clang only in RELEASE builds, so that the DFAKE_MUTEX will be present in the compiled IR for DEBUG builds. Testing: - I have been unable to repro the actual crash despite trying a large variety of different things. However, with additional logging added its clear that the MemPool is being used concurrently, which is incorrect. - Ran a full exhaustive build in both DEBUG and RELEASE. Change-Id: I751cad7e6b75c9d95d7ad029bbd1e52fe09e8a29 --- M be/CMakeLists.txt M be/src/runtime/mem-pool.cc M be/src/runtime/mem-pool.h M be/src/runtime/runtime-filter-bank.cc M be/src/runtime/runtime-filter-bank.h M be/src/util/min-max-filter-test.cc M be/src/util/min-max-filter.cc M be/src/util/min-max-filter.h 8 files changed, 76 insertions(+), 47 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/50/11650/2 -- To view, visit http://gerrit.cloudera.org:8080/11650 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I751cad7e6b75c9d95d7ad029bbd1e52fe09e8a29 Gerrit-Change-Number: 11650 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-7639: Move concurrent UDF tests to a custom cluster test
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11701 Change subject: IMPALA-7639: Move concurrent UDF tests to a custom cluster test .. IMPALA-7639: Move concurrent UDF tests to a custom cluster test Two test_udfs.py tests (test_native_functions_race and test_concurrent_jar_drop_use) spawn dozens of connections to test Impala behavior under concurrency. These connections use up frontend service threads and can cause shell tests to timeout when trying to connect. This moves both tests to a new TestUdfConcurrency custom cluster test. The new custom cluster test uses a larger fe_service_threads value to allow full concurrency. The tests run serially and cannot impact other tests. This also reduces the test dimensions for test_native_functions_race so that it runs one configuration rather than eight. Change-Id: I3f255823167a4dd807a07276f630ef02435900a3 --- A tests/custom_cluster/test_udf_concurrency.py M tests/query_test/test_udfs.py 2 files changed, 206 insertions(+), 162 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/11701/1 -- To view, visit http://gerrit.cloudera.org:8080/11701 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I3f255823167a4dd807a07276f630ef02435900a3 Gerrit-Change-Number: 11701 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11696 ) Change subject: IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1064/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11696 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Gerrit-Change-Number: 11696 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 16 Oct 2018 21:12:24 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11699 ) Change subject: Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code" .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/1065/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/11699 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5ab7f2f6317f1281928af5ea6cf3fd7d0c6e0a09 Gerrit-Change-Number: 11699 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Tue, 16 Oct 2018 21:10:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11625 ) Change subject: IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver" .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11625 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I679469ed7f27944f75004ec4b16d513e6ea6b544 Gerrit-Change-Number: 11625 Gerrit-PatchSet: 6 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 16 Oct 2018 21:05:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11625 ) Change subject: IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver" .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3320/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11625 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I679469ed7f27944f75004ec4b16d513e6ea6b544 Gerrit-Change-Number: 11625 Gerrit-PatchSet: 6 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 16 Oct 2018 21:05:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver"
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/11625 ) Change subject: IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver" .. Patch Set 5: gvo failure was unrelated, I filed IMPALA-7716 -- To view, visit http://gerrit.cloudera.org:8080/11625 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I679469ed7f27944f75004ec4b16d513e6ea6b544 Gerrit-Change-Number: 11625 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 16 Oct 2018 21:04:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11696 ) Change subject: IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1063/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11696 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Gerrit-Change-Number: 11696 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 16 Oct 2018 20:59:51 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code"
Lars Volker has removed Sailesh Mukil from this change. ( http://gerrit.cloudera.org:8080/11699 ) Change subject: Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code" .. Removed reviewer Sailesh Mukil. -- To view, visit http://gerrit.cloudera.org:8080/11699 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: deleteReviewer Gerrit-Change-Id: I5ab7f2f6317f1281928af5ea6cf3fd7d0c6e0a09 Gerrit-Change-Number: 11699 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Philip Zeyliger has posted comments on this change. (
http://gerrit.cloudera.org:8080/11696 )
Change subject: IMPALA-7713: Add test coverage for catalogd restart when
authorization is enabled
..
Patch Set 4: Code-Review+2
(2 comments)
I think this is fine. You could make the asserts tighter, but I don't care too
much: they're clear enough.
http://gerrit.cloudera.org:8080/#/c/11696/4/tests/authorization/test_authorization.py
File tests/authorization/test_authorization.py:
http://gerrit.cloudera.org:8080/#/c/11696/4/tests/authorization/test_authorization.py@399
PS4, Line 399: assert len(result.data) == 1
:
TestAuthorization._check_privileges(result.data[0].split("\t"),
: ["database",
"functional", "", "", "",
:"all", "false"])
If you really wanted to, you could make 399 part of _check_privileges(). And
you could probably make this entirely:
assert_equals([[...]], map(lambda x: x.split(..), result.data))
or something
http://gerrit.cloudera.org:8080/#/c/11696/4/tests/authorization/test_authorization.py@443
PS4, Line 443: # We ignore the create_time since this can be NULL depending
on the Sentry refresh.
Where is create_time here? I don't quite see what this is skipping.
--
To view, visit http://gerrit.cloudera.org:8080/11696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369
Gerrit-Change-Number: 11696
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Tue, 16 Oct 2018 20:48:55 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code"
Lars Volker has removed Tim Armstrong from this change. ( http://gerrit.cloudera.org:8080/11699 ) Change subject: Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code" .. Removed reviewer Tim Armstrong. -- To view, visit http://gerrit.cloudera.org:8080/11699 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: deleteReviewer Gerrit-Change-Id: I5ab7f2f6317f1281928af5ea6cf3fd7d0c6e0a09 Gerrit-Change-Number: 11699 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code"
Lars Volker has reverted this change. ( http://gerrit.cloudera.org:8080/11699 ) Change subject: Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code" .. Uploaded patch set 1. -- To view, visit http://gerrit.cloudera.org:8080/11699 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: revert Gerrit-Change-Id: I5ab7f2f6317f1281928af5ea6cf3fd7d0c6e0a09 Gerrit-Change-Number: 11699 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code"
Hello Philip Zeyliger, Sailesh Mukil, Tim Armstrong, Impala Public Jenkins, I'd like you to do a code review. Please visit http://gerrit.cloudera.org:8080/11699 to review the following change. Change subject: Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code" .. Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code" This reverts commit b996e0f0c8ad4ee6e4edaaeb92420ef18a3f6e26. Change-Id: I5ab7f2f6317f1281928af5ea6cf3fd7d0c6e0a09 --- M fe/src/test/resources/log4j.properties.template 1 file changed, 0 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/11699/1 -- To view, visit http://gerrit.cloudera.org:8080/11699 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5ab7f2f6317f1281928af5ea6cf3fd7d0c6e0a09 Gerrit-Change-Number: 11699 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11625 ) Change subject: IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver" .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11625 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I679469ed7f27944f75004ec4b16d513e6ea6b544 Gerrit-Change-Number: 11625 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 16 Oct 2018 20:43:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7351: Add estimates to Exchange node
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11692 ) Change subject: IMPALA-7351: Add estimates to Exchange node .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1062/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11692 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd Gerrit-Change-Number: 11692 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Tue, 16 Oct 2018 20:42:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Fredy Wijaya has posted comments on this change. (
http://gerrit.cloudera.org:8080/11696 )
Change subject: IMPALA-7713: Add test coverage for catalogd restart when
authorization is enabled
..
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11696/3/tests/authorization/test_authorization.py
File tests/authorization/test_authorization.py:
http://gerrit.cloudera.org:8080/#/c/11696/3/tests/authorization/test_authorization.py@409
PS3, Line 409:
: result = self.client.execute("show grant role %s_baz" %
unique_role)
: assert len(result.data) == 1
:
TestAuthorization._check_privileges(result.data[0].split("\t"),
: ["database",
"functional_avro", "", "", "",
:"all", "false"])
:
: self.role_cleanup(unique_role)
: try:
: self.client.execute("create role %s_foo" % unique_role)
: self.client.execute("create role %s_bar" % unique_role)
: self.client.execute("create role %s_baz" % unique_role)
: self.client.execute("grant all on database functional to
role %s_foo" %
: unique_role)
: self.client.execute("grant all on database functional_kudu
to role %s_bar" %
: unique_role)
: self.client.execute("grant all on database functional
> Can you wrap this in a function (can be local) and then run it twice?
Done
--
To view, visit http://gerrit.cloudera.org:8080/11696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369
Gerrit-Change-Number: 11696
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Tue, 16 Oct 2018 20:39:57 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Fredy Wijaya has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/11696 ) Change subject: IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled .. IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled This patch adds a test coverage for catalogd restart when authorization is enabled to ensure all privileges in the impalad's catalogs get reset after the catalogd restart to avoid stale privileges in the impalad's catalogs, which can pose a security issue. Testing: - Ran all E2E authorization tests - Added a new test Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 --- M tests/authorization/test_authorization.py M tests/conftest.py 2 files changed, 72 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/11696/4 -- To view, visit http://gerrit.cloudera.org:8080/11696 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Gerrit-Change-Number: 11696 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Philip Zeyliger has posted comments on this change. (
http://gerrit.cloudera.org:8080/11696 )
Change subject: IMPALA-7713: Add test coverage for catalogd restart when
authorization is enabled
..
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11696/3/tests/authorization/test_authorization.py
File tests/authorization/test_authorization.py:
http://gerrit.cloudera.org:8080/#/c/11696/3/tests/authorization/test_authorization.py@409
PS3, Line 409: result = self.client.execute("show grant role %s_foo" %
unique_role)
: assert len(result.data) == 1
:
TestAuthorization._check_privileges(result.data[0].split("\t"),
: ["database",
"functional", "", "", "",
:"all", "false"])
:
: result = self.client.execute("show grant role %s_bar" %
unique_role)
: assert len(result.data) == 1
:
TestAuthorization._check_privileges(result.data[0].split("\t"),
: ["database",
"functional_kudu", "", "", "",
:"all", "false"])
:
: result = self.client.execute("show grant role %s_baz" %
unique_role)
: assert len(result.data) == 1
:
TestAuthorization._check_privileges(result.data[0].split("\t"),
: ["database",
"functional_avro", "", "", "",
:"all", "false"])
Can you wrap this in a function (can be local) and then run it twice?
--
To view, visit http://gerrit.cloudera.org:8080/11696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369
Gerrit-Change-Number: 11696
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Tue, 16 Oct 2018 20:37:13 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Fredy Wijaya has posted comments on this change. (
http://gerrit.cloudera.org:8080/11696 )
Change subject: IMPALA-7713: Add test coverage for catalogd restart when
authorization is enabled
..
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/11696/2/tests/authorization/test_authorization.py
File tests/authorization/test_authorization.py:
http://gerrit.cloudera.org:8080/#/c/11696/2/tests/authorization/test_authorization.py@397
PS2, Line 397: self.role_cleanup(unique_role)
> We use "unique database" in some places. Roles are global, so we should may
Using unique_database will create a database which we don't need. Furthermore,
it also requires a special CREATE privilege which needs to be set up, but is
unnecessary for this particular test. I decided to create a new fixture called
unique_role instead, which returns a unique role name.
http://gerrit.cloudera.org:8080/#/c/11696/2/tests/authorization/test_authorization.py@413
PS2, Line 413:"all", "false"])
:
: result = self.client.execute("show grant role %s_bar" %
unique_role)
: assert len(result.data) == 1
:
TestAuthorization._check_privileges(result.data[0].split("\t"),
: ["database",
"functional_kudu", "", "", "",
:"all", "false"])
:
: result = self.client.execute("show grant role %s_baz" %
unique_role)
: assert len(result.data) == 1
:
TestAuthorization._check_privileges(result.data[0].split("\t"),
: ["database",
"functional_avro", "", "", "",
:"all", "false"])
:
:
self._start_impala_cluster(["--catalogd_args=--sentry_config=%s" %
: SENTRY_CONFIG_FILE,
"--restart_catalogd_only"])
:
> Can you run this both before and after the restart?
Done
--
To view, visit http://gerrit.cloudera.org:8080/11696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369
Gerrit-Change-Number: 11696
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Tue, 16 Oct 2018 20:25:45 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Fredy Wijaya has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/11696 ) Change subject: IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled .. IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled This patch adds a test coverage for catalogd restart when authorization is enabled to ensure all privileges in the impalad's catalogs get reset after the catalogd restart to avoid stale privileges in the impalad's catalogs, which can pose a security issue. Testing: - Ran all E2E authorization tests - Added a new test Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 --- M tests/authorization/test_authorization.py M tests/conftest.py 2 files changed, 87 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/11696/3 -- To view, visit http://gerrit.cloudera.org:8080/11696 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Gerrit-Change-Number: 11696 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11696 ) Change subject: IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1061/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11696 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Gerrit-Change-Number: 11696 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 16 Oct 2018 20:18:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7351: Add estimates to Exchange node
Bikramjeet Vig has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11692 Change subject: IMPALA-7351: Add estimates to Exchange node .. IMPALA-7351: Add estimates to Exchange node Added rough estimates for exchange node and a justification of the method in the in-line comments. Testing: Updated Planner tests. Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd --- M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/planner/ExchangeNode.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test M testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds-all.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-all.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-nested.test 11 files changed, 856 insertions(+), 797 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/11692/2 -- To view, visit http://gerrit.cloudera.org:8080/11692 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd Gerrit-Change-Number: 11692 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Philip Zeyliger has posted comments on this change. (
http://gerrit.cloudera.org:8080/11696 )
Change subject: IMPALA-7713: Add test coverage for catalogd restart when
authorization is enabled
..
Patch Set 2:
(2 comments)
Thanks for adding the test!
http://gerrit.cloudera.org:8080/#/c/11696/2/tests/authorization/test_authorization.py
File tests/authorization/test_authorization.py:
http://gerrit.cloudera.org:8080/#/c/11696/2/tests/authorization/test_authorization.py@397
PS2, Line 397: role_name_prefix = "cr_role"
We use "unique database" in some places. Roles are global, so we should maybe
use unique role_name_prefix here to make sure this is clean in that sense.
http://gerrit.cloudera.org:8080/#/c/11696/2/tests/authorization/test_authorization.py@413
PS2, Line 413: result = self.client.execute("show grant role %s_foo" %
role_name_prefix)
: assert len(result.data) == 1
:
TestAuthorization._check_privileges(result.data[0].split("\t"),
: ["database",
"functional", "", "", "",
:"all", "false"])
:
: result = self.client.execute("show grant role %s_bar" %
role_name_prefix)
: assert len(result.data) == 1
:
TestAuthorization._check_privileges(result.data[0].split("\t"),
: ["database",
"functional_kudu", "", "", "",
:"all", "false"])
:
: result = self.client.execute("show grant role %s_baz" %
role_name_prefix)
: assert len(result.data) == 1
:
TestAuthorization._check_privileges(result.data[0].split("\t"),
: ["database",
"functional_avro", "", "", "",
:"all", "false"])
Can you run this both before and after the restart?
--
To view, visit http://gerrit.cloudera.org:8080/11696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369
Gerrit-Change-Number: 11696
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Tue, 16 Oct 2018 20:03:23 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11696 Change subject: IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled .. IMPALA-7713: Add test coverage for catalogd restart when authorization is enabled This patch adds a test coverage for catalogd restart when authorization is enabled to ensure all privileges in the impalad's catalogs get reset after the catalogd restart to avoid stale privileges in the impalad's catalogs, which can pose a security issue. Testing: - Ran all E2E authorization tests - Added a new test Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 --- M tests/authorization/test_authorization.py 1 file changed, 58 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/11696/2 -- To view, visit http://gerrit.cloudera.org:8080/11696 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib9a168697401cf0b83c7a193fa477888b48cb369 Gerrit-Change-Number: 11696 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya
[Impala-ASF-CR] IMPALA-6661 Make NaN values equal for grouping purposes.
Michal Ostrowski has posted comments on this change. ( http://gerrit.cloudera.org:8080/11535 ) Change subject: IMPALA-6661 Make NaN values equal for grouping purposes. .. Patch Set 15: (1 comment) > Patch Set 14: > > (1 comment) http://gerrit.cloudera.org:8080/#/c/11535/14/testdata/workloads/functional-query/queries/QueryTest/joins.test File testdata/workloads/functional-query/queries/QueryTest/joins.test: http://gerrit.cloudera.org:8080/#/c/11535/14/testdata/workloads/functional-query/queries/QueryTest/joins.test@798 PS14, Line 798: QUERY > Yes I think t1.c1 = t2.c1 is not the same as t1.c1 <=> t2.c1; Since your pa Test in exprs.test -- To view, visit http://gerrit.cloudera.org:8080/11535 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I996c4a2e1934fd887046ed0c55457b7285375086 Gerrit-Change-Number: 11535 Gerrit-PatchSet: 15 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 16 Oct 2018 18:57:09 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11638 ) Change subject: IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs .. Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/11638/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11638/2//COMMIT_MSG@22 PS2, Line 22: tight loop in parallel shells > worthwhile making an e2e test for this? we have a couple of these for query IIRC I added a few sleeps in some codepaths to repro this, since the race window was small. Lemme try to write an e-e test and see if it repros without the patch. http://gerrit.cloudera.org:8080/#/c/11638/2/fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java File fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java: http://gerrit.cloudera.org:8080/#/c/11638/2/fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java@145 PS2, Line 145: cause_ == null > This could be guarded equivalently at the call-site with isLoaded(). Here i you mean use !isLoaded() here instead of (cause_ == null)? http://gerrit.cloudera.org:8080/#/c/11638/2/fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java@149 PS2, Line 149: more meaningful lookup status > perhaps TABLE_NOT_LOADED? will do. -- To view, visit http://gerrit.cloudera.org:8080/11638 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705 Gerrit-Change-Number: 11638 Gerrit-PatchSet: 2 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 16 Oct 2018 17:44:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/11673 ) Change subject: IMPALA-7076: [DOCS] Document ALTER TABLE / VIEW SET OWNER statement .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/11673/1/docs/topics/impala_alter_table.xml File docs/topics/impala_alter_table.xml: http://gerrit.cloudera.org:8080/#/c/11673/1/docs/topics/impala_alter_table.xml@269 PS1, Line 269: An : owner of a table has effectively the ALL Done Also update this similar to the one in ALTER VIEW SET OWNER. We should also update the one in ALTER DATABASE SET OWNER. http://gerrit.cloudera.org:8080/#/c/11673/1/docs/topics/impala_alter_view.xml File docs/topics/impala_alter_view.xml: http://gerrit.cloudera.org:8080/#/c/11673/1/docs/topics/impala_alter_view.xml@105 PS1, Line 105: An owner of a view has effectively the ALL Not a good Sentry doc to link. Waiting for a short info to add here from Fr We can rewrite it like this "When object ownership is enabled in Sentry, an owner of a view can have "ALL with GRANT" or "ALL without GRANT", but ". -- To view, visit http://gerrit.cloudera.org:8080/11673 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I203a800855a413069a40c728dfa157939ea15caf Gerrit-Change-Number: 11673 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Oct 2018 17:33:13 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11625 ) Change subject: IMPALA-7678: Reapply "IMPALA-7660: Support ECDH ciphers for debug webserver" .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3319/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/11625 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I679469ed7f27944f75004ec4b16d513e6ea6b544 Gerrit-Change-Number: 11625 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 16 Oct 2018 16:54:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6658: improve Parquet RLE for low bit widths
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11582 ) Change subject: IMPALA-6658: improve Parquet RLE for low bit widths .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1060/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11582 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I191a581d3f699b6669e48ac9dc39c76ed77c4a76 Gerrit-Change-Number: 11582 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 16 Oct 2018 16:13:54 + Gerrit-HasComments: No
