[Impala-ASF-CR] IMPALA-9256: Refactor constraint information into a separate class.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15213 ) Change subject: IMPALA-9256: Refactor constraint information into a separate class. .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5338/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/15213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3f1c441c24df84d2d0791ffe94dff60d039a3341 Gerrit-Change-Number: 15213 Gerrit-PatchSet: 1 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 14 Feb 2020 05:37:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 14 Feb 2020 04:26:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. IMPALA-9304: Support starting Hive with Ranger in minicluster Add a new flag -with_ranger in testdata/bin/run-hive-server.sh to start Hive with Ranger integration. The relative configuration files are generated in bin/create-test-configuration.sh using a new varient ranger_auth in hive-site.xml.py. Only Hive3 is supported. Current limitation: Can't use different username in Beeline by the -n option. "select current_user()" keeps returning my username, while "select logged_in_user()" can return the username given by -n option but it's not used in authorization. Tests: - Ran bin/create-test-configuration.sh and verified the generated hive-site_ranger_auth.xml contains Ranger configurations. - Ran testdata/bin/run-hive-server.sh -with_ranger. Verified column masking and row filtering policies took effect in Beeline. - Added test in test_ranger.py for this mode. Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Reviewed-on: http://gerrit.cloudera.org:8080/15189 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M bin/create-test-configuration.sh M fe/src/test/resources/hive-site.xml.py M testdata/bin/run-hive-server.sh A testdata/workloads/functional-query/queries/QueryTest/hive_ranger_integration.test M tests/authorization/test_ranger.py M tests/common/impala_connection.py M tests/common/skip.py 7 files changed, 93 insertions(+), 5 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 7 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-8690 (prep 3): Factor out common code for cache implementations
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15179 ) Change subject: IMPALA-8690 (prep 3): Factor out common code for cache implementations .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15179 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I67294244a3e8a2812f1482fe786bf7f8e6ce054e Gerrit-Change-Number: 15179 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Fri, 14 Feb 2020 03:12:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9383: Fix hang in hs2-http server with large, chunks request
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15218 ) Change subject: IMPALA-9383: Fix hang in hs2-http server with large, chunks request .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15218 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If822fb9cd2d7f2b0f5e36879fd7ccbba9a217ad0 Gerrit-Change-Number: 15218 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 14 Feb 2020 01:45:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8870: Bump up guava version from 14.0.1 to 28.1-jre
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15214 ) Change subject: IMPALA-8870: Bump up guava version from 14.0.1 to 28.1-jre .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5220/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15214 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9690a926953a8d3c3872277680b4be0551546c68 Gerrit-Change-Number: 15214 Gerrit-PatchSet: 3 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 14 Feb 2020 00:43:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8870: Bump up guava version from 14.0.1 to 28.1-jre
Fang-Yu Rao has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/15214 ) Change subject: IMPALA-8870: Bump up guava version from 14.0.1 to 28.1-jre .. IMPALA-8870: Bump up guava version from 14.0.1 to 28.1-jre This patch bumps up the version of Guava libraries from 14.0.1 to 28.1-jre. Due to some changes in Guava's API's, we modify the call sites accordingly. Note that in order to instruct the Java classes under the directory of $IMPALA_HOME/common/yarn-extras to use the new Guava libraries, we also explicitly added a dependency in the corresponding pom.xml file. Change-Id: I9690a926953a8d3c3872277680b4be0551546c68 --- M common/yarn-extras/pom.xml M common/yarn-extras/src/main/java/org/apache/impala/yarn/server/resourcemanager/scheduler/fair/AllocationFileLoaderService.java M fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java M fe/src/main/java/org/apache/impala/analysis/AggregateInfoBase.java M fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java M fe/src/main/java/org/apache/impala/analysis/AnalyticInfo.java M fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/main/java/org/apache/impala/analysis/BoolLiteral.java M fe/src/main/java/org/apache/impala/analysis/CaseExpr.java M fe/src/main/java/org/apache/impala/analysis/CastExpr.java M fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java M fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/main/java/org/apache/impala/analysis/IsNullPredicate.java M fe/src/main/java/org/apache/impala/analysis/NullLiteral.java M fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java M fe/src/main/java/org/apache/impala/analysis/Path.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/StringLiteral.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/Column.java M fe/src/main/java/org/apache/impala/catalog/ColumnStats.java M fe/src/main/java/org/apache/impala/catalog/DataSource.java M fe/src/main/java/org/apache/impala/catalog/FileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/TableLoader.java M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java M fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java M fe/src/main/java/org/apache/impala/planner/DataPartition.java M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java M fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java M fe/src/main/java/org/apache/impala/planner/HashJoinNode.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/NestedLoopJoinNode.java M fe/src/main/java/org/apache/impala/planner/ScanNode.java M fe/src/main/java/org/apache/impala/planner/SortNode.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/util/JvmPauseMonitor.java M fe/src/test/java/org/apache/impala/catalog/local/CatalogdMetaProviderTest.java M impala-parent/pom.xml 48 files changed, 104 insertions(+), 94 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/15214/3 -- To view, visit http://gerrit.cloudera.org:8080/15214 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9690a926953a8d3c3872277680b4be0551546c68 Gerrit-Change-Number: 15214 Gerrit-PatchSet: 3 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-9384: Improve Impala shell usability by automatically enable live progress in the interactive mode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15219 ) Change subject: IMPALA-9384: Improve Impala shell usability by automatically enable live_progress in the interactive mode .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5219/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15219 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3765b775f663fa227e59728acffe4d5ea9a5e2d3 Gerrit-Change-Number: 15219 Gerrit-PatchSet: 1 Gerrit-Owner: Alice Fan Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 13 Feb 2020 23:50:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9107 (part 1): Add scripts to produce an m2 archive
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/14562 ) Change subject: IMPALA-9107 (part 1): Add scripts to produce an m2 archive .. Patch Set 13: Rebased this, nothing has changed. I'm thinking it would be good to get this in and try to use the archive. If we later decide to do our own maven cache, we can turn this off. -- To view, visit http://gerrit.cloudera.org:8080/14562 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I043912f5fbc7cf24ee80b2855354656aa587ca9f Gerrit-Change-Number: 14562 Gerrit-PatchSet: 13 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 13 Feb 2020 23:40:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5337/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 13 Feb 2020 23:36:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 13 Feb 2020 23:36:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 5: > Patch Set 5: > > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5331/ Failed in a flaky test (IMPALA-9091) again... https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/9206 -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 13 Feb 2020 23:33:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8909: Fix incorrectly omitting implicit cast in pre-insert sort
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15217 ) Change subject: IMPALA-8909: Fix incorrectly omitting implicit cast in pre-insert sort .. Patch Set 1: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5334/ -- To view, visit http://gerrit.cloudera.org:8080/15217 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icca8ab1456a3b840a47833119c9d4fd31a1fff90 Gerrit-Change-Number: 15217 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 13 Feb 2020 23:29:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7031: Improve error message for invalid query handle
Alice Fan has abandoned this change. ( http://gerrit.cloudera.org:8080/12926 ) Change subject: IMPALA-7031: Improve error message for invalid query handle .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/12926 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I56983d40e0542bc734ec5a66c339b5131b7b56c8 Gerrit-Change-Number: 12926 Gerrit-PatchSet: 12 Gerrit-Owner: Alice Fan Gerrit-Reviewer: Alice Fan Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. Patch Set 13: Thanks for the contribution! -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 13 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Thu, 13 Feb 2020 23:08:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9384: Improve Impala shell usability by automatically enable live progress in the interactive mode
Alice Fan has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15219 Change subject: IMPALA-9384: Improve Impala shell usability by automatically enable live_progress in the interactive mode .. IMPALA-9384: Improve Impala shell usability by automatically enable live_progress in the interactive mode Currently, shell option live_progress is set to False by default. The patch makes Impala shell automatically enable it in the interactive mode when users do not set the option in both configure file and command line. This will help to improve usability by automatically showing the query processing status to users. Testing: - Updated and added tests in test_shell_interactive.py - Successfully ran all other shell interactive related tests Change-Id: I3765b775f663fa227e59728acffe4d5ea9a5e2d3 --- M shell/impala_shell.py M tests/shell/test_shell_interactive.py 2 files changed, 15 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/15219/1 -- To view, visit http://gerrit.cloudera.org:8080/15219 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I3765b775f663fa227e59728acffe4d5ea9a5e2d3 Gerrit-Change-Number: 15219 Gerrit-PatchSet: 1 Gerrit-Owner: Alice Fan Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-9366: Remove embedded pointer reference in PhjBuilder::CodegenInsertRuntimeFilters
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15186 ) Change subject: IMPALA-9366: Remove embedded pointer reference in PhjBuilder::CodegenInsertRuntimeFilters .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5333/ -- To view, visit http://gerrit.cloudera.org:8080/15186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I09037b5832b037536bde9324f7a2a1077854 Gerrit-Change-Number: 15186 Gerrit-PatchSet: 3 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 13 Feb 2020 22:57:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8960 (prep 1): Copy Kudu cache code to be/src/util/cache
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15169 ) Change subject: IMPALA-8960 (prep 1): Copy Kudu cache code to be/src/util/cache .. Patch Set 6: Code-Review+2 Carry +2 -- To view, visit http://gerrit.cloudera.org:8080/15169 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0e7b8b852b56613803dea125456fa9e8b3736d76 Gerrit-Change-Number: 15169 Gerrit-PatchSet: 6 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 13 Feb 2020 22:23:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8690 (prep 3): Factor out common code for cache implementations
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15179 ) Change subject: IMPALA-8690 (prep 3): Factor out common code for cache implementations .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5336/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/15179 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I67294244a3e8a2812f1482fe786bf7f8e6ce054e Gerrit-Change-Number: 15179 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 13 Feb 2020 22:23:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. Patch Set 12: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 12 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Thu, 13 Feb 2020 21:43:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI This change appends the socket address (HOST:PORT) of the client who made the request to close a session or cancel a query from the coordinator's debug WebUI. Existing statuses: "Cancelled from Impala's debug web interface" "Session closed from Impala's debug web interface" New statuses: "Cancelled from Impala's debug web interface by client at :" "Session closed from Impala's debug web interface by client at :" Testing: -Verified visually that the status message is printed in the impalad log with the socket address when one cancels a query or closes a session. -Added a new e2e test to verify that the new status gets printed in runtime profile and coordinator log when a query is cancelled in this way. -Made log asserts more robust by adding a timeout/wait value. Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Reviewed-on: http://gerrit.cloudera.org:8080/14782 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/kudu/util/web_callback_registry.h M be/src/service/impala-http-handler.cc M be/src/util/webserver.cc M tests/common/impala_test_suite.py M tests/observability/test_log_fragments.py M tests/webserver/test_web_pages.py 6 files changed, 77 insertions(+), 30 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 13 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran
[Impala-ASF-CR] WIP: Asynchronous code generation
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/15105 ) Change subject: WIP: Asynchronous code generation .. Patch Set 9: (1 comment) http://gerrit.cloudera.org:8080/#/c/15105/9/be/src/codegen/codegen-fn-ptr.h File be/src/codegen/codegen-fn-ptr.h: http://gerrit.cloudera.org:8080/#/c/15105/9/be/src/codegen/codegen-fn-ptr.h@41 PS9, Line 41: static constexpr std::memory_order mem_order_ = std::memory_order_acq_rel; just passing through: I think you have it correct here. Relaxed is insuffiicent because it would permit reordering of the writing of the code and the storing of the pointer. In other words, your compilation thread does things like: 1. allocate memory for function machine code 2. write code to allocated memory 3. publish pointer and without "release" semantics on the publishing of the pointer, there is no guarantee that #2 and #3 won't be reordered. In practice, this is really unlikely to happen, because there are probably some other locking operations in between #2 and #3 and because x86 is a TSO architecture, but worth doing it right regardless. Also worth noting that acq/rel semantics on x86 isn't expensive, they are just simple load/store operations. It only prevents compiler reordering. So, there is little to gain by using relaxed. -- To view, visit http://gerrit.cloudera.org:8080/15105 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia7cbfa7c6734dcf03641629429057d6a4194aa6b Gerrit-Change-Number: 15105 Gerrit-PatchSet: 9 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 13 Feb 2020 21:18:56 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 5: Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5331/ -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 13 Feb 2020 21:03:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9383: Fix hang in hs2-http server with large, chunks request
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15218 ) Change subject: IMPALA-9383: Fix hang in hs2-http server with large, chunks request .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5335/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/15218 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If822fb9cd2d7f2b0f5e36879fd7ccbba9a217ad0 Gerrit-Change-Number: 15218 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 13 Feb 2020 20:52:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9383: Fix hang in hs2-http server with large, chunks request
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15218 ) Change subject: IMPALA-9383: Fix hang in hs2-http server with large, chunks request .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5218/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15218 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If822fb9cd2d7f2b0f5e36879fd7ccbba9a217ad0 Gerrit-Change-Number: 15218 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 13 Feb 2020 20:51:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8712: Make ExecQueryFInstances async
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15154 ) Change subject: IMPALA-8712: Make ExecQueryFInstances async .. Patch Set 5: (4 comments) http://gerrit.cloudera.org:8080/#/c/15154/5/be/src/runtime/coordinator-backend-state.cc File be/src/runtime/coordinator-backend-state.cc: http://gerrit.cloudera.org:8080/#/c/15154/5/be/src/runtime/coordinator-backend-state.cc@195 PS5, Line 195: rpc_latency_ I'm not sure if this variable is thread safe anymore - I think this Cb is being invoked by a kRPC thread right? but it gets read later on in Coordinator::FinishBackendStartup without holding onto the BackendState lock_ I think the same issue applies to exec_rpc_status_ It looks like reads of last_report_time_ms_ get around this by acquiring the lock_ inside the BackendState::last_report_time_ms() method. http://gerrit.cloudera.org:8080/#/c/15154/5/be/src/runtime/coordinator-backend-state.cc@227 PS5, Line 227: SCOPED_TIMER(exec_serialization_timer_); move this to right before the ThriftSerializer is called? otherwise if the below code throws an error or returns, this could be non-zero, even if no serialization is done, which seems odd http://gerrit.cloudera.org:8080/#/c/15154/5/be/src/runtime/coordinator.cc File be/src/runtime/coordinator.cc: http://gerrit.cloudera.org:8080/#/c/15154/5/be/src/runtime/coordinator.cc@447 PS5, Line 447: ThriftSerializer serializer(true); should SCOPED_TIMER(exec_serialization_timer_); go here as well? http://gerrit.cloudera.org:8080/#/c/15154/5/be/src/runtime/coordinator.cc@982 PS5, Line 982: DCHECK(exec_rpcs_complete_.Load()) << "Exec() must be called first."; Will this break the ImpalaServer UnresponsiveBackendThread thread? It calls this method for each registered query, and if I'm reading the code in ImpalaServer::ExecuteInternal correctly, it register the query before calling Coordinator::Exec I think some of the other DCHECKs might suffer from the same issue - specifically any that are called due to a HTTP request from the Web UI. Since those are triggered asynchronously there could be a race condition between when the query is registered, when the HTTP request is fired, and when Coordinator::Exec is actually called. For example, BackendsToJson is called by ImpalaHttpHandler::QueryBackendsHandler. -- To view, visit http://gerrit.cloudera.org:8080/15154 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I33ec96e5885af094c294cd3a76c242995263ba32 Gerrit-Change-Number: 15154 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 13 Feb 2020 20:22:24 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9383: Fix hang in hs2-http server with large, chunks request
Thomas Tauber-Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15218 Change subject: IMPALA-9383: Fix hang in hs2-http server with large, chunks request .. IMPALA-9383: Fix hang in hs2-http server with large, chunks request The issue is that in each call to THttpTransport::read(), it always starts by calling refill(), which tries to read more data off the socket, but for chunked requests each call to read() only processes a single chunk. So, if more than one chunk is read off the socket at a time (which happens with large requests due to an algorithm that increases the amount of data read off the socket with each read), you can end up with more chunks still needing to be processed but no more data to read off the socket, and the next call to THttpTransport::read() will hang when it calls refill(). The solution is to not always call refill() at the beginning of each call to THttpTransport::read(). This works because the functions that actually process the data, readLine() and readContent() will call refill() themselves anyways if they actually need more data. Testing: - Added a BE test that uses curl to send chunked requests to the hs2 http server. Change-Id: If822fb9cd2d7f2b0f5e36879fd7ccbba9a217ad0 --- M be/src/rpc/CMakeLists.txt R be/src/rpc/hs2-http-test.cc M be/src/transport/THttpTransport.cpp 3 files changed, 65 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/15218/1 -- To view, visit http://gerrit.cloudera.org:8080/15218 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: If822fb9cd2d7f2b0f5e36879fd7ccbba9a217ad0 Gerrit-Change-Number: 15218 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-9075: Add support for reading zstd text files
Xiaomeng Zhang has posted comments on this change. (
http://gerrit.cloudera.org:8080/15023 )
Change subject: IMPALA-9075: Add support for reading zstd text files
..
Patch Set 5:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/15023/5/be/src/util/decompress.cc
File be/src/util/decompress.cc:
http://gerrit.cloudera.org:8080/#/c/15023/5/be/src/util/decompress.cc@616
PS5, Line 616: Status ZstandardDecompressor::Init() {
> Init() gets called for both block and streaming code paths. Since it's only
Do you have suggestions where should it be init? I can't put it inside
ProcessBlockStreaming as it might be called multiple times for one stream, and
we should use one dctx for one stream.
--
To view, visit http://gerrit.cloudera.org:8080/15023
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2adce9fe00190558525fa5cd3d50cf5e0f0b0aa4
Gerrit-Change-Number: 15023
Gerrit-PatchSet: 5
Gerrit-Owner: Xiaomeng Zhang
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: Andrew Sherman
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Xiaomeng Zhang
Gerrit-Comment-Date: Thu, 13 Feb 2020 19:45:45 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8909: Fix incorrectly omitting implicit cast in pre-insert sort
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15217 ) Change subject: IMPALA-8909: Fix incorrectly omitting implicit cast in pre-insert sort .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5217/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15217 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icca8ab1456a3b840a47833119c9d4fd31a1fff90 Gerrit-Change-Number: 15217 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 13 Feb 2020 19:10:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9199: Add support for single query retries on cluster membership changes
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14824 ) Change subject: IMPALA-9199: Add support for single query retries on cluster membership changes .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/14824/5/be/src/service/impala-server.cc File be/src/service/impala-server.cc: http://gerrit.cloudera.org:8080/#/c/14824/5/be/src/service/impala-server.cc@1025 PS5, Line 1025: query_ctx->query_id = UuidToQueryId(random_generator()()); > I sort of did this in the previous patch update. I added a client_query_id_ I'll try thinking through this some more. One challenge with creating an internal vs. user facing id is that a common debugging practice is to take user facing id and use it to grep through the logs. A lot of the log statements contain the query id they are associated with. So the user would have to manually figure out the mapping from the user facing id (perhaps it got the id from the runtime profile or the client logs) and the internal id. -- To view, visit http://gerrit.cloudera.org:8080/14824 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e4a0e72a9bf8ec10b91639aefd81bef17886ddd Gerrit-Change-Number: 14824 Gerrit-PatchSet: 5 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 13 Feb 2020 19:06:57 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9199: Add support for single query retries on cluster membership changes
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14824 ) Change subject: IMPALA-9199: Add support for single query retries on cluster membership changes .. Patch Set 7: Instead of creating a separate doc, think it might be easier to just post comments here. I've done some re-factoring locally and this is what I have so far: * Created a "QueryDriver", which (as suggested by Thomas) is a layer of abstraction of on top of ClientRequestState * "QueryDriver" owns the ClientRequestState; all access to the ClientRequestState goes through the QueryDriver * Since the first patch will only support a single retry, the QueryDriver has a regular client_request_state_ and a retried_client_request_state_ (one for the original query and the other for the retried query) * QueryDriver owns the TExecRequest that is used by both the original and retried queries * The "transparent" part of this feature is now handled by the QueryDriver (since the QueryDriver owns all access to the ClientRequestState) * The QueryDriver has two methods: GetActiveClientRequestState() and GetClientRequestState(query_id) * GetActiveClientRequestState() returns the CRS for the "active" query - e.g. if the query has not been retried yet, it returns the original query; if the query has been retried, it returns the retried query * GetClientRequestState(query_id) returns the CRS for the query that matches the given 'query_id' - this allows clients to get the CRS of exactly the query_id they are looking for * ImpalaServer creates a QueryDriver for each query; the ImpalaServer::client_request_state_map_ has been replaced with the ImpalaServer::query_driver_map_ which maps query ids to QueryDrivers * The query_driver_map_ can have key, values pairs with the same value (e.g. the same QueryDriver) * Moved all of the retry code (along with the retry threadpool) into QueryDriver (specifically QueryDriver::RetryQueryFromThreadPool) Example: * Client submits a query for execution * Impala creates a QueryDriver for the query * QueryDriver::CreateClientRequestState creates a ClientRequestState for the query (this CRS is owned by the QueryDriver) * ImpalaServer::RegisterQuery "registers" the query and adds an entry to ImpalaServer::query_driver_map_ that maps the query id to the QueryDriver * QueryDriver::RunFrontendPlanner creates the TExecRequest (owned by the QueryDriver) * Eventually, the query fails and is retried * QueryDriver::RetryQueryFromThreadPool contains all the retry logic (moved from ImpalaServer) * It cancels the original query, creates the new one and runs it * It calls ImpalaServer::RegisterQuery again, but with the retried query id * It manipulates some internal pointers so that retried_client_request_state_ now points to the CRS of the retried query -- To view, visit http://gerrit.cloudera.org:8080/14824 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e4a0e72a9bf8ec10b91639aefd81bef17886ddd Gerrit-Change-Number: 14824 Gerrit-PatchSet: 7 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 13 Feb 2020 19:05:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8909: Fix incorrectly omitting implicit cast in pre-insert sort
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15217 ) Change subject: IMPALA-8909: Fix incorrectly omitting implicit cast in pre-insert sort .. Patch Set 1: Code-Review+2 I think this is the right fix - it should ensure that the root type of the exprs doesn't get changed. There are some broader challenges with the whole implicit cast concept. -- To view, visit http://gerrit.cloudera.org:8080/15217 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icca8ab1456a3b840a47833119c9d4fd31a1fff90 Gerrit-Change-Number: 15217 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 13 Feb 2020 18:40:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8909: Fix incorrectly omitting implicit cast in pre-insert sort
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15217 ) Change subject: IMPALA-8909: Fix incorrectly omitting implicit cast in pre-insert sort .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5334/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/15217 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icca8ab1456a3b840a47833119c9d4fd31a1fff90 Gerrit-Change-Number: 15217 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 13 Feb 2020 18:29:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8870: Bump up guava version from 14.0.1 to 28.1-jre
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15214 ) Change subject: IMPALA-8870: Bump up guava version from 14.0.1 to 28.1-jre .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java File fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java: http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@67 PS1, Line 67: import static java.util.concurrent.TimeUnit.MILLISECONDS; nit, I think parts of the code use TimeUnit.MILLISECONDS which is probably more clear. May be remove the static import? -- To view, visit http://gerrit.cloudera.org:8080/15214 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9690a926953a8d3c3872277680b4be0551546c68 Gerrit-Change-Number: 15214 Gerrit-PatchSet: 1 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 13 Feb 2020 18:31:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8587: show grant does not produce correct privileges
Fang-Yu Rao has abandoned this change. ( http://gerrit.cloudera.org:8080/14356 ) Change subject: IMPALA-8587: show grant does not produce correct privileges .. Abandoned IMPALA-8587 has been merged upstream so we abandon this patch set. -- To view, visit http://gerrit.cloudera.org:8080/14356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I8511656fe386a37a66d20e07ce1b875190bc4b65 Gerrit-Change-Number: 14356 Gerrit-PatchSet: 4 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-8909: Fix incorrectly omitting implicit cast in pre-insert sort
Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15217 Change subject: IMPALA-8909: Fix incorrectly omitting implicit cast in pre-insert sort .. IMPALA-8909: Fix incorrectly omitting implicit cast in pre-insert sort Inserts can add a sort node that orders the rows by partitioning and Kudu primary key columns (aka. clustered insert). The issue occurred when the target column was a timestamp and the source was an expression that returned a string (e.g. concat()). Impala adds an implicit cast to convert the strings to timestamps before sorting, but this cast was incorrectly removed later during expression substitution. This led to hitting a DCHECK in debug builds and a (not too informative) error message in release mode. Note that the cast in question is not visible in EXPLAIN outputs. Explain should contain implicit casts from explain_level=2 since https://gerrit.cloudera.org/#/c/11719/ , but it is still not shown in some expressions. I consider this to be a separate issue. Testing: - added an EE test that used to crash - ran planner / sort / kudu_insert tests Change-Id: Icca8ab1456a3b840a47833119c9d4fd31a1fff90 --- M fe/src/main/java/org/apache/impala/planner/SortNode.java M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test 2 files changed, 18 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/15217/1 -- To view, visit http://gerrit.cloudera.org:8080/15217 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Icca8ab1456a3b840a47833119c9d4fd31a1fff90 Gerrit-Change-Number: 15217 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer
[Impala-ASF-CR] IMPALA-9256: Refactor constraint information into a separate class.
Anurag Mantripragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/15213 ) Change subject: IMPALA-9256: Refactor constraint information into a separate class. .. Patch Set 1: The build errors are due to row filtering. Unrelated to this change. -- To view, visit http://gerrit.cloudera.org:8080/15213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3f1c441c24df84d2d0791ffe94dff60d039a3341 Gerrit-Change-Number: 15213 Gerrit-PatchSet: 1 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 13 Feb 2020 18:17:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9366: Remove embedded pointer reference in PhjBuilder::CodegenInsertRuntimeFilters
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15186 ) Change subject: IMPALA-9366: Remove embedded pointer reference in PhjBuilder::CodegenInsertRuntimeFilters .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I09037b5832b037536bde9324f7a2a1077854 Gerrit-Change-Number: 15186 Gerrit-PatchSet: 3 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 13 Feb 2020 18:03:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9366: Remove embedded pointer reference in PhjBuilder::CodegenInsertRuntimeFilters
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15186 ) Change subject: IMPALA-9366: Remove embedded pointer reference in PhjBuilder::CodegenInsertRuntimeFilters .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5333/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I09037b5832b037536bde9324f7a2a1077854 Gerrit-Change-Number: 15186 Gerrit-PatchSet: 3 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 13 Feb 2020 18:03:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15199 ) Change subject: IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output .. Patch Set 2: IMO it would be better to format the output of 'show table stats [impala-kudu-table-name]' as follows: +-+ | # Rows | +-+ | [total-num-rows] | +-+ Having the partition information with 'NA' can be confusing, it makes it seem like there is a bug in Impala or Kudu that is causing the stats to not be available. Plus it doesn't add any extra information. Added Thomas to the review to see what he thinks as well. -- To view, visit http://gerrit.cloudera.org:8080/15199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ice4b8df65f0a53fe14b8fbe35d82c9887ab9a041 Gerrit-Change-Number: 15199 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 13 Feb 2020 17:45:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3343: Make impala-shell compatible with python 3.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15132 ) Change subject: IMPALA-3343: Make impala-shell compatible with python 3. .. Patch Set 12: (4 comments) I did a pass over the change and had some high level questions/suggestions. I haven't focused yet on the string-handling part. http://gerrit.cloudera.org:8080/#/c/15132/12//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/15132/12//COMMIT_MSG@39 PS12, Line 39: USE_THRIFT11_GEN_PY=true impala-py.test --shell_executable=//bin/impala-shell -sv shell/test_shell_commandline.py Is thrift 11 a requirement for python 3 support? http://gerrit.cloudera.org:8080/#/c/15132/12//COMMIT_MSG@115 PS12, Line 115: What ultimately seemed like a better approach was to try to weed out as many Is there any way to document or enforce that the strings in the code are a particular type of string? It might help at least to ensure that some parts of the shell code are clean. http://gerrit.cloudera.org:8080/#/c/15132/12//COMMIT_MSG@116 PS12, Line 116: existing spurious str.encode() and str.decode() calls as I could. I'm not sure Also, do you have any references to best practices or other info about python string handling? http://gerrit.cloudera.org:8080/#/c/15132/12/shell/impala_shell.py File shell/impala_shell.py: http://gerrit.cloudera.org:8080/#/c/15132/12/shell/impala_shell.py@51 PS12, Line 51: try: Would it be worth factoring out some of these compatibility patterns into a separate file? E.g. you could import like this. from compat import xrange, basestring It might be overkill, but it did occur to me while reading through -- To view, visit http://gerrit.cloudera.org:8080/15132 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb75e162bac0faeae3e12106c15da39cbfb8b462 Gerrit-Change-Number: 15132 Gerrit-PatchSet: 12 Gerrit-Owner: David Knupp Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 13 Feb 2020 17:03:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. Patch Set 11: Agreed, I'll rerun -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 11 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Thu, 13 Feb 2020 16:53:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. Patch Set 12: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 12 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Thu, 13 Feb 2020 16:53:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14782 ) Change subject: IMPALA-9182: Print the socket address of the client closing a session or cancelling a query from the WebUI .. Patch Set 12: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5332/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf74ad06ce1c40fab4ce37de6b7ca78e3e520b43 Gerrit-Change-Number: 14782 Gerrit-PatchSet: 12 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Thu, 13 Feb 2020 16:53:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9366: Remove embedded pointer reference in PhjBuilder::CodegenInsertRuntimeFilters
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15186 ) Change subject: IMPALA-9366: Remove embedded pointer reference in PhjBuilder::CodegenInsertRuntimeFilters .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I09037b5832b037536bde9324f7a2a1077854 Gerrit-Change-Number: 15186 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 13 Feb 2020 16:46:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5331/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 13 Feb 2020 16:11:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8870: Bump up guava version from 14.0.1 to 28.1-jre
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15214 ) Change subject: IMPALA-8870: Bump up guava version from 14.0.1 to 28.1-jre .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java File fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java: http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java@765 PS1, Line 765: queryStr.length() - 1 The endIndex is exclusive so I think we should use queryStr.length(). Otherwise an empty string will results in -1 which will cause String.subSequence() throw an IndexOutOfBoundsException. Actually String implements CharSequence interface. So we don't need to call subSequence() here. -- To view, visit http://gerrit.cloudera.org:8080/15214 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9690a926953a8d3c3872277680b4be0551546c68 Gerrit-Change-Number: 15214 Gerrit-PatchSet: 1 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 13 Feb 2020 15:22:46 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 5: > Patch Set 5: Verified-1 > > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5330/ Caused by two unrelative failures: 1. https://jenkins.impala.io/job/clang-tidy-ub1604/10263 ++ sudo wget -nv https://www-us.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz Unable to establish SSL connection. 2. https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/9199 failed by IMPALA-9152 -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 13 Feb 2020 13:56:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 5: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5330/ -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 13 Feb 2020 13:50:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9256: Refactor constraint information into a separate class.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15213 ) Change subject: IMPALA-9256: Refactor constraint information into a separate class. .. Patch Set 1: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5329/ -- To view, visit http://gerrit.cloudera.org:8080/15213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3f1c441c24df84d2d0791ffe94dff60d039a3341 Gerrit-Change-Number: 15213 Gerrit-PatchSet: 1 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 13 Feb 2020 13:39:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4224: execute separate join builds fragments
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/14859 )
Change subject: IMPALA-4224: execute separate join builds fragments
..
Patch Set 40: Code-Review+1
(5 comments)
Went through the java part and tests. Only found some potential beautifications.
I also went through the c++ part in a past PS, it seemed ok though I didn't
dive into it deep enough.
http://gerrit.cloudera.org:8080/#/c/14859/38/fe/src/main/java/org/apache/impala/planner/PlanFragment.java
File fe/src/main/java/org/apache/impala/planner/PlanFragment.java:
http://gerrit.cloudera.org:8080/#/c/14859/38/fe/src/main/java/org/apache/impala/planner/PlanFragment.java@304
PS38, Line 304: Preconditions.checkState(sink_ instanceof JoinBuildSink);
Check in the loops looks a bit weird to me - I would prefer to check it outside
even if it makes the code longer.
http://gerrit.cloudera.org:8080/#/c/14859/38/fe/src/main/java/org/apache/impala/planner/PlanFragment.java@407
PS38, Line 407:
Preconditions.checkState(perInstanceResourceProfile_.isValid());
:
Preconditions.checkState(perBackendResourceProfile_.isValid());
:
Preconditions.checkArgument(perInstanceInitialMemReservationTotalClaims_ > -1);
:
Preconditions.checkArgument(perBackendInitialMemReservationTotalClaims_ > -1);
:
Preconditions.checkArgument(producedRuntimeFiltersMemReservationBytes_ > -1);
:
Preconditions.checkArgument(consumedGlobalRuntimeFiltersMemReservationBytes_ >
-1);
I think that this is not the best place for these checks - we combine two
invariants:
1. toThrift() should be only called after computeResourceProfile()
2. all of these resource estimates should be filled at the end of
computeResourceProfile()
Note that there is a comment around line 67 that describes a state machine with
a function called finalize() which doesn't exist. It would be nice to update
the comment.
http://gerrit.cloudera.org:8080/#/c/14859/40/fe/src/main/java/org/apache/impala/planner/PlanNode.java
File fe/src/main/java/org/apache/impala/planner/PlanNode.java:
http://gerrit.cloudera.org:8080/#/c/14859/40/fe/src/main/java/org/apache/impala/planner/PlanNode.java@476
PS40, Line 476: // ExchangeNodes and separated join builds) to have no
children.
: if (this instanceof ExchangeNode) {
: msg.num_children = 0;
: return;
: } else if (this instanceof JoinNode &&
((JoinNode)this).hasSeparateBuild()) {
: // The join build is in a separate fragment so only the
left child should be
: // serialized.
: msg.num_children = 1;
: Preconditions.checkState(children_.size() == 2);
: children_.get(0).treeToThriftHelper(container);
: } else {
: msg.num_children = children_.size();
: for (PlanNode child: children_) {
: child.treeToThriftHelper(container);
: }
: }
Can't we generalize this by checking if fragment_ is the same as in the child?
http://gerrit.cloudera.org:8080/#/c/14859/40/fe/src/main/java/org/apache/impala/planner/ResourceProfile.java
File fe/src/main/java/org/apache/impala/planner/ResourceProfile.java:
http://gerrit.cloudera.org:8080/#/c/14859/40/fe/src/main/java/org/apache/impala/planner/ResourceProfile.java@157
PS40, Line 157: combine
Is sum() still useful after creating combine? Are there case where specifically
want buffer sizes to be undefined?
http://gerrit.cloudera.org:8080/#/c/14859/40/fe/src/main/java/org/apache/impala/planner/ResourceProfile.java@172
PS40, Line 172: returns the max
We return invalid instead of max if I understand correctly
--
To view, visit http://gerrit.cloudera.org:8080/14859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4403c8e62d9c13854e7830602ee613f8efc80c58
Gerrit-Change-Number: 14859
Gerrit-PatchSet: 40
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Thu, 13 Feb 2020 12:40:39 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP: Asynchronous code generation
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15105 ) Change subject: WIP: Asynchronous code generation .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5216/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15105 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia7cbfa7c6734dcf03641629429057d6a4194aa6b Gerrit-Change-Number: 15105 Gerrit-PatchSet: 9 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 13 Feb 2020 11:50:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6663: Expose current DDL metrics on WebUI
Norbert Luksa has posted comments on this change. ( http://gerrit.cloudera.org:8080/13806 ) Change subject: IMPALA-6663: Expose current DDL metrics on WebUI .. Patch Set 13: Code-Review+1 (2 comments) Left a small nit, but lgtm. http://gerrit.cloudera.org:8080/#/c/13806/8/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/13806/8/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@427 PS8, Line 427: case DROP_FUNCTION: > It is possible that I misunderstood the concerns here, but I will try to an I see, I misunderstood something, thanks for clarifying. http://gerrit.cloudera.org:8080/#/c/13806/13/fe/src/test/java/org/apache/impala/catalog/monitor/CatalogFinalizeDmlCounterTest.java File fe/src/test/java/org/apache/impala/catalog/monitor/CatalogFinalizeDmlCounterTest.java: http://gerrit.cloudera.org:8080/#/c/13806/13/fe/src/test/java/org/apache/impala/catalog/monitor/CatalogFinalizeDmlCounterTest.java@34 PS13, Line 34: private final String TEST_UPDATE_SQL = "UPDATE table SET c3 = " : + "'test';"; nit: this string could be written without concatenation. -- To view, visit http://gerrit.cloudera.org:8080/13806 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0ed76f134bad6d3b3d4dce132365a53a01e9512a Gerrit-Change-Number: 13806 Gerrit-PatchSet: 13 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 13 Feb 2020 11:41:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP: Asynchronous code generation
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/15105 ) Change subject: WIP: Asynchronous code generation .. Patch Set 9: (1 comment) http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/hdfs-scanner.h File be/src/exec/hdfs-scanner.h: http://gerrit.cloudera.org:8080/#/c/15105/6/be/src/exec/hdfs-scanner.h@462 PS6, Line 462: decltype(auto) invoke(ThisT This, CodegenFnPt > Since this is a wrapper function I think decltype(auto) should be preferred Thanks, I didn't know that. -- To view, visit http://gerrit.cloudera.org:8080/15105 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia7cbfa7c6734dcf03641629429057d6a4194aa6b Gerrit-Change-Number: 15105 Gerrit-PatchSet: 9 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 13 Feb 2020 11:04:19 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP: Asynchronous code generation
Daniel Becker has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/15105 ) Change subject: WIP: Asynchronous code generation .. WIP: Asynchronous code generation This commit introduces optional asynchronous code generation. Asynchronous code generation means that instead of waiting for codegen to finish, the query starts in interpreted mode while codegen is done on another thread. All the function pointers that point to codegen'd functions are changed to be atomic, wrapped in a CodegenFnPtr. These are initialised to nullptr and as long as they are nullptr, the corresponding interpreted functions are used (as before). When code generation is ready, the funtion pointers are set by the codegen thread. No synchronisation is needed as the function pointers are atomic and it is not a problem if, at a given moment, only a subset of the codegen'd function pointers are set and the rest are interpreted. Asynchronous code generation can be turned on using the ASYNC_CODEGEN boolean query option. TODO: The default should be synchronous codegen for now. TODO: Testing. TODO: Benchmarks. Change-Id: Ia7cbfa7c6734dcf03641629429057d6a4194aa6b --- M be/src/benchmarks/hash-benchmark.cc A be/src/codegen/codegen-fn-ptr.h M be/src/codegen/llvm-codegen-test.cc M be/src/codegen/llvm-codegen.cc M be/src/codegen/llvm-codegen.h M be/src/exec/grouping-aggregator.cc M be/src/exec/grouping-aggregator.h M be/src/exec/hdfs-avro-scanner.cc M be/src/exec/hdfs-avro-scanner.h M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/hdfs-scanner.cc M be/src/exec/hdfs-scanner.h M be/src/exec/hdfs-sequence-scanner.cc M be/src/exec/hdfs-text-scanner.cc M be/src/exec/non-grouping-aggregator.cc M be/src/exec/non-grouping-aggregator.h M be/src/exec/parquet/hdfs-parquet-scanner.cc M be/src/exec/parquet/hdfs-parquet-scanner.h M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/partitioned-hash-join-builder.h M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/partitioned-hash-join-node.h M be/src/exec/select-node.cc M be/src/exec/select-node.h M be/src/exec/topn-node.cc M be/src/exec/topn-node.h M be/src/exec/union-node.cc M be/src/exec/union-node.h M be/src/exprs/expr-codegen-test.cc M be/src/exprs/scalar-expr.cc M be/src/exprs/scalar-expr.h M be/src/exprs/scalar-expr.inline.h M be/src/exprs/scalar-fn-call.cc M be/src/exprs/scalar-fn-call.h M be/src/runtime/fragment-instance-state.cc M be/src/runtime/krpc-data-stream-sender.cc M be/src/runtime/krpc-data-stream-sender.h M be/src/runtime/runtime-state.h M be/src/service/query-options.cc M be/src/service/query-options.h M be/src/util/tuple-row-compare.cc M be/src/util/tuple-row-compare.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift 45 files changed, 487 insertions(+), 227 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/15105/9 -- To view, visit http://gerrit.cloudera.org:8080/15105 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia7cbfa7c6734dcf03641629429057d6a4194aa6b Gerrit-Change-Number: 15105 Gerrit-PatchSet: 9 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8870: Bump up guava version from 14.0.1 to 28.1-jre
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/15214 ) Change subject: IMPALA-8870: Bump up guava version from 14.0.1 to 28.1-jre .. Patch Set 1: Code-Review+1 (9 comments) http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@2071 PS1, Line 2071: createUnstarted I would prefer to remove the static import as this is not as clear as Stopwatch.createUnstarted().start() would be. Stopwatch.createStarted() would be even better. http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/catalog/Table.java File fe/src/main/java/org/apache/impala/catalog/Table.java: http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/catalog/Table.java@414 PS1, Line 414: createUnstarted Same as in CatalogServiceCatalog.java http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/catalog/TableLoader.java File fe/src/main/java/org/apache/impala/catalog/TableLoader.java: http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/catalog/TableLoader.java@59 PS1, Line 59: createUnstarted Same as in CatalogServiceCatalog.java http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java File fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java: http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@110 PS1, Line 110: import static com.google.common.base.Stopwatch.createUnstarted; Same as in CatalogServiceCatalog.java http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@199 PS1, Line 199: import static com.google.common.base.Stopwatch.createUnstarted; Same as in CatalogServiceCatalog.java http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1752 PS1, Line 1752: ; double ; http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/util/JvmPauseMonitor.java File fe/src/main/java/org/apache/impala/util/JvmPauseMonitor.java: http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/util/JvmPauseMonitor.java@189 PS1, Line 189: createUnstarted I would prefer the longer Stopwatch.createUnstarted() http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/main/java/org/apache/impala/util/JvmPauseMonitor.java@190 PS1, Line 190: Stopwatch timeSinceDeadlockCheck = createUnstarted().start(); Same as in CatalogServiceCatalog.java http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/test/java/org/apache/impala/catalog/local/CatalogdMetaProviderTest.java File fe/src/test/java/org/apache/impala/catalog/local/CatalogdMetaProviderTest.java: http://gerrit.cloudera.org:8080/#/c/15214/1/fe/src/test/java/org/apache/impala/catalog/local/CatalogdMetaProviderTest.java@290 PS1, Line 290: Stopwatch sw = createUnstarted().start(); Same as in CatalogServiceCatalog.java -- To view, visit http://gerrit.cloudera.org:8080/15214 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9690a926953a8d3c3872277680b4be0551546c68 Gerrit-Change-Number: 15214 Gerrit-PatchSet: 1 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 13 Feb 2020 10:21:26 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5330/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 13 Feb 2020 09:01:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9304: Support starting Hive with Ranger in minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15189 ) Change subject: IMPALA-9304: Support starting Hive with Ranger in minicluster .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01e3a195b00a98388244a922a1a79e65146cec42 Gerrit-Change-Number: 15189 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 13 Feb 2020 09:01:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9256: Refactor constraint information into a separate class.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15213 ) Change subject: IMPALA-9256: Refactor constraint information into a separate class. .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5329/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/15213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3f1c441c24df84d2d0791ffe94dff60d039a3341 Gerrit-Change-Number: 15213 Gerrit-PatchSet: 1 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 13 Feb 2020 08:45:17 + Gerrit-HasComments: No
