[Impala-ASF-CR] Revert "Fix docs for catalogd automatic invalidate flags"
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13523 ) Change subject: Revert "Fix docs for catalogd automatic invalidate flags" .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13523 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I14b1b27491a50209d4e80f206e0ecbfb397db81b Gerrit-Change-Number: 13523 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 07 Jun 2019 05:11:40 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Fix docs for catalogd automatic invalidate flags"
Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13523 ) Change subject: Revert "Fix docs for catalogd automatic invalidate flags" .. Revert "Fix docs for catalogd automatic invalidate flags" This reverts commit 5f1b00c0809657bd3fa07567670ac88714f7f253. Change-Id: I14b1b27491a50209d4e80f206e0ecbfb397db81b Reviewed-on: http://gerrit.cloudera.org:8080/13523 Tested-by: Impala Public Jenkins Reviewed-by: Todd Lipcon --- M docs/topics/impala_metadata.xml 1 file changed, 4 insertions(+), 3 deletions(-) Approvals: Impala Public Jenkins: Verified Todd Lipcon: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/13523 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I14b1b27491a50209d4e80f206e0ecbfb397db81b Gerrit-Change-Number: 13523 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/13525/2/tests/authorization/test_ranger.py File tests/authorization/test_ranger.py: http://gerrit.cloudera.org:8080/#/c/13525/2/tests/authorization/test_ranger.py@566 PS2, Line 566: invalid_impala_client, : statement, : user=invalid_user) it's weird that you need two different clients, but you're passing the user info here in the query instead of in the client creation. I guess this is probably a weird artifact of the test client code or something? Would it be straight-forward to refactor/fix this? If not, mind adding some comment explaining what's going on here? http://gerrit.cloudera.org:8080/#/c/13525/2/tests/authorization/test_ranger.py@578 PS2, Line 578: HTTP 403 Error is there any way we can get rid of the HTTP error part of the error message? This seems very specific to the underlying implementation of ranger and just confusing to a user -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 07 Jun 2019 05:05:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8436: Prohibit write/alter operations on materialized view
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13503 ) Change subject: IMPALA-8436: Prohibit write/alter operations on materialized view .. Patch Set 4: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4426/ -- To view, visit http://gerrit.cloudera.org:8080/13503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idcd619303e19b5a2551876a63d67569c76bd22f0 Gerrit-Change-Number: 13503 Gerrit-PatchSet: 4 Gerrit-Owner: Sudhanshu Arora Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sudhanshu Arora Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 07 Jun 2019 03:57:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1)
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13522 ) Change subject: IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1) .. Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/13522/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13522/4//COMMIT_MSG@28 PS4, Line 28: - Passed core tests Did you do any testing on S3 or other object stores? Not sure what the mtime story is there. http://gerrit.cloudera.org:8080/#/c/13522/4/be/src/exec/hdfs-scan-node-base.h File be/src/exec/hdfs-scan-node-base.h: http://gerrit.cloudera.org:8080/#/c/13522/4/be/src/exec/hdfs-scan-node-base.h@278 PS4, Line 278: /// Old API for compatibility with text scanners (e.g. LZO text scanner). Maybe we can remove this one now? Since I think the version of the LZO scanner we're building against has been upgraded. It's mostly orthogonal but removing an overload makes it easier to see that callers are calling the right overload. -- To view, visit http://gerrit.cloudera.org:8080/13522 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I48b7ed60d6ab9104b993237b4fe23de5dc058672 Gerrit-Change-Number: 13522 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 03:14:09 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8630: Include partition id when calculating consistent remote placement
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13545 ) Change subject: IMPALA-8630: Include partition id when calculating consistent remote placement .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/13545/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13545/1//COMMIT_MSG@17 PS1, Line 17: This adds the partition_id in the hash, so files with the same How are the partition IDs generated? I'm wondering if there might be a less severe version of the same problem where there are collisions in partition IDs between different tables? I think the general idea of adding something unique to the hash per-partition makes sense, just wanting to make sure I understand the solution. -- To view, visit http://gerrit.cloudera.org:8080/13545 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I46c739fc31af539af2b3509e2a161f4e29f44d7b Gerrit-Change-Number: 13545 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 03:03:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13542 ) Change subject: Fix integration of kudu-hive.jar .. Fix integration of kudu-hive.jar IMPALA-8503 added downloading kudu-hive.jar and adding it to HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to start with Kudu's HMS plugin. There are two problems with this that are fixed by this patch: - Previously, we fully specify the expected jar filename based on the value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH but this is overly restrictive for users who may wish to override this value in impala-config-branch.sh to build their own branch with a different version of the kudu-hive.jar This patch relaxes this restriction by adding any jar containing the string kudu-hive in IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH - In bootstrap_toolchain, we don't download a package if its directory already exists. Since the 'kudu' and 'kudu-java' packages download to the same directory, this led to a race condition where 'kudu-java' might not be downloaded if 'kudu' had already been unpacked when it started. This patch fixes this by inspecting the contents of the Kudu package directory to look for specific files expected for each Kudu package. Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Reviewed-on: http://gerrit.cloudera.org:8080/13542 Reviewed-by: Thomas Marshall Reviewed-by: Hao Hao Tested-by: Impala Public Jenkins --- M bin/bootstrap_toolchain.py M bin/impala-config.sh M testdata/bin/run-hive-server.sh 3 files changed, 21 insertions(+), 5 deletions(-) Approvals: Thomas Marshall: Looks good to me, approved Hao Hao: Looks good to me, but someone else must approve Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 ) Change subject: Fix integration of kudu-hive.jar .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 02:18:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8633 : Insert event should not error when table does not exists
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13548 ) Change subject: IMPALA-8633 : Insert event should not error when table does not exists .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3531/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13548 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I961cd7cbede4c248dba538c7fabb4bc708e49693 Gerrit-Change-Number: 13548 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 07 Jun 2019 01:33:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8630: Include partition id when calculating consistent remote placement
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13545 ) Change subject: IMPALA-8630: Include partition id when calculating consistent remote placement .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3530/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13545 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I46c739fc31af539af2b3509e2a161f4e29f44d7b Gerrit-Change-Number: 13545 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Jun 2019 01:15:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8633 : Insert event should not error when table does not exists
Vihang Karajgaonkar has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/13548 ) Change subject: IMPALA-8633 : Insert event should not error when table does not exists .. IMPALA-8633 : Insert event should not error when table does not exists When a insert event is received by the events processor it is possible that the table has already been removed from metastore. In such cases, the reloadTable method throws a NoSuchObjectException which is unhandled. This causes EventProcessor to go in NEEDS_INVALIDATE state. The patch adds exception handling for such cases so that the error can be ignored and a warning is logged. Ideally, if the table has been removed from metastore, this insert event should be followed by a drop-table event and hence the table is cleaned up from the catalog subsequently. Hence Event processor does not need to remove the table from the catalog. Also, the patch adds the exception trace to the error messages during processing of Insert events to improve debugging ability in case of such errors. Testing: Refactored the existing test for insert events into util methods which can be reused for other tests. Added a new test case which generates a insert event on a table which has been dropped. This test reproduces the scenario seen by debugging test failures in IMPALA-8567. The newly added test succeeds after the patch. Change-Id: I961cd7cbede4c248dba538c7fabb4bc708e49693 --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 5 files changed, 180 insertions(+), 131 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/13548/4 -- To view, visit http://gerrit.cloudera.org:8080/13548 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I961cd7cbede4c248dba538c7fabb4bc708e49693 Gerrit-Change-Number: 13548 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8633 : Insert event should not error when table does not exists
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/13548 ) Change subject: IMPALA-8633 : Insert event should not error when table does not exists .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/13548/2/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java File fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java: http://gerrit.cloudera.org:8080/#/c/13548/2/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java@631 PS2, Line 631: msTbl = metaStoreClient.getHiveClient().getTable(TEST_DB_NAME, createInsertDropTable); > line too long (92 > 90) Done http://gerrit.cloudera.org:8080/#/c/13548/2/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java@739 PS2, Line 739: Path parentPath = partition == null ? new Path(msTbl.getSd().getLocation()) : new Path(partition.getSd().getLocation()); > line too long (124 > 90) Done -- To view, visit http://gerrit.cloudera.org:8080/13548 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I961cd7cbede4c248dba538c7fabb4bc708e49693 Gerrit-Change-Number: 13548 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 07 Jun 2019 01:14:39 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1)
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13522 ) Change subject: IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1) .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4427/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13522 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I48b7ed60d6ab9104b993237b4fe23de5dc058672 Gerrit-Change-Number: 13522 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 00:32:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8630: Include partition id when calculating consistent remote placement
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13545 Change subject: IMPALA-8630: Include partition id when calculating consistent remote placement .. IMPALA-8630: Include partition id when calculating consistent remote placement Consistent remote placement currently uses the relative filename within a partition for the consistent hash. If the relative filenames for different partitions have a simple naming scheme, then multiple partitions may have files of the same name. This is true for some tables written by Hive (e.g. in our minicluster the tpcds.store_sales has this problem). This can lead to unbalanced placement of remote ranges. This adds the partition_id in the hash, so files with the same name from different partitions will have different hashes. Change-Id: I46c739fc31af539af2b3509e2a161f4e29f44d7b --- M be/src/scheduling/scheduler.cc 1 file changed, 5 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/13545/1 -- To view, visit http://gerrit.cloudera.org:8080/13545 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I46c739fc31af539af2b3509e2a161f4e29f44d7b Gerrit-Change-Number: 13545 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
[Impala-ASF-CR] IMPALA-8629: Adjust new KuduStorageHandler package
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13541 ) Change subject: IMPALA-8629: Adjust new KuduStorageHandler package .. Patch Set 2: As expected the HMS sync tests failed, but it looks like everything else is passing: https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/6144/ -- To view, visit http://gerrit.cloudera.org:8080/13541 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 Gerrit-Change-Number: 13541 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 00:13:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8629: Adjust new KuduStorageHandler package
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13541 ) Change subject: IMPALA-8629: Adjust new KuduStorageHandler package .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4423/ -- To view, visit http://gerrit.cloudera.org:8080/13541 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 Gerrit-Change-Number: 13541 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 00:10:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1)
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13522 ) Change subject: IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1) .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3527/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13522 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I48b7ed60d6ab9104b993237b4fe23de5dc058672 Gerrit-Change-Number: 13522 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 06 Jun 2019 23:32:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8567 : Improve failure messages for insert events
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13548 ) Change subject: IMPALA-8567 : Improve failure messages for insert events .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3528/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13548 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I961cd7cbede4c248dba538c7fabb4bc708e49693 Gerrit-Change-Number: 13548 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 06 Jun 2019 23:30:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8536: Add Scalable Pool Configuration to Admission Controller.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/13307 ) Change subject: IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. .. Patch Set 7: (21 comments) Addressing review comments, main changes are: Switched to using ClusterMembershipMgr to get cluster membership. Enhanced ClusterMembershipMgr to add a count of non-quiescing backends to the Snapshot. Added a simple unit test for AdmissionControoler::PoolDisabled. Added metrics for derived configuration values. http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller-test.cc File be/src/scheduling/admission-controller-test.cc: http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller-test.cc@279 PS3, Line 279: // AdmissionController. > Consider breaking this up into a few more targeted tests, I think it makes I have tidied up and added comments. http://gerrit.cloudera.org:8080/#/c/13307/6/be/src/scheduling/admission-controller-test.cc File be/src/scheduling/admission-controller-test.cc: http://gerrit.cloudera.org:8080/#/c/13307/6/be/src/scheduling/admission-controller-test.cc@71 PS6, Line 71: > Comment could be more descriptive. Done http://gerrit.cloudera.org:8080/#/c/13307/6/be/src/scheduling/admission-controller-test.cc@74 PS6, Line 74: QuerySchedule* MakeQuerySchedule(string request_pool_name, TPoolConfig& config, > DCHECK_GT Done http://gerrit.cloudera.org:8080/#/c/13307/6/be/src/scheduling/admission-controller-test.cc@175 PS6, Line 175: } > Extra spaces after /// Done http://gerrit.cloudera.org:8080/#/c/13307/6/be/src/scheduling/admission-controller-test.cc@204 PS6, Line 204: } > Comment is a little uninformative. Maybe "Make an AdmissionController with Done http://gerrit.cloudera.org:8080/#/c/13307/5/be/src/scheduling/admission-controller.h File be/src/scheduling/admission-controller.h: http://gerrit.cloudera.org:8080/#/c/13307/5/be/src/scheduling/admission-controller.h@663 PS5, Line 663: > nit: Uses a Done http://gerrit.cloudera.org:8080/#/c/13307/5/be/src/scheduling/admission-controller.h@664 PS5, Line 664: pool is configured. : static std::string GetMaxMemForPoolDescription( : const TPoolConfig& pool_config, int64_t latest_cluster_size); > nit: not sure if this detail is relevant there. maybe remove this? Done http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.h File be/src/scheduling/admission-controller.h: http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.h@603 PS3, Line 603: > Not a common term but just reviewing IMPALA-8460 drilled into me that the n Ah now I understand what you didn't like. Going with your original suggestion 'latest_cluster_size' http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.cc File be/src/scheduling/admission-controller.cc: http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.cc@1396 PS3, Line 1396: uge( > I thought about this more and I think not counting the quiescing backends w Thanks for raising the question.I am now not counting the quiescing backends. http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.cc@1444 PS3, Line 1444: tu > should be OR (||) YES! http://gerrit.cloudera.org:8080/#/c/13307/5/be/src/scheduling/admission-controller.cc File be/src/scheduling/admission-controller.cc: http://gerrit.cloudera.org:8080/#/c/13307/5/be/src/scheduling/admission-controller.cc@1051 PS5, Line 1051: > nit: outdated comment Done http://gerrit.cloudera.org:8080/#/c/13307/6/be/src/scheduling/admission-controller.cc File be/src/scheduling/admission-controller.cc: http://gerrit.cloudera.org:8080/#/c/13307/6/be/src/scheduling/admission-controller.cc@1390 PS6, Line 1390: metrics_.max_queued_queries_multiple = parent_->metrics_group_->AddDoubleGauge( > I think we need to be a little careful with this being O(cluster size). If Thanks, new implementation uses ClusterMembershipMgr and the value is added to value in ClusterMembershipMgr::Snapshot . http://gerrit.cloudera.org:8080/#/c/13307/6/be/src/scheduling/admission-controller.cc@1445 PS6, Line 1445: > Should this be ||? Since we shouldn't be able to submit queries to a pool i Yes! http://gerrit.cloudera.org:8080/#/c/13307/5/tests/custom_cluster/test_admission_controller.py File tests/custom_cluster/test_admission_controller.py: http://gerrit.cloudera.org:8080/#/c/13307/5/tests/custom_cluster/test_admission_controller.py@994 PS5, Line 994: dmission > nit: Impalads Done http://gerrit.cloudera.org:8080/#/c/13307/5/tests/custom_cluster/test_admission_controller.py@1002 PS5, Line 1002: :param expected_rejection_reason: a string expected to be in the reason for rejection. > nit: by convention, we usually prefix a helper method with a double undersc Done
[Impala-ASF-CR] IMPALA-8536: Add Scalable Pool Configuration to Admission Controller.
Andrew Sherman has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/13307 ) Change subject: IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. .. IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. Add 3 configuration parameters to Admission Controller that scale with the number of hosts in the resource pool. These parameters are specified to the Impalad through the -llama_site_path flag which points to a Llama XML configuration file. The new configuration parameters are: + Max Running Queries Multiple - this floating point number is multiplied by the current total number of executors at runtime to give the maximum number of concurrently running queries allowed in the pool. This calculation is rounded up to the nearest integer so the result will always be at least one as long as the parameter is non-zero. + Max Queued Queries Multiple - this floating point number is multiplied by the current total number of executors at runtime to give the maximum number of queries that can be queued in the pool. This calculation is rounded up to the nearest integer so the result will always be at least one as long as the parameter is non-zero. + Max Memory Multiple - this number of bytes is multiplied by the current total number of executors at runtime to give the maximum memory available across the cluster for the pool. If any of these parameters have zero value then they will be ignored. In this case the corresponding non-scalable parameters will be used, if they are set. The new parameters are exposed through the webui. At various points in the code Admission Controller looks at the Pool Config objects to find non-scalable parameters such as the max number of queries that can run in the pool. These access have been encapsulated in functions that return the scalable version of the configuration value if the new scalable parameters are being used. Diagnostic messages are enhanced to show the origin of the encapsulated parameters. TESTING All end-to-end tests are running clean with ASAN. The unit test admission-controller-test.cc has been expanded to test the newly added code. Added an end-to-end test that adds and removes Impalads from a minicluster. Change-Id: If47508728124076f3b9200c27cffc989f7a4f188 --- M be/src/runtime/exec-env.cc M be/src/scheduling/admission-controller-test.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M be/src/scheduling/cluster-membership-mgr-test.cc M be/src/scheduling/cluster-membership-mgr.cc M be/src/scheduling/cluster-membership-mgr.h M common/thrift/ImpalaInternalService.thrift M common/thrift/generate_metrics.py M common/thrift/metrics.json M fe/src/main/java/org/apache/impala/util/RequestPoolService.java M fe/src/test/java/org/apache/impala/util/TestRequestPoolService.java M fe/src/test/resources/fair-scheduler-test.xml M fe/src/test/resources/fair-scheduler-test2.xml M fe/src/test/resources/llama-site-test.xml M fe/src/test/resources/llama-site-test2.xml M testdata/workloads/functional-query/queries/QueryTest/admission-max-min-mem-limits.test M testdata/workloads/functional-query/queries/QueryTest/admission-reject-mem-estimate.test M testdata/workloads/functional-query/queries/QueryTest/admission-reject-min-reservation.test M tests/common/impala_cluster.py M tests/custom_cluster/test_admission_controller.py M tests/webserver/test_web_pages.py M www/admission_controller.tmpl 23 files changed, 1,356 insertions(+), 298 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/13307/7 -- To view, visit http://gerrit.cloudera.org:8080/13307 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If47508728124076f3b9200c27cffc989f7a4f188 Gerrit-Change-Number: 13307 Gerrit-PatchSet: 7 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8567 : Improve failure messages for insert events
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13548 ) Change subject: IMPALA-8567 : Improve failure messages for insert events .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/13548/2/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java File fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java: http://gerrit.cloudera.org:8080/#/c/13548/2/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java@631 PS2, Line 631: msTbl = metaStoreClient.getHiveClient().getTable(TEST_DB_NAME, createInsertDropTable); line too long (92 > 90) http://gerrit.cloudera.org:8080/#/c/13548/2/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java@739 PS2, Line 739: Path parentPath = partition == null ? new Path(msTbl.getSd().getLocation()) : new Path(partition.getSd().getLocation()); line too long (124 > 90) -- To view, visit http://gerrit.cloudera.org:8080/13548 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I961cd7cbede4c248dba538c7fabb4bc708e49693 Gerrit-Change-Number: 13548 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 06 Jun 2019 22:52:32 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8567 : Improve failure messages for insert events
Vihang Karajgaonkar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13548 Change subject: IMPALA-8567 : Improve failure messages for insert events .. IMPALA-8567 : Improve failure messages for insert events This patch adds the exception to the error message during processing of Insert events to improve debugging ability in case of errors. Also, refactors the existing tests for insert events into util methods which can be reused for other tests. Added a new test case which generates a insert event on a table which has been dropped. This scenario was seen while debugging test failures reported in IMPALA-8567. Although, in this case the test does not error out. Testing done: Ran existing tests for events processor Change-Id: I961cd7cbede4c248dba538c7fabb4bc708e49693 --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 4 files changed, 163 insertions(+), 127 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/13548/2 -- To view, visit http://gerrit.cloudera.org:8080/13548 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I961cd7cbede4c248dba538c7fabb4bc708e49693 Gerrit-Change-Number: 13548 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada
[Impala-ASF-CR] IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1)
Hello Michael Ho, Lars Volker, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13522 to look at the new patch set (#4). Change subject: IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1) .. IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1) The file handle cache uses the mtime to distinguish different versions of a file separate. For example, if a file at mtime=1 is overwritten with a version at mtime=2, the old file handle from mtime=1 will not be used for the mtime=2 version. In some codepaths, for legacy reasons, the mtime would be unconditionally set to -1, and this eliminates the ability to distinguish between different versions of files. There is no need to set the mtime to -1. It seems to be a legacy bit of cruft. This removes the mtime=-1 behavior for HDFS scan ranges. It removes mtime from BufferOpts and plumbs the mtime through the scan range codepaths separately. Local non-HDFS files do not use the mtime, so those continue to use mtime=-1. Testing: - Passed core tests Change-Id: I48b7ed60d6ab9104b993237b4fe23de5dc058672 --- M be/src/exec/base-sequence-scanner.cc M be/src/exec/hdfs-orc-scanner.cc M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/hdfs-scanner.cc M be/src/exec/hdfs-text-scanner.cc M be/src/exec/parquet/hdfs-parquet-scanner.cc M be/src/exec/parquet/parquet-column-readers.cc M be/src/exec/parquet/parquet-page-index.cc M be/src/exec/scanner-context.cc M be/src/runtime/io/data-cache-test.cc M be/src/runtime/io/disk-io-mgr-stress.cc M be/src/runtime/io/disk-io-mgr-test.cc M be/src/runtime/io/handle-cache.inline.h M be/src/runtime/io/request-ranges.h M be/src/runtime/io/scan-range.cc M be/src/runtime/tmp-file-mgr.cc 17 files changed, 65 insertions(+), 61 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/13522/4 -- To view, visit http://gerrit.cloudera.org:8080/13522 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I48b7ed60d6ab9104b993237b4fe23de5dc058672 Gerrit-Change-Number: 13522 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8436: Prohibit write/alter operations on materialized view
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13503 ) Change subject: IMPALA-8436: Prohibit write/alter operations on materialized view .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4426/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idcd619303e19b5a2551876a63d67569c76bd22f0 Gerrit-Change-Number: 13503 Gerrit-PatchSet: 4 Gerrit-Owner: Sudhanshu Arora Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sudhanshu Arora Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Jun 2019 22:25:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration
Thomas Marshall has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13409 ) Change subject: IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration .. IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration This commit intends to support the actual handling of ALTER/RENAME TABLE DDL for managed Kudu tables with Kudu's integration with the Hive Metastore. However, currently Kudu is considered as the source of truth of the table schema, so when ALTER TABLE (ADD/DROP COLUMN/RANGE_PARTITION), Impala always directly alters/loads the Kudu tables. Thus, this commit only updates RENAME TABLE DDL, so that after the table is renamed in the Kudu, relies on Kudu to rename the table in the HMS. Change-Id: If7155e0b385b8ad81eda0a84277bc85171a88269 Reviewed-on: http://gerrit.cloudera.org:8080/13409 Reviewed-by: Grant Henke Reviewed-by: Alexey Serbin Reviewed-by: Thomas Marshall Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test A testdata/workloads/functional-query/queries/QueryTest/kudu_hms_alter.test M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 5 files changed, 679 insertions(+), 16 deletions(-) Approvals: Grant Henke: Looks good to me, but someone else must approve Alexey Serbin: Looks good to me, but someone else must approve Thomas Marshall: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/13409 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If7155e0b385b8ad81eda0a84277bc85171a88269 Gerrit-Change-Number: 13409 Gerrit-PatchSet: 6 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13409 ) Change subject: IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13409 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If7155e0b385b8ad81eda0a84277bc85171a88269 Gerrit-Change-Number: 13409 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 22:11:50 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 ) Change subject: Fix integration of kudu-hive.jar .. Patch Set 4: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 21:20:08 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 ) Change subject: Fix integration of kudu-hive.jar .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3524/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 21:05:47 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13544 ) Change subject: Fix integration of kudu-hive.jar .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3525/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13544 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia874efa63d9114716da3b2ac8921e78b518bdae5 Gerrit-Change-Number: 13544 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 06 Jun 2019 20:50:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13409 ) Change subject: IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13409 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If7155e0b385b8ad81eda0a84277bc85171a88269 Gerrit-Change-Number: 13409 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 20:37:14 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 ) Change subject: Fix integration of kudu-hive.jar .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3523/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 20:34:10 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Thomas Marshall has abandoned this change. ( http://gerrit.cloudera.org:8080/13544 ) Change subject: Fix integration of kudu-hive.jar .. Abandoned pushed by mistake -- To view, visit http://gerrit.cloudera.org:8080/13544 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Ia874efa63d9114716da3b2ac8921e78b518bdae5 Gerrit-Change-Number: 13544 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 ) Change subject: Fix integration of kudu-hive.jar .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4424/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 20:33:20 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 ) Change subject: Fix integration of kudu-hive.jar .. Patch Set 4: Code-Review+2 carrying forward -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 20:33:07 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Thomas Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13544 Change subject: Fix integration of kudu-hive.jar .. Fix integration of kudu-hive.jar IMPALA-8503 added downloading kudu-hive.jar and adding it to HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to start with Kudu's HMS plugin. There are two problems with this that are fixed by this patch: - Previously, we fully specify the expected jar filename based on the value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH but this is overly restrictive for users who may wish to override this value in impala-config-branch.sh to build their own branch with a different version of the kudu-hive.jar This patch relaxes this restriction by adding any jar containing the string kudu-hive in IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH - In bootstrap_toolchain, we don't download a package if its directory already exists. Since the 'kudu' and 'kudu-java' packages download to the same directory, this led to a race condition where 'kudu-java' might not be downloaded if 'kudu' had already been unpacked when it started. This patch fixes this by inspecting the contents of the Kudu package directory to look for specific files expected for each Kudu package. Change-Id: Ia874efa63d9114716da3b2ac8921e78b518bdae5 --- M bin/bootstrap_toolchain.py M bin/impala-config.sh M testdata/bin/run-hive-server.sh 3 files changed, 21 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/44/13544/1 -- To view, visit http://gerrit.cloudera.org:8080/13544 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia874efa63d9114716da3b2ac8921e78b518bdae5 Gerrit-Change-Number: 13544 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Hello Lars Volker, Hao Hao, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13542 to look at the new patch set (#4). Change subject: Fix integration of kudu-hive.jar .. Fix integration of kudu-hive.jar IMPALA-8503 added downloading kudu-hive.jar and adding it to HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to start with Kudu's HMS plugin. There are two problems with this that are fixed by this patch: - Previously, we fully specify the expected jar filename based on the value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH but this is overly restrictive for users who may wish to override this value in impala-config-branch.sh to build their own branch with a different version of the kudu-hive.jar This patch relaxes this restriction by adding any jar containing the string kudu-hive in IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH - In bootstrap_toolchain, we don't download a package if its directory already exists. Since the 'kudu' and 'kudu-java' packages download to the same directory, this led to a race condition where 'kudu-java' might not be downloaded if 'kudu' had already been unpacked when it started. This patch fixes this by inspecting the contents of the Kudu package directory to look for specific files expected for each Kudu package. Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 --- M bin/bootstrap_toolchain.py M bin/impala-config.sh M testdata/bin/run-hive-server.sh 3 files changed, 21 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/13542/4 -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 ) Change subject: Fix integration of kudu-hive.jar .. Patch Set 3: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py File bin/bootstrap_toolchain.py: http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py@433 PS2, Line 433: debug > Both the 'debug' and the 'release' directories will be present regardless o Thx for the explanation, might be good to leave that in a comment here for future reference. -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 20:26:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Hello Lars Volker, Hao Hao, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13542 to look at the new patch set (#3). Change subject: Fix integration of kudu-hive.jar .. Fix integration of kudu-hive.jar IMPALA-8503 added downloading kudu-hive.jar and adding it to HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to start with Kudu's HMS plugin. There are two problems with this that are fixed by this patch: - Previously, we fully specify the expected jar filename based on the value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH but this is overly restrictive for users who may wish to override this value in impala-config-branch.sh to build their own branch with a different version of the kudu-hive.jar This patch relaxes this restriction by adding any jar containing the string kudu-hive in IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH - In bootstrap_toolchain, we don't download a package if its directory already exists. Since the 'kudu' and 'kudu-java' packages download to the same directory, this led to a race condition where 'kudu-java' might not be downloaded if 'kudu' had already been unpacked when it started. This patch fixes this by inspecting the contents of the Kudu package directory to look for specific files expected for each Kudu package. Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 --- M bin/bootstrap_toolchain.py M bin/impala-config.sh M testdata/bin/run-hive-server.sh 3 files changed, 19 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/13542/3 -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Thomas Marshall has posted comments on this change. (
http://gerrit.cloudera.org:8080/13542 )
Change subject: Fix integration of kudu-hive.jar
..
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py
File bin/bootstrap_toolchain.py:
http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py@433
PS2, Line 433: debug
> does this affect release builds?
Both the 'debug' and the 'release' directories will be present regardless of
the build type because we always just unpack and keep everything in the tarball.
http://gerrit.cloudera.org:8080/#/c/13542/2/testdata/bin/run-hive-server.sh
File testdata/bin/run-hive-server.sh:
http://gerrit.cloudera.org:8080/#/c/13542/2/testdata/bin/run-hive-server.sh@105
PS2, Line 105: ${IMPALA_KUDU_JAVA_HOME}/*
> I think this could just be
Done
--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:24:54 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/13409 ) Change subject: IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration .. Patch Set 5: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13409 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If7155e0b385b8ad81eda0a84277bc85171a88269 Gerrit-Change-Number: 13409 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 20:16:46 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Lars Volker has posted comments on this change. (
http://gerrit.cloudera.org:8080/13542 )
Change subject: Fix integration of kudu-hive.jar
..
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py
File bin/bootstrap_toolchain.py:
http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py@433
PS2, Line 433: debug
does this affect release builds?
http://gerrit.cloudera.org:8080/#/c/13542/2/testdata/bin/run-hive-server.sh
File testdata/bin/run-hive-server.sh:
http://gerrit.cloudera.org:8080/#/c/13542/2/testdata/bin/run-hive-server.sh@105
PS2, Line 105: ${IMPALA_KUDU_JAVA_HOME}/*
I think this could just be
for file in ${IMPALA_KUDU_JAVA_HOME}/*kudu-hive*jar; do
or you could use find | while.
--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:11:13 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Fix integration of kudu-hive.jar
Hello Hao Hao, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13542 to look at the new patch set (#2). Change subject: Fix integration of kudu-hive.jar .. Fix integration of kudu-hive.jar IMPALA-8503 added downloading kudu-hive.jar and adding it to HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to start with Kudu's HMS plugin. There are two problems with this that are fixed by this patch: - Previously, we fully specify the expected jar filename based on the value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH but this is overly restrictive for users who may wish to override this value in impala-config-branch.sh to build their own branch with a different version of the kudu-hive.jar This patch relaxes this restriction by adding any jar containing the string kudu-hive in IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH - In bootstrap_toolchain, we don't download a package if its directory already exists. Since the 'kudu' and 'kudu-java' packages download to the same directory, this led to a race condition where 'kudu-java' might not be downloaded if 'kudu' had already been unpacked when it started. This patch fixes this by inspecting the contents of the Kudu package directory to look for specific files expected for each Kudu package. Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 --- M bin/bootstrap_toolchain.py M bin/impala-config.sh M testdata/bin/run-hive-server.sh 3 files changed, 23 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/13542/2 -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] Improve adding kudu-hive.jar to HADOOP CLASSPATH
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 ) Change subject: Improve adding kudu-hive.jar to HADOOP_CLASSPATH .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3522/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 06 Jun 2019 19:08:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8629: Adjust new KuduStorageHandler package
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13541 ) Change subject: IMPALA-8629: Adjust new KuduStorageHandler package .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4423/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13541 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 Gerrit-Change-Number: 13541 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 18:37:23 + Gerrit-HasComments: No
[Impala-ASF-CR] Improve adding kudu-hive.jar to HADOOP CLASSPATH
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 ) Change subject: Improve adding kudu-hive.jar to HADOOP_CLASSPATH .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 06 Jun 2019 18:35:59 + Gerrit-HasComments: No
[Impala-ASF-CR] Improve adding kudu-hive.jar to HADOOP CLASSPATH
Thomas Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13542 Change subject: Improve adding kudu-hive.jar to HADOOP_CLASSPATH .. Improve adding kudu-hive.jar to HADOOP_CLASSPATH IMPALA-8503 added kudu-hive.jar to HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to start with Kudu's HMS plugin. It accomplished this by fully specify the expected jar filename based on the value of IMPALA_KUDU_JAVA_VERSION, but this is overly restrictive for users who may wish to override this value in impala-config-branch.sh to build their own branch with a different version of the kudu-hive.jar This patch relaxes this restriction by adding any jar containing the string kudu-hive in IMPALA_KUDU_HOME to HADOOP_CLASSPATH Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 --- M testdata/bin/run-hive-server.sh 1 file changed, 7 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/13542/1 -- To view, visit http://gerrit.cloudera.org:8080/13542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7 Gerrit-Change-Number: 13542 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall
[Impala-ASF-CR] IMPALA-8629: Adjust new KuduStorageHandler package
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13541 ) Change subject: IMPALA-8629: Adjust new KuduStorageHandler package .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3521/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13541 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 Gerrit-Change-Number: 13541 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 18:16:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1)
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/13522 ) Change subject: IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1) .. Patch Set 3: This fixes mtime plumbing. There will be another change to add some extra validation to HdfsParquetScanner::ProcessFooter() when we read a larger footer. This passed core tests, I'm running exhaustive. I'm thinking through appropriate tests to add. -- To view, visit http://gerrit.cloudera.org:8080/13522 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I48b7ed60d6ab9104b993237b4fe23de5dc058672 Gerrit-Change-Number: 13522 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 06 Jun 2019 18:16:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8629: Adjust new KuduStorageHandler package
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13541 ) Change subject: IMPALA-8629: Adjust new KuduStorageHandler package .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3520/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13541 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 Gerrit-Change-Number: 13541 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 18:16:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1)
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13522 to look at the new patch set (#3). Change subject: IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1) .. IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1) The file handle cache uses the mtime to distinguish different versions of a file separate. For example, if a file at mtime=1 is overwritten with a version at mtime=2, the old file handle from mtime=1 will not be used for the mtime=2 version. In some codepaths, for legacy reasons, the mtime would be unconditionally set to -1, and this eliminates the ability to distinguish between different versions of files. There is no need to set the mtime to -1. It seems to be a legacy bit of cruft. This removes the mtime=-1 behavior for HDFS scan ranges. It removes mtime from BufferOpts and plumbs the mtime through the scan range codepaths separately. Local non-HDFS files do not use the mtime, so those continue to use mtime=-1. Testing: - Passed core tests Change-Id: I48b7ed60d6ab9104b993237b4fe23de5dc058672 --- M be/src/exec/base-sequence-scanner.cc M be/src/exec/hdfs-orc-scanner.cc M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/hdfs-scanner.cc M be/src/exec/hdfs-text-scanner.cc M be/src/exec/parquet/hdfs-parquet-scanner.cc M be/src/exec/parquet/parquet-column-readers.cc M be/src/exec/parquet/parquet-page-index.cc M be/src/exec/scanner-context.cc M be/src/runtime/io/disk-io-mgr-stress.cc M be/src/runtime/io/disk-io-mgr-test.cc M be/src/runtime/io/handle-cache.inline.h M be/src/runtime/io/request-ranges.h M be/src/runtime/io/scan-range.cc M be/src/runtime/tmp-file-mgr.cc 16 files changed, 62 insertions(+), 59 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/13522/3 -- To view, visit http://gerrit.cloudera.org:8080/13522 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I48b7ed60d6ab9104b993237b4fe23de5dc058672 Gerrit-Change-Number: 13522 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8629: Adjust new KuduStorageHandler package
Hello Thomas Marshall, Hao Hao, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13541 to look at the new patch set (#2). Change subject: IMPALA-8629: Adjust new KuduStorageHandler package .. IMPALA-8629: Adjust new KuduStorageHandler package This patch changes the new KuduStorageHandler package from “org.apache.kudu.hive” to “org.apache.hadoop.hive.kudu”. This is being done to ensure the stand-in storage handler can be a real storage handler when a Hive integration is added in the future. The “org.apache.hadoop.hive” package is the standard package all Hive storage handlers lives under. Additionally this patch updates the stand-in InputFormat, OutputFormat, and SerDe entries for Kudu. This allows a future Hive integration to read HMS tables/entries created by Impala. Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 --- M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 6 files changed, 18 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/13541/2 -- To view, visit http://gerrit.cloudera.org:8080/13541 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 Gerrit-Change-Number: 13541 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8629: Adjust new KuduStorageHandler package
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/13541 )
Change subject: IMPALA-8629: Adjust new KuduStorageHandler package
..
Patch Set 1:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/13541/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java:
http://gerrit.cloudera.org:8080/#/c/13541/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java@206
PS1, Line 206: "tblproperties
('storage_handler'='org.apache.hadoop.hive.kudu.KuduStorageHandler')",
> line too long (93 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/13541/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java@447
PS1, Line 447:
"'storage_handler'='org.apache.hadoop.hive.kudu.KuduStorageHandler')",
kuduMasters));
> line too long (93 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/13541/1/fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
File fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java:
http://gerrit.cloudera.org:8080/#/c/13541/1/fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java@341
PS1, Line 341:
"'storage_handler'='org.apache.hadoop.hive.kudu.KuduStorageHandler')",
kuduMasters),
> line too long (92 > 90)
Done
--
To view, visit http://gerrit.cloudera.org:8080/13541
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473
Gerrit-Change-Number: 13541
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Thu, 06 Jun 2019 17:36:54 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8629: Adjust new KuduStorageHandler package
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13541 ) Change subject: IMPALA-8629: Adjust new KuduStorageHandler package .. Patch Set 1: This is just a quick find and replace patch. Given it's dependence on the Kudu side change (https://gerrit.cloudera.org/#/c/13540/), I don't expect all tests to pass yet. -- To view, visit http://gerrit.cloudera.org:8080/13541 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 Gerrit-Change-Number: 13541 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 17:32:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8629: Adjust new KuduStorageHandler package
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/13541 )
Change subject: IMPALA-8629: Adjust new KuduStorageHandler package
..
Patch Set 1:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/13541/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java:
http://gerrit.cloudera.org:8080/#/c/13541/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java@206
PS1, Line 206: "tblproperties
('storage_handler'='org.apache.hadoop.hive.kudu.KuduStorageHandler')",
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/13541/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java@447
PS1, Line 447:
"'storage_handler'='org.apache.hadoop.hive.kudu.KuduStorageHandler')",
kuduMasters));
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/13541/1/fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
File fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java:
http://gerrit.cloudera.org:8080/#/c/13541/1/fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java@341
PS1, Line 341:
"'storage_handler'='org.apache.hadoop.hive.kudu.KuduStorageHandler')",
kuduMasters),
line too long (92 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/13541
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473
Gerrit-Change-Number: 13541
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Thu, 06 Jun 2019 17:32:19 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8629: Adjust new KuduStorageHandler package
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13541 Change subject: IMPALA-8629: Adjust new KuduStorageHandler package .. IMPALA-8629: Adjust new KuduStorageHandler package This patch changes the new KuduStorageHandler package from “org.apache.kudu.hive” to “org.apache.hadoop.hive.kudu”. This is being done to ensure the stand-in storage handler can be a real storage handler when a Hive integration is added in the future. The “org.apache.hadoop.hive” package is the standard package all Hive storage handlers lives under. Additionally this patch updates the stand-in InputFormat, OutputFormat, and SerDe entries for Kudu. This allows a future Hive integration to read HMS tables/entries created by Impala. Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 --- M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 6 files changed, 15 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/13541/1 -- To view, visit http://gerrit.cloudera.org:8080/13541 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 Gerrit-Change-Number: 13541 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[Impala-ASF-CR] IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13409 ) Change subject: IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration .. Patch Set 5: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13409 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If7155e0b385b8ad81eda0a84277bc85171a88269 Gerrit-Change-Number: 13409 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 17:20:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13409 ) Change subject: IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4422/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13409 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If7155e0b385b8ad81eda0a84277bc85171a88269 Gerrit-Change-Number: 13409 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 16:33:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8436: Prohibit write/alter operations on materialized view
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13503 ) Change subject: IMPALA-8436: Prohibit write/alter operations on materialized view .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3519/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idcd619303e19b5a2551876a63d67569c76bd22f0 Gerrit-Change-Number: 13503 Gerrit-PatchSet: 4 Gerrit-Owner: Sudhanshu Arora Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sudhanshu Arora Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Jun 2019 16:26:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8436: Prohibit write/alter operations on materialized view
Sudhanshu Arora has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13503 Change subject: IMPALA-8436: Prohibit write/alter operations on materialized view .. IMPALA-8436: Prohibit write/alter operations on materialized view Instead of creating an in memory instance of View, we were creating instance of HdfsTable. Modified the code to create instance of View for materialized view. Testing Done: - Added tests in AnalyzerTest. Change-Id: Idcd619303e19b5a2551876a63d67569c76bd22f0 --- M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java M testdata/datasets/functional/functional_schema_template.sql 4 files changed, 55 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/13503/4 -- To view, visit http://gerrit.cloudera.org:8080/13503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Idcd619303e19b5a2551876a63d67569c76bd22f0 Gerrit-Change-Number: 13503 Gerrit-PatchSet: 4 Gerrit-Owner: Sudhanshu Arora Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Sudhanshu Arora Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3518/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 06 Jun 2019 14:56:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Fredy Wijaya has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 This patch bumps CDP_BUILD_NUMBER to 1153860 to pull RANGER-2437 fix that provides better error messages when granting/revoking with invalid user/group. Testing: - Added E2E test to test grant/revoke with invalid principal - Ran core tests - Ran the build with USE_CDP_HIVE=true Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 --- M bin/impala-config.sh M tests/authorization/test_ranger.py 2 files changed, 42 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/13525/2 -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13409 ) Change subject: IMPALA-8506: Support RENAME TABLE statement with Kudu/HMS integration .. Patch Set 5: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4421/ -- To view, visit http://gerrit.cloudera.org:8080/13409 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If7155e0b385b8ad81eda0a84277bc85171a88269 Gerrit-Change-Number: 13409 Gerrit-PatchSet: 5 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 06 Jun 2019 06:28:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 06 Jun 2019 06:26:14 + Gerrit-HasComments: No
