[Impala-ASF-CR] IMPALA-8460: Simplify cluster membership management
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13207 ) Change subject: IMPALA-8460: Simplify cluster membership management .. Patch Set 19: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4358/ -- To view, visit http://gerrit.cloudera.org:8080/13207 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3cf9a8bb060d0c6e9ec8868b7b21ce01f8740a3 Gerrit-Change-Number: 13207 Gerrit-PatchSet: 19 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 08:49:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8596: Add debugging output to assertion in test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13482 ) Change subject: IMPALA-8596: Add debugging output to assertion in test .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4361/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13482 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9b649600ad42fea85bfc3020c8d880b690d65dd Gerrit-Change-Number: 13482 Gerrit-PatchSet: 2 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 09:49:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8596: Add debugging output to assertion in test
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/13482 ) Change subject: IMPALA-8596: Add debugging output to assertion in test .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13482 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9b649600ad42fea85bfc3020c8d880b690d65dd Gerrit-Change-Number: 13482 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 09:48:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8596: Add debugging output to assertion in test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13482 ) Change subject: IMPALA-8596: Add debugging output to assertion in test .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13482 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9b649600ad42fea85bfc3020c8d880b690d65dd Gerrit-Change-Number: 13482 Gerrit-PatchSet: 2 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 09:49:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8369: Fix HMS integration tests for Hive 3
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13475 to look at the new patch set (#3). Change subject: IMPALA-8369: Fix HMS integration tests for Hive 3 .. IMPALA-8369: Fix HMS integration tests for Hive 3 Hive 3 changed the output format of the 'describe formatted ' command. Added a new function that can parse the new output. Hive 3 also got more permissive during scanning, e.g. it can scan a Parquet column of INT type while the HMS metadata for that column is STRING. It does automatic conversion during scanning. However, it doesn't do the opposite, i.e. when the Parquet column type is STRING and the HMS column type is INT it raises an exception. Extended the test with this case. Change-Id: I51c91c7fc706175295b78abaacf47a86156714ce (cherry picked from commit 5fb83e73e83d5a6496553d88daa9a2510e5b97bd) --- M tests/metadata/test_hms_integration.py 1 file changed, 52 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/13475/3 -- To view, visit http://gerrit.cloudera.org:8080/13475 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I51c91c7fc706175295b78abaacf47a86156714ce Gerrit-Change-Number: 13475 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8369: Fix HMS integration tests for Hive 3
Zoltan Borok-Nagy has posted comments on this change. (
http://gerrit.cloudera.org:8080/13475 )
Change subject: IMPALA-8369: Fix HMS integration tests for Hive 3
..
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/13475/2/tests/metadata/test_hms_integration.py
File tests/metadata/test_hms_integration.py:
http://gerrit.cloudera.org:8080/#/c/13475/2/tests/metadata/test_hms_integration.py@217
PS2, Line 217: output = self.run_stmt_in_hive('describe formatted %s %s' %
(table, column))
: if HIVE_MAJOR_VERSION == 2:
: return self.parse_
> nit: fits to 1 line
Done
http://gerrit.cloudera.org:8080/#/c/13475/2/tests/metadata/test_hms_integration.py@632
PS2, Line 632: if HIVE_MAJOR_VERSION == 2:
> Can you add a comment about Hive 3 becoming more tolerant?
Done. Added some other comments as well.
http://gerrit.cloudera.org:8080/#/c/13475/2/tests/metadata/test_hms_integration.py@646
PS2, Line 646: # Insert STRING value, it will create a Parquet file
where column 'y'
> nit: + 2 indentation
Fits single line.
http://gerrit.cloudera.org:8080/#/c/13475/2/tests/metadata/test_hms_integration.py@650
PS2, Line 650: self.run_stmt_in_hive('alter table %s change y y int' %
table_name)
> nit: + 2 indentation
Done
--
To view, visit http://gerrit.cloudera.org:8080/13475
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51c91c7fc706175295b78abaacf47a86156714ce
Gerrit-Change-Number: 13475
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Fri, 31 May 2019 10:14:32 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8369: Fix HMS integration tests for Hive 3
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/13475 ) Change subject: IMPALA-8369: Fix HMS integration tests for Hive 3 .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13475 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51c91c7fc706175295b78abaacf47a86156714ce Gerrit-Change-Number: 13475 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 10:20:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8369: Add HIVE MAJOR VERSION section to EE tests + some fixes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13472 ) Change subject: IMPALA-8369: Add HIVE_MAJOR_VERSION section to EE tests + some fixes .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4362/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13472 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c3cff05ed7080b655b6af64ea09c0691e7dd931 Gerrit-Change-Number: 13472 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 31 May 2019 10:26:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8369: Add HIVE MAJOR VERSION section to EE tests + some fixes
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/13472 ) Change subject: IMPALA-8369: Add HIVE_MAJOR_VERSION section to EE tests + some fixes .. Patch Set 2: Restarting tests as the failures seem to be unrelated issues on dockerized builds. -- To view, visit http://gerrit.cloudera.org:8080/13472 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c3cff05ed7080b655b6af64ea09c0691e7dd931 Gerrit-Change-Number: 13472 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 31 May 2019 10:27:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8369: Fix HMS integration tests for Hive 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13475 ) Change subject: IMPALA-8369: Fix HMS integration tests for Hive 3 .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13475 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51c91c7fc706175295b78abaacf47a86156714ce Gerrit-Change-Number: 13475 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 10:27:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8369: Fix HMS integration tests for Hive 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13475 ) Change subject: IMPALA-8369: Fix HMS integration tests for Hive 3 .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4363/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13475 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51c91c7fc706175295b78abaacf47a86156714ce Gerrit-Change-Number: 13475 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 10:27:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8369: Fix HMS integration tests for Hive 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13475 ) Change subject: IMPALA-8369: Fix HMS integration tests for Hive 3 .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3460/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13475 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51c91c7fc706175295b78abaacf47a86156714ce Gerrit-Change-Number: 13475 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 10:50:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8369: Add HIVE MAJOR VERSION section to EE tests + some fixes
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/13472 ) Change subject: IMPALA-8369: Add HIVE_MAJOR_VERSION section to EE tests + some fixes .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13472 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c3cff05ed7080b655b6af64ea09c0691e7dd931 Gerrit-Change-Number: 13472 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 11:00:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8595: Support TLSv1.2 with Python < 2.7.9 in shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13457 ) Change subject: IMPALA-8595: Support TLSv1.2 with Python < 2.7.9 in shell .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13457 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3fb6510f4b556bd8c6b1e86380379aba8be4b805 Gerrit-Change-Number: 13457 Gerrit-PatchSet: 7 Gerrit-Owner: Robbie Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 11:01:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7369: part 1: Implement TRUNC, DATE TRUNC, EXTRACT, DATE PART functions for DATE
Zoltan Borok-Nagy has posted comments on this change. (
http://gerrit.cloudera.org:8080/13363 )
Change subject: IMPALA-7369: part 1: Implement TRUNC, DATE_TRUNC, EXTRACT,
DATE_PART functions for DATE
..
Patch Set 6:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/benchmarks/date-benchmark.cc
File be/src/benchmarks/date-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/benchmarks/date-benchmark.cc@89
PS6, Line 89: vector date;
nit: missing '_' at the end here and for the following member variables.
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/expr-test.cc
File be/src/exprs/expr-test.cc:
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/expr-test.cc@7626
PS6, Line 7626: "extract(date '2006-05-12', 'minute')"
should be 'expr'.
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/expr-test.cc@7632
PS6, Line 7632: "extract(date '2006-05-12', 'minute')"
should be 'expr'.
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/expr-test.cc@7661
PS6, Line 7661: "date_part('minute', date '2006-05-12')"
should be 'expr'
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/expr-test.cc@7667
PS6, Line 7667: "date_part('minute', date '2006-05-12')"
should be expr.
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/udf-builtins.h
File be/src/exprs/udf-builtins.h:
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/udf-builtins.h@68
PS6, Line 68: ///WW : Same day of the week as the first day of the year
: ///W : Same day of the week as the first day of the month
I'm not sure I understand these. Maybe you could add a '*' at end and explain
at the bottom of the comment.
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/udf-builtins.h@125
PS6, Line 125: ///HOUR: The hour field (0–23).
: ///MINUTE: The minutes field (0–59).
: ///SECOND: The seconds field (0–59).
: ///MILLISECONDS: The milliseconds fraction in the seconds.
: ///MICROSECONDS: The microseconds fraction in the seconds.
very nit: maybe too much wording about stuff that it doesn't accept. Simple
enumeration would be enough IMHO, i.e. HOUR, MINUTE, ...
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/udf-builtins.cc
File be/src/exprs/udf-builtins.cc:
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/udf-builtins.cc@661
PS6, Line 661: typename Val,
: typename Value,
UdfType, InternalType?
--
To view, visit http://gerrit.cloudera.org:8080/13363
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I843358a45eb5faa2c134994600546fc1d0a797c8
Gerrit-Change-Number: 13363
Gerrit-PatchSet: 6
Gerrit-Owner: Attila Jeges
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Fri, 31 May 2019 12:08:29 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8491: Non-root user in container
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13451 ) Change subject: IMPALA-8491: Non-root user in container .. IMPALA-8491: Non-root user in container Set a default USER in the Dockerfile per best practices so that consumers of the container don't accidentally run as root. The default user is "impala" if the container is run in docker without specifying a user. Various frameworks, including kubernetes, will run the container with an arbitrary user and group ID set. This causes issues with some Hadoop libraries, which depend on the user having a name. This is generally not the case because inside the container usernames are resolved with the container's /etc/passwd. To work around this, the entrypoint script checks if the current user has a name and if not, assigns it one (either dummyuser or $HADOOP_USER_NAME). Remove the umask setting that was required to make logs modifiable by the host user - this is not needed for our tests since the host host and container users now match up. Also run apt-get clean in Dockerfile to reduce cruft in the image. Change-Id: I0bea9f44a8199851ed04fbef8caf4a2350ae2c0e Reviewed-on: http://gerrit.cloudera.org:8080/13451 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M bin/start-impala-cluster.py M docker/daemon_entrypoint.sh M docker/impala_base/Dockerfile 3 files changed, 21 insertions(+), 6 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I0bea9f44a8199851ed04fbef8caf4a2350ae2c0e Gerrit-Change-Number: 13451 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8491: Non-root user in container
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13451 ) Change subject: IMPALA-8491: Non-root user in container .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0bea9f44a8199851ed04fbef8caf4a2350ae2c0e Gerrit-Change-Number: 13451 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 12:19:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7369: part 1: Implement TRUNC, DATE TRUNC, EXTRACT, DATE PART functions for DATE
Attila Jeges has posted comments on this change. (
http://gerrit.cloudera.org:8080/13363 )
Change subject: IMPALA-7369: part 1: Implement TRUNC, DATE_TRUNC, EXTRACT,
DATE_PART functions for DATE
..
Patch Set 6:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/benchmarks/date-benchmark.cc
File be/src/benchmarks/date-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/benchmarks/date-benchmark.cc@89
PS6, Line 89: vector date;
> nit: missing '_' at the end here and for the following member variables.
Done
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/expr-test.cc
File be/src/exprs/expr-test.cc:
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/expr-test.cc@7626
PS6, Line 7626: "extract(date '2006-05-12', 'minute')"
> should be 'expr'.
Done. Not sure, how I messed these up :)
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/expr-test.cc@7632
PS6, Line 7632: "extract(date '2006-05-12', 'minute')"
> should be 'expr'.
Done
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/expr-test.cc@7661
PS6, Line 7661: "date_part('minute', date '2006-05-12')"
> should be 'expr'
Done
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/expr-test.cc@7667
PS6, Line 7667: "date_part('minute', date '2006-05-12')"
> should be expr.
Done
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/udf-builtins.h
File be/src/exprs/udf-builtins.h:
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/udf-builtins.h@68
PS6, Line 68: ///WW : Same day of the week as the first day of the year
: ///W : Same day of the week as the first day of the month
> I'm not sure I understand these. Maybe you could add a '*' at end and expla
Reworded the comment, hope it makes more sense now.
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/udf-builtins.h@125
PS6, Line 125: ///HOUR: The hour field (0–23).
: ///MINUTE: The minutes field (0–59).
: ///SECOND: The seconds field (0–59).
: ///MILLISECONDS: The milliseconds fraction in the seconds.
: ///MICROSECONDS: The microseconds fraction in the seconds.
> very nit: maybe too much wording about stuff that it doesn't accept. Simple
Done
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/udf-builtins.cc
File be/src/exprs/udf-builtins.cc:
http://gerrit.cloudera.org:8080/#/c/13363/6/be/src/exprs/udf-builtins.cc@661
PS6, Line 661: typename Val,
: typename Value,
> UdfType, InternalType?
Done
--
To view, visit http://gerrit.cloudera.org:8080/13363
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I843358a45eb5faa2c134994600546fc1d0a797c8
Gerrit-Change-Number: 13363
Gerrit-PatchSet: 6
Gerrit-Owner: Attila Jeges
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Fri, 31 May 2019 13:47:24 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7369: part 1: Implement TRUNC, DATE TRUNC, EXTRACT, DATE PART functions for DATE
Attila Jeges has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/13363 ) Change subject: IMPALA-7369: part 1: Implement TRUNC, DATE_TRUNC, EXTRACT, DATE_PART functions for DATE .. IMPALA-7369: part 1: Implement TRUNC, DATE_TRUNC, EXTRACT, DATE_PART functions for DATE These functions are somewhat similar in that each of them takes a DATE argument and a time unit to work with. They work identically to the corresponding TIMESTAMP functions. The only difference is that the DATE functions don't accept time-of-day units. TRUNC(DATE d, STRING unit) Truncates a DATE value to the specified time unit. The 'unit' argument is case insensitive. This argument string can be one of: S, , YEAR, SYEAR, YYY, YY, Y: Year. Q: Quarter. MONTH, MON, MM, RM: Month. DDD, DD, J: Day. DAY, DY, D: Starting day (Monday) of the week. WW: Truncates to the most recent date, no later than 'd', which is on the same day of the week as the first day of year. W: Truncates to the most recent date, no later than 'd', which is on the same day of the week as the first day of month. The impelementation mirrors Impala's TRUNC(TIMESTAMP ts, STRING unit) function. Hive and Oracle SQL have a similar function too. Reference: http://docs.oracle.com/cd/B19306_01/server.102/b14200/functions201.htm . DATE_TRUNC(STRING unit, DATE d) Truncates a DATE value to the specified precision. The 'unit' argument is case insensitive. This argument string can be one of: DAY, WEEK, MONTH, YEAR, DECADE, CENTURY, MILLENNIUM. The implementation mirrors Impala's DATE_TRUNC(STRING unit, TIMESTAMP ts) function. Vertica has a similar function too. Reference: https://my.vertica.com/docs/8.1.x/HTML/index.htm#Authoring/ SQLReferenceManual/Functions/Date-Time/DATE_TRUNC.htm . EXTRACT(DATE d, STRING unit), EXTRACT(unit FROM DATE d) Returns one of the numeric date fields from a DATE value. The 'unit' string can be one of YEAR, QUARTER, MONTH, DAY. This argument value is case-insensitive. The implementation mirrors that Impala's EXTRACT(TIMESTAMP ts, STRING unit). Hive and Oracle SQL have a similar function too. Reference: http://docs.oracle.com/cd/B19306_01/server.102/b14200/functions050.htm . DATE_PART(STRING unit, DATE date) Similar to EXTRACT(), with the argument order reversed. Supports the same date units as EXTRACT(). The implementation mirrors Impala's DATE_PART(STRING unit, TIMESTAMP ts) function. Change-Id: I843358a45eb5faa2c134994600546fc1d0a797c8 --- M be/src/benchmarks/CMakeLists.txt A be/src/benchmarks/date-benchmark.cc M be/src/exprs/expr-test.cc M be/src/exprs/udf-builtins-ir.cc M be/src/exprs/udf-builtins.cc M be/src/exprs/udf-builtins.h M be/src/runtime/date-test.cc M be/src/runtime/date-value.cc M be/src/runtime/date-value.h M common/function-registry/impala_functions.py M fe/src/main/java/org/apache/impala/analysis/ExtractFromExpr.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java 12 files changed, 1,375 insertions(+), 304 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/13363/7 -- To view, visit http://gerrit.cloudera.org:8080/13363 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I843358a45eb5faa2c134994600546fc1d0a797c8 Gerrit-Change-Number: 13363 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-7369: part 1: Implement TRUNC, DATE TRUNC, EXTRACT, DATE PART functions for DATE
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/13363 ) Change subject: IMPALA-7369: part 1: Implement TRUNC, DATE_TRUNC, EXTRACT, DATE_PART functions for DATE .. Patch Set 7: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13363 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I843358a45eb5faa2c134994600546fc1d0a797c8 Gerrit-Change-Number: 13363 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 13:57:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8600 : Reload partition does not work for transactional tables
Sudhanshu Arora has posted comments on this change. (
http://gerrit.cloudera.org:8080/13467 )
Change subject: IMPALA-8600 : Reload partition does not work for transactional
tables
..
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13467/1/tests/custom_cluster/test_event_processing.py
File tests/custom_cluster/test_event_processing.py:
http://gerrit.cloudera.org:8080/#/c/13467/1/tests/custom_cluster/test_event_processing.py@76
PS1, Line 76:self.run_stmt_in_hive("create table %s.%s (id int, val
int) TBLPROPERTIES "
Nit: We can create another string representing TBLPROPERTIES and append that
string to create statement. That will localize the checks for is_transactional
to one place.
--
To view, visit http://gerrit.cloudera.org:8080/13467
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8fffad27cd54f722efba1758ff44a45468f9514a
Gerrit-Change-Number: 13467
Gerrit-PatchSet: 1
Gerrit-Owner: Vihang Karajgaonkar
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Sudhanshu Arora
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Fri, 31 May 2019 14:17:22 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7369: part 1: Implement TRUNC, DATE TRUNC, EXTRACT, DATE PART functions for DATE
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/13363 ) Change subject: IMPALA-7369: part 1: Implement TRUNC, DATE_TRUNC, EXTRACT, DATE_PART functions for DATE .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13363 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I843358a45eb5faa2c134994600546fc1d0a797c8 Gerrit-Change-Number: 13363 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 14:27:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7369: part 1: Implement TRUNC, DATE TRUNC, EXTRACT, DATE PART functions for DATE
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13363 ) Change subject: IMPALA-7369: part 1: Implement TRUNC, DATE_TRUNC, EXTRACT, DATE_PART functions for DATE .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3461/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13363 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I843358a45eb5faa2c134994600546fc1d0a797c8 Gerrit-Change-Number: 13363 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 14:28:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7369: part 1: Implement TRUNC, DATE TRUNC, EXTRACT, DATE PART functions for DATE
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13363 ) Change subject: IMPALA-7369: part 1: Implement TRUNC, DATE_TRUNC, EXTRACT, DATE_PART functions for DATE .. Patch Set 8: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13363 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I843358a45eb5faa2c134994600546fc1d0a797c8 Gerrit-Change-Number: 13363 Gerrit-PatchSet: 8 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 14:32:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7369: part 1: Implement TRUNC, DATE TRUNC, EXTRACT, DATE PART functions for DATE
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13363 ) Change subject: IMPALA-7369: part 1: Implement TRUNC, DATE_TRUNC, EXTRACT, DATE_PART functions for DATE .. Patch Set 8: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4364/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13363 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I843358a45eb5faa2c134994600546fc1d0a797c8 Gerrit-Change-Number: 13363 Gerrit-PatchSet: 8 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 14:32:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8596: Add debugging output to assertion in test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13482 ) Change subject: IMPALA-8596: Add debugging output to assertion in test .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13482 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9b649600ad42fea85bfc3020c8d880b690d65dd Gerrit-Change-Number: 13482 Gerrit-PatchSet: 2 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 15:28:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8596: Add debugging output to assertion in test
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13482 ) Change subject: IMPALA-8596: Add debugging output to assertion in test .. IMPALA-8596: Add debugging output to assertion in test It looks like the assertion can sporadically fail, but inspecting the thrift profile did not show any suspicious fields. This change adds debug output to help identify the issue if we hit it again. Change-Id: If9b649600ad42fea85bfc3020c8d880b690d65dd Reviewed-on: http://gerrit.cloudera.org:8080/13482 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M tests/query_test/test_observability.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13482 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If9b649600ad42fea85bfc3020c8d880b690d65dd Gerrit-Change-Number: 13482 Gerrit-PatchSet: 3 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8369: Add HIVE MAJOR VERSION section to EE tests + some fixes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13472 ) Change subject: IMPALA-8369: Add HIVE_MAJOR_VERSION section to EE tests + some fixes .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13472 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c3cff05ed7080b655b6af64ea09c0691e7dd931 Gerrit-Change-Number: 13472 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 16:52:42 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-8425: Prototype to skip unneeded Kudu binaries for docker image
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13488 Change subject: WIP IMPALA-8425: Prototype to skip unneeded Kudu binaries for docker image .. WIP IMPALA-8425: Prototype to skip unneeded Kudu binaries for docker image Change-Id: I73efec0e8c1cceefd70c71f95fad8c1677e75e72 --- M docker/CMakeLists.txt 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/13488/1 -- To view, visit http://gerrit.cloudera.org:8080/13488 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I73efec0e8c1cceefd70c71f95fad8c1677e75e72 Gerrit-Change-Number: 13488 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
[Impala-ASF-CR] IMPALA-8595: Support TLSv1.2 with Python < 2.7.9 in shell
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13457 ) Change subject: IMPALA-8595: Support TLSv1.2 with Python < 2.7.9 in shell .. Patch Set 7: Code-Review+2 The change looks good. I ran exhaustive tests on CentOS7 and confirmed that the tls12 test ran and succeeded. I'm running some more tests on CentOS6 to confirm this didn't break older versions. You'll need to rebase as well. -- To view, visit http://gerrit.cloudera.org:8080/13457 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3fb6510f4b556bd8c6b1e86380379aba8be4b805 Gerrit-Change-Number: 13457 Gerrit-PatchSet: 7 Gerrit-Owner: Robbie Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 17:21:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8369: Fix HMS integration tests for Hive 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13475 ) Change subject: IMPALA-8369: Fix HMS integration tests for Hive 3 .. Patch Set 4: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4363/ -- To view, visit http://gerrit.cloudera.org:8080/13475 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51c91c7fc706175295b78abaacf47a86156714ce Gerrit-Change-Number: 13475 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 17:32:55 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-8425: Prototype to skip unneeded Kudu binaries for docker image
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13488 ) Change subject: WIP IMPALA-8425: Prototype to skip unneeded Kudu binaries for docker image .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3462/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13488 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I73efec0e8c1cceefd70c71f95fad8c1677e75e72 Gerrit-Change-Number: 13488 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 17:40:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8369: Fix HMS integration tests for Hive 3
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/13475 ) Change subject: IMPALA-8369: Fix HMS integration tests for Hive 3 .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/13475/4/tests/metadata/test_hms_integration.py File tests/metadata/test_hms_integration.py: http://gerrit.cloudera.org:8080/#/c/13475/4/tests/metadata/test_hms_integration.py@205 PS4, Line 205: parse_hive3_describe_formatted_output The new format of the the command in hive looks very weird. I wonder if its bug. -- To view, visit http://gerrit.cloudera.org:8080/13475 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51c91c7fc706175295b78abaacf47a86156714ce Gerrit-Change-Number: 13475 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 17:53:55 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504: Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13318 ) Change subject: IMPALA-8504: Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 13: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4367/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13318 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I465673d749221bd5f3772814b1c22c2673a53f5c Gerrit-Change-Number: 13318 Gerrit-PatchSet: 13 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 17:56:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8536: Add Scalable Pool Configuration to Admission Controller.
Andrew Sherman has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/13307 ) Change subject: IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. .. IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. Add 3 configuration parameters to Admission Controller that scale with the number of hosts in the resource pool. These parameters are specified to the Impalad through the -llama_site_path flag which points to a Llama XML configuration file. The new configuration parameters are: + Max Running Queries Multiple - this floating point number is multiplied by the current total number of executors at runtime to give the maximum number of concurrently running queries allowed in the pool. This calculation is rounded up to the nearest integer so the result will always be at least one as long as the parameter is non-zero. + Max Queued Queries Multiple - this floating point number is multiplied by the current total number of executors at runtime to give the maximum number of queries that can be queued in the pool. This calculation is rounded up to the nearest integer so the result will always be at least one as long as the parameter is non-zero. + Max Memory Multiple - this number of bytes is multiplied by the current total number of executors at runtime to give the maximum memory available across the cluster for the pool. If any of these parameters have zero value then they will be ignored. In this case the corresponding non-scalable parameters will be used, if they are set. The new parameters are exposed through the webui. At various points in the code Admission Controller looks at the Pool Config objects to find non-scalable parameters such as the max number of queries that can run in the pool. These access have been encapsulated in functions that return the scalable version of the configuration value if the new scalable parameters are being used. Diagnostic messages are enhanced to show the origin of the encapsulated parameters. TESTING All end-to-end tests are running clean with ASAN. The unit test admission-controller-test.cc has been expanded to test the newly added code. Added an end-to-end test that adds and removes Impalads from a minicluster. Change-Id: If47508728124076f3b9200c27cffc989f7a4f188 --- M be/src/scheduling/admission-controller-test.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M common/thrift/ImpalaInternalService.thrift M common/thrift/generate_metrics.py M common/thrift/metrics.json M fe/src/main/java/org/apache/impala/util/RequestPoolService.java M fe/src/test/java/org/apache/impala/util/TestRequestPoolService.java M fe/src/test/resources/fair-scheduler-test.xml M fe/src/test/resources/fair-scheduler-test2.xml M fe/src/test/resources/llama-site-test.xml M fe/src/test/resources/llama-site-test2.xml M testdata/workloads/functional-query/queries/QueryTest/admission-max-min-mem-limits.test M testdata/workloads/functional-query/queries/QueryTest/admission-reject-mem-estimate.test M testdata/workloads/functional-query/queries/QueryTest/admission-reject-min-reservation.test M tests/common/impala_cluster.py M tests/custom_cluster/test_admission_controller.py M tests/webserver/test_web_pages.py M www/admission_controller.tmpl 19 files changed, 1,187 insertions(+), 270 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/13307/4 -- To view, visit http://gerrit.cloudera.org:8080/13307 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If47508728124076f3b9200c27cffc989f7a4f188 Gerrit-Change-Number: 13307 Gerrit-PatchSet: 4 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8536: Add Scalable Pool Configuration to Admission Controller.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/13307 ) Change subject: IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. .. Patch Set 4: (22 comments) Thanks Tim and Bikram for the thorough reviews - see new patch set 4 for fixes. http://gerrit.cloudera.org:8080/#/c/13307/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13307/1//COMMIT_MSG@21 PS1, Line 21: + Max Queued Queries Multiple - this floating point number is multiplied > I think this is ok for now. I thought about it some more and having one opt Done http://gerrit.cloudera.org:8080/#/c/13307/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13307/3//COMMIT_MSG@14 PS3, Line 14: The new configuration parameters are: > mention that a value of zero means it wont be used and will default to the Thanks http://gerrit.cloudera.org:8080/#/c/13307/3//COMMIT_MSG@15 PS3, Line 15: Max Running > nit: use "Max Requests" to be consistent OK, this is complicated, like all naming things. Yes it is "max_requests" in thrift. But in CM (which is what I am really taking about in the commit msg) it is "Max Running Queries". And in the webui it is "Max concurrent queries" So there is no totally consistent value. I think "queries" is clearer than "requests" but we can talk more! http://gerrit.cloudera.org:8080/#/c/13307/3//COMMIT_MSG@26 PS3, Line 26: this number of bytes is multiplied by the : current total number of executors at runtime to give the maximum : memory available acro > can you elaborate on what this number is like you did for the ones above? A Done http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller-test.cc File be/src/scheduling/admission-controller-test.cc: http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller-test.cc@503 PS3, Line 503: stats.local_stats_.num_queued = 10; > ScopedFlagSetter? Done http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller-test.cc@545 PS3, Line 545: stats.agg_num_queued_ = host_count; > Would it make sense to test that the pool stats were actually modified in t Done http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller-test.cc@555 PS3, Line 555: } > Would it make sense to test that the pool stats were actually modified in t Done http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.h File be/src/scheduling/admission-controller.h: http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.h@582 PS3, Line 582: bool CanAdmitRequest(const QuerySchedule& schedule, const TPoolConfig& pool_cfg, > I'd prefer to use int64_t for consistency (reasonable people can differ abo Thanks http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.h@603 PS3, Line 603: _size_snapshot, > nit: maybe use something like latest_cluster_size. the snapshot in the name The ClusterMembershipMgr that will come in IMPALA-8460 has a method GetSnapshot() to fetch the cluster membership. The cluster size used by AC will eventually come from there which is why I chose it. What was that made you expect a struct? If this is a common Impala term then I should change it. http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.h@661 PS3, Line 661: /// If it can be determined that no queries can currently be run, then zero > It would help to clarify if max_to_dequeue can be 0 if Status is non-OK. If Yes, that is much clearer thanks. http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.cc File be/src/scheduling/admission-controller.cc: http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.cc@1000 PS3, Line 1000: int64_t max_to_dequeue = : GetMaxToDequeue(queue, stats, pool_config, cluster_size_snapshot); : VLOG_RPC << "Dequeue thread will try to admit " << max_to_dequeue << " requests" :<< ", pool=" << pool_name > nit: if this functionality is completely moved over to a method, maybe we c Done http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.cc@1007 PS3, Line 1007: > I agree, would prefer returning max_to_dequeue itself and handling the <= 0 Done http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.cc@1007 PS3, Line 1007: > I think the old behaviour had the advantage of logging something if nothing Yes, thanks, I moved this logic below the logging. http://gerrit.cloudera.org:8080/#/c/13307/3/be/src/scheduling/admission-controller.cc@1070 PS3, Line 1070: return min(stats->local_stats().num_queued, > Shouldn't this use the ceil() logic from GetMaxRequestsForPool()? Or at lea Yes, should be a double, this was a bug
[Impala-ASF-CR] IMPALA-8460: Simplify cluster membership management
Hello Michael Ho, Thomas Marshall, Tim Armstrong, Bikramjeet Vig, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13207 to look at the new patch set (#20). Change subject: IMPALA-8460: Simplify cluster membership management .. IMPALA-8460: Simplify cluster membership management This change adds a class to track cluster membership called ClusterMembershipMgr. It replaces the logic that was partially duplicated between the ImpalaServer and the Coordinator and makes sure that the local backend descriptor is consistent (IMPALA-8469). The ClusterMembershipMgr maintains a view of the cluster membership and incorporates incoming updates from the statestore. It also registers the local backend with the statestore after startup. Clients can obtain a consistent, immutable snapshot of the current cluster membership from the ClusterMembershipMgr. Additionally, callbacks can be registered to receive notifications of cluster membership changes. The ImpalaServer and Frontend use this mechanism. This change also generalizes the fix for IMPALA-7665: updates from the statestore to the cluster membership topic are only made visible to the rest of the local server after a post-recovery grace period has elapsed. As part of this the flag 'failed_backends_query_cancellation_grace_period_ms' is replaced with 'statestore_subscriber_recovery_grace_period_ms'. To tell the initial startup from post-recovery, a new metric 'statestore-subscriber.num-connection-failures' is exposed by the daemon, which tracks the total number of connection failures to the statestore over the lifetime process lifetime. This change also unifies the naming of executor-related classes, in particular it renames "BackendConfig" to "ExecutorGroup". In anticipation of a subsequent change (IMPALA-8484), it adds maps to store multiple executor groups. This change also disables the generation of default operators from the thrift files and instead adds explicit implementations for the ones that we rely on. This forces us to explicitly specify comparators when manipulating containers of thrift structs and will help prevent accidental bugs. Testing: This change adds a backend unit test for the new cluster membership manager. The observable behavior of Impala does not change, and the existing scheduler unit test and end to end tests should make sure of that. Change-Id: Ib3cf9a8bb060d0c6e9ec8868b7b21ce01f8740a3 --- M be/src/benchmarks/scheduler-benchmark.cc M be/src/common/logging.h M be/src/common/thread-debug-info-test.cc M be/src/experiments/hash-ring-util.cc M be/src/gutil/strings/split.cc M be/src/gutil/strings/split.h M be/src/rpc/thrift-util-test.cc M be/src/rpc/thrift-util.cc M be/src/runtime/exec-env.cc M be/src/runtime/exec-env.h M be/src/runtime/krpc-data-stream-mgr.cc M be/src/runtime/query-state.h M be/src/scheduling/CMakeLists.txt D be/src/scheduling/backend-config-test.cc D be/src/scheduling/backend-config.cc D be/src/scheduling/backend-config.h A be/src/scheduling/cluster-membership-mgr-test.cc A be/src/scheduling/cluster-membership-mgr.cc A be/src/scheduling/cluster-membership-mgr.h A be/src/scheduling/cluster-membership-test-util.cc A be/src/scheduling/cluster-membership-test-util.h A be/src/scheduling/executor-group-test.cc A be/src/scheduling/executor-group.cc A be/src/scheduling/executor-group.h M be/src/scheduling/query-schedule.cc M be/src/scheduling/query-schedule.h M be/src/scheduling/scheduler-test-util.cc M be/src/scheduling/scheduler-test-util.h M be/src/scheduling/scheduler-test.cc M be/src/scheduling/scheduler.cc M be/src/scheduling/scheduler.h M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M be/src/service/impala-server.h M be/src/statestore/statestore-subscriber.cc M be/src/statestore/statestore-subscriber.h M be/src/statestore/statestore.cc M be/src/testutil/in-process-servers.cc M be/src/util/container-util.h M be/src/util/network-util.cc M be/src/util/network-util.h M be/src/util/runtime-profile-test.cc M be/src/util/uid-util-test.cc M common/thrift/CMakeLists.txt M common/thrift/Frontend.thrift M common/thrift/StatestoreService.thrift M common/thrift/metrics.json M fe/src/main/java/org/apache/impala/service/Frontend.java M tests/custom_cluster/test_coordinators.py M tests/custom_cluster/test_restart_services.py 50 files changed, 1,994 insertions(+), 1,027 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/13207/20 -- To view, visit http://gerrit.cloudera.org:8080/13207 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib3cf9a8bb060d0c6e9ec8868b7b21ce01f8740a3 Gerrit-Change-Number: 13207 Gerrit-PatchSet: 20 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lar
[Impala-ASF-CR] IMPALA-8460: Simplify cluster membership management
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/13207 ) Change subject: IMPALA-8460: Simplify cluster membership management .. Patch Set 19: > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4358/ The build hit a race condition where for a short time during statestore re-connection the connection had already been established, but the last_recovery_duration_metric_ had not yet been updated. To fix this, I added a new metric (see updated commit message) that keeps track of the actual number of reconnects. -- To view, visit http://gerrit.cloudera.org:8080/13207 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3cf9a8bb060d0c6e9ec8868b7b21ce01f8740a3 Gerrit-Change-Number: 13207 Gerrit-PatchSet: 19 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 18:05:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8460: Simplify cluster membership management
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/13207 ) Change subject: IMPALA-8460: Simplify cluster membership management .. Patch Set 20: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13207 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3cf9a8bb060d0c6e9ec8868b7b21ce01f8740a3 Gerrit-Change-Number: 13207 Gerrit-PatchSet: 20 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 18:17:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8502: Bump CDH BUILD NUMBER and Kudu version
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13481 ) Change subject: IMPALA-8502: Bump CDH_BUILD_NUMBER and Kudu version .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4368/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13481 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c9c7d264f08454d9da8ad2d06ead643214ee23e Gerrit-Change-Number: 13481 Gerrit-PatchSet: 3 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 18:19:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13319 ) Change subject: IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4369/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4bd1488ced51840ec986d29ed371e26168abcc76 Gerrit-Change-Number: 13319 Gerrit-PatchSet: 2 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 18:19:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8460: Simplify cluster membership management
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13207 ) Change subject: IMPALA-8460: Simplify cluster membership management .. Patch Set 21: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13207 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3cf9a8bb060d0c6e9ec8868b7b21ce01f8740a3 Gerrit-Change-Number: 13207 Gerrit-PatchSet: 21 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 18:24:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8460: Simplify cluster membership management
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13207 ) Change subject: IMPALA-8460: Simplify cluster membership management .. Patch Set 21: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4370/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13207 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3cf9a8bb060d0c6e9ec8868b7b21ce01f8740a3 Gerrit-Change-Number: 13207 Gerrit-PatchSet: 21 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 18:24:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8502: Bump CDH BUILD NUMBER and Kudu version
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13481 ) Change subject: IMPALA-8502: Bump CDH_BUILD_NUMBER and Kudu version .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13481 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c9c7d264f08454d9da8ad2d06ead643214ee23e Gerrit-Change-Number: 13481 Gerrit-PatchSet: 3 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 18:39:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8536: Add Scalable Pool Configuration to Admission Controller.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13307 ) Change subject: IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3463/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13307 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If47508728124076f3b9200c27cffc989f7a4f188 Gerrit-Change-Number: 13307 Gerrit-PatchSet: 4 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 18:41:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8460: Simplify cluster membership management
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13207 ) Change subject: IMPALA-8460: Simplify cluster membership management .. Patch Set 20: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3464/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13207 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3cf9a8bb060d0c6e9ec8868b7b21ce01f8740a3 Gerrit-Change-Number: 13207 Gerrit-PatchSet: 20 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 18:43:24 + Gerrit-HasComments: No
[native-toolchain-CR] Bump Kudu version to 84086fe
Thomas Marshall has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/13479 ) Change subject: Bump Kudu version to 84086fe .. Bump Kudu version to 84086fe This pulls in changes needed for the Impala side of the Kudu/HMS integration work. Change-Id: Ie63a8baf1564f63d6ec9c0b953e2fa4119f51190 --- M buildall.sh 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/79/13479/2 -- To view, visit http://gerrit.cloudera.org:8080/13479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie63a8baf1564f63d6ec9c0b953e2fa4119f51190 Gerrit-Change-Number: 13479 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall
[native-toolchain-CR] Bump Kudu version to 84086fe
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13479 ) Change subject: Bump Kudu version to 84086fe .. Patch Set 2: see https://gerrit.cloudera.org/#/c/13481/ -- To view, visit http://gerrit.cloudera.org:8080/13479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie63a8baf1564f63d6ec9c0b953e2fa4119f51190 Gerrit-Change-Number: 13479 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 18:49:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8553,IMPALA-8552: fix checks for remote cluster
Hello David Knupp, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13386 to look at the new patch set (#4). Change subject: IMPALA-8553,IMPALA-8552: fix checks for remote cluster .. IMPALA-8553,IMPALA-8552: fix checks for remote cluster Apparently IMPALA_REMOTE_URL is not generally used for remote cluster tests: only --testing_remote_cluster is reliably set. Fix the is_remote_cluster() implementation to take into account REMOTE_DATA_LOAD and --testing_remote_cluster in addition to IMPALA_REMOTE_URL. Consistently use is_remote_cluster() in other tests instead of checking the pytest flag directly. There were a few lifecycle headaches with how ImpalaTestClusterProperties is used: * common.environ is imported from conftest, which means that the top-level code in the file runs *before* pytest command-line arguments have been registered and parsed. * ImpalaTestClusterProperties is used by various code, like build_flavor_timeout(), which runs before pytest command-line arguments have been parsed. * ImpalaTestClusterProperties is called from non-pytest scripts like start-impala-cluster.py, so the command-line arguments are not available. I dealt with the above challenges by making a couple of changes to do the detection later: * Lazily initializing a singleton ImpalaTestClusterProperties. This was not strictly necessary but makes the whole problem less sensitive to import order and module dependencies. * Removing the caching of the local/remote build calculation. ImpalaTestClusterProperties is instantiated outside of python tests, but is_remote_cluster() is only called from python tests, so if we check flags in is_remote_cluster() we'll get the right results reliably. As a workaround to unblock remote tests, also assume catalog_v1 if accessing the web UI fails. Testing: Ran core tests against a regular minicluster. Ran tests against a remote cluster Change-Id: Ifa6b2a1391f53121d3d7c00c5cf0a57590899ce4 --- M tests/common/environ.py M tests/common/skip.py M tests/custom_cluster/test_admission_controller.py M tests/custom_cluster/test_automatic_invalidation.py M tests/hs2/test_hs2.py M tests/metadata/test_compute_stats.py M tests/metadata/test_hms_integration.py M tests/metadata/test_metadata_query_statements.py M tests/metadata/test_refresh_partition.py M tests/query_test/test_kudu.py M tests/query_test/test_mt_dop.py M tests/query_test/test_spilling.py M tests/shell/test_shell_commandline.py M tests/shell/util.py M tests/webserver/test_web_pages.py 15 files changed, 93 insertions(+), 60 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/13386/4 -- To view, visit http://gerrit.cloudera.org:8080/13386 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifa6b2a1391f53121d3d7c00c5cf0a57590899ce4 Gerrit-Change-Number: 13386 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8553,IMPALA-8552: fix checks for remote cluster
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13386 ) Change subject: IMPALA-8553,IMPALA-8552: fix checks for remote cluster .. Patch Set 4: I ran some tests against remote clusters and ran into a bunch of issues with the pytest command line flags. I had to rework the lifecycle of ImpalaTestClusterProperties a bunch. Good news is that it now works properly. -- To view, visit http://gerrit.cloudera.org:8080/13386 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifa6b2a1391f53121d3d7c00c5cf0a57590899ce4 Gerrit-Change-Number: 13386 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 19:27:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8553,IMPALA-8552: fix checks for remote cluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13386 ) Change subject: IMPALA-8553,IMPALA-8552: fix checks for remote cluster .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4371/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13386 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifa6b2a1391f53121d3d7c00c5cf0a57590899ce4 Gerrit-Change-Number: 13386 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 19:27:22 + Gerrit-HasComments: No
[native-toolchain-CR] Bump Kudu version to 84086fe
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13479 ) Change subject: Bump Kudu version to 84086fe .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie63a8baf1564f63d6ec9c0b953e2fa4119f51190 Gerrit-Change-Number: 13479 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 19:28:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8425: part 1: reduce size of binaries in container
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13487 Change subject: IMPALA-8425: part 1: reduce size of binaries in container .. IMPALA-8425: part 1: reduce size of binaries in container * Symlink impalad/catalog/statestored inside container. This doesn't seem to really save any space - there's some kind of deduplication going on. * Don't include libfesupport.so, which shouldn't be needed. * strip debug symbols from the binary. * Only include the libkuduclient.so libraries for Kudu This shaves ~1.1GB from the image size- 250MB as a result of the impalad binary changes and the remainder from the Kudu changs. Change-Id: I95ff479bedd3b93e6569e72f03f42acd9dba8b14 --- M docker/daemon_entrypoint.sh M docker/impala_base/Dockerfile M docker/setup_build_context.py 3 files changed, 32 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/13487/3 -- To view, visit http://gerrit.cloudera.org:8080/13487 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I95ff479bedd3b93e6569e72f03f42acd9dba8b14 Gerrit-Change-Number: 13487 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[native-toolchain-CR] Bump Kudu version to 84086fe
Thomas Marshall has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13479 ) Change subject: Bump Kudu version to 84086fe .. Bump Kudu version to 84086fe This pulls in changes needed for the Impala side of the Kudu/HMS integration work. Change-Id: Ie63a8baf1564f63d6ec9c0b953e2fa4119f51190 Reviewed-on: http://gerrit.cloudera.org:8080/13479 Reviewed-by: Tim Armstrong Tested-by: Thomas Marshall --- M buildall.sh 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Tim Armstrong: Looks good to me, approved Thomas Marshall: Verified -- To view, visit http://gerrit.cloudera.org:8080/13479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie63a8baf1564f63d6ec9c0b953e2fa4119f51190 Gerrit-Change-Number: 13479 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong
[native-toolchain-CR] Bump Kudu version to 84086fe
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13479 ) Change subject: Bump Kudu version to 84086fe .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie63a8baf1564f63d6ec9c0b953e2fa4119f51190 Gerrit-Change-Number: 13479 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 19:41:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8425: part 1: reduce size of binaries in container
Hello Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13487 to look at the new patch set (#4). Change subject: IMPALA-8425: part 1: reduce size of binaries in container .. IMPALA-8425: part 1: reduce size of binaries in container * Symlink impalad/catalog/statestored inside container. This doesn't seem to really save any space - there's some kind of deduplication going on. * Don't include libfesupport.so, which shouldn't be needed. * strip debug symbols from the binary. * Only include the libkuduclient.so libraries for Kudu This shaves ~1.1GB from the image size- 250MB as a result of the impalad binary changes and the remainder from the Kudu changs. Change-Id: I95ff479bedd3b93e6569e72f03f42acd9dba8b14 --- M docker/daemon_entrypoint.sh M docker/impala_base/Dockerfile M docker/setup_build_context.py 3 files changed, 34 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/13487/4 -- To view, visit http://gerrit.cloudera.org:8080/13487 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I95ff479bedd3b93e6569e72f03f42acd9dba8b14 Gerrit-Change-Number: 13487 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7369: part 1: Implement TRUNC, DATE TRUNC, EXTRACT, DATE PART functions for DATE
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13363 ) Change subject: IMPALA-7369: part 1: Implement TRUNC, DATE_TRUNC, EXTRACT, DATE_PART functions for DATE .. Patch Set 8: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4364/ -- To view, visit http://gerrit.cloudera.org:8080/13363 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I843358a45eb5faa2c134994600546fc1d0a797c8 Gerrit-Change-Number: 13363 Gerrit-PatchSet: 8 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 31 May 2019 20:00:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8553,IMPALA-8552: fix checks for remote cluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13386 ) Change subject: IMPALA-8553,IMPALA-8552: fix checks for remote cluster .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3465/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13386 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifa6b2a1391f53121d3d7c00c5cf0a57590899ce4 Gerrit-Change-Number: 13386 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 20:06:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8425: part 1: reduce size of binaries in container
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13487 ) Change subject: IMPALA-8425: part 1: reduce size of binaries in container .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3466/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13487 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I95ff479bedd3b93e6569e72f03f42acd9dba8b14 Gerrit-Change-Number: 13487 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 20:25:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8509. Lazily evaluate LOAD sections during data load
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13252 ) Change subject: IMPALA-8509. Lazily evaluate LOAD sections during data load .. Patch Set 4: Looks like the jenkins job got gc'ed - any idea what happened? -- To view, visit http://gerrit.cloudera.org:8080/13252 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifc64bb5cac4fda675607672329c04c5caf810d99 Gerrit-Change-Number: 13252 Gerrit-PatchSet: 4 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 31 May 2019 20:29:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13319 ) Change subject: IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/13319/2/bin/bootstrap_toolchain.py File bin/bootstrap_toolchain.py: http://gerrit.cloudera.org:8080/#/c/13319/2/bin/bootstrap_toolchain.py@424 PS2, Line 424: addtional_file = None So as written this isn't going to work for non-cdh supported platforms (eg. ubuntu14). That's because 'kudu' won't be added to the list of cdh components on such platforms (see line 573). On those platforms, we download the platform-specific Kudu bits from the toolchain instead (see line 542), but we'll still want to download these jars from cdh_components because the toolchain doesn't produce the jars. How about, we have two kudu components - call the Java one just 'kudu' and always add it to cdh_components on line 563. Then have another one, call it 'kudu-c++' or similar, which gets adds to cdh_components on line 573, and then modify the 'if' on line 425 to check for that component name. -- To view, visit http://gerrit.cloudera.org:8080/13319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4bd1488ced51840ec986d29ed371e26168abcc76 Gerrit-Change-Number: 13319 Gerrit-PatchSet: 2 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 20:29:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Add decimal parquet to TableFormatInfo.KNOWN FILE FORMATS
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13398 ) Change subject: Add decimal_parquet to TableFormatInfo.KNOWN_FILE_FORMATS .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4372/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13398 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fc9c0ec82318e740a06493c1b4e4719c7162db Gerrit-Change-Number: 13398 Gerrit-PatchSet: 3 Gerrit-Owner: David Knupp Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 31 May 2019 20:29:47 + Gerrit-HasComments: No
[Impala-ASF-CR] Add decimal parquet to TableFormatInfo.KNOWN FILE FORMATS
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13398 ) Change subject: Add decimal_parquet to TableFormatInfo.KNOWN_FILE_FORMATS .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13398 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fc9c0ec82318e740a06493c1b4e4719c7162db Gerrit-Change-Number: 13398 Gerrit-PatchSet: 3 Gerrit-Owner: David Knupp Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 31 May 2019 20:29:46 + Gerrit-HasComments: No
[Impala-ASF-CR] Add decimal parquet to TableFormatInfo.KNOWN FILE FORMATS
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13398 ) Change subject: Add decimal_parquet to TableFormatInfo.KNOWN_FILE_FORMATS .. Patch Set 3: I kicked off the jenkins job :) Seemed safe to do -- To view, visit http://gerrit.cloudera.org:8080/13398 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fc9c0ec82318e740a06493c1b4e4719c7162db Gerrit-Change-Number: 13398 Gerrit-PatchSet: 3 Gerrit-Owner: David Knupp Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 31 May 2019 20:30:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8425: part 1: reduce size of binaries in container
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13487 ) Change subject: IMPALA-8425: part 1: reduce size of binaries in container .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3467/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13487 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I95ff479bedd3b93e6569e72f03f42acd9dba8b14 Gerrit-Change-Number: 13487 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 20:35:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8564: Add table/view create time in the lineage graph
Fredy Wijaya has uploaded a new patch set (#17). (
http://gerrit.cloudera.org:8080/13399 )
Change subject: IMPALA-8564: Add table/view create time in the lineage graph
..
IMPALA-8564: Add table/view create time in the lineage graph
This patch adds table/view create time in the lineage graph. This is
needed for Impala/Atlas integration. See ATLAS-3080.
Below is an example of the updated lineage graph.
{
"queryText":"create table lineage_test_tbl as select int_col, tinyint_col
from functional.alltypes",
"queryId":"0:0",
"hash":"407f23b24758ffcb2ac445b9703f5c44",
"user":"dummy_user",
"timestamp":1547867921,
"edges":[
{
"sources":[
1
],
"targets":[
0
],
"edgeType":"PROJECTION"
},
{
"sources":[
3
],
"targets":[
2
],
"edgeType":"PROJECTION"
}
],
"vertices":[
{
"id":0,
"vertexType":"COLUMN",
"vertexId":"int_col",
"metadata":{
"tableName":"default.lineage_test_tbl",
"tableCreateTime":1559151337
}
},
{
"id":1,
"vertexType":"COLUMN",
"vertexId":"functional.alltypes.int_col",
"metadata":{
"tableName":"functional.alltypes",
"tableCreateTime":1559151317
}
},
{
"id":2,
"vertexType":"COLUMN",
"vertexId":"tinyint_col",
"metadata":{
"tableName":"default.lineage_test_tbl",
"tableCreateTime":1559151337
}
},
{
"id":3,
"vertexType":"COLUMN",
"vertexId":"functional.alltypes.tinyint_col",
"metadata":{
"tableName":"functional.alltypes",
"tableCreateTime":1559151317
}
}
]
}
Testing:
- Updated lineage tests in PlannerTest
- Updated test_lineage.py
- Ran all FE tests
Change-Id: If4f578d7b299a76c30323b10a883ba32f8713d82
---
M be/src/service/client-request-state.h
M be/src/service/frontend.cc
M be/src/service/impala-server.cc
M be/src/util/lineage-util.h
M common/thrift/CatalogService.thrift
M common/thrift/LineageGraph.thrift
M fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java
M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java
M testdata/workloads/functional-planner/queries/PlannerTest/lineage.test
M tests/custom_cluster/test_lineage.py
15 files changed, 2,136 insertions(+), 623 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/13399/17
--
To view, visit http://gerrit.cloudera.org:8080/13399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If4f578d7b299a76c30323b10a883ba32f8713d82
Gerrit-Change-Number: 13399
Gerrit-PatchSet: 17
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Anonymous Coward (498)
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8564: Add table/view create time in the lineage graph
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/13399 ) Change subject: IMPALA-8564: Add table/view create time in the lineage graph .. Patch Set 17: Code-Review+2 > Patch Set 15: Verified-1 > > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4356/ Fixed a subtle bug in the lineage unit test where Vertex instances in the set of vertices and the Vertex instances in the set of Edges are different instances. In the previous patch, I got lucky with my local testing due to the timestamps in lineage.test were exactly the same as the ones created in the metastore. I blew up the metastore and re-created the tables and was able to reproduce the issue and fixed it. -- To view, visit http://gerrit.cloudera.org:8080/13399 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If4f578d7b299a76c30323b10a883ba32f8713d82 Gerrit-Change-Number: 13399 Gerrit-PatchSet: 17 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 31 May 2019 21:20:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8564: Add table/view create time in the lineage graph
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13399 ) Change subject: IMPALA-8564: Add table/view create time in the lineage graph .. Patch Set 17: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4373/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13399 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If4f578d7b299a76c30323b10a883ba32f8713d82 Gerrit-Change-Number: 13399 Gerrit-PatchSet: 17 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 31 May 2019 21:21:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8564: Add table/view create time in the lineage graph
Fredy Wijaya has posted comments on this change. (
http://gerrit.cloudera.org:8080/13399 )
Change subject: IMPALA-8564: Add table/view create time in the lineage graph
..
Patch Set 17:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13399/17/fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java
File fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java:
http://gerrit.cloudera.org:8080/#/c/13399/17/fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java@338
PS17, Line 338: if (v != null) {
> can you add comments explaining under what circumstances there would be mis
This will never happen, just being defensive about it. I can remove it.
http://gerrit.cloudera.org:8080/#/c/13399/17/fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java@726
PS17, Line 726: // We want to use the same Vertex references when
building the edge.
> can you explain why?
So in the unit test, we're modifying the timestamp in the vertices due to
dynamic nature of the timestamp. However I didn't modify the vertex instances
in the MultiEdge. The normal and easier way is to simply use the same Vertex
instances in both Set and MultiEdge.
--
To view, visit http://gerrit.cloudera.org:8080/13399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If4f578d7b299a76c30323b10a883ba32f8713d82
Gerrit-Change-Number: 13399
Gerrit-PatchSet: 17
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Anonymous Coward (498)
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Fri, 31 May 2019 21:24:11 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8536: Add Scalable Pool Configuration to Admission Controller.
Andrew Sherman has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/13307 ) Change subject: IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. .. IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. Add 3 configuration parameters to Admission Controller that scale with the number of hosts in the resource pool. These parameters are specified to the Impalad through the -llama_site_path flag which points to a Llama XML configuration file. The new configuration parameters are: + Max Running Queries Multiple - this floating point number is multiplied by the current total number of executors at runtime to give the maximum number of concurrently running queries allowed in the pool. This calculation is rounded up to the nearest integer so the result will always be at least one as long as the parameter is non-zero. + Max Queued Queries Multiple - this floating point number is multiplied by the current total number of executors at runtime to give the maximum number of queries that can be queued in the pool. This calculation is rounded up to the nearest integer so the result will always be at least one as long as the parameter is non-zero. + Max Memory Multiple - this number of bytes is multiplied by the current total number of executors at runtime to give the maximum memory available across the cluster for the pool. If any of these parameters have zero value then they will be ignored. In this case the corresponding non-scalable parameters will be used, if they are set. The new parameters are exposed through the webui. At various points in the code Admission Controller looks at the Pool Config objects to find non-scalable parameters such as the max number of queries that can run in the pool. These access have been encapsulated in functions that return the scalable version of the configuration value if the new scalable parameters are being used. Diagnostic messages are enhanced to show the origin of the encapsulated parameters. TESTING All end-to-end tests are running clean with ASAN. The unit test admission-controller-test.cc has been expanded to test the newly added code. Added an end-to-end test that adds and removes Impalads from a minicluster. Change-Id: If47508728124076f3b9200c27cffc989f7a4f188 --- M be/src/scheduling/admission-controller-test.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M common/thrift/ImpalaInternalService.thrift M common/thrift/generate_metrics.py M common/thrift/metrics.json M fe/src/main/java/org/apache/impala/util/RequestPoolService.java M fe/src/test/java/org/apache/impala/util/TestRequestPoolService.java M fe/src/test/resources/fair-scheduler-test.xml M fe/src/test/resources/fair-scheduler-test2.xml M fe/src/test/resources/llama-site-test.xml M fe/src/test/resources/llama-site-test2.xml M testdata/workloads/functional-query/queries/QueryTest/admission-max-min-mem-limits.test M testdata/workloads/functional-query/queries/QueryTest/admission-reject-mem-estimate.test M testdata/workloads/functional-query/queries/QueryTest/admission-reject-min-reservation.test M tests/common/impala_cluster.py M tests/custom_cluster/test_admission_controller.py M tests/webserver/test_web_pages.py M www/admission_controller.tmpl 19 files changed, 1,193 insertions(+), 270 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/13307/5 -- To view, visit http://gerrit.cloudera.org:8080/13307 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If47508728124076f3b9200c27cffc989f7a4f188 Gerrit-Change-Number: 13307 Gerrit-PatchSet: 5 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8536: Add Scalable Pool Configuration to Admission Controller.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/13307 ) Change subject: IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. .. Patch Set 5: It's actually easy (form a code point of view) to ignore quiescing machines - see see patch set 5. -- To view, visit http://gerrit.cloudera.org:8080/13307 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If47508728124076f3b9200c27cffc989f7a4f188 Gerrit-Change-Number: 13307 Gerrit-PatchSet: 5 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 21:37:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/13375 )
Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with
Kudu/HMS integration
..
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java
File fe/src/main/java/org/apache/impala/catalog/KuduTable.java:
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@231
PS4, Line 231: if (hmsHosts != null && kuduHmsHosts != null
&&hmsHosts.equals(kuduHmsHosts)) {
nit: separate by a space
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@234
PS4, Line 234: throw new ImpalaRuntimeException(
: String.format("Kudu is integrated with a different
Hive Metastore " +
: "than that used by Impala, Kudu is configured to
use the HMS: " +
: "%s, while Impala is configured to use the HMS:
%s",
: kuduHmsUris, hmsUris));
readability nit: remove 'else' and move this out of the scope if/else scope.
--
To view, visit http://gerrit.cloudera.org:8080/13375
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502
Gerrit-Change-Number: 13375
Gerrit-PatchSet: 4
Gerrit-Owner: Hao Hao
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Fri, 31 May 2019 21:46:10 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8564: Add table/view create time in the lineage graph
Fredy Wijaya has uploaded a new patch set (#18). (
http://gerrit.cloudera.org:8080/13399 )
Change subject: IMPALA-8564: Add table/view create time in the lineage graph
..
IMPALA-8564: Add table/view create time in the lineage graph
This patch adds table/view create time in the lineage graph. This is
needed for Impala/Atlas integration. See ATLAS-3080.
Below is an example of the updated lineage graph.
{
"queryText":"create table lineage_test_tbl as select int_col, tinyint_col
from functional.alltypes",
"queryId":"0:0",
"hash":"407f23b24758ffcb2ac445b9703f5c44",
"user":"dummy_user",
"timestamp":1547867921,
"edges":[
{
"sources":[
1
],
"targets":[
0
],
"edgeType":"PROJECTION"
},
{
"sources":[
3
],
"targets":[
2
],
"edgeType":"PROJECTION"
}
],
"vertices":[
{
"id":0,
"vertexType":"COLUMN",
"vertexId":"int_col",
"metadata":{
"tableName":"default.lineage_test_tbl",
"tableCreateTime":1559151337
}
},
{
"id":1,
"vertexType":"COLUMN",
"vertexId":"functional.alltypes.int_col",
"metadata":{
"tableName":"functional.alltypes",
"tableCreateTime":1559151317
}
},
{
"id":2,
"vertexType":"COLUMN",
"vertexId":"tinyint_col",
"metadata":{
"tableName":"default.lineage_test_tbl",
"tableCreateTime":1559151337
}
},
{
"id":3,
"vertexType":"COLUMN",
"vertexId":"functional.alltypes.tinyint_col",
"metadata":{
"tableName":"functional.alltypes",
"tableCreateTime":1559151317
}
}
]
}
Testing:
- Updated lineage tests in PlannerTest
- Updated test_lineage.py
- Ran all FE tests
Change-Id: If4f578d7b299a76c30323b10a883ba32f8713d82
---
M be/src/service/client-request-state.h
M be/src/service/frontend.cc
M be/src/service/impala-server.cc
M be/src/util/lineage-util.h
M common/thrift/CatalogService.thrift
M common/thrift/LineageGraph.thrift
M fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java
M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java
M testdata/workloads/functional-planner/queries/PlannerTest/lineage.test
M tests/custom_cluster/test_lineage.py
15 files changed, 2,101 insertions(+), 622 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/13399/18
--
To view, visit http://gerrit.cloudera.org:8080/13399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If4f578d7b299a76c30323b10a883ba32f8713d82
Gerrit-Change-Number: 13399
Gerrit-PatchSet: 18
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Anonymous Coward (498)
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8564: Add table/view create time in the lineage graph
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13399 ) Change subject: IMPALA-8564: Add table/view create time in the lineage graph .. Patch Set 17: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3468/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13399 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If4f578d7b299a76c30323b10a883ba32f8713d82 Gerrit-Change-Number: 13399 Gerrit-PatchSet: 17 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 31 May 2019 22:01:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8564: Add table/view create time in the lineage graph
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13399 ) Change subject: IMPALA-8564: Add table/view create time in the lineage graph .. Patch Set 18: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3470/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13399 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If4f578d7b299a76c30323b10a883ba32f8713d82 Gerrit-Change-Number: 13399 Gerrit-PatchSet: 18 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 31 May 2019 22:18:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin
Hello Thomas Marshall, Alexey Serbin, Andrew Wong, Adar Dembo, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13319 to look at the new patch set (#3). Change subject: IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin .. IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin This patch allows to start the Hive Metasotre with Kudu plugin which is required for enabling Kudu's integration with the HMS. The Kudu plugin is downloaded and extracted from native-toolchain S3 bucket. Change-Id: I4bd1488ced51840ec986d29ed371e26168abcc76 --- M bin/bootstrap_toolchain.py M bin/impala-config.sh M fe/src/test/resources/hive-site.xml.py M testdata/bin/run-hive-server.sh 4 files changed, 19 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/13319/3 -- To view, visit http://gerrit.cloudera.org:8080/13319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4bd1488ced51840ec986d29ed371e26168abcc76 Gerrit-Change-Number: 13319 Gerrit-PatchSet: 3 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8536: Add Scalable Pool Configuration to Admission Controller.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13307 ) Change subject: IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3469/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13307 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If47508728124076f3b9200c27cffc989f7a4f188 Gerrit-Change-Number: 13307 Gerrit-PatchSet: 5 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 22:28:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13319 ) Change subject: IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/13319/2/bin/bootstrap_toolchain.py File bin/bootstrap_toolchain.py: http://gerrit.cloudera.org:8080/#/c/13319/2/bin/bootstrap_toolchain.py@424 PS2, Line 424: # Kudu is the only component that's platform dependent. > So as written this isn't going to work for non-cdh supported platforms (eg. I see, thanks for the suggestion! I opt for naming the kudu java package 'kudu-java' to avoid touching much existing code. -- To view, visit http://gerrit.cloudera.org:8080/13319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4bd1488ced51840ec986d29ed371e26168abcc76 Gerrit-Change-Number: 13319 Gerrit-PatchSet: 3 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 22:29:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8425: part 1: reduce size of binaries in container
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13487 ) Change subject: IMPALA-8425: part 1: reduce size of binaries in container .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13487 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I95ff479bedd3b93e6569e72f03f42acd9dba8b14 Gerrit-Change-Number: 13487 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 31 May 2019 22:32:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13319 ) Change subject: IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4374/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4bd1488ced51840ec986d29ed371e26168abcc76 Gerrit-Change-Number: 13319 Gerrit-PatchSet: 3 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 22:35:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8502: Bump CDH BUILD NUMBER and Kudu version
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13481 ) Change subject: IMPALA-8502: Bump CDH_BUILD_NUMBER and Kudu version .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4375/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13481 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c9c7d264f08454d9da8ad2d06ead643214ee23e Gerrit-Change-Number: 13481 Gerrit-PatchSet: 4 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 22:49:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8502: Bump CDH BUILD NUMBER and Kudu version
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13481 ) Change subject: IMPALA-8502: Bump CDH_BUILD_NUMBER and Kudu version .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13481 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c9c7d264f08454d9da8ad2d06ead643214ee23e Gerrit-Change-Number: 13481 Gerrit-PatchSet: 4 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 22:51:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin
Thomas Marshall has posted comments on this change. (
http://gerrit.cloudera.org:8080/13319 )
Change subject: IMPALA-8503: allow the Hive Metastore to start with kudu-hive
plugin
..
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13319/3/bin/bootstrap_toolchain.py
File bin/bootstrap_toolchain.py:
http://gerrit.cloudera.org:8080/#/c/13319/3/bin/bootstrap_toolchain.py@571
PS3, Line 571: cdh_components += [Package("kudu-java")]
Could you add a comment here like:
Always download Kudu's jars regardless of USE_CDH_KUDU since they aren't
platform dependent and aren't packaged by the toolchain.
http://gerrit.cloudera.org:8080/#/c/13319/3/bin/impala-config.sh
File bin/impala-config.sh:
http://gerrit.cloudera.org:8080/#/c/13319/3/bin/impala-config.sh@670
PS3, Line 670: export
IMPALA_KUDU_JAVA_VERSION=${IMPALA_KUDU_VERSION-"1.10.0-cdh6.x-SNAPSHOT"}
I don't think this logic works - you'll get a weird result in the case that
USE_CDH_KUDU=false, since your java version will be getting set to the random
git hash above.
Could you just modify the existing env var KUDU_JAVA_VERSION on line 174 to
have IMPALA_ as a prefix? (and fix any place that var is used, I think the only
place is impala-parent/pom.xml)
Also, please do a full build with USE_CDH_KUDU=false set in order to confirm
that it works (you don't have to actually do this on a non-CDH platform, just
set the env variable, and also please delete your toolchain directory before
doing it and verify that the contents of cdh_components/kudu... are as
expected, i.e. the kudu-hive jar and not the binaries)
--
To view, visit http://gerrit.cloudera.org:8080/13319
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4bd1488ced51840ec986d29ed371e26168abcc76
Gerrit-Change-Number: 13319
Gerrit-PatchSet: 3
Gerrit-Owner: Hao Hao
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Fri, 31 May 2019 23:03:11 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13319 ) Change subject: IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3471/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4bd1488ced51840ec986d29ed371e26168abcc76 Gerrit-Change-Number: 13319 Gerrit-PatchSet: 3 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 23:09:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/13375 )
Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with
Kudu/HMS integration
..
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java
File fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java:
http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java@120
PS4, Line 120: /**
:* Return the value of thrift URI for the remote Hive
Metastore.
:*/
: public static String
getHiveMetastoreUrisKeyValue(IMetaStoreClient client)
: throws ConfigValSecurityException, TException {
: return client.getConfigValue(
: HIVE_METASTORE_URIS_KEY, DEFAULT_HIVE_METASTORE_URIS);
: }
Why to create this shortcut instead of just using getMetastoreConfigValue()
below specifying the HIVE_METASTORE_URIS_KEY ?
--
To view, visit http://gerrit.cloudera.org:8080/13375
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502
Gerrit-Change-Number: 13375
Gerrit-PatchSet: 4
Gerrit-Owner: Hao Hao
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Marshall
Gerrit-Comment-Date: Fri, 31 May 2019 23:12:19 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8504: Support CREATE TABLE statement with Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13318 ) Change subject: IMPALA-8504: Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 13: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13318 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I465673d749221bd5f3772814b1c22c2673a53f5c Gerrit-Change-Number: 13318 Gerrit-PatchSet: 13 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 23:24:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504: Support CREATE TABLE statement with Kudu/HMS integration
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13318 ) Change subject: IMPALA-8504: Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 13: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13318 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I465673d749221bd5f3772814b1c22c2673a53f5c Gerrit-Change-Number: 13318 Gerrit-PatchSet: 13 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 23:33:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8504: Support CREATE TABLE statement with Kudu/HMS integration
Thomas Marshall has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13318 ) Change subject: IMPALA-8504: Support CREATE TABLE statement with Kudu/HMS integration .. IMPALA-8504: Support CREATE TABLE statement with Kudu/HMS integration This commit adds support for the syntax of CREATE TABLE (and CTAS) statements for managed Kudu tables with Kudu/HMS integration. A follow up patch will address the actual handling of CREATE TABLE statement with Kudu/HMS integration. For a managed table the syntax remains the same. However, the detailed changes includes: 1) Kudu table will always be created with the new Kudu storage handler 'org.apache.kudu.hive.KuduStorageHandler' even when Kudu/HMS integration is disabled. The legacy storage handler will be eventually deprecated. 2) When Kudu/HMS integration is enabled, the Kudu table underneath the managed HMS table will follow the naming convention 'db_name.table_name' instead of 'impala::db_name.table_name'. 3) Add 'kudu.table_id' table property to be used with Kudu/HMS integration. This commit also extracts Kudu-related DDL parsing and analyzing tests, so that they can be run with or without Kudu/HMS integration enabled. Change-Id: I465673d749221bd5f3772814b1c22c2673a53f5c Reviewed-on: http://gerrit.cloudera.org:8080/13318 Tested-by: Impala Public Jenkins Reviewed-by: Thomas Marshall --- M bin/run-all-tests.sh M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/KuduUtil.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java A fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java A fe/src/test/java/org/apache/impala/analysis/AuditingKuduTest.java M fe/src/test/java/org/apache/impala/analysis/AuditingTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java A fe/src/test/java/org/apache/impala/customservice/CustomServiceRunner.java A fe/src/test/java/org/apache/impala/customservice/KuduHMSIntegrationTest.java M testdata/cluster/node_templates/common/etc/init.d/kudu-master M tests/query_test/test_kudu.py 16 files changed, 920 insertions(+), 577 deletions(-) Approvals: Impala Public Jenkins: Verified Thomas Marshall: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/13318 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I465673d749221bd5f3772814b1c22c2673a53f5c Gerrit-Change-Number: 13318 Gerrit-PatchSet: 14 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13375 ) Change subject: IMPALA-8504 (part 2): Support CREATE TABLE statement with Kudu/HMS integration .. Patch Set 4: (6 comments) http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java File fe/src/main/java/org/apache/impala/catalog/KuduTable.java: http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@172 PS4, Line 172: org.apache.hadoop.hive.metastore.api.Table msTbl, : boolean isHMSIntegrationEnabled these can fit on the same line http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1930 PS4, Line 1930: !KuduTable.isHMSIntegrationEnabledAndValidate(masterHosts, : hmsUris formatting - put everything after the '=' on the next line, indented by 4 http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java File fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java: http://gerrit.cloudera.org:8080/#/c/13375/4/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java@87 PS4, Line 87: public static final String HIVE_METASTORE_URIS_KEY = : "hive.metastore.uris"; This can fit on 1 line http://gerrit.cloudera.org:8080/#/c/13375/4/tests/custom_cluster/test_kudu.py File tests/custom_cluster/test_kudu.py: http://gerrit.cloudera.org:8080/#/c/13375/4/tests/custom_cluster/test_kudu.py@129 PS4, Line 129: manged typo http://gerrit.cloudera.org:8080/#/c/13375/4/tests/query_test/test_kudu.py File tests/query_test/test_kudu.py: http://gerrit.cloudera.org:8080/#/c/13375/4/tests/query_test/test_kudu.py@93 PS4, Line 93: @SkipIfKudu.hms_integration_enabled Why would a test like this, which is focused on scanning kudu tables, need to be skipped if hms integration is turned on? http://gerrit.cloudera.org:8080/#/c/13375/4/tests/query_test/test_kudu.py@175 PS4, Line 175: @SkipIfKudu.hms_integration_enabled I assume that for cases like these, where the table is modified outside of Impala, we expect that Impala will automatically pick up the changes and the error won't occur? -- To view, visit http://gerrit.cloudera.org:8080/13375 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icffe412395f47f5e07d97bad457020770cfa7502 Gerrit-Change-Number: 13375 Gerrit-PatchSet: 4 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 23:38:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8502: Bump CDH BUILD NUMBER and Kudu version
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13481 ) Change subject: IMPALA-8502: Bump CDH_BUILD_NUMBER and Kudu version .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13481 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c9c7d264f08454d9da8ad2d06ead643214ee23e Gerrit-Change-Number: 13481 Gerrit-PatchSet: 3 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 31 May 2019 23:49:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13319 ) Change subject: IMPALA-8503: allow the Hive Metastore to start with kudu-hive plugin .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4bd1488ced51840ec986d29ed371e26168abcc76 Gerrit-Change-Number: 13319 Gerrit-PatchSet: 2 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Sat, 01 Jun 2019 00:10:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8460: Simplify cluster membership management
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13207 ) Change subject: IMPALA-8460: Simplify cluster membership management .. Patch Set 21: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4370/ -- To view, visit http://gerrit.cloudera.org:8080/13207 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3cf9a8bb060d0c6e9ec8868b7b21ce01f8740a3 Gerrit-Change-Number: 13207 Gerrit-PatchSet: 21 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 01 Jun 2019 00:41:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8553,IMPALA-8552: fix checks for remote cluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13386 ) Change subject: IMPALA-8553,IMPALA-8552: fix checks for remote cluster .. Patch Set 4: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4371/ -- To view, visit http://gerrit.cloudera.org:8080/13386 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifa6b2a1391f53121d3d7c00c5cf0a57590899ce4 Gerrit-Change-Number: 13386 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 01 Jun 2019 00:53:46 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Added the section on object ownership
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13478 ) Change subject: [DOCS] Added the section on object ownership .. [DOCS] Added the section on object ownership Change-Id: Iff48684f457ef19a27524adfbcc2ae5e098320a3 Reviewed-on: http://gerrit.cloudera.org:8080/13478 Tested-by: Impala Public Jenkins Reviewed-by: Alex Rodoni --- M docs/topics/impala_authorization.xml 1 file changed, 39 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Verified Alex Rodoni: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/13478 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iff48684f457ef19a27524adfbcc2ae5e098320a3 Gerrit-Change-Number: 13478 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [DOCS] Added the section on object ownership
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13478 ) Change subject: [DOCS] Added the section on object ownership .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13478 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iff48684f457ef19a27524adfbcc2ae5e098320a3 Gerrit-Change-Number: 13478 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Sat, 01 Jun 2019 01:46:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8460: Simplify cluster membership management
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13207 ) Change subject: IMPALA-8460: Simplify cluster membership management .. Patch Set 22: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4376/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13207 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3cf9a8bb060d0c6e9ec8868b7b21ce01f8740a3 Gerrit-Change-Number: 13207 Gerrit-PatchSet: 22 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 01 Jun 2019 01:47:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8460: Simplify cluster membership management
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13207 ) Change subject: IMPALA-8460: Simplify cluster membership management .. Patch Set 22: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13207 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3cf9a8bb060d0c6e9ec8868b7b21ce01f8740a3 Gerrit-Change-Number: 13207 Gerrit-PatchSet: 22 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 01 Jun 2019 01:47:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8460: Simplify cluster membership management
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/13207 ) Change subject: IMPALA-8460: Simplify cluster membership management .. Patch Set 21: > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4370/ The build hit an unrelated, flaky issue: IMPALA-8567 -- To view, visit http://gerrit.cloudera.org:8080/13207 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3cf9a8bb060d0c6e9ec8868b7b21ce01f8740a3 Gerrit-Change-Number: 13207 Gerrit-PatchSet: 21 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 01 Jun 2019 01:47:38 + Gerrit-HasComments: No
[Impala-ASF-CR] Add decimal parquet to TableFormatInfo.KNOWN FILE FORMATS
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13398 ) Change subject: Add decimal_parquet to TableFormatInfo.KNOWN_FILE_FORMATS .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4372/ -- To view, visit http://gerrit.cloudera.org:8080/13398 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fc9c0ec82318e740a06493c1b4e4719c7162db Gerrit-Change-Number: 13398 Gerrit-PatchSet: 3 Gerrit-Owner: David Knupp Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Sat, 01 Jun 2019 02:04:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8502: Bump CDH BUILD NUMBER and Kudu version
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13481 ) Change subject: IMPALA-8502: Bump CDH_BUILD_NUMBER and Kudu version .. Patch Set 4: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13481 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c9c7d264f08454d9da8ad2d06ead643214ee23e Gerrit-Change-Number: 13481 Gerrit-PatchSet: 4 Gerrit-Owner: Hao Hao Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Sat, 01 Jun 2019 02:33:43 + Gerrit-HasComments: No
