[Impala-ASF-CR] IMPALA-8585: Skip most ACID EE tests in non-HDFS builds
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/13921 ) Change subject: IMPALA-8585: Skip most ACID EE tests in non-HDFS builds .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13921 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5f67654805b770d86090b0d1015e144147d73f40 Gerrit-Change-Number: 13921 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 26 Jul 2019 06:36:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8571[WIP]: improve QueryEventHook execution
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13748 ) Change subject: IMPALA-8571[WIP]: improve QueryEventHook execution .. Patch Set 17: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4015/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/13748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643 Gerrit-Change-Number: 13748 Gerrit-PatchSet: 17 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Fri, 26 Jul 2019 05:16:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8571[WIP]: improve QueryEventHook execution
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13748 ) Change subject: IMPALA-8571[WIP]: improve QueryEventHook execution .. Patch Set 17: (1 comment) http://gerrit.cloudera.org:8080/#/c/13748/17/be/src/util/hook-metrics.cc File be/src/util/hook-metrics.cc: http://gerrit.cloudera.org:8080/#/c/13748/17/be/src/util/hook-metrics.cc@58 PS17, Line 58: line has trailing whitespace -- To view, visit http://gerrit.cloudera.org:8080/13748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643 Gerrit-Change-Number: 13748 Gerrit-PatchSet: 17 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Fri, 26 Jul 2019 04:36:57 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8571[WIP]: improve QueryEventHook execution
Hello Bharath Vissapragada, Fredy Wijaya, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13748
to look at the new patch set (#17).
Change subject: IMPALA-8571[WIP]: improve QueryEventHook execution
..
IMPALA-8571[WIP]: improve QueryEventHook execution
(WIP because still need to implement polling of metrics by backend)
This commit hardens guarantees around QueryEventHook execution by
adding the following features:
*hook timeout/cancellation*
A timeout for hook execution can be configured through the backend
flag `query_event_hook_timeout_s`, which specified a timeout value
in seconds. If a hook has not completed execution within this timeout
(measured from hook submission, not execution) then the hook task
will be cancelled in order to free up resources.
*hook rejection*
The hook execution engine now has a fixed-capacity work queue whose
capacity can be configured through the backend flag
`query_event_hook_queue_capacity`. This queue is used to store
hook tasks that are submitted when there are no free threads
available for hook execution. All hook tasks submitted when the
queue is at capacity will be rejected and logged without affecting
the result of the query.
*hook performance metrics*
The following hook metrics are captured:
*query-event-hook.${hook_method}.execution-rejections*
Counter indicating how many submitted tasks have been rejected
due to a full work queue
*query-event-hook.${hook_method}.execution-exceptions*
Counter indicating how many tasks have thrown an exception
during execution
*query-event-hook.${hook_method}.execution-timeouts*
Counter indicating how many tasks have been cancelled due to
not completing within {@code hookTimeout_s} of submission.
*query-event-hook.${hook_method}.execution-submissions*
Counter indicating the number of times ${hookClass}.${method}
has been submitted for execution.
*query-event-hook.${hook_method}.mean-execution-time*
Mean time in [ns] that ${hook_name} has taken to complete,
whether normally or by error (e.g. timeout or exception).
*query-event-hook.${hook_method}.mean-queued-time*
Mean time in [ns] between hook task submission
and hook task execution. This indicates how long a hook
task has been queued waiting to execute.
Testing:
- added unit tests for new features
- re-ran existing E2E tests
Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643
---
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/util/CMakeLists.txt
M be/src/util/backend-gflag-util.cc
A be/src/util/hook-metrics-test.cc
A be/src/util/hook-metrics.cc
A be/src/util/hook-metrics.h
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
A fe/src/main/java/org/apache/impala/hooks/FixedCapacityQueryHookExecutor.java
M fe/src/main/java/org/apache/impala/hooks/QueryEventHookManager.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
A
fe/src/test/java/org/apache/impala/hooks/FixedCapacityQueryHookExecutorTest.java
M fe/src/test/java/org/apache/impala/hooks/QueryEventHookManagerTest.java
M query-event-hook-api/src/main/java/org/apache/impala/hooks/QueryEventHook.java
19 files changed, 1,376 insertions(+), 114 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/13748/17
--
To view, visit http://gerrit.cloudera.org:8080/13748
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643
Gerrit-Change-Number: 13748
Gerrit-PatchSet: 17
Gerrit-Owner: radford nguyen
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] WIP: Add debugging tools to our docker images
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/13895 ) Change subject: WIP: Add debugging tools to our docker images .. Patch Set 1: Code-Review+1 I'll let others weigh in, but I think having these tools makes life much simpler. -- To view, visit http://gerrit.cloudera.org:8080/13895 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I47c7aa7076cebfa3bfad2029fb1da9e64364f0e6 Gerrit-Change-Number: 13895 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Fri, 26 Jul 2019 04:23:07 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: Add debugging tools to our docker images
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/13895 ) Change subject: WIP: Add debugging tools to our docker images .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/13895/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13895/1//COMMIT_MSG@13 PS1, Line 13: 984MB 846MB -- To view, visit http://gerrit.cloudera.org:8080/13895 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I47c7aa7076cebfa3bfad2029fb1da9e64364f0e6 Gerrit-Change-Number: 13895 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Fri, 26 Jul 2019 04:20:01 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8489: Partitions created by RECOVER PARTITIONS fail to create insert events with IllegalStateException.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13860 ) Change subject: IMPALA-8489: Partitions created by RECOVER PARTITIONS fail to create insert events with IllegalStateException. .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13860 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idef7f6aadff2868047c861ebfcc05d65f080eab9 Gerrit-Change-Number: 13860 Gerrit-PatchSet: 5 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 26 Jul 2019 03:39:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8434: retain tables and functions in altering database
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13904 ) Change subject: IMPALA-8434: retain tables and functions in altering database .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13904 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3dc9857fd2733e20cf10fbe17bb1a4670d7d015 Gerrit-Change-Number: 13904 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 26 Jul 2019 03:28:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8434: retain tables and functions in altering database
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13904 ) Change subject: IMPALA-8434: retain tables and functions in altering database .. IMPALA-8434: retain tables and functions in altering database In the legacy catalog implementation (ImpaladCatalog), when altering a database, the tables and functions in it will disappear until we run INVALIDATE METADATA to reset the cache. The cause is that we just replace the old Db object with the new one deserialized from the TDatabase. We should migrate the existing tables and functions to the new Db object. Tests: - Add test_metadata_after_alter_database for the bug. - Run Core tests Change-Id: Ia3dc9857fd2733e20cf10fbe17bb1a4670d7d015 Reviewed-on: http://gerrit.cloudera.org:8080/13904 Reviewed-by: Vihang Karajgaonkar Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java M tests/metadata/test_ddl.py 2 files changed, 29 insertions(+), 1 deletion(-) Approvals: Vihang Karajgaonkar: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/13904 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia3dc9857fd2733e20cf10fbe17bb1a4670d7d015 Gerrit-Change-Number: 13904 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl
Michael Ho has posted comments on this change. (
http://gerrit.cloudera.org:8080/13883 )
Change subject: IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and
BufferedPRS impl
..
Patch Set 11:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/13883/11/be/src/exec/buffered-plan-root-sink.h
File be/src/exec/buffered-plan-root-sink.h:
http://gerrit.cloudera.org:8080/#/c/13883/11/be/src/exec/buffered-plan-root-sink.h@64
PS11, Line 64: //
nit: ///
http://gerrit.cloudera.org:8080/#/c/13883/11/be/src/exec/buffered-plan-root-sink.cc
File be/src/exec/buffered-plan-root-sink.cc:
http://gerrit.cloudera.org:8080/#/c/13883/11/be/src/exec/buffered-plan-root-sink.cc@27
PS11, Line 27: 10
This should be a static constant
http://gerrit.cloudera.org:8080/#/c/13883/11/be/src/exec/buffered-plan-root-sink.cc@54
PS11, Line 54: while (batch_queue_->IsFull()) {
: is_full_.Wait(l);
: RETURN_IF_CANCELLED(state);
: }
May hang for a while if the sink is already cancelled when we get here, it may
be more safer to do:
while (!state->cancelled() && batch_queue->IsFull()) {
is_full_.Wait(l);
}
RETURN_IF_CANCELLED(state);
http://gerrit.cloudera.org:8080/#/c/13883/11/be/src/exec/buffered-plan-root-sink.cc@103
PS11, Line 103: rows_available_.NotifyOne();
: consumer_eos_.NotifyOne();
: is_full_.NotifyOne();
I think keeping NotifyAll() make sense for the cancellation path. Same for
Close(). NotifyOne() makes assumption about the number of waiters on these
condition variables, which seems dicey for the cancellation path and it doesn't
hurt to do a NotifyAll() for the slow path.
http://gerrit.cloudera.org:8080/#/c/13883/11/be/src/exec/buffered-plan-root-sink.cc@123
PS11, Line 123: // For now, if num_results < batch->num_rows(), we
terminate returning results
: // early.
: if (num_results > 0 && num_results < batch->num_rows()) {
Not sure I understand this part ? Shouldn't this function still need to insert
up to 'num_results' rows into 'results' in this case ?
In other words, if the row batch at the front of the queue is not completely
consumed, don't you need to track how many rows have been consumed from it ?
http://gerrit.cloudera.org:8080/#/c/13883/11/be/src/exec/buffered-plan-root-sink.cc@127
PS11, Line 127: consumer_eos_.NotifyOne();
May need to do is_full_.NotifyOne() here in case the sender was blocked. Seems
better to refactor this code (if possible) to always return at line 147.
http://gerrit.cloudera.org:8080/#/c/13883/11/be/src/exec/scan-node.cc
File be/src/exec/scan-node.cc:
http://gerrit.cloudera.org:8080/#/c/13883/11/be/src/exec/scan-node.cc@262
PS11, Line 262: ADD_TIMER(parent->runtime_profile(),
"RowBatchQueueGetWaitTime"),
: ADD_TIMER(parent->runtime_profile(),
"RowBatchQueuePutWaitTime"))
The code seems easier to follow if we keep the ADD_TIMER() macro at the old
call sites and simply reference those timers here.
http://gerrit.cloudera.org:8080/#/c/13883/11/be/src/util/blocking-queue.h
File be/src/util/blocking-queue.h:
http://gerrit.cloudera.org:8080/#/c/13883/11/be/src/util/blocking-queue.h@108
PS11, Line 108: get_wait_timer_)
get_wait_timer_ != nullptr
--
To view, visit http://gerrit.cloudera.org:8080/13883
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b1bb4b9c6f6e92c70e8fbee6ccdf48c2f85b7be
Gerrit-Change-Number: 13883
Gerrit-PatchSet: 11
Gerrit-Owner: Sahil Takiar
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Sahil Takiar
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Fri, 26 Jul 2019 02:31:40 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8339: Add local executor blacklist to coordinators
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/13868 ) Change subject: IMPALA-8339: Add local executor blacklist to coordinators .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.cc File be/src/scheduling/executor-blacklist.cc: http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.cc@29 PS3, Line 29: 0 disables blacklisting > My problem with setting it to 120 is that it makes it possible for users to I see. May be it's simpler to not have this padding flag at all and use a constant padding percentage instead (e.g. 20 ~ 30%). The consequence of having a padding value too small is that we may end up removing a bad executor from a node too quickly. However, this is not catastrophic once we implement the transparent query retry in the follow-up patch, right ? I am okay with a a flag which disables blacklisting in case something goes tremendously wrong. -- To view, visit http://gerrit.cloudera.org:8080/13868 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iacb6e73b84042c33cd475b82470a975d04ee9b74 Gerrit-Change-Number: 13868 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Fri, 26 Jul 2019 02:30:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13883 ) Change subject: IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl .. Patch Set 10: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4013/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/13883 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9b1bb4b9c6f6e92c70e8fbee6ccdf48c2f85b7be Gerrit-Change-Number: 13883 Gerrit-PatchSet: 10 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 26 Jul 2019 01:40:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13883 ) Change subject: IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl .. Patch Set 11: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4014/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/13883 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9b1bb4b9c6f6e92c70e8fbee6ccdf48c2f85b7be Gerrit-Change-Number: 13883 Gerrit-PatchSet: 11 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 26 Jul 2019 01:34:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8717: impala-shell support for HS2 HTTP endpoint
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13746 ) Change subject: IMPALA-8717: impala-shell support for HS2 HTTP endpoint .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4012/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8323950857dfe1c1dfd5377fde79f87bc2ce9534 Gerrit-Change-Number: 13746 Gerrit-PatchSet: 12 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 26 Jul 2019 01:22:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13918 ) Change subject: IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4011/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15d9a842ab37ebc34b9fde5917137ff2961d870a Gerrit-Change-Number: 13918 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 26 Jul 2019 01:11:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/13883 ) Change subject: IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl .. Patch Set 9: (10 comments) http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/exec/buffered-plan-root-sink.cc File be/src/exec/buffered-plan-root-sink.cc: http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/exec/buffered-plan-root-sink.cc@65 PS9, Line 65: rows_available_.NotifyAll(); > It's better to drop the lock before calling notify, so that the woken threa Done. Switched to NotifyOne() as well as per the recommendation in IMPALA-6125 I looked through the other uses of Notify and made similar changes, but I couldn't change all of them because it would introduce race conditions. http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/exec/buffered-plan-root-sink.cc@74 PS9, Line 74: way > wake Done http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/exec/buffered-plan-root-sink.cc@104 PS9, Line 104: Status BufferedPlanRootSink::GetNext( > I saw a null pointer dereference causing a crash in the function. Emailed S Done http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/exec/buffered-plan-root-sink.cc@121 PS9, Line 121: *eos = true; > Are you missing a consumer_eos_.NotifyAll() here ? Done http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/exec/plan-root-sink.h File be/src/exec/plan-root-sink.h: http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/exec/plan-root-sink.h@84 PS9, Line 84: approriate > typo Yeah, I decided just to split this up into two methods since it makes more sense. http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/exec/plan-root-sink.h@107 PS9, Line 107: Send() > stale Done http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/runtime/blocking-row-batch-queue.cc File be/src/runtime/blocking-row-batch-queue.cc: http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/runtime/blocking-row-batch-queue.cc@66 PS9, Line 66: row_batches_put_timer_->Set(batch_queue_->total_put_wait_time()); : row_batches_get_timer_->Set(batch_queue_->total_get_wait_time()); > I believe the TODO has more to do with not using this pattern of setting th Done http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/runtime/deque-row-batch-queue.h File be/src/runtime/deque-row-batch-queue.h: http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/runtime/deque-row-batch-queue.h@57 PS9, Line 57: remanining > typo. Done http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/runtime/deque-row-batch-queue.h@68 PS9, Line 68: int max_batches_; > const Done http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/runtime/deque-row-batch-queue.cc File be/src/runtime/deque-row-batch-queue.cc: http://gerrit.cloudera.org:8080/#/c/13883/9/be/src/runtime/deque-row-batch-queue.cc@61 PS9, Line 61: result->Reset(); > I was able to trigger a segfault from a null pointer here by running "selec Done -- To view, visit http://gerrit.cloudera.org:8080/13883 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9b1bb4b9c6f6e92c70e8fbee6ccdf48c2f85b7be Gerrit-Change-Number: 13883 Gerrit-PatchSet: 9 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 26 Jul 2019 01:01:54 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/13883 ) Change subject: IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl .. Patch Set 11: > (4 comments) > > I played around with this a little bit and was definitely able to > see the spooling behaviour by suspending impala-shell. I ran into a > couple of crashes though - I think the problems might be around > null row batches being dequeued from the queue Thanks for doing some testing. I recently switched from using a std::queue to using a std:deque and think I got my push / pop functions mixed up. Regardless, the issue is fixed now. I'm able to run "select distinct * from tpch.lineitem limit 1" successfully and cancelling the query midway doesn't cause any crashes. I also looped the tests in https://gerrit.cloudera.org/#/c/13907/ for about an hour and everything passed. -- To view, visit http://gerrit.cloudera.org:8080/13883 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9b1bb4b9c6f6e92c70e8fbee6ccdf48c2f85b7be Gerrit-Change-Number: 13883 Gerrit-PatchSet: 11 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 26 Jul 2019 00:54:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl
Hello Michael Ho, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13883 to look at the new patch set (#11). Change subject: IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl .. IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl Improves the encapsulation of RowBatchQueue by the doing the following re-factoring: * Renames RowBatchQueue to BlockingRowBatchQueue which is more indicitive of what the queue does * Re-factors the timers managed by the scan-node into the BlockingRowBatchQueue implementation * Favors composition over inheritance by re-factoring BlockingRowBatchQueue to own a BlockingQueue rather than extending one The re-factoring lays the groundwork for introducing a generic RowBatchQueue that all RowBatch queues inherit from. Adds a new DequeRowBatchQueue which is a simple wrapper around a std::deque that (1) stores unique_ptr to queued RowBatch-es and (2) has a maximum capacity. Implements BufferedPlanRootSink using the new DequeRowBatchQueue. DequeRowBatchQueue is generic enough that replacing it with a SpillableQueue (queue backed by a BufferedTupleStream) should be straightforward. BufferedPlanRootSink is synchronized to protect access to DequeRowBatchQueue since the queue is not thread safe. BufferedPlanRootSink FlushFinal blocks until the consumer thread has processed all RowBatches. This ensures that the coordinator fragment stays alive until all results are fetched, but allows all other fragments to be shutdown immediately. Testing: * Running core tests * Updated tests/query_test/test_result_spooling.py Change-Id: I9b1bb4b9c6f6e92c70e8fbee6ccdf48c2f85b7be --- M be/src/exec/blocking-plan-root-sink.cc M be/src/exec/blocking-plan-root-sink.h M be/src/exec/buffered-plan-root-sink.cc M be/src/exec/buffered-plan-root-sink.h M be/src/exec/hdfs-scan-node.cc M be/src/exec/kudu-scan-node.cc M be/src/exec/plan-root-sink.cc M be/src/exec/plan-root-sink.h M be/src/exec/scan-node.cc M be/src/exec/scan-node.h M be/src/exec/scanner-context.cc M be/src/runtime/CMakeLists.txt A be/src/runtime/blocking-row-batch-queue.cc A be/src/runtime/blocking-row-batch-queue.h A be/src/runtime/deque-row-batch-queue.cc A be/src/runtime/deque-row-batch-queue.h D be/src/runtime/row-batch-queue.cc D be/src/runtime/row-batch-queue.h M be/src/util/blocking-queue.h M tests/query_test/test_result_spooling.py 20 files changed, 550 insertions(+), 214 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/13883/11 -- To view, visit http://gerrit.cloudera.org:8080/13883 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9b1bb4b9c6f6e92c70e8fbee6ccdf48c2f85b7be Gerrit-Change-Number: 13883 Gerrit-PatchSet: 11 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13883 ) Change subject: IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl .. Patch Set 10: (3 comments) http://gerrit.cloudera.org:8080/#/c/13883/10/be/src/exec/buffered-plan-root-sink.cc File be/src/exec/buffered-plan-root-sink.cc: http://gerrit.cloudera.org:8080/#/c/13883/10/be/src/exec/buffered-plan-root-sink.cc@61 PS10, Line 61: // Adding a batch should always be successful because the queue should always be open line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/13883/10/be/src/exec/buffered-plan-root-sink.cc@62 PS10, Line 62: // when Send is called, and the call to is_full_.Wait(l) ensures spaces is available. line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/13883/10/be/src/exec/buffered-plan-root-sink.cc@122 PS10, Line 122: // For now, if num_results < batch->num_rows(), we terminate returning results early. line too long (91 > 90) -- To view, visit http://gerrit.cloudera.org:8080/13883 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9b1bb4b9c6f6e92c70e8fbee6ccdf48c2f85b7be Gerrit-Change-Number: 13883 Gerrit-PatchSet: 10 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 26 Jul 2019 00:51:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl
Hello Michael Ho, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13883 to look at the new patch set (#10). Change subject: IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl .. IMPALA-8779, IMPALA-8780: RowBatchQueue re-factoring and BufferedPRS impl Improves the encapsulation of RowBatchQueue by the doing the following re-factoring: * Renames RowBatchQueue to BlockingRowBatchQueue which is more indicitive of what the queue does * Re-factors the timers managed by the scan-node into the BlockingRowBatchQueue implementation * Favors composition over inheritance by re-factoring BlockingRowBatchQueue to own a BlockingQueue rather than extending one The re-factoring lays the groundwork for introducing a generic RowBatchQueue that all RowBatch queues inherit from. Adds a new DequeRowBatchQueue which is a simple wrapper around a std::deque that (1) stores unique_ptr to queued RowBatch-es and (2) has a maximum capacity. Implements BufferedPlanRootSink using the new DequeRowBatchQueue. DequeRowBatchQueue is generic enough that replacing it with a SpillableQueue (queue backed by a BufferedTupleStream) should be straightforward. BufferedPlanRootSink is synchronized to protect access to DequeRowBatchQueue since the queue is not thread safe. BufferedPlanRootSink FlushFinal blocks until the consumer thread has processed all RowBatches. This ensures that the coordinator fragment stays alive until all results are fetched, but allows all other fragments to be shutdown immediately. Testing: * Running core tests * Updated tests/query_test/test_result_spooling.py Change-Id: I9b1bb4b9c6f6e92c70e8fbee6ccdf48c2f85b7be --- M be/src/exec/blocking-plan-root-sink.cc M be/src/exec/blocking-plan-root-sink.h M be/src/exec/buffered-plan-root-sink.cc M be/src/exec/buffered-plan-root-sink.h M be/src/exec/hdfs-scan-node.cc M be/src/exec/kudu-scan-node.cc M be/src/exec/plan-root-sink.cc M be/src/exec/plan-root-sink.h M be/src/exec/scan-node.cc M be/src/exec/scan-node.h M be/src/exec/scanner-context.cc M be/src/runtime/CMakeLists.txt A be/src/runtime/blocking-row-batch-queue.cc A be/src/runtime/blocking-row-batch-queue.h A be/src/runtime/deque-row-batch-queue.cc A be/src/runtime/deque-row-batch-queue.h D be/src/runtime/row-batch-queue.cc D be/src/runtime/row-batch-queue.h M be/src/util/blocking-queue.h M tests/query_test/test_result_spooling.py 20 files changed, 548 insertions(+), 214 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/13883/10 -- To view, visit http://gerrit.cloudera.org:8080/13883 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9b1bb4b9c6f6e92c70e8fbee6ccdf48c2f85b7be Gerrit-Change-Number: 13883 Gerrit-PatchSet: 10 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8717: impala-shell support for HS2 HTTP endpoint
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13746 ) Change subject: IMPALA-8717: impala-shell support for HS2 HTTP endpoint .. Patch Set 12: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4661/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8323950857dfe1c1dfd5377fde79f87bc2ce9534 Gerrit-Change-Number: 13746 Gerrit-PatchSet: 12 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 26 Jul 2019 00:42:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8717: impala-shell support for HS2 HTTP endpoint
Hello Thomas Tauber-Marshall, David Knupp, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13746 to look at the new patch set (#12). Change subject: IMPALA-8717: impala-shell support for HS2 HTTP endpoint .. IMPALA-8717: impala-shell support for HS2 HTTP endpoint Adds impala-shell support to connect to HiveServer2 HTTP endpoint. Relies on toolchain change at https://gerrit.cloudera.org/#/c/13725/. Use --protocol='hs2-http' to enable this behavior. Example usages: --- impala-shell --protocol='hs2-http' (No auth) impala-shell --protocol='hs2-http' --ldap -u. (PLAIN auth) impala-shell --protocol-'hs2-http' --ssl --ca_cert... (TLS) impala-shell --protocol='hs2-http' --ldap --ssl --ca_cert... (LDAP + TLS) Limitations: --- - Does not support Kerberos (-k) due to lack ot SPNEGO support. Testing: - Parameterized existing shell tests to support this combination. - Added shell test coverage for LDAP auth. Change-Id: I8323950857dfe1c1dfd5377fde79f87bc2ce9534 --- M be/src/service/impala-server.cc M bin/impala-config.sh A fe/src/test/java/org/apache/impala/customcluster/LdapImpalaShellTest.java M shell/impala_client.py M shell/impala_shell.py M shell/option_parser.py M tests/common/impala_cluster.py M tests/common/impala_service.py M tests/common/impala_test_suite.py M tests/common/test_dimensions.py M tests/conftest.py M tests/custom_cluster/test_client_ssl.py M tests/run-tests.py M tests/shell/test_shell_commandline.py M tests/shell/test_shell_interactive.py M tests/shell/util.py 16 files changed, 296 insertions(+), 49 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/13746/12 -- To view, visit http://gerrit.cloudera.org:8080/13746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8323950857dfe1c1dfd5377fde79f87bc2ce9534 Gerrit-Change-Number: 13746 Gerrit-PatchSet: 12 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8717: impala-shell support for HS2 HTTP endpoint
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13746 ) Change subject: IMPALA-8717: impala-shell support for HS2 HTTP endpoint .. Patch Set 11: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4010/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/13746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8323950857dfe1c1dfd5377fde79f87bc2ce9534 Gerrit-Change-Number: 13746 Gerrit-PatchSet: 11 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 26 Jul 2019 00:35:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13918 ) Change subject: IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/13918/1/be/src/transport/THttpServer.cpp File be/src/transport/THttpServer.cpp: http://gerrit.cloudera.org:8080/#/c/13918/1/be/src/transport/THttpServer.cpp@256 PS1, Line 256: sprintf(buff, > This is for the error case, i.e. if a client attempts to connect without pr Done -- To view, visit http://gerrit.cloudera.org:8080/13918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15d9a842ab37ebc34b9fde5917137ff2961d870a Gerrit-Change-Number: 13918 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 26 Jul 2019 00:30:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8636: Implement INSERT for insert-only ACID tables
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13559 ) Change subject: IMPALA-8636: Implement INSERT for insert-only ACID tables .. Patch Set 15: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13559 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id6c36fa6902676f06b4e38730f737becfc7c06ad Gerrit-Change-Number: 13559 Gerrit-PatchSet: 15 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 26 Jul 2019 00:30:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server
Hello Todd Lipcon, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13918 to look at the new patch set (#2). Change subject: IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server .. IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server IMPALA-8538 added support for http connections to the hs2 server along with LDAP auth support. This patch adds support for authorizing with Kerberos via SPNEGO. Testing: - Added a unit test that runs a minikdc and an Impala server against it. Currently disabled until curl is available in the toolchain. - Tested manually on a real cluster with a full Kerberos setup and beeline. Change-Id: I15d9a842ab37ebc34b9fde5917137ff2961d870a --- M be/src/rpc/CMakeLists.txt M be/src/rpc/TAcceptQueueServer.cpp M be/src/rpc/auth-provider.h M be/src/rpc/authentication.cc A be/src/rpc/kerberos-test.cc M be/src/rpc/thrift-server.cc M be/src/rpc/thrift-server.h M be/src/service/impala-server.cc M be/src/transport/THttpServer.cpp M be/src/transport/THttpServer.h M common/thrift/metrics.json 11 files changed, 402 insertions(+), 98 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/13918/2 -- To view, visit http://gerrit.cloudera.org:8080/13918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I15d9a842ab37ebc34b9fde5917137ff2961d870a Gerrit-Change-Number: 13918 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-7486: Add specialized estimation scheme for dedicated coordinators
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13740 ) Change subject: IMPALA-7486: Add specialized estimation scheme for dedicated coordinators .. Patch Set 13: Code-Review+1 (4 comments) Looks good except for a couple of minor things. I'll +1, Lars might want to take another look too http://gerrit.cloudera.org:8080/#/c/13740/13/be/src/scheduling/admission-controller.h File be/src/scheduling/admission-controller.h: http://gerrit.cloudera.org:8080/#/c/13740/13/be/src/scheduling/admission-controller.h@806 PS13, Line 806: weather whether http://gerrit.cloudera.org:8080/#/c/13740/13/be/src/scheduling/admission-controller.cc File be/src/scheduling/admission-controller.cc: http://gerrit.cloudera.org:8080/#/c/13740/13/be/src/scheduling/admission-controller.cc@401 PS13, Line 401: node_mem maybe mem_to_admit or backend_mem_to_admit, to avoid introducing the "node" terminology. http://gerrit.cloudera.org:8080/#/c/13740/13/be/src/service/client-request-state.cc File be/src/service/client-request-state.cc: http://gerrit.cloudera.org:8080/#/c/13740/13/be/src/service/client-request-state.cc@70 PS13, Line 70: DECLARE_bool(is_executor); needed? http://gerrit.cloudera.org:8080/#/c/13740/13/testdata/workloads/functional-query/queries/QueryTest/dedicated-coord-mem-estimates.test File testdata/workloads/functional-query/queries/QueryTest/dedicated-coord-mem-estimates.test: PS13: In some ways it would be better if these were planner tests - they would run a bit faster and it would be a bit easier to update the expected output on planner changes, but I think this is OK - these should not change with any frequency. -- To view, visit http://gerrit.cloudera.org:8080/13740 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2b94e7293b91dec8a18491079c34923eadd94b21 Gerrit-Change-Number: 13740 Gerrit-PatchSet: 13 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 26 Jul 2019 00:16:53 + Gerrit-HasComments: Yes
[Impala-ASF-CR] modified MetaStoreEventsProcessor to catch multiple invalid configuration keys in one go.
sharanitha.har...@cloudera.com has abandoned this change. ( http://gerrit.cloudera.org:8080/13894 ) Change subject: modified MetaStoreEventsProcessor to catch multiple invalid configuration keys in one go. .. Abandoned made some changes to rectify the same patch -- To view, visit http://gerrit.cloudera.org:8080/13894 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Ie6f4b22a810714c7f2a3c35b7010781a8a071493 Gerrit-Change-Number: 13894 Gerrit-PatchSet: 1 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8717: impala-shell support for HS2 HTTP endpoint
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/13746 ) Change subject: IMPALA-8717: impala-shell support for HS2 HTTP endpoint .. Patch Set 11: Missed a bunch of asserts, I fixed the tests and I'm running the test suite again. Lets see how it goes. -- To view, visit http://gerrit.cloudera.org:8080/13746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8323950857dfe1c1dfd5377fde79f87bc2ce9534 Gerrit-Change-Number: 13746 Gerrit-PatchSet: 11 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 23:57:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8717: impala-shell support for HS2 HTTP endpoint
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13746 ) Change subject: IMPALA-8717: impala-shell support for HS2 HTTP endpoint .. Patch Set 11: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4660/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8323950857dfe1c1dfd5377fde79f87bc2ce9534 Gerrit-Change-Number: 13746 Gerrit-PatchSet: 11 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 23:55:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8717: impala-shell support for HS2 HTTP endpoint
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13746 ) Change subject: IMPALA-8717: impala-shell support for HS2 HTTP endpoint .. Patch Set 11: (1 comment) http://gerrit.cloudera.org:8080/#/c/13746/11/tests/common/impala_test_suite.py File tests/common/impala_test_suite.py: http://gerrit.cloudera.org:8080/#/c/13746/11/tests/common/impala_test_suite.py@222 PS11, Line 222: f flake8: E999 SyntaxError: invalid syntax -- To view, visit http://gerrit.cloudera.org:8080/13746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8323950857dfe1c1dfd5377fde79f87bc2ce9534 Gerrit-Change-Number: 13746 Gerrit-PatchSet: 11 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 23:55:19 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8717: impala-shell support for HS2 HTTP endpoint
Hello Thomas Tauber-Marshall, David Knupp, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13746 to look at the new patch set (#11). Change subject: IMPALA-8717: impala-shell support for HS2 HTTP endpoint .. IMPALA-8717: impala-shell support for HS2 HTTP endpoint Adds impala-shell support to connect to HiveServer2 HTTP endpoint. Relies on toolchain change at https://gerrit.cloudera.org/#/c/13725/. Use --protocol='hs2-http' to enable this behavior. Example usages: --- impala-shell --protocol='hs2-http' (No auth) impala-shell --protocol='hs2-http' --ldap -u. (PLAIN auth) impala-shell --protocol-'hs2-http' --ssl --ca_cert... (TLS) impala-shell --protocol='hs2-http' --ldap --ssl --ca_cert... (LDAP + TLS) Limitations: --- - Does not support Kerberos (-k) due to lack ot SPNEGO support. Testing: - Parameterized existing shell tests to support this combination. - Added shell test coverage for LDAP auth. Change-Id: I8323950857dfe1c1dfd5377fde79f87bc2ce9534 --- M be/src/service/impala-server.cc M bin/impala-config.sh A fe/src/test/java/org/apache/impala/customcluster/LdapImpalaShellTest.java M shell/impala_client.py M shell/impala_shell.py M shell/option_parser.py M tests/common/impala_cluster.py M tests/common/impala_service.py M tests/common/impala_test_suite.py M tests/common/test_dimensions.py M tests/conftest.py M tests/custom_cluster/test_client_ssl.py M tests/run-tests.py M tests/shell/test_shell_commandline.py M tests/shell/test_shell_interactive.py M tests/shell/util.py 16 files changed, 296 insertions(+), 49 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/13746/11 -- To view, visit http://gerrit.cloudera.org:8080/13746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8323950857dfe1c1dfd5377fde79f87bc2ce9534 Gerrit-Change-Number: 13746 Gerrit-PatchSet: 11 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13900 ) Change subject: IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes .. Patch Set 5: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/412/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13900 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I46c1941269feff18306863f784aa36f5037da1db Gerrit-Change-Number: 13900 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 25 Jul 2019 23:29:51 + Gerrit-HasComments: No
[Impala-ASF-CR] This commit adds support to display the metric last-synced-event-id as Catalogd/metrics#evnts page whereas previously it was displayed only on the catalod/events page. Added code to th
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13925 ) Change subject: This commit adds support to display the metric last-synced-event-id as Catalogd/metrics#evnts page whereas previously it was displayed only on the catalod/events page. Added code to the MetaStoreEventsProcessorTest class under testEventProcessorMetrics() .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4008/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13925 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75b966f2b4eaafbcf7d80358f53501bb7ade67e7 Gerrit-Change-Number: 13925 Gerrit-PatchSet: 1 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 25 Jul 2019 23:20:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13900 ) Change subject: IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes .. Patch Set 5: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/412/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13900 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I46c1941269feff18306863f784aa36f5037da1db Gerrit-Change-Number: 13900 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 25 Jul 2019 23:18:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes
Hello Lars Volker, Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13900 to look at the new patch set (#5). Change subject: IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes .. IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes - The following options were documented: - parquet_read_page_index - parquet_write_page_index - parquet_page_row_count_limit Change-Id: I46c1941269feff18306863f784aa36f5037da1db --- M docs/impala.ditamap M docs/topics/impala_parquet.xml A docs/topics/impala_parquet_page_row_count_limit.xml A docs/topics/impala_parquet_read_page_index.xml A docs/topics/impala_parquet_write_page_index.xml 5 files changed, 255 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/00/13900/5 -- To view, visit http://gerrit.cloudera.org:8080/13900 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I46c1941269feff18306863f784aa36f5037da1db Gerrit-Change-Number: 13900 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8339: Add local executor blacklist to coordinators
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13868 ) Change subject: IMPALA-8339: Add local executor blacklist to coordinators .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4007/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13868 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iacb6e73b84042c33cd475b82470a975d04ee9b74 Gerrit-Change-Number: 13868 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 25 Jul 2019 23:16:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13900 ) Change subject: IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes .. Patch Set 4: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/411/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13900 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I46c1941269feff18306863f784aa36f5037da1db Gerrit-Change-Number: 13900 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 25 Jul 2019 23:14:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8339: Add local executor blacklist to coordinators
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13868 ) Change subject: IMPALA-8339: Add local executor blacklist to coordinators .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4009/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13868 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iacb6e73b84042c33cd475b82470a975d04ee9b74 Gerrit-Change-Number: 13868 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 25 Jul 2019 23:10:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7486: Add specialized estimation scheme for dedicated coordinators
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13740 ) Change subject: IMPALA-7486: Add specialized estimation scheme for dedicated coordinators .. Patch Set 13: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4006/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13740 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2b94e7293b91dec8a18491079c34923eadd94b21 Gerrit-Change-Number: 13740 Gerrit-PatchSet: 13 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 23:00:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8549: Add support for scanning DEFLATE text files
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13857 ) Change subject: IMPALA-8549: Add support for scanning DEFLATE text files .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4005/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13857 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I45e41ab5a12637d396fef0812a09d71fa839b27a Gerrit-Change-Number: 13857 Gerrit-PatchSet: 5 Gerrit-Owner: Ethan Xue Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Ethan Xue Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Thu, 25 Jul 2019 22:56:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes
Hello Lars Volker, Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13900 to look at the new patch set (#4). Change subject: IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes .. IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes - The following options were documented: - parquet_read_page_index - parquet_write_page_index - parquet_page_row_count_limit Change-Id: I46c1941269feff18306863f784aa36f5037da1db --- M docs/impala.ditamap A docs/topics/impala_parquet_page_row_count_limit.xml A docs/topics/impala_parquet_read_page_index.xml A docs/topics/impala_parquet_write_page_index.xml 4 files changed, 252 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/00/13900/4 -- To view, visit http://gerrit.cloudera.org:8080/13900 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I46c1941269feff18306863f784aa36f5037da1db Gerrit-Change-Number: 13900 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13900 ) Change subject: IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes .. Patch Set 4: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/411/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13900 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I46c1941269feff18306863f784aa36f5037da1db Gerrit-Change-Number: 13900 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 25 Jul 2019 22:49:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13900 ) Change subject: IMPALA-7991 IMPALA-8575: [DOCS] Document the query options for Parquet page indexes .. Patch Set 2: (4 comments) http://gerrit.cloudera.org:8080/#/c/13900/3/docs/topics/impala_parquet_write_page_index.xml File docs/topics/impala_parquet_write_page_index.xml: http://gerrit.cloudera.org:8080/#/c/13900/3/docs/topics/impala_parquet_write_page_index.xml@48 PS3, Line 48: : This option enables the same optimization as the PARQUET_READ_STATISTICS : for writing at the finer grained page level. : > This option is only about writing, so this paragraph doesn't belong here. Done http://gerrit.cloudera.org:8080/#/c/13900/3/docs/topics/impala_parquet_write_page_index.xml@62 PS3, Line 62: : For simple predicates of the forms:or : , where is LT, : LE, GE, GT, and EQ : > This isn't really belong to the write path. Done http://gerrit.cloudera.org:8080/#/c/13900/3/docs/topics/impala_parquet_write_page_index.xml@73 PS3, Line 73: Read statistics from Parquet pages and use : them in query processing. > Write the Parquet page index when creating Parquet files. Done http://gerrit.cloudera.org:8080/#/c/13900/3/docs/topics/impala_parquet_write_page_index.xml@78 PS3, Line 78: Do not use Parquet write page statistics. > Do not write the Parquet page index when creating Parquet files. Done -- To view, visit http://gerrit.cloudera.org:8080/13900 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I46c1941269feff18306863f784aa36f5037da1db Gerrit-Change-Number: 13900 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 25 Jul 2019 22:49:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8339: Add local executor blacklist to coordinators
Hello Michael Ho, Lars Volker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13868 to look at the new patch set (#5). Change subject: IMPALA-8339: Add local executor blacklist to coordinators .. IMPALA-8339: Add local executor blacklist to coordinators This patch adds the concept of a blacklist of executors to the coordinator, which removes executors from consideration for query scheduling. Blacklisting decisions are local to a given coordinator and are not included in statestore updates. The intention is to allow coordinators to be more aggressive about deciding that an exeutor is unhealthy or unavailable, to minimize failed queries in environments where cluster membership may be more variable, rather than having to wait on the statestore heartbeat mechanism to decide that the executor is down. For the first patch, executors will only be blacklisted if the KRPC status for Exec() is an error. Followup work will add blacklisting of executors in more complex scenarios, eg. if an executor appears to be a straggler. Testing: - Added a case to the cluster mgr BE unit test that uses blacklisting. - Added e2e test cases for killing and restarting an impalad. TODO - Add an e2e test case where an impalad becomes briefly unreachable. - Manual/stress tests on a real cluster. Change-Id: Iacb6e73b84042c33cd475b82470a975d04ee9b74 --- M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/coordinator.cc M be/src/scheduling/CMakeLists.txt M be/src/scheduling/admission-controller.cc M be/src/scheduling/cluster-membership-mgr-test.cc M be/src/scheduling/cluster-membership-mgr.cc M be/src/scheduling/cluster-membership-mgr.h A be/src/scheduling/executor-blacklist.cc A be/src/scheduling/executor-blacklist.h M be/src/scheduling/query-schedule.h M be/src/scheduling/scheduler.cc M be/src/statestore/statestore.cc M be/src/statestore/statestore.h A tests/custom_cluster/test_blacklist.py 15 files changed, 733 insertions(+), 35 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/13868/5 -- To view, visit http://gerrit.cloudera.org:8080/13868 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iacb6e73b84042c33cd475b82470a975d04ee9b74 Gerrit-Change-Number: 13868 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-8339: Add local executor blacklist to coordinators
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13868 ) Change subject: IMPALA-8339: Add local executor blacklist to coordinators .. Patch Set 5: (5 comments) http://gerrit.cloudera.org:8080/#/c/13868/4/be/src/scheduling/executor-blacklist.cc File be/src/scheduling/executor-blacklist.cc: http://gerrit.cloudera.org:8080/#/c/13868/4/be/src/scheduling/executor-blacklist.cc@196 PS4, Line 196: bool ExecutorBlacklist::eqBePort( > line too long (92 > 90) Done http://gerrit.cloudera.org:8080/#/c/13868/4/tests/custom_cluster/test_blacklist.py File tests/custom_cluster/test_blacklist.py: http://gerrit.cloudera.org:8080/#/c/13868/4/tests/custom_cluster/test_blacklist.py@45 PS4, Line 45: > flake8: E501 line too long (107 > 90 characters) Done http://gerrit.cloudera.org:8080/#/c/13868/4/tests/custom_cluster/test_blacklist.py@70 PS4, Line 70: > flake8: E501 line too long (107 > 90 characters) Done http://gerrit.cloudera.org:8080/#/c/13868/4/tests/custom_cluster/test_blacklist.py@79 PS4, Line 79: > flake8: E501 line too long (107 > 90 characters) Done http://gerrit.cloudera.org:8080/#/c/13868/4/tests/custom_cluster/test_blacklist.py@110 PS4, Line 110: > flake8: E501 line too long (107 > 90 characters) Done -- To view, visit http://gerrit.cloudera.org:8080/13868 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iacb6e73b84042c33cd475b82470a975d04ee9b74 Gerrit-Change-Number: 13868 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 25 Jul 2019 22:44:59 + Gerrit-HasComments: Yes
[Impala-ASF-CR] This commit adds support to display the metric last-synced-event-id as Catalogd/metrics#evnts page whereas previously it was displayed only on the catalod/events page. Added code to th
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13925 ) Change subject: This commit adds support to display the metric last-synced-event-id as Catalogd/metrics#evnts page whereas previously it was displayed only on the catalod/events page. Added code to the MetaStoreEventsProcessorTest class under testEventProcessorMetrics() .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/13925/1/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java File fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java: http://gerrit.cloudera.org:8080/#/c/13925/1/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java@1447 PS1, Line 1447: createTable(TEST_DB_NAME, "tbl_shouldtestEventProcessorMetr_skipped", tblParams, true); line too long (91 > 90) -- To view, visit http://gerrit.cloudera.org:8080/13925 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75b966f2b4eaafbcf7d80358f53501bb7ade67e7 Gerrit-Change-Number: 13925 Gerrit-PatchSet: 1 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 25 Jul 2019 22:39:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] This commit adds support to display the metric last-synced-event-id as Catalogd/metrics#evnts page whereas previously it was displayed only on the catalod/events page. Added code to th
sharanitha.har...@cloudera.com has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13925 Change subject: This commit adds support to display the metric last-synced-event-id as Catalogd/metrics#evnts page whereas previously it was displayed only on the catalod/events page. Added code to the MetaStoreEventsProcessorTest class under testEventProcessorMetrics() .. This commit adds support to display the metric last-synced-event-id as Catalogd/metrics#evnts page whereas previously it was displayed only on the catalod/events page. Added code to the MetaStoreEventsProcessorTest class under testEventProcessorMetrics() and testEventProcessorWhenNotActive() to check 1)If the metric updated as it should when new events were processed. 2)If the metric was not set when the event processor was not active. Change-Id: I75b966f2b4eaafbcf7d80358f53501bb7ade67e7 --- M be/src/util/event-metrics.cc M be/src/util/event-metrics.h M common/thrift/JniCatalog.thrift M common/thrift/metrics.json M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 6 files changed, 37 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/13925/1 -- To view, visit http://gerrit.cloudera.org:8080/13925 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I75b966f2b4eaafbcf7d80358f53501bb7ade67e7 Gerrit-Change-Number: 13925 Gerrit-PatchSet: 1 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8339: Add local executor blacklist to coordinators
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/13868 )
Change subject: IMPALA-8339: Add local executor blacklist to coordinators
..
Patch Set 4:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/13868/4/be/src/scheduling/executor-blacklist.cc
File be/src/scheduling/executor-blacklist.cc:
http://gerrit.cloudera.org:8080/#/c/13868/4/be/src/scheduling/executor-blacklist.cc@196
PS4, Line 196: bool ExecutorBlacklist::eqBePort(const TBackendDescriptor&
be_desc, const Entry& existing) {
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/13868/4/tests/custom_cluster/test_blacklist.py
File tests/custom_cluster/test_blacklist.py:
http://gerrit.cloudera.org:8080/#/c/13868/4/tests/custom_cluster/test_blacklist.py@45
PS4, Line 45: t
flake8: E501 line too long (107 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/13868/4/tests/custom_cluster/test_blacklist.py@70
PS4, Line 70: t
flake8: E501 line too long (107 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/13868/4/tests/custom_cluster/test_blacklist.py@79
PS4, Line 79: t
flake8: E501 line too long (107 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/13868/4/tests/custom_cluster/test_blacklist.py@110
PS4, Line 110: t
flake8: E501 line too long (107 > 90 characters)
--
To view, visit http://gerrit.cloudera.org:8080/13868
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iacb6e73b84042c33cd475b82470a975d04ee9b74
Gerrit-Change-Number: 13868
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Tauber-Marshall
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Thomas Tauber-Marshall
Gerrit-Comment-Date: Thu, 25 Jul 2019 22:35:58 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8339: Add local executor blacklist to coordinators
Hello Michael Ho, Lars Volker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13868 to look at the new patch set (#4). Change subject: IMPALA-8339: Add local executor blacklist to coordinators .. IMPALA-8339: Add local executor blacklist to coordinators This patch adds the concept of a blacklist of executors to the coordinator, which removes executors from consideration for query scheduling. Blacklisting decisions are local to a given coordinator and are not included in statestore updates. The intention is to allow coordinators to be more aggressive about deciding that an exeutor is unhealthy or unavailable, to minimize failed queries in environments where cluster membership may be more variable, rather than having to wait on the statestore heartbeat mechanism to decide that the executor is down. For the first patch, executors will only be blacklisted if the KRPC status for Exec() is an error. Followup work will add blacklisting of executors in more complex scenarios, eg. if an executor appears to be a straggler. Testing: - Added a case to the cluster mgr BE unit test that uses blacklisting. - Added e2e test cases for killing and restarting an impalad. TODO - Add an e2e test case where an impalad becomes briefly unreachable. - Manual/stress tests on a real cluster. Change-Id: Iacb6e73b84042c33cd475b82470a975d04ee9b74 --- M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/coordinator.cc M be/src/scheduling/CMakeLists.txt M be/src/scheduling/admission-controller.cc M be/src/scheduling/cluster-membership-mgr-test.cc M be/src/scheduling/cluster-membership-mgr.cc M be/src/scheduling/cluster-membership-mgr.h A be/src/scheduling/executor-blacklist.cc A be/src/scheduling/executor-blacklist.h M be/src/scheduling/query-schedule.h M be/src/scheduling/scheduler.cc M be/src/statestore/statestore.cc M be/src/statestore/statestore.h A tests/custom_cluster/test_blacklist.py 15 files changed, 728 insertions(+), 35 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/13868/4 -- To view, visit http://gerrit.cloudera.org:8080/13868 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iacb6e73b84042c33cd475b82470a975d04ee9b74 Gerrit-Change-Number: 13868 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-8339: Add local executor blacklist to coordinators
Thomas Tauber-Marshall has posted comments on this change. (
http://gerrit.cloudera.org:8080/13868 )
Change subject: IMPALA-8339: Add local executor blacklist to coordinators
..
Patch Set 4:
(19 comments)
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/admission-controller.cc
File be/src/scheduling/admission-controller.cc:
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/admission-controller.cc@737
PS3, Line 737: string
> nit: std:: not needed
Done
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/cluster-membership-mgr.cc
File be/src/scheduling/cluster-membership-mgr.cc:
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/cluster-membership-mgr.cc@272
PS3, Line 272: already be on the blacklist or probation.
> This may be a case worth double checking. If a node is restarted and re-reg
Yes, this DCHECK is correct - if a node is restarted, when it re-registers with
the statestore the old subscriber is always unregistered first, see:
https://github.com/apache/impala/blob/master/be/src/statestore/statestore.cc#L623
This is also required by the loop above, which will fail if AddExecutor() is
called with an existing executor.
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/cluster-membership-mgr.cc@273
PS3, Line 273: DCHECK
> DCHECK_EQ
Done
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/cluster-membership-mgr.cc@350
PS3, Line 350: exists
> Given the for-loop above, can an executor belong to more than one executor
Done
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/cluster-membership-mgr.cc@355
PS3, Line 355: exists = true;
> This is repeated at multiple places in this function. May make sense to fac
Done
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.h
File be/src/scheduling/executor-blacklist.h:
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.h@32
PS3, Line 32:
> FindAndRemove()
Done
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.h@38
PS3, Line 38: When a blacklisted executor has passed this timeout
: /// and Maintenance() is called, the executor is put on
'probation'
> May help to also document what if an executor on blacklist is removed durin
Done
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.h@42
PS3, Line 42: There
> nit: typo
Done
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.h@79
PS3, Line 79:
> May be clearer to call it "probation_list" or something so it can relates t
Done
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.h@124
PS3, Line 124:
> typo
Done
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.cc
File be/src/scheduling/executor-blacklist.cc:
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.cc@29
PS3, Line 29: 0 disables blacklisting
> This seems a bit weird. Setting it to 0 means the blacklist timeout == stat
My problem with setting it to 120 is that it makes it possible for users to set
it to < 100, which doesn't seem like it would ever be the right thing to do
(eg. for executors that go down, it would make it very likely they would be
incorrectly unblacklisted).
Maybe it would be better to just add another flag, blacklisting_enabled or
similar. I was trying to minimize the number of flags, but I guess it doesn't
really matter. I don't think we'll be formally documenting these or expecting
customers to commonly set them, they're just intended as emergency safety
valves.
Or we could do something like disable blacklisting if this is negative.
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.cc@39
PS3, Line 39: ExecutorBlacklist::Blacklist(const TBackendDescriptor& be_desc) {
> Should this check if blacklisting is disabled at the beginning ?
I put the check in ClusterMembershipMgr::Blacklist to avoid copying the
Snapshot if blacklisting is disabled.
I'll at least add a DCHECK here.
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.cc@54
PS3, Line 54: nicMillis()
> Why not MonotonicMillis() ? This field shouldn't need to be shared across m
Done
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.cc@83
PS3, Line 83: if (remove_it == be_descs.end()) {
: // Executor wasn't on the blacklist.
: return NOT_BLACKLISTED;
: }
: St
> This seems to be used at more than one functions. May be worth factoring it
Done
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-blacklist.cc@99
PS3, Line 99:for (auto entry_it : e
> Is remove_it still valid after calling erase above ?
Done
http://gerrit.cloudera.org:8080/#/c/13868/3/be/src/scheduling/executor-bl
[Impala-ASF-CR] IMPALA-7486: Add specialized estimation scheme for dedicated coordinators
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/13740 ) Change subject: IMPALA-7486: Add specialized estimation scheme for dedicated coordinators .. Patch Set 13: (1 comment) http://gerrit.cloudera.org:8080/#/c/13740/1/fe/src/main/java/org/apache/impala/planner/Planner.java File fe/src/main/java/org/apache/impala/planner/Planner.java: http://gerrit.cloudera.org:8080/#/c/13740/1/fe/src/main/java/org/apache/impala/planner/Planner.java@295 PS1, Line 295: Preconditions.checkState(request.isSetPer_host_mem_estimate()); > I think we want to have at least some planner tests so that we have direct Done -- To view, visit http://gerrit.cloudera.org:8080/13740 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2b94e7293b91dec8a18491079c34923eadd94b21 Gerrit-Change-Number: 13740 Gerrit-PatchSet: 13 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 22:20:35 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7486: Add specialized estimation scheme for dedicated coordinators
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13740 ) Change subject: IMPALA-7486: Add specialized estimation scheme for dedicated coordinators .. Patch Set 13: (1 comment) http://gerrit.cloudera.org:8080/#/c/13740/13/be/src/util/backend-gflag-util.cc File be/src/util/backend-gflag-util.cc: http://gerrit.cloudera.org:8080/#/c/13740/13/be/src/util/backend-gflag-util.cc@158 PS13, Line 158: cfg.__set_use_dedicated_coordinator_estimates(FLAGS_use_dedicated_coordinator_estimates); line too long (91 > 90) -- To view, visit http://gerrit.cloudera.org:8080/13740 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2b94e7293b91dec8a18491079c34923eadd94b21 Gerrit-Change-Number: 13740 Gerrit-PatchSet: 13 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 22:20:29 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7486: Add specialized estimation scheme for dedicated coordinators
Hello Andrew Sherman, Lars Volker, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13740 to look at the new patch set (#13). Change subject: IMPALA-7486: Add specialized estimation scheme for dedicated coordinators .. IMPALA-7486: Add specialized estimation scheme for dedicated coordinators This patch computes two memory estimates in the frontend: an estimate for any host that is an executor (including a combined coordinator and executor) and an estimate for a dedicated coordinator. This is computed regardless of whether it is a dedicated coordinator or not. Admission control then, in the case when the coordinator is dedicated, uses the coordinator memory estimate for the coordinator node and the executor memory estimate for all other nodes. Other highlights: - if MEM_LIMIT query option is set, it is applied to all backends - the min_query_mem_limit pool config is not enforced on the dedicated coordinator estimates unless MEM_LIMIT query option is set - the lower cap on estimates and the admission checks based on the min mem limit required for reservation are applied separately on coordinator's and executors' mem requirements. - Added a hidden startup option 'use_dedicated_coordinator_estimates' which if set to false, reverts to previous estimation behavior. Testing: - Added unit test for admission/rejection in dedicated coordinator clusters. - Added end to end tests. Change-Id: I2b94e7293b91dec8a18491079c34923eadd94b21 --- M be/src/runtime/coordinator.cc M be/src/scheduling/admission-controller-test.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M be/src/scheduling/query-schedule.cc M be/src/scheduling/query-schedule.h M be/src/scheduling/scheduler.cc M be/src/service/client-request-state.cc M be/src/service/impala-http-handler.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M common/thrift/Frontend.thrift M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/Frontend.java A testdata/workloads/functional-query/queries/QueryTest/dedicated-coord-mem-estimates.test M tests/custom_cluster/test_admission_controller.py M www/admission_controller.tmpl 19 files changed, 776 insertions(+), 108 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/13740/13 -- To view, visit http://gerrit.cloudera.org:8080/13740 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2b94e7293b91dec8a18491079c34923eadd94b21 Gerrit-Change-Number: 13740 Gerrit-PatchSet: 13 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8549: Add support for scanning DEFLATE text files
Hello Abhishek Rawat, Sahil Takiar, Bikramjeet Vig, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13857 to look at the new patch set (#5). Change subject: IMPALA-8549: Add support for scanning DEFLATE text files .. IMPALA-8549: Add support for scanning DEFLATE text files Hadoop tools such as Hive and MapReduce support reading and writing text files compressed using the deflate algorithm. In Hadoop, the zlib library (an implementation of the DEFLATE algorithm) is used to compress text files into .DEFLATE files, which are not in the raw deflate format but rather the zlib format (the zlib library supports three flavors of deflate, and Hadoop is using the flavor that compresses data into deflate with zlib wrappings rather than the raw deflate format) This patch adds support to Impala for scanning .DEFLATE files of tables stored as text. To avoid confusion, it should be noted that although these files have a compression type of DEFLATE in Impala, they should be treated as if their compression type is ZLIB. Testing: There is a pre-existing unit test that validates compressing/decompressing data with compression type DEFLATE. Also, modified existing end-to-end testing that simulates querying files of various formats and compression types. All core tests pass. Change-Id: I45e41ab5a12637d396fef0812a09d71fa839b27a --- M be/src/exec/hdfs-text-scanner.cc M be/src/exec/hdfs-text-scanner.h M be/src/util/codec.cc M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/functional-query_exhaustive.csv M tests/query_test/test_compressed_formats.py 6 files changed, 20 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/13857/5 -- To view, visit http://gerrit.cloudera.org:8080/13857 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I45e41ab5a12637d396fef0812a09d71fa839b27a Gerrit-Change-Number: 13857 Gerrit-PatchSet: 5 Gerrit-Owner: Ethan Xue Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Ethan Xue Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar
[Impala-ASF-CR] IMPALA-8606: Don't load table meta for GET TABLES in local catalog mode
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13874 ) Change subject: IMPALA-8606: Don't load table meta for GET_TABLES in local catalog mode .. IMPALA-8606: Don't load table meta for GET_TABLES in local catalog mode In LocalCatalog implementation, LocalDb#getTable will always return a completely loaded table containing all the meta of columns, partitions, files, etc. It's time consuming if we implement the GET_TABLES HiveServer2 operation based on this interface, since GET_TABLES only requires table names, table types and table comments, while this interface will trigger catalogd to fully load the table meta. It becomes worse when we do this for all the tables. This patch introduces a new interface, getTableIfCached, to return a LocalIncompleteTable object if the corresponding table is unloaded, which requires no round trips to the catalogd. It's used to boost the GET_TABLES performance in LocalCatalog mode. Tests - Testing in a HMS with 100 dbs and 3000 tables, without this patch it takes ~2mins in GET_TABLES for all tables on a cold started cluster. With this patch, the time reduces to ~1s. - Testing in HUE-4.4.0 with a db with 3000 tables, the performance is the same as using legacy catalog implementation. Change-Id: Ia8bbab7efdf8e629abe09d89ae3bd770e3feaccb Reviewed-on: http://gerrit.cloudera.org:8080/13874 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/catalog/Catalog.java M fe/src/main/java/org/apache/impala/catalog/Db.java M fe/src/main/java/org/apache/impala/catalog/FeCatalog.java M fe/src/main/java/org/apache/impala/catalog/FeDb.java M fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java A fe/src/main/java/org/apache/impala/catalog/local/LocalIncompleteTable.java M fe/src/main/java/org/apache/impala/service/MetadataOp.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java M fe/src/test/java/org/apache/impala/service/JdbcTest.java M tests/hs2/test_hs2.py 11 files changed, 156 insertions(+), 15 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13874 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia8bbab7efdf8e629abe09d89ae3bd770e3feaccb Gerrit-Change-Number: 13874 Gerrit-PatchSet: 14 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-8606: Don't load table meta for GET TABLES in local catalog mode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13874 ) Change subject: IMPALA-8606: Don't load table meta for GET_TABLES in local catalog mode .. Patch Set 13: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13874 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia8bbab7efdf8e629abe09d89ae3bd770e3feaccb Gerrit-Change-Number: 13874 Gerrit-PatchSet: 13 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 25 Jul 2019 21:59:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8489: Partitions created by RECOVER PARTITIONS fail to create insert events with IllegalStateException.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13860 ) Change subject: IMPALA-8489: Partitions created by RECOVER PARTITIONS fail to create insert events with IllegalStateException. .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4004/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13860 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idef7f6aadff2868047c861ebfcc05d65f080eab9 Gerrit-Change-Number: 13860 Gerrit-PatchSet: 5 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 25 Jul 2019 21:52:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8616: [DOCS] Document --disconnected session timeout flag for HS2
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13903 ) Change subject: IMPALA-8616: [DOCS] Document --disconnected_session_timeout flag for HS2 .. IMPALA-8616: [DOCS] Document --disconnected_session_timeout flag for HS2 Change-Id: I2c178116ec9cb8abf619c9fdcfb2697bcfb1de5c Reviewed-on: http://gerrit.cloudera.org:8080/13903 Tested-by: Impala Public Jenkins Reviewed-by: Thomas Tauber-Marshall --- M docs/topics/impala_client.xml 1 file changed, 47 insertions(+), 22 deletions(-) Approvals: Impala Public Jenkins: Verified Thomas Tauber-Marshall: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/13903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2c178116ec9cb8abf619c9fdcfb2697bcfb1de5c Gerrit-Change-Number: 13903 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-8795 : [WIP] Enable event polling by default in tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13922 ) Change subject: IMPALA-8795 : [WIP] Enable event polling by default in tests .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4003/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13922 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7279349d4900e24fbcf558f290549496844ce138 Gerrit-Change-Number: 13922 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 25 Jul 2019 21:35:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8616: [DOCS] Document --disconnected session timeout flag for HS2
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13903 ) Change subject: IMPALA-8616: [DOCS] Document --disconnected_session_timeout flag for HS2 .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2c178116ec9cb8abf619c9fdcfb2697bcfb1de5c Gerrit-Change-Number: 13903 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 25 Jul 2019 21:26:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8616: [DOCS] Document --disconnected session timeout flag for HS2
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13903 ) Change subject: IMPALA-8616: [DOCS] Document --disconnected_session_timeout flag for HS2 .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/410/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2c178116ec9cb8abf619c9fdcfb2697bcfb1de5c Gerrit-Change-Number: 13903 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 25 Jul 2019 21:24:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8489: Partitions created by RECOVER PARTITIONS fail to create insert events with IllegalStateException.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13860 ) Change subject: IMPALA-8489: Partitions created by RECOVER PARTITIONS fail to create insert events with IllegalStateException. .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4659/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13860 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idef7f6aadff2868047c861ebfcc05d65f080eab9 Gerrit-Change-Number: 13860 Gerrit-PatchSet: 5 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 25 Jul 2019 21:14:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8616: [DOCS] Document --disconnected session timeout flag for HS2
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13903 ) Change subject: IMPALA-8616: [DOCS] Document --disconnected_session_timeout flag for HS2 .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/410/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2c178116ec9cb8abf619c9fdcfb2697bcfb1de5c Gerrit-Change-Number: 13903 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 25 Jul 2019 21:11:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8616: [DOCS] Document --disconnected session timeout flag for HS2
Hello Thomas Tauber-Marshall, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13903 to look at the new patch set (#2). Change subject: IMPALA-8616: [DOCS] Document --disconnected_session_timeout flag for HS2 .. IMPALA-8616: [DOCS] Document --disconnected_session_timeout flag for HS2 Change-Id: I2c178116ec9cb8abf619c9fdcfb2697bcfb1de5c --- M docs/topics/impala_client.xml 1 file changed, 47 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/13903/2 -- To view, visit http://gerrit.cloudera.org:8080/13903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2c178116ec9cb8abf619c9fdcfb2697bcfb1de5c Gerrit-Change-Number: 13903 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-8616: [DOCS] Document --disconnected session timeout flag for HS2
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13903 ) Change subject: IMPALA-8616: [DOCS] Document --disconnected_session_timeout flag for HS2 .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/13903/1/docs/topics/impala_client.xml File docs/topics/impala_client.xml: http://gerrit.cloudera.org:8080/#/c/13903/1/docs/topics/impala_client.xml@132 PS1, Line 132: Impala closes any session started over that connection > There's always exactly one session per beeswax connection, created when the Done -- To view, visit http://gerrit.cloudera.org:8080/13903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2c178116ec9cb8abf619c9fdcfb2697bcfb1de5c Gerrit-Change-Number: 13903 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 25 Jul 2019 21:09:55 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8489: Partitions created by RECOVER PARTITIONS fail to create insert events with IllegalStateException.
Anurag Mantripragada has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/13860 ) Change subject: IMPALA-8489: Partitions created by RECOVER PARTITIONS fail to create insert events with IllegalStateException. .. IMPALA-8489: Partitions created by RECOVER PARTITIONS fail to create insert events with IllegalStateException. createInsertEvents() uses partition ids to keep track of the files added to the partitions by the insert by finding the delta of files in a partition before and after load() call. However, if partitions are marked dirty (for eg.: partitions created by RECOVER PARTITIONS), load() will drop and re-create them which will change the partition ids. createInsertEvents() then cannot find these parittions and fails with exception. In this patch, partitions are tracked by partition names instead of partition ids so drop + reload will not affect the logic. Testing: 1. Ran TestRecoverPartitions.test_post_invalidate() which was failing. 2. Ran MetastoreEventProcessorTest FE tests. Change-Id: Idef7f6aadff2868047c861ebfcc05d65f080eab9 --- M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java 2 files changed, 36 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/13860/5 -- To view, visit http://gerrit.cloudera.org:8080/13860 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Idef7f6aadff2868047c861ebfcc05d65f080eab9 Gerrit-Change-Number: 13860 Gerrit-PatchSet: 5 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8795 : [WIP] Enable event polling by default in tests
Vihang Karajgaonkar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13922 Change subject: IMPALA-8795 : [WIP] Enable event polling by default in tests .. IMPALA-8795 : [WIP] Enable event polling by default in tests Most of the fixes for event processing are committed. This change enables the feature by default for all the tests so that eventually we can turn it on out of the box. Testing done: 1. Ran all the tests with the patch and confirmed that there are no test failures 2. [WIP] Running all the tests with CDP build. Change-Id: I7279349d4900e24fbcf558f290549496844ce138 --- M docker/catalogd/Dockerfile 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/13922/1 -- To view, visit http://gerrit.cloudera.org:8080/13922 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I7279349d4900e24fbcf558f290549496844ce138 Gerrit-Change-Number: 13922 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8434: retain tables and functions in altering database
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13904 ) Change subject: IMPALA-8434: retain tables and functions in altering database .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4658/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13904 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3dc9857fd2733e20cf10fbe17bb1a4670d7d015 Gerrit-Change-Number: 13904 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 25 Jul 2019 20:53:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8434: retain tables and functions in altering database
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/13904 ) Change subject: IMPALA-8434: retain tables and functions in altering database .. Patch Set 4: Code-Review+2 Thanks for making the suggested changes. Looks good to me. -- To view, visit http://gerrit.cloudera.org:8080/13904 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3dc9857fd2733e20cf10fbe17bb1a4670d7d015 Gerrit-Change-Number: 13904 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 25 Jul 2019 20:18:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8636: Implement INSERT for insert-only ACID tables
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/13559 )
Change subject: IMPALA-8636: Implement INSERT for insert-only ACID tables
..
Patch Set 15: Code-Review+2
(3 comments)
Only minor comments.
http://gerrit.cloudera.org:8080/#/c/13559/15//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/13559/15//COMMIT_MSG@17
PS15, Line 17: The write id is also set the for the HDFS table sinks. The sinks
write the
: files at their final destination which is an ACID base or delta
directory.
nit: please wrap at 72
http://gerrit.cloudera.org:8080/#/c/13559/15/be/src/service/client-request-state.cc
File be/src/service/client-request-state.cc:
http://gerrit.cloudera.org:8080/#/c/13559/15/be/src/service/client-request-state.cc@1116
PS15, Line 1116: Status debug_action_status = DebugAction(
: query_options(), "CLIENT_REQUEST_UPDATE_CATALOG");
: Status status = !debug_action_status.ok() ?
debug_action_status :
: client.DoRpc(
: &CatalogServiceClientWrapper::UpdateCatalog,
catalog_update, &resp);
nit: could be bit simpler IMO as
Status status = DebugAction(query_options(), "CLIENT_REQUEST_UPDATE_CATALOG");
if (status.ok()) {
status = client.DoRpc(
&CatalogServiceClientWrapper::UpdateCatalog, catalog_update,
&resp);
}
http://gerrit.cloudera.org:8080/#/c/13559/15/testdata/workloads/functional-query/queries/QueryTest/acid-insert-fail.test
File
testdata/workloads/functional-query/queries/QueryTest/acid-insert-fail.test:
http://gerrit.cloudera.org:8080/#/c/13559/15/testdata/workloads/functional-query/queries/QueryTest/acid-insert-fail.test@1
PS15, Line 1:
It would be also useful to test (dynamic) partitioned inserts with different
failure modes. I'am ok with doing this in a different patch.
--
To view, visit http://gerrit.cloudera.org:8080/13559
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id6c36fa6902676f06b4e38730f737becfc7c06ad
Gerrit-Change-Number: 13559
Gerrit-PatchSet: 15
Gerrit-Owner: Zoltan Borok-Nagy
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Thu, 25 Jul 2019 20:10:45 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5149: Provide query profile in JSON format
Michael Ho has posted comments on this change. (
http://gerrit.cloudera.org:8080/13801 )
Change subject: IMPALA-5149: Provide query profile in JSON format
..
Patch Set 9:
(23 comments)
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/service/impala-hs2-server.cc
File be/src/service/impala-hs2-server.cc:
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/service/impala-hs2-server.cc@988
PS9, Line 988: }else {
nit: missing blank space after }
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/service/impala-server.cc@101
PS9, Line 101: #include
: #include
: #include
: #include
Why not group these together with other rapidjson #include above ? There seems
to be some duplicates.
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/service/impala-server.cc@783
PS9, Line 783: DCHECK(format == TRuntimeProfileFormat::STRING);
DCHECK_EQ(format, TRuntimeProfileFormat::STRING);
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/service/impala-server.cc@818
PS9, Line 818: DCHECK(format == TRuntimeProfileFormat::STRING);
DCHECK_EQ(format, TRuntimeProfileFormat::STRING);
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/util/runtime-profile-counters.h
File be/src/util/runtime-profile-counters.h:
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/util/runtime-profile-counters.h@126
PS9, Line 126: void ToJson(rapidjson::Document& document, rapidjson::Value*
val) const {
void ToJson(rapidjson::Document& document, rapidjson::Value* val) const
override {
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/util/runtime-profile-counters.h@131
PS9, Line 131: val->AddMember("current_value", current_value(),
document.GetAllocator());
Actually, do you need to output this in json ? It seems sufficient to just call
Counter::ToJson() which should add the higher water mark stored in "value_" ?
"current_value_" seems to be internal state used for supporting TryAdd() only.
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/util/runtime-profile-counters.h@165
PS9, Line 165: void ToJson(rapidjson::Document& document, rapidjson::Value*
val) const {
: Counter::ToJson(document, val);
: rapidjson::Value::MemberIterator type_itr =
val->FindMember("kind");
: DCHECK(type_itr != val->MemberEnd());
: type_itr->value.SetString("DerivedCounter");
: val->AddMember("counter_fn_val", counter_fn_(),
document.GetAllocator());
: }
If you take the suggestion in Counter::ToJson(), you probably don't need this
function.
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/util/runtime-profile-counters.h@219
PS9, Line 219: rapidjson::Value::MemberIterator type_itr =
val->FindMember("kind");
: DCHECK(type_itr != val->MemberEnd());
: type_itr->value.SetString("AveragedCounter");
See comments in Counter::ToJson() on why we may not need this.
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/util/runtime-profile-counters.h@222
PS9, Line 222: if (unit_ == TUnit::type::DOUBLE_VALUE){
: val->AddMember("current_double_sum", current_double_sum_,
document.GetAllocator());
: } else {
: val->AddMember("current_int_sum", current_int_sum_,
document.GetAllocator());
: }
Same question as above. Do you need to store the internal states (i.e. sum) in
the json output object ? Isn't "average" the only state we need ?
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/util/runtime-profile-counters.h@290
PS9, Line 290: void ToJson(rapidjson::Document& document, rapidjson::Value*
val) const {
virtual override
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/util/runtime-profile-counters.h@292
PS9, Line 292: val->RemoveMember("kind");
Why is "kind" removed ? May be worth adding a comment here.
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/util/runtime-profile-counters.h@406
PS9, Line 406: void ToJson(rapidjson::Document& document, rapidjson::Value*
value);
Please add a comment about the output format.
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/util/runtime-profile-counters.h@452
PS9, Line 452: void ToJson(rapidjson::Document& document, rapidjson::Value*
val);
It'd be nice to add a comment about the output format.
http://gerrit.cloudera.org:8080/#/c/13801/9/be/src/util/runtime-profile-counters.h@583
PS9, Line 583: void ToJson(rapidjson::Document& document, rapidjson::Value*
val) const {
: Counter::ToJson(document, val);
: rapidjson::Value::MemberIterator type_itr =
val->FindMember("kind");
: type_itr->value.SetString("ConcurrentTimerCounter");
: val->AddMember("concurrent_stop_watch_total_time",
:
[Impala-ASF-CR] IMPALA-8585: Skip most ACID EE tests in non-HDFS builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13921 ) Change subject: IMPALA-8585: Skip most ACID EE tests in non-HDFS builds .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4002/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13921 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5f67654805b770d86090b0d1015e144147d73f40 Gerrit-Change-Number: 13921 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 25 Jul 2019 19:56:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8794: Skipping testPiggyback* in Hive 3 builds
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13920 ) Change subject: IMPALA-8794: Skipping testPiggyback* in Hive 3 builds .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4001/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13920 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ice8f52caa20021ad9e36cbebe62a2bb247a4273c Gerrit-Change-Number: 13920 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 25 Jul 2019 19:46:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8434: retain tables and functions in altering database
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13904 ) Change subject: IMPALA-8434: retain tables and functions in altering database .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4000/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13904 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3dc9857fd2733e20cf10fbe17bb1a4670d7d015 Gerrit-Change-Number: 13904 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 25 Jul 2019 19:32:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8585: Skip most ACID EE tests in non-HDFS builds
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13921 to look at the new patch set (#2). Change subject: IMPALA-8585: Skip most ACID EE tests in non-HDFS builds .. IMPALA-8585: Skip most ACID EE tests in non-HDFS builds Currently these tests rely on Hive to insert data into ACID tables (+ also compaction in some cases), which fails on non-HDFS filesystems (e.g. S3). Change-Id: I5f67654805b770d86090b0d1015e144147d73f40 --- M tests/query_test/test_acid.py 1 file changed, 22 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/13921/2 -- To view, visit http://gerrit.cloudera.org:8080/13921 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5f67654805b770d86090b0d1015e144147d73f40 Gerrit-Change-Number: 13921 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8585: Skip most ACID EE tests in non-HDFS builds
Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13921 Change subject: IMPALA-8585: Skip most ACID EE tests in non-HDFS builds .. IMPALA-8585: Skip most ACID EE tests in non-HDFS builds Currently these tests rely on Hive to insert data into ACID tables (+ also compaction in some cases), which fails on non-HDFS filesystems (e.g. S3). Change-Id: I5f67654805b770d86090b0d1015e144147d73f40 --- M tests/query_test/test_acid.py 1 file changed, 22 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/13921/1 -- To view, visit http://gerrit.cloudera.org:8080/13921 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5f67654805b770d86090b0d1015e144147d73f40 Gerrit-Change-Number: 13921 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer
[Impala-ASF-CR] IMPALA-8794: Skipping testPiggyback* in Hive 3 builds
Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13920 Change subject: IMPALA-8794: Skipping testPiggyback* in Hive 3 builds .. IMPALA-8794: Skipping testPiggyback* in Hive 3 builds The tests turned out to be very flaky with Hive 3. Skipping them for now to make the builds green. Change-Id: Ice8f52caa20021ad9e36cbebe62a2bb247a4273c --- M fe/src/test/java/org/apache/impala/catalog/local/CatalogdMetaProviderTest.java 1 file changed, 13 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/13920/1 -- To view, visit http://gerrit.cloudera.org:8080/13920 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ice8f52caa20021ad9e36cbebe62a2bb247a4273c Gerrit-Change-Number: 13920 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer
[Impala-ASF-CR] IMPALA-8434: retain tables and functions in altering database
Quanlong Huang has posted comments on this change. (
http://gerrit.cloudera.org:8080/13904 )
Change subject: IMPALA-8434: retain tables and functions in altering database
..
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/13904/2/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java
File fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java:
http://gerrit.cloudera.org:8080/#/c/13904/2/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java@392
PS2, Line 392: addDb(newDb);
Thanks to Vihang's comment in person. Will move this below after adding
existing tables and functions to avoid race of seeing the empty new database.
http://gerrit.cloudera.org:8080/#/c/13904/2/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java@394
PS2, Line 394: CatalogObjectVersionSet.INSTANCE.updateVersions(
: existingDb.getCatalogVersion(), catalogVersion);
:
CatalogObjectVersionSet.INSTANCE.removeAll(existingDb.getTables());
: CatalogObjectVersionSet.INSTANCE.removeAll(
: existingDb.getFunctions(null, new PatternMatcher()));
: // IMPALA-8434: add back the existing tables/functions.
Note that their version
: // counters in CatalogObjectVersionSet have been
decreased by the above removeAll
: // statements, meaning their references from the old db
are deleted since the old
: // db object has been replaced by newDb.
: for (Table tbl: existingDb.getTables()) {
: newDb.addTable(tbl);
: }
> Oops I only went one level deep and checked tableCache_.add(table); Didn't
Never mind :)
http://gerrit.cloudera.org:8080/#/c/13904/2/tests/metadata/test_ddl.py
File tests/metadata/test_ddl.py:
http://gerrit.cloudera.org:8080/#/c/13904/2/tests/metadata/test_ddl.py@241
PS2, Line 241: self.client.execute("create table {0}.tbl (i
int)".format(unique_database))
> nit: how about merging this with the above test?
I think they are testing different points. The above one is for the
functionality of altering owner for the database. This one is for checking
metadata according to the bug. It'd be better to let them act as unit tests. So
if someone fails them in the future, it'll be easier to know exactly what
happens.
--
To view, visit http://gerrit.cloudera.org:8080/13904
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia3dc9857fd2733e20cf10fbe17bb1a4670d7d015
Gerrit-Change-Number: 13904
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Comment-Date: Thu, 25 Jul 2019 18:50:58 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8434: retain tables and functions in altering database
Hello Bharath Vissapragada, Vihang Karajgaonkar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13904 to look at the new patch set (#3). Change subject: IMPALA-8434: retain tables and functions in altering database .. IMPALA-8434: retain tables and functions in altering database In the legacy catalog implementation (ImpaladCatalog), when altering a database, the tables and functions in it will disappear until we run INVALIDATE METADATA to reset the cache. The cause is that we just replace the old Db object with the new one deserialized from the TDatabase. We should migrate the existing tables and functions to the new Db object. Tests: - Add test_metadata_after_alter_database for the bug. - Run Core tests Change-Id: Ia3dc9857fd2733e20cf10fbe17bb1a4670d7d015 --- M fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java M tests/metadata/test_ddl.py 2 files changed, 29 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/13904/3 -- To view, visit http://gerrit.cloudera.org:8080/13904 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia3dc9857fd2733e20cf10fbe17bb1a4670d7d015 Gerrit-Change-Number: 13904 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8641: Added Zstd to impala file formats.xml
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13919 ) Change subject: IMPALA-8641: Added Zstd to impala_file_formats.xml .. IMPALA-8641: Added Zstd to impala_file_formats.xml Change-Id: I82133b982c472e408d12ce46fc61124627428911 Reviewed-on: http://gerrit.cloudera.org:8080/13919 Reviewed-by: Alex Rodoni Tested-by: Impala Public Jenkins --- M docs/topics/impala_file_formats.xml M docs/topics/impala_parquet.xml 2 files changed, 6 insertions(+), 4 deletions(-) Approvals: Alex Rodoni: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/13919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I82133b982c472e408d12ce46fc61124627428911 Gerrit-Change-Number: 13919 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8641: Added Zstd to impala file formats.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13919 ) Change subject: IMPALA-8641: Added Zstd to impala_file_formats.xml .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/409/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82133b982c472e408d12ce46fc61124627428911 Gerrit-Change-Number: 13919 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 25 Jul 2019 18:42:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8641: Added Zstd to impala file formats.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13919 ) Change subject: IMPALA-8641: Added Zstd to impala_file_formats.xml .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/408/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82133b982c472e408d12ce46fc61124627428911 Gerrit-Change-Number: 13919 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 25 Jul 2019 18:40:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8641: Added Zstd to impala file formats.xml
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13919 ) Change subject: IMPALA-8641: Added Zstd to impala_file_formats.xml .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82133b982c472e408d12ce46fc61124627428911 Gerrit-Change-Number: 13919 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 25 Jul 2019 18:37:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8641: Added Zstd to impala file formats.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13919 ) Change subject: IMPALA-8641: Added Zstd to impala_file_formats.xml .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/409/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82133b982c472e408d12ce46fc61124627428911 Gerrit-Change-Number: 13919 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 25 Jul 2019 18:37:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8641: Added Zstd to impala file formats.xml
Hello Abhishek Rawat, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13919 to look at the new patch set (#2). Change subject: IMPALA-8641: Added Zstd to impala_file_formats.xml .. IMPALA-8641: Added Zstd to impala_file_formats.xml Change-Id: I82133b982c472e408d12ce46fc61124627428911 --- M docs/topics/impala_file_formats.xml M docs/topics/impala_parquet.xml 2 files changed, 6 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/13919/2 -- To view, visit http://gerrit.cloudera.org:8080/13919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I82133b982c472e408d12ce46fc61124627428911 Gerrit-Change-Number: 13919 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[native-toolchain-CR] Patch Avro to 1.7.4-p5 to add DATE type support.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13898 ) Change subject: Patch Avro to 1.7.4-p5 to add DATE type support. .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13898 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I70aae9f16805e3ee806c39e9121b6525d83845b3 Gerrit-Change-Number: 13898 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 18:29:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8641: Added Zstd to impala file formats.xml
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13919 ) Change subject: IMPALA-8641: Added Zstd to impala_file_formats.xml .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82133b982c472e408d12ce46fc61124627428911 Gerrit-Change-Number: 13919 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 25 Jul 2019 18:25:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8641: Added Zstd to impala file formats.xml
Abhishek Rawat has posted comments on this change. ( http://gerrit.cloudera.org:8080/13919 ) Change subject: IMPALA-8641: Added Zstd to impala_file_formats.xml .. Patch Set 1: Code-Review+1 Looks good. Thanks! -- To view, visit http://gerrit.cloudera.org:8080/13919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82133b982c472e408d12ce46fc61124627428911 Gerrit-Change-Number: 13919 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 25 Jul 2019 18:23:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8641: Added Zstd to impala file formats.xml
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13919 Change subject: IMPALA-8641: Added Zstd to impala_file_formats.xml .. IMPALA-8641: Added Zstd to impala_file_formats.xml Change-Id: I82133b982c472e408d12ce46fc61124627428911 --- M docs/topics/impala_file_formats.xml 1 file changed, 5 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/13919/1 -- To view, visit http://gerrit.cloudera.org:8080/13919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I82133b982c472e408d12ce46fc61124627428911 Gerrit-Change-Number: 13919 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-8641: Added Zstd to impala file formats.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13919 ) Change subject: IMPALA-8641: Added Zstd to impala_file_formats.xml .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/408/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13919 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82133b982c472e408d12ce46fc61124627428911 Gerrit-Change-Number: 13919 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 25 Jul 2019 18:16:23 + Gerrit-HasComments: No
[native-toolchain-CR] Patch Avro to 1.7.4-p5 to add DATE type support.
Attila Jeges has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/13898 ) Change subject: Patch Avro to 1.7.4-p5 to add DATE type support. .. Patch Avro to 1.7.4-p5 to add DATE type support. This patch is needed because the Avro C library doesn't support logical types yet. This is a quick-and-dirty patch to add DATE type without a proper logical type implementation. Note, that DECIMAL type support was added similarly in 1.7.4-p2. This will be used in "IMPALA-8198: DATE: Read from avro." TESTING: I built the toolchain locally and using jenkins. I built impala (including IMPALA-8198) with the new toolchain and ran the core test suite. Change-Id: I70aae9f16805e3ee806c39e9121b6525d83845b3 --- M buildall.sh A source/avro/avro-1.7.4-patches/0005-Changes-to-Avro-C-library-to-allow-parsing-DATE.patch 2 files changed, 219 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/98/13898/2 -- To view, visit http://gerrit.cloudera.org:8080/13898 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I70aae9f16805e3ee806c39e9121b6525d83845b3 Gerrit-Change-Number: 13898 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13918 ) Change subject: IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/13918/1/be/src/transport/THttpServer.cpp File be/src/transport/THttpServer.cpp: http://gerrit.cloudera.org:8080/#/c/13918/1/be/src/transport/THttpServer.cpp@256 PS1, Line 256: h << "WWW-Authenticate: Negotiate" << CRLF; > shouldn't we be using a Negotiate header which includes the token returned This is for the error case, i.e. if a client attempts to connect without providing a 'Authorization: Negotiate' header at all. I guess you're saying that in that case I should be passing an empty token to kudu::gssapi::SpnegoStep and returning whatever the response is? -- To view, visit http://gerrit.cloudera.org:8080/13918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15d9a842ab37ebc34b9fde5917137ff2961d870a Gerrit-Change-Number: 13918 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 25 Jul 2019 18:00:51 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13918 ) Change subject: IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/13918/1/be/src/rpc/authentication.cc File be/src/rpc/authentication.cc: http://gerrit.cloudera.org:8080/#/c/13918/1/be/src/rpc/authentication.cc@565 PS1, Line 565: connection_context->return_headers; nit: std::move() around this http://gerrit.cloudera.org:8080/#/c/13918/1/be/src/transport/THttpServer.cpp File be/src/transport/THttpServer.cpp: http://gerrit.cloudera.org:8080/#/c/13918/1/be/src/transport/THttpServer.cpp@256 PS1, Line 256: h << "WWW-Authenticate: Negotiate" << CRLF; shouldn't we be using a Negotiate header which includes the token returned by gssapi here? I guess this will work for certain configurations where the server's initial challenge is empty, but I think there might be cases where this isn't the case. -- To view, visit http://gerrit.cloudera.org:8080/13918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15d9a842ab37ebc34b9fde5917137ff2961d870a Gerrit-Change-Number: 13918 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 25 Jul 2019 17:47:33 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8434: retain tables and functions in altering database
Bharath Vissapragada has posted comments on this change. (
http://gerrit.cloudera.org:8080/13904 )
Change subject: IMPALA-8434: retain tables and functions in altering database
..
Patch Set 2: Code-Review+2
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13904/2/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java
File fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java:
http://gerrit.cloudera.org:8080/#/c/13904/2/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java@394
PS2, Line 394: CatalogObjectVersionSet.INSTANCE.updateVersions(
: existingDb.getCatalogVersion(), catalogVersion);
:
CatalogObjectVersionSet.INSTANCE.removeAll(existingDb.getTables());
: CatalogObjectVersionSet.INSTANCE.removeAll(
: existingDb.getFunctions(null, new PatternMatcher()));
: // IMPALA-8434: add back the existing tables/functions.
Note that their version
: // counters in CatalogObjectVersionSet have been
decreased by the above removeAll
: // statements, meaning their references from the old db
are deleted since the old
: // db object has been replaced by newDb.
: for (Table tbl: existingDb.getTables()) {
: newDb.addTable(tbl);
: }
> Db.addTable and Db.addFunction will finally add back the versions to Catalo
Oops I only went one level deep and checked tableCache_.add(table); Didn't
realize that it was updating the CatalogObjectVersionSet underneath. Sorry for
the false alarm.
http://gerrit.cloudera.org:8080/#/c/13904/2/tests/metadata/test_ddl.py
File tests/metadata/test_ddl.py:
http://gerrit.cloudera.org:8080/#/c/13904/2/tests/metadata/test_ddl.py@241
PS2, Line 241: self.client.execute("create table {0}.tbl (i
int)".format(unique_database))
nit: how about merging this with the above test?
--
To view, visit http://gerrit.cloudera.org:8080/13904
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia3dc9857fd2733e20cf10fbe17bb1a4670d7d015
Gerrit-Change-Number: 13904
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Comment-Date: Thu, 25 Jul 2019 17:44:52 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13918 ) Change subject: IMPALA-8783: Add Kerberos SPNEGO support to the http hs2 server .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/3999/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/13918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15d9a842ab37ebc34b9fde5917137ff2961d870a Gerrit-Change-Number: 13918 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 25 Jul 2019 17:42:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8641: Document compression codec zstd in Parquet
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13910 ) Change subject: IMPALA-8641: Document compression codec zstd in Parquet .. Patch Set 2: I will update impala_file_formats.xml in a separate patch. -- To view, visit http://gerrit.cloudera.org:8080/13910 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id55dbc3297ec1560b04e4da2c93cc1aeb1e6fb2f Gerrit-Change-Number: 13910 Gerrit-PatchSet: 2 Gerrit-Owner: Abhishek Rawat Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 17:41:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7486: Add specialized estimation scheme for dedicated coordinators
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13740 ) Change subject: IMPALA-7486: Add specialized estimation scheme for dedicated coordinators .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3998/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13740 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2b94e7293b91dec8a18491079c34923eadd94b21 Gerrit-Change-Number: 13740 Gerrit-PatchSet: 12 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 17:32:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8641: Document compression codec zstd in Parquet
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13910 ) Change subject: IMPALA-8641: Document compression codec zstd in Parquet .. Patch Set 2: Otherwise it looks good. -- To view, visit http://gerrit.cloudera.org:8080/13910 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id55dbc3297ec1560b04e4da2c93cc1aeb1e6fb2f Gerrit-Change-Number: 13910 Gerrit-PatchSet: 2 Gerrit-Owner: Abhishek Rawat Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 17:28:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8641: Document compression codec zstd in Parquet
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13910 ) Change subject: IMPALA-8641: Document compression codec zstd in Parquet .. Patch Set 2: I just noticed that we need to add zstd to the list of parquet codecs in topics/impala_file_formats.xml -- To view, visit http://gerrit.cloudera.org:8080/13910 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id55dbc3297ec1560b04e4da2c93cc1aeb1e6fb2f Gerrit-Change-Number: 13910 Gerrit-PatchSet: 2 Gerrit-Owner: Abhishek Rawat Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Jul 2019 17:28:26 + Gerrit-HasComments: No
