[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 03 Nov 2020 05:28:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables
wangsheng has posted comments on this change. ( http://gerrit.cloudera.org:8080/16599 ) Change subject: IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables .. Patch Set 4: Code-Review+1 (1 comment) Thanks for changes, Gabor. Apart from ' adding new reserved words' problem which Tim mentioned above, this patch LGTM. http://gerrit.cloudera.org:8080/#/c/16599/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16599/1//COMMIT_MSG@14 PS1, Line 14: ++-+-+-+ : | creation_time | snapshot_id | parent_id | is_current_ancestor | : ++-+-+-+ : | 2020-10-13 14:01:07.234000 | 4400379706200951771 | NULL | TRUE| : | 2020-10-13 14:01:19.307000 | 4221472712544505868 | 4400379706200951771 | TRUE| : ++-+-+-+ > That should be a different query in my opinion. Here what I try to reproduc Thanks for explain, maybe we can support snapshot in another patch. -- To view, visit http://gerrit.cloudera.org:8080/16599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5 Gerrit-Change-Number: 16599 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Tue, 03 Nov 2020 02:22:54 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7606/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 9 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 01:39:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16676 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 2: (1 comment) The change looks good from a content point of view. I have one small style point. http://gerrit.cloudera.org:8080/#/c/16676/2/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test File testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test: http://gerrit.cloudera.org:8080/#/c/16676/2/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test@7 PS2, Line 7: # IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit : # the memory limit. The mem_limit is reduced from 170MB to 168MB since the memory : # shortage observed at exchange nodes can be as small as 8KB without exceeding : # the limit of 170MB. >From a purely style note, I think this comment is not necessary. The comment >about IMPALA-5485 is describing why this test exists and is useful to a >reader. The details about adjustments to the memory limit to avoid flakiness >are important for the history, but it is usually not something a reader of >this file needs to know about. All the history is accessible by git blame, so >they can find the commit that set the mem_limit=168m if they need. So, I would >lean toward not including a comment for this. -- To view, visit http://gerrit.cloudera.org:8080/16676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 Gerrit-Change-Number: 16676 Gerrit-PatchSet: 2 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 03 Nov 2020 01:30:07 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10283: Fix IllegalStateException in applying incremental partition updates
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/16649 ) Change subject: IMPALA-10283: Fix IllegalStateException in applying incremental partition updates .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/16649/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16649/2//COMMIT_MSG@14 PS2, Line 14: aka > nit, replacing with "via" makes more sense to me. Done http://gerrit.cloudera.org:8080/#/c/16649/2//COMMIT_MSG@28 PS2, Line 28: So one of the coordinator will have a table version : that is lower than the latest version in catalogd but larger than the : last sent table version in catalogd > IIUC, we are saying that the table in the coordinator cache is already at a Sorry, not this case. Let's say last sent catalog version: v1 current catalog version of the table in coordinator: v2 table catalog version in current topic-update: v3 The case is when v1 < v2 < v3, v2 will have some partition instances of v3. So when applying partition updates of v3, we will see duplicated partition instances. Let me update the commit message with an example. -- To view, visit http://gerrit.cloudera.org:8080/16649 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1657684f8853b76b1524475a3b3c35fa22a0e36e Gerrit-Change-Number: 16649 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 03 Nov 2020 01:24:51 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10283: Fix IllegalStateException in applying incremental partition updates
Hello Vihang Karajgaonkar, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16649
to look at the new patch set (#3).
Change subject: IMPALA-10283: Fix IllegalStateException in applying incremental
partition updates
..
IMPALA-10283: Fix IllegalStateException in applying incremental partition
updates
When incremental metadata updates are enabled (by default), catalogd
sends incremental partition updates based on the last sent table
snapshot. Coordinators will apply these partition updates on their
existing table snapshots.
Each partition update is via a partition instance. Partition instances
are identified by partition ids. Each partition instance is a snapshot
of the metadata of a partition. When applying incremental partition
updates, ImpaladCatalog#addTable() has a Precondition check assuming
that new partition updates should not be duplicated with existing
partition ids.
The motivation of this check is to detect whether catalogd is sending
duplicate partition updates. However, it could be hitted when the
coordinator has a newer version of the table than the last sent table
snapshot in catalogd. This happens when two coordinators both execute
DMLs on the same table (e.g. insert into different partitions), and the
DMLs finish within a catalog topic update time window. Note that
coordinator will receive a table snapshot from catalogd as a response of
the DML request. So one of the coordinator will have a table version
that is lower than the latest version in catalogd but larger than the
last sent table version in catalogd. For an example, let's see the
following sequence of events on a table:
t0: coord1 and coord2 both have the latest version as catalogd
t1: coord1 executes a DML to add a partition p2
t2: coord2 executes a DML to add another partition p3
t3: catalogd sends topic update with {p2, p3}
t1 and t2 happen inside a topic-update window. So catalogd will send the
update of {p2, p3}. The following table shows the table version and
corresponding partition instances in each server.
++---+--+---+
|| catalogd | coordinator1 | coordinator2 |
++---+--+---+
| t0 | v0:{p1} | v0:{p1} | v0:{p1} |
++---+--+---+
| t1 | v1:{p1,p2}| v1:{p1,p2} | v0:{p1} |
++---+--+---+
| t2 | v2:{p1,p2,p3} | v1:{p1,p2} | v2:{p1,p2,p3} |
++---+--+---+
At t3, coordinator2 will skip the table update since it already has a
version equal to the one in the topic update. However, on coordinator1,
the table version is smaller than v2, so it will apply the incremental
updates of {p2,p3} and then hit the Precondition check complaining that
p2 already exists.
It's legal that a coordinator has got some partition instances in the
DML responses. So we can't assume that all partition updates in a topic
update don't exist in the coordinator. This patch removes this
Precondition check to accept this case.
Tests:
- Add a test to reproduce the scenario mentioned above. It fails
without this patch.
Change-Id: I1657684f8853b76b1524475a3b3c35fa22a0e36e
---
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java
A tests/custom_cluster/test_incremental_metadata_updates.py
3 files changed, 79 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/16649/3
--
To view, visit http://gerrit.cloudera.org:8080/16649
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1657684f8853b76b1524475a3b3c35fa22a0e36e
Gerrit-Change-Number: 16649
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has posted comments on this change. (
http://gerrit.cloudera.org:8080/16621 )
Change subject: IMPALA-3816: Codegen perf critical loops in Sorter
..
Patch Set 9:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc
File be/src/runtime/sorter.cc:
http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc@1215
PS8, Line 1215: llvm::Function* fn =
codegen->GetFunction(IRFunction::TUPLE_SORTER_SORT_HELPER, true);
> Looks like there is only on call to SortHelper() which is from Sorter::Tupl
Did some additional testing with instrumentation and found the return addresses
reported via __builtin_return_address(0) in the LLVM and non-LLVM version of
SortHelper() are quite different: In the range of 0x7f7f12135exx for LLVM and
0x18c2yyy in non-LLVM. Note that __builtin_return_address(0) returns the
address of the instruction after a function call.
167 Status Sorter::TupleSorter::SortHelper(TupleIterator begin, TupleIterator
end) {
168 // Use insertion sort for smaller sequences.
169
170 printf("Enter SortHelper(): return_address(0)=%p\n",
__builtin_return_address(0));
171
172 while (end.index() - begin.index() > INSERTION_THRESHOLD) {
173 // Select a pivot and call Partition() to split the tuples in [begin,
end) into two
174 // groups (<= pivot and >= pivot) in-place. 'cut' is the index of the
first tuple in
175 // the second group.
176 Tuple* pivot = SelectPivot(begin, end);
177 TupleIterator cut;
178 RETURN_IF_ERROR(Partition(begin, end, pivot, &cut));
179
180 // Recurse on the smaller partition. This limits stack size to log(n)
stack frames.
181 if (cut.index() - begin.index() < end.index() - cut.index()) {
182 // Left partition is smaller.
183 printf("call SortHelper() for [begin, cut]\n");
184 RETURN_IF_ERROR(SortHelper(begin, cut));
185 begin = cut;
186 } else {
187 // Right partition is equal or smaller.
188 printf("call SortHelper() for [cut, end]\n");
189 RETURN_IF_ERROR(SortHelper(cut, end));
190 end = cut;
191 }
192 }
193
194 if (begin.index() < end.index()) RETURN_IF_ERROR(InsertionSort(begin,
end));
195
196 printf("Exit SortHelper(): return_address(0)=%p\n",
__builtin_return_address(0));
197 return Status::OK();
198 }
--
To view, visit http://gerrit.cloudera.org:8080/16621
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b
Gerrit-Change-Number: 16621
Gerrit-PatchSet: 9
Gerrit-Owner: Qifan Chen
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Qifan Chen
Gerrit-Reviewer: Sahil Takiar
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Tue, 03 Nov 2020 01:21:09 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. IMPALA-3816: Codegen perf critical loops in Sorter This fix added the functionality to codegen method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit test to order 7300 rows from table functional.alltypes, the speedup of the code-gen version over non-code-gen version of the method is around 26%. In another unit test to partially order 2880404 rows, the speedup is around 20%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 123 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/9 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 9 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-10058: Use commit hash as version for Kudu java artifacts
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16686 ) Change subject: IMPALA-10058: Use commit hash as version for Kudu java artifacts .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16686 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a6c9676f4521d6709393143d3e82533486164d3 Gerrit-Change-Number: 16686 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 01:17:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9180 (part 3): Remove legacy backend port
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16533 ) Change subject: IMPALA-9180 (part 3): Remove legacy backend port .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16533 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic6909a8da449b4d25ee98037b3eb459af4850dc6 Gerrit-Change-Number: 16533 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 03 Nov 2020 00:56:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9180 (part 3): Remove legacy backend port
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16533 ) Change subject: IMPALA-9180 (part 3): Remove legacy backend port .. IMPALA-9180 (part 3): Remove legacy backend port The legacy Thrift based Impala internal service has been removed so the backend port 22000 can be freed up. This patch set flag be_port as a REMOVED_FLAG and all infrastructures around it are cleaned up. StatestoreSubscriber::subscriber_id is set as hostname + krpc_port. Testing: - Passed the exhaustive test. Change-Id: Ic6909a8da449b4d25ee98037b3eb459af4850dc6 Reviewed-on: http://gerrit.cloudera.org:8080/16533 Reviewed-by: Thomas Tauber-Marshall Tested-by: Impala Public Jenkins --- M be/src/common/global-flags.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/data-stream-test.cc M be/src/runtime/exec-env.cc M be/src/runtime/krpc-data-stream-sender.cc M be/src/scheduling/schedule-state.h M be/src/scheduling/scheduler.cc M bin/start-impala-cluster.py M common/protobuf/admission_control_service.proto M common/protobuf/control_service.proto M common/protobuf/statestore_service.proto M infra/deploy/deploy.py M tests/common/impala_cluster.py M tests/common/impala_service.py M tests/custom_cluster/test_query_retries.py M tests/shell/test_shell_interactive.py 16 files changed, 28 insertions(+), 38 deletions(-) Approvals: Thomas Tauber-Marshall: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16533 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic6909a8da449b4d25ee98037b3eb459af4850dc6 Gerrit-Change-Number: 16533 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7605/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 03 Nov 2020 00:15:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6629/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 03 Nov 2020 00:03:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7604/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 02 Nov 2020 23:59:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/16690/1/tests/common/impala_service.py File tests/common/impala_service.py: http://gerrit.cloudera.org:8080/#/c/16690/1/tests/common/impala_service.py@136 PS1, Line 136: d > flake8: E303 too many blank lines (2) Done -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 02 Nov 2020 23:54:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16690 to look at the new patch set (#2). Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. IMPALA-9864: Produce a minidump when TestValidateMetrics fails After running end-to-end tests, run-tests.py runs verifiers to check that a set of metrics are zero. When this fails, it can indicate a hung query fragment or other resource leak (see IMPALA-9842 for example). To track this down, it is useful to have a minidump, so this adds a step to have every Impalad generate a minidump (by sending SIGUSR1) when we hit the timeout. Also, the current error message dumps a bunch of unformatted JSON from our Web UI. This is hard to read and painful to cut/paste. This now dumps that JSON to files in a diagnostic directory under the logs directory. The JSON is formatted in a readable way. These files would be preserved along with the rest of the logs directory for automated runs. The new error message looks like this: E AssertionError: Metric impala-server.num-queries-registered did not reach value 0 in 60s. E Dumping debug webpages in JSON format... E Dumped memz JSON to $IMPALA_HOME/logs/metric_timeout_diags_1604359071/json/memz.json E Dumped metrics JSON to $IMPALA_HOME/logs/metric_timeout_diags_1604359071/json/metrics.json E Dumped queries JSON to $IMPALA_HOME/logs/metric_timeout_diags_1604359071/json/queries.json E Dumped sessions JSON to $IMPALA_HOME/logs/metric_timeout_diags_1604359071/json/sessions.json E Dumped threadz JSON to $IMPALA_HOME/logs/metric_timeout_diags_1604359071/json/threadz.json E Dumped rpcz JSON to $IMPALA_HOME/logs/metric_timeout_diags_1604359071/json/rpcz.json E Dumping minidumps for 3 running impalads... E Dumped minidump for PID 2709 E Dumped minidump for PID 2714 E Dumped minidump for PID 2721 This also fixes various flake8 errors (unnecessary imports, etc), so now impala_service.py is flake8 clean. Testing: - Tried out the dump function on my developer machine - Verified the minidumps exist - Verified the JSON is readable Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d --- M tests/common/impala_service.py 1 file changed, 70 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/16690/2 -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16690 Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. IMPALA-9864: Produce a minidump when TestValidateMetrics fails After running end-to-end tests, run-tests.py runs verifiers to check that a set of metrics are zero. When this fails, it can indicate a hung query fragment or other resource leak (see IMPALA-9842 for example). To track this down, it is useful to have a minidump, so this adds a step to have every Impalad generate a minidump (by sending SIGUSR1) when we hit the timeout. Also, the current error message dumps a bunch of unformatted JSON from our Web UI. This is hard to read and painful to cut/paste. This now dumps that JSON to files in a diagnostic directory under the logs directory. The JSON is formatted in a readable way. These files would be preserved along with the rest of the logs directory for automated runs. The new error message looks like this: E AssertionError: Metric impala-server.num-queries-registered did not reach value 0 in 60s. E Dumping debug webpages in JSON format... E Dumped memz JSON to $IMPALA_HOME/logs/metric_timeout_diags_1604359071/json/memz.json E Dumped metrics JSON to $IMPALA_HOME/logs/metric_timeout_diags_1604359071/json/metrics.json E Dumped queries JSON to $IMPALA_HOME/logs/metric_timeout_diags_1604359071/json/queries.json E Dumped sessions JSON to $IMPALA_HOME/logs/metric_timeout_diags_1604359071/json/sessions.json E Dumped threadz JSON to $IMPALA_HOME/logs/metric_timeout_diags_1604359071/json/threadz.json E Dumped rpcz JSON to $IMPALA_HOME/logs/metric_timeout_diags_1604359071/json/rpcz.json E Dumping minidumps for 3 running impalads... E Dumped minidump for PID 2709 E Dumped minidump for PID 2714 E Dumped minidump for PID 2721 Testing: - Tried out the dump function on my developer machine - Verified the minidumps exist - Verified the JSON is readable Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d --- M tests/common/impala_service.py 1 file changed, 62 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/16690/1 -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/16690/1/tests/common/impala_service.py File tests/common/impala_service.py: http://gerrit.cloudera.org:8080/#/c/16690/1/tests/common/impala_service.py@136 PS1, Line 136: d flake8: E303 too many blank lines (2) -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 02 Nov 2020 23:39:20 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10058: Use commit hash as version for Kudu java artifacts
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16686 ) Change subject: IMPALA-10058: Use commit hash as version for Kudu java artifacts .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6628/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/16686 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a6c9676f4521d6709393143d3e82533486164d3 Gerrit-Change-Number: 16686 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 02 Nov 2020 19:54:51 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-10058: Use commit hash as version for Kudu java artifacts
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16685 ) Change subject: IMPALA-10058: Use commit hash as version for Kudu java artifacts .. Patch Set 1: This is the corresponding change to https://gerrit.cloudera.org/#/c/16686/ which uses a toolchain produced by this. -- To view, visit http://gerrit.cloudera.org:8080/16685 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I352fdcb02ce3c11432cb5d6e7fdc564fe9b3c038 Gerrit-Change-Number: 16685 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 02 Nov 2020 19:53:53 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-10305: Sync Kudu security code change for FIPS
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/16684 ) Change subject: WIP IMPALA-10305: Sync Kudu security code change for FIPS .. Patch Set 2: Yes, it's still WIP since not all patches are merged into Kudu upstream yet. Thanks. -- To view, visit http://gerrit.cloudera.org:8080/16684 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1aa5e69bf8470e3cfb316e8b6354ad29c28223f4 Gerrit-Change-Number: 16684 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 02 Nov 2020 19:51:45 + Gerrit-HasComments: No
[Impala-ASF-CR] Replace some TODOs with specific JIRA references
Tim Armstrong has abandoned this change. ( http://gerrit.cloudera.org:8080/11392 ) Change subject: Replace some TODOs with specific JIRA references .. Abandoned Cleaning up some old CRs -- To view, visit http://gerrit.cloudera.org:8080/11392 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I8d008b62e11b8057cade503721046cd696c3ca77 Gerrit-Change-Number: 11392 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation
Tim Armstrong has abandoned this change. ( http://gerrit.cloudera.org:8080/11055 ) Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation .. Abandoned Cleaning up some old CRs -- To view, visit http://gerrit.cloudera.org:8080/11055 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c Gerrit-Change-Number: 11055 Gerrit-PatchSet: 4 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Rahul Shivu Mahadev Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] WIP IMPALA-10305: Sync Kudu security code change for FIPS
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16684 ) Change subject: WIP IMPALA-10305: Sync Kudu security code change for FIPS .. Patch Set 2: Is this still a WIP? I can review once it's ready -- To view, visit http://gerrit.cloudera.org:8080/16684 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1aa5e69bf8470e3cfb316e8b6354ad29c28223f4 Gerrit-Change-Number: 16684 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 02 Nov 2020 19:45:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10185 Use bool stats for selectivity calculations.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16462 ) Change subject: IMPALA-10185 Use bool stats for selectivity calculations. .. Patch Set 2: I'm ready to +2 after the minor code cleanup -- To view, visit http://gerrit.cloudera.org:8080/16462 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I95c1c7c915bf6bca13fe006c0531c33988187d12 Gerrit-Change-Number: 16462 Gerrit-PatchSet: 2 Gerrit-Owner: Shant Hovsepian Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 02 Nov 2020 19:44:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9180 (part 3): Remove legacy backend port
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16533 ) Change subject: IMPALA-9180 (part 3): Remove legacy backend port .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6627/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16533 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic6909a8da449b4d25ee98037b3eb459af4850dc6 Gerrit-Change-Number: 16533 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 02 Nov 2020 19:30:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9180 (part 3): Remove legacy backend port
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/16533 ) Change subject: IMPALA-9180 (part 3): Remove legacy backend port .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16533 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic6909a8da449b4d25ee98037b3eb459af4850dc6 Gerrit-Change-Number: 16533 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 02 Nov 2020 19:30:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9180 (part 3): Remove legacy backend port
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16533 ) Change subject: IMPALA-9180 (part 3): Remove legacy backend port .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7603/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16533 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic6909a8da449b4d25ee98037b3eb459af4850dc6 Gerrit-Change-Number: 16533 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 02 Nov 2020 19:22:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9180 (part 3): Remove legacy backend port
Wenzhe Zhou has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/16533 ) Change subject: IMPALA-9180 (part 3): Remove legacy backend port .. IMPALA-9180 (part 3): Remove legacy backend port The legacy Thrift based Impala internal service has been removed so the backend port 22000 can be freed up. This patch set flag be_port as a REMOVED_FLAG and all infrastructures around it are cleaned up. StatestoreSubscriber::subscriber_id is set as hostname + krpc_port. Testing: - Passed the exhaustive test. Change-Id: Ic6909a8da449b4d25ee98037b3eb459af4850dc6 --- M be/src/common/global-flags.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/data-stream-test.cc M be/src/runtime/exec-env.cc M be/src/runtime/krpc-data-stream-sender.cc M be/src/scheduling/schedule-state.h M be/src/scheduling/scheduler.cc M bin/start-impala-cluster.py M common/protobuf/admission_control_service.proto M common/protobuf/control_service.proto M common/protobuf/statestore_service.proto M infra/deploy/deploy.py M tests/common/impala_cluster.py M tests/common/impala_service.py M tests/custom_cluster/test_query_retries.py M tests/shell/test_shell_interactive.py 16 files changed, 28 insertions(+), 38 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16533/4 -- To view, visit http://gerrit.cloudera.org:8080/16533 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic6909a8da449b4d25ee98037b3eb459af4850dc6 Gerrit-Change-Number: 16533 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-9180 (part 3): Remove legacy backend port
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/16533 ) Change subject: IMPALA-9180 (part 3): Remove legacy backend port .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/16533/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16533/3//COMMIT_MSG@18 PS3, Line 18: - Did manually testing on a dev cluster, verified that CM with fixing :work with new version of Impala, which set subscriber_id as hostname :+ krpc_port. Verified the subscriber_ids of Impalad in Statestore Web :Server. The health check of Impala cluster on CM worked as expected. : - Passed CDH L0 test on cdpd-master branch. As this is Apache Impala, its better to leave out info about any vendor-specific work that you did, as its not really relevant here http://gerrit.cloudera.org:8080/#/c/16533/3/be/src/runtime/data-stream-test.cc File be/src/runtime/data-stream-test.cc: http://gerrit.cloudera.org:8080/#/c/16533/3/be/src/runtime/data-stream-test.cc@77 PS3, Line 77: Thrift krpc -- To view, visit http://gerrit.cloudera.org:8080/16533 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic6909a8da449b4d25ee98037b3eb459af4850dc6 Gerrit-Change-Number: 16533 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 02 Nov 2020 18:49:21 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10283: Fix IllegalStateException in applying incremental partition updates
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/16649 ) Change subject: IMPALA-10283: Fix IllegalStateException in applying incremental partition updates .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/16649/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16649/2//COMMIT_MSG@14 PS2, Line 14: aka nit, replacing with "via" makes more sense to me. http://gerrit.cloudera.org:8080/#/c/16649/2//COMMIT_MSG@28 PS2, Line 28: So one of the coordinator will have a table version : that is lower than the latest version in catalogd but larger than the : last sent table version in catalogd IIUC, we are saying that the table in the coordinator cache is already at a later version (due to DML response) than one in the topic-update? In this case shouldn't we be skipping the table altogether here https://github.com/apache/impala/blob/c3b5cf8b4c807fa4136b64addb1e7e8be8aaf6c5/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java#L472 since the thrift table's version is lower than the table version in the coordinator cache? -- To view, visit http://gerrit.cloudera.org:8080/16649 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1657684f8853b76b1524475a3b3c35fa22a0e36e Gerrit-Change-Number: 16649 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 02 Nov 2020 18:44:44 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10252: fix invalid runtime filters for outer joins
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16622 ) Change subject: IMPALA-10252: fix invalid runtime filters for outer joins .. Patch Set 5: Updated the commit message as requested. Shant, I think hasNullRejectingConjucts (sp) in Analyzer.java handles at least this case correctly - it does call isTrueWithNullSlots() on the expression. I guess it's possible that it might handle more complex expressions incorrectly, e.g. if the expression has slots from both sides of the join and is false when all slots are null but true if a subset of slots is null. [localhost.EXAMPLE.COM:21050] default> set ENABLE_OUTER_JOIN_TO_INNER_TRANSFORMATION=1; ENABLE_OUTER_JOIN_TO_INNER_TRANSFORMATION set to 1 [localhost.EXAMPLE.COM:21050] default> explain select * from functional.alltypes t1 left outer join functional.alltypestiny t2 on t1.id = t2.id where zeroifnull(t2.int_col) = 0; Query: explain select * from functional.alltypes t1 left outer join functional.alltypestiny t2 on t1.id = t2.id where zeroifnull(t2.int_col) = 0 ++ | Explain String | ++ | Max Per-Host Resource Reservation: Memory=1.98MB Threads=5 | | Per-Host Resource Estimates: Memory=163MB | | Codegen disabled by planner| || | PLAN-ROOT SINK | | | | | 04:EXCHANGE [UNPARTITIONED]| | | | | 02:HASH JOIN [LEFT OUTER JOIN, BROADCAST] | | | hash predicates: t1.id = t2.id | | | other predicates: zeroifnull(t2.int_col) = 0| | | row-size=178B cardinality=7.30K | | | | | |--03:EXCHANGE [BROADCAST] | | | | | | | 01:SCAN HDFS [functional.alltypestiny t2] | | | HDFS partitions=4/4 files=4 size=460B| | | row-size=89B cardinality=8 | | | | | 00:SCAN HDFS [functional.alltypes t1] | |HDFS partitions=24/24 files=24 size=478.45KB| |row-size=89B cardinality=7.30K | ++ Fetched 22 row(s) in 0.05s [localhost.EXAMPLE.COM:21050] default> explain select * from functional.alltypes t1 left outer join functional.alltypestiny t2 on t1.id = t2.id where t2.int_col = 0; Query: explain select * from functional.alltypes t1 left outer join functional.alltypestiny t2 on t1.id = t2.id where t2.int_col = 0 ++ | Explain String | ++ | Max Per-Host Resource Reservation: Memory=2.98MB Threads=5 | | Per-Host Resource Estimates: Memory=163MB | | Codegen disabled by planner| || | PLAN-ROOT SINK | | | | | 04:EXCHANGE [UNPARTITIONED]| | | | | 02:HASH JOIN [INNER JOIN, BROADCAST] | | | hash predicates: t1.id = t2.id | | | runtime filters: RF000 <- t2.id | | | row-size=178B cardinality=4 | | | | | |--03:EXCHANGE [BROADCAST] | | | | | | | 01:SCAN HDFS [functional.alltypestiny t2] | | | HDFS partitions=4/4 files=4 size=460B| | | predicates: t2.int_col = 0 | | | row-size=89B cardinality=4 | | | | | 00:SCAN HDFS [functional.alltypes t1] | |HDFS partitions=24/24 files=24 size=478.45KB| |runtime filters: RF000 -> t1.id | |row-size=89B cardinality=7.30K | ++
[Impala-ASF-CR] IMPALA-10252: fix invalid runtime filters for outer joins
Hello Aman Sinha, Qifan Chen, Shant Hovsepian, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16622 to look at the new patch set (#6). Change subject: IMPALA-10252: fix invalid runtime filters for outer joins .. IMPALA-10252: fix invalid runtime filters for outer joins The planner generates runtime filters for non-join conjuncts assigned to LEFT OUTER and FULL OUTER JOIN nodes. This is correct in many cases where NULLs stemming from unmatched rows would result in the predicate evaluating to false. E.g. x = y is always false if y is NULL. However, it is incorrect if the NULL returned from the unmatched row can result in the predicate evaluating to true. E.g. x = isnull(y, 1) can return true even if y is NULL. The fix is to detect cases when the source expression from the left input of the join returns non-NULL for null inputs and then skip generating the filter. Examples of expressions that may be affected by this change are COALESCE and ISNULL. Testing: Added regression tests: * Planner tests for LEFT OUTER and FULL OUTER where the runtime filter was incorrectly generated before this patch. * Enabled end-to-end test that was previously failing. * Added a new runtime filter test that will execute on both Parquet and Kudu (which are subtly different because of nullability of slots). Ran exhaustive tests. Change-Id: I507af1cc8df15bca21e0d8555019997812087261 --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java M testdata/workloads/functional-planner/queries/PlannerTest/runtime-filter-propagation.test M testdata/workloads/functional-query/queries/QueryTest/runtime_filters.test M testdata/workloads/functional-query/queries/QueryTest/subquery.test M tests/query_test/test_queries.py 6 files changed, 144 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/16622/6 -- To view, visit http://gerrit.cloudera.org:8080/16622 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I507af1cc8df15bca21e0d8555019997812087261 Gerrit-Change-Number: 16622 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-10306: [DOCS] add note about TZ offset
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16689 ) Change subject: IMPALA-10306: [DOCS] add note about TZ offset .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/606/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/16689 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I693d15f1230dd7eebcbf2a16657a3850943749e1 Gerrit-Change-Number: 16689 Gerrit-PatchSet: 1 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 02 Nov 2020 18:09:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10306: [DOCS] add note about TZ offset
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16689 ) Change subject: IMPALA-10306: [DOCS] add note about TZ offset .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/606/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/16689 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I693d15f1230dd7eebcbf2a16657a3850943749e1 Gerrit-Change-Number: 16689 Gerrit-PatchSet: 1 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 02 Nov 2020 17:49:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10306: [DOCS] add note about TZ offset
Shajini Thayasingh has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16689 Change subject: IMPALA-10306: [DOCS] add note about TZ offset .. IMPALA-10306: [DOCS] add note about TZ offset added a note that says that TZ offset will not be included in the output of this function even if the offset is provided as input Change-Id: I693d15f1230dd7eebcbf2a16657a3850943749e1 --- M docs/topics/impala_datetime_functions.xml 1 file changed, 4 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/16689/1 -- To view, visit http://gerrit.cloudera.org:8080/16689 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I693d15f1230dd7eebcbf2a16657a3850943749e1 Gerrit-Change-Number: 16689 Gerrit-PatchSet: 1 Gerrit-Owner: Shajini Thayasingh
[Impala-ASF-CR] IMPALA-10234: Add support for cookie authentication to impala-shell
Thomas Tauber-Marshall has posted comments on this change. (
http://gerrit.cloudera.org:8080/16660 )
Change subject: IMPALA-10234: Add support for cookie authentication to
impala-shell
..
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/16660/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/16660/3//COMMIT_MSG@14
PS3, Line 14: - Unit tests were added to test cookie handling methods.
Can you also add a check against the metrics in LdapImpalaShellTest.java, eg.
like what I've done in this patch:
https://github.com/twmarshall/impala/commit/d74cd55713f7bdd1de629c64d8b4c3a9e08b2785
http://gerrit.cloudera.org:8080/#/c/16660/3//COMMIT_MSG@15
PS3, Line 15: - Tested e2e manually.
It would be good to be sure that we've tested this with the main HTTP proxies
that Impala is expected to work with (Knox and nginx). Probably this is fine to
leave as a follow up task.
http://gerrit.cloudera.org:8080/#/c/16660/3/shell/ImpalaHttpClient.py
File shell/ImpalaHttpClient.py:
http://gerrit.cloudera.org:8080/#/c/16660/3/shell/ImpalaHttpClient.py@251
PS3, Line 251: # A '401 Unauthorized' response might mean that we tried
cookie-based authentication
: # with an expired cookie.
We should explicitly detect this by looking at the returned headers for a
impala.auth cookie with a max-age of 0 (see GetDeleteCookie() in
authentication-util.cc)
http://gerrit.cloudera.org:8080/#/c/16660/3/shell/impala_client.py
File shell/impala_client.py:
http://gerrit.cloudera.org:8080/#/c/16660/3/shell/impala_client.py@406
PS3, Line 406: return {"Authorization": basic_auth_header}
So I think the way most clients usually handle this is to just send the
Authorization header on every request, even if there's an auth cookie. There's
basically no downside (just a very small amount of additional data transmitted
per request), and it avoids having to re-send the payload if the cookie is
rejected.
It would also allow you to make this patch a lot simpler, I think, eg. you
wouldn't need the 'custom_headers_func'.
--
To view, visit http://gerrit.cloudera.org:8080/16660
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icb0bc6e0f58f236866ca9913a2e63d97d5148f51
Gerrit-Change-Number: 16660
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Jeges
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Tauber-Marshall
Gerrit-Comment-Date: Mon, 02 Nov 2020 17:39:12 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9180 (part 3): Remove legacy backend port
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16533 ) Change subject: IMPALA-9180 (part 3): Remove legacy backend port .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7602/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16533 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic6909a8da449b4d25ee98037b3eb459af4850dc6 Gerrit-Change-Number: 16533 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 02 Nov 2020 16:52:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9180 (part 3): Remove legacy backend port
Wenzhe Zhou has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16533 ) Change subject: IMPALA-9180 (part 3): Remove legacy backend port .. IMPALA-9180 (part 3): Remove legacy backend port The legacy Thrift based Impala internal service has been removed so the backend port 22000 can be freed up. This patch set flag be_port as a REMOVED_FLAG and all infrastructures around it are cleaned up. StatestoreSubscriber::subscriber_id is set as hostname + krpc_port. Testing: - Passed the exhaustive test. - Did manually testing on a dev cluster, verified that CM with fixing work with new version of Impala, which set subscriber_id as hostname + krpc_port. Verified the subscriber_ids of Impalad in Statestore Web Server. The health check of Impala cluster on CM worked as expected. - Passed CDH L0 test on cdpd-master branch. Change-Id: Ic6909a8da449b4d25ee98037b3eb459af4850dc6 --- M be/src/common/global-flags.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/data-stream-test.cc M be/src/runtime/exec-env.cc M be/src/runtime/krpc-data-stream-sender.cc M be/src/scheduling/schedule-state.h M be/src/scheduling/scheduler.cc M bin/start-impala-cluster.py M common/protobuf/admission_control_service.proto M common/protobuf/control_service.proto M common/protobuf/statestore_service.proto M infra/deploy/deploy.py M tests/common/impala_cluster.py M tests/common/impala_service.py M tests/custom_cluster/test_query_retries.py M tests/shell/test_shell_interactive.py 16 files changed, 27 insertions(+), 37 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16533/3 -- To view, visit http://gerrit.cloudera.org:8080/16533 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic6909a8da449b4d25ee98037b3eb459af4850dc6 Gerrit-Change-Number: 16533 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[native-toolchain-CR] IMPALA-10145,IMPALA-10299: Apply unicode decoding bug fixes to thrift-0.11.0
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16688 ) Change subject: IMPALA-10145,IMPALA-10299: Apply unicode decoding bug fixes to thrift-0.11.0 .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16688 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id16b04248f2db3033bef3ab26b7ba8205768c9af Gerrit-Change-Number: 16688 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Comment-Date: Mon, 02 Nov 2020 16:04:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16599 ) Change subject: IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7601/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5 Gerrit-Change-Number: 16599 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Mon, 02 Nov 2020 14:10:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16599 ) Change subject: IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables .. Patch Set 4: (3 comments) http://gerrit.cloudera.org:8080/#/c/16599/2/common/thrift/Frontend.thrift File common/thrift/Frontend.thrift: http://gerrit.cloudera.org:8080/#/c/16599/2/common/thrift/Frontend.thrift@292 PS2, Line 292: DESCRIBE HISTORY > DESCRIBE HISTORY Done http://gerrit.cloudera.org:8080/#/c/16599/2/fe/src/main/java/org/apache/impala/analysis/DescribeHistoryStmt.java File fe/src/main/java/org/apache/impala/analysis/DescribeHistoryStmt.java: http://gerrit.cloudera.org:8080/#/c/16599/2/fe/src/main/java/org/apache/impala/analysis/DescribeHistoryStmt.java@31 PS2, Line 31: DESCRIBE HISTORY s > DESCRIBE HISTORY Done http://gerrit.cloudera.org:8080/#/c/16599/2/fe/src/main/java/org/apache/impala/service/Frontend.java File fe/src/main/java/org/apache/impala/service/Frontend.java: http://gerrit.cloudera.org:8080/#/c/16599/2/fe/src/main/java/org/apache/impala/service/Frontend.java@1077 PS2, Line 1077: DESCRIBE HISTORY q > DESCRIBE HISTORY Done -- To view, visit http://gerrit.cloudera.org:8080/16599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5 Gerrit-Change-Number: 16599 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Mon, 02 Nov 2020 13:58:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16599 ) Change subject: IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables .. Patch Set 4: (2 comments) PS4 is a rebase with master http://gerrit.cloudera.org:8080/#/c/16599/2/be/src/service/client-request-state.cc File be/src/service/client-request-state.cc: http://gerrit.cloudera.org:8080/#/c/16599/2/be/src/service/client-request-state.cc@426 PS2, Line 426: DESCRIBE_HISTORY: > Should be DESCRIBE_HISTORY Done http://gerrit.cloudera.org:8080/#/c/16599/2/be/src/service/frontend.h File be/src/service/frontend.h: http://gerrit.cloudera.org:8080/#/c/16599/2/be/src/service/frontend.h@99 PS2, Line 99: Call > nit: typo Done -- To view, visit http://gerrit.cloudera.org:8080/16599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5 Gerrit-Change-Number: 16599 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Mon, 02 Nov 2020 13:54:07 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables
Hello Zoltan Borok-Nagy, wangsheng, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16599 to look at the new patch set (#4). Change subject: IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables .. IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables The DESCRIBE HISTORY works for Iceberg tables and displays the snapshot history of the table. An example output: DESCRIBE HISTORY iceberg_multi_snapshots; ++-+-+-+ | creation_time | snapshot_id | parent_id | is_current_ancestor | ++-+-+-+ | 2020-10-13 14:01:07.234000 | 4400379706200951771 | NULL| TRUE | | 2020-10-13 14:01:19.307000 | 4221472712544505868 | 4400379706200951771 | TRUE | ++-+-+-+ The purpose here was to have similar output with this new feature as what SparkSql returns for "SELECT * from tablename.history". See "History" section of https://iceberg.apache.org/spark/#inspecting-tables Testing: - iceberg-negative.test was extended to check that DESCRIBE HISTORY is not applicable for non-Iceberg tables. - iceberg-table-history.test: Covers basic usage of DESCRIBE HISTORY. Tests on tables created with Impala and also with Spark. Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5 --- M be/src/service/client-request-state.cc M be/src/service/frontend.cc M be/src/service/frontend.h M common/thrift/Frontend.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java A fe/src/main/java/org/apache/impala/analysis/DescribeHistoryStmt.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java M fe/src/main/jflex/sql-scanner.flex M testdata/data/README A testdata/data/iceberg_test/iceberg_multi_snapshots/data/2b4c7b0f52b60a94-5308a4d8_52928493_data.0.parq A testdata/data/iceberg_test/iceberg_multi_snapshots/data/97455b925ff67dcf-b56951ca_1396743290_data.0.parq A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/559bbd83-47bf-420e-bd3b-0800ca89c81c-m0.avro A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/aa86e03c-db78-4ff2-afbd-905cbab0f4b7-m0.avro A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/snap-1034149846385246343-1-559bbd83-47bf-420e-bd3b-0800ca89c81c.avro A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/snap-1860064961840897739-1-aa86e03c-db78-4ff2-afbd-905cbab0f4b7.avro A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/v1.metadata.json A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/v2.metadata.json A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/v3.metadata.json A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/version-hint.text M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test A testdata/workloads/functional-query/queries/QueryTest/iceberg-table-history.test M tests/query_test/test_iceberg.py 26 files changed, 403 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/16599/4 -- To view, visit http://gerrit.cloudera.org:8080/16599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5 Gerrit-Change-Number: 16599 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng
