[native-toolchain-CR] IMPALA-10145,IMPALA-10299: Apply unicode decoding bug fixes to thrift-0.11.0
Quanlong Huang has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/16688 )
Change subject: IMPALA-10145,IMPALA-10299: Apply unicode decoding bug fixes to
thrift-0.11.0
..
IMPALA-10145,IMPALA-10299: Apply unicode decoding bug fixes to thrift-0.11.0
After we bump the impala-shell dependent thrift version to 0.11.0, we
hit some bugs in decoding malformed utf8 characters, which crash the
impala-shell or cause it hanging forever. Before we bump the thrift
version, impala-shell is able to print incomplete utf8 characters as
some replaced utf8 symbols, e.g.
impala-shell> select substr("引擎", 1, 4);
引�
impala-shell> select unhex("aa");
�
The cause is that thrift changes its internal strings representation
from bytes to unicode after 0.10 (THRIFT-3503) to support Python3, which
follows the "unicode sandwich" rule -- namely "bytes on the outside,
unicode on the inside, encode/decode at the edges". However, the error
handling method is not specified so we hit the decoding error. We need
patches of THRIFT-2087 and THRIFT-5303 to improve its robustness.
THRIFT-5303 is enough to resolve the issue we hitted since we mostly use
the _fast_decode code path. Backporting THRIFT-2087 as well in case we
use the normal decoding code path somewhere.
Tests:
- Verify the issue is resolved after bumping the impala-shell dependent
thrift version to 0.11.0-p4.
Change-Id: Id16b04248f2db3033bef3ab26b7ba8205768c9af
Reviewed-on: http://gerrit.cloudera.org:8080/16688
Reviewed-by: Csaba Ringhofer
Tested-by: Quanlong Huang
---
M buildall.sh
A
source/thrift/thrift-0.11.0-patches/0003-THRIFT-2087-Python-compiler-replace-non-utf-8-char-w.patch
A
source/thrift/thrift-0.11.0-patches/0004-THRIFT-5303-Fix-missing-error-handling-in-using-PyUn.patch
3 files changed, 55 insertions(+), 1 deletion(-)
Approvals:
Csaba Ringhofer: Looks good to me, approved
Quanlong Huang: Verified
--
To view, visit http://gerrit.cloudera.org:8080/16688
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Id16b04248f2db3033bef3ab26b7ba8205768c9af
Gerrit-Change-Number: 16688
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Quanlong Huang
[native-toolchain-CR] IMPALA-10145,IMPALA-10299: Apply unicode decoding bug fixes to thrift-0.11.0
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/16688 ) Change subject: IMPALA-10145,IMPALA-10299: Apply unicode decoding bug fixes to thrift-0.11.0 .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16688 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id16b04248f2db3033bef3ab26b7ba8205768c9af Gerrit-Change-Number: 16688 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 04 Nov 2020 07:23:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. IMPALA-9864: Produce a minidump when TestValidateMetrics fails After running end-to-end tests, run-tests.py runs verifiers to check that a set of metrics are zero. When this fails, it can indicate a hung query fragment or other resource leak (see IMPALA-9842 for example). To track this down, it is useful to have a minidump, so this adds a step to have every Impalad/Catalogd generate a minidump (by sending SIGUSR1) when we hit the timeout. Also, the current error message dumps a bunch of unformatted JSON from our Web UI. This is hard to read and painful to cut/paste. This now dumps that JSON to files in a diagnostic directory under the logs directory. The JSON is formatted in a readable way. These files would be preserved along with the rest of the logs directory for automated runs. The new error message looks like this: E AssertionError: Metric impala-server.num-queries-registered did not reach value 0 in 60s. E Dumping debug webpages in JSON format... E Dumped memz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20201103_13:51:02/json/memz.json E Dumped metrics JSON to $IMPALA_HOME/logs/metric_timeout_diags_20201103_13:51:02/json/metrics.json E Dumped queries JSON to $IMPALA_HOME/logs/metric_timeout_diags_20201103_13:51:02/json/queries.json E Dumped sessions JSON to $IMPALA_HOME/logs/metric_timeout_diags_20201103_13:51:02/json/sessions.json E Dumped threadz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20201103_13:51:02/json/threadz.json E Dumped rpcz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20201103_13:51:02/json/rpcz.json E Dumping minidumps for impalads/catalogds... E Dumped minidump for Impalad PID 2709 E Dumped minidump for Impalad PID 2714 E Dumped minidump for Impalad PID 2721 E Dumped minidump for Catalogd PID 2627 This also fixes various flake8 errors (unnecessary imports, etc), so now impala_service.py is flake8 clean. Testing: - Tried out the dump function on my developer machine - Verified the minidumps exist - Verified the JSON is readable Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Reviewed-on: http://gerrit.cloudera.org:8080/16690 Reviewed-by: Qifan Chen Reviewed-by: Csaba Ringhofer Tested-by: Impala Public Jenkins --- M tests/common/impala_service.py 1 file changed, 89 insertions(+), 21 deletions(-) Approvals: Qifan Chen: Looks good to me, but someone else must approve Csaba Ringhofer: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Wed, 04 Nov 2020 03:31:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10305: Sync Kudu's FIPS compliant changes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16684 ) Change subject: IMPALA-10305: Sync Kudu's FIPS compliant changes .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7613/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16684 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1aa5e69bf8470e3cfb316e8b6354ad29c28223f4 Gerrit-Change-Number: 16684 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 04 Nov 2020 02:29:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10305: Sync Kudu's FIPS compliant changes
Wenzhe Zhou has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/16684 ) Change subject: IMPALA-10305: Sync Kudu's FIPS compliant changes .. IMPALA-10305: Sync Kudu's FIPS compliant changes kudu-3210 added FIPS compliant changes. This patch ports the following patches for kudu-3210 into Impala source tree: http://gerrit.cloudera.org:8080/16631/ http://gerrit.cloudera.org:8080/16657/ http://gerrit.cloudera.org:8080/16658/ http://gerrit.cloudera.org:8080/16659/ Testing: - Passed exhausive tests. Change-Id: I1aa5e69bf8470e3cfb316e8b6354ad29c28223f4 --- M be/src/kudu/security/CMakeLists.txt M be/src/kudu/security/ca/cert_management-test.cc M be/src/kudu/security/crypto.cc M be/src/kudu/security/openssl_util.cc M be/src/kudu/security/openssl_util.h M be/src/kudu/security/tls_context.cc M be/src/kudu/security/tls_handshake.cc M be/src/kudu/security/token-test.cc M be/src/kudu/util/flags.cc M be/src/kudu/util/flags.h M be/src/kudu/util/test_util.cc M be/src/kudu/util/test_util.h 12 files changed, 187 insertions(+), 73 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/16684/4 -- To view, visit http://gerrit.cloudera.org:8080/16684 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1aa5e69bf8470e3cfb316e8b6354ad29c28223f4 Gerrit-Change-Number: 16684 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-10305: Sync Kudu's FIPS compliant changes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16684 ) Change subject: IMPALA-10305: Sync Kudu's FIPS compliant changes .. Patch Set 3: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/7612/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/16684 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1aa5e69bf8470e3cfb316e8b6354ad29c28223f4 Gerrit-Change-Number: 16684 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 04 Nov 2020 02:04:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10305: Sync Kudu's FIPS compliant changes
Wenzhe Zhou has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16684 ) Change subject: IMPALA-10305: Sync Kudu's FIPS compliant changes .. IMPALA-10305: Sync Kudu's FIPS compliant changes kudu-3210 added FIPS compliant changes. This patch ports the following patches for kudu-3210 into Impala source tree: http://gerrit.cloudera.org:8080/16631/ http://gerrit.cloudera.org:8080/16657/ http://gerrit.cloudera.org:8080/16658/ http://gerrit.cloudera.org:8080/16659/ Testing: - Passed exhausive tests. Change-Id: I1aa5e69bf8470e3cfb316e8b6354ad29c28223f4 --- M be/src/kudu/security/CMakeLists.txt M be/src/kudu/security/ca/cert_management-test.cc M be/src/kudu/security/crypto.cc M be/src/kudu/security/openssl_util.cc M be/src/kudu/security/openssl_util.h M be/src/kudu/security/tls_context.cc M be/src/kudu/security/tls_handshake.cc M be/src/kudu/security/token-test.cc M be/src/kudu/util/flags.cc M be/src/kudu/util/flags.h M be/src/kudu/util/test_util.cc M be/src/kudu/util/test_util.h 12 files changed, 187 insertions(+), 73 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/16684/3 -- To view, visit http://gerrit.cloudera.org:8080/16684 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1aa5e69bf8470e3cfb316e8b6354ad29c28223f4 Gerrit-Change-Number: 16684 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 11: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6630/ -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 11 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 04 Nov 2020 01:20:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 11: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 11 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 04 Nov 2020 00:16:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6671: Skip locked tables from topic updates
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16549 ) Change subject: IMPALA-6671: Skip locked tables from topic updates .. Patch Set 10: I took a quick look over the changes and I think this makes sense to me, thanks for working on it. I need to make a bit of time to do a detailed review. -- To view, visit http://gerrit.cloudera.org:8080/16549 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic657b96edbcdc94c6b906e7ca59291f4e4715655 Gerrit-Change-Number: 16549 Gerrit-PatchSet: 10 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 04 Nov 2020 00:15:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 3: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/16690/3/tests/common/impala_service.py File tests/common/impala_service.py: http://gerrit.cloudera.org:8080/#/c/16690/3/tests/common/impala_service.py@181 PS3, Line 181: "-f", hmm, this sounds weird, I don't remember such issues on my desktop -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 03 Nov 2020 22:57:38 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 3: Code-Review+1 (2 comments) Thanks a lot for addressing the comments! http://gerrit.cloudera.org:8080/#/c/16690/2/tests/common/impala_service.py File tests/common/impala_service.py: http://gerrit.cloudera.org:8080/#/c/16690/2/tests/common/impala_service.py@139 PS2, Line 139: > Changed this to used datetime, so the directory would have a name like: Done http://gerrit.cloudera.org:8080/#/c/16690/2/tests/common/impala_service.py@159 PS2, Line 159: "metric > Statestore is usually less interesting, because it is mostly a publisher/su Done -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 03 Nov 2020 22:28:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7611/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 03 Nov 2020 22:27:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6631/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 03 Nov 2020 22:06:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 3: (3 comments) http://gerrit.cloudera.org:8080/#/c/16690/2/tests/common/impala_service.py File tests/common/impala_service.py: http://gerrit.cloudera.org:8080/#/c/16690/2/tests/common/impala_service.py@139 PS2, Line 139: > It may help diagnose later on by putting the date and the timestamp (in a f Changed this to used datetime, so the directory would have a name like: metric_timeout_diags_20201103_13:51:32 http://gerrit.cloudera.org:8080/#/c/16690/2/tests/common/impala_service.py@140 PS2, Line 140: Impala processes (impalad, catalogd, statestored) have a signal handler for > optional: I think it would improve readability to extract some parts to sep I split out the logic for dumping JSON to a file and the logic for requesting a minidump. One hurdle is that I'm constructing the assert message as I go, so I'm leaving this as one big function for the time being. http://gerrit.cloudera.org:8080/#/c/16690/2/tests/common/impala_service.py@159 PS2, Line 159: "metric > May be useful to get a minidump for statestored? I was not be able to see c Statestore is usually less interesting, because it is mostly a publisher/subscriber daemon. There aren't per-query resources on statestored. I changed the code to also dump catalogd. Interestingly, pgrep has trouble finding it unless I look at the whole commandline (e.g. pgrep -f). Seems like a minor bug. -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 03 Nov 2020 22:06:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Hello Qifan Chen, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16690 to look at the new patch set (#3). Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. IMPALA-9864: Produce a minidump when TestValidateMetrics fails After running end-to-end tests, run-tests.py runs verifiers to check that a set of metrics are zero. When this fails, it can indicate a hung query fragment or other resource leak (see IMPALA-9842 for example). To track this down, it is useful to have a minidump, so this adds a step to have every Impalad/Catalogd generate a minidump (by sending SIGUSR1) when we hit the timeout. Also, the current error message dumps a bunch of unformatted JSON from our Web UI. This is hard to read and painful to cut/paste. This now dumps that JSON to files in a diagnostic directory under the logs directory. The JSON is formatted in a readable way. These files would be preserved along with the rest of the logs directory for automated runs. The new error message looks like this: E AssertionError: Metric impala-server.num-queries-registered did not reach value 0 in 60s. E Dumping debug webpages in JSON format... E Dumped memz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20201103_13:51:02/json/memz.json E Dumped metrics JSON to $IMPALA_HOME/logs/metric_timeout_diags_20201103_13:51:02/json/metrics.json E Dumped queries JSON to $IMPALA_HOME/logs/metric_timeout_diags_20201103_13:51:02/json/queries.json E Dumped sessions JSON to $IMPALA_HOME/logs/metric_timeout_diags_20201103_13:51:02/json/sessions.json E Dumped threadz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20201103_13:51:02/json/threadz.json E Dumped rpcz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20201103_13:51:02/json/rpcz.json E Dumping minidumps for impalads/catalogds... E Dumped minidump for Impalad PID 2709 E Dumped minidump for Impalad PID 2714 E Dumped minidump for Impalad PID 2721 E Dumped minidump for Catalogd PID 2627 This also fixes various flake8 errors (unnecessary imports, etc), so now impala_service.py is flake8 clean. Testing: - Tried out the dump function on my developer machine - Verified the minidumps exist - Verified the JSON is readable Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d --- M tests/common/impala_service.py 1 file changed, 89 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/16690/3 -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 11: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 11 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 21:34:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 11: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6630/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 11 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 21:34:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6671: Skip locked tables from topic updates
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16549 ) Change subject: IMPALA-6671: Skip locked tables from topic updates .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7610/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16549 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic657b96edbcdc94c6b906e7ca59291f4e4715655 Gerrit-Change-Number: 16549 Gerrit-PatchSet: 10 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 03 Nov 2020 20:57:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6671: Skip locked tables from topic updates
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16549 ) Change subject: IMPALA-6671: Skip locked tables from topic updates .. Patch Set 10: (3 comments) http://gerrit.cloudera.org:8080/#/c/16549/10/tests/metadata/test_topic_update_frequency.py File tests/metadata/test_topic_update_frequency.py: http://gerrit.cloudera.org:8080/#/c/16549/10/tests/metadata/test_topic_update_frequency.py@50 PS10, Line 50: l flake8: E501 line too long (146 > 90 characters) http://gerrit.cloudera.org:8080/#/c/16549/10/tests/metadata/test_topic_update_frequency.py@109 PS10, Line 109: l flake8: E501 line too long (145 > 90 characters) http://gerrit.cloudera.org:8080/#/c/16549/10/tests/metadata/test_topic_update_frequency.py@177 PS10, Line 177: f flake8: E126 continuation line over-indented for hanging indent -- To view, visit http://gerrit.cloudera.org:8080/16549 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic657b96edbcdc94c6b906e7ca59291f4e4715655 Gerrit-Change-Number: 16549 Gerrit-PatchSet: 10 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 03 Nov 2020 20:37:41 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6671: Skip locked tables from topic updates
Vihang Karajgaonkar has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/16549 ) Change subject: IMPALA-6671: Skip locked tables from topic updates .. IMPALA-6671: Skip locked tables from topic updates This change adds a mechanism for topic-update thread to skip a table which is locked for more than a configurable interval from the topic updates. This is especially useful in scenarios where long running operations on a locked table (refresh, recover partitions, compute stats) block the topic update thread. This causes unrelated queries which are waiting on metadata via topic updates (catalog-v1 mode) to unnecessarily block. The ideal solution of this problem would be to make HdfsTable immutable so that there is no need for table lock. But that is large change and not easily portable to older releases of Impala. It would be taken up as a separate patch. This change introduces 2 new configurations for catalogd: 1. topic_update_tbl_max_wait_time_ms: This defines the maximum time in msecs the topic update thread waits on a locked table before skipping the table from that iteration of topic updates. The default value is 500. 2. catalog_max_lock_skipped_topic_updates: This defines the maximum number of distinct lock operations which are skipped by topic update thread due to lock contention. Once this limit is reached, topic update thread will block until it acquires the table lock and adds it to the updates. Testing: 1. Added a test case which introduces a simulated delay in a few potentially long running statements. This causes the table to be locked for a long time. The topic update thread skips that table from updates and unrelated queries are unblocked since they receive the required metadata from updates. 2. Added a test where multiple threads run blocking statements in a loop to stress the table lock. It makes sure that topic update thread is not starved and eventually blocks on table lock by hitting the limit defined by catalog_max_lock_skipped_topic_updates. 2. Ran exhaustive tests with default configurations. Change-Id: Ic657b96edbcdc94c6b906e7ca59291f4e4715655 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/analysis/CopyTestCaseStmt.java M fe/src/main/java/org/apache/impala/catalog/Catalog.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/TopicUpdateLog.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/catalog/CatalogObjectToFromThriftTest.java A tests/metadata/test_topic_update_frequency.py 13 files changed, 647 insertions(+), 149 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/16549/10 -- To view, visit http://gerrit.cloudera.org:8080/16549 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic657b96edbcdc94c6b906e7ca59291f4e4715655 Gerrit-Change-Number: 16549 Gerrit-PatchSet: 10 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 10: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 10 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 20:32:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10058: Use commit hash as version for Kudu java artifacts
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16686 ) Change subject: IMPALA-10058: Use commit hash as version for Kudu java artifacts .. IMPALA-10058: Use commit hash as version for Kudu java artifacts This uses a new version of the native toolchain where Kudu now uses the commit hash as the version for its jars. This means that IMPALA_KUDU_VERSION is the same as IMPALA_KUDU_JAVA_VERSION, so this consolidates everything to use IMPALA_KUDU_VERSION. This also eliminates SNAPSHOT versions for the Kudu jars. Kudu changed one error message, so this updates the impacted tests. Testing: - Ran a core job Change-Id: I1a6c9676f4521d6709393143d3e82533486164d3 Reviewed-on: http://gerrit.cloudera.org:8080/16686 Reviewed-by: Tim Armstrong Tested-by: Impala Public Jenkins --- M bin/impala-config.sh M java/pom.xml M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test M testdata/workloads/functional-query/queries/QueryTest/kudu_hms_alter.test 4 files changed, 11 insertions(+), 11 deletions(-) Approvals: Tim Armstrong: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16686 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I1a6c9676f4521d6709393143d3e82533486164d3 Gerrit-Change-Number: 16686 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[native-toolchain-CR] IMPALA-10058: Use commit hash as version for Kudu java artifacts
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16685 ) Change subject: IMPALA-10058: Use commit hash as version for Kudu java artifacts .. Patch Set 1: Verified+1 I ran a native toolchain build with this, then built Impala with it and ran a core job. Adding +1 verified. -- To view, visit http://gerrit.cloudera.org:8080/16685 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I352fdcb02ce3c11432cb5d6e7fdc564fe9b3c038 Gerrit-Change-Number: 16685 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 20:05:10 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-10058: Use commit hash as version for Kudu java artifacts
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16685 ) Change subject: IMPALA-10058: Use commit hash as version for Kudu java artifacts .. IMPALA-10058: Use commit hash as version for Kudu java artifacts This changes the Kudu build so that it no longer produces jars with SNAPSHOT versions. Before this change, the Kudu jars would have a version like 1.13.0-SNAPSHOT, and now it is just the commit hash. This should eliminate version conflicts where the solution is to clear the .m2 cache. This also bumps the Kudu commit hash to the latest. This is needed because existing Kudu toolchain directories have the old version format. Testing: - Ran an Impala build with a toolchain produced by this Change-Id: I352fdcb02ce3c11432cb5d6e7fdc564fe9b3c038 Reviewed-on: http://gerrit.cloudera.org:8080/16685 Reviewed-by: Tim Armstrong Tested-by: Joe McDonnell --- M buildall.sh M source/kudu/build.sh 2 files changed, 6 insertions(+), 1 deletion(-) Approvals: Tim Armstrong: Looks good to me, approved Joe McDonnell: Verified -- To view, visit http://gerrit.cloudera.org:8080/16685 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I352fdcb02ce3c11432cb5d6e7fdc564fe9b3c038 Gerrit-Change-Number: 16685 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[native-toolchain-CR] IMPALA-10058: Use commit hash as version for Kudu java artifacts
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16685 ) Change subject: IMPALA-10058: Use commit hash as version for Kudu java artifacts .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16685 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I352fdcb02ce3c11432cb5d6e7fdc564fe9b3c038 Gerrit-Change-Number: 16685 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 19:55:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16676 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7609/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 Gerrit-Change-Number: 16676 Gerrit-PatchSet: 3 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 03 Nov 2020 17:53:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16676 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 3: Code-Review+2 Thanks for fixing this! -- To view, visit http://gerrit.cloudera.org:8080/16676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 Gerrit-Change-Number: 16676 Gerrit-PatchSet: 3 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 03 Nov 2020 17:43:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16676 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/16676/2/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test File testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test: http://gerrit.cloudera.org:8080/#/c/16676/2/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test@7 PS2, Line 7: set mem_limit=168m; : set num_scanner_threads=1; : select * : from tpch_parquet.lin > From a purely style note, I think this comment is not necessary. The commen Done -- To view, visit http://gerrit.cloudera.org:8080/16676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 Gerrit-Change-Number: 16676 Gerrit-PatchSet: 3 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 03 Nov 2020 17:31:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Qifan Chen has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16676 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit This fix improved the robustness of the test by lowering the memory limit from 170MB to 168MB. In 10 local tests, the memory allocation shortage at EXCHANGE_NODE was as low as 8KB without exceeding the limit of 170MB. Testing: 1. Unit testing with 200 test iterations. Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 --- M testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/16676/3 -- To view, visit http://gerrit.cloudera.org:8080/16676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 Gerrit-Change-Number: 16676 Gerrit-PatchSet: 3 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-10279: Import CPC functionality from DataSketches
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16645 ) Change subject: IMPALA-10279: Import CPC functionality from DataSketches .. Patch Set 7: (3 comments) http://gerrit.cloudera.org:8080/#/c/16645/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16645/6//COMMIT_MSG@11 PS6, Line 11: : First, I updated our existing snapshot of DataSketches to the : following commit:2b84e213067b681b696ec883d245ddf911790ff2 > All change files have been updated I checked manually the commits between the git hash we used previously and the one that you bumped our snapshot to. Apparently there are some changes that went into the repo since the 2 snapshots but aren't visible here in your patch. (at least I don't see the files that were changed in the intermittent commits being changed here) Example commits where either kll, hll or common was changed: https://github.com/apache/incubator-datasketches-cpp/commit/7cafbef53f475b966c1984dce395a940c6cc93f5 https://github.com/apache/incubator-datasketches-cpp/commit/3135dc4a71af05150bd8864aaf0ff803532a554f https://github.com/apache/incubator-datasketches-cpp/commit/608a741a4d0db9078feddc165cb080f5bb43e351 https://github.com/apache/incubator-datasketches-cpp/commit/fa9516767ffece0c8aa3246302135d6614ae2deb https://github.com/apache/incubator-datasketches-cpp/commit/c5be98a81c5947e5100868b90fb0f50aba4a9334 Are you sure all the files here are in the Impala repo (common/ hll/ kll) are updated to the desired snapshot? http://gerrit.cloudera.org:8080/#/c/16645/6/be/src/exprs/datasketches-test.cc File be/src/exprs/datasketches-test.cc: http://gerrit.cloudera.org:8080/#/c/16645/6/be/src/exprs/datasketches-test.cc@114 PS6, Line 114: } > The test code comes from the official example: thx! I just wanted to make sure that this result is deterministic and is the very same regardless of how many times we run it. Note, KLL wasn't deterministic and I had to check if the result is in a given range. http://gerrit.cloudera.org:8080/#/c/16645/7/be/src/thirdparty/datasketches/README.md File be/src/thirdparty/datasketches/README.md: http://gerrit.cloudera.org:8080/#/c/16645/7/be/src/thirdparty/datasketches/README.md@16 PS7, Line 16: https://github.com/apache/incubator-datasketches-cpp/ Could you provide a link here that points to the particular snapshot that you are upgrading to? e.g. https://github.com/apache/incubator-datasketches-cpp/tree/2b84e213067b681b696ec883d245ddf911790ff2 -- To view, visit http://gerrit.cloudera.org:8080/16645 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1e8d0c2e80df95fa84af82c64d493df9bbb34a8c Gerrit-Change-Number: 16645 Gerrit-PatchSet: 7 Gerrit-Owner: Fucun Chu Gerrit-Reviewer: Fucun Chu Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 03 Nov 2020 16:33:09 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16599 ) Change subject: IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7608/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5 Gerrit-Change-Number: 16599 Gerrit-PatchSet: 5 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Tue, 03 Nov 2020 16:09:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10306: [DOCS] add note about TZ offset
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16689 ) Change subject: IMPALA-10306: [DOCS] add note about TZ offset .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/16689/1/docs/topics/impala_datetime_functions.xml File docs/topics/impala_datetime_functions.xml: http://gerrit.cloudera.org:8080/#/c/16689/1/docs/topics/impala_datetime_functions.xml@1416 PS1, Line 1416: /> > There is a similar usage note already in place here (timezone_conversion_ca Shajini, if you plan to add your comment to a common place that covers all the timestamp conversion functions as Tamas suggests than please ignore my comment above about bigint inputs and such :) because that is valid only for this particular function. -- To view, visit http://gerrit.cloudera.org:8080/16689 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I693d15f1230dd7eebcbf2a16657a3850943749e1 Gerrit-Change-Number: 16689 Gerrit-PatchSet: 1 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 03 Nov 2020 16:08:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10306: [DOCS] add note about TZ offset
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16689 ) Change subject: IMPALA-10306: [DOCS] add note about TZ offset .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/16689/1/docs/topics/impala_datetime_functions.xml File docs/topics/impala_datetime_functions.xml: http://gerrit.cloudera.org:8080/#/c/16689/1/docs/topics/impala_datetime_functions.xml@1390 PS1, Line 1390: TZ > To keep the document's consistency, could you change "TZ" to "timezone". I think the issues with FROM_UNIXTIME() is not that Impala doesn't store timezone information internally, but that this particular function receives a bigint number (seconds since epoch) as an input and that doesn't hold any TZ info. This comment you wrote would be a good one for any other timstamp conversion functions that receive a timestamp and returns a formatted string. -- To view, visit http://gerrit.cloudera.org:8080/16689 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I693d15f1230dd7eebcbf2a16657a3850943749e1 Gerrit-Change-Number: 16689 Gerrit-PatchSet: 1 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 03 Nov 2020 16:06:39 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables
Gabor Kaszab has posted comments on this change. (
http://gerrit.cloudera.org:8080/16599 )
Change subject: IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables
..
Patch Set 5:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/16599/3/fe/src/main/jflex/sql-scanner.flex
File fe/src/main/jflex/sql-scanner.flex:
http://gerrit.cloudera.org:8080/#/c/16599/3/fe/src/main/jflex/sql-scanner.flex@152
PS3, Line 152: keywordMap.put("hudiparquet",
SqlParserSymbols.KW_HUDIPARQUET);
> We should be careful with adding new reserved words (it's best to avoid it)
I managed to address this. There was some conflict with the DESCRIBE (table)
statement and I had to merge them together and refactor a bit so that the
parser could accept everything as expected.
http://gerrit.cloudera.org:8080/#/c/16599/5/testdata/workloads/functional-query/queries/QueryTest/iceberg-table-history.test
File
testdata/workloads/functional-query/queries/QueryTest/iceberg-table-history.test:
http://gerrit.cloudera.org:8080/#/c/16599/5/testdata/workloads/functional-query/queries/QueryTest/iceberg-table-history.test@17
PS5, Line 17: '2020-08-31 07:58:08.44'
Apparently, when I run these tests in jenkins this check fails because some
kind of time adjustment happens and "set timezone" above has no effect on the
result.
--
To view, visit http://gerrit.cloudera.org:8080/16599
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5
Gerrit-Change-Number: 16599
Gerrit-PatchSet: 5
Gerrit-Owner: Gabor Kaszab
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Reviewer: wangsheng
Gerrit-Comment-Date: Tue, 03 Nov 2020 16:00:28 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 2: (2 comments) Looks good to me! http://gerrit.cloudera.org:8080/#/c/16690/2/tests/common/impala_service.py File tests/common/impala_service.py: http://gerrit.cloudera.org:8080/#/c/16690/2/tests/common/impala_service.py@139 PS2, Line 139: format(int(time())) It may help diagnose later on by putting the date and the timestamp (in a format close to what is shown in *INFO etc log files) into the path. Example: $IMPALA_HOME/logs/metric_timeout_diags_10302020_09:00:39.258774/ >From impalad.INFO I1103 09:00:39.258778 227319 runtime-state.cc:196] 70495a87ff023170:975863720003] Error from query 70495a87ff023170:97586372: Row with null value violates nullability constraint on table 'impa la::default.table_kudu'. http://gerrit.cloudera.org:8080/#/c/16690/2/tests/common/impala_service.py@159 PS2, Line 159: impalad May be useful to get a minidump for statestored? I was not be able to see catalogd running on my box, although the start up message lists it. -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 03 Nov 2020 15:51:30 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables
Hello Zoltan Borok-Nagy, wangsheng, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16599 to look at the new patch set (#5). Change subject: IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables .. IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables The DESCRIBE HISTORY works for Iceberg tables and displays the snapshot history of the table. An example output: DESCRIBE HISTORY iceberg_multi_snapshots; ++-+-+-+ | creation_time | snapshot_id | parent_id | is_current_ancestor | ++-+-+-+ | 2020-10-13 14:01:07.234000 | 4400379706200951771 | NULL| TRUE | | 2020-10-13 14:01:19.307000 | 4221472712544505868 | 4400379706200951771 | TRUE | ++-+-+-+ The purpose here was to have similar output with this new feature as what SparkSql returns for "SELECT * from tablename.history". See "History" section of https://iceberg.apache.org/spark/#inspecting-tables Testing: - iceberg-negative.test was extended to check that DESCRIBE HISTORY is not applicable for non-Iceberg tables. - iceberg-table-history.test: Covers basic usage of DESCRIBE HISTORY. Tests on tables created with Impala and also with Spark. Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5 --- M be/src/service/client-request-state.cc M be/src/service/frontend.cc M be/src/service/frontend.h M common/thrift/Frontend.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java A fe/src/main/java/org/apache/impala/analysis/DescribeHistoryStmt.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M testdata/data/README A testdata/data/iceberg_test/iceberg_multi_snapshots/data/2b4c7b0f52b60a94-5308a4d8_52928493_data.0.parq A testdata/data/iceberg_test/iceberg_multi_snapshots/data/97455b925ff67dcf-b56951ca_1396743290_data.0.parq A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/559bbd83-47bf-420e-bd3b-0800ca89c81c-m0.avro A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/aa86e03c-db78-4ff2-afbd-905cbab0f4b7-m0.avro A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/snap-1034149846385246343-1-559bbd83-47bf-420e-bd3b-0800ca89c81c.avro A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/snap-1860064961840897739-1-aa86e03c-db78-4ff2-afbd-905cbab0f4b7.avro A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/v1.metadata.json A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/v2.metadata.json A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/v3.metadata.json A testdata/data/iceberg_test/iceberg_multi_snapshots/metadata/version-hint.text M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test A testdata/workloads/functional-query/queries/QueryTest/iceberg-table-history.test M tests/query_test/test_iceberg.py 26 files changed, 421 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/16599/5 -- To view, visit http://gerrit.cloudera.org:8080/16599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5 Gerrit-Change-Number: 16599 Gerrit-PatchSet: 5 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7607/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 10 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 14:45:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. IMPALA-3816: Codegen perf critical loops in Sorter This fix added the functionality to codegen recursive method Sorter::TupleSorter::SortHelper() in sorter, which improves the performance for both the sort and the partial sort operators. In one unit test to order 7300 rows from table functional.alltypes, the speedup of the code-gen version over non-code-gen version of the method is around 65%. In another unit test to partially order 2880404 rows, the speedup is around 61%. Testing: 1. Unit testing; 2. Ran Core tests successfully. Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b --- M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter-ir.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 9 files changed, 133 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16621/10 -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 10 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-3816: Codegen perf critical loops in Sorter
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16621 ) Change subject: IMPALA-3816: Codegen perf critical loops in Sorter .. Patch Set 10: (1 comment) http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc File be/src/runtime/sorter.cc: http://gerrit.cloudera.org:8080/#/c/16621/8/be/src/runtime/sorter.cc@1215 PS8, Line 1215: LlvmCodeGen* codegen = state->codegen(); > Did some additional testing with instrumentation and found the return addre Added the logic to replace the call sites (two recursive calls) within the LLVMed SorterHelp with good improvement (~65% and ~61% over non-LLVM version, respectively). -- To view, visit http://gerrit.cloudera.org:8080/16621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie08137449d4a7b554ca8b8650260f8bd72e0a81b Gerrit-Change-Number: 16621 Gerrit-PatchSet: 10 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 03 Nov 2020 14:25:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10306: [DOCS] add note about TZ offset
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/16689 ) Change subject: IMPALA-10306: [DOCS] add note about TZ offset .. Patch Set 1: (3 comments) Hi Shajini, thank you for the change. I have left some comments inline. http://gerrit.cloudera.org:8080/#/c/16689/1/docs/topics/impala_datetime_functions.xml File docs/topics/impala_datetime_functions.xml: http://gerrit.cloudera.org:8080/#/c/16689/1/docs/topics/impala_datetime_functions.xml@1390 PS1, Line 1390: TZ To keep the document's consistency, could you change "TZ" to "timezone". http://gerrit.cloudera.org:8080/#/c/16689/1/docs/topics/impala_datetime_functions.xml@1416 PS1, Line 1416: /> There is a similar usage note already in place here (timezone_conversion_caveat), I think the 'Usage notes' would be a better place for the new paragraph. Also instead of it could be . http://gerrit.cloudera.org:8080/#/c/16689/1/docs/topics/impala_datetime_functions.xml@1420 PS1, Line 1420: nit: empty tab -- To view, visit http://gerrit.cloudera.org:8080/16689 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I693d15f1230dd7eebcbf2a16657a3850943749e1 Gerrit-Change-Number: 16689 Gerrit-PatchSet: 1 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 03 Nov 2020 10:56:27 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16599 ) Change subject: IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables .. Patch Set 4: I'm still working on not introducing a new keyword for "HISTORY". Currently there is some difficulties not to collide with DESCRIBE tablename in the .cup file generation. Let me see if I an find a workaround for this. -- To view, visit http://gerrit.cloudera.org:8080/16599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5 Gerrit-Change-Number: 16599 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Tue, 03 Nov 2020 09:44:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9864: Produce a minidump when TestValidateMetrics fails
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16690 ) Change subject: IMPALA-9864: Produce a minidump when TestValidateMetrics fails .. Patch Set 2: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/16690/2/tests/common/impala_service.py File tests/common/impala_service.py: http://gerrit.cloudera.org:8080/#/c/16690/2/tests/common/impala_service.py@140 PS2, Line 140: if not os.path.exists(diag_dir): optional: I think it would improve readability to extract some parts to separate functions, e.g. dump_debug_pages(self, diag_dir) and maybe trigger_impalad_minidumps() -- To view, visit http://gerrit.cloudera.org:8080/16690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16d26052d0664ee0b115e3611cd96047d8ada19d Gerrit-Change-Number: 16690 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 03 Nov 2020 08:56:02 + Gerrit-HasComments: Yes
