[Impala-ASF-CR] IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views
Fredy Wijaya has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/10132 ) Change subject: IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views .. IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views This patch fixes an issue where in ALTER VIEW the storage descriptor is created with a new instance instead of reusing the existing storage descriptor. This causes an issue where some HMS attributes become nullable causing a NullPointerException. The patch also differentiates between updating view attributes for CREATE VIEW and ALTER VIEW. Testing: - Ran all front-end tests - Added a new end-to-end test - Ran the all end-to-end metadata tests Change-Id: Ica2fb0c4f4b09cdf36eeb4911a1cbe7e98381d9e --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/metadata/test_ddl.py 2 files changed, 46 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/32/10132/5 -- To view, visit http://gerrit.cloudera.org:8080/10132 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ica2fb0c4f4b09cdf36eeb4911a1cbe7e98381d9e Gerrit-Change-Number: 10132 Gerrit-PatchSet: 5 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Fredy Wijaya
[Impala-ASF-CR] IMPALA-5690: Part 1: Rename ostream operators for thrift types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9168 ) Change subject: IMPALA-5690: Part 1: Rename ostream operators for thrift types .. Patch Set 15: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2344/ -- To view, visit http://gerrit.cloudera.org:8080/9168 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c303997411237e988ef960157f781776f6fcb60 Gerrit-Change-Number: 9168 Gerrit-PatchSet: 15 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 20 Apr 2018 06:33:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5690: Part 1: Rename ostream operators for thrift types
Tianyi Wang has posted comments on this change. ( http://gerrit.cloudera.org:8080/9168 ) Change subject: IMPALA-5690: Part 1: Rename ostream operators for thrift types .. Patch Set 15: Code-Review+2 rebase python fixes -- To view, visit http://gerrit.cloudera.org:8080/9168 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c303997411237e988ef960157f781776f6fcb60 Gerrit-Change-Number: 9168 Gerrit-PatchSet: 15 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 20 Apr 2018 06:33:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5893: Remove old kinit code for Impala 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9941 ) Change subject: IMPALA-5893: Remove old kinit code for Impala 3 .. Patch Set 6: Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2343/ -- To view, visit http://gerrit.cloudera.org:8080/9941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b Gerrit-Change-Number: 9941 Gerrit-PatchSet: 6 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Fri, 20 Apr 2018 06:23:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views
Fredy Wijaya has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/10132 ) Change subject: IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views .. IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views This patch fixes an issue where in ALTER VIEW the storage descriptor is created with a new instance instead of reusing the existing storage descriptor. This causes an issue where some HMS attributes become nullable causing a NullPointerException. The patch also differentiates between updating view attributes for CREATE VIEW and ALTER VIEW. Testing: - Ran all front-end tests - Added a new end-to-end test - Ran the all end-to-end metadata tests Change-Id: Ica2fb0c4f4b09cdf36eeb4911a1cbe7e98381d9e --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/metadata/test_ddl.py 2 files changed, 46 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/32/10132/4 -- To view, visit http://gerrit.cloudera.org:8080/10132 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ica2fb0c4f4b09cdf36eeb4911a1cbe7e98381d9e Gerrit-Change-Number: 10132 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Fredy Wijaya
[Impala-ASF-CR] IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views
Fredy Wijaya has posted comments on this change. (
http://gerrit.cloudera.org:8080/10132 )
Change subject: IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views
..
Patch Set 3:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/10132/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/10132/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1916
PS2, Line 1916: if (params.isSetComment() && params.getComment() != null) {
> extra space after &&
Done
http://gerrit.cloudera.org:8080/#/c/10132/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1935
PS2, Line 1935: if (params.isSetComment() && params.getComment() != null) {
> name and db is expected to be set right?
Done
http://gerrit.cloudera.org:8080/#/c/10132/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1937
PS2, Line 1937: }
> extra space after &&
Done
http://gerrit.cloudera.org:8080/#/c/10132/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1940
PS2, Line 1940: }
> combine with L1942
Done
http://gerrit.cloudera.org:8080/#/c/10132/2/tests/metadata/test_ddl.py
File tests/metadata/test_ddl.py:
http://gerrit.cloudera.org:8080/#/c/10132/2/tests/metadata/test_ddl.py@366
PS2, Line 366: # IMPALA-6896: Tests that altered views can be described by
all impalads.
> The message should describe what is being tested, and not just have the JIR
Done
http://gerrit.cloudera.org:8080/#/c/10132/2/tests/metadata/test_ddl.py@379
PS2, Line 379: for impalad in impalads[1:]:
> Instead of sleeping, let's run the loop below until we find "alltypesagg" i
Done
--
To view, visit http://gerrit.cloudera.org:8080/10132
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ica2fb0c4f4b09cdf36eeb4911a1cbe7e98381d9e
Gerrit-Change-Number: 10132
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Alex Behm
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Comment-Date: Fri, 20 Apr 2018 06:12:38 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views
Fredy Wijaya has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/10132 ) Change subject: IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views .. IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views This patch fixes an issue where in ALTER VIEW the storage descriptor is created with a new instance instead of reusing the existing storage descriptor. This causes an issue where some HMS attributes become nullable causing a NullPointerException. The patch also differentiates between updating view attributes for CREATE VIEW and ALTER VIEW. Testing: - Ran all front-end tests - Added a new end-to-end test - Ran the all end-to-end metadata tests Change-Id: Ica2fb0c4f4b09cdf36eeb4911a1cbe7e98381d9e --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/metadata/test_ddl.py 2 files changed, 47 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/32/10132/3 -- To view, visit http://gerrit.cloudera.org:8080/10132 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ica2fb0c4f4b09cdf36eeb4911a1cbe7e98381d9e Gerrit-Change-Number: 10132 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Fredy Wijaya
[Impala-ASF-CR] IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views
Alex Behm has posted comments on this change. (
http://gerrit.cloudera.org:8080/10132 )
Change subject: IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views
..
Patch Set 2:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/10132/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/10132/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1916
PS2, Line 1916: if (params.isSetComment() && params.getComment() != null) {
extra space after &&
http://gerrit.cloudera.org:8080/#/c/10132/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1935
PS2, Line 1935: view.setDbName(params.getView_name().getDb_name());
name and db is expected to be set right?
http://gerrit.cloudera.org:8080/#/c/10132/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1937
PS2, Line 1937: if (params.isSetComment() && params.getComment() != null) {
extra space after &&
http://gerrit.cloudera.org:8080/#/c/10132/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1940
PS2, Line 1940: StorageDescriptor sd = view.getSd();
combine with L1942
http://gerrit.cloudera.org:8080/#/c/10132/2/tests/metadata/test_ddl.py
File tests/metadata/test_ddl.py:
http://gerrit.cloudera.org:8080/#/c/10132/2/tests/metadata/test_ddl.py@366
PS2, Line 366: # IMPALA-6896: NPE in describe formatted on views.
The message should describe what is being tested, and not just have the JIRA
title. For example:
Tests that altered views can be described by all impalads.
http://gerrit.cloudera.org:8080/#/c/10132/2/tests/metadata/test_ddl.py@379
PS2, Line 379: time.sleep(5)
Instead of sleeping, let's run the loop below until we find "alltypesagg" in
the output of all the describes
--
To view, visit http://gerrit.cloudera.org:8080/10132
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ica2fb0c4f4b09cdf36eeb4911a1cbe7e98381d9e
Gerrit-Change-Number: 10132
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Alex Behm
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Comment-Date: Fri, 20 Apr 2018 05:27:52 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Add new performance test workload - tpcds-unmodified used by Impala Performance Tests
Nithya Janarthanan has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/9973 ) Change subject: Add new performance test workload - tpcds-unmodified used by Impala Performance Tests .. Add new performance test workload - tpcds-unmodified used by Impala Performance Tests Description: Impala versions prior to 2.5 didn't have Runtimefilters, which made TPC-DS queries run very slow, so queries under tpcds have explicit partition filters to workaround the limitation. Post Impala 2.5 adding tpcds-unmodified which has the unmodified version of the workload to provide more coverage Testing: Ran the performance tests using the new workload and all tests passed Change-Id: I3957621d88b80fffc8fc89fd8104a58137b86e92 --- A testdata/workloads/tpcds-unmodified/queries/tpcds-q1.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q10.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q11.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q12.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q13.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q14.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q15.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q16.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q17.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q18.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q19.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q2.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q20.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q21.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q22.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q23.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q25.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q26.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q27.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q28.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q29.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q3.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q30.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q31.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q32.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q33.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q34.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q35.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q36.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q37.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q38.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q39.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q4.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q40.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q41.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q42.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q43.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q44.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q45.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q46.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q47.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q48.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q49.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q5.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q50.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q51.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q52.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q53.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q54.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q55.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q56.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q57.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q58.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q59.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q6.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q60.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q61.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q62.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q63.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q64.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q65.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q66.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q68.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q69.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q7.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q70.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q71.test A testdata/workloads/tpcds-unmodified/queries/tpcds-q72.test A testdata/workloads/tpcds-unmodified/queries/tp
[Impala-ASF-CR] IMPALA-6819: Add new queries to targeted-perf workload
Nithya Janarthanan has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/9979 ) Change subject: IMPALA-6819: Add new queries to targeted-perf workload .. IMPALA-6819: Add new queries to targeted-perf workload Description: Adding new queries to the targeted-perf workload that is used by Impala performance tests run via $IMPALA_HOME/bin/run-workload.py Testing: Ran the performance tests for the targeted-perf workload and all the tests passed Change-Id: I5c415924d0bb6da1b1f5df6cb16b95a1d2eaa3ab --- A testdata/workloads/targeted-perf/queries/primitive_conjunct_ordering_1.test A testdata/workloads/targeted-perf/queries/primitive_conjunct_ordering_2.test A testdata/workloads/targeted-perf/queries/primitive_conjunct_ordering_3.test A testdata/workloads/targeted-perf/queries/primitive_conjunct_ordering_4.test A testdata/workloads/targeted-perf/queries/primitive_count_star.test A testdata/workloads/targeted-perf/queries/primitive_decimal_arithmetic.test A testdata/workloads/targeted-perf/queries/primitive_filter_bigint_in_list.test A testdata/workloads/targeted-perf/queries/primitive_intrinsic_appx_median.test A testdata/workloads/targeted-perf/queries/primitive_intrinsic_to_date.test M testdata/workloads/targeted-perf/queries/primitive_long_predicate.test A testdata/workloads/targeted-perf/queries/primitive_many_fragments.test A testdata/workloads/targeted-perf/queries/primitive_many_independent_fragments.test A testdata/workloads/targeted-perf/queries/primitive_orderby_bigint_expression.test A testdata/workloads/targeted-perf/queries/primitive_shuffle_1mb_rows.test 14 files changed, 675 insertions(+), 171 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/9979/4 -- To view, visit http://gerrit.cloudera.org:8080/9979 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5c415924d0bb6da1b1f5df6cb16b95a1d2eaa3ab Gerrit-Change-Number: 9979 Gerrit-PatchSet: 4 Gerrit-Owner: Nithya Janarthanan Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Mostafa Mokhtar
[Impala-ASF-CR] IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10132 Change subject: IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views .. IMPALA-6896: NullPointerException in DESCRIBE FORMATTED on views This patch fixes an issue where in ALTER VIEW the storage descriptor is created with a new instance instead of reusing the existing storage descriptor. This causes an issue where some HMS attributes become nullable causing a NullPointerException. The patch also differentiates between updating view attributes for CREATE VIEW and ALTER VIEW. Testing: - Ran all front-end tests - Added a new end-to-end test - Ran the all end-to-end metadata tests Change-Id: Ica2fb0c4f4b09cdf36eeb4911a1cbe7e98381d9e --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/metadata/test_ddl.py 2 files changed, 46 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/32/10132/2 -- To view, visit http://gerrit.cloudera.org:8080/10132 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ica2fb0c4f4b09cdf36eeb4911a1cbe7e98381d9e Gerrit-Change-Number: 10132 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya
[Impala-ASF-CR] IMPALA-5690: Part 1: Rename ostream operators for thrift types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9168 ) Change subject: IMPALA-5690: Part 1: Rename ostream operators for thrift types .. Patch Set 14: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2340/ -- To view, visit http://gerrit.cloudera.org:8080/9168 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c303997411237e988ef960157f781776f6fcb60 Gerrit-Change-Number: 9168 Gerrit-PatchSet: 14 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 20 Apr 2018 04:53:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5893: Remove old kinit code for Impala 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9941 ) Change subject: IMPALA-5893: Remove old kinit code for Impala 3 .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2343/ -- To view, visit http://gerrit.cloudera.org:8080/9941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b Gerrit-Change-Number: 9941 Gerrit-PatchSet: 6 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Fri, 20 Apr 2018 02:32:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5893: Remove old kinit code for Impala 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9941 ) Change subject: IMPALA-5893: Remove old kinit code for Impala 3 .. Patch Set 6: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2341/ -- To view, visit http://gerrit.cloudera.org:8080/9941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b Gerrit-Change-Number: 9941 Gerrit-PatchSet: 6 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Fri, 20 Apr 2018 02:03:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6518,IMPALA-6340: Check that decimal types are compatible in FE
Taras Bobrovytsky has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/9930 ) Change subject: IMPALA-6518,IMPALA-6340: Check that decimal types are compatible in FE .. IMPALA-6518,IMPALA-6340: Check that decimal types are compatible in FE In this patch we implement strict decimal type checking in the FE in various situations when DECIMAL_V2 is enabled. What is affected: - Union. If we union two decimals and it is not possible to come up with a decimal that will be able to contain all the digits, an error is thrown. For example, the union(decimal(20, 10), decimal(20, 20)) returns decimal(30, 20). However, for union(decimal(38, 0), decimal(38, 38)) the ideal return type would be decimal(76,38), but this is too large, so an error is thrown. - Insert. If we are inserting a decimal value into a column where we are not guaranteed that all digits will fit, an error is thrown. For example, inserting a decimal(38,0) value into a decimal(38,38) column. - Functions such as coalesce(). If we are unable to determine the output type that guarantees that all digits will fit from all the arguments, an error is thrown. For example, coalesce(decimal(38,38), decimal(38,0)) will throw an error. - Hash Join. When joining on two decimals, if a type cannot be determined that both columns can be cast to, we throw an error. For example, join on decimal(38,0) and decimal(38,38) will result in an error. To avoid these errors, you need to use CAST() on some of the decimals. In this patch we also change the output decimal calculation of decimal round, truncate and related functions. If these functions are a no-op, the resulting decimal type is the same as the input type. Testing: - Ran a core build and almost all tests passed. I still need to fix A few tests in PlannerTest.testJoinOrder. Change-Id: Id406f4189e01a909152985fabd5cca7a1527a568 --- M be/src/exprs/expr-test.cc M fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/main/java/org/apache/impala/analysis/CaseExpr.java M fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/main/java/org/apache/impala/analysis/InPredicate.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/LikePredicate.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/analysis/PartitionSpec.java M fe/src/main/java/org/apache/impala/analysis/RangePartition.java M fe/src/main/java/org/apache/impala/analysis/StatementBase.java M fe/src/main/java/org/apache/impala/analysis/TimestampArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/TypesUtil.java M fe/src/main/java/org/apache/impala/catalog/Function.java M fe/src/main/java/org/apache/impala/catalog/ScalarType.java M fe/src/main/java/org/apache/impala/catalog/Type.java M fe/src/main/java/org/apache/impala/planner/HashJoinNode.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java M fe/src/test/java/org/apache/impala/analysis/TypesUtilTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java M testdata/workloads/functional-planner/queries/PlannerTest/complex-types-file-formats.test M testdata/workloads/functional-planner/queries/PlannerTest/insert.test M testdata/workloads/functional-planner/queries/PlannerTest/join-order.test M testdata/workloads/functional-planner/queries/PlannerTest/joins.test M testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test M testdata/workloads/functional-planner/queries/PlannerTest/nested-loop-join.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-kudu.test M testdata/workloads/functional-planner/queries/PlannerTest/union.test M testdata/workloads/functional-query/queries/QueryTest/aggregation.test M testdata/workloads/functional-query/queries/QueryTest/avro-writer.test M testdata/workloads/functional-query/queries/QueryTest/decimal-exprs.test M testdata/workloads/functional-query/queries/QueryTest/decimal.test M testdata/workloads/functional-query/queries/QueryTest/exprs.test M testdata/workloads/functional-query/queries/QueryTest/insert.test M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test M testdata/workloads/functional-query/queries/QueryTe
[Impala-ASF-CR] WIP: IMPALA-6518,IMPALA-6340: Check that decimal types are compatible in FE
Taras Bobrovytsky has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/9930 ) Change subject: WIP: IMPALA-6518,IMPALA-6340: Check that decimal types are compatible in FE .. WIP: IMPALA-6518,IMPALA-6340: Check that decimal types are compatible in FE In this patch we implement strict decimal type checking in the FE in various situations when DECIMAL_V2 is enabled. What is affected: - Union. If we union two decimals and it is not possible to come up with a decimal that will be able to contain all the digits, an error is thrown. For example, the union(decimal(20, 10), decimal(20, 20)) returns decimal(30, 20). However, for union(decimal(38, 0), decimal(38, 38)) the ideal return type would be decimal(76,38), but this is too large, so an error is thrown. - Insert. If we are inserting a decimal value into a column where we are not guaranteed that all digits will fit, an error is thrown. For example, inserting a decimal(38,0) value into a decimal(38,38) column. - Functions such as coalesce(). If we are unable to determine the output type that guarantees that all digits will fit from all the arguments, an error is thrown. For example, coalesce(decimal(38,38), decimal(38,0)) will throw an error. - Hash Join. When joining on two decimals, if a type cannot be determined that both columns can be cast to, we throw an error. For example, join on decimal(38,0) and decimal(38,38) will result in an error. To avoid these errors, you need to use CAST() on some of the decimals. In this patch we also change the output decimal calculation of decimal round, truncate and related functions. If these functions are a no-op, the resulting decimal type is the same as the input type. Testing: - Ran a core build and almost all tests passed. I still need to fix A few tests in PlannerTest.testJoinOrder. Change-Id: Id406f4189e01a909152985fabd5cca7a1527a568 --- M be/src/exprs/expr-test.cc M fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/main/java/org/apache/impala/analysis/CaseExpr.java M fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/main/java/org/apache/impala/analysis/InPredicate.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/LikePredicate.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/analysis/PartitionSpec.java M fe/src/main/java/org/apache/impala/analysis/RangePartition.java M fe/src/main/java/org/apache/impala/analysis/StatementBase.java M fe/src/main/java/org/apache/impala/analysis/TimestampArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/TypesUtil.java M fe/src/main/java/org/apache/impala/catalog/Function.java M fe/src/main/java/org/apache/impala/catalog/ScalarType.java M fe/src/main/java/org/apache/impala/catalog/Type.java M fe/src/main/java/org/apache/impala/planner/HashJoinNode.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java M fe/src/test/java/org/apache/impala/analysis/TypesUtilTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java M testdata/workloads/functional-planner/queries/PlannerTest/complex-types-file-formats.test M testdata/workloads/functional-planner/queries/PlannerTest/insert.test M testdata/workloads/functional-planner/queries/PlannerTest/join-order.test M testdata/workloads/functional-planner/queries/PlannerTest/joins.test M testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test M testdata/workloads/functional-planner/queries/PlannerTest/nested-loop-join.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-kudu.test M testdata/workloads/functional-planner/queries/PlannerTest/union.test M testdata/workloads/functional-query/queries/QueryTest/aggregation.test M testdata/workloads/functional-query/queries/QueryTest/avro-writer.test M testdata/workloads/functional-query/queries/QueryTest/decimal-exprs.test M testdata/workloads/functional-query/queries/QueryTest/decimal.test M testdata/workloads/functional-query/queries/QueryTest/exprs.test M testdata/workloads/functional-query/queries/QueryTest/insert.test M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test M testdata/workloads/functional-query/queri
[Impala-ASF-CR] IMPALA-6518,IMPALA-6340: Check that decimal types are compatible in FE
Taras Bobrovytsky has posted comments on this change. (
http://gerrit.cloudera.org:8080/9930 )
Change subject: IMPALA-6518,IMPALA-6340: Check that decimal types are
compatible in FE
..
Patch Set 2:
(19 comments)
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java
File fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java@377
PS2, Line 377: Analyzer analyzer, AnalyticWindow.Boundary boundary)
throws AnalysisException {
> analyzer arg not needed anymore
Done
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java
File fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java@217
PS2, Line 217: boolean strict = analyzer.isDecimalV2() &&
(t0.isDecimal() || t1.isDecimal());
> remove?
Done
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/Expr.java
File fe/src/main/java/org/apache/impala/analysis/Expr.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/Expr.java@423
PS2, Line 423:* Otherwise, if the function signature contains wildcard
decimals, each wildcard child
> Describe how the decimalV2 argument changes the behavior of this function.
Done
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/Expr.java@451
PS2, Line 451: argTypes.append(childType.toString());
> childType.toSql()
Done
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java
File fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java@453
PS2, Line 453: if (analyzer.isDecimalV2() && digitsBefore + digitsAfter >
38) return Type.INVALID;
> I think it's clearer to not call createClippedDecimaltype() for V2 at all b
Done
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/TypesUtil.java
File fe/src/main/java/org/apache/impala/analysis/TypesUtil.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/TypesUtil.java@72
PS2, Line 72: if (decimalV2 && digitsBefore + digitsAfter > 38) return
Type.INVALID;
> Let's avoid calling createClippedDecimalType() for V2
Done
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/catalog/Type.java
File fe/src/main/java/org/apache/impala/catalog/Type.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/catalog/Type.java@310
PS2, Line 310: Type t1, Type t2, boolean strict) {
> move into previous line
Done
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/planner/HashJoinNode.java
File fe/src/main/java/org/apache/impala/planner/HashJoinNode.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/planner/HashJoinNode.java@91
PS2, Line 91: "Operand types: %s = %s.", eqPred.toSql(),
t0.toString(), t1.toString()));
> also use toSql() for types
Done
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java@1838
PS2, Line 1838: AnalyzesOk("select coalesce(1.8, cast(0 as
decimal(38,38)))", decimalV1Ctx);
> Don't we already have tests for this below in TestDecimalFunctions()? I thi
I forgot about those. Added explicit context for v1 and v2 there.
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java@1840
PS2, Line 1840: "Cannot resolve DECIMAL types of the
_impala_builtins.coalesce(DECIMAL(2,1), " +
> Should we maybe omit the database when printing the FunctionName? Most of t
How do we avoid printing the function name? It gets printed when we call
fn.getName().
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java@1843
PS2, Line 1843: String query = "with x as ( " +
> Why have a WITH clause? A UNION alone should suffice
Done
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java@2538
PS2, Line 2538: String.format("Cannot resolve DECIMAL precision and
scale from NULL type in _impala_builtins.%s function.", alias));
> long lines (here and below)
Done
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.ja
[Impala-ASF-CR] IMPALA-6869: [DOCS] Update Known Issues doc for 2.12
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10101 ) Change subject: IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/10101/3/docs/topics/impala_known_issues.xml File docs/topics/impala_known_issues.xml: http://gerrit.cloudera.org:8080/#/c/10101/3/docs/topics/impala_known_issues.xml@481 PS3, Line 481: When you access large metadata : objects in catalogd Web UI, you can encounter two issues: : : : : The output string is too large to easily navigate and can bog : down your browser. : : : : Serializing to a string can exceed memory limits and crash : catalogd. : : > I'd simplify this to: "Printing large objects (> 1 GB) can crash catalogd." Done. And moved this issue to Known Issues: Crash and Hangs -- To view, visit http://gerrit.cloudera.org:8080/10101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 Gerrit-Change-Number: 10101 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 20 Apr 2018 00:55:22 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6869: [DOCS] Update Known Issues doc for 2.12
Hello Sailesh Mukil, Tim Armstrong, Impala Public Jenkins, Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10101 to look at the new patch set (#4). Change subject: IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 .. IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 - Updated the fixed versions for the issues fixed in 2.12 or earlier. - Added new known issues open in 2.12. Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 --- M docs/topics/impala_known_issues.xml 1 file changed, 107 insertions(+), 101 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/10101/4 -- To view, visit http://gerrit.cloudera.org:8080/10101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 Gerrit-Change-Number: 10101 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6869: [DOCS] Update Known Issues doc for 2.12
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10101 ) Change subject: IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/10101/3/docs/topics/impala_known_issues.xml File docs/topics/impala_known_issues.xml: http://gerrit.cloudera.org:8080/#/c/10101/3/docs/topics/impala_known_issues.xml@481 PS3, Line 481: When you access large metadata : objects in catalogd Web UI, you can encounter two issues: : : : : The output string is too large to easily navigate and can bog : down your browser. : : : : Serializing to a string can exceed memory limits and crash : catalogd. : : I'd simplify this to: "Printing large objects (> 1 GB) can crash catalogd." -- To view, visit http://gerrit.cloudera.org:8080/10101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 Gerrit-Change-Number: 10101 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 20 Apr 2018 00:47:21 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6869: [DOCS] Update Known Issues doc for 2.12
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10101 ) Change subject: IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 .. Patch Set 3: > I'm unclear about the criteria for including a known issue here. > For example, here are two known issues that may be worth noting > here: > https://issues.apache.org/jira/browse/IMPALA-6841 > https://issues.apache.org/jira/browse/IMPALA-6671 I added these 2 issues, but I could use help better describing the issues. -- To view, visit http://gerrit.cloudera.org:8080/10101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 Gerrit-Change-Number: 10101 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 20 Apr 2018 00:38:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6869: [DOCS] Update Known Issues doc for 2.12
Hello Sailesh Mukil, Tim Armstrong, Impala Public Jenkins, Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10101 to look at the new patch set (#3). Change subject: IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 .. IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 - Updated the fixed versions for the issues fixed in 2.12 or earlier. - Added new known issues open in 2.12. Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 --- M docs/topics/impala_known_issues.xml 1 file changed, 119 insertions(+), 101 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/10101/3 -- To view, visit http://gerrit.cloudera.org:8080/10101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 Gerrit-Change-Number: 10101 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10080 ) Change subject: IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0 .. IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0 Change-Id: If8416ac0abb7ea1b918ba53b9533af27182fbe89 Cherry-picks: not for 2.x. Reviewed-on: http://gerrit.cloudera.org:8080/10080 Reviewed-by: Alex Behm Tested-by: Impala Public Jenkins --- M docs/topics/impala_upgrading.xml 1 file changed, 158 insertions(+), 2 deletions(-) Approvals: Alex Behm: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If8416ac0abb7ea1b918ba53b9533af27182fbe89 Gerrit-Change-Number: 10080 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10080 ) Change subject: IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0 .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If8416ac0abb7ea1b918ba53b9533af27182fbe89 Gerrit-Change-Number: 10080 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 20 Apr 2018 00:04:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10080 ) Change subject: IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0 .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-docs-submit/255/ -- To view, visit http://gerrit.cloudera.org:8080/10080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If8416ac0abb7ea1b918ba53b9533af27182fbe89 Gerrit-Change-Number: 10080 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 23:53:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0
Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/10080 ) Change subject: IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0 .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If8416ac0abb7ea1b918ba53b9533af27182fbe89 Gerrit-Change-Number: 10080 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 23:50:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6821: Push down limits into Kudu
Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/10119 ) Change subject: IMPALA-6821: Push down limits into Kudu .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/10119/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/10119/1//COMMIT_MSG@24 PS1, Line 24: - For the query 'select count(*) from tpch_kudu.lineitem limit This query doesn't have a limit on the scan. You need to use an inline view. -- To view, visit http://gerrit.cloudera.org:8080/10119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibe35e70065d8706b575e24fe20902cd405b49941 Gerrit-Change-Number: 10119 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Alex Behm Gerrit-Comment-Date: Thu, 19 Apr 2018 23:49:46 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6880: disable flaky bloom filter test
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10122 ) Change subject: IMPALA-6880: disable flaky bloom filter test .. IMPALA-6880: disable flaky bloom filter test This test is made flaky by IMPALA-6338. While that is being worked on, temporarily disable this test. Change-Id: I595645b0f2875614294adc7abb4572aec1be8ad5 Reviewed-on: http://gerrit.cloudera.org:8080/10122 Reviewed-by: Vuk Ercegovac Tested-by: Impala Public Jenkins --- M testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test 1 file changed, 2 insertions(+), 1 deletion(-) Approvals: Vuk Ercegovac: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10122 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I595645b0f2875614294adc7abb4572aec1be8ad5 Gerrit-Change-Number: 10122 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6880: disable flaky bloom filter test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10122 ) Change subject: IMPALA-6880: disable flaky bloom filter test .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10122 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I595645b0f2875614294adc7abb4572aec1be8ad5 Gerrit-Change-Number: 10122 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 23:43:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p4
Tianyi Wang has posted comments on this change. (
http://gerrit.cloudera.org:8080/9300 )
Change subject: IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p4
..
Patch Set 11:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc
File be/src/rpc/thrift-server-test.cc:
http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc@362
PS11, Line 362: {TLSv1_1, {SSLTLS, TLSv1_1}},
> The behavior is covered in this test we are commenting on. I didn't think i
The thrift patch is at https://gerrit.cloudera.org/#/c/10129/. With that and
the latest patchset in this change, TLSv1_x behaves like TLSv1_x_plus now.
I didn't remove this test. Ensuring that all configurations work may still have
some value.
--
To view, visit http://gerrit.cloudera.org:8080/9300
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I639227721502eaa10398d9490ff6ac63aa71b3a6
Gerrit-Change-Number: 9300
Gerrit-PatchSet: 11
Gerrit-Owner: Tianyi Wang
Gerrit-Reviewer: Dan Hecht
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Sailesh Mukil
Gerrit-Reviewer: Thomas Tauber-Marshall
Gerrit-Reviewer: Tianyi Wang
Gerrit-Reviewer: anujphadke
Gerrit-Comment-Date: Thu, 19 Apr 2018 23:25:33 +
Gerrit-HasComments: Yes
[native-toolchain-CR] thrift-0.9.3-p4: forward compatibility of TLS protocols
Tianyi Wang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10129 Change subject: thrift-0.9.3-p4: forward compatibility of TLS protocols .. thrift-0.9.3-p4: forward compatibility of TLS protocols This patch adds thrift-0.9.3-p4. In thrift-0.9.3-p3, TLS protocols are not forward-compatible. A server using TLSv1_x only works with a client using the same protocol. This patch changes thrift into using SSLv23_method() and disabling undesired protocols using flags. TLSv1_x will behave like TLSv1_x_plus in thrift-0.9.0-p11 and is compatible with later versions. Change-Id: Ifdca94a9426feff5ab52a80cf4da669a3fbfe812 --- M buildall.sh A source/thrift/thrift-0.9.3-patches/0004-TLS-forward-compatibility.patch 2 files changed, 53 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/29/10129/1 -- To view, visit http://gerrit.cloudera.org:8080/10129 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ifdca94a9426feff5ab52a80cf4da669a3fbfe812 Gerrit-Change-Number: 10129 Gerrit-PatchSet: 1 Gerrit-Owner: Tianyi Wang
[Impala-ASF-CR] IMPALA-6314: Add run time scalar subquery check for uncorrelated subqueries
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/9005 )
Change subject: IMPALA-6314: Add run time scalar subquery check for
uncorrelated subqueries
..
Patch Set 23:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/9005/23/fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java:
http://gerrit.cloudera.org:8080/#/c/9005/23/fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java@1395
PS23, Line 1395: AnalyzesOk("select count(1) from functional.alltypes " +
> Maybe we should think again the case we discussed in PS9 SingleNodePlanner.
It sounds like trying to modify the predicate assignment code is a pretty
open-ended thing. We should probably just reject problematic queries for now
and move towards getting the current code in.
--
To view, visit http://gerrit.cloudera.org:8080/9005
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0f52b93a60eeacedd242a2f17fa6b99c4fc38e06
Gerrit-Change-Number: 9005
Gerrit-PatchSet: 23
Gerrit-Owner: Zoltan Borok-Nagy
Gerrit-Reviewer: Alex Behm
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Dimitris Tsirogiannis
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Thu, 19 Apr 2018 23:22:14 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] [Preview]IMPALA-5216: Make admission control queuing async
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/10060 )
Change subject: [Preview]IMPALA-5216: Make admission control queuing async
..
Patch Set 2:
(25 comments)
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/runtime/coordinator.h
File be/src/runtime/coordinator.h:
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/runtime/coordinator.h@329
PS2, Line 329: bool execution_started_ = false;
leftover?
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/scheduling/admission-controller.h
File be/src/scheduling/admission-controller.h:
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/scheduling/admission-controller.h@45
PS2, Line 45: AdmissionStatus
Maybe it should be AdmissionOutcome? This is really the final outcome of
admission rather than the intermediate states, right?
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/scheduling/admission-controller.h@199
PS2, Line 199: /// returns an OK status, schedule->is_admitted() is true and
admit_status is ADMITTED.
Is (admit_status.IsSet() && admit_status.Get() == ADMITTED) equivalent to
schedule->is_admitted()? If so we should just remove the redundancy somehow.
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/scheduling/admission-controller.h@395
PS2, Line 395: Promise* admit_status;
What owns the memory of admit_status?
http://gerrit.cloudera.org:8080/#/c/10060/1/be/src/scheduling/admission-controller.cc
File be/src/scheduling/admission-controller.cc:
http://gerrit.cloudera.org:8080/#/c/10060/1/be/src/scheduling/admission-controller.cc@518
PS1, Line 518:
admit_status->TrySet(AdmissionStatus::REJECTED_OR_TIMED_OUT);
> This was my initial intuition too, but like you mentioned, I saw that sched
Yeah, I actually think it would be good to remove the is_admitted_ flag out of
QuerySchedule to make it more consistent.
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/scheduling/admission-controller.cc
File be/src/scheduling/admission-controller.cc:
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/scheduling/admission-controller.cc@505
PS2, Line 505: lock_guard lock(admission_ctrl_lock_);
Do we need to worry about races between admit_status being set to CANCELLED and
it being set below? What happens if the TrySet(ADMITTED) call below fails
because the query was cancelled? Are the admission results, etc correct?
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/scheduling/admission-controller.cc@576
PS2, Line 576:
admit_status->TrySet(AdmissionStatus::REJECTED_OR_TIMED_OUT);
It would be nice to log the outcome of admission in all cases. Maybe we should
have a class that is a wrapper around the promise and guarantees that the
outcome of admission is logged?
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/scheduling/admission-controller.cc@595
PS2, Line 595: if (queue->Remove(&queue_node)) {
One line?
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/scheduling/admission-controller.cc@885
PS2, Line 885: // TODO: Maybe dont even check cancelled here, just try
admitting it and let the
+1 to removing a code path if it isn't totally necessary. As long as we catch
it before starting fragments it seems ok to detect the cancellation slightly
later.
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/scheduling/admission-controller.cc@901
PS2, Line 901: DCHECK
We should really log the actual value if the DCHECK fails. I wish we have a
convenience macro for comparing two strongly-typed enums. Might be worth adding?
http://gerrit.cloudera.org:8080/#/c/10060/1/be/src/scheduling/query-schedule.h
File be/src/scheduling/query-schedule.h:
http://gerrit.cloudera.org:8080/#/c/10060/1/be/src/scheduling/query-schedule.h@274
PS1, Line 274:
> the only reason I kept this was because the coordinator has a const referen
Can the coordinator have a reference to Promise then? Having the
redundant state and inconsistency between them seems confusing.
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/service/client-request-state.h
File be/src/service/client-request-state.h:
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/service/client-request-state.h@279
PS2, Line 279: /// admit or reject or mark timed out) or on cancellation of
query (to mark cancelled).
Maybe slightly expand on how cancellation works. I think a few important things
are:
* It can be called from client-RPC-handling threads.
* Cancellation is performed by AdmitQuery() or DequeueLoop() asynchronously.
* TrySet(CANCELLED) ensures async cancellation of the query is queued or about
to be queued or is a no-op if the query has already been admitted.
http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/service/client-request-state.h@282
PS2, Line 282: Promise admit_status_;
After reading through the rest of the patch I feel like it might be beneficial
to have a lightweight abstraction around admit_status_ to force callers to use
[Impala-ASF-CR] IMPALA-6518,IMPALA-6340: Check that decimal types are compatible in FE
Alex Behm has posted comments on this change. (
http://gerrit.cloudera.org:8080/9930 )
Change subject: IMPALA-6518,IMPALA-6340: Check that decimal types are
compatible in FE
..
Patch Set 2:
(22 comments)
Looks pretty good to me
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java
File fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java@377
PS2, Line 377: Analyzer analyzer, AnalyticWindow.Boundary boundary)
throws AnalysisException {
analyzer arg not needed anymore
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java
File fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java@217
PS2, Line 217: boolean strict = analyzer.isDecimalV2() &&
(t0.isDecimal() || t1.isDecimal());
remove?
http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/analysis/Expr.java
File fe/src/main/java/org/apache/impala/analysis/Expr.java:
http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/analysis/Expr.java@440
PS1, Line 440: for (int i = 0; i < children_.size(); ++i) {
> Not a good idea because if ignoreWildcardDecimals is true we don't want to
It's confusing to me because this check is inside the loop, but
resolvedWildcardType and ignoreWildcardDecimals are not changed in the loop, so
it seems more logical to move this check outside.
Would it not work to check:
if (resolvedWildcardType != null && resolvedWildcardType.isInvalid() &&
!ignoreWildcardDecimals) {
// error
}
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/Expr.java
File fe/src/main/java/org/apache/impala/analysis/Expr.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/Expr.java@423
PS2, Line 423:* Otherwise, if the function signature contains wildcard
decimals, each wildcard child
Describe how the decimalV2 argument changes the behavior of this function.
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/Expr.java@451
PS2, Line 451: argTypes.append(childType.toString());
childType.toSql()
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java
File fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java@453
PS2, Line 453: if (analyzer.isDecimalV2() && digitsBefore + digitsAfter >
38) return Type.INVALID;
I think it's clearer to not call createClippedDecimaltype() for V2 at all
because we require no clipping to happen. You can just return the appropriate
type directly.
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/TypesUtil.java
File fe/src/main/java/org/apache/impala/analysis/TypesUtil.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/analysis/TypesUtil.java@72
PS2, Line 72: if (decimalV2 && digitsBefore + digitsAfter > 38) return
Type.INVALID;
Let's avoid calling createClippedDecimalType() for V2
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/catalog/Type.java
File fe/src/main/java/org/apache/impala/catalog/Type.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/catalog/Type.java@310
PS2, Line 310: Type t1, Type t2, boolean strict) {
move into previous line
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/planner/HashJoinNode.java
File fe/src/main/java/org/apache/impala/planner/HashJoinNode.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/main/java/org/apache/impala/planner/HashJoinNode.java@91
PS2, Line 91: "Operand types: %s = %s.", eqPred.toSql(),
t0.toString(), t1.toString()));
also use toSql() for types
http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java:
http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java@2570
PS1, Line 2570: testDecimalExpr(String.format("%s(1.23)", alias),
> Should we be generating the exception in the function call expr then? That
Makes sense, printing the toSql() will be more tricky. Let's defer.
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java:
http://gerrit.cloudera.org:8080/#/c/9930/2/fe/src/test/java/org/a
[Impala-ASF-CR] [Preview]IMPALA-5216: Make admission control queuing async
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/10060 ) Change subject: [Preview]IMPALA-5216: Make admission control queuing async .. Patch Set 2: (2 comments) missed a small change regarding update of query state in the impala-beeswax-server, will do in the next patch http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/runtime/coordinator.h File be/src/runtime/coordinator.h: http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/runtime/coordinator.h@328 PS2, Line 328: /// True if and only if Exec() has been called and returned with an OK status. : bool execution_started_ = false; forgot to remove this. will do in the next patch http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/runtime/coordinator.cc File be/src/runtime/coordinator.cc: http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/runtime/coordinator.cc@160 PS2, Line 160: // Ensure 'this' Coordinator object can be safely destroyed. : DCHECK(query_status_.ok()); : query_status_ = prepare_status; : CancelInternal(); > this is somewhat subtle logic, so it'd be good to combine it with the tail makes sense, will do in the next patch -- To view, visit http://gerrit.cloudera.org:8080/10060 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I989cf5b259afb8f5bc5c35590c94961c81ce88bf Gerrit-Change-Number: 10060 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Apr 2018 22:48:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3
Tianyi Wang has posted comments on this change. (
http://gerrit.cloudera.org:8080/9300 )
Change subject: IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3
..
Patch Set 11:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc
File be/src/rpc/thrift-server-test.cc:
http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc@362
PS11, Line 362: {TLSv1_1, {SSLTLS, TLSv1_1}},
> Sounds like we're missing a test?
The behavior is covered in this test we are commenting on. I didn't think it
through and just removed TLSv1_*_plus from this test and that is how Michael
found it wrong.
By the way, if we patch thrift so that TLSv1_1 works as TLSv1_1_plus in thrift
0.9.0, this test might become meaningless - any combination will work because
they will all negotiate into using TLS 1.2. So we might have to remove this
test.
--
To view, visit http://gerrit.cloudera.org:8080/9300
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I639227721502eaa10398d9490ff6ac63aa71b3a6
Gerrit-Change-Number: 9300
Gerrit-PatchSet: 11
Gerrit-Owner: Tianyi Wang
Gerrit-Reviewer: Dan Hecht
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Sailesh Mukil
Gerrit-Reviewer: Thomas Tauber-Marshall
Gerrit-Reviewer: Tianyi Wang
Gerrit-Reviewer: anujphadke
Gerrit-Comment-Date: Thu, 19 Apr 2018 22:44:09 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3
Dan Hecht has posted comments on this change. (
http://gerrit.cloudera.org:8080/9300 )
Change subject: IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3
..
Patch Set 11:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc
File be/src/rpc/thrift-server-test.cc:
http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc@362
PS11, Line 362: {TLSv1_1, {SSLTLS, TLSv1_1}},
> Talked to Tianyi in person about this.
Sounds like we're missing a test?
--
To view, visit http://gerrit.cloudera.org:8080/9300
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I639227721502eaa10398d9490ff6ac63aa71b3a6
Gerrit-Change-Number: 9300
Gerrit-PatchSet: 11
Gerrit-Owner: Tianyi Wang
Gerrit-Reviewer: Dan Hecht
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Sailesh Mukil
Gerrit-Reviewer: Thomas Tauber-Marshall
Gerrit-Reviewer: Tianyi Wang
Gerrit-Reviewer: anujphadke
Gerrit-Comment-Date: Thu, 19 Apr 2018 22:33:27 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0
Hello Philip Zeyliger, Sailesh Mukil, Alex Behm, Impala Public Jenkins, Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10080 to look at the new patch set (#4). Change subject: IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0 .. IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0 Change-Id: If8416ac0abb7ea1b918ba53b9533af27182fbe89 Cherry-picks: not for 2.x. --- M docs/topics/impala_upgrading.xml 1 file changed, 158 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/10080/4 -- To view, visit http://gerrit.cloudera.org:8080/10080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If8416ac0abb7ea1b918ba53b9533af27182fbe89 Gerrit-Change-Number: 10080 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10080 ) Change subject: IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0 .. Patch Set 3: (6 comments) http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml File docs/topics/impala_upgrading.xml: http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml@150 PS3, Line 150: identifier, e.g. a table name, enclose the word in quotes. > enclose the word in backticks. Done http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml@155 PS3, Line 155: startup flag: --reserved_words_version=2.11.0. > Note that this startup option will eventually be deprecated. Done http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml@184 PS3, Line 184: To conform to the SQL standard, Impala no longer performs alias > probably better to just link to the documentation and not repeat all exampl Done http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml@190 PS3, Line 190: For example, the following statements will result in syntax errors. > They will not result in syntax errors, but error in semantic analysis (prob Done http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml@234 PS3, Line 234: The PARQUET_ARRAY_RESOLUTION query option controls > Remove this paragraph and merge the last sentence linking to the query opti Done http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml@247 PS3, Line 247: >clustered hint is enabled by default. The hint inserts a > The hint adds a local sort... Done -- To view, visit http://gerrit.cloudera.org:8080/10080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If8416ac0abb7ea1b918ba53b9533af27182fbe89 Gerrit-Change-Number: 10080 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 22:33:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/10051/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/10051/3//COMMIT_MSG@7 PS3, Line 7: 6917 > Um... I know the horse is already gone, but this is actually IMPALA-5917. :/ We'll survive. -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 3 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 22:32:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [Preview]IMPALA-5216: Make admission control queuing async
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10060 ) Change subject: [Preview]IMPALA-5216: Make admission control queuing async .. Patch Set 2: (1 comment) Just looked at the coordinator change since you mentioned that earlier. http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/runtime/coordinator.cc File be/src/runtime/coordinator.cc: http://gerrit.cloudera.org:8080/#/c/10060/2/be/src/runtime/coordinator.cc@160 PS2, Line 160: // Ensure 'this' Coordinator object can be safely destroyed. : DCHECK(query_status_.ok()); : query_status_ = prepare_status; : CancelInternal(); this is somewhat subtle logic, so it'd be good to combine it with the tail of FinishBackendStartup() and ensure it's not missed with future RETURN_IF_ERROR usage in this method. How about either renaming this function ExecInternal() and then make Exec() handle the error by doing cancellation and setting query_status_. Or okay with me to adapt the "goto error" pattern for this routine. -- To view, visit http://gerrit.cloudera.org:8080/10060 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I989cf5b259afb8f5bc5c35590c94961c81ce88bf Gerrit-Change-Number: 10060 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Apr 2018 22:31:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/10051/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/10051/3//COMMIT_MSG@7 PS3, Line 7: 6917 Um... I know the horse is already gone, but this is actually IMPALA-5917. -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 3 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 22:28:54 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3
Michael Ho has posted comments on this change. (
http://gerrit.cloudera.org:8080/9300 )
Change subject: IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3
..
Patch Set 11:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc
File be/src/rpc/thrift-server-test.cc:
http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc@362
PS11, Line 362: {TLSv1_1, {SSLTLS, TLSv1_1}},
> My understanding of the test is that the thrift server started with certain
Talked to Tianyi in person about this.
This patch as-is will break compatibility. In particular, the code in Thrift
(https://github.com/apache/thrift/blob/0.9.3/lib/cpp/src/thrift/transport/TSSLSocket.cpp#L139-L153)
will only support the specified TLS version.
In other words, if we specify FLAGS_minimum_ssl_version as "tlsv1", a client
used to be able to speak to the server using TLSv1, TLSv1.1 and TLSv1.2 in the
old code. With this patch, a client can only talk to the thrift server using
TLSv1 as the thrift code now uses TLSv1_method() which only understands TLSv1
(https://www.openssl.org/docs/man1.0.2/ssl/SSL_CTX_new.html).
As Tianyi suggested, patching Thrift is one way of preserving this
compatibility.
Sailesh, what do you think ?
--
To view, visit http://gerrit.cloudera.org:8080/9300
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I639227721502eaa10398d9490ff6ac63aa71b3a6
Gerrit-Change-Number: 9300
Gerrit-PatchSet: 11
Gerrit-Owner: Tianyi Wang
Gerrit-Reviewer: Dan Hecht
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Sailesh Mukil
Gerrit-Reviewer: Thomas Tauber-Marshall
Gerrit-Reviewer: Tianyi Wang
Gerrit-Reviewer: anujphadke
Gerrit-Comment-Date: Thu, 19 Apr 2018 22:26:39 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6869: [DOCS] Update Known Issues doc for 2.12
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10101 ) Change subject: IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 .. Patch Set 2: I'm unclear about the criteria for including a known issue here. For example, here are two known issues that may be worth noting here: https://issues.apache.org/jira/browse/IMPALA-6841 https://issues.apache.org/jira/browse/IMPALA-6671 -- To view, visit http://gerrit.cloudera.org:8080/10101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 Gerrit-Change-Number: 10101 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 22:19:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5893: Remove old kinit code for Impala 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9941 ) Change subject: IMPALA-5893: Remove old kinit code for Impala 3 .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2341/ -- To view, visit http://gerrit.cloudera.org:8080/9941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b Gerrit-Change-Number: 9941 Gerrit-PatchSet: 6 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 19 Apr 2018 22:07:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5893: Remove old kinit code for Impala 3
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/9941 ) Change subject: IMPALA-5893: Remove old kinit code for Impala 3 .. Patch Set 6: Code-Review+2 Hit flaky test. Rebase, carry +2. -- To view, visit http://gerrit.cloudera.org:8080/9941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b Gerrit-Change-Number: 9941 Gerrit-PatchSet: 6 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 19 Apr 2018 22:07:14 + Gerrit-HasComments: No
[Impala-ASF-CR](2.x) IMPALA-6847: work around high memory estimates for AC
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10121 ) Change subject: IMPALA-6847: work around high memory estimates for AC .. IMPALA-6847: work around high memory estimates for AC Adds MAX_MEM_ESTIMATE_FOR_ADMISSION query option, which takes effect if and only if * Memory-based admission control is enabled for the pool * No mem_limit is set (i.e. best practices are not being followed) In that case min(MAX_MEM_ESTIMATE_FOR_ADMISSION, mem_estimate) is used for admission control instead of mem_estimate. This provides a way to override the planner's estimate if it happens to be incorrect and are preventing the query from running. Setting MEM_LIMIT is usually a better alternative but sometimes it is not feasible to set MEM_LIMIT for each individual query. Testing: Added an admission control test to verify that query option allows queries with high estimates to run. Also tested manually on a minicluster started with: start-impala-cluster.py --impalad_args='-vmodule admission-controller=3 \ -default_pool_mem_limit 12884901888' Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e Reviewed-on: http://gerrit.cloudera.org:8080/10058 Reviewed-by: Tim Armstrong Tested-by: Impala Public Jenkins Reviewed-on: http://gerrit.cloudera.org:8080/10121 --- M be/src/service/query-options-test.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M fe/src/main/java/org/apache/impala/service/Frontend.java A testdata/workloads/functional-query/queries/QueryTest/admission-reject-mem-estimate.test M tests/custom_cluster/test_admission_controller.py 8 files changed, 135 insertions(+), 11 deletions(-) Approvals: Tim Armstrong: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: merged Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e Gerrit-Change-Number: 10121 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR](2.x) IMPALA-6847: work around high memory estimates for AC
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10121 ) Change subject: IMPALA-6847: work around high memory estimates for AC .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e Gerrit-Change-Number: 10121 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Apr 2018 22:06:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3
Michael Ho has posted comments on this change. (
http://gerrit.cloudera.org:8080/9300 )
Change subject: IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3
..
Patch Set 11:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc
File be/src/rpc/thrift-server-test.cc:
http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc@362
PS11, Line 362: {TLSv1_1, {SSLTLS, TLSv1_1}},
> I realized I didn't think it through.
My understanding of the test is that the thrift server started with certain
version of OpenSSL should be backward compatible with client using <= server
version.
So if the server is using TLSv1_1, I would expect the whitelist to be SSLTLS,
TLSv1_0 and TLSv1_1.
--
To view, visit http://gerrit.cloudera.org:8080/9300
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I639227721502eaa10398d9490ff6ac63aa71b3a6
Gerrit-Change-Number: 9300
Gerrit-PatchSet: 11
Gerrit-Owner: Tianyi Wang
Gerrit-Reviewer: Dan Hecht
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Sailesh Mukil
Gerrit-Reviewer: Thomas Tauber-Marshall
Gerrit-Reviewer: Tianyi Wang
Gerrit-Reviewer: anujphadke
Gerrit-Comment-Date: Thu, 19 Apr 2018 22:03:57 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0
Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/10080 ) Change subject: IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0 .. Patch Set 3: (6 comments) http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml File docs/topics/impala_upgrading.xml: http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml@150 PS3, Line 150: identifier, e.g. a table name, enclose the word in quotes. enclose the word in backticks. (*not* quotes) http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml@155 PS3, Line 155: startup flag: --reserved_words_version=2.11.0. Note that this startup option will eventually be deprecated. http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml@184 PS3, Line 184: To conform to the SQL standard, Impala no longer performs alias probably better to just link to the documentation and not repeat all examples here http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml@190 PS3, Line 190: For example, the following statements will result in syntax errors. They will not result in syntax errors, but error in semantic analysis (probably better to not list the examples here) http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml@234 PS3, Line 234: The PARQUET_ARRAY_RESOLUTION query option controls Remove this paragraph and merge the last sentence linking to the query options docu page with the paragraph in 230? http://gerrit.cloudera.org:8080/#/c/10080/3/docs/topics/impala_upgrading.xml@247 PS3, Line 247: >clustered hint is enabled by default. The hint inserts a The hint adds a local sort... (better to not use "insert" since it could be misunderstood) -- To view, visit http://gerrit.cloudera.org:8080/10080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If8416ac0abb7ea1b918ba53b9533af27182fbe89 Gerrit-Change-Number: 10080 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 22:01:33 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6884: Skip test misaligned orc stripes on invalid platforms
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10117 ) Change subject: IMPALA-6884: Skip test_misaligned_orc_stripes on invalid platforms .. IMPALA-6884: Skip test_misaligned_orc_stripes on invalid platforms test_misaligned_orc_stripes relies on generating files with a specific number of HDFS blocks. This is not possible on S3, ADLS, or Isilon, so the test needs to be skipped on those platforms. Additionally, it relies on running multiple impalads, so it needs to be skipped on local filesystem tests. Change-Id: I8395b4eb06847a35c630b3ef43e289960d78382a Reviewed-on: http://gerrit.cloudera.org:8080/10117 Reviewed-by: Vuk Ercegovac Tested-by: Impala Public Jenkins --- M tests/query_test/test_scanners.py 1 file changed, 4 insertions(+), 0 deletions(-) Approvals: Vuk Ercegovac: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10117 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8395b4eb06847a35c630b3ef43e289960d78382a Gerrit-Change-Number: 10117 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6884: Skip test misaligned orc stripes on invalid platforms
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10117 ) Change subject: IMPALA-6884: Skip test_misaligned_orc_stripes on invalid platforms .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10117 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8395b4eb06847a35c630b3ef43e289960d78382a Gerrit-Change-Number: 10117 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 21:58:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6733: [DOCS] Impala 3.0 Release Notes
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10090 ) Change subject: IMPALA-6733: [DOCS] Impala 3.0 Release Notes .. Patch Set 3: Sailesh and Vuk, Could you review this and approve? -- To view, visit http://gerrit.cloudera.org:8080/10090 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I971323993aa6dced0791476e3e798d2f246fab5c Gerrit-Change-Number: 10090 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 21:53:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5893: Remove old kinit code for Impala 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9941 ) Change subject: IMPALA-5893: Remove old kinit code for Impala 3 .. Patch Set 5: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2334/ -- To view, visit http://gerrit.cloudera.org:8080/9941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b Gerrit-Change-Number: 9941 Gerrit-PatchSet: 5 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 19 Apr 2018 21:49:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5690: Part 1: Rename ostream operators for thrift types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9168 ) Change subject: IMPALA-5690: Part 1: Rename ostream operators for thrift types .. Patch Set 14: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2340/ -- To view, visit http://gerrit.cloudera.org:8080/9168 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c303997411237e988ef960157f781776f6fcb60 Gerrit-Change-Number: 9168 Gerrit-PatchSet: 14 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Apr 2018 21:25:02 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] A typo fix introduced in impala-6748
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10126 ) Change subject: [DOCS] A typo fix introduced in impala-6748 .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10126 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idb89c97d2cc470960b88f14f038252d1d46778d0 Gerrit-Change-Number: 10126 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 19 Apr 2018 21:24:55 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] A typo fix introduced in impala-6748
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10126 ) Change subject: [DOCS] A typo fix introduced in impala-6748 .. [DOCS] A typo fix introduced in impala-6748 Change-Id: Idb89c97d2cc470960b88f14f038252d1d46778d0 Reviewed-on: http://gerrit.cloudera.org:8080/10126 Reviewed-by: Alex Rodoni Tested-by: Impala Public Jenkins --- M docs/shared/impala_common.xml 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Alex Rodoni: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10126 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Idb89c97d2cc470960b88f14f038252d1d46778d0 Gerrit-Change-Number: 10126 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-5690: Part 1: Rename ostream operators for thrift types
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/9168 ) Change subject: IMPALA-5690: Part 1: Rename ostream operators for thrift types .. Patch Set 14: I think so, looks like it was +2ed. -- To view, visit http://gerrit.cloudera.org:8080/9168 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c303997411237e988ef960157f781776f6fcb60 Gerrit-Change-Number: 9168 Gerrit-PatchSet: 14 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Apr 2018 21:24:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5690: Part 1: Rename ostream operators for thrift types
Tianyi Wang has posted comments on this change. ( http://gerrit.cloudera.org:8080/9168 ) Change subject: IMPALA-5690: Part 1: Rename ostream operators for thrift types .. Patch Set 14: > Can we merge this? I think we can. If you agree I will merge it. -- To view, visit http://gerrit.cloudera.org:8080/9168 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c303997411237e988ef960157f781776f6fcb60 Gerrit-Change-Number: 9168 Gerrit-PatchSet: 14 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Apr 2018 21:22:29 + Gerrit-HasComments: No
[Impala-ASF-CR](2.x) IMPALA-6847: work around high memory estimates for AC
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10121 ) Change subject: IMPALA-6847: work around high memory estimates for AC .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e Gerrit-Change-Number: 10121 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Apr 2018 21:16:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5690: Part 1: Rename ostream operators for thrift types
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/9168 ) Change subject: IMPALA-5690: Part 1: Rename ostream operators for thrift types .. Patch Set 14: Can we merge this? -- To view, visit http://gerrit.cloudera.org:8080/9168 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c303997411237e988ef960157f781776f6fcb60 Gerrit-Change-Number: 9168 Gerrit-PatchSet: 14 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Apr 2018 21:13:45 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] A typo fix introduced in impala-6748
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10126 ) Change subject: [DOCS] A typo fix introduced in impala-6748 .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-docs-submit/254/ -- To view, visit http://gerrit.cloudera.org:8080/10126 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idb89c97d2cc470960b88f14f038252d1d46778d0 Gerrit-Change-Number: 10126 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 19 Apr 2018 21:08:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6131: Track time of last statistics update in metadata
Lars Volker has posted comments on this change. (
http://gerrit.cloudera.org:8080/10116 )
Change subject: IMPALA-6131: Track time of last statistics update in metadata
..
Patch Set 3:
(11 comments)
http://gerrit.cloudera.org:8080/#/c/10116/3/fe/src/main/java/org/apache/impala/catalog/KuduTable.java
File fe/src/main/java/org/apache/impala/catalog/KuduTable.java:
http://gerrit.cloudera.org:8080/#/c/10116/3/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@240
PS3, Line 240: // HMS updates transient_lastDdlTime if it is removed.
Can you explain in a comment why we need to clone the table instead of removing
the key from msTable_?
http://gerrit.cloudera.org:8080/#/c/10116/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/10116/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@728
PS3, Line 728:* and 'numUpdatedColumns', respectively.
This comment should point out that it updates impala.lastComputeStatsTime, and
that it only does so when running compute stats.
http://gerrit.cloudera.org:8080/#/c/10116/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@782
PS3, Line 782: &&
Why do we only update this if both are set? Why not if we're just updating one?
http://gerrit.cloudera.org:8080/#/c/10116/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2857
PS3, Line 2857:* longer period of time.
Explain the bool parameter in the comment including what value the lastDdlTime
is set to if it's set to true. Since we already describe the HMS API we could
also mention what happens with missing fields (they get removed).
http://gerrit.cloudera.org:8080/#/c/10116/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2862
PS3, Line 2862: long lastDdlTime = -1;
unused?
http://gerrit.cloudera.org:8080/#/c/10116/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3221
PS3, Line 3221:* Updates the lastDdlTime of the table if new partitions
were created.
Assuming this line is still correct, can you add a comment where this happens?
e.g. "This call also updates lastDdlTime"
http://gerrit.cloudera.org:8080/#/c/10116/3/tests/metadata/test_last_ddl_time_update.py
File tests/metadata/test_last_ddl_time_update.py:
http://gerrit.cloudera.org:8080/#/c/10116/3/tests/metadata/test_last_ddl_time_update.py@50
PS3, Line 50: # initialize last compute stats time
This should be more verbose, explain why it is necessary to initialize it.
http://gerrit.cloudera.org:8080/#/c/10116/3/tests/metadata/test_last_ddl_time_update.py@75
PS3, Line 75: self.run_test("compute stats %s" % FQ_TBL_NAME,
We should also have a test for computing incremental stats, for sampled stats,
and for manually updating both table and column stats.
http://gerrit.cloudera.org:8080/#/c/10116/3/tests/metadata/test_last_ddl_time_update.py@145
PS3, Line 145:expect_changed_ddl_time, expect_changed_stat_time
= False):
I think we should have defaults for both expect_* parameters here, or for none.
http://gerrit.cloudera.org:8080/#/c/10116/3/tests/metadata/test_last_ddl_time_update.py@161
PS3, Line 161: beforeStatTime =
table.parameters[LAST_COMPUTE_STATS_TIME_KEY]
nit: beforeStatsTime
http://gerrit.cloudera.org:8080/#/c/10116/3/tests/metadata/test_last_ddl_time_update.py@180
PS3, Line 180: if expect_changed_stat_time:
stats_time, given the key is lastComputeStatsTime, too.
--
To view, visit http://gerrit.cloudera.org:8080/10116
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I59a671ac29d352bd92ce40d5cb6662bb23f146b5
Gerrit-Change-Number: 10116
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Thu, 19 Apr 2018 21:08:16 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] [DOCS] A typo fix introduced in impala-6748
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10126 ) Change subject: [DOCS] A typo fix introduced in impala-6748 .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10126 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idb89c97d2cc470960b88f14f038252d1d46778d0 Gerrit-Change-Number: 10126 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Comment-Date: Thu, 19 Apr 2018 21:08:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4025: Part 1: Add percentile disc aggregation function
Dimitris Tsirogiannis has posted comments on this change. (
http://gerrit.cloudera.org:8080/9777 )
Change subject: IMPALA-4025: Part 1: Add percentile_disc aggregation function
..
Patch Set 4:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java
File fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java:
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java@99
PS4, Line 99: private ExprSubstitutionMap outputGroupingExprSmap_ = new
ExprSubstitutionMap();
This seems like a subset of outputTupleSmap_. If it's just for checking whether
an expr is bounded by grouping exprs, why can't we use outputTupleSmap_?
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@353
PS4, Line 353: boolean topStmtContainsPercentile() { return containsPercentile;
}
I haven't seen yet where this is used but I feel this is not correct. A stmt
may have a hierarchy of analyzers each one corresponding to a select block and
each one with or without a percentile function. So "topStmtContainsPercentile"
in the context of single analyzer seems kind of off.
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java
File fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java:
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java@468
PS4, Line 468: mergeAggInputFn_.analyze(analyzer);
Are you sure this is correct? Won't this call Expr.analyzer() which then
subsequently calls FunctionCallExpr.analyzeImpl()?
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
File fe/src/main/java/org/apache/impala/analysis/SelectStmt.java:
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java@63
PS4, Line 63: // slotRefs directly refer to table(s) in the from clause.
Can you plz expand the comment a bit to describe why this is needed?
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java
File fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java:
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@1140
PS4, Line 1140: Those slotRefs
Unfinished sentence
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@1143
PS4, Line 1143: slotRef
mention what this slotRef represents
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@1163
PS4, Line 1163: +
nit: add a space before '+'
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java:
http://gerrit.cloudera.org:8080/#/c/9777/4/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java@2124
PS4, Line 2124: "from functional.alltypesagg group by tinyint_col");
Can you also add a case with a WITH clause?
--
To view, visit http://gerrit.cloudera.org:8080/9777
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iacef7b3fcd74c4c73d88400ce27307c3baa0121e
Gerrit-Change-Number: 9777
Gerrit-PatchSet: 4
Gerrit-Owner: Tianyi Wang
Gerrit-Reviewer: Dimitris Tsirogiannis
Gerrit-Reviewer: Tianyi Wang
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Thu, 19 Apr 2018 21:07:05 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] [DOCS] A typo fix introduced in impala-6748
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10126 Change subject: [DOCS] A typo fix introduced in impala-6748 .. [DOCS] A typo fix introduced in impala-6748 Change-Id: Idb89c97d2cc470960b88f14f038252d1d46778d0 --- M docs/shared/impala_common.xml 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/10126/1 -- To view, visit http://gerrit.cloudera.org:8080/10126 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Idb89c97d2cc470960b88f14f038252d1d46778d0 Gerrit-Change-Number: 10126 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-6869: [DOCS] Update Known Issues doc for 2.12
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10101 ) Change subject: IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 .. Patch Set 1: (2 comments) > (2 comments) > > I was scanning for a trace of krpc and 5058 and didn't see it > (either mentioned before as an issue and now resolved). Perhaps > I've misunderstood the purpose of this doc? These doc lists currently open bugs/issues. I discussed with Michael Ho and Greg about KRPC, and we decided as important as the feature is, it is not user-facing, thus no mention in docs. http://gerrit.cloudera.org:8080/#/c/10101/1/docs/topics/impala_known_issues.xml File docs/topics/impala_known_issues.xml: http://gerrit.cloudera.org:8080/#/c/10101/1/docs/topics/impala_known_issues.xml@140 PS1, Line 140: crash > to Done http://gerrit.cloudera.org:8080/#/c/10101/1/docs/topics/impala_known_issues.xml@332 PS1, Line 332: spent > spend Done -- To view, visit http://gerrit.cloudera.org:8080/10101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 Gerrit-Change-Number: 10101 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 20:53:50 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6869: [DOCS] Update Known Issues doc for 2.12
Hello Sailesh Mukil, Tim Armstrong, Impala Public Jenkins, Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10101 to look at the new patch set (#2). Change subject: IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 .. IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 - Updated the fixed versions for the issues fixed in 2.12 or earlier. - Added new known issues open in 2.12. Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 --- M docs/topics/impala_known_issues.xml 1 file changed, 76 insertions(+), 101 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/10101/2 -- To view, visit http://gerrit.cloudera.org:8080/10101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 Gerrit-Change-Number: 10101 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0
Hello Philip Zeyliger, Sailesh Mukil, Alex Behm, Impala Public Jenkins, Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10080 to look at the new patch set (#3). Change subject: IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0 .. IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0 Change-Id: If8416ac0abb7ea1b918ba53b9533af27182fbe89 Cherry-picks: not for 2.x. --- M docs/topics/impala_upgrading.xml 1 file changed, 187 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/10080/3 -- To view, visit http://gerrit.cloudera.org:8080/10080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If8416ac0abb7ea1b918ba53b9533af27182fbe89 Gerrit-Change-Number: 10080 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10080 ) Change subject: IMPALA-6860: [DOCS] Upgrade considerations for Impala 3.0 .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/10080/2/docs/topics/impala_upgrading.xml File docs/topics/impala_upgrading.xml: http://gerrit.cloudera.org:8080/#/c/10080/2/docs/topics/impala_upgrading.xml@288 PS2, Line 288: > Another important upgrade consideration is with the new fine-grained permis Done -- To view, visit http://gerrit.cloudera.org:8080/10080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If8416ac0abb7ea1b918ba53b9533af27182fbe89 Gerrit-Change-Number: 10080 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 20:47:14 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6869: [DOCS] Update Known Issues doc for 2.12
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10101 ) Change subject: IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 .. Patch Set 1: (2 comments) I was scanning for a trace of krpc and 5058 and didn't see it (either mentioned before as an issue and now resolved). Perhaps I've misunderstood the purpose of this doc? http://gerrit.cloudera.org:8080/#/c/10101/1/docs/topics/impala_known_issues.xml File docs/topics/impala_known_issues.xml: http://gerrit.cloudera.org:8080/#/c/10101/1/docs/topics/impala_known_issues.xml@140 PS1, Line 140: crash to http://gerrit.cloudera.org:8080/#/c/10101/1/docs/topics/impala_known_issues.xml@332 PS1, Line 332: spent spend -- To view, visit http://gerrit.cloudera.org:8080/10101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 Gerrit-Change-Number: 10101 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 20:45:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 2 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 20:31:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. IMPALA-6917: Limit impalad mem-limit to 12GB. This changes the memlimit choosing code in start-impala-cluster to have a maximum of 12GB. In 68GB machines, this has the effect of reducing the memlimit from ~15.8GB to 12GB. On machines with less than 51.4 GB of RAM, this has no effect. I'm tinkering with this threshold because ASAN builds sometimes die at the hands of the OOM killer on m2.4xlarge machines (68GB). My theory for why it's only sometimes is that during the parallel tests, our memory usage could vary widely depending on which tests and queries execute. End-users don't see this code; this is only used by our minicluster tests to test Impala. I have run the ASAN build with this change successfully, though I've found this particular OOM seems to come and go. Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Reviewed-on: http://gerrit.cloudera.org:8080/10051 Reviewed-by: Philip Zeyliger Tested-by: Impala Public Jenkins --- M bin/start-impala-cluster.py 1 file changed, 8 insertions(+), 2 deletions(-) Approvals: Philip Zeyliger: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 3 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6880: disable flaky bloom filter test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10122 ) Change subject: IMPALA-6880: disable flaky bloom filter test .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2337/ -- To view, visit http://gerrit.cloudera.org:8080/10122 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I595645b0f2875614294adc7abb4572aec1be8ad5 Gerrit-Change-Number: 10122 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 19:48:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6880: disable flaky bloom filter test
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10122 ) Change subject: IMPALA-6880: disable flaky bloom filter test .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10122 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I595645b0f2875614294adc7abb4572aec1be8ad5 Gerrit-Change-Number: 10122 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 18:53:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6070: Further improvements to test-with-docker.
Philip Zeyliger has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10123 Change subject: IMPALA-6070: Further improvements to test-with-docker. .. IMPALA-6070: Further improvements to test-with-docker. This commit tackles a few additions and improvements to test-with-docker. In general, I'm adding workloads (e.g., exhaustive, rat-check), tuning memory setting and parallelism, and trying to speed things up. Bug fixes: * Embarassingly, I was still skipping thrift-server-test in the backend tests. This was a mistake in handling feedback from my last review. * I made the timeline a little bit taller to clip less. Adding workloads: * I added the RAT licensing check. * I added exhaustive runs. This led me to model the suites a little it more in Python, with a class representing a suite with a bunch of data about the suite. It's not perfect and still coupled with the entrypoint.sh shell script, but it feels workable. As part of adding exhaustive tests, I had to re-work the timeout handling, since now different suites meaningfully have different timeouts. Speed ups: * To speed up test runs, I added a mechanism to split py.test suites into multiple shards with a py.test argument. This involved a little bit of work in conftest.py, and exposing $RUN_CUSTOM_CLUSTER_TESTS_ARGS in run-all-tests.sh. Furthermore, I moved a bit more logic about managing the list of suites into Python. * Doing the full build with "-notests" and only building the backend tests in the relevant target that needs them. This speeds up "docker commit" significantly by removing about 20GB from the container. I had to indicates that expr-codegen-test depends on expr-codegen-test-ir, which was missing. * I sped up copying the Kudu data: previously I did both a move and a copy; now I'm doing a move followed by a move. One of the moves is cross-filesystem so is slow, but this does half the amount of copying. Memory usage: * I tweaked the memlimit_gb settings to have a higher default. I've been fighting empirically to have the tests run well on c4.8xlarge and m4.10xlarge. The more memory a minicluster and test suite run uses, the fewer parallel suites we can run. By observing the peak processes at the tail of a run (with a new "memory_usage" function that uses a ps/sort/awk trick) and by observing peak container total_rss, I found that we had several JVMs that didn't have Xmx settings set. I added Xms/Xmx settings in a few places: * The non-first Impalad does very little JVM work, so having an Xmx keeps it small, even in the parallel tests. * Datanodes do work, but they essentially were never garbage collecting, because JVM defaults let them use up to 1/4th the machine memory. (I observed this based on RSS at the end of the run; nothing fancier.) Adding Xms/Xmx settings helped. * Similarly, I piped the settings through to HBase. A few daemons still run without resource limitations, but they don't seem to be a problem. Change-Id: I43fe124f00340afa21ad1eeb6432d6d50151ca7c --- M be/src/exprs/CMakeLists.txt M bin/run-all-tests.sh M docker/entrypoint.sh M docker/monitor.py M docker/test-with-docker.py M docker/timeline.html.template M testdata/bin/run-hbase.sh M testdata/cluster/node_templates/common/etc/init.d/hdfs-common M tests/conftest.py M tests/run-tests.py 10 files changed, 426 insertions(+), 148 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/23/10123/1 -- To view, visit http://gerrit.cloudera.org:8080/10123 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I43fe124f00340afa21ad1eeb6432d6d50151ca7c Gerrit-Change-Number: 10123 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal
[Impala-ASF-CR] IMPALA-6880: disable flaky bloom filter test
Thomas Tauber-Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10122 Change subject: IMPALA-6880: disable flaky bloom filter test .. IMPALA-6880: disable flaky bloom filter test This test is made flaky by IMPALA-6338. While that is being worked on, temporarily disable this test. Change-Id: I595645b0f2875614294adc7abb4572aec1be8ad5 --- M testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test 1 file changed, 2 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/10122/1 -- To view, visit http://gerrit.cloudera.org:8080/10122 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I595645b0f2875614294adc7abb4572aec1be8ad5 Gerrit-Change-Number: 10122 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-6869: [DOCS] Update Known Issues doc for 2.12
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10101 ) Change subject: IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 .. Patch Set 1: Probably best for Vuk and/or Sailesh to review this since they've had their finger on the pulse for 2.12 -- To view, visit http://gerrit.cloudera.org:8080/10101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 Gerrit-Change-Number: 10101 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 18:49:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6869: [DOCS] Update Known Issues doc for 2.12
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10101 ) Change subject: IMPALA-6869: [DOCS] Update Known Issues doc for 2.12 .. Patch Set 1: Could someone review this for 2.12? -- To view, visit http://gerrit.cloudera.org:8080/10101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4638be7e488546287e3555945bb691a588ec6f09 Gerrit-Change-Number: 10101 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 18:45:55 + Gerrit-HasComments: No
[Impala-ASF-CR](2.x) IMPALA-6847: work around high memory estimates for AC
Hello Impala Public Jenkins, I'd like you to do a code review. Please visit http://gerrit.cloudera.org:8080/10121 to review the following change. Change subject: IMPALA-6847: work around high memory estimates for AC .. IMPALA-6847: work around high memory estimates for AC Adds MAX_MEM_ESTIMATE_FOR_ADMISSION query option, which takes effect if and only if * Memory-based admission control is enabled for the pool * No mem_limit is set (i.e. best practices are not being followed) In that case min(MAX_MEM_ESTIMATE_FOR_ADMISSION, mem_estimate) is used for admission control instead of mem_estimate. This provides a way to override the planner's estimate if it happens to be incorrect and are preventing the query from running. Setting MEM_LIMIT is usually a better alternative but sometimes it is not feasible to set MEM_LIMIT for each individual query. Testing: Added an admission control test to verify that query option allows queries with high estimates to run. Also tested manually on a minicluster started with: start-impala-cluster.py --impalad_args='-vmodule admission-controller=3 \ -default_pool_mem_limit 12884901888' Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e Reviewed-on: http://gerrit.cloudera.org:8080/10058 Reviewed-by: Tim Armstrong Tested-by: Impala Public Jenkins --- M be/src/service/query-options-test.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M fe/src/main/java/org/apache/impala/service/Frontend.java A testdata/workloads/functional-query/queries/QueryTest/admission-reject-mem-estimate.test M tests/custom_cluster/test_admission_controller.py 8 files changed, 135 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/10121/1 -- To view, visit http://gerrit.cloudera.org:8080/10121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: newchange Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e Gerrit-Change-Number: 10121 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-6886: [DOCS] Removed the missed keyref for impala cluster sizing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10118 ) Change subject: IMPALA-6886: [DOCS] Removed the missed keyref for impala_cluster_sizing .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10118 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieab3eac8027c57a9a1a30a77ac0e07686a92f2b0 Gerrit-Change-Number: 10118 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 19 Apr 2018 18:00:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6886: [DOCS] Removed the missed keyref for impala cluster sizing
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10118 ) Change subject: IMPALA-6886: [DOCS] Removed the missed keyref for impala_cluster_sizing .. IMPALA-6886: [DOCS] Removed the missed keyref for impala_cluster_sizing Change-Id: Ieab3eac8027c57a9a1a30a77ac0e07686a92f2b0 Reviewed-on: http://gerrit.cloudera.org:8080/10118 Reviewed-by: Alex Rodoni Tested-by: Impala Public Jenkins --- M docs/impala_keydefs.ditamap M docs/topics/impala_new_features.xml 2 files changed, 2 insertions(+), 4 deletions(-) Approvals: Alex Rodoni: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10118 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ieab3eac8027c57a9a1a30a77ac0e07686a92f2b0 Gerrit-Change-Number: 10118 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-6884: Skip test misaligned orc stripes on invalid platforms
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10117 ) Change subject: IMPALA-6884: Skip test_misaligned_orc_stripes on invalid platforms .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2335/ -- To view, visit http://gerrit.cloudera.org:8080/10117 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8395b4eb06847a35c630b3ef43e289960d78382a Gerrit-Change-Number: 10117 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 17:59:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5893: Remove old kinit code for Impala 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9941 ) Change subject: IMPALA-5893: Remove old kinit code for Impala 3 .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2334/ -- To view, visit http://gerrit.cloudera.org:8080/9941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b Gerrit-Change-Number: 9941 Gerrit-PatchSet: 5 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 19 Apr 2018 17:50:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5893: Remove old kinit code for Impala 3
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/9941 ) Change subject: IMPALA-5893: Remove old kinit code for Impala 3 .. Patch Set 5: Code-Review+2 (1 comment) GVO failed due to PyPi issue. Rebase, Carry +2. http://gerrit.cloudera.org:8080/#/c/9941/4/be/src/rpc/authentication.cc File be/src/rpc/authentication.cc: http://gerrit.cloudera.org:8080/#/c/9941/4/be/src/rpc/authentication.cc@a80 PS4, Line 80: > Maybe this needs to be moved to the "graveyard" rather than removed altoget Yes, I added a reference in the graveyard now. -- To view, visit http://gerrit.cloudera.org:8080/9941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b Gerrit-Change-Number: 9941 Gerrit-PatchSet: 5 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 19 Apr 2018 17:50:19 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5893: Remove old kinit code for Impala 3
Hello Michael Ho, Dan Hecht, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/9941 to look at the new patch set (#5). Change subject: IMPALA-5893: Remove old kinit code for Impala 3 .. IMPALA-5893: Remove old kinit code for Impala 3 We've gone through a couple of releases with Kudu's kinit as the default way to use kerberos and we've not come across any major issues. Since we're going to have a major release soon, it's time to get rid of the old Kinit code that's largely unused for a while now. Testing: Made sure that our current kerberos tests continue to work without the old code. Cherry-picks: not for 2.x Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b --- M be/src/common/global-flags.cc M be/src/rpc/auth-provider.h M be/src/rpc/authentication.cc M be/src/rpc/rpc-mgr-kerberized-test.cc M be/src/rpc/thrift-server-test.cc M be/src/testutil/mini-kdc-wrapper.h 6 files changed, 11 insertions(+), 110 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/9941/5 -- To view, visit http://gerrit.cloudera.org:8080/9941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b Gerrit-Change-Number: 9941 Gerrit-PatchSet: 5 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-6884: Skip test misaligned orc stripes on invalid platforms
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10117 ) Change subject: IMPALA-6884: Skip test_misaligned_orc_stripes on invalid platforms .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10117 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8395b4eb06847a35c630b3ef43e289960d78382a Gerrit-Change-Number: 10117 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 17:44:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6886: [DOCS] Removed the missed keyref for impala cluster sizing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10118 ) Change subject: IMPALA-6886: [DOCS] Removed the missed keyref for impala_cluster_sizing .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-docs-submit/253/ -- To view, visit http://gerrit.cloudera.org:8080/10118 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieab3eac8027c57a9a1a30a77ac0e07686a92f2b0 Gerrit-Change-Number: 10118 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 19 Apr 2018 17:41:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6886: [DOCS] Removed the missed keyref for impala cluster sizing
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10118 ) Change subject: IMPALA-6886: [DOCS] Removed the missed keyref for impala_cluster_sizing .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10118 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieab3eac8027c57a9a1a30a77ac0e07686a92f2b0 Gerrit-Change-Number: 10118 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Comment-Date: Thu, 19 Apr 2018 17:41:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6821: Push down limits into Kudu
Thomas Tauber-Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10119 Change subject: IMPALA-6821: Push down limits into Kudu .. IMPALA-6821: Push down limits into Kudu This patch takes advantage of a recent change in Kudu (KUDU-16) that exposes the ability to set limits on KuduScanners. Since each KuduScanner corresponds to a scan token, and there will be multiple scan tokens per query, this is just a performance optimization in cases where the limit is smaller than the number of rows per token, and Impala still needs to apply the limit on our side for cases where the limit is greater than the number of rows per token. Testing: - Updated test_limit to run against Kudu by moving the few test cases that Kudu cannot support to a different test. - For the query 'select * from tpch_kudu.lineitem limit 1', a best case perf scenario for this change where the limit is highly effective, the time spent in the Kudu scan node was reduced from 6.107ms to 3.498ms (avg over 3 runs). - For the query 'select count(*) from tpch_kudu.lineitem limit 100', a worst case perf scenario for this change where the limit is ineffective, the time spent in the Kudu scan node was essentially unchanged, 62.387ms previously vs. 64.168ms (avg over 3 runs). Change-Id: Ibe35e70065d8706b575e24fe20902cd405b49941 --- M be/src/exec/kudu-scanner.cc A testdata/workloads/functional-query/queries/QueryTest/limit-nondeterministic.test M testdata/workloads/functional-query/queries/QueryTest/limit.test M tests/query_test/test_queries.py 4 files changed, 87 insertions(+), 73 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/10119/1 -- To view, visit http://gerrit.cloudera.org:8080/10119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ibe35e70065d8706b575e24fe20902cd405b49941 Gerrit-Change-Number: 10119 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-6886: [DOCS] Removed the missed keyref for impala cluster sizing
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10118 Change subject: IMPALA-6886: [DOCS] Removed the missed keyref for impala_cluster_sizing .. IMPALA-6886: [DOCS] Removed the missed keyref for impala_cluster_sizing Change-Id: Ieab3eac8027c57a9a1a30a77ac0e07686a92f2b0 --- M docs/impala_keydefs.ditamap M docs/topics/impala_new_features.xml 2 files changed, 2 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/10118/1 -- To view, visit http://gerrit.cloudera.org:8080/10118 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ieab3eac8027c57a9a1a30a77ac0e07686a92f2b0 Gerrit-Change-Number: 10118 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-6884: Skip test misaligned orc stripes on invalid platforms
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10117 Change subject: IMPALA-6884: Skip test_misaligned_orc_stripes on invalid platforms .. IMPALA-6884: Skip test_misaligned_orc_stripes on invalid platforms test_misaligned_orc_stripes relies on generating files with a specific number of HDFS blocks. This is not possible on S3, ADLS, or Isilon, so the test needs to be skipped on those platforms. Additionally, it relies on running multiple impalads, so it needs to be skipped on local filesystem tests. Change-Id: I8395b4eb06847a35c630b3ef43e289960d78382a --- M tests/query_test/test_scanners.py 1 file changed, 4 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/10117/1 -- To view, visit http://gerrit.cloudera.org:8080/10117 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8395b4eb06847a35c630b3ef43e289960d78382a Gerrit-Change-Number: 10117 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2332/ -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 2 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 16:37:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6917: Limit impalad mem-limit to 12GB.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/10051 ) Change subject: IMPALA-6917: Limit impalad mem-limit to 12GB. .. Patch Set 2: Code-Review+2 Rebase. -- To view, visit http://gerrit.cloudera.org:8080/10051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8024414c5c23bb42cce912d8f34cd0b787e0e39a Gerrit-Change-Number: 10051 Gerrit-PatchSet: 2 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Apr 2018 16:37:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5842: Write page index in Parquet files
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/9693 ) Change subject: IMPALA-5842: Write page index in Parquet files .. Patch Set 10: (1 comment) http://gerrit.cloudera.org:8080/#/c/9693/10/be/src/exec/hdfs-parquet-table-writer.cc File be/src/exec/hdfs-parquet-table-writer.cc: http://gerrit.cloudera.org:8080/#/c/9693/10/be/src/exec/hdfs-parquet-table-writer.cc@301 PS10, Line 301: std::vector min_values_; > Could we not just track the size of page_stats.min_value in a memtracker in Options 1 or 2 seem fine to me. -- To view, visit http://gerrit.cloudera.org:8080/9693 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icbacf7fe3b7672e3ce719261ecef445b16f8dec9 Gerrit-Change-Number: 9693 Gerrit-PatchSet: 10 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Anonymous Coward #248 Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 19 Apr 2018 16:13:21 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6131: Track time of last statistics update in metadata
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/10116 ) Change subject: IMPALA-6131: Track time of last statistics update in metadata .. Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/10116/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/10116/2//COMMIT_MSG@28 PS2, Line 28: ablo > typo, and missing 'to' Done http://gerrit.cloudera.org:8080/#/c/10116/2//COMMIT_MSG@30 PS2, Line 30: > + 'be'? Done http://gerrit.cloudera.org:8080/#/c/10116/2/fe/src/main/java/org/apache/impala/catalog/KuduTable.java File fe/src/main/java/org/apache/impala/catalog/KuduTable.java: http://gerrit.cloudera.org:8080/#/c/10116/2/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@239 PS2, Line 239: msTable_.deepCopy(); > Why do we deep copy it now? Do be honest, I am not sure about this part - without deep copy, DESCRIBE TABLE did not work as intended for Kudu tables, transient_lastDdlTime was missing. I consider the Kudu part to be "work in progress". Note that CatalogOpExecutor always deep copies the tables before calling applyAlterTable(), which calls HmsClient's alter_table(), so this seams to be the normal way to do it in Impala. -- To view, visit http://gerrit.cloudera.org:8080/10116 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I59a671ac29d352bd92ce40d5cb6662bb23f146b5 Gerrit-Change-Number: 10116 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 19 Apr 2018 15:59:54 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6131: Track time of last statistics update in metadata
Hello Lars Volker, Zoltan Borok-Nagy, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10116 to look at the new patch set (#3). Change subject: IMPALA-6131: Track time of last statistics update in metadata .. IMPALA-6131: Track time of last statistics update in metadata The timestamp of the last COMPUTE STATS operation is saved to table property "impala.lastComputeStatsTime". The format is the same as in "transient_lastDdlTime", so the two can be compared to check if the schema has changed since computing statistics. Other changes: - handling of "transient_lastDdlTime" is simplified to use HMS's default mechanism by removing this property before calling alter_table() - computing / setting stats on HDFS tables no longer increases "transient_lastDdlTime" - for Kudu behavior, see notes Notes: - Kudu tables work a bit differently, because they overwrite their HMS representation whenever they are reloaded from Kudu. This means that "transient_lastDdlTime" is increased during several operations, for example any operation that loads the table after INVALIDATE METADATA. I did not change this behavior. Kudu will be able to sync its tables to HMS in the near future (see KUDU-2191), so the Kudu metadata handling in Impala may need to be redesigned. Testing: tests/metadata/test_last_ddl_time_update.py is extended by - also checking "impala.lastComputeStatsTime" - testing more SQL statements - tests for Kudu tables Note that test_last_ddl_time_update.py is ran only in exhaustive testing. Change-Id: I59a671ac29d352bd92ce40d5cb6662bb23f146b5 --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/metadata/test_last_ddl_time_update.py 5 files changed, 105 insertions(+), 116 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/10116/3 -- To view, visit http://gerrit.cloudera.org:8080/10116 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I59a671ac29d352bd92ce40d5cb6662bb23f146b5 Gerrit-Change-Number: 10116 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-6131: Track time of last statistics update in metadata
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/10116 ) Change subject: IMPALA-6131: Track time of last statistics update in metadata .. Patch Set 2: (3 comments) Looks good, only had some nit comments and questions. http://gerrit.cloudera.org:8080/#/c/10116/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/10116/2//COMMIT_MSG@28 PS2, Line 28: ablo typo, and missing 'to' http://gerrit.cloudera.org:8080/#/c/10116/2//COMMIT_MSG@30 PS2, Line 30: + 'be'? http://gerrit.cloudera.org:8080/#/c/10116/2/fe/src/main/java/org/apache/impala/catalog/KuduTable.java File fe/src/main/java/org/apache/impala/catalog/KuduTable.java: http://gerrit.cloudera.org:8080/#/c/10116/2/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@239 PS2, Line 239: msTable_.deepCopy(); Why do we deep copy it now? Don't we need to set msTable_ if there were no exceptions? -- To view, visit http://gerrit.cloudera.org:8080/10116 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I59a671ac29d352bd92ce40d5cb6662bb23f146b5 Gerrit-Change-Number: 10116 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 19 Apr 2018 15:16:01 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3307: Add support for IANA time-zone db
Zoltan Borok-Nagy has posted comments on this change. (
http://gerrit.cloudera.org:8080/9986 )
Change subject: IMPALA-3307: Add support for IANA time-zone db
..
Patch Set 2:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/9986/2/be/src/benchmarks/convert-timestamp-benchmark.cc
File be/src/benchmarks/convert-timestamp-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/9986/2/be/src/benchmarks/convert-timestamp-benchmark.cc@150
PS2, Line 150: void AddTestDataDateTimes(vector& data, int n,
const string& startstr) {
Since it's in a benchmark it doesn't really matter, anyway, some nit comments:
- output parameters should be listed last and they should be pointers
- in this particular case I think it would be better to return
vector
http://gerrit.cloudera.org:8080/#/c/9986/2/be/src/benchmarks/convert-timestamp-benchmark.cc@174
PS2, Line 174: > >
Nit: since C++11 you don't need to put spaces between right angle brackets.
http://gerrit.cloudera.org:8080/#/c/9986/2/be/src/benchmarks/convert-timestamp-benchmark.cc@228
PS2, Line 228: const shared_ptr > data_
Nit: I think using 'const vector&' would be simpler. I don't really see
why we need shared_ptr here.
Also, in 'const shared_ptr' only the pointer is const, the pointed object is
not.
http://gerrit.cloudera.org:8080/#/c/9986/2/be/src/benchmarks/convert-timestamp-benchmark.cc@319
PS2, Line 319: boost_throw_if_date_out_of_range(local_time.date());
Is this function call really needed here? Don't we trust boost that it
validates the date correctly?
http://gerrit.cloudera.org:8080/#/c/9986/2/be/src/benchmarks/convert-timestamp-benchmark.cc@608
PS2, Line 608: if (cctz_utc_to_unix_data.get_result() !=
glibc_utc_to_unix_data.get_result()) {
: cerr << "cctz/glibc utc_to_unix results do not match!" <<
endl;
: return 1;
: }
: if (boost_utc_to_unix_data.get_result() !=
glibc_utc_to_unix_data.get_result()) {
: cerr << "boost/glibc utc_to_unix results do not match!" <<
endl;
: return 1;
: }
The other benchmarks don't need this validity check?
It could be implemented in a helper function that takes a vector of TestData.
http://gerrit.cloudera.org:8080/#/c/9986/2/be/src/exprs/timestamp-functions.cc
File be/src/exprs/timestamp-functions.cc:
http://gerrit.cloudera.org:8080/#/c/9986/2/be/src/exprs/timestamp-functions.cc@122
PS2, Line 122: or
Nit: of
http://gerrit.cloudera.org:8080/#/c/9986/2/be/src/exprs/timezone_db.h
File be/src/exprs/timezone_db.h:
http://gerrit.cloudera.org:8080/#/c/9986/2/be/src/exprs/timezone_db.h@21
PS2, Line 21: boost
I think we should use the unordered_map class from the C++ STL.
http://gerrit.cloudera.org:8080/#/c/9986/2/be/src/exprs/timezone_db.h@56
PS2, Line 56: TZ_MAP
TimezoneMap? I don't think a typename should be all capitals.
--
To view, visit http://gerrit.cloudera.org:8080/9986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I93c1fbffe81f067919706e30db0a34d0e58e7e77
Gerrit-Change-Number: 9986
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Thu, 19 Apr 2018 14:20:32 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6131: Track time of last statistics update in metadata
Csaba Ringhofer has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/10116 ) Change subject: IMPALA-6131: Track time of last statistics update in metadata .. IMPALA-6131: Track time of last statistics update in metadata The timestamp of the last COMPUTE STATS operation is saved to table property "impala.lastComputeStatsTime". The format is the same as in "transient_lastDdlTime", so the two can be compared to check if the schema has changed since computing statistics. Other changes: - handling of "transient_lastDdlTime" is simplified to use HMS's default mechanism by removing this property before calling alter_table() - computing / setting stats on HDFS tables no longer increases "transient_lastDdlTime" - for Kudu behavior, see notes Notes: - Kudu tables work a bit differently, because they overwrite their HMS representation whenever they are reloaded from Kudu. This means that "transient_lastDdlTime" is increased during several operations, for example any operation that loads the table after INVALIDATE METADATA. I did not change this behavior. Kudu will be ablo sync its tables to HMS in the near future (see KUDU-2191), so the Kudu metadata handling in Impala may need to redesigned. Testing: tests/metadata/test_last_ddl_time_update.py is extended by - also checking "impala.lastComputeStatsTime" - testing more SQL statements - tests for Kudu tables Note that test_last_ddl_time_update.py is ran only in exhaustive testing. Change-Id: I59a671ac29d352bd92ce40d5cb6662bb23f146b5 --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/metadata/test_last_ddl_time_update.py 5 files changed, 105 insertions(+), 116 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/10116/2 -- To view, visit http://gerrit.cloudera.org:8080/10116 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I59a671ac29d352bd92ce40d5cb6662bb23f146b5 Gerrit-Change-Number: 10116 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer
