[Impala-ASF-CR] IMPALA-11378: Allow INSERT OVERWRITE for bucket tranforms in some cases
Tamas Mate has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/18649 ) Change subject: IMPALA-11378: Allow INSERT OVERWRITE for bucket tranforms in some cases .. IMPALA-11378: Allow INSERT OVERWRITE for bucket tranforms in some cases This change has been considered only for Iceberg tables mainly for table maintenance reasons. Iceberg table writes create new snapshots and these can accumulate over time. This commit allows a simple form of compaction of these snapshots, later the old snapshots can be expired. All types of INSERT OVERWRITES have been blocked in case partition evolution is in place, to avoid unexpected data loss. For bucketed tables, the following syntax is allowed to be executed: INSERT OVERWRITE ice_tbl SELECT * FROM ice_tbl; The source and target table has to be the same and specified, only SELECT '*' select_queries are allowed. These requirements are also in place to avoid unexpected data loss. Testing: - Added e2e tests. Change-Id: Ibd1bc19d839297246eadeb754cdeeec1e306098a --- M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test M testdata/workloads/functional-query/queries/QueryTest/iceberg-overwrite.test 3 files changed, 88 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/18649/3 -- To view, visit http://gerrit.cloudera.org:8080/18649 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibd1bc19d839297246eadeb754cdeeec1e306098a Gerrit-Change-Number: 18649 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-11378: Allow INSERT OVERWRITE for bucket tranforms in some cases
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18649 ) Change subject: IMPALA-11378: Allow INSERT OVERWRITE for bucket tranforms in some cases .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10958/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18649 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibd1bc19d839297246eadeb754cdeeec1e306098a Gerrit-Change-Number: 18649 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 13 Jul 2022 13:03:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11034: Resolve schema of old data files in migrated Iceberg tables
Gergely Fürnstáhl has posted comments on this change. ( http://gerrit.cloudera.org:8080/18639 ) Change subject: IMPALA-11034: Resolve schema of old data files in migrated Iceberg tables .. Patch Set 14: (2 comments) http://gerrit.cloudera.org:8080/#/c/18639/14/be/src/exec/parquet/parquet-metadata-utils.cc File be/src/exec/parquet/parquet-metadata-utils.cc: http://gerrit.cloudera.org:8080/#/c/18639/14/be/src/exec/parquet/parquet-metadata-utils.cc@847 PS14, Line 847: break > We could return idx here and INVALID_ID at L850, in case we didn't find a c I would not return INVALID_ID on L850. idx >= size signals that we ran out of columns, INVALID_ID signals that the file is corrupted and handled differently by the caller http://gerrit.cloudera.org:8080/#/c/18639/12/tests/query_test/test_iceberg.py File tests/query_test/test_iceberg.py: http://gerrit.cloudera.org:8080/#/c/18639/12/tests/query_test/test_iceberg.py@106 PS12, Line 106: time.sleep(1) > you should add annotation "@SkipIf.not_hdfs" S3A supports "hadoop fs" calls and "hdfs dfs" is translated into that. Tested it on S3 environment and the test passes, no need to skip -- To view, visit http://gerrit.cloudera.org:8080/18639 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I77570bbfc2fcc60c2756812d7210110e8cc11ccc Gerrit-Change-Number: 18639 Gerrit-PatchSet: 14 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 13 Jul 2022 14:03:17 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11355: Add STRING overloads for hour/minute/second/millisecond
Hello Riza Suminto, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18718
to look at the new patch set (#6).
Change subject: IMPALA-11355: Add STRING overloads for
hour/minute/second/millisecond
..
IMPALA-11355: Add STRING overloads for hour/minute/second/millisecond
IMPALA-9531 dropped support for "dateless timestamps",
e.g. cast("12:05:05" as timestamp) now returns NULL.
This led to breaking functions like minute("12:05:05"), as minute()
expects a timestamp, and Impala adds an implicit cast, so what actually
happens is minute(cast("12:05:05" as timestamp)), which returns NULL.
This change adds overloads for similar functions that take STRING
instead of TIMESTAMP parameter. The same functions already take a
STRING parameter in Hive and mySQL.
The changes in the parser mainly restore code removed in IMPALA-9531.
Note that these functions could be potentially optimized by returning
parts of the parse result without converting them to boost time first,
but this is not done here to make the change minimal.
Testing:
- restored related tests in expr-test and added some new ones for
malformed time-of-day strings
- added benchmarks for the new overloads and fixed the ones for the
old functions (they tested NULL)
Change-Id: I6cc1c851ee71ab4fcc58105c7e9931155a483679
---
M be/src/benchmarks/expr-benchmark.cc
M be/src/exprs/expr-test.cc
M be/src/exprs/timestamp-functions-ir.cc
M be/src/exprs/timestamp-functions.h
M be/src/runtime/date-parse-util.cc
M be/src/runtime/datetime-simple-date-format-parser.cc
M be/src/runtime/datetime-simple-date-format-parser.h
M be/src/runtime/timestamp-parse-util.cc
M be/src/runtime/timestamp-parse-util.h
M common/function-registry/impala_functions.py
10 files changed, 204 insertions(+), 80 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/18718/6
--
To view, visit http://gerrit.cloudera.org:8080/18718
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6cc1c851ee71ab4fcc58105c7e9931155a483679
Gerrit-Change-Number: 18718
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Riza Suminto
[Impala-ASF-CR] IMPALA-11034: Resolve schema of old data files in migrated Iceberg tables
Gergely Fürnstáhl has uploaded a new patch set (#16). ( http://gerrit.cloudera.org:8080/18639 ) Change subject: IMPALA-11034: Resolve schema of old data files in migrated Iceberg tables .. IMPALA-11034: Resolve schema of old data files in migrated Iceberg tables When external tables are converted to Iceberg, the data files remain intact, thus missing field IDs. Previously, Impala used name based column resolution in this case. Added a feature to traverse through the data files before column resolution and assign field IDs the same way as iceberg would, to be able to use field ID based column resolutions. Testing: Default resolution method was changed to field id for migrated tables, existing tests use that from now. Added new tests to cover edge cases with complex types and schema evolution. Change-Id: I77570bbfc2fcc60c2756812d7210110e8cc11ccc --- M be/src/exec/orc-metadata-utils.cc M be/src/exec/orc-metadata-utils.h M be/src/exec/parquet/parquet-metadata-utils.cc M be/src/exec/parquet/parquet-metadata-utils.h M testdata/data/README A testdata/data/iceberg_test/iceberg_migrated_alter_test/00_0 A testdata/data/iceberg_test/iceberg_migrated_alter_test/metadata/c9f83a82-60f4-443b-9ca4-359cad16fe12-m0.avro A testdata/data/iceberg_test/iceberg_migrated_alter_test/metadata/snap-2941076094076108396-1-c9f83a82-60f4-443b-9ca4-359cad16fe12.avro A testdata/data/iceberg_test/iceberg_migrated_alter_test/metadata/v1.metadata.json A testdata/data/iceberg_test/iceberg_migrated_alter_test/metadata/v2.metadata.json A testdata/data/iceberg_test/iceberg_migrated_alter_test/metadata/version-hint.text A testdata/data/iceberg_test/iceberg_migrated_alter_test_orc/00_0 A testdata/data/iceberg_test/iceberg_migrated_alter_test_orc/metadata/340a3b82-71e3-4f50-b030-aecb5a5ea730-m0.avro A testdata/data/iceberg_test/iceberg_migrated_alter_test_orc/metadata/snap-2205107170480729038-1-340a3b82-71e3-4f50-b030-aecb5a5ea730.avro A testdata/data/iceberg_test/iceberg_migrated_alter_test_orc/metadata/v1.metadata.json A testdata/data/iceberg_test/iceberg_migrated_alter_test_orc/metadata/v2.metadata.json A testdata/data/iceberg_test/iceberg_migrated_alter_test_orc/metadata/version-hint.text A testdata/data/iceberg_test/iceberg_migrated_complex_test/00_0 A testdata/data/iceberg_test/iceberg_migrated_complex_test/metadata/152e384f-2851-44b7-9ada-1bfbec74e9fc-m0.avro A testdata/data/iceberg_test/iceberg_migrated_complex_test/metadata/snap-3911840040574896148-1-152e384f-2851-44b7-9ada-1bfbec74e9fc.avro A testdata/data/iceberg_test/iceberg_migrated_complex_test/metadata/v1.metadata.json A testdata/data/iceberg_test/iceberg_migrated_complex_test/metadata/v2.metadata.json A testdata/data/iceberg_test/iceberg_migrated_complex_test/metadata/version-hint.text A testdata/data/iceberg_test/iceberg_migrated_complex_test_orc/00_0 A testdata/data/iceberg_test/iceberg_migrated_complex_test_orc/metadata/8588fd4b-13c1-4451-80ad-5cf71a959b94-m0.avro A testdata/data/iceberg_test/iceberg_migrated_complex_test_orc/metadata/snap-3622599918649152504-1-8588fd4b-13c1-4451-80ad-5cf71a959b94.avro A testdata/data/iceberg_test/iceberg_migrated_complex_test_orc/metadata/v1.metadata.json A testdata/data/iceberg_test/iceberg_migrated_complex_test_orc/metadata/v2.metadata.json A testdata/data/iceberg_test/iceberg_migrated_complex_test_orc/metadata/version-hint.text A testdata/workloads/functional-query/queries/QueryTest/iceberg-migrated-table-field-id-resolution.test M tests/common/file_utils.py M tests/query_test/test_iceberg.py 32 files changed, 1,874 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/18639/16 -- To view, visit http://gerrit.cloudera.org:8080/18639 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I77570bbfc2fcc60c2756812d7210110e8cc11ccc Gerrit-Change-Number: 18639 Gerrit-PatchSet: 16 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-11355: Add STRING overloads for hour/minute/second/millisecond
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/18718 )
Change subject: IMPALA-11355: Add STRING overloads for
hour/minute/second/millisecond
..
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18718/5/be/src/exprs/expr-test.cc
File be/src/exprs/expr-test.cc:
http://gerrit.cloudera.org:8080/#/c/18718/5/be/src/exprs/expr-test.cc@6967
PS5, Line 6967: TestIsNull("minute('09-10-11')", TYPE_INT);
> Should it be "hour('838:59:59')"?
changed it to hour - note that it doesn't matter too much, as all these
functions return NULL if any component of time is invalid
>MySQL's hour function seems to accept string where the hour part is more than
>23.
Checked with Hive and it doesn't accept it. I think that not accepting it is
the safest way.
--
To view, visit http://gerrit.cloudera.org:8080/18718
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6cc1c851ee71ab4fcc58105c7e9931155a483679
Gerrit-Change-Number: 18718
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Riza Suminto
Gerrit-Comment-Date: Wed, 13 Jul 2022 14:06:18 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11355: Add STRING overloads for hour/minute/second/millisecond
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18718 ) Change subject: IMPALA-11355: Add STRING overloads for hour/minute/second/millisecond .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10959/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18718 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6cc1c851ee71ab4fcc58105c7e9931155a483679 Gerrit-Change-Number: 18718 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Wed, 13 Jul 2022 14:24:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11034: Resolve schema of old data files in migrated Iceberg tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18639 ) Change subject: IMPALA-11034: Resolve schema of old data files in migrated Iceberg tables .. Patch Set 16: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10960/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18639 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I77570bbfc2fcc60c2756812d7210110e8cc11ccc Gerrit-Change-Number: 18639 Gerrit-PatchSet: 16 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 13 Jul 2022 14:26:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11355: Add STRING overloads for hour/minute/second/millisecond
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/18718 ) Change subject: IMPALA-11355: Add STRING overloads for hour/minute/second/millisecond .. Patch Set 6: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/18718 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6cc1c851ee71ab4fcc58105c7e9931155a483679 Gerrit-Change-Number: 18718 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Wed, 13 Jul 2022 15:08:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-886: Support displaying HBase cols in the order from HMS
Hello Quanlong Huang, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18635 to look at the new patch set (#5). Change subject: IMPALA-886: Support displaying HBase cols in the order from HMS .. IMPALA-886: Support displaying HBase cols in the order from HMS Before this patch catalogd always ordered HBase columns by lexicographically by family/qualifier. This is incompatible with other table formats and the way Hive handles HBase tables, where the order comes from HMS as defined during CREATE TABLE. I don't know of any valid reason behind this old behavior, it probably just made the implementation a bit easier by doing the ordering in FE instead of BE - the BE actually needs this ordering during scanning as the HBase API returns results in this order, but this should have no effect on other parts of Impala. Added flag use_hms_column_order_for_hbase_tables (used by catalogd) to decide whether to do this reordering: - true: keep HMS order - false: reorder by family/qualifier [default] The old way is kept as default to avoid breaking existing workloads, but it would make sense to change it in the next major release. Note that a query option would be more convenient to use, but it would be much harder to implement it as the order is decided during loading in catalogd. Testing: - added custom cluster test for use_hms_column_order_for_hbase_tables = true Change-Id: Ibc5df8b803f2ae3b93951765326cdaea706e3563 --- M be/src/exec/hbase-scan-node.cc M be/src/exec/hbase-scan-node.h M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/catalog/FeHBaseTable.java M fe/src/main/java/org/apache/impala/catalog/HBaseColumn.java M fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java A testdata/workloads/functional-query/queries/QueryTest/hbase-hms-column-order.test A tests/custom_cluster/test_hbase_hms_column_order.py 10 files changed, 156 insertions(+), 30 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/18635/5 -- To view, visit http://gerrit.cloudera.org:8080/18635 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibc5df8b803f2ae3b93951765326cdaea706e3563 Gerrit-Change-Number: 18635 Gerrit-PatchSet: 5 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-886: Support displaying HBase cols in the order from HMS
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18635 ) Change subject: IMPALA-886: Support displaying HBase cols in the order from HMS .. Patch Set 5: (4 comments) http://gerrit.cloudera.org:8080/#/c/18635/4/be/src/util/backend-gflag-util.cc File be/src/util/backend-gflag-util.cc: http://gerrit.cloudera.org:8080/#/c/18635/4/be/src/util/backend-gflag-util.cc@189 PS4, Line 189: DEFINE_bool(use_hms_column_order_for_hbase_tables, false, > nit: duplicated blank line Done http://gerrit.cloudera.org:8080/#/c/18635/4/fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java File fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java: http://gerrit.cloudera.org:8080/#/c/18635/4/fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java@136 PS4, Line 136: columns.get(i); > nit: use 'columns' directly Done http://gerrit.cloudera.org:8080/#/c/18635/4/testdata/workloads/functional-query/queries/QueryTest/hbase-hms-column-order.test File testdata/workloads/functional-query/queries/QueryTest/hbase-hms-column-order.test: http://gerrit.cloudera.org:8080/#/c/18635/4/testdata/workloads/functional-query/queries/QueryTest/hbase-hms-column-order.test@5 PS4, Line 5: = > nit: needs a space after '=' Done http://gerrit.cloudera.org:8080/#/c/18635/4/testdata/workloads/functional-query/queries/QueryTest/hbase-hms-column-order.test@31 PS4, Line 31: > It'd be nice if we can copy some tests from testdata/workloads/functional-p Added a test with predicate on string key and checked the result + that the predicate is pushed down to the plan. -- To view, visit http://gerrit.cloudera.org:8080/18635 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibc5df8b803f2ae3b93951765326cdaea706e3563 Gerrit-Change-Number: 18635 Gerrit-PatchSet: 5 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 13 Jul 2022 16:14:49 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-886: Support displaying HBase cols in the order from HMS
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18635 ) Change subject: IMPALA-886: Support displaying HBase cols in the order from HMS .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/18635/6/tests/custom_cluster/test_hbase_hms_column_order.py File tests/custom_cluster/test_hbase_hms_column_order.py: http://gerrit.cloudera.org:8080/#/c/18635/6/tests/custom_cluster/test_hbase_hms_column_order.py@43 PS6, Line 43: c flake8: E126 continuation line over-indented for hanging indent -- To view, visit http://gerrit.cloudera.org:8080/18635 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibc5df8b803f2ae3b93951765326cdaea706e3563 Gerrit-Change-Number: 18635 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 13 Jul 2022 16:18:59 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-886: Support displaying HBase cols in the order from HMS
Hello Quanlong Huang, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18635 to look at the new patch set (#6). Change subject: IMPALA-886: Support displaying HBase cols in the order from HMS .. IMPALA-886: Support displaying HBase cols in the order from HMS Before this patch catalogd always ordered HBase columns by lexicographically by family/qualifier. This is incompatible with other table formats and the way Hive handles HBase tables, where the order comes from HMS as defined during CREATE TABLE. I don't know of any valid reason behind this old behavior, it probably just made the implementation a bit easier by doing the ordering in FE instead of BE - the BE actually needs this ordering during scanning as the HBase API returns results in this order, but this should have no effect on other parts of Impala. Added flag use_hms_column_order_for_hbase_tables (used by catalogd) to decide whether to do this reordering: - true: keep HMS order - false: reorder by family/qualifier [default] The old way is kept as default to avoid breaking existing workloads, but it would make sense to change it in the next major release. Note that a query option would be more convenient to use, but it would be much harder to implement it as the order is decided during loading in catalogd. Testing: - added custom cluster test for use_hms_column_order_for_hbase_tables = true Change-Id: Ibc5df8b803f2ae3b93951765326cdaea706e3563 --- M be/src/exec/hbase-scan-node.cc M be/src/exec/hbase-scan-node.h M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/catalog/FeHBaseTable.java M fe/src/main/java/org/apache/impala/catalog/HBaseColumn.java M fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java A testdata/workloads/functional-query/queries/QueryTest/hbase-hms-column-order.test A tests/custom_cluster/test_hbase_hms_column_order.py 10 files changed, 155 insertions(+), 30 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/18635/6 -- To view, visit http://gerrit.cloudera.org:8080/18635 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibc5df8b803f2ae3b93951765326cdaea706e3563 Gerrit-Change-Number: 18635 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-886: Support displaying HBase cols in the order from HMS
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18635 ) Change subject: IMPALA-886: Support displaying HBase cols in the order from HMS .. Patch Set 6: Patcher 6 is just rebase with resolving some conflict due to new backend configs. -- To view, visit http://gerrit.cloudera.org:8080/18635 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibc5df8b803f2ae3b93951765326cdaea706e3563 Gerrit-Change-Number: 18635 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 13 Jul 2022 16:20:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-886: Support displaying HBase cols in the order from HMS
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18635 ) Change subject: IMPALA-886: Support displaying HBase cols in the order from HMS .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10961/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18635 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibc5df8b803f2ae3b93951765326cdaea706e3563 Gerrit-Change-Number: 18635 Gerrit-PatchSet: 5 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 13 Jul 2022 16:32:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-886: Support displaying HBase cols in the order from HMS
Hello Quanlong Huang, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18635 to look at the new patch set (#7). Change subject: IMPALA-886: Support displaying HBase cols in the order from HMS .. IMPALA-886: Support displaying HBase cols in the order from HMS Before this patch catalogd always ordered HBase columns lexicographically by family/qualifier. This is incompatible with other table formats and the way Hive handles HBase tables, where the order comes from HMS as defined during CREATE TABLE. I don't know of any valid reason behind this old behavior, it probably just made the implementation a bit easier by doing the ordering in FE instead of BE - the BE actually needs this ordering during scanning as the HBase API returns results in this order, but this should have no effect on other parts of Impala. Added flag use_hms_column_order_for_hbase_tables (used by catalogd) to decide whether to do this reordering: - true: keep HMS order - false: reorder by family/qualifier [default] The old way is kept as default to avoid breaking existing workloads, but it would make sense to change it in the next major release. Note that a query option would be more convenient to use, but it would be much harder to implement it as the order is decided during loading in catalogd. Testing: - added custom cluster test for use_hms_column_order_for_hbase_tables = true Change-Id: Ibc5df8b803f2ae3b93951765326cdaea706e3563 --- M be/src/exec/hbase-scan-node.cc M be/src/exec/hbase-scan-node.h M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/catalog/FeHBaseTable.java M fe/src/main/java/org/apache/impala/catalog/HBaseColumn.java M fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java A testdata/workloads/functional-query/queries/QueryTest/hbase-hms-column-order.test A tests/custom_cluster/test_hbase_hms_column_order.py 10 files changed, 155 insertions(+), 30 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/18635/7 -- To view, visit http://gerrit.cloudera.org:8080/18635 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibc5df8b803f2ae3b93951765326cdaea706e3563 Gerrit-Change-Number: 18635 Gerrit-PatchSet: 7 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-886: Support displaying HBase cols in the order from HMS
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18635 ) Change subject: IMPALA-886: Support displaying HBase cols in the order from HMS .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10962/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18635 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibc5df8b803f2ae3b93951765326cdaea706e3563 Gerrit-Change-Number: 18635 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 13 Jul 2022 16:39:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11257: Fix CMake warnings for module names and cmake minimum required
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18444 to look at the new patch set (#7). Change subject: IMPALA-11257: Fix CMake warnings for module names and cmake_minimum_required .. IMPALA-11257: Fix CMake warnings for module names and cmake_minimum_required This fixes a few different CMake warnings: 1. This removes cmake_minimum_required invocations except for the top-most CMakeLists.txt. This eliminates the warnings like this: Compatibility with CMake < 2.8.12 will be removed from a future version of CMake. Update the VERSION argument value or use a ... suffix to tell CMake that the project does not need compatibility with older versions. Moving to a later version also required setting CMAKE_ENABLE_EXPORTS to continue exporting symbols. 2. This modifies the module names so that they match the corresponding module names from Find*.cmake. This is mostly dealing with case differences. This address warnings like: The package name passed to `find_package_handle_standard_args` (PROTOBUF) does not match the name of the calling package (Protobuf). This can lead to problems in calling code that expects `find_package` result variables (e.g., `_FOUND`) to follow a certain pattern. This fixed the detection logic for KerberosPrograms, and so it required adding more Kerberos packages to bin/bootstrap_build.sh. 3. This adds a missing .cc suffix. This addresses the following warning: CMake Warning (dev) at be/src/util/CMakeLists.txt:141 (add_library): Policy CMP0115 is not set: Source file extensions must be explicit. Run "cmake --help-policy CMP0115" for policy details. Use the cmake_policy command to set the policy and suppress this warning. These fixes mostly match how these warnings were handled in Apache Kudu. Testing: - Ran GVO Change-Id: I2a97dd07cdd0831e90882a2035415ac71d670147 --- M CMakeLists.txt M be/CMakeLists.txt M be/src/exec/CMakeLists.txt M be/src/exec/parquet/CMakeLists.txt M be/src/testutil/CMakeLists.txt M be/src/util/CMakeLists.txt M bin/bootstrap_build.sh M cmake_modules/FindCrcutil.cmake M cmake_modules/FindFlatBuffers.cmake M cmake_modules/FindKerberos.cmake M cmake_modules/FindKerberosPrograms.cmake M cmake_modules/FindLibEv.cmake M cmake_modules/FindLibUnwind.cmake M cmake_modules/FindProtobuf.cmake M common/fbs/CMakeLists.txt M common/function-registry/CMakeLists.txt M common/protobuf/CMakeLists.txt M common/thrift/CMakeLists.txt 18 files changed, 22 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/44/18444/7 -- To view, visit http://gerrit.cloudera.org:8080/18444 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2a97dd07cdd0831e90882a2035415ac71d670147 Gerrit-Change-Number: 18444 Gerrit-PatchSet: 7 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11257: Fix CMake warnings for module names and cmake minimum required
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18444 ) Change subject: IMPALA-11257: Fix CMake warnings for module names and cmake_minimum_required .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10963/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18444 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2a97dd07cdd0831e90882a2035415ac71d670147 Gerrit-Change-Number: 18444 Gerrit-PatchSet: 7 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Jul 2022 18:28:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11355: Add STRING overloads for hour/minute/second/millisecond
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/18718 ) Change subject: IMPALA-11355: Add STRING overloads for hour/minute/second/millisecond .. Patch Set 6: (2 comments) http://gerrit.cloudera.org:8080/#/c/18718/6/be/src/benchmarks/expr-benchmark.cc File be/src/benchmarks/expr-benchmark.cc: http://gerrit.cloudera.org:8080/#/c/18718/6/be/src/benchmarks/expr-benchmark.cc@940 PS6, Line 940: /*benchmarks.push_back(&BenchmarkLiterals); why comments out? http://gerrit.cloudera.org:8080/#/c/18718/6/be/src/exprs/timestamp-functions-ir.cc File be/src/exprs/timestamp-functions-ir.cc: http://gerrit.cloudera.org:8080/#/c/18718/6/be/src/exprs/timestamp-functions-ir.cc@300 PS6, Line 300: nit: extra spaces -- To view, visit http://gerrit.cloudera.org:8080/18718 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6cc1c851ee71ab4fcc58105c7e9931155a483679 Gerrit-Change-Number: 18718 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 13 Jul 2022 19:36:46 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA 11430: Support custom hash schema for Kudu range tables
Hello Mahesh Reddy, Quanlong Huang, Alexey Serbin, Abhishek Chennaka, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18676 to look at the new patch set (#9). Change subject: IMPALA 11430: Support custom hash schema for Kudu range tables .. IMPALA 11430: Support custom hash schema for Kudu range tables KUDU-2671 added support for custom hash partition specification at the range partition level. This patch adds CREATE TABLE and ALTER TABLE syntax to to allow Kudu custom hash schema to be specified through Impala. In addition, a new SHOW HASH SCHEMA statement has been added to allow display of the hash schema information for each partition. Testing: Tests added to kudu_partition_ddl.test Change-Id: I981056e0827f4957580706d6e73742e4e6743c1c --- M common/thrift/CatalogObjects.thrift M common/thrift/Frontend.thrift M fe/pom.xml M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/AlterTableAddDropRangePartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/KuduPartitionParam.java M fe/src/main/java/org/apache/impala/analysis/RangePartition.java M fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M testdata/cluster/node_templates/common/etc/kudu/master.conf.tmpl M testdata/workloads/functional-query/queries/QueryTest/kudu_partition_ddl.test 14 files changed, 284 insertions(+), 26 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/18676/9 -- To view, visit http://gerrit.cloudera.org:8080/18676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I981056e0827f4957580706d6e73742e4e6743c1c Gerrit-Change-Number: 18676 Gerrit-PatchSet: 9 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA 11430: Support custom hash schema for Kudu range tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18676 ) Change subject: IMPALA 11430: Support custom hash schema for Kudu range tables .. Patch Set 9: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/10964/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/18676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I981056e0827f4957580706d6e73742e4e6743c1c Gerrit-Change-Number: 18676 Gerrit-PatchSet: 9 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 13 Jul 2022 23:07:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9482: Support for BINARY columns
Steve Carlin has posted comments on this change. (
http://gerrit.cloudera.org:8080/16066 )
Change subject: IMPALA-9482: Support for BINARY columns
..
Patch Set 14:
(7 comments)
Hopefully these comments are helpful. Logic seems good, these are all pretty
much nits.
Thanks!
http://gerrit.cloudera.org:8080/#/c/16066/14/be/src/runtime/types.cc
File be/src/runtime/types.cc:
http://gerrit.cloudera.org:8080/#/c/16066/14/be/src/runtime/types.cc@141
PS14, Line 141: return TPrimitiveType::BINARY;
Should we return INVALID_TYPE here because of DCHECK(false) so that we catch
the problem earlier on?
http://gerrit.cloudera.org:8080/#/c/16066/14/be/src/service/hs2-util.cc
File be/src/service/hs2-util.cc:
http://gerrit.cloudera.org:8080/#/c/16066/14/be/src/service/hs2-util.cc@872
PS14, Line 872: AuxColumnType aux_type(columnType);
If this is only used in the TYPE_STRING, perhaps it should be initialized there.
http://gerrit.cloudera.org:8080/#/c/16066/14/be/src/service/hs2-util.cc@950
PS14, Line 950: type_entry.__set_type(thrift::TTypeId::STRING_TYPE);
If DCHECK is false, do we want to set this to STRING_TYPE? Or maybe we should
set it to INVALID_TYPE? Not sure what is best.
http://gerrit.cloudera.org:8080/#/c/16066/14/be/src/service/impala-beeswax-server.cc
File be/src/service/impala-beeswax-server.cc:
http://gerrit.cloudera.org:8080/#/c/16066/14/be/src/service/impala-beeswax-server.cc@212
PS14, Line 212: return "string";
How come we don't return TypeToOdbcString here?
(sidenote, I'm rusty on my C++, but why is TypeToOdbcString not inlined?)
http://gerrit.cloudera.org:8080/#/c/16066/14/fe/src/main/java/org/apache/impala/analysis/CastExpr.java
File fe/src/main/java/org/apache/impala/analysis/CastExpr.java:
http://gerrit.cloudera.org:8080/#/c/16066/14/fe/src/main/java/org/apache/impala/analysis/CastExpr.java@187
PS14, Line 187: // No built-in function needed for BINARY <-> STRING
conversion.
I think the comment should also reflect other datatypes versus binary are not
in the support matrix.
http://gerrit.cloudera.org:8080/#/c/16066/14/fe/src/main/java/org/apache/impala/analysis/CastExpr.java@357
PS14, Line 357: boolean isStringBinaryCast =
I may be understanding this wrong...
This is slightly inconsistent with the above code. In the above code, we
assume that if one is binary, the other must be either binary or string. Do we
need to check explicitly here that the other type is string? I understand why,
but then maybe we should have a Preconditions check sayin git is either of type
string or binary?
Not sure what is best here.
http://gerrit.cloudera.org:8080/#/c/16066/14/fe/src/main/java/org/apache/impala/analysis/LikePredicate.java
File fe/src/main/java/org/apache/impala/analysis/LikePredicate.java:
http://gerrit.cloudera.org:8080/#/c/16066/14/fe/src/main/java/org/apache/impala/analysis/LikePredicate.java@116
PS14, Line 116: private static boolean isLikeableType(Type type) {
No real comment here. But I feel the other types might get upset for not being
liked.
--
To view, visit http://gerrit.cloudera.org:8080/16066
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I36861a9ca6c2047b0d76862507c86f7f153bc582
Gerrit-Change-Number: 16066
Gerrit-PatchSet: 14
Gerrit-Owner: Csaba Ringhofer
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Steve Carlin
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Thu, 14 Jul 2022 00:01:00 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11426: Fix quotes not paired in doc 'impala alter table.xml'
yx91...@126.com has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18723 Change subject: IMPALA-11426: Fix quotes not paired in doc 'impala_alter_table.xml' .. IMPALA-11426: Fix quotes not paired in doc 'impala_alter_table.xml' Change-Id: Ic29292cd3c6247fbf7c604d0618e5fbae53b2f01 --- M docs/topics/impala_alter_table.xml 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/23/18723/3 -- To view, visit http://gerrit.cloudera.org:8080/18723 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic29292cd3c6247fbf7c604d0618e5fbae53b2f01 Gerrit-Change-Number: 18723 Gerrit-PatchSet: 3 Gerrit-Owner: Anonymous Coward
[Impala-ASF-CR] IMPALA-11426: Fix quotes not paired in doc 'impala alter table.xml'
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18723 ) Change subject: IMPALA-11426: Fix quotes not paired in doc 'impala_alter_table.xml' .. Patch Set 3: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/662/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/18723 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic29292cd3c6247fbf7c604d0618e5fbae53b2f01 Gerrit-Change-Number: 18723 Gerrit-PatchSet: 3 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Jul 2022 02:53:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11426: Fix quotes not paired in doc 'impala alter table.xml'
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18723 ) Change subject: IMPALA-11426: Fix quotes not paired in doc 'impala_alter_table.xml' .. Patch Set 3: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/662/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/18723 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic29292cd3c6247fbf7c604d0618e5fbae53b2f01 Gerrit-Change-Number: 18723 Gerrit-PatchSet: 3 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Jul 2022 02:59:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables This commit implements cloning between Kudu tables. Cloing Kudu tables from other types of tables is not implemented, because the table c
Xiaoqing Gao has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18729 Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables This commit implements cloning between Kudu tables. Cloing Kudu tables from other types of tables is not implemented, because the table creation statement is not completely consistent. .. IMPALA-4052: CREATE TABLE LIKE for Kudu tables This commit implements cloning between Kudu tables. Cloing Kudu tables from other types of tables is not implemented, because the table creation statement is not completely consistent. Testing: - e2e tests Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 --- M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java A testdata/workloads/functional-query/queries/QueryTest/kudu_create_table_like_table.test M tests/query_test/test_kudu.py 4 files changed, 156 insertions(+), 12 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18729/1 -- To view, visit http://gerrit.cloudera.org:8080/18729 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 Gerrit-Change-Number: 18729 Gerrit-PatchSet: 1 Gerrit-Owner: Xiaoqing Gao
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables This commit implements cloning between Kudu tables. Cloing Kudu tables from other types of tables is not implemented, because the table c
Xiaoqing Gao has abandoned this change. ( http://gerrit.cloudera.org:8080/18729 ) Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables This commit implements cloning between Kudu tables. Cloing Kudu tables from other types of tables is not implemented, because the table creation statement is not completely consistent. .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/18729 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 Gerrit-Change-Number: 18729 Gerrit-PatchSet: 1 Gerrit-Owner: Xiaoqing Gao
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables This commit implements cloning between Kudu tables. Cloing Kudu tables from other types of tables is not implemented, because the table c
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/18729 )
Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables This commit
implements cloning between Kudu tables. Cloing Kudu tables from other types of
tables is not implemented, because the table creation statement is not
completely consistent.
..
Patch Set 1:
(9 comments)
http://gerrit.cloudera.org:8080/#/c/18729/1/fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java
File fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java:
http://gerrit.cloudera.org:8080/#/c/18729/1/fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java@217
PS1, Line 217: private void ensureCreateKuduTableParamsValid(FeTable
srcTable) throws AnalysisException {
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/18729/1/fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java@219
PS1, Line 219: if ((fileFormat_ == THdfsFileFormat.KUDU &&
!KuduTable.isKuduTable(srcTable.getMetaStoreTable())) || (fileFormat_ != null
&& fileFormat_ != THdfsFileFormat.KUDU &&
KuduTable.isKuduTable(srcTable.getMetaStoreTable( {
line too long (223 > 90)
http://gerrit.cloudera.org:8080/#/c/18729/1/fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java@221
PS1, Line 221: throw new AnalysisException(srcTable.getFullName() + "
cannot be cloned because SORT BY is not supported for Kudu tables.");
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/18729/1/fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java@227
PS1, Line 227: throw new AnalysisException(srcTable.getFullName() + "
cannot be cloned because it cannot be cloned into a " +
targeTableFileFormat.toString() + " table.");
line too long (162 > 90)
http://gerrit.cloudera.org:8080/#/c/18729/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/18729/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3776
PS1, Line 3776: TCreateTableParams createTableParams =
extractCreateTableParams(params, tblName, (KuduTable)srcTable, tbl);
line too long (113 > 90)
http://gerrit.cloudera.org:8080/#/c/18729/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3787
PS1, Line 3787: private TCreateTableParams
extractCreateTableParams(TCreateTableLikeParams params, TableName tblName,
KuduTable kuduTable,
line too long (124 > 90)
http://gerrit.cloudera.org:8080/#/c/18729/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3801
PS1, Line 3801: throw new UnsupportedOperationException("CREATE TABLE
LIKE is not supported for Kudu tables hava range partition.");
line too long (124 > 90)
http://gerrit.cloudera.org:8080/#/c/18729/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3809
PS1, Line 3809: tableProperties.put(KuduTable.KEY_TABLE_NAME,
String.format("%s::%s:%s", params.getOwner(), tblName.getDb(),
tblName.getTbl()));
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/18729/1/tests/query_test/test_kudu.py
File tests/query_test/test_kudu.py:
http://gerrit.cloudera.org:8080/#/c/18729/1/tests/query_test/test_kudu.py@92
PS1, Line 92: a
flake8: E501 line too long (96 > 90 characters)
--
To view, visit http://gerrit.cloudera.org:8080/18729
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42
Gerrit-Change-Number: 18729
Gerrit-PatchSet: 1
Gerrit-Owner: Xiaoqing Gao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Thu, 14 Jul 2022 03:36:23 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables This commit implements cloning between Kudu tables. Cloing Kudu tables from other types of tables is not implemented, because the table c
Xiaoqing Gao has restored this change. ( http://gerrit.cloudera.org:8080/18729 ) Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables This commit implements cloning between Kudu tables. Cloing Kudu tables from other types of tables is not implemented, because the table creation statement is not completely consistent. .. Restored -- To view, visit http://gerrit.cloudera.org:8080/18729 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: restore Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 Gerrit-Change-Number: 18729 Gerrit-PatchSet: 1 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18729 to look at the new patch set (#2). Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables .. IMPALA-4052: CREATE TABLE LIKE for Kudu tables This commit implements cloning between Kudu tables. Cloing Kudu tables from other types of tables is not implemented, because the table creation statement is not completely consistent. Testing: - e2e tests Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 --- M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java A testdata/workloads/functional-query/queries/QueryTest/kudu_create_table_like_table.test M tests/query_test/test_kudu.py 4 files changed, 157 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18729/2 -- To view, visit http://gerrit.cloudera.org:8080/18729 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 Gerrit-Change-Number: 18729 Gerrit-PatchSet: 2 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18729 ) Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/18729/2/fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java File fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java: http://gerrit.cloudera.org:8080/#/c/18729/2/fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java@169 PS2, Line 169: line has trailing whitespace http://gerrit.cloudera.org:8080/#/c/18729/2/tests/query_test/test_kudu.py File tests/query_test/test_kudu.py: http://gerrit.cloudera.org:8080/#/c/18729/2/tests/query_test/test_kudu.py@90 PS2, Line 90: a flake8: E501 line too long (96 > 90 characters) -- To view, visit http://gerrit.cloudera.org:8080/18729 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 Gerrit-Change-Number: 18729 Gerrit-PatchSet: 2 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Jul 2022 03:42:56 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18729 ) Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/10965/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/18729 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 Gerrit-Change-Number: 18729 Gerrit-PatchSet: 1 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Jul 2022 03:46:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18729 ) Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10966/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18729 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 Gerrit-Change-Number: 18729 Gerrit-PatchSet: 2 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Jul 2022 04:03:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18729 to look at the new patch set (#3). Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables .. IMPALA-4052: CREATE TABLE LIKE for Kudu tables This commit implements cloning between Kudu tables. Cloing Kudu tables from other types of tables is not implemented, because the table creation statement is not completely consistent. Testing: - e2e tests Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 --- M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java A testdata/workloads/functional-query/queries/QueryTest/kudu_create_table_like_table.test M tests/query_test/test_kudu.py 4 files changed, 159 insertions(+), 12 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18729/3 -- To view, visit http://gerrit.cloudera.org:8080/18729 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 Gerrit-Change-Number: 18729 Gerrit-PatchSet: 3 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18729 ) Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/18729/3/tests/query_test/test_kudu.py File tests/query_test/test_kudu.py: http://gerrit.cloudera.org:8080/#/c/18729/3/tests/query_test/test_kudu.py@91 PS3, Line 91: ' flake8: E126 continuation line over-indented for hanging indent -- To view, visit http://gerrit.cloudera.org:8080/18729 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 Gerrit-Change-Number: 18729 Gerrit-PatchSet: 3 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Jul 2022 04:09:38 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18729 ) Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10967/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18729 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 Gerrit-Change-Number: 18729 Gerrit-PatchSet: 3 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Jul 2022 04:30:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18729 to look at the new patch set (#4). Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables .. IMPALA-4052: CREATE TABLE LIKE for Kudu tables This commit implements cloning between Kudu tables. Cloing Kudu tables from other types of tables is not implemented, because the table creation statement is not completely consistent. Testing: - e2e tests Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 --- M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java A testdata/workloads/functional-query/queries/QueryTest/kudu_create_table_like_table.test M tests/query_test/test_kudu.py 4 files changed, 154 insertions(+), 12 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18729/4 -- To view, visit http://gerrit.cloudera.org:8080/18729 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 Gerrit-Change-Number: 18729 Gerrit-PatchSet: 4 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-4052: CREATE TABLE LIKE for Kudu tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18729 ) Change subject: IMPALA-4052: CREATE TABLE LIKE for Kudu tables .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10968/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18729 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3d276a6465301dbcfed17bb713aca06367d9a42 Gerrit-Change-Number: 18729 Gerrit-PatchSet: 4 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Jul 2022 06:04:28 + Gerrit-HasComments: No
