[Impala-ASF-CR] IMPALA-11834: Fix Iceberg LOAD DATA hdfsDelete JVM crash
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/19410 ) Change subject: IMPALA-11834: Fix Iceberg LOAD DATA hdfsDelete JVM crash .. Patch Set 1: I don't think a test would work with HDFS and for S3 we would likely need 2 buckets. I am working on to test it manually. -- To view, visit http://gerrit.cloudera.org:8080/19410 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifb8f6ebf5b7100e69c1b02137d03fe70c331c30f Gerrit-Change-Number: 19410 Gerrit-PatchSet: 1 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Noemi Pap-Takacs Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 11 Jan 2023 07:53:36 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19383 ) Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu .. Patch Set 9: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/12145/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/19383 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd Gerrit-Change-Number: 19383 Gerrit-PatchSet: 9 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 11 Jan 2023 07:22:26 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu
Wenzhe Zhou has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/19383 ) Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu .. WIP IMPALA-11809: Support non unique primary key for Kudu Kudu engine adds support for non unique primary key. It adds a column 'auto_increment_id' automatically in a table which has non unique primary key. The non unique primary key and 'auto_increment_id' form unique composite primary key. This patch integrated new version of Kudu which support non unique primary key, added syntactic support for creating table with non unique primary key. Example: CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING) PARTITION BY HASH (i) partitions 3 STORED as KUDU; CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i)) PARTITION BY HASH (i) partitions 3 STORED as KUDU; CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id) PARTITION BY HASH (id) partitions 3 STORED as KUDU AS SELECT id, string_col FROM functional.alltypes WHERE id = 10; Kudu assign values for column 'auto_increment_id' automatically when inserting rows so insertion statements don't need to specify values for column 'auto_increment_id'. SELECT statement does not show 'auto_increment_id' column unless the column is explicitly specified in select list. UPSERT operation is not supported now for Kudu table with auto incrementing column due to limitation in Kudu engine. When creating a Kudu table, specifying PRIMARY KEY is optional. If there is no primary key attribute specified, the partition key columes will be promoted as non unique primary key if those columns are the beginning columns of the table. New column "key_unique" is added to the output of 'describe' table command for Kudu table. Testing: - Integrated new version of Kudu built on local machine, ran manual test in impala-shell with queries to create tables with non unique primary key, and tested insert/update/delete operations for Kudu tables with non unique primary key. - Added front end and end to end unit tests. Passed query_test/test_kudu.py and custom_cluster/test_kudu.py on local environment with new version of Kudu built on local machine. - TODO build toolchian with new version of Kudu, including the commits for KUDU-1945. Run core test. Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd --- M common/thrift/CatalogObjects.thrift M common/thrift/JniCatalog.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/AlterTableAddColsStmt.java M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java M fe/src/main/java/org/apache/impala/analysis/TableDef.java M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java M fe/src/main/java/org/apache/impala/catalog/Db.java M fe/src/main/java/org/apache/impala/catalog/FeDb.java M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java M fe/src/main/java/org/apache/impala/catalog/KuduColumn.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java M fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/KuduUtil.java M fe/src/main/jflex/sql-scanner.flex M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test M testdata/workloads/functional-query/queries/QueryTest/kudu_hms_alter.test M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test M testdata/workloads/functional-query/queries/QueryTest/kudu_partition_ddl.test M testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 40 files changed, 1,105 insertions(+), 191 deletions(-)
[Impala-ASF-CR] IMPALA-11662: Improve 'refresh iceberg tbl on oss' performance
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19379 ) Change subject: IMPALA-11662: Improve 'refresh iceberg_tbl_on_oss' performance .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12144/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19379 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2ee8b6b7559e6590698b46ef1d574e55ed52f9a Gerrit-Change-Number: 19379 Gerrit-PatchSet: 8 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 11 Jan 2023 04:50:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11662: Improve 'refresh iceberg tbl on oss' performance
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19379 ) Change subject: IMPALA-11662: Improve 'refresh iceberg_tbl_on_oss' performance .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12143/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19379 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If2ee8b6b7559e6590698b46ef1d574e55ed52f9a Gerrit-Change-Number: 19379 Gerrit-PatchSet: 7 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 11 Jan 2023 04:42:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11662: Improve 'refresh iceberg tbl on oss' performance
lipeng...@apache.org has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/19379 ) Change subject: IMPALA-11662: Improve 'refresh iceberg_tbl_on_oss' performance .. IMPALA-11662: Improve 'refresh iceberg_tbl_on_oss' performance Iceberg provides rich metadata, the cost of directory listing on OSS service e.g. S3A is higher than the cost on HDFS, we could create the file descriptors from Iceberg metadata instead of using org.apache.hadoop.fs.FileSystem#listFiles. The only thing missing there is the last_modification_time of the files. But since Iceberg files are immutable, we could just come up with a special timestamp for these files. At the same time, we can also construct file descriptors ourselves during time travel to reduce the cost of requests with OSS services. Test: * existing tests * test on COS with my local test environment Change-Id: If2ee8b6b7559e6590698b46ef1d574e55ed52f9a --- M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java M fe/src/main/java/org/apache/impala/catalog/FileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java A fe/src/main/java/org/apache/impala/catalog/IcebergFileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java 8 files changed, 334 insertions(+), 100 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/19379/8 -- To view, visit http://gerrit.cloudera.org:8080/19379 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If2ee8b6b7559e6590698b46ef1d574e55ed52f9a Gerrit-Change-Number: 19379 Gerrit-PatchSet: 8 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 ) Change subject: IMPALA-10893: Use old schema during iceberg time travel. .. Patch Set 6: Verify tests fail in test_time_travel. It seems like this test is being run with local catalog mode. I can reproduce by running with local catalog mode. I will investigate further... -- To view, visit http://gerrit.cloudera.org:8080/19380 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab Gerrit-Change-Number: 19380 Gerrit-PatchSet: 6 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 11 Jan 2023 04:04:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 ) Change subject: IMPALA-10893: Use old schema during iceberg time travel. .. Patch Set 6: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8958/ -- To view, visit http://gerrit.cloudera.org:8080/19380 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab Gerrit-Change-Number: 19380 Gerrit-PatchSet: 6 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 10 Jan 2023 23:05:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11375 Impala shell outputs details of each RPC
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19388 ) Change subject: IMPALA-11375 Impala shell outputs details of each RPC .. Patch Set 15: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12142/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19388 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36f8dbc96726aa2a573133acbe8a558299381f8b Gerrit-Change-Number: 19388 Gerrit-PatchSet: 15 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jason Fehr Gerrit-Comment-Date: Tue, 10 Jan 2023 21:56:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11375 Impala shell outputs details of each RPC
Jason Fehr has uploaded a new patch set (#15). ( http://gerrit.cloudera.org:8080/19388 ) Change subject: IMPALA-11375 Impala shell outputs details of each RPC .. IMPALA-11375 Impala shell outputs details of each RPC When the Impala shell is using the hs2 protocol, it makes multiple RPCs to the Impala daemon. These calls pass Thrift objects back and forth. This change adds the '--show_rpc' which outputs the details of the RPCs to stdout and the '--rpc_file' flag which outputs the RPC details to the specified file path. RPC details include: - operation name - request attempt count - Impala session/query ids (if applicable) - call duration - call status (success/failure) - request Thrift objects - response Thrift objects Certain information is not included in the RPC details: - Thrift object attributes named 'secret' or 'password' are redacted. - Thrift objects with a type of TRowSet or TGetRuntimeProfileResp are not include as the information contained within them is already available in the standard output from the Impala shell. Testing: - Added new tests in the end-to-end test suite. Change-Id: I36f8dbc96726aa2a573133acbe8a558299381f8b --- M bin/impala-shell.sh M shell/impala_client.py M shell/impala_shell.py M shell/impala_shell_config_defaults.py M shell/make_shell_tarball.sh M shell/option_parser.py M shell/packaging/make_python_package.sh A shell/thrift_printer.py M tests/shell/test_shell_commandline.py 9 files changed, 565 insertions(+), 147 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/19388/15 -- To view, visit http://gerrit.cloudera.org:8080/19388 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I36f8dbc96726aa2a573133acbe8a558299381f8b Gerrit-Change-Number: 19388 Gerrit-PatchSet: 15 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jason Fehr
[Impala-ASF-CR] IMPALA-11778: Printing maps may produce invalid json
Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/19364 )
Change subject: IMPALA-11778: Printing maps may produce invalid json
..
IMPALA-11778: Printing maps may produce invalid json
Impala allows non-string types, for example numbers, to be keys in maps.
We print maps as json objects, but json objects only allow string keys.
If the Impala map has for example an INT key, the printed json is
invalid.
For example, in Impala the following two maps are not the same:
{1: "a", 2: "b"}
{"1": "a", "2": "b"}
The first map has INT keys, the second has STRING keys. Only the second
one is valid json.
Hive has the same behaviour as Impala, i.e. it produces invalid json if
the map keys have a non-string type.
This change introduces the STRINGIFY_MAP_KEYS query option that, when
set to true, converts non-string keys to strings. The default value of
the new query option is false because
- conversion to string causes loss of information and
- setting it to true would be a breaking change.
Testing:
- Added tests in nested-map-in-select-list.test and map_null_keys.test
that check the behaviour when STRINGIFY_MAP_KEYS is set to true.
Change-Id: I1820036a1c614c34ae5d70ac4fe79a992c9bce3a
Reviewed-on: http://gerrit.cloudera.org:8080/19364
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
---
M be/src/runtime/complex-value-writer.h
M be/src/runtime/complex-value-writer.inline.h
M be/src/service/hs2-util.cc
M be/src/service/hs2-util.h
M be/src/service/impala-beeswax-server.cc
M be/src/service/impala-hs2-server.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/service/query-result-set.cc
M be/src/service/query-result-set.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test
M
testdata/workloads/functional-query/queries/QueryTest/nested-map-in-select-list.test
14 files changed, 209 insertions(+), 98 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/19364
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1820036a1c614c34ae5d70ac4fe79a992c9bce3a
Gerrit-Change-Number: 19364
Gerrit-PatchSet: 5
Gerrit-Owner: Daniel Becker
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-11778: Printing maps may produce invalid json
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19364 ) Change subject: IMPALA-11778: Printing maps may produce invalid json .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/19364 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1820036a1c614c34ae5d70ac4fe79a992c9bce3a Gerrit-Change-Number: 19364 Gerrit-PatchSet: 4 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 10 Jan 2023 20:50:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11661: Added new api in MetastoreServiceHandler for find next compact2 method
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19140 ) Change subject: IMPALA-11661: Added new api in MetastoreServiceHandler for find_next_compact2 method .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12141/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9f1663c16d2649c9c455e6dffde02894819b2761 Gerrit-Change-Number: 19140 Gerrit-PatchSet: 4 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Reviewer: Yu-Wen Lai Gerrit-Comment-Date: Tue, 10 Jan 2023 20:37:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11661: Added new api in MetastoreServiceHandler for find next compact2 method
Hello Quanlong Huang, Daniel Becker, Yu-Wen Lai, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19140 to look at the new patch set (#4). Change subject: IMPALA-11661: Added new api in MetastoreServiceHandler for find_next_compact2 method .. IMPALA-11661: Added new api in MetastoreServiceHandler for find_next_compact2 method In the MetastoreServiceHandler class, the 'find_next_compact2' API implementation is missing which is causing test failures. This needs to be fixed by adding implementation of the 'find_next_compact2' api in the MetastoreServiceHandler class Testing: Added a test 'test_compaction_apis()' to verify that the 'find_next_compact2' api in HMS is reachable from impala. The same test can be used in the future to test newly added apis in HMS. Change-Id: I9f1663c16d2649c9c455e6dffde02894819b2761 --- M fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java M tests/custom_cluster/test_metastore_service.py 2 files changed, 46 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/19140/4 -- To view, visit http://gerrit.cloudera.org:8080/19140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9f1663c16d2649c9c455e6dffde02894819b2761 Gerrit-Change-Number: 19140 Gerrit-PatchSet: 4 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Reviewer: Yu-Wen Lai
[Impala-ASF-CR] IMPALA-11661: Added new api in MetastoreServiceHandler for find next compact2 method
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19140 ) Change subject: IMPALA-11661: Added new api in MetastoreServiceHandler for find_next_compact2 method .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/19140/4/tests/custom_cluster/test_metastore_service.py File tests/custom_cluster/test_metastore_service.py: http://gerrit.cloudera.org:8080/#/c/19140/4/tests/custom_cluster/test_metastore_service.py@1007 PS4, Line 1007: o flake8: F841 local variable 'optionalCi' is assigned to but never used -- To view, visit http://gerrit.cloudera.org:8080/19140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9f1663c16d2649c9c455e6dffde02894819b2761 Gerrit-Change-Number: 19140 Gerrit-PatchSet: 4 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Reviewer: Yu-Wen Lai Gerrit-Comment-Date: Tue, 10 Jan 2023 20:17:09 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11375: Impala shell outputs details of each RPC
Abhishek Rawat has removed a vote on this change. Change subject: IMPALA-11375: Impala shell outputs details of each RPC .. Removed Code-Review+2 by Impala Public Jenkins -- To view, visit http://gerrit.cloudera.org:8080/19388 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I36f8dbc96726aa2a573133acbe8a558299381f8b Gerrit-Change-Number: 19388 Gerrit-PatchSet: 14 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jason Fehr
[Impala-ASF-CR] WIP IMPALA-11809: Support non unique primary key for Kudu
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/19383 ) Change subject: WIP IMPALA-11809: Support non unique primary key for Kudu .. Patch Set 8: (4 comments) http://gerrit.cloudera.org:8080/#/c/19383/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java File fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java: http://gerrit.cloudera.org:8080/#/c/19383/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java@107 PS8, Line 107: A primary key is required for a Kudu table Is this the right Error message for this case? Additionally, if a user wants to create multiple partition levels, the columns on which the partitions are being created have to be specified first, right? In that case does the order matter as long as all the columns are the beginning of the table? http://gerrit.cloudera.org:8080/#/c/19383/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java@715 PS8, Line 715: add Do we need an equivalent test for dropping of the columns? http://gerrit.cloudera.org:8080/#/c/19383/8/fe/src/test/java/org/apache/impala/analysis/ParserTest.java File fe/src/test/java/org/apache/impala/analysis/ParserTest.java: http://gerrit.cloudera.org:8080/#/c/19383/8/fe/src/test/java/org/apache/impala/analysis/ParserTest.java@2798 PS8, Line 2798: i INT PRIMARY KEY, NON UNIQUE PRIMARY KEY(i) This looks a bit confusing for the end user where is declared as both PRIMARY KEY and NON UNIQUE PRIMARY KEY. Do we want to allow this? Additionally what if we do something like below? What would the result be? i INT NON UNIQUE PRIMARY KEY, PRIMARY KEY(i) http://gerrit.cloudera.org:8080/#/c/19383/8/testdata/workloads/functional-query/queries/QueryTest/kudu_create.test File testdata/workloads/functional-query/queries/QueryTest/kudu_create.test: http://gerrit.cloudera.org:8080/#/c/19383/8/testdata/workloads/functional-query/queries/QueryTest/kudu_create.test@535 PS8, Line 535: A primary key is required for a Kudu table As pointed out before, maybe a more helpful message might help here? -- To view, visit http://gerrit.cloudera.org:8080/19383 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd Gerrit-Change-Number: 19383 Gerrit-PatchSet: 8 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 10 Jan 2023 19:25:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11604 Planner changes for CPU usage
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19033 ) Change subject: IMPALA-11604 Planner changes for CPU usage .. Patch Set 30: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12140/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19033 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If32dc770dfffcdd0be2ba789a7720952c68a Gerrit-Change-Number: 19033 Gerrit-PatchSet: 30 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 10 Jan 2023 18:46:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11604 Planner changes for CPU usage
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19033 ) Change subject: IMPALA-11604 Planner changes for CPU usage .. Patch Set 30: ps30 is a rebase of ps29. -- To view, visit http://gerrit.cloudera.org:8080/19033 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If32dc770dfffcdd0be2ba789a7720952c68a Gerrit-Change-Number: 19033 Gerrit-PatchSet: 30 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 10 Jan 2023 18:26:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11604 Planner changes for CPU usage
Riza Suminto has uploaded a new patch set (#30) to the change originally created by Qifan Chen. ( http://gerrit.cloudera.org:8080/19033 ) Change subject: IMPALA-11604 Planner changes for CPU usage .. IMPALA-11604 Planner changes for CPU usage This patch augments IMPALA-10992 by establishing an infrastructure to allow weighted total amount of data to process to be used as a new factor in the definition and selection of an executor group. In this patch the weight component is set to 1. Two flavors of the weighted amount of data processed are enabled in this patch: the minimal amount and the maximal amount. The former is the processing cost along a path with the largest sum of processing costs, reflecting the best level of parallelism among fragments. The latter is the sum of that of every fragment, reflecting the worst level of parallelism. The geometric mean of the minimal and the maximal is used as the estimated processing cost of the query. A fragment's total amount of data processed is the sum of that of every node in the fragment. The weighted amount of data processed is computed with a general formula as follows. Processing cost is a pair: PC(D, N), where D = I * C * W where D is the weighted amount of data processed N is number of instances I is input cardinality C is expression evaluation cost per row, set to 1 W is average row size A description of the computation for each kind of plan node is given below. 1. Aggregation node: C and W are the sum of the costs and partial row widths for each AggregateInfo object. 2. AnalyticEval node: C is sum of the evaluation costs for analytic functions, partition by equal and order by equal predicate; 3. CardinalityCheck node: Both C and I are 1; 4. DataSource scan node: C is computed from a subset of the selection predicates excluding data source accepted predicates; 5. EmptySet node: I is 0; 6. Exchange node: A modification of the general formula when in broadcast mode: D = D * number of receivers; 7. Hash join node: probe side = PC(I0 * C(equi-join predicate) * W, N) + PC(output cardinality * C(other join predicate) * W, N) build side = PC(I1 * C(equi-join predicate) * W, N) 8. Hbase scan node: N is 1 9. Hdfs and Kudu scan node: N is mt_dop when query option mt_dop >= 1, otherwise N is number of nodes * max scan threads; 10. Nested loop join node: When the right child is not a SingularRowSrc node: probe side = PC(I0 * C(equi-join predicate) * W, N) + PC(output cardinality * C(other join predicate) * W, N) build side = PC(I1 * C(equi-join predicate) * W, N) When the right child is a SingularRowSrc node: probe side = PC(I0 * W, N) build side = PC(I0 * I1 * W, N) 11. Select node: Use the general formula; 12. SingularRowSrc node: Since the node is involved once per input in nested loop join, the contribution of this node is computed in nested loop join; 13. Sort node: C is the evalation cost for the sort expression and W is the width of the intermediate tuple being sorted; 14. Subplan node: C is 1. I is the multiplication of the cardinality of the left and the right child; 15. Union node: C is the cost of materializing rows from all non pass-through children. W is the width of all non pass-through children; 16. Unnest node: I is the cardinality of the containing subplan node and C is 1. The processing cost for the data sink of a fragment is also computed and added to that of the fragment. This patch also assumes that the number of instances of execution overlaps each other when there is a discrepancy of instances among nodes in a single fragment. For example when there are 6 scan thread and 3 aggregation threads in a single fragment, 3 threads are used for both scan and aggregate, and 3 other threads are used for the scan. As an example, the best and worst processing cost for TPCDS large query q14a and tiny q19 are as follows. Processing cost for query: q14a: Best case: total=1271804127, numInstances=9, perInstance=141311569 Worst case: total=5169193752, numInstances=12, perInstance=430766146 q19: Best case: total=1082950, numInstances=15, perInstance=72196 Worst case: total=1082950, numInstances=15, perInstance=72196 Testing: 1. Unit testing by examining the best and the worst processing cost computed for all plan nodes in all fragments for a small set of queries; 2. Core tests. Change-Id: If32dc770dfffcdd0be2ba789a7720952c68a --- M common/thrift/Frontend.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/SortInfo.java M fe/src/main/java/org/apache/impala/planner/AggregationNode.java M
[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 ) Change subject: IMPALA-10893: Use old schema during iceberg time travel. .. Patch Set 6: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/12139/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/19380 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab Gerrit-Change-Number: 19380 Gerrit-PatchSet: 6 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 10 Jan 2023 18:09:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 ) Change subject: IMPALA-10893: Use old schema during iceberg time travel. .. Patch Set 6: Code-Review+2 Carry forward +2 -- To view, visit http://gerrit.cloudera.org:8080/19380 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab Gerrit-Change-Number: 19380 Gerrit-PatchSet: 6 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 10 Jan 2023 17:49:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.
Hello Tamas Mate, Gabor Kaszab, Zoltan Borok-Nagy, Gergely Fürnstáhl, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19380 to look at the new patch set (#6). Change subject: IMPALA-10893: Use old schema during iceberg time travel. .. IMPALA-10893: Use old schema during iceberg time travel. Before this change the schema used during Iceberg Time Travel was the current schema of the table. With this change we will use the schema from the point specified by the Time Travel parameters. The parameters used by an Iceberg Time Travel query are part of the FROM clause of the query. Previously analysis of the Time Travel parameters took place after the table Path was resolved, at which point some schema information is cached. In order to use the old schema during iceberg time travel however we need to ensure that the version of the Table that is used is always the version specified by the Time Travel parameters. To do this we have to move the analysis of the Time Travel parameters inside the code that resolves the Path. Add a new implementation of FeIcebergTable that represents an Iceberg table involved in Time Travel. This is implemented by embedding a reference to the base Iceberg Table. All methods that are not Time Travel related are delegated to the base table. The Time Travel related methods use the historic Iceberg schema. TESTING: - Add a new file iceberg_util.py to hold the snapshot utility code that was developed for the in-progress IMPALA-11482. - Extend the existing Iceberg Time Travel tests to check the schema. - Add a test that shows time travel working with columns masking. The column masking configuration is not tightly coupled to the schema so it is possible to mask historical columns. Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java M fe/src/main/java/org/apache/impala/analysis/DescriptorTable.java M fe/src/main/java/org/apache/impala/analysis/TableRef.java M fe/src/main/java/org/apache/impala/analysis/TimeTravelSpec.java M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java M fe/src/main/java/org/apache/impala/catalog/IcebergPositionDeleteTable.java M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java A fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M tests/authorization/test_ranger.py M tests/common/iceberg_test_suite.py M tests/query_test/test_iceberg.py A tests/util/iceberg_util.py 16 files changed, 1,036 insertions(+), 98 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/19380/6 -- To view, visit http://gerrit.cloudera.org:8080/19380 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab Gerrit-Change-Number: 19380 Gerrit-PatchSet: 6 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 ) Change subject: IMPALA-10893: Use old schema during iceberg time travel. .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8958/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/19380 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab Gerrit-Change-Number: 19380 Gerrit-PatchSet: 6 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 10 Jan 2023 17:53:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11013 (part 1): Support 'MIGRATE TABLE' for external Hdfs tables
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19397 ) Change subject: IMPALA-11013 (part 1): Support 'MIGRATE TABLE' for external Hdfs tables .. Patch Set 4: (3 comments) Went through the code quickly. My main concern is that it always migrates to a 'hadoop.tables' catalog and would be good if we could be more flexible about this. And if we can, HiveCatalog would be a better default. http://gerrit.cloudera.org:8080/#/c/19397/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19397/4//COMMIT_MSG@23 PS4, Line 23: not temporary tables What do you mean by 'temporary table'? In Impala we don't support such tables. http://gerrit.cloudera.org:8080/#/c/19397/4//COMMIT_MSG@30 PS4, Line 30: querie nit: query http://gerrit.cloudera.org:8080/#/c/19397/4/fe/src/main/java/org/apache/impala/util/MigrateTableUtil.java File fe/src/main/java/org/apache/impala/util/MigrateTableUtil.java: http://gerrit.cloudera.org:8080/#/c/19397/4/fe/src/main/java/org/apache/impala/util/MigrateTableUtil.java@71 PS4, Line 71: IcebergHadoopTables.createTable Would it be possible to use Iceberg's Catalogs API? https://github.com/apache/iceberg/blob/ce6c95025d33aac663ecdfb99ff92f2d71731263/mr/src/main/java/org/apache/iceberg/mr/Catalogs.java#L151 This way the user could specify any Iceberg catalog. And our default catalog could be Hive Catalog, which is the Catalog being used for plain CREATE TABLE STORED AS ICEBERG statements already. -- To view, visit http://gerrit.cloudera.org:8080/19397 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I91e6a9cfe099c263f17b5506d6db459b79ad31a5 Gerrit-Change-Number: 19397 Gerrit-PatchSet: 4 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 10 Jan 2023 17:06:55 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11778: Printing maps may produce invalid json
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19364 ) Change subject: IMPALA-11778: Printing maps may produce invalid json .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12138/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19364 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1820036a1c614c34ae5d70ac4fe79a992c9bce3a Gerrit-Change-Number: 19364 Gerrit-PatchSet: 3 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 10 Jan 2023 16:00:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11778: Printing maps may produce invalid json
Daniel Becker has uploaded a new patch set (#3). (
http://gerrit.cloudera.org:8080/19364 )
Change subject: IMPALA-11778: Printing maps may produce invalid json
..
IMPALA-11778: Printing maps may produce invalid json
Impala allows non-string types, for example numbers, to be keys in maps.
We print maps as json objects, but json objects only allow string keys.
If the Impala map has for example an INT key, the printed json is
invalid.
For example, in Impala the following two maps are not the same:
{1: "a", 2: "b"}
{"1": "a", "2": "b"}
The first map has INT keys, the second has STRING keys. Only the second
one is valid json.
Hive has the same behaviour as Impala, i.e. it produces invalid json if
the map keys have a non-string type.
This change introduces the STRINGIFY_MAP_KEYS query option that, when
set to true, converts non-string keys to strings. The default value of
the new query option is false because
- conversion to string causes loss of information and
- setting it to true would be a breaking change.
Testing:
- Added tests in nested-map-in-select-list.test and map_null_keys.test
that check the behaviour when STRINGIFY_MAP_KEYS is set to true.
Change-Id: I1820036a1c614c34ae5d70ac4fe79a992c9bce3a
---
M be/src/runtime/complex-value-writer.h
M be/src/runtime/complex-value-writer.inline.h
M be/src/service/hs2-util.cc
M be/src/service/hs2-util.h
M be/src/service/impala-beeswax-server.cc
M be/src/service/impala-hs2-server.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/service/query-result-set.cc
M be/src/service/query-result-set.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test
M
testdata/workloads/functional-query/queries/QueryTest/nested-map-in-select-list.test
14 files changed, 209 insertions(+), 98 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/64/19364/3
--
To view, visit http://gerrit.cloudera.org:8080/19364
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1820036a1c614c34ae5d70ac4fe79a992c9bce3a
Gerrit-Change-Number: 19364
Gerrit-PatchSet: 3
Gerrit-Owner: Daniel Becker
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-11778: Printing maps may produce invalid json
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19364 ) Change subject: IMPALA-11778: Printing maps may produce invalid json .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/19364 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1820036a1c614c34ae5d70ac4fe79a992c9bce3a Gerrit-Change-Number: 19364 Gerrit-PatchSet: 4 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 10 Jan 2023 15:40:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11778: Printing maps may produce invalid json
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19364 ) Change subject: IMPALA-11778: Printing maps may produce invalid json .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8957/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/19364 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1820036a1c614c34ae5d70ac4fe79a992c9bce3a Gerrit-Change-Number: 19364 Gerrit-PatchSet: 4 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 10 Jan 2023 15:40:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11778: Printing maps may produce invalid json
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19364 ) Change subject: IMPALA-11778: Printing maps may produce invalid json .. Patch Set 2: Code-Review+2 (2 comments) http://gerrit.cloudera.org:8080/#/c/19364/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19364/2//COMMIT_MSG@20 PS2, Line 20: : Hive has the same behaviour. Can you make it clearer that Hive also produces invalid JSON? http://gerrit.cloudera.org:8080/#/c/19364/2/be/src/service/query-options.h File be/src/service/query-options.h: http://gerrit.cloudera.org:8080/#/c/19364/2/be/src/service/query-options.h@284 PS2, Line 284: REGULAR I would consider this advanced (should affect very few users) -- To view, visit http://gerrit.cloudera.org:8080/19364 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1820036a1c614c34ae5d70ac4fe79a992c9bce3a Gerrit-Change-Number: 19364 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 10 Jan 2023 15:03:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 ) Change subject: IMPALA-10893: Use old schema during iceberg time travel. .. Patch Set 5: Code-Review+2 LGTM and nice tests! -- To view, visit http://gerrit.cloudera.org:8080/19380 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab Gerrit-Change-Number: 19380 Gerrit-PatchSet: 5 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 10 Jan 2023 14:06:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11834: Fix Iceberg LOAD DATA hdfsDelete JVM crash
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19410 ) Change subject: IMPALA-11834: Fix Iceberg LOAD DATA hdfsDelete JVM crash .. Patch Set 1: The change LGTM. Is it possible to add tests for this fix? -- To view, visit http://gerrit.cloudera.org:8080/19410 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifb8f6ebf5b7100e69c1b02137d03fe70c331c30f Gerrit-Change-Number: 19410 Gerrit-PatchSet: 1 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Noemi Pap-Takacs Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 10 Jan 2023 13:04:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11834: Fix Iceberg LOAD DATA hdfsDelete JVM crash
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19410 ) Change subject: IMPALA-11834: Fix Iceberg LOAD DATA hdfsDelete JVM crash .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12137/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19410 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifb8f6ebf5b7100e69c1b02137d03fe70c331c30f Gerrit-Change-Number: 19410 Gerrit-PatchSet: 1 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Noemi Pap-Takacs Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 10 Jan 2023 12:13:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11834: Fix Iceberg LOAD DATA hdfsDelete JVM crash
Tamas Mate has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19410 Change subject: IMPALA-11834: Fix Iceberg LOAD DATA hdfsDelete JVM crash .. IMPALA-11834: Fix Iceberg LOAD DATA hdfsDelete JVM crash The LOAD DATA statement could crash the JVM when there were differences between the 'fs.defaultFS' and loaded paths. This happened because the hdfsFS object was initialized with the default FS instead of the correct ones for the paths. This commit fixes the hdfsFS object initialization. Testing: - Ran the existing tests locally. Change-Id: Ifb8f6ebf5b7100e69c1b02137d03fe70c331c30f --- M be/src/service/client-request-state.cc M be/src/service/client-request-state.h 2 files changed, 15 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/10/19410/1 -- To view, visit http://gerrit.cloudera.org:8080/19410 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ifb8f6ebf5b7100e69c1b02137d03fe70c331c30f Gerrit-Change-Number: 19410 Gerrit-PatchSet: 1 Gerrit-Owner: Tamas Mate
[Impala-ASF-CR] WIP IMPALA-11833: Fixed manifest length in snapshot files
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19409 ) Change subject: WIP IMPALA-11833: Fixed manifest_length in snapshot files .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12136/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19409 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I258055998a1d41b7f6047b6879e919834ed2c247 Gerrit-Change-Number: 19409 Gerrit-PatchSet: 1 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 10 Jan 2023 11:12:28 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-11833: Fixed manifest length in snapshot files
Gergely Fürnstáhl has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19409 Change subject: WIP IMPALA-11833: Fixed manifest_length in snapshot files .. WIP IMPALA-11833: Fixed manifest_length in snapshot files Some manifest files were manually edited to support multiple environments, namely hdfs://localhost:20500 were cropped from every path. The snapshot files contain the length of these manifest files, which was not adjusted. Testing: - TBD Change-Id: I258055998a1d41b7f6047b6879e919834ed2c247 --- M testdata/data/iceberg_test/hadoop_catalog/ice/airports_parquet/metadata/snap-2304960110511088609-1-2d65964e-90ea-4442-bab5-71a67b84dfd9.avro M testdata/data/iceberg_test/hadoop_catalog/ice/complextypestbl_iceberg_orc/metadata/snap-8747481058330439933-1-46b4a907-2ff3-4799-ba4a-074d04734265.avro M testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_alltypes_part/metadata/snap-6167994413873848621-1-283c54cb-5a45-4a2c-bca8-4bfa0e61cdbd.avro M testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes/metadata/snap-728158873687794725-1-5c80922f-01b5-4d52-bc93-6505be3b977b.avro M testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes_orc/metadata/snap-1041485290740594175-1-a72290c9-c518-4719-8502-6c83a881de07.avro 5 files changed, 0 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/09/19409/1 -- To view, visit http://gerrit.cloudera.org:8080/19409 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I258055998a1d41b7f6047b6879e919834ed2c247 Gerrit-Change-Number: 19409 Gerrit-PatchSet: 1 Gerrit-Owner: Gergely Fürnstáhl
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12135/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 10 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 10 Jan 2023 10:28:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. There is a limitation: zipping unnest for arrays that are within a struct is not supported, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 23 files changed, 917 insertions(+), 247 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/10 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 10 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] WIP IMPALA-5323: Support Kudu BINARY
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18868 ) Change subject: WIP IMPALA-5323: Support Kudu BINARY .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12134/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18868 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iff701a4b3a09ce7b6982c5d238e65f3d4f3d1151 Gerrit-Change-Number: 18868 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 10 Jan 2023 10:00:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11661: Added new api in MetastoreServiceHandler for find next compact2 method
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19140 ) Change subject: IMPALA-11661: Added new api in MetastoreServiceHandler for find_next_compact2 method .. Patch Set 3: Code-Review+1 (2 comments) Some nits, otherwise LGTM. http://gerrit.cloudera.org:8080/#/c/19140/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19140/3//COMMIT_MSG@12 PS3, Line 12: need Nit: needs http://gerrit.cloudera.org:8080/#/c/19140/3/tests/custom_cluster/test_metastore_service.py File tests/custom_cluster/test_metastore_service.py: http://gerrit.cloudera.org:8080/#/c/19140/3/tests/custom_cluster/test_metastore_service.py@989 PS3, Line 989: talk Nit: talk to -- To view, visit http://gerrit.cloudera.org:8080/19140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9f1663c16d2649c9c455e6dffde02894819b2761 Gerrit-Change-Number: 19140 Gerrit-PatchSet: 3 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Reviewer: Yu-Wen Lai Gerrit-Comment-Date: Tue, 10 Jan 2023 09:40:39 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7969: Always admit trivial queries immediately
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19214 ) Change subject: IMPALA-7969: Always admit trivial queries immediately .. Patch Set 5: Code-Review+1 Thanks for the changes! -- To view, visit http://gerrit.cloudera.org:8080/19214 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2a729764e3055d7eb11900c96c82ff53eb261f91 Gerrit-Change-Number: 19214 Gerrit-PatchSet: 5 Gerrit-Owner: Yida Wu Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Tue, 10 Jan 2023 09:31:29 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-5323: Support Kudu BINARY
Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18868 Change subject: WIP IMPALA-5323: Support Kudu BINARY .. WIP IMPALA-5323: Support Kudu BINARY Also changed ColumnType to include the info whether it is binary or string. The current implementation can crash during codegen. Change-Id: Iff701a4b3a09ce7b6982c5d238e65f3d4f3d1151 --- M be/src/exec/file-metadata-utils.cc M be/src/exec/hbase/hbase-scan-node.cc M be/src/exec/hbase/hbase-table-writer.cc M be/src/exec/hdfs-scanner-ir.cc M be/src/exec/hdfs-scanner.cc M be/src/exec/hdfs-text-table-writer.cc M be/src/exec/kudu/kudu-table-sink.cc M be/src/exec/kudu/kudu-util-ir.cc M be/src/exec/kudu/kudu-util.cc M be/src/exec/parquet/hdfs-parquet-table-writer.cc M be/src/exec/parquet/parquet-metadata-utils.cc M be/src/exec/parquet/parquet-metadata-utils.h M be/src/exec/rcfile/hdfs-rcfile-scanner.cc M be/src/exec/text-converter.cc M be/src/exec/text-converter.h M be/src/exec/text-converter.inline.h M be/src/exec/text/hdfs-text-scanner.cc M be/src/exprs/kudu-partition-expr.cc M be/src/runtime/descriptors.cc M be/src/runtime/descriptors.h M be/src/runtime/types.cc M be/src/runtime/types.h M be/src/service/hs2-util.cc M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M fe/src/main/java/org/apache/impala/util/KuduUtil.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/binary-type.test M tests/query_test/test_scanners.py 29 files changed, 107 insertions(+), 99 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/18868/2 -- To view, visit http://gerrit.cloudera.org:8080/18868 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iff701a4b3a09ce7b6982c5d238e65f3d4f3d1151 Gerrit-Change-Number: 18868 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.
Gergely Fürnstáhl has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 ) Change subject: IMPALA-10893: Use old schema during iceberg time travel. .. Patch Set 5: Code-Review+1 Thanks for the work, LGTM! -- To view, visit http://gerrit.cloudera.org:8080/19380 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab Gerrit-Change-Number: 19380 Gerrit-PatchSet: 5 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 10 Jan 2023 08:12:18 + Gerrit-HasComments: No
