[Impala-ASF-CR] IMPALA-9448: (Addendum) Avoid generic recreate for Ozone bucket

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19082 )

Change subject: IMPALA-9448: (Addendum) Avoid generic recreate for Ozone bucket
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11494/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19082
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I31a3813fac6f2d68859ba7fed53b5c5a82502780
Gerrit-Change-Number: 19082
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 30 Sep 2022 05:47:48 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9448: (Addendum) Avoid generic recreate for Ozone bucket

2022-09-29 Thread Michael Smith (Code Review)
Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19082

to look at the new patch set (#2).

Change subject: IMPALA-9448: (Addendum) Avoid generic recreate for Ozone bucket
..

IMPALA-9448: (Addendum) Avoid generic recreate for Ozone bucket

Avoids using `hadoop fs` to recreate an Ozone bucket during snapshot
load, as this removes the encryption key.

Change-Id: I31a3813fac6f2d68859ba7fed53b5c5a82502780
---
M testdata/bin/load-test-warehouse-snapshot.sh
1 file changed, 7 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/19082/2
--
To view, visit http://gerrit.cloudera.org:8080/19082
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I31a3813fac6f2d68859ba7fed53b5c5a82502780
Gerrit-Change-Number: 19082
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-3119: DDL support for bucketed tables

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19055 )

Change subject: IMPALA-3119: DDL support for bucketed tables
..


Patch Set 5:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11493/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19055
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I919b4d4139bc3a7784fa6fdb6f064e25666d548e
Gerrit-Change-Number: 19055
Gerrit-PatchSet: 5
Gerrit-Owner: Baike Xia 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 30 Sep 2022 04:35:53 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-3119: DDL support for bucketed tables

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19055 )

Change subject: IMPALA-3119: DDL support for bucketed tables
..


Patch Set 5:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/19055/5/common/thrift/CatalogObjects.thrift
File common/thrift/CatalogObjects.thrift:

http://gerrit.cloudera.org:8080/#/c/19055/5/common/thrift/CatalogObjects.thrift@196
PS5, Line 196: // When create bucketd table, need to know about bucket's type, 
bucket's columns and number.
line too long (92 > 90)


http://gerrit.cloudera.org:8080/#/c/19055/5/fe/src/main/java/org/apache/impala/analysis/TableDef.java
File fe/src/main/java/org/apache/impala/analysis/TableDef.java:

http://gerrit.cloudera.org:8080/#/c/19055/5/fe/src/main/java/org/apache/impala/analysis/TableDef.java@792
PS5, Line 792: if (bucketDesc.getBucket_columns() == null || 
bucketDesc.getBucket_columns().size() == 0) {
line too long (95 > 90)


http://gerrit.cloudera.org:8080/#/c/19055/5/fe/src/main/java/org/apache/impala/util/BucketUtils.java
File fe/src/main/java/org/apache/impala/util/BucketUtils.java:

http://gerrit.cloudera.org:8080/#/c/19055/5/fe/src/main/java/org/apache/impala/util/BucketUtils.java@40
PS5, Line 40: TBucketType bucketType = 
TBucketType.valueOf(params.get(Table.HIVE_IMPALA_BUCKET_TYPE));
line too long (92 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/19055
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I919b4d4139bc3a7784fa6fdb6f064e25666d548e
Gerrit-Change-Number: 19055
Gerrit-PatchSet: 5
Gerrit-Owner: Baike Xia 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 30 Sep 2022 04:29:10 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-3119: DDL support for bucketed tables

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19055 )

Change subject: IMPALA-3119: DDL support for bucketed tables
..


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8643/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/19055
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I919b4d4139bc3a7784fa6fdb6f064e25666d548e
Gerrit-Change-Number: 19055
Gerrit-PatchSet: 5
Gerrit-Owner: Baike Xia 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 30 Sep 2022 04:23:13 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11606: add 'untracked memory' metric.

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19029 )

Change subject: IMPALA-11606: add 'untracked memory' metric.
..


Patch Set 3:

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8641/


--
To view, visit http://gerrit.cloudera.org:8080/19029
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib16e00109d732f759c96c7a967eb1cc32124a03f
Gerrit-Change-Number: 19029
Gerrit-PatchSet: 3
Gerrit-Owner: Xiang Yang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Xianqing He 
Gerrit-Comment-Date: Fri, 30 Sep 2022 04:23:11 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-3119: DDL support for bucketed tables

2022-09-29 Thread Baike Xia (Code Review)
Baike Xia has uploaded a new patch set (#5). ( 
http://gerrit.cloudera.org:8080/19055 )

Change subject: IMPALA-3119: DDL support for bucketed tables
..

IMPALA-3119: DDL support for bucketed tables

Add syntactic support for creating bucketed table.
The specific syntax is as follows:
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.]table_name(
   col_name data_type
   [constraint_specification]
   [COMMENT 'col_comment']
   [, ...]
 )
 [PARTITIONED BY (col_name data_type [COMMENT 'col_comment'], ...)]
 [BUCKETED BY HASH([column [, column ...]])|RANDOM INTO 24 BUCKETS
 [SORT BY ([column [, column ...]])]
 [COMMENT 'table_comment']
 [ROW FORMAT row_format]
 [WITH SERDEPROPERTIES ('key1'='value1', 'key2'='value2', ...)]
 [STORED AS file_format]
 [LOCATION 'hdfs_path']
 [CACHED IN 'pool_name' [WITH REPLICATION = integer] | UNCACHED]
 [TBLPROPERTIES ('key1'='value1', 'key2'='value2', ...)]

Instructions:
1. CLUSTERED BY of Hive is not supported, because HINT has the keyword;
2. The bucket partitioning algorithm contains HASH, RANDOM, KUDU_HASH.
  The default value is HASH;
3. INTO 24 BUCKETS, specifies the number of buckets, the default value
  is 16;
4. Create Bucketed Table statements that do not support Kudu and
  Iceberg tables, but for a Kudu table,
  the hash partition is equivalent to a bucket,
  and the optimization rule applies to join query;
5. In the current version, alter operations(add/drop/change/replace
 columns) on bucketed tables are not supported;

This COMMIT is the first subtask of IMPALA-3118.

Change-Id: I919b4d4139bc3a7784fa6fdb6f064e25666d548e
---
M common/thrift/CatalogObjects.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/TableDef.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
A fe/src/main/java/org/apache/impala/util/BucketUtils.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M testdata/workloads/functional-query/queries/QueryTest/create-table.test
15 files changed, 408 insertions(+), 21 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/55/19055/5
--
To view, visit http://gerrit.cloudera.org:8080/19055
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I919b4d4139bc3a7784fa6fdb6f064e25666d548e
Gerrit-Change-Number: 19055
Gerrit-PatchSet: 5
Gerrit-Owner: Baike Xia 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-11555: external Udf support decimal(*, *)

2022-09-29 Thread Minghui Zhu (Code Review)
Minghui Zhu has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19021 )

Change subject: IMPALA-11555: external Udf support decimal(*, *)
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19021/3//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19021/3//COMMIT_MSG@9
PS3, Line 9: Increase syntax parsing and remove some restrictions.
> To help others to understand the sense of "decimal(*, *)", could you please
I have written in https://issues.apache.org/jira/browse/IMPALA-11555



--
To view, visit http://gerrit.cloudera.org:8080/19021
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09f19da5b1660ed0b882497fc321e6cc1b9b55d0
Gerrit-Change-Number: 19021
Gerrit-PatchSet: 3
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Anonymous Coward 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Minghui Zhu 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Fri, 30 Sep 2022 04:04:20 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11606: add 'untracked memory' metric.

2022-09-29 Thread Xianqing He (Code Review)
Xianqing He has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19029 )

Change subject: IMPALA-11606: add 'untracked memory' metric.
..


Patch Set 3: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/19029
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib16e00109d732f759c96c7a967eb1cc32124a03f
Gerrit-Change-Number: 19029
Gerrit-PatchSet: 3
Gerrit-Owner: Xiang Yang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Xianqing He 
Gerrit-Comment-Date: Fri, 30 Sep 2022 04:03:16 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11558: Ensure one Kudu client created (FE) for the specified Kudu master addresses

2022-09-29 Thread Xiang Yang (Code Review)
Xiang Yang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19046 )

Change subject: IMPALA-11558: Ensure one Kudu client created (FE) for the 
specified Kudu master addresses
..


Patch Set 4: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/19046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1003556d3afc8e8216142cac4007a4c99046caeb
Gerrit-Change-Number: 19046
Gerrit-PatchSet: 4
Gerrit-Owner: Xianqing He 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Reviewer: Xiang Yang 
Gerrit-Reviewer: Xianqing He 
Gerrit-Comment-Date: Fri, 30 Sep 2022 03:51:19 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11235: Support Pluggable Authentication for Impala

2022-09-29 Thread Xiang Yang (Code Review)
Xiang Yang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19042 )

Change subject: IMPALA-11235: Support Pluggable Authentication for Impala
..


Patch Set 3:

(9 comments)

Thanks for your contribution, minghui!

http://gerrit.cloudera.org:8080/#/c/19042/3//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19042/3//COMMIT_MSG@1
PS3, Line 1: Parent: 296e9441 (IMPALA-11599/IMPALA-11605: GCC 10: Fix gdb 
and change to flatbuffers 1.9)
can we give this authentication implementation a more explict name instead of 
the 'plugin'? such as commandline authentication .etc.
can you add docs like 
'https://impala.apache.org/docs/build/html/topics/impala_ldap.html'?


http://gerrit.cloudera.org:8080/#/c/19042/3/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:

http://gerrit.cloudera.org:8080/#/c/19042/3/be/src/rpc/authentication.cc@309
PS3, Line 309: LOG(ERROR) << "Plugin authentication failed for empty 
password。";
nit: don't use chinese punctuation.


http://gerrit.cloudera.org:8080/#/c/19042/3/bin/plugin_for_test.sh
File bin/plugin_for_test.sh:

http://gerrit.cloudera.org:8080/#/c/19042/3/bin/plugin_for_test.sh@1
PS3, Line 1: #!/usr/bin/env bash
lack of apache License header.


http://gerrit.cloudera.org:8080/#/c/19042/3/bin/plugin_for_test.sh@18
PS3, Line 18: exit 1
use a different exit code to distinguish from line 4.


http://gerrit.cloudera.org:8080/#/c/19042/3/fe/src/test/java/org/apache/impala/customcluster/PluginHS2Test.java
File fe/src/test/java/org/apache/impala/customcluster/PluginHS2Test.java:

http://gerrit.cloudera.org:8080/#/c/19042/3/fe/src/test/java/org/apache/impala/customcluster/PluginHS2Test.java@1
PS3, Line 1: // or more contributor license agreements.  See the NOTICE file
Incomplete lincense header.


http://gerrit.cloudera.org:8080/#/c/19042/3/fe/src/test/java/org/apache/impala/customcluster/PluginHS2Test.java@53
PS3, Line 53:  * plugin authentication is being used.
nit: is used.


http://gerrit.cloudera.org:8080/#/c/19042/3/fe/src/test/java/org/apache/impala/customcluster/PluginHS2Test.java@67
PS3, Line 67:   static void verifySuccess(TStatus status) throws Exception {
can use org.apache.hive.jdbc.Utils.verifySuccess() instead.


http://gerrit.cloudera.org:8080/#/c/19042/3/fe/src/test/java/org/apache/impala/customcluster/PluginHS2Test.java@155
PS3, Line 155: transport.open();
do we need close THttpClient at the end of the test?


http://gerrit.cloudera.org:8080/#/c/19042/3/fe/src/test/java/org/apache/impala/customcluster/PluginImpylaHttpTest.java
File fe/src/test/java/org/apache/impala/customcluster/PluginImpylaHttpTest.java:

http://gerrit.cloudera.org:8080/#/c/19042/3/fe/src/test/java/org/apache/impala/customcluster/PluginImpylaHttpTest.java@38
PS3, Line 38:  * Impyla HTTP connectivity tests with LDAP authentication.
What's the difference with LdapImpylaHttpTest?  I see many Duplicate codes, can 
we refactor it to  base class and inherited class mode?



--
To view, visit http://gerrit.cloudera.org:8080/19042
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieaaef6bffe641ece30e7fc3756688ef30722dfe1
Gerrit-Change-Number: 19042
Gerrit-PatchSet: 3
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Minghui Zhu 
Gerrit-Reviewer: Xiang Yang 
Gerrit-Comment-Date: Fri, 30 Sep 2022 03:47:56 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-3119: DDL support for bucketed tables

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19055 )

Change subject: IMPALA-3119: DDL support for bucketed tables
..


Patch Set 4:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/11492/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/19055
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I919b4d4139bc3a7784fa6fdb6f064e25666d548e
Gerrit-Change-Number: 19055
Gerrit-PatchSet: 4
Gerrit-Owner: Baike Xia 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 30 Sep 2022 03:30:30 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-3119: DDL support for bucketed tables

2022-09-29 Thread Baike Xia (Code Review)
Baike Xia has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/19055


Change subject: IMPALA-3119: DDL support for bucketed tables
..

IMPALA-3119: DDL support for bucketed tables

Add syntactic support for creating bucketed table.
The specific syntax is as follows:
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.]table_name(
   col_name data_type
   [constraint_specification]
   [COMMENT 'col_comment']
   [, ...]
 )
 [PARTITIONED BY (col_name data_type [COMMENT 'col_comment'], ...)]
 [BUCKETED BY HASH([column [, column ...]])|RANDOM INTO 24 BUCKETS
 [SORT BY ([column [, column ...]])]
 [COMMENT 'table_comment']
 [ROW FORMAT row_format]
 [WITH SERDEPROPERTIES ('key1'='value1', 'key2'='value2', ...)]
 [STORED AS file_format]
 [LOCATION 'hdfs_path']
 [CACHED IN 'pool_name' [WITH REPLICATION = integer] | UNCACHED]
 [TBLPROPERTIES ('key1'='value1', 'key2'='value2', ...)]

Instructions:
1. CLUSTERED BY of Hive is not supported, because HINT has the keyword;
2. The bucket partitioning algorithm contains HASH, RANDOM, KUDU_HASH.
  The default value is HASH;
3. INTO 24 BUCKETS, specifies the number of buckets, the default value
  is 16;
4. Create Bucketed Table statements that do not support Kudu and
  Iceberg tables, but for a Kudu table,
  the hash partition is equivalent to a bucket,
  and the optimization rule applies to join query;
5. In the current version, alter operations(add/drop/change/replace
 columns) on bucketed tables are not supported;

This COMMIT is the first subtask of IMPALA-3118.

Change-Id: I919b4d4139bc3a7784fa6fdb6f064e25666d548e
---
M common/thrift/CatalogObjects.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/TableDef.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
A fe/src/main/java/org/apache/impala/util/BucketUtils.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M testdata/workloads/functional-query/queries/QueryTest/create-table.test
16 files changed, 411 insertions(+), 21 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/55/19055/4
--
To view, visit http://gerrit.cloudera.org:8080/19055
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I919b4d4139bc3a7784fa6fdb6f064e25666d548e
Gerrit-Change-Number: 19055
Gerrit-PatchSet: 4
Gerrit-Owner: Baike Xia 


[Impala-ASF-CR] IMPALA-11566: Provide SQL parsing capabilities to other applications in the form of jar packages

2022-09-29 Thread Xiang Yang (Code Review)
Xiang Yang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19023 )

Change subject: IMPALA-11566: Provide SQL parsing capabilities to other 
applications in the form of jar packages
..


Patch Set 2:

(4 comments)

Hi minghui, thanks for your contribution! I post some comments.

http://gerrit.cloudera.org:8080/#/c/19023/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19023/2//COMMIT_MSG@1
PS2, Line 1: Parent: 19114c72 (IMPALA-11578: Exclude locality test for 
remote FS)
can  you add some UT?


http://gerrit.cloudera.org:8080/#/c/19023/2/fe/src/main/java/org/apache/impala/analysis/StmtNode.java
File fe/src/main/java/org/apache/impala/analysis/StmtNode.java:

http://gerrit.cloudera.org:8080/#/c/19023/2/fe/src/main/java/org/apache/impala/analysis/StmtNode.java@49
PS2, Line 49:   @SuppressWarnings("unchecked")
why use @SuppressWarnings here?


http://gerrit.cloudera.org:8080/#/c/19023/2/fe/src/main/java/org/apache/impala/catalog/ScalarFunction.java
File fe/src/main/java/org/apache/impala/catalog/ScalarFunction.java:

http://gerrit.cloudera.org:8080/#/c/19023/2/fe/src/main/java/org/apache/impala/catalog/ScalarFunction.java@82
PS2, Line 82: if (BackendConfig.INSTANCE != null && 
BackendConfig.INSTANCE.skipLookupSymbol()) {
what is skipLookupSymbol() for?


http://gerrit.cloudera.org:8080/#/c/19023/2/fe/src/main/java/org/apache/impala/util/StmtVisitor.java
File fe/src/main/java/org/apache/impala/util/StmtVisitor.java:

http://gerrit.cloudera.org:8080/#/c/19023/2/fe/src/main/java/org/apache/impala/util/StmtVisitor.java@26
PS2, Line 26: List visit(StmtNode node);
Why don't you use or extend org.apache.impala.util.Visitor?
If this interface is necessary, can you add some description javadoc?



--
To view, visit http://gerrit.cloudera.org:8080/19023
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I246b088310f25b809d9e598c0ecc6613d9749ee3
Gerrit-Change-Number: 19023
Gerrit-PatchSet: 2
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Minghui Zhu 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Xiang Yang 
Gerrit-Comment-Date: Fri, 30 Sep 2022 02:43:42 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11612: fix ORDER BY expression rewrite bug

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19049 )

Change subject: IMPALA-11612: fix ORDER BY expression rewrite bug
..


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8642/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/19049
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I438c61c647e3f99665c94ea15e3a9d98b9ff9912
Gerrit-Change-Number: 19049
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Fri, 30 Sep 2022 02:10:27 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8592: Add support for insert events for 'LOAD DATA' statements from Impala

2022-09-29 Thread Quanlong Huang (Code Review)
Quanlong Huang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19052 )

Change subject: IMPALA-8592: Add support for insert events for 'LOAD DATA' 
statements from Impala
..


Patch Set 1:

(3 comments)

This is a pretty nice fix!

http://gerrit.cloudera.org:8080/#/c/19052/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19052/1//COMMIT_MSG@16
PS1, Line 16: - Run existing test_load.py
We also need tests to verify the INSERT events. Could you add some tests in 
tests/metadata/test_event_processing.py, e.g. add some cases for LOAD DATA in 
test_event_based_replication?


http://gerrit.cloudera.org:8080/#/c/19052/1/be/src/service/client-request-state.cc
File be/src/service/client-request-state.cc:

http://gerrit.cloudera.org:8080/#/c/19052/1/be/src/service/client-request-state.cc@806
PS1, Line 806: response.partition_name
nit: Could you add a comment mentioning that the partition_name is an empty 
string for unpartitioned tables? It's the same as DataSink::ROOT_PARTITION_KEY.


http://gerrit.cloudera.org:8080/#/c/19052/1/be/src/service/client-request-state.cc@809
PS1, Line 809:   catalog_update.__set_header(TCatalogServiceRequestHeader());
 :   catalog_update.header.__set_requesting_user(effective_user());
 :   
catalog_update.header.__set_client_ip(session()->network_address.hostname);
 :   
catalog_update.header.__set_want_minimal_response(FLAGS_use_local_catalog);
 :   catalog_update.header.__set_redacted_sql_stmt(
 :   query_ctx_.client_request.__isset.redacted_stmt ?
 :   query_ctx_.client_request.redacted_stmt : 
query_ctx_.client_request.stmt);
nit: these duplicate the code in ClientRequestState::ExecLoadDataRequestImpl(). 
It'd be nice to extract them into a method like 
GetCatalogServiceRequestHeader(), and simplify the code to

  catalog_update.__set_header(GetCatalogServiceRequestHeader())



--
To view, visit http://gerrit.cloudera.org:8080/19052
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7f1b470f40e0aaf891c9f3f327af393b2f9c74bc
Gerrit-Change-Number: 19052
Gerrit-PatchSet: 1
Gerrit-Owner: Yu-Wen Lai 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Yu-Wen Lai 
Gerrit-Comment-Date: Fri, 30 Sep 2022 01:55:21 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-10436: Require lower privilege for external Kudu table creation

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/17640 )

Change subject: IMPALA-10436: Require lower privilege for external Kudu table 
creation
..


Patch Set 7:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11491/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/17640
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7936e1d8c48696169f7ad7ad92abe44a26eea3c4
Gerrit-Change-Number: 17640
Gerrit-PatchSet: 7
Gerrit-Owner: Fang-Yu Rao 
Gerrit-Reviewer: Aman Sinha 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Fang-Yu Rao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Fri, 30 Sep 2022 01:18:24 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-10436: Require lower privilege for external Kudu table creation

2022-09-29 Thread Fang-Yu Rao (Code Review)
Fang-Yu Rao has uploaded a new patch set (#7). ( 
http://gerrit.cloudera.org:8080/17640 )

Change subject: IMPALA-10436: Require lower privilege for external Kudu table 
creation
..

IMPALA-10436: Require lower privilege for external Kudu table creation

This patch lowers the privilege requirement for external Kudu table
creation. Before this patch, a user was required to have the ALL
privilege on SERVER if the user wanted to create an external Kudu table.
In this patch we introduce a new type of resources called storage
handler URI and a new access type called RWSTORAGE that will be
supported by Apache Ranger once RANGER-3281 is resolved, which in turn
depends on the release of Apache Hive 4.0 that consists of HIVE-24705.

Specifically, after this patch, a user will be allowed to create an
external Kudu table as long as the user is granted the RWSTORAGE
privilege on the resource specified by a storage handler URI that points
to an existing Kudu table.

For instance, in order for a user 'non_owner' to create an external Kudu
table based on an existing Kudu table 'impala::tpch_kudu.nation', it
suffices to execute the following command as an administrator to grant
the necessary privilege to the requesting user, where "localhost" is the
default address of Kudu master host assuming there is only one single
master host in this example.

GRANT RWSTORAGE ON
STORAGEHANDLER_URI 'kudu://localhost/impala::tpch_kudu.nation'
TO USER non_owner

One may be wondering why we do not simply cancel the privilege check
that required the ALL privilege on SERVER for external Kudu table
creation. One scenario in which such relaxation is not secure is when
the owner or the creator of the existing Kudu table is different from
the requesting user who wants to create an external Kudu table in
Impala. Not requiring any additional privilege check would allow a user
without any privilege to retrieve the contents of the existing Kudu
table.

On the other hand, after this patch we still require a user to have the
ALL privilege on SERVER when the table property of
'kudu.master_addresses' is specified in a query that tries to create a
Kudu table whether or not the table is external. To be more specific,
the user 'non_owner' would be able to create an external Kudu table
using the following statement once being granted the RWSTORAGE privilege
on the specified storage handler URI above.

CREATE EXTERNAL TABLE default.kudu_tbl STORED AS KUDU
TBLPROPERTIES ('kudu.table_name'='impala::tpch_kudu.nation')

However, the following query submitted by the same user would be
rejected due to the user 'non_owner' not being granted the ALL privilege
on SERVER.

CREATE EXTERNAL TABLE default.kudu_tbl STORED AS KUDU
TBLPROPERTIES ('kudu.table_name'='impala::tpch_kudu.nation',
'kudu.master_addresses'='localhost')

We do not relax such a requirement in that specifying the addresses of
Kudu master hosts to connect should still be considered as an
administrative operation.

Testing:
 - Added various FE and E2E tests to verify Impala's behavior after this
   patch with respect to external Kudu table creation.
 - Verified that this patch passes the core tests in the DEBUG build.

Change-Id: I7936e1d8c48696169f7ad7ad92abe44a26eea3c4
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java
M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java
A fe/src/main/java/org/apache/impala/analysis/StorageHandlerUri.java
M fe/src/main/java/org/apache/impala/authorization/Authorizable.java
M fe/src/main/java/org/apache/impala/authorization/AuthorizableFactory.java
A 
fe/src/main/java/org/apache/impala/authorization/AuthorizableStorageHandlerUri.java
M 
fe/src/main/java/org/apache/impala/authorization/DefaultAuthorizableFactory.java
M fe/src/main/java/org/apache/impala/authorization/Privilege.java
M fe/src/main/java/org/apache/impala/authorization/PrivilegeRequestBuilder.java
M 
fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
M 
fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
M 
fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpalaResourceBuilder.java
M 
fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java
M testdata/workloads/functional-query/queries/QueryTest/grant_revoke.test
M 

[Impala-ASF-CR] IMPALA-9448: (Addendum) Avoid generic recreate for Ozone bucket

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19082 )

Change subject: IMPALA-9448: (Addendum) Avoid generic recreate for Ozone bucket
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11490/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19082
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I31a3813fac6f2d68859ba7fed53b5c5a82502780
Gerrit-Change-Number: 19082
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 30 Sep 2022 00:16:49 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11562: (Addendum) Update core-site to include ofs impl

2022-09-29 Thread Michael Smith (Code Review)
Hello weic...@apache.org, Joe McDonnell, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19081

to look at the new patch set (#2).

Change subject: IMPALA-11562: (Addendum) Update core-site to include ofs impl
..

IMPALA-11562: (Addendum) Update core-site to include ofs impl

Updates core-site to include the ofs implementation rather than o3fs. I
believe this still worked because both ofs and o3fs implementations live
in the same jar file, so including either statement results in both
classes loading. However we should use the correct statement to avoid
issues in the future.

Change-Id: I41cbc875bef29e730ee2afcbda42f07017119593
---
M testdata/cluster/node_templates/common/etc/hadoop/conf/core-site.xml.py
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/19081/2
--
To view, visit http://gerrit.cloudera.org:8080/19081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I41cbc875bef29e730ee2afcbda42f07017119593
Gerrit-Change-Number: 19081
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Anonymous Coward 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 


[Impala-ASF-CR] IMPALA-9448: (Addendum) Avoid generic recreate for Ozone bucket

2022-09-29 Thread Michael Smith (Code Review)
Michael Smith has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/19082


Change subject: IMPALA-9448: (Addendum) Avoid generic recreate for Ozone bucket
..

IMPALA-9448: (Addendum) Avoid generic recreate for Ozone bucket

Avoids using `hadoop fs` to recreate an Ozone bucket during snapshot
load, as this removes the encryption key.

Change-Id: I31a3813fac6f2d68859ba7fed53b5c5a82502780
---
M testdata/bin/load-test-warehouse-snapshot.sh
1 file changed, 7 insertions(+), 4 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/19082/1
--
To view, visit http://gerrit.cloudera.org:8080/19082
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I31a3813fac6f2d68859ba7fed53b5c5a82502780
Gerrit-Change-Number: 19082
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith 


[Impala-ASF-CR] IMPALA-11606: add 'untracked memory' metric.

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19029 )

Change subject: IMPALA-11606: add 'untracked memory' metric.
..


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8641/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/19029
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib16e00109d732f759c96c7a967eb1cc32124a03f
Gerrit-Change-Number: 19029
Gerrit-PatchSet: 3
Gerrit-Owner: Xiang Yang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Xianqing He 
Gerrit-Comment-Date: Thu, 29 Sep 2022 23:21:26 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11627: Build Impala with cdw dependencies

2022-09-29 Thread Yu-Wen Lai (Code Review)
Yu-Wen Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18994 )

Change subject: IMPALA-11627: Build Impala with cdw dependencies
..


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18994/6/java/pom.xml
File java/pom.xml:

http://gerrit.cloudera.org:8080/#/c/18994/6/java/pom.xml@109
PS6, Line 109:
nit: wrong indentation?



--
To view, visit http://gerrit.cloudera.org:8080/18994
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id379030f4b314e139c875584eee438b7416d89a4
Gerrit-Change-Number: 18994
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Yu-Wen Lai 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 29 Sep 2022 23:03:25 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11562: Update core-site to include ofs impl

2022-09-29 Thread Anonymous Coward (Code Review)
weic...@apache.org has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19081 )

Change subject: IMPALA-11562: Update core-site to include ofs impl
..


Patch Set 1:

I guess it was working before because we actually specified it in Hadoop's 
core-default.xml: https://github.com/apache/hadoop/pull/2276/files


--
To view, visit http://gerrit.cloudera.org:8080/19081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I41cbc875bef29e730ee2afcbda42f07017119593
Gerrit-Change-Number: 19081
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Anonymous Coward 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Comment-Date: Thu, 29 Sep 2022 22:35:27 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11562: Update core-site to include ofs impl

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19081 )

Change subject: IMPALA-11562: Update core-site to include ofs impl
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11489/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I41cbc875bef29e730ee2afcbda42f07017119593
Gerrit-Change-Number: 19081
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Anonymous Coward 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Comment-Date: Thu, 29 Sep 2022 22:06:11 +
Gerrit-HasComments: No


[Impala-ASF-CR] Speed up default configuration for Docker-based tests

2022-09-29 Thread Joe McDonnell (Code Review)
Joe McDonnell has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/19038 )

Change subject: Speed up default configuration for Docker-based tests
..

Speed up default configuration for Docker-based tests

Docker-based parallelized test runs have proven themselves to be quite a
bit faster than regular core or exhaustive mode builds. While regular
sequential builds have also enjoyed shorter runtimes recently,
Docker-based parallel builds still enjoy a speed advantage.

Scheduling the parallel build segments is currently driven from the
test driver script test-with-docker.py, and the order in which the
segments are considered is currently hard-coded. The ordering was
originally devised experimentally, by timing several test runs, then
ordering the test segments based on expected duration, from longest
to shortest.

The average wall-clock run times for various test segments have changed
since this original ordering was committed: FE tests have gotten
significantly longer, while upgrading the default worker instance
type cut shortened the serial phase(s) of E2E tests.

This patch makes two changes to achieve a shorter overall run time for
the Docker-based tests:
1. Reorders the default scheduling order of the test segments, based
   on currently measured durations
2. Increases the default suite concurrency for execution hosts:
   bumps suite concurrency from 4 to 5 for machines with memory sizes
   between 96 and 140 GBs (the currently used worker size)

The latter change is also based on measurements: memory usage reports for
total peak memory (RSS) and peak memory (RSS) per test segment both
showed significant amounts of unused memory on the current default
worker instance size (having 32 CPUs and 128 GB of RAM).
Experiments showed that this machine size can reliable handle five
concurrent containerized test sessions with some safety margin remaining,
so the patch increases the default concurrency for this machine
category.

with both changes applied the duration of a core-mode test run with
default settings is reduced from 2h45 to 2h25 (on average).

Tested by running the Docker-based default test suite in core mode,
with Ubuntu 16.04 and Rocky Linux 8.5 base images.

Change-Id: Ifb609bcfb10e9f9b281cc6b375c36c9638db168b
Reviewed-on: http://gerrit.cloudera.org:8080/19038
Reviewed-by: Michael Smith 
Reviewed-by: Joe McDonnell 
Tested-by: Impala Public Jenkins 
---
M docker/test-with-docker.py
1 file changed, 3 insertions(+), 3 deletions(-)

Approvals:
  Michael Smith: Looks good to me, but someone else must approve
  Joe McDonnell: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/19038
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ifb609bcfb10e9f9b281cc6b375c36c9638db168b
Gerrit-Change-Number: 19038
Gerrit-PatchSet: 2
Gerrit-Owner: Laszlo Gaal 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Laszlo Gaal 
Gerrit-Reviewer: Michael Smith 


[Impala-ASF-CR] IMPALA-11562: Update core-site to include ofs impl

2022-09-29 Thread Michael Smith (Code Review)
Michael Smith has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/19081


Change subject: IMPALA-11562: Update core-site to include ofs impl
..

IMPALA-11562: Update core-site to include ofs impl

Updates core-site to include the ofs implementation rather than o3fs. I
believe this still worked because both ofs and o3fs implementations live
in the same jar file, so including either statement results in both
classes loading. However we should use the correct statement to avoid
issues in the future.

Change-Id: I41cbc875bef29e730ee2afcbda42f07017119593
---
M testdata/cluster/node_templates/common/etc/hadoop/conf/core-site.xml.py
1 file changed, 1 insertion(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/19081/1
--
To view, visit http://gerrit.cloudera.org:8080/19081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I41cbc875bef29e730ee2afcbda42f07017119593
Gerrit-Change-Number: 19081
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith 


[Impala-ASF-CR] IMPALA-10851: Codegen for structs

2022-09-29 Thread Michael Smith (Code Review)
Michael Smith has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18526 )

Change subject: IMPALA-10851: Codegen for structs
..


Patch Set 12: Code-Review-1

(2 comments)

http://gerrit.cloudera.org:8080/#/c/18526/12/be/src/runtime/descriptors.cc
File be/src/runtime/descriptors.cc:

http://gerrit.cloudera.org:8080/#/c/18526/12/be/src/runtime/descriptors.cc@972
PS12, Line 972: case TYPE_ARRAY: { // CollectionVal has same memory layout 
as StringVal.
My point in an earlier comment was that https://gerrit.cloudera.org/c/18736/ 
added TYPE_MAP to some of these cases, and I think it needs to be added here 
and line 1024 after the rebase.


http://gerrit.cloudera.org:8080/#/c/18526/10/tests/query_test/test_nested_types.py
File tests/query_test/test_nested_types.py:

http://gerrit.cloudera.org:8080/#/c/18526/10/tests/query_test/test_nested_types.py@143
PS10, Line 143: if new_vector.get_value('exec_option')['disable_codegen'] 
== 'False':
> I reversed the order because in non-exhaustive builds only a subset of the
Reversing order seems fine, but I'd add a comment why. I'd also add the other 
options always set to 0 (even when they're not used) because I think it's less 
confusing if they're in the initial setup and not conditional down in the tests.



--
To view, visit http://gerrit.cloudera.org:8080/18526
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5272c3f095fd9f07877104ee03c8e43d0c4ec0b6
Gerrit-Change-Number: 18526
Gerrit-PatchSet: 12
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Comment-Date: Thu, 29 Sep 2022 18:16:59 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11606: add 'untracked memory' metric.

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19029 )

Change subject: IMPALA-11606: add 'untracked memory' metric.
..


Patch Set 3: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8640/


--
To view, visit http://gerrit.cloudera.org:8080/19029
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib16e00109d732f759c96c7a967eb1cc32124a03f
Gerrit-Change-Number: 19029
Gerrit-PatchSet: 3
Gerrit-Owner: Xiang Yang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Xianqing He 
Gerrit-Comment-Date: Thu, 29 Sep 2022 18:08:06 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11621: Remove hiveserver2.pid when shutting down HiveServer2

2022-09-29 Thread Joe McDonnell (Code Review)
Joe McDonnell has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/19051 )

Change subject: IMPALA-11621: Remove hiveserver2.pid when shutting down 
HiveServer2
..

IMPALA-11621: Remove hiveserver2.pid when shutting down HiveServer2

In HIVE-22193, Hive added graceful shutdown for HiveServer2.
This modified some of the startup scripts to maintain a
pid file. On startup, it verified that the pid is not already
running. Impala uses the hive startup script, but it kills
HiveServer2 with testdata/bin/kill-java-service.sh. If the
pid file outlives the process, then the OS may reuse the
pid and this can cause problems on startup.

This modifies testdata/bin/kill-hive-server.sh to remove
the pid file. testdata/bin/kill-java-service.sh would fail
if it did not kill HiveServer2, so it should be safe to
remove the pid file after the kill-java-service.sh call.

Testing:
 - Verified the hiveserver2.pid file is removed after shutting down.

Change-Id: I813626d06829a86854c6d2c1715f0c5f5109836d
Reviewed-on: http://gerrit.cloudera.org:8080/19051
Reviewed-by: Joe McDonnell 
Tested-by: Joe McDonnell 
---
M testdata/bin/kill-hive-server.sh
1 file changed, 5 insertions(+), 0 deletions(-)

Approvals:
  Joe McDonnell: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/19051
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I813626d06829a86854c6d2c1715f0c5f5109836d
Gerrit-Change-Number: 19051
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 


[Impala-ASF-CR] IMPALA-11621: Remove hiveserver2.pid when shutting down HiveServer2

2022-09-29 Thread Joe McDonnell (Code Review)
Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19051 )

Change subject: IMPALA-11621: Remove hiveserver2.pid when shutting down 
HiveServer2
..


Patch Set 2: Verified+1 Code-Review+2

Only commit message changed, going ahead.


--
To view, visit http://gerrit.cloudera.org:8080/19051
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I813626d06829a86854c6d2c1715f0c5f5109836d
Gerrit-Change-Number: 19051
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Thu, 29 Sep 2022 16:11:17 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11621: Remove hiveserver2.pid when shutting down HiveServer2

2022-09-29 Thread Joe McDonnell (Code Review)
Hello Quanlong Huang, Csaba Ringhofer, Michael Smith, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19051

to look at the new patch set (#2).

Change subject: IMPALA-11621: Remove hiveserver2.pid when shutting down 
HiveServer2
..

IMPALA-11621: Remove hiveserver2.pid when shutting down HiveServer2

In HIVE-22193, Hive added graceful shutdown for HiveServer2.
This modified some of the startup scripts to maintain a
pid file. On startup, it verified that the pid is not already
running. Impala uses the hive startup script, but it kills
HiveServer2 with testdata/bin/kill-java-service.sh. If the
pid file outlives the process, then the OS may reuse the
pid and this can cause problems on startup.

This modifies testdata/bin/kill-hive-server.sh to remove
the pid file. testdata/bin/kill-java-service.sh would fail
if it did not kill HiveServer2, so it should be safe to
remove the pid file after the kill-java-service.sh call.

Testing:
 - Verified the hiveserver2.pid file is removed after shutting down.

Change-Id: I813626d06829a86854c6d2c1715f0c5f5109836d
---
M testdata/bin/kill-hive-server.sh
1 file changed, 5 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/19051/2
--
To view, visit http://gerrit.cloudera.org:8080/19051
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I813626d06829a86854c6d2c1715f0c5f5109836d
Gerrit-Change-Number: 19051
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 


[Impala-ASF-CR] IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES statement with single value

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19048 )

Change subject: IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES 
statement with single value
..


Patch Set 7: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8639/


-- 
To view, visit http://gerrit.cloudera.org:8080/19048
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I952377ed14eba26e3774e7776eb81a95d1d8e76f
Gerrit-Change-Number: 19048
Gerrit-PatchSet: 7
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Peter Rozsa 
Gerrit-Comment-Date: Thu, 29 Sep 2022 16:03:14 +
Gerrit-HasComments: No


[Impala-ASF-CR] Prototype to provide an option to install Java 11

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19031 )

Change subject: Prototype to provide an option to install Java 11
..


Patch Set 11:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11488/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icc1dbd3f6a2279840218dc1da2b60077e211a328
Gerrit-Change-Number: 19031
Gerrit-PatchSet: 11
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Thu, 29 Sep 2022 15:55:59 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-10851: Codegen for structs

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18526 )

Change subject: IMPALA-10851: Codegen for structs
..


Patch Set 12:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11487/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18526
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5272c3f095fd9f07877104ee03c8e43d0c4ec0b6
Gerrit-Change-Number: 18526
Gerrit-PatchSet: 12
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Comment-Date: Thu, 29 Sep 2022 15:56:13 +
Gerrit-HasComments: No


[Impala-ASF-CR] Prototype to provide an option to install Java 11

2022-09-29 Thread Joe McDonnell (Code Review)
Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19031

to look at the new patch set (#11).

Change subject: Prototype to provide an option to install Java 11
..

Prototype to provide an option to install Java 11

Change-Id: Icc1dbd3f6a2279840218dc1da2b60077e211a328
---
M bin/impala-config.sh
M docker/CMakeLists.txt
M docker/daemon_entrypoint.sh
M docker/install_os_packages.sh
4 files changed, 111 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/19031/11
--
To view, visit http://gerrit.cloudera.org:8080/19031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Icc1dbd3f6a2279840218dc1da2b60077e211a328
Gerrit-Change-Number: 19031
Gerrit-PatchSet: 11
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 


[Impala-ASF-CR] IMPALA-10851: Codegen for structs

2022-09-29 Thread Daniel Becker (Code Review)
Daniel Becker has uploaded a new patch set (#12). ( 
http://gerrit.cloudera.org:8080/18526 )

Change subject: IMPALA-10851: Codegen for structs
..

IMPALA-10851: Codegen for structs

IMPALA-9495 added support for struct types in SELECT lists but only with
codegen turned off. This commit implements codegen for struct types.

To facilitate this, code generation for reading and writing 'AnyVal's
has been refactored. A new class, 'CodegenAnyValReadWriteInfo' is
introduced. This class is an interface between sources and destinations,
one of which is an 'AnyVal' object: sources generate an instance of this
class and destinations take that instance and use it to write the value.

The other side can for example be tuples from which we read (in the case
of 'SlotRef') or tuples we write into (in case of materialisation, see
Tuple::CodegenMaterializeExprs()).  The main advantage is that sources
do not have to know how to write their destinations, only how to read
the values (and vice versa).

Testing:
  - enabling tests for structs in select list in codegen mode in
tests/query_test/test_nested_types.py

Change-Id: I5272c3f095fd9f07877104ee03c8e43d0c4ec0b6
---
M be/src/codegen/CMakeLists.txt
A be/src/codegen/codegen-anyval-read-write-info.cc
A be/src/codegen/codegen-anyval-read-write-info.h
M be/src/codegen/codegen-anyval.cc
M be/src/codegen/codegen-anyval.h
M be/src/codegen/gen_ir_descriptions.py
M be/src/codegen/impala-ir.cc
M be/src/codegen/llvm-codegen.cc
M be/src/codegen/llvm-codegen.h
M be/src/exec/aggregator.cc
M be/src/exec/filter-context.cc
M be/src/exec/hash-table.cc
M be/src/exec/hash-table.h
M be/src/exprs/CMakeLists.txt
M be/src/exprs/hive-udf-call-ir.cc
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M be/src/exprs/kudu-partition-expr.cc
A be/src/exprs/scalar-expr-evaluator-ir.cc
M be/src/exprs/scalar-expr-evaluator.h
M be/src/exprs/slot-ref.cc
M be/src/exprs/slot-ref.h
M be/src/runtime/descriptors.cc
M be/src/runtime/descriptors.h
M be/src/runtime/fragment-state.cc
M be/src/runtime/fragment-state.h
M be/src/runtime/krpc-data-stream-sender.cc
M be/src/runtime/tuple.cc
M be/src/udf/udf-internal.h
M be/src/udf/udf-ir.cc
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M 
testdata/workloads/functional-query/queries/QueryTest/nested-struct-in-select-list.test
M tests/authorization/test_ranger.py
M tests/query_test/test_nested_types.py
35 files changed, 2,200 insertions(+), 808 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/18526/12
--
To view, visit http://gerrit.cloudera.org:8080/18526
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5272c3f095fd9f07877104ee03c8e43d0c4ec0b6
Gerrit-Change-Number: 18526
Gerrit-PatchSet: 12
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 


[Impala-ASF-CR] IMPALA-11621: Remove hiveserver2.pid when shutting down HiveServer2

2022-09-29 Thread Joe McDonnell (Code Review)
Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19051 )

Change subject: IMPALA-11621: Remove hiveserver2.pid when shutting down 
HiveServer2
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19051/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19051/1//COMMIT_MSG@23
PS1, Line 23: remove
> typo
Thanks for pointing that out, will fix.



--
To view, visit http://gerrit.cloudera.org:8080/19051
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I813626d06829a86854c6d2c1715f0c5f5109836d
Gerrit-Change-Number: 19051
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Thu, 29 Sep 2022 15:34:19 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11627: Build Impala with cdw dependencies

2022-09-29 Thread Csaba Ringhofer (Code Review)
Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/18994

to look at the new patch set (#6).

Change subject: IMPALA-11627: Build Impala with cdw dependencies
..

IMPALA-11627: Build Impala with cdw dependencies

The change switches to download cdw versions of Hive dependencies.
As cdw depends on older versions of some components than cdp,
this is a step back from some components, but this didn't lead
to test failures.

For maven artifects both cdp and cdw versions are downloaded
and added as repositiories, so all component can find the
versions it depends on.

Other changes:
- updated resource-requirements.test due to different number
  of files tpch_orc_def.lineitem
- added redirections/implementations for new HMS APIs in
  MetastoreService
- fixes the bug mentioned in IMPALA-11397

Change-Id: Id379030f4b314e139c875584eee438b7416d89a4
---
M bin/impala-config.sh
M bin/jenkins/archive_m2_directory.sh
M fe/pom.xml
M 
fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java
M fe/src/main/java/org/apache/impala/catalog/metastore/HmsApiNameEnum.java
M 
fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java
M java/pom.xml
M 
testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
8 files changed, 249 insertions(+), 39 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/18994/6
--
To view, visit http://gerrit.cloudera.org:8080/18994
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id379030f4b314e139c875584eee438b7416d89a4
Gerrit-Change-Number: 18994
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-11606: add 'untracked memory' metric.

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19029 )

Change subject: IMPALA-11606: add 'untracked memory' metric.
..


Patch Set 3:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11486/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19029
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib16e00109d732f759c96c7a967eb1cc32124a03f
Gerrit-Change-Number: 19029
Gerrit-PatchSet: 3
Gerrit-Owner: Xiang Yang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Xianqing He 
Gerrit-Comment-Date: Thu, 29 Sep 2022 13:19:43 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11606: add 'untracked memory' metric.

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19029 )

Change subject: IMPALA-11606: add 'untracked memory' metric.
..


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8640/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/19029
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib16e00109d732f759c96c7a967eb1cc32124a03f
Gerrit-Change-Number: 19029
Gerrit-PatchSet: 3
Gerrit-Owner: Xiang Yang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Xianqing He 
Gerrit-Comment-Date: Thu, 29 Sep 2022 13:05:34 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11606: add 'untracked memory' metric.

2022-09-29 Thread Xiang Yang (Code Review)
Hello Xianqing He, Jian Zhang, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19029

to look at the new patch set (#3).

Change subject: IMPALA-11606: add 'untracked memory' metric.
..

IMPALA-11606: add 'untracked memory' metric.

Add a gauge metric 'untracked memroy' to record the memory size that
not tracked by the 'process' mem-tracker.

Change-Id: Ib16e00109d732f759c96c7a967eb1cc32124a03f
---
M be/src/runtime/mem-tracker.cc
M be/src/runtime/mem-tracker.h
M common/thrift/metrics.json
3 files changed, 41 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/19029/3
--
To view, visit http://gerrit.cloudera.org:8080/19029
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib16e00109d732f759c96c7a967eb1cc32124a03f
Gerrit-Change-Number: 19029
Gerrit-PatchSet: 3
Gerrit-Owner: Xiang Yang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Xianqing He 


[Impala-ASF-CR] IMPALA-10851: Codegen for structs

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18526 )

Change subject: IMPALA-10851: Codegen for structs
..


Patch Set 11:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11485/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18526
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5272c3f095fd9f07877104ee03c8e43d0c4ec0b6
Gerrit-Change-Number: 18526
Gerrit-PatchSet: 11
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Comment-Date: Thu, 29 Sep 2022 12:50:00 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-10851: Codegen for structs

2022-09-29 Thread Daniel Becker (Code Review)
Daniel Becker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18526 )

Change subject: IMPALA-10851: Codegen for structs
..


Patch Set 11:

Thanks Michael Smith for the review. First I rebased the change on master, I 
will address the comments in the next patch set.


--
To view, visit http://gerrit.cloudera.org:8080/18526
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5272c3f095fd9f07877104ee03c8e43d0c4ec0b6
Gerrit-Change-Number: 18526
Gerrit-PatchSet: 11
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Comment-Date: Thu, 29 Sep 2022 12:28:53 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-10851: Codegen for structs

2022-09-29 Thread Daniel Becker (Code Review)
Daniel Becker has uploaded a new patch set (#11). ( 
http://gerrit.cloudera.org:8080/18526 )

Change subject: IMPALA-10851: Codegen for structs
..

IMPALA-10851: Codegen for structs

IMPALA-9495 added support for struct types in SELECT lists but only with
codegen turned off. This commit implements codegen for struct types.

To facilitate this, code generation for reading and writing 'AnyVal's
has been refactored. A new class, 'CodegenAnyValReadWriteInfo' is
introduced. This class is an interface between sources and destinations,
one of which is an 'AnyVal' object: sources generate an instance of this
class and destinations take that instance and use it to write the value.

The other side can for example be tuples from which we read (in the case
of 'SlotRef') or tuples we write into (in case of materialisation, see
Tuple::CodegenMaterializeExprs()).  The main advantage is that sources
do not have to know how to write their destinations, only how to read
the values (and vice versa).

Testing:
  - enabling tests for structs in select list in codegen mode in
tests/query_test/test_nested_types.py

Change-Id: I5272c3f095fd9f07877104ee03c8e43d0c4ec0b6
---
M be/src/codegen/CMakeLists.txt
A be/src/codegen/codegen-anyval-read-write-info.cc
A be/src/codegen/codegen-anyval-read-write-info.h
M be/src/codegen/codegen-anyval.cc
M be/src/codegen/codegen-anyval.h
M be/src/codegen/gen_ir_descriptions.py
M be/src/codegen/impala-ir.cc
M be/src/codegen/llvm-codegen.cc
M be/src/codegen/llvm-codegen.h
M be/src/exec/aggregator.cc
M be/src/exec/filter-context.cc
M be/src/exec/hash-table.cc
M be/src/exec/hash-table.h
M be/src/exprs/CMakeLists.txt
M be/src/exprs/hive-udf-call-ir.cc
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M be/src/exprs/kudu-partition-expr.cc
A be/src/exprs/scalar-expr-evaluator-ir.cc
M be/src/exprs/scalar-expr-evaluator.h
M be/src/exprs/slot-ref.cc
M be/src/exprs/slot-ref.h
M be/src/runtime/descriptors.cc
M be/src/runtime/descriptors.h
M be/src/runtime/fragment-state.cc
M be/src/runtime/fragment-state.h
M be/src/runtime/krpc-data-stream-sender.cc
M be/src/runtime/tuple.cc
M be/src/udf/udf-internal.h
M be/src/udf/udf-ir.cc
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M 
testdata/workloads/functional-query/queries/QueryTest/nested-struct-in-select-list.test
M tests/authorization/test_ranger.py
M tests/query_test/test_nested_types.py
35 files changed, 2,190 insertions(+), 807 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/18526/11
--
To view, visit http://gerrit.cloudera.org:8080/18526
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5272c3f095fd9f07877104ee03c8e43d0c4ec0b6
Gerrit-Change-Number: 18526
Gerrit-PatchSet: 11
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 


[Impala-ASF-CR] IMPALA-11558: Ensure one Kudu client created (FE) for the specified Kudu master addresses

2022-09-29 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19046 )

Change subject: IMPALA-11558: Ensure one Kudu client created (FE) for the 
specified Kudu master addresses
..


Patch Set 4: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/19046
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1003556d3afc8e8216142cac4007a4c99046caeb
Gerrit-Change-Number: 19046
Gerrit-PatchSet: 4
Gerrit-Owner: Xianqing He 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Reviewer: Xiang Yang 
Gerrit-Reviewer: Xianqing He 
Gerrit-Comment-Date: Thu, 29 Sep 2022 12:14:56 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES statement with single value

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19048 )

Change subject: IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES 
statement with single value
..


Patch Set 6:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11484/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19048
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I952377ed14eba26e3774e7776eb81a95d1d8e76f
Gerrit-Change-Number: 19048
Gerrit-PatchSet: 6
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Peter Rozsa 
Gerrit-Comment-Date: Thu, 29 Sep 2022 11:11:04 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES statement with single value

2022-09-29 Thread Daniel Becker (Code Review)
Daniel Becker has uploaded a new patch set (#6). ( 
http://gerrit.cloudera.org:8080/19048 )

Change subject: IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES 
statement with single value
..

IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES statement with
single value

If the query contains a VALUES clause with a single value, the 'analyzed
query' in the level 2 explain plan contains a UNION where both sides are
the value from the VALUES clause. For

INSERT INTO double_tbl VALUES (-0.43149576573887316);

we get

SELECT CAST(-0.43149576573887316 AS DECIMAL(17,17))
UNION
SELECT CAST(-0.43149576573887316 AS DECIMAL(17,17));

This is because of a bug in 'SetOperationStmt::toSql()'; 'ValuesStmt's
are implemented as special 'UnionStmt's, which derive from
'SetOperationStmt'. 'SetOperationStmt::toSql()' implicitly assumes that
there are at least 2 operands and always prints the first one and the
last one separately, even if they are the same.

Note that 'ValuesStmt' is the only 'SetOperationStmt' that may contain a
single operand - it is syntactically impossible in SQL to create other
'SetOperationStmt's with only one operand.

This patch modifies 'SetOperationStmt::toSql()' so that if there is only
one operand, only that operand is printed (so there is no reference to
the set operation in the resulting string). This change does not affect
how such queries are handled, only how the analyzed query is printed.

Testing:
 - Extended AnalyzeStmtsTest.TestValuesStmt so it now also checks what
   VALUES clauses with one and two operands are rewritten to during
   analysis.

Change-Id: I952377ed14eba26e3774e7776eb81a95d1d8e76f
---
M fe/src/main/java/org/apache/impala/analysis/SetOperationStmt.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java
M 
testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
4 files changed, 31 insertions(+), 15 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/19048/6
--
To view, visit http://gerrit.cloudera.org:8080/19048
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I952377ed14eba26e3774e7776eb81a95d1d8e76f
Gerrit-Change-Number: 19048
Gerrit-PatchSet: 6
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Peter Rozsa 


[Impala-ASF-CR] IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES statement with single value

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19048 )

Change subject: IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES 
statement with single value
..


Patch Set 7:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8639/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/19048
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I952377ed14eba26e3774e7776eb81a95d1d8e76f
Gerrit-Change-Number: 19048
Gerrit-PatchSet: 7
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Peter Rozsa 
Gerrit-Comment-Date: Thu, 29 Sep 2022 10:51:13 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES statement with single value

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19048 )

Change subject: IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES 
statement with single value
..


Patch Set 7: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/19048
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I952377ed14eba26e3774e7776eb81a95d1d8e76f
Gerrit-Change-Number: 19048
Gerrit-PatchSet: 7
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Peter Rozsa 
Gerrit-Comment-Date: Thu, 29 Sep 2022 10:51:12 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES statement with single value

2022-09-29 Thread Daniel Becker (Code Review)
Daniel Becker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19048 )

Change subject: IMPALA-10356: Fix SetOperationStmt::toSql() for VALUES 
statement with single value
..


Patch Set 5:

The error is strange, doesn't seem to be related:
/home/ubuntu/Impala/toolchain/toolchain-packages-gcc10.4.0/binutils-2.35.1/bin/ld.gold:
 out of memory
collect2: error: ld returned 1 exit status


--
To view, visit http://gerrit.cloudera.org:8080/19048
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I952377ed14eba26e3774e7776eb81a95d1d8e76f
Gerrit-Change-Number: 19048
Gerrit-PatchSet: 5
Gerrit-Owner: Daniel Becker 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Peter Rozsa 
Gerrit-Comment-Date: Thu, 29 Sep 2022 10:41:52 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11624: Bump Impyla dependency to 0.18.0

2022-09-29 Thread Riza Suminto (Code Review)
Riza Suminto has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18456 )

Change subject: IMPALA-11624: Bump Impyla dependency to 0.18.0
..


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18456/6/bin/impala-config.sh
File bin/impala-config.sh:

http://gerrit.cloudera.org:8080/#/c/18456/6/bin/impala-config.sh@179
PS6, Line 179: shell/ext-py
Hi Csaba. Just wondering, what will happen with shell/ext-py/thrift-0.14.2 ? Do 
we need to replace it with source code from thrift-0.16.0 ?



--
To view, visit http://gerrit.cloudera.org:8080/18456
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7265558b0e07959c606cba73cd251c3edfcb3ed5
Gerrit-Change-Number: 18456
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Thu, 29 Sep 2022 10:04:41 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11624: Bump Impyla dependency to 0.18.0

2022-09-29 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18456 )

Change subject: IMPALA-11624: Bump Impyla dependency to 0.18.0
..


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18456/5//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/18456/5//COMMIT_MSG@7
PS5, Line 7: IMPALA-11624: Bump Impyla dependency to 0.18.0
> nit: Should you add Jira number?
Created a Jira about this.



--
To view, visit http://gerrit.cloudera.org:8080/18456
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7265558b0e07959c606cba73cd251c3edfcb3ed5
Gerrit-Change-Number: 18456
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Thu, 29 Sep 2022 09:17:18 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11624: Bump Impyla dependency to 0.18.0

2022-09-29 Thread Csaba Ringhofer (Code Review)
Hello Wenzhe Zhou, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/18456

to look at the new patch set (#6).

Change subject: IMPALA-11624: Bump Impyla dependency to 0.18.0
..

IMPALA-11624: Bump Impyla dependency to 0.18.0

IMPALA_THRIFT_PY_VERSION is also bumped to 0.16.0p3.
As 0.16.0p3 Thrift does not contain Python related
patches and Impyla 0.18.0 depends on Thrift 0.16.0,
now we are consistently using Thrift 0.16.0 in all
Python code.

Change-Id: I7265558b0e07959c606cba73cd251c3edfcb3ed5
---
M bin/impala-config.sh
M infra/python/deps/requirements.txt
M shell/packaging/requirements.txt
3 files changed, 3 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/56/18456/6
--
To view, visit http://gerrit.cloudera.org:8080/18456
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7265558b0e07959c606cba73cd251c3edfcb3ed5
Gerrit-Change-Number: 18456
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Wenzhe Zhou 


[Impala-ASF-CR] IMPALA-11621: Remove hiveserver2.pid when shutting down HiveServer2

2022-09-29 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19051 )

Change subject: IMPALA-11621: Remove hiveserver2.pid when shutting down 
HiveServer2
..


Patch Set 1: Code-Review+2

(1 comment)

I think I have bumped into the issue in https://gerrit.cloudera.org/#/c/18994/

http://gerrit.cloudera.org:8080/#/c/19051/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19051/1//COMMIT_MSG@23
PS1, Line 23: remove
typo



--
To view, visit http://gerrit.cloudera.org:8080/19051
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I813626d06829a86854c6d2c1715f0c5f5109836d
Gerrit-Change-Number: 19051
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Thu, 29 Sep 2022 08:19:44 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-10902: In the case of outer join connections, skip unnecessary BE check

2022-09-29 Thread Xianqing He (Code Review)
Xianqing He has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/17830 )

Change subject: IMPALA-10902: In the case of outer join connections, skip 
unnecessary BE check
..


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/17830/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:

http://gerrit.cloudera.org:8080/#/c/17830/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@3737
PS4, Line 3737: } else if (expr instanceof BetweenPredicate) {
Just wonder whether exists this case since BetweenPredicate rewrites into an 
equivalent conjunctive/disjunctive CompoundPredicate in analyzer



--
To view, visit http://gerrit.cloudera.org:8080/17830
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If6f4db9b5aa00ee5117118696c58c07ffaaf0b66
Gerrit-Change-Number: 17830
Gerrit-PatchSet: 4
Gerrit-Owner: pengdou 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jian Zhang 
Gerrit-Reviewer: Xianqing He 
Gerrit-Comment-Date: Thu, 29 Sep 2022 08:18:16 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11612: fix ORDER BY expression rewrite bug

2022-09-29 Thread Anonymous Coward (Code Review)
Hello Quanlong Huang, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19049

to look at the new patch set (#3).

Change subject: IMPALA-11612: fix ORDER BY expression rewrite bug
..

IMPALA-11612: fix ORDER BY expression rewrite bug

When the query SQL is overwritten, both the selectList and groupByList
fields are preprocessed except for the orderByList fields, the result
of rewoten orderByList is not same as selectList and groupByList.
It occur exception in method SelectStmt.verifyAggregation.
So we should add preprocessing before the orderByList be rewrote.

Change-Id: I438c61c647e3f99665c94ea15e3a9d98b9ff9912
---
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java
2 files changed, 20 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/19049/3
--
To view, visit http://gerrit.cloudera.org:8080/19049
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I438c61c647e3f99665c94ea15e3a9d98b9ff9912
Gerrit-Change-Number: 19049
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 


[Impala-ASF-CR] Prototype to provide an option to install Java 11

2022-09-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19031 )

Change subject: Prototype to provide an option to install Java 11
..


Patch Set 10:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11483/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icc1dbd3f6a2279840218dc1da2b60077e211a328
Gerrit-Change-Number: 19031
Gerrit-PatchSet: 10
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Thu, 29 Sep 2022 06:12:12 +
Gerrit-HasComments: No