date:20220920

Hello Wenzhe Zhou,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19022

to look at the new patch set (#2).

Change subject: IMPALA-11599: Build gdb with the OS compiler
..

IMPALA-11599: Build gdb with the OS compiler

With the switch to a newer GDB that accompanied
the GCC 10.4 upgrade, GDB now has a dependency on
libstdc++. gdb from the GCC 10.4 toolchain fails
on Centos 7 with messages like:
gdb: /../lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by 
gdb)
gdb: /../lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by 
gdb)
gdb: /../lib64/libstdc++.so.6: version `CXXABI_1.3.11' not found (required by 
gdb)
Checking the gdb binary, its RPATH is not setup
correctly, so it is not able to find the libstdc++
that ships with GCC 10 and Centos 7's libstdc++
is too old.

Since Impala only needs the gdb binary and setting the
RPATH is difficult for gdb, this switches to building
gdb with the OS compilers / libstdc++. That avoids the
need to fix the RPATH.

Testing:
 - Ran build, downloaded the binary on Centos 7, verified
   it ran properly

Change-Id: I0b794f3f28e3932c08ced0952bc1ba342c3dc0f6
---
M buildall.sh
M functions.sh
M init-compiler.sh
3 files changed, 16 insertions(+), 13 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/native-toolchain 
refs/changes/22/19022/2
--
To view, visit http://gerrit.cloudera.org:8080/19022
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0b794f3f28e3932c08ced0952bc1ba342c3dc0f6
Gerrit-Change-Number: 19022
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Wenzhe Zhou

[native-toolchain-CR] IMPALA-11599: Build gdb with the OS compiler

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19022 )

Change subject: IMPALA-11599: Build gdb with the OS compiler
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19022/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19022/1//COMMIT_MSG@16
PS1, Line 16: it's
> nit: its
Done



--
To view, visit http://gerrit.cloudera.org:8080/19022
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0b794f3f28e3932c08ced0952bc1ba342c3dc0f6
Gerrit-Change-Number: 19022
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Wed, 21 Sep 2022 04:54:13 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11160: Ignore stale ALTER PARTITION events on transactional tables

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/19020 )

Change subject: IMPALA-11160: Ignore stale ALTER_PARTITION events on 
transactional tables
..

IMPALA-11160: Ignore stale ALTER_PARTITION events on transactional tables

When applying ALTER_PARTITION events on transactional tables, we refresh
the partition using the metadata in events if
hms_event_incremental_refresh_transactional_table is enabled (which is
the default). This could be wrong if the ALTER_PARTITION event is stale.
The partition metadata will be rolled back to a stale state.

This patch compares the eventId with the createEventId of the table and
ignores those ALTER_PARTITION events that have older (smaller) event
ids. Note that we already do this for many other event types,
ALTER_PARTITION is somehow missing the checks.

Eventually we should depend on the lastSyncedEventId and replace
createEventId with it. The self-event detection can also be replaced
since self-events are also stale events. These will be addressed in
IMPALA-10976.

Tests
- Verified locally with local-catalog mode and event-processor enabled
  and iterated test_acid_compute_stats for 1400 times. Without the fix,
  the test would fail in tens of runs.

Change-Id: I5bb8cfc213093f3bbd0359c7084b277a3bd5264a
Reviewed-on: http://gerrit.cloudera.org:8080/19020
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
1 file changed, 5 insertions(+), 0 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/19020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I5bb8cfc213093f3bbd0359c7084b277a3bd5264a
Gerrit-Change-Number: 19020
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Yu-Wen Lai

[Impala-ASF-CR] IMPALA-11160: Ignore stale ALTER PARTITION events on transactional tables

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19020 )

Change subject: IMPALA-11160: Ignore stale ALTER_PARTITION events on 
transactional tables
..


Patch Set 2: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/19020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5bb8cfc213093f3bbd0359c7084b277a3bd5264a
Gerrit-Change-Number: 19020
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Yu-Wen Lai 
Gerrit-Comment-Date: Wed, 21 Sep 2022 04:20:13 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11566: Provide SQL parsing capabilities to other applications in the form of jar packages

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19023 )

Change subject: IMPALA-11566: Provide SQL parsing capabilities to other 
applications in the form of jar packages
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11396/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19023
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I246b088310f25b809d9e598c0ecc6613d9749ee3
Gerrit-Change-Number: 19023
Gerrit-PatchSet: 2
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 21 Sep 2022 04:12:06 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11566: Provide SQL parsing capabilities to other applications in the form of jar packages

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19023 )

Change subject: IMPALA-11566: Provide SQL parsing capabilities to other 
applications in the form of jar packages
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11395/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19023
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I246b088310f25b809d9e598c0ecc6613d9749ee3
Gerrit-Change-Number: 19023
Gerrit-PatchSet: 1
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 21 Sep 2022 04:04:35 +
Gerrit-HasComments: No

[native-toolchain-CR] IMPALA-11599: Build gdb with the OS compiler

2022-09-20 Thread Wenzhe Zhou (Code Review)

Wenzhe Zhou has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19022 )

Change subject: IMPALA-11599: Build gdb with the OS compiler
..


Patch Set 1: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19022/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19022/1//COMMIT_MSG@16
PS1, Line 16: it's
nit: its



--
To view, visit http://gerrit.cloudera.org:8080/19022
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0b794f3f28e3932c08ced0952bc1ba342c3dc0f6
Gerrit-Change-Number: 19022
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Wed, 21 Sep 2022 04:00:56 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11566: Provide SQL parsing capabilities to other applications in the form of jar packages

Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19023

to look at the new patch set (#2).

Change subject: IMPALA-11566: Provide SQL parsing capabilities to other 
applications in the form of jar packages
..

IMPALA-11566: Provide SQL parsing capabilities to other applications in the 
form of jar packages

We have a requirement to rewrite the parameters of a function in SQL, and there 
will be more requirements related to SQL parsing in the future. Before, we used 
jsqlParser for parsing. In order to be more compatible with impala sql, we want 
to use impala's parser.

Not all statements currently support the toSql method, so to provide complete 
parsing capabilities, it is also necessary to support the toSql method of other 
statements, such as AlterTableStmt. But in our scenario, there are only select 
and use statements, which are already supported. So all we need to do is to 
analyze all the functions in sql and type out the parser package.

We can rely on it in the pom in another project:


org.apache.impala
impala-parser
${impala.version}


And then we can rewrite the sql:

public String parse(String sql) {
  StatementBase node = Parser.parse(sql);
  FunctionCallExprVisitor functionCallExprVisitor = new 
FunctionCallExprVisitor();
  node.accept(functionCallExprVisitor);
  List funcCalls = functionCallExprVisitor.getFuncCalls();
  // do something with funcCalla
  return node.toSql();
}

Change-Id: I246b088310f25b809d9e598c0ecc6613d9749ee3
---
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/pom.xml
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/StmtNode.java
M fe/src/main/java/org/apache/impala/catalog/ScalarFunction.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
A fe/src/main/java/org/apache/impala/util/FunctionCallExprVisitor.java
A fe/src/main/java/org/apache/impala/util/StmtVisitor.java
9 files changed, 162 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/23/19023/2
--
To view, visit http://gerrit.cloudera.org:8080/19023
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I246b088310f25b809d9e598c0ecc6613d9749ee3
Gerrit-Change-Number: 19023
Gerrit-PatchSet: 2
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] IMPALA-11566: Provide SQL parsing capabilities to other applications in the form of jar packages

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19023 )

Change subject: IMPALA-11566: Provide SQL parsing capabilities to other 
applications in the form of jar packages
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19023/1/fe/src/main/java/org/apache/impala/util/FunctionCallExprVisitor.java
File fe/src/main/java/org/apache/impala/util/FunctionCallExprVisitor.java:

http://gerrit.cloudera.org:8080/#/c/19023/1/fe/src/main/java/org/apache/impala/util/FunctionCallExprVisitor.java@35
PS1, Line 35:
line has trailing whitespace



--
To view, visit http://gerrit.cloudera.org:8080/19023
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I246b088310f25b809d9e598c0ecc6613d9749ee3
Gerrit-Change-Number: 19023
Gerrit-PatchSet: 1
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 21 Sep 2022 03:43:56 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11566: Provide SQL parsing capabilities to other applications in the form of jar packages

Minghui Zhu has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/19023


Change subject: IMPALA-11566: Provide SQL parsing capabilities to other 
applications in the form of jar packages
..

IMPALA-11566: Provide SQL parsing capabilities to other applications in the 
form of jar packages

We have a requirement to rewrite the parameters of a function in SQL, and there 
will be more requirements related to SQL parsing in the future. Before, we used 
jsqlParser for parsing. In order to be more compatible with impala sql, we want 
to use impala's parser.

Not all statements currently support the toSql method, so to provide complete 
parsing capabilities, it is also necessary to support the toSql method of other 
statements, such as AlterTableStmt. But in our scenario, there are only select 
and use statements, which are already supported. So all we need to do is to 
analyze all the functions in sql and type out the parser package.

We can rely on it in the pom in another project:


org.apache.impala
impala-parser
${impala.version}


And then we can rewrite the sql:

public String parse(String sql) {
  StatementBase node = Parser.parse(sql);
  FunctionCallExprVisitor functionCallExprVisitor = new 
FunctionCallExprVisitor();
  node.accept(functionCallExprVisitor);
  List funcCalls = functionCallExprVisitor.getFuncCalls();
  // do something with funcCalla
  return node.toSql();
}

Change-Id: I246b088310f25b809d9e598c0ecc6613d9749ee3
---
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/pom.xml
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/StmtNode.java
M fe/src/main/java/org/apache/impala/catalog/ScalarFunction.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
A fe/src/main/java/org/apache/impala/util/FunctionCallExprVisitor.java
A fe/src/main/java/org/apache/impala/util/StmtVisitor.java
9 files changed, 162 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/23/19023/1
--
To view, visit http://gerrit.cloudera.org:8080/19023
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I246b088310f25b809d9e598c0ecc6613d9749ee3
Gerrit-Change-Number: 19023
Gerrit-PatchSet: 1
Gerrit-Owner: Minghui Zhu

[native-toolchain-CR] IMPALA-11599: Build gdb with the OS compiler

Joe McDonnell has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/19022


Change subject: IMPALA-11599: Build gdb with the OS compiler
..

IMPALA-11599: Build gdb with the OS compiler

With the switch to a newer GDB that accompanied
the GCC 10.4 upgrade, GDB now has a dependency on
libstdc++. gdb from the GCC 10.4 toolchain fails
on Centos 7 with messages like:
gdb: /../lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by 
gdb)
gdb: /../lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by 
gdb)
gdb: /../lib64/libstdc++.so.6: version `CXXABI_1.3.11' not found (required by 
gdb)
Checking the gdb binary, it's RPATH is not setup
correctly, so it is not able to find the libstdc++
that ships with GCC 10 and Centos 7's libstdc++
is too old.

Since Impala only needs the gdb binary and setting the
RPATH is difficult for gdb, this switches to building
gdb with the OS compilers / libstdc++. That avoids the
need to fix the RPATH.

Testing:
 - Ran build, downloaded the binary on Centos 7, verified
   it ran properly

Change-Id: I0b794f3f28e3932c08ced0952bc1ba342c3dc0f6
---
M buildall.sh
M functions.sh
M init-compiler.sh
3 files changed, 16 insertions(+), 13 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/native-toolchain 
refs/changes/22/19022/1
--
To view, visit http://gerrit.cloudera.org:8080/19022
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0b794f3f28e3932c08ced0952bc1ba342c3dc0f6
Gerrit-Change-Number: 19022
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell

[Impala-ASF-CR] IMPALA-11418: A statement that returns at most one row need not to spool results

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18711 )

Change subject: IMPALA-11418: A statement that returns at most one row need not 
to spool results
..


Patch Set 12:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11394/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18711
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icd4d73c21106048df68a270cf03d4abd56bd3aac
Gerrit-Change-Number: 18711
Gerrit-PatchSet: 12
Gerrit-Owner: Xianqing He 
Gerrit-Reviewer: Aman Sinha 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Reviewer: Xianqing He 
Gerrit-Comment-Date: Wed, 21 Sep 2022 03:16:15 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11555: external Udf support decimal(, )

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19021 )

Change subject: IMPALA-11555: external Udf support decimal(*, *)
..


Patch Set 3:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11393/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19021
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09f19da5b1660ed0b882497fc321e6cc1b9b55d0
Gerrit-Change-Number: 19021
Gerrit-PatchSet: 3
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 21 Sep 2022 03:14:05 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11555: external Udf support decimal(, )

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19021 )

Change subject: IMPALA-11555: external Udf support decimal(*, *)
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11392/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19021
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09f19da5b1660ed0b882497fc321e6cc1b9b55d0
Gerrit-Change-Number: 19021
Gerrit-PatchSet: 2
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 21 Sep 2022 03:02:14 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11418: A statement that returns at most one row need not to spool results

2022-09-20 Thread Xianqing He (Code Review)

Xianqing He has uploaded a new patch set (#12). ( 
http://gerrit.cloudera.org:8080/18711 )

Change subject: IMPALA-11418: A statement that returns at most one row need not 
to spool results
..

IMPALA-11418: A statement that returns at most one row need not to spool results

A query that returns at most one row can run more efficiently without
result spooling. If result spooling is enabled, it will set the
minimum memory reservation in PlanRootSink, e.g. 'select 1' minimum
memory reservation is 4MB.

This optimization can reduce the statement's resource reservation and
the exception 'Failed to get minimum memory reservation' when the host
memory limit not available.

Testing:
- Add tests in result-spooling.test

Change-Id: Icd4d73c21106048df68a270cf03d4abd56bd3aac
---
M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M 
testdata/workloads/functional-planner/queries/PlannerTest/bloom-filter-assignment.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters-hdfs-num-rows-est-enabled.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering-disabled.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
M testdata/workloads/functional-planner/queries/PlannerTest/result-spooling.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/runtime-filter-query-options.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q13.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q16.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q23a.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q32.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q38.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q48.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q87.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q92.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q94.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q95.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q96.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q97.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpch-all.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpch-kudu.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpch-nested.test
M 
testdata/workloads/functional-query/queries/QueryTest/admission-max-min-mem-limits.test
M 
testdata/workloads/functional-query/queries/QueryTest/dedicated-coord-mem-estimates.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test
M tests/query_test/test_observability.py
31 files changed, 313 insertions(+), 197 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/11/18711/12
--
To view, visit http://gerrit.cloudera.org:8080/18711
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Icd4d73c21106048df68a270cf03d4abd56bd3aac
Gerrit-Change-Number: 18711
Gerrit-PatchSet: 12
Gerrit-Owner: Xianqing He 
Gerrit-Reviewer: Aman Sinha 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Reviewer: Xianqing He

[Impala-ASF-CR] IMPALA-11555: external Udf support decimal(, )

Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19021

to look at the new patch set (#3).

Change subject: IMPALA-11555: external Udf support decimal(*, *)
..

IMPALA-11555: external Udf support decimal(*, *)

Increase syntax parsing and remove some restrictions.

Change-Id: I09f19da5b1660ed0b882497fc321e6cc1b9b55d0
---
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java
M fe/src/main/java/org/apache/impala/catalog/ScalarType.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
4 files changed, 17 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/19021/3
--
To view, visit http://gerrit.cloudera.org:8080/19021
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I09f19da5b1660ed0b882497fc321e6cc1b9b55d0
Gerrit-Change-Number: 19021
Gerrit-PatchSet: 3
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] IMPALA-11418: A statement that returns at most one row need not to spool results

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18711 )

Change subject: IMPALA-11418: A statement that returns at most one row need not 
to spool results
..


Patch Set 12:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8594/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/18711
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icd4d73c21106048df68a270cf03d4abd56bd3aac
Gerrit-Change-Number: 18711
Gerrit-PatchSet: 12
Gerrit-Owner: Xianqing He 
Gerrit-Reviewer: Aman Sinha 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Reviewer: Xianqing He 
Gerrit-Comment-Date: Wed, 21 Sep 2022 02:55:10 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11555: external Udf support decimal(, )

Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19021

to look at the new patch set (#2).

Change subject: IMPALA-11555: external Udf support decimal(*, *)
..

IMPALA-11555: external Udf support decimal(*, *)

Increase syntax parsing and remove some restrictions.

Change-Id: I09f19da5b1660ed0b882497fc321e6cc1b9b55d0
---
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java
M fe/src/main/java/org/apache/impala/catalog/ScalarType.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
4 files changed, 17 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/19021/2
--
To view, visit http://gerrit.cloudera.org:8080/19021
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I09f19da5b1660ed0b882497fc321e6cc1b9b55d0
Gerrit-Change-Number: 19021
Gerrit-PatchSet: 2
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] IMPALA-11555: external Udf support decimal(, )

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19021 )

Change subject: IMPALA-11555: external Udf support decimal(*, *)
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19021/2/fe/src/test/java/org/apache/impala/analysis/ParserTest.java
File fe/src/test/java/org/apache/impala/analysis/ParserTest.java:

http://gerrit.cloudera.org:8080/#/c/19021/2/fe/src/test/java/org/apache/impala/analysis/ParserTest.java@2255
PS2, Line 2255: ParsesOk("CREATE AGGREGATE FUNCTIOn Foo(TIMESTAMP, 
DECIMAL(*, *)) " +
line has trailing whitespace



--
To view, visit http://gerrit.cloudera.org:8080/19021
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09f19da5b1660ed0b882497fc321e6cc1b9b55d0
Gerrit-Change-Number: 19021
Gerrit-PatchSet: 2
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 21 Sep 2022 02:42:03 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11160: Ignore stale ALTER PARTITION events on transactional tables

2022-09-20 Thread Quanlong Huang (Code Review)

Quanlong Huang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19020 )

Change subject: IMPALA-11160: Ignore stale ALTER_PARTITION events on 
transactional tables
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19020/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19020/1//COMMIT_MSG@25
PS1, Line 25: Tests
> Thanks Quanlong for catching this. I agree with Csaba that we should add mo
That's a good point. Thank Csaba and Yu-Wen!



--
To view, visit http://gerrit.cloudera.org:8080/19020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5bb8cfc213093f3bbd0359c7084b277a3bd5264a
Gerrit-Change-Number: 19020
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Yu-Wen Lai 
Gerrit-Comment-Date: Tue, 20 Sep 2022 23:13:11 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11160: Ignore stale ALTER PARTITION events on transactional tables

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19020 )

Change subject: IMPALA-11160: Ignore stale ALTER_PARTITION events on 
transactional tables
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/19020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5bb8cfc213093f3bbd0359c7084b277a3bd5264a
Gerrit-Change-Number: 19020
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Yu-Wen Lai 
Gerrit-Comment-Date: Tue, 20 Sep 2022 23:13:54 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11160: Ignore stale ALTER PARTITION events on transactional tables

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19020 )

Change subject: IMPALA-11160: Ignore stale ALTER_PARTITION events on 
transactional tables
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8593/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/19020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5bb8cfc213093f3bbd0359c7084b277a3bd5264a
Gerrit-Change-Number: 19020
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Yu-Wen Lai 
Gerrit-Comment-Date: Tue, 20 Sep 2022 23:13:55 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11160: Ignore stale ALTER PARTITION events on transactional tables

2022-09-20 Thread Yu-Wen Lai (Code Review)

Yu-Wen Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19020 )

Change subject: IMPALA-11160: Ignore stale ALTER_PARTITION events on 
transactional tables
..


Patch Set 1: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19020/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19020/1//COMMIT_MSG@25
PS1, Line 25: Tests
> The solution looks good, but one thing bugs me: shouldn't the original bug
Thanks Quanlong for catching this. I agree with Csaba that we should add more 
tests around event processing. I just created a follow-up Jira IMPALA-11598.



--
To view, visit http://gerrit.cloudera.org:8080/19020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5bb8cfc213093f3bbd0359c7084b277a3bd5264a
Gerrit-Change-Number: 19020
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Yu-Wen Lai 
Gerrit-Comment-Date: Tue, 20 Sep 2022 17:10:39 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11583: Use Iceberg API to update stats

2022-09-20 Thread Zoltan Borok-Nagy (Code Review)

Zoltan Borok-Nagy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18995 )

Change subject: IMPALA-11583: Use Iceberg API to update stats
..


Patch Set 6:

(3 comments)

Thanks for the comments.

http://gerrit.cloudera.org:8080/#/c/18995/4/testdata/workloads/functional-query/queries/QueryTest/iceberg-compute-stats.test
File 
testdata/workloads/functional-query/queries/QueryTest/iceberg-compute-stats.test:

http://gerrit.cloudera.org:8080/#/c/18995/4/testdata/workloads/functional-query/queries/QueryTest/iceberg-compute-stats.test@77
PS4, Line 77: '','numRows ','2   '
> I am not sure which operation sets this property - can you unset it before
Added UNSET.
Also opened IMPALA-11597.


http://gerrit.cloudera.org:8080/#/c/18995/4/testdata/workloads/functional-query/queries/QueryTest/iceberg-compute-stats.test@86
PS4, Line 86: show column stats ice_alltypes;
:  RESULTS
> Shouldn't we deny this operation, or at least return a warning?
Users can set numRows for non-HMS integrated Iceberg tables.

I could also add this check probably to 
https://github.com/apache/impala/blob/cff286e7512e9d1e2ff2b4ea033d3e575f54b353/fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java#L153

But there is no complete list of Iceberg-managed table properties, so we will 
run into this repeatedly.


http://gerrit.cloudera.org:8080/#/c/18995/4/testdata/workloads/functional-query/queries/QueryTest/iceberg-compute-stats.test@111
PS4, Line 111: 'p_d
> Can you also add a drop stats statement?
Done



--
To view, visit http://gerrit.cloudera.org:8080/18995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I46b6e0a5a65e18e5aaf2a007ec0242b28e0fed92
Gerrit-Change-Number: 18995
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tamas Mate 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Tue, 20 Sep 2022 15:58:33 +
Gerrit-HasComments: Yes

[native-toolchain-CR] IMPALA-9999: Build the toolchain with GCC 10

Joe McDonnell has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/18806 )

Change subject: IMPALA-: Build the toolchain with GCC 10
..

IMPALA-: Build the toolchain with GCC 10

This switches the toolchain to build with GCC 10.
This required several version changes and patches
due to new warnings or changes in default settings:
1. LLVM is patched to fix compilation in lli
(type mismatch on char)
2. crcutil is upgraded to the latest to handle the
two digit GCC version. The new version incorporates
the two patches previously used.
3. libunwind is upgraded to 1.5.1 due to GCC 10
switch to -fno-common by default. The new
version incorporates a couple of the previous
patches.
4. Breakpad is upgraded to a recent commit (from late
March 2022), which uses a newer version of lss
that can compile with GCC 10.
5. Flatbuffers is upgraded to 1.12.0, which fixes
a constness casting issue
6. The TPC-DS library is patched to use -fcommon,
as GCC 10 switched to -fno-common by default.
7. GDB fails to compile on ARM due to the switch
to -fno-common, so this adds newer GDB versions
without that issue.

This upgrade GCC to 10.4 and binutils to 2.35.1.
This changes GCC's build command to build GCC using
link time optimization. This should speed up GCC's
execution, but it has no other impact. Building GCC
with LTO requires GCC compilation to use an updated
binutils, so this modifies the build script to build
binutils first and use that for GCC.

GCC 10.4 requires a patch to libstdc++ to fix missing
noexcepts, otherwise Clang compilation of Impala fails.

Testing:
- This has built on all supported platforms, and
Impala can build and operate with this toolchain.

Change-Id: I2fbfcdcb497ad60772bf0a7837e4043ae75bcfaa
Reviewed-on: http://gerrit.cloudera.org:8080/18806
Reviewed-by: Michael Smith
Reviewed-by: Wenzhe Zhou
Tested-by: Joe McDonnell
---
M buildall.sh
M functions.sh
M init-compiler.sh
M init.sh
A
source/breakpad/breakpad-e09741c609dcd5f5274d40182c5e2cc9a002d5ba-patches/0001-Add-basic-support-for-dwz-dwarf-extension.patch
A
source/breakpad/breakpad-e09741c609dcd5f5274d40182c5e2cc9a002d5ba-patches/0002-Build-breakpad-e09741c6-on-ppc64le.patch
M source/gcc/build.sh
A
source/gcc/gcc-10.4.0-patches/0001-libstdc-Fix-inconsistent-noexcept-specific-for-valar.patch
M source/gdb/build.sh
A
source/libunwind/libunwind-1.5.0-patches/0001-libunwind-trace-cache-destructor.patch
A source/llvm/llvm-5.0.1-patches/0005-PATCH-Fix-lli-compilation-on-gcc8.patch
A
source/tpc-ds/tpc-ds-2.1.0-patches/0001-PATCH-Fix-compilation-on-gcc10-fcommon.patch
12 files changed, 710 insertions(+), 31 deletions(-)

Approvals:
Michael Smith: Looks good to me, but someone else must approve
Wenzhe Zhou: Looks good to me, approved
Joe McDonnell: Verified

--
To view, visit http://gerrit.cloudera.org:8080/18806
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I2fbfcdcb497ad60772bf0a7837e4043ae75bcfaa
Gerrit-Change-Number: 18806
Gerrit-PatchSet: 6
Gerrit-Owner: Joe McDonnell
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Michael Smith
Gerrit-Reviewer: Wenzhe Zhou

[Impala-ASF-CR] IMPALA-9999: Switch to GCC 10.4

Joe McDonnell has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/18134 )

Change subject: IMPALA-: Switch to GCC 10.4
..

IMPALA-: Switch to GCC 10.4

This upgrades GCC and libstdc++ to version 10.4. This
required patching or upgrading several dependencies
so they could compile with GCC 10. The toolchain
companion change has details on what items needed
to be upgraded and why.

The toolchain companion change switches GCC to build
with toolchain binutils rather than host binutils. This
means that the python virtualenv initialization needs
to include binutils on the path.

This disables two warnings introduced in the new GCC
versions (Wclass-memaccess and Winit-list-lifetime).
These two warnings occur in our code and also in
dependencies like LLVM and rapidjson. These are not
critical warnings, so they can be addressed
independently and reenabled later.

Binary sizes increase, particulary when including
debug symbols:
 | GCC 7.5 | GCC 10.4
impalad RELEASE stripped |  83204768   |  88702824
impalad RELEASE  | 707278904   | 971711456
impalad DEBUG stripped   | 106677672   |  97391944
impalad DEBUG| 725864760   | 867647512

Testing:
 - Multiple test jobs (core, release exhaustive, ASAN)
 - Performance testing for TPC-H and TPC-DS shows
   a modest improvement (2-4%).
 - Code compiles without warnings on debug and release

Change-Id: Ibe6857b822925226d39fd4d6413457ef6bbaabec
Reviewed-on: http://gerrit.cloudera.org:8080/18134
Reviewed-by: Michael Smith 
Tested-by: Impala Public Jenkins 
Reviewed-by: Csaba Ringhofer 
---
M be/CMakeLists.txt
M be/src/codegen/llvm-codegen-test.cc
M be/src/runtime/string-value.inline.h
M be/src/udf/udf.cc
M be/src/util/parquet-reader.cc
M bin/impala-config.sh
M fe/pom.xml
M infra/python/bootstrap_virtualenv.py
8 files changed, 51 insertions(+), 20 deletions(-)

Approvals:
  Michael Smith: Looks good to me, but someone else must approve
  Impala Public Jenkins: Verified
  Csaba Ringhofer: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/18134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ibe6857b822925226d39fd4d6413457ef6bbaabec
Gerrit-Change-Number: 18134
Gerrit-PatchSet: 16
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Abhishek Rawat 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith

[native-toolchain-CR] IMPALA-9999: Build the toolchain with GCC 10

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18806 )

Change subject: IMPALA-: Build the toolchain with GCC 10
..


Patch Set 5: Verified+1

Build passes and Impala is able to build/run with the resulting toolchain.

Impala change here: https://gerrit.cloudera.org/#/c/18134/


--
To view, visit http://gerrit.cloudera.org:8080/18806
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2fbfcdcb497ad60772bf0a7837e4043ae75bcfaa
Gerrit-Change-Number: 18806
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Tue, 20 Sep 2022 15:49:45 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11583: Use Iceberg API to update stats

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18995 )

Change subject: IMPALA-11583: Use Iceberg API to update stats
..


Patch Set 6:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11391/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I46b6e0a5a65e18e5aaf2a007ec0242b28e0fed92
Gerrit-Change-Number: 18995
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tamas Mate 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Tue, 20 Sep 2022 15:30:22 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11583: Use Iceberg API to update stats

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18995 )

Change subject: IMPALA-11583: Use Iceberg API to update stats
..


Patch Set 5:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11390/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I46b6e0a5a65e18e5aaf2a007ec0242b28e0fed92
Gerrit-Change-Number: 18995
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tamas Mate 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Tue, 20 Sep 2022 15:20:15 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11583: Use Iceberg API to update stats

2022-09-20 Thread Zoltan Borok-Nagy (Code Review)

Hello Tamas Mate, Gergely Fürnstáhl, Csaba Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/18995

to look at the new patch set (#6).

Change subject: IMPALA-11583: Use Iceberg API to update stats
..

IMPALA-11583: Use Iceberg API to update stats

Before this patch we used HMS API alter_table() to update an Iceberg
table's statistics. 'alter_table()' API calls are unsafe for Iceberg
tables as they overwrite the whole HMS table, including the table
property 'metadata_location' which must always point to the latest
snapshot. Hence concurrent modification to the same table could be
reverted by COMPUTE STATS.

In this patch we are using Iceberg API to update Iceberg tables.
Also, table-level stats (e.g. numRows, totalSize, totalFiles) are not
set as Iceberg keeps them up-to-date.

DROP STATS has been also modified to not drop table-level stats for
HMS-integrated Iceberg tables.

Testing:
 * added e2e tests for COMPUTE STATS
 * added e2e tests for DROP STATS
 * manually tested concurrent Hive INSERT and Impala COMPUTE STATS
   using latest Hive
 * opened IMPALA-11590 to add automated interop tests with Hive

Change-Id: I46b6e0a5a65e18e5aaf2a007ec0242b28e0fed92
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
A 
testdata/workloads/functional-query/queries/QueryTest/iceberg-compute-stats.test
M tests/query_test/test_iceberg.py
3 files changed, 413 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/18995/6
--
To view, visit http://gerrit.cloudera.org:8080/18995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I46b6e0a5a65e18e5aaf2a007ec0242b28e0fed92
Gerrit-Change-Number: 18995
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tamas Mate 
Gerrit-Reviewer: Zoltan Borok-Nagy

[Impala-ASF-CR] IMPALA-10660: Impala shell prints DOUBLEs with less precision in HS2 than beeswax

2022-09-20 Thread Daniel Becker (Code Review)

Daniel Becker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18990 )

Change subject: IMPALA-10660: Impala shell prints DOUBLEs with less precision 
in HS2 than beeswax
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18990/5/shell/impala_shell.py
File shell/impala_shell.py:

http://gerrit.cloudera.org:8080/#/c/18990/5/shell/impala_shell.py@1939
PS5, Line 1939: format_str = "{:%s}" % format_specification
The user input was inserted after the ':' in Patch Set 3 as well, wasn't it? On 
the other hand I tend to agree that this is not a security issue, I'm fine with 
both solutions.

I don't think providing a huge value for 'width' (or other parameters) is a 
security issue, the user only slows down their own impala shell.



--
To view, visit http://gerrit.cloudera.org:8080/18990
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424339266be66437941be8bafaa83fa0f2dfbd4e
Gerrit-Change-Number: 18990
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Peter Rozsa 
Gerrit-Comment-Date: Tue, 20 Sep 2022 15:03:04 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11583: Use Iceberg API to update stats

2022-09-20 Thread Zoltan Borok-Nagy (Code Review)

Hello Tamas Mate, Gergely Fürnstáhl, Csaba Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/18995

to look at the new patch set (#5).

Change subject: IMPALA-11583: Use Iceberg API to update stats
..

IMPALA-11583: Use Iceberg API to update stats

Before this patch we used HMS API alter_table() to update an Iceberg
table's statistics. 'alter_table()' API calls are unsafe for Iceberg
tables as they overwrite the whole HMS table, including the table
property 'metadata_location' which must always point to the latest
snapshot. Hence concurrent modification to the same table could be
reverted by COMPUTE STATS.

In this patch we are using Iceberg API to update Iceberg tables.
Also, table-level stats (e.g. numRows, totalSize, totalFiles) are not
set as Iceberg keeps them up-to-date.

DROP STATS has been also modified to not drop table-level stats for
HMS-integrated Iceberg tables.

Testing:
 * added e2e tests for COMPUTE STATS
 * added e2e tests for DROP STATS
 * manually tested concurrent Hive INSERT and Impala COMPUTE STATS
   using latest Hive
 * opened IMPALA-11590 to add automated interop tests with Hive

Change-Id: I46b6e0a5a65e18e5aaf2a007ec0242b28e0fed92
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
A 
testdata/workloads/functional-query/queries/QueryTest/iceberg-compute-stats.test
M tests/query_test/test_iceberg.py
3 files changed, 397 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/18995/5
--
To view, visit http://gerrit.cloudera.org:8080/18995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I46b6e0a5a65e18e5aaf2a007ec0242b28e0fed92
Gerrit-Change-Number: 18995
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tamas Mate 
Gerrit-Reviewer: Zoltan Borok-Nagy

[Impala-ASF-CR] IMPALA-10660: Impala shell prints DOUBLEs with less precision in HS2 than beeswax

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18990 )

Change subject: IMPALA-10660: Impala shell prints DOUBLEs with less precision 
in HS2 than beeswax
..


Patch Set 5:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11389/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18990
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424339266be66437941be8bafaa83fa0f2dfbd4e
Gerrit-Change-Number: 18990
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Peter Rozsa 
Gerrit-Comment-Date: Tue, 20 Sep 2022 15:02:59 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7942 (part 1): Add query hints for table cardinalities

2022-09-20 Thread Qifan Chen (Code Review)

Qifan Chen has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18829 )

Change subject: IMPALA-7942 (part 1): Add query hints for table cardinalities
..


Patch Set 9:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18829/5/fe/src/main/java/org/apache/impala/analysis/TableRef.java
File fe/src/main/java/org/apache/impala/analysis/TableRef.java:

http://gerrit.cloudera.org:8080/#/c/18829/5/fe/src/main/java/org/apache/impala/analysis/TableRef.java@176
PS5, Line 176: tableNumRowsHint_
> Thanks for suggestion, Qifan.
Yeah, the logic to clean up the hash table could be a little bit complicated. I 
wonder if the timing to do can be after the analyze(), here.

https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java#L472

In addition, hash table lookup and throw exception can be done inside method 
analyzeHint().
https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java#L70

Hash table population can be done in the constructor for TableRef.



--
To view, visit http://gerrit.cloudera.org:8080/18829
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9f0c773f4e67782a1428db64062f68afbd257af7
Gerrit-Change-Number: 18829
Gerrit-PatchSet: 9
Gerrit-Owner: wangsheng 
Gerrit-Reviewer: Aman Sinha 
Gerrit-Reviewer: Fang-Yu Rao 
Gerrit-Reviewer: Fucun Chu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Qifan Chen 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: wangsheng 
Gerrit-Comment-Date: Tue, 20 Sep 2022 14:57:01 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10660: Impala shell prints DOUBLEs with less precision in HS2 than beeswax

Hello Daniel Becker, Csaba Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/18990

to look at the new patch set (#5).

Change subject: IMPALA-10660: Impala shell prints DOUBLEs with less precision 
in HS2 than beeswax
..

IMPALA-10660: Impala shell prints DOUBLEs with less precision in HS2 than 
beeswax

This change adds a shell option called "fp_format_specification"
which manipulates the print format of floating-point values.
It lets the user to specify a Python-based format specification
expression (https://docs.python.org/2.7/library/string.html#formatspec)
which will get parsed and applied to floating-point
column values. The default value is None, in this case the
formatting is the same as the state before this change.
This option does not support the beeswax protocol.

Tests: command line tests for various formatting options and
   for invalid formatting option

Change-Id: I424339266be66437941be8bafaa83fa0f2dfbd4e
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/make_shell_tarball.sh
M shell/option_parser.py
M shell/packaging/make_python_package.sh
A shell/value_converter.py
M tests/shell/test_shell_commandline.py
8 files changed, 280 insertions(+), 44 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/18990/5
--
To view, visit http://gerrit.cloudera.org:8080/18990
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I424339266be66437941be8bafaa83fa0f2dfbd4e
Gerrit-Change-Number: 18990
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Peter Rozsa

[Impala-ASF-CR] IMPALA-10660: Impala shell prints DOUBLEs with less precision in HS2 than beeswax

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18990 )

Change subject: IMPALA-10660: Impala shell prints DOUBLEs with less precision 
in HS2 than beeswax
..


Patch Set 4:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11388/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18990
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424339266be66437941be8bafaa83fa0f2dfbd4e
Gerrit-Change-Number: 18990
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Peter Rozsa 
Gerrit-Comment-Date: Tue, 20 Sep 2022 14:25:18 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10660: Impala shell prints DOUBLEs with less precision in HS2 than beeswax

Peter Rozsa has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18990 )

Change subject: IMPALA-10660: Impala shell prints DOUBLEs with less precision 
in HS2 than beeswax
..


Patch Set 3:

(6 comments)

> (3 comments)
 >
 > Are there any security implications of accepting user input as
 > format strings? Can we say it is completely safe?

I changed the format parsing to a more strict approach: now the user input is 
interpreted with a ":" prefix, which forces it to be a format specification 
(https://docs.python.org/3/library/string.html#formatspec). Now, the remaining 
specification has a limited set of options:
- fill: fills the remaining width with the given character, can make 
numerically invalid string, eg.: 1000.0 but they are printable, unicode 
characters are not allowed
- align: 4 options, cannot enter invalid option
- sign: 3 options, same as align
- width: arbitrary number of digits, can make the printing slow with huge 
values, for example width of 1 is printing for 1 minute
- grouping_option 2 options, same as align and sign
- precision: arbitrary number of digits, same as width
- type: 8 options, same as the previous non-terminals with closed option set

The only concern that I can see is the arbitrary number of digits for width and 
precision, it can slow down the shell to never give back result. There's no 
option that can accept named fields and identifiers, these are restricted, 
because these elements must precede the ":" token.

http://gerrit.cloudera.org:8080/#/c/18990/2/shell/impala_client.py
File shell/impala_client.py:

http://gerrit.cloudera.org:8080/#/c/18990/2/shell/impala_client.py@63
PS2, Line 63: TTypeId.SMALLINT_TYPE: operator.attrgetter('i16Val'),
> Update the comment section above, as this has changed.
Done


http://gerrit.cloudera.org:8080/#/c/18990/2/shell/option_parser.py
File shell/option_parser.py:

http://gerrit.cloudera.org:8080/#/c/18990/2/shell/option_parser.py@324
PS2, Line 324: hs2_fp_format", type="s
> It would be nice to contain the info in the name that this only applies to
Done


http://gerrit.cloudera.org:8080/#/c/18990/2/shell/option_parser.py@326
PS2, Line 326: help
> Can you add info also about the default Python behavior and the value neede
Done


http://gerrit.cloudera.org:8080/#/c/18990/2/shell/value_converter.py
File shell/value_converter.py:

http://gerrit.cloudera.org:8080/#/c/18990/2/shell/value_converter.py@32
PS2, Line 32: def __init__(self):
> Is it intentional that FLOAT and DOUBLE are not in this map? Don't we need
FLOAT and DOUBLE added back
Other, non-numeric types are extracted as string so no conversion is needed.


http://gerrit.cloudera.org:8080/#/c/18990/2/shell/value_converter.py@32
PS2, Line 32: def __init__(self):
> Is it intentional that it is a class (static) variable? I think it would be
Done


http://gerrit.cloudera.org:8080/#/c/18990/2/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:

http://gerrit.cloudera.org:8080/#/c/18990/2/tests/shell/test_shell_commandline.py@1312
PS2, Line 1312: mat
> The value was chosen to match with beeswax, right? I think that we could sk
Done



--
To view, visit http://gerrit.cloudera.org:8080/18990
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424339266be66437941be8bafaa83fa0f2dfbd4e
Gerrit-Change-Number: 18990
Gerrit-PatchSet: 3
Gerrit-Owner: Peter Rozsa 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Peter Rozsa 
Gerrit-Comment-Date: Tue, 20 Sep 2022 14:04:33 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10660: Impala shell prints DOUBLEs with less precision in HS2 than beeswax

Hello Daniel Becker, Csaba Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/18990

to look at the new patch set (#4).

Change subject: IMPALA-10660: Impala shell prints DOUBLEs with less precision 
in HS2 than beeswax
..

IMPALA-10660: Impala shell prints DOUBLEs with less precision in HS2 than 
beeswax

This change adds a shell option called "fp_format_specification"
which manipulates the print format of floating-point values.
It lets the user to specify a Python-based format specification
expression (https://docs.python.org/2.7/library/string.html#formatspec)
which will get parsed and applied to floating-point
column values. The default value is None, in this case the
formatting is the same as the state before this change.
This option does not support the beeswax protocol.

Tests: command line tests for various formatting options and
   for invalid formatting option

Change-Id: I424339266be66437941be8bafaa83fa0f2dfbd4e
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/make_shell_tarball.sh
M shell/option_parser.py
M shell/packaging/make_python_package.sh
A shell/value_converter.py
M tests/shell/test_shell_commandline.py
8 files changed, 281 insertions(+), 44 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/18990/4
--
To view, visit http://gerrit.cloudera.org:8080/18990
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I424339266be66437941be8bafaa83fa0f2dfbd4e
Gerrit-Change-Number: 18990
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] IMPALA-10660: Impala shell prints DOUBLEs with less precision in HS2 than beeswax

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18990 )

Change subject: IMPALA-10660: Impala shell prints DOUBLEs with less precision 
in HS2 than beeswax
..


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18990/4/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:

http://gerrit.cloudera.org:8080/#/c/18990/4/tests/shell/test_shell_commandline.py@1425
PS4, Line 1425: d
flake8: E303 too many blank lines (2)



--
To view, visit http://gerrit.cloudera.org:8080/18990
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424339266be66437941be8bafaa83fa0f2dfbd4e
Gerrit-Change-Number: 18990
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Peter Rozsa 
Gerrit-Comment-Date: Tue, 20 Sep 2022 14:05:25 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10660: Impala shell prints DOUBLEs with less precision in HS2 than beeswax

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18990 )

Change subject: IMPALA-10660: Impala shell prints DOUBLEs with less precision 
in HS2 than beeswax
..


Patch Set 3:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11387/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18990
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424339266be66437941be8bafaa83fa0f2dfbd4e
Gerrit-Change-Number: 18990
Gerrit-PatchSet: 3
Gerrit-Owner: Peter Rozsa 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 20 Sep 2022 10:20:09 +
Gerrit-HasComments: No

[Impala-ASF-CR](branch-4.1.1) IMPALA-11528: Catalogd should start up with a corrupt Hive function.

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19019 )

Change subject: IMPALA-11528: Catalogd should start up with a corrupt Hive 
function.
..


Patch Set 4: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/19019
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: branch-4.1.1
Gerrit-MessageType: comment
Gerrit-Change-Id: I54e7a1df6d018ba6cf5ecf32dc9946edf86e2112
Gerrit-Change-Number: 19019
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Steve Carlin 
Gerrit-Reviewer: Tamas Mate 
Gerrit-Comment-Date: Tue, 20 Sep 2022 10:07:08 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10660: Impala shell prints DOUBLEs with less precision in HS2 than beeswax

Hello Daniel Becker, Csaba Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/18990

to look at the new patch set (#3).

Change subject: IMPALA-10660: Impala shell prints DOUBLEs with less precision 
in HS2 than beeswax
..

IMPALA-10660: Impala shell prints DOUBLEs with less precision in HS2 than 
beeswax

This change adds a shell option called "fp_format_specification"
which manipulates the print format of floating-point values.
It lets the user to specify a Python-based format specification
expression (https://docs.python.org/2.7/library/string.html#formatspec)
which will get parsed and applied to floating-point
column values. The default value is None, in this case the
formatting is the same as the state before this change.
This option does not support the beeswax protocol.

Tests: command line tests for various formatting options and
   for invalid formatting option

Change-Id: I424339266be66437941be8bafaa83fa0f2dfbd4e
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/make_shell_tarball.sh
M shell/option_parser.py
M shell/packaging/make_python_package.sh
A shell/value_converter.py
M tests/shell/test_shell_commandline.py
8 files changed, 271 insertions(+), 44 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/18990/3
--
To view, visit http://gerrit.cloudera.org:8080/18990
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I424339266be66437941be8bafaa83fa0f2dfbd4e
Gerrit-Change-Number: 18990
Gerrit-PatchSet: 3
Gerrit-Owner: Peter Rozsa 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Daniel Becker 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] IMPALA-7942 (part 1): Add query hints for table cardinalities

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18829 )

Change subject: IMPALA-7942 (part 1): Add query hints for table cardinalities
..


Patch Set 9:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11386/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18829
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9f0c773f4e67782a1428db64062f68afbd257af7
Gerrit-Change-Number: 18829
Gerrit-PatchSet: 9
Gerrit-Owner: wangsheng 
Gerrit-Reviewer: Aman Sinha 
Gerrit-Reviewer: Fang-Yu Rao 
Gerrit-Reviewer: Fucun Chu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Qifan Chen 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: wangsheng 
Gerrit-Comment-Date: Tue, 20 Sep 2022 08:54:59 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7942 (part 1): Add query hints for table cardinalities

2022-09-20 Thread wangsheng (Code Review)

wangsheng has uploaded a new patch set (#9). ( 
http://gerrit.cloudera.org:8080/18829 )

Change subject: IMPALA-7942 (part 1): Add query hints for table cardinalities
..

IMPALA-7942 (part 1): Add query hints for table cardinalities

Currently, we run 'COMPUTE STATS' command to compute table stats
which is very useful for query planning. Without these stats, a
query plan may not be optimal. However, these stats may not be
available, up to date, or valid. To workaround this problem,
this patch adds a new query hint: 'TABLE_NUM_ROWS', We can use
this new hint after a hdfs or kudu table in query like this:

  * select col from t /* +TABLE_NUM_ROWS(1000) */;

If set, Impala will use this value as table scanned rows when
table no stats or has corrput stats. This hint value will not
valid if table stats is normal.

Testing:
- Added new fe test in 'PlannerTest'
- Added new fe test in 'AnalyzeStmtsTest' for negative cases

Change-Id: I9f0c773f4e67782a1428db64062f68afbd257af7
---
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/ScanNode.java
M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
A 
testdata/workloads/functional-planner/queries/PlannerTest/table-cardinality-hint.test
8 files changed, 146 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18829/9
--
To view, visit http://gerrit.cloudera.org:8080/18829
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9f0c773f4e67782a1428db64062f68afbd257af7
Gerrit-Change-Number: 18829
Gerrit-PatchSet: 9
Gerrit-Owner: wangsheng 
Gerrit-Reviewer: Aman Sinha 
Gerrit-Reviewer: Fang-Yu Rao 
Gerrit-Reviewer: Fucun Chu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Qifan Chen 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: wangsheng

[Impala-ASF-CR] IMPALA-7942 (part 1): Add query hints for table cardinalities

2022-09-20 Thread wangsheng (Code Review)

wangsheng has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18829 )

Change subject: IMPALA-7942 (part 1): Add query hints for table cardinalities
..


Patch Set 8:

(2 comments)

I found that when executing test, the cardinality of 
functional_parquet.alltypes seems different in each jenkins test.
Refer to: 
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/17575/testReport/junit/org.apache.impala.planner/PlannerTest/testTableCardinalityHint/
Besides, I found that cardinality of functional_parquet.alltypes are different 
in test file, such as: mt-dop-validation.test, parquet-filtering.test, 
parquet-stats-agg.test. Since these test cases not use VALIDATE_CARDINALITY.
But this seems not happen to functional.alltypes, why?

http://gerrit.cloudera.org:8080/#/c/18829/5/fe/src/main/java/org/apache/impala/analysis/TableRef.java
File fe/src/main/java/org/apache/impala/analysis/TableRef.java:

http://gerrit.cloudera.org:8080/#/c/18829/5/fe/src/main/java/org/apache/impala/analysis/TableRef.java@176
PS5, Line 176: tableNumRowsHint_
> IMHO, that the table references are in different scope should not matter, a
Thanks for suggestion, Qifan.
I try this may, but find a problem. We can add a static map to reserve each 
involved table and related row hint. Bug when do we clean this map?
If we do not clean this map, when submit same query second time, this map 
already contains involved table in first submit and not been clean. Then second 
submit will invalid rows hint.
We need to clean this map after sql parse complete, but it seem that we don't 
know when does sql parse end in TableRef.java


http://gerrit.cloudera.org:8080/#/c/18829/6/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java:

http://gerrit.cloudera.org:8080/#/c/18829/6/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java@5046
PS6, Line 5046: Syntax error in line 1
> Sure. Your current code and the test are good.
Done



--
To view, visit http://gerrit.cloudera.org:8080/18829
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9f0c773f4e67782a1428db64062f68afbd257af7
Gerrit-Change-Number: 18829
Gerrit-PatchSet: 8
Gerrit-Owner: wangsheng 
Gerrit-Reviewer: Aman Sinha 
Gerrit-Reviewer: Fang-Yu Rao 
Gerrit-Reviewer: Fucun Chu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Qifan Chen 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: wangsheng 
Gerrit-Comment-Date: Tue, 20 Sep 2022 08:26:39 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11555: external Udf support decimal(, )

2022-09-20 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19021 )

Change subject: IMPALA-11555: external Udf support decimal(*, *)
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/11385/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/19021
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09f19da5b1660ed0b882497fc321e6cc1b9b55d0
Gerrit-Change-Number: 19021
Gerrit-PatchSet: 1
Gerrit-Owner: Minghui Zhu 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 20 Sep 2022 08:08:50 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11555: external Udf support decimal(, )