zabetak commented on code in PR #6523:
URL: https://github.com/apache/hive/pull/6523#discussion_r3830285452
##########
ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out:
##########
@@ -1885,37 +1885,40 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart_double_hour_n0
- filterExpr: ((UDFToDouble(hour) = 11.0D) and
UDFToDouble(UDFToInteger((hr / 2.0D))) is not null) (type: boolean)
+ filterExpr: ((UDFToDouble(hour) = 11.0D) and hr is not null)
(type: boolean)
Statistics: Num rows: 2 Data size: 188 Basic stats: COMPLETE
Column stats: COMPLETE
Filter Operator
- predicate: ((UDFToDouble(hour) = 11.0D) and
UDFToDouble(UDFToInteger((hr / 2.0D))) is not null) (type: boolean)
+ predicate: ((UDFToDouble(hour) = 11.0D) and hr is not
null) (type: boolean)
Statistics: Num rows: 1 Data size: 94 Basic stats:
COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToDouble(UDFToInteger((hr / 2.0D)))
(type: double)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats:
COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: double)
- null sort order: z
- sort order: +
- Map-reduce partition columns: _col0 (type: double)
+ Filter Operator
+ predicate: _col0 is not null (type: boolean)
Review Comment:
The plans are roughly the same so let's not worry too much about it. If it
ever becomes a problem then we can look more into it.
##########
iceberg/iceberg-handler/src/test/results/positive/dynamic_partition_pruning.q.out:
##########
@@ -1458,37 +1458,40 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart_double_hour_n0
- filterExpr: ((UDFToDouble(hour) = 11.0D) and CAST(
UDFToInteger((hr / 2.0D)) AS STRING) is not null) (type: boolean)
+ filterExpr: ((UDFToDouble(hour) = 11.0D) and hr is not null)
(type: boolean)
Statistics: Num rows: 2 Data size: 188 Basic stats: COMPLETE
Column stats: COMPLETE
Filter Operator
- predicate: ((UDFToDouble(hour) = 11.0D) and CAST(
UDFToInteger((hr / 2.0D)) AS STRING) is not null) (type: boolean)
+ predicate: ((UDFToDouble(hour) = 11.0D) and hr is not
null) (type: boolean)
Review Comment:
I am not sure if point 1 and point 3 are fully distinct. I suspect that both
are due to different simplifications.
##########
ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out:
##########
@@ -581,10 +581,10 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- filterExpr: CAST( ds AS DATE) is not null (type: boolean)
+ filterExpr: day(CAST( ds AS DATE)) is not null (type:
boolean)
Review Comment:
I also checked the Calcite codebase and indeed its risky to infer the safety
of the operator based on the `SqlKind.EXTRACT`. In general, I am thinking that
deriving safety from the `SqlKind` is brittle and in most if not all cases we
should use the `SqlOperator` interface.
We could possibly override the
`org.apache.calcite.sql.fun.SqlExtractFunction#isSafeOperator` but this won't
help much since the implementation in Hive is not extending this class. A HIVE
Jira ticket is probably the best path forward. Please create one once you get a
chance.
##########
ql/src/test/results/clientpositive/llap/input8.q.out:
##########
@@ -36,14 +36,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 25 Data size: 191 Basic stats:
COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats:
COMPLETE Column stats: COMPLETE
Select Operator
- expressions: null (type: string), null (type: int), null
(type: double)
+ expressions: null (type: string),
UDFToInteger((UDFToDouble(key) - null)) (type: int), null (type: double)
Review Comment:
Once a follow-up ticket is created mention it here and we can mark the
discussion as resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
