rubenada commented on code in PR #6523:
URL: https://github.com/apache/hive/pull/6523#discussion_r3823186935
##########
ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out:
##########
@@ -1885,37 +1885,40 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart_double_hour_n0
- filterExpr: ((UDFToDouble(hour) = 11.0D) and
UDFToDouble(UDFToInteger((hr / 2.0D))) is not null) (type: boolean)
+ filterExpr: ((UDFToDouble(hour) = 11.0D) and hr is not null)
(type: boolean)
Statistics: Num rows: 2 Data size: 188 Basic stats: COMPLETE
Column stats: COMPLETE
Filter Operator
- predicate: ((UDFToDouble(hour) = 11.0D) and
UDFToDouble(UDFToInteger((hr / 2.0D))) is not null) (type: boolean)
+ predicate: ((UDFToDouble(hour) = 11.0D) and hr is not
null) (type: boolean)
Statistics: Num rows: 1 Data size: 94 Basic stats:
COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToDouble(UDFToInteger((hr / 2.0D)))
(type: double)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats:
COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: double)
- null sort order: z
- sort order: +
- Map-reduce partition columns: _col0 (type: double)
+ Filter Operator
+ predicate: _col0 is not null (type: boolean)
Review Comment:
@zabetak According to Claude, a possible explanation for this behavior would
be in HiveFilterProjectTransposeRule. Before the upgrade, the predicate was
pushed down, and at the end of the rule, it was detected as redundant (it was
already down the tree) so it was not placed on the new Project.
With the upgrade, when the predicate is pushed down, it gets combined with
the other existing predicate, simplification kicks in, and the new Calcite code
simplifies it `UDFToDouble(UDFToInteger((hr / 2.0D))) is not null` ==> `hr is
not null`; but then, when the isRedundant check of the rule is done, it cannot
find the predicate (because it has been simplified, so it is "hidden" under
another form), so the "old" predicate (`UDFToDouble(UDFToInteger((hr / 2.0D)))
is not null`) is placed also on the final Project.
Could this be a possible explanation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
