[
https://issues.apache.org/jira/browse/HIVE-25766?focusedWorklogId=690131&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-690131
]
ASF GitHub Bot logged work on HIVE-25766:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Dec/21 16:13
Start Date: 03/Dec/21 16:13
Worklog Time Spent: 10m
Work Description: asolimando commented on a change in pull request #2839:
URL: https://github.com/apache/hive/pull/2839#discussion_r762066925
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFilterProjectTransposeRule.java
##########
@@ -339,6 +339,11 @@ private void check(Filter filter) {
final RexNode filterCondition = simplify.simplify(filter.getCondition());
final Set<Integer> inputRefs =
HiveCalciteUtil.getInputRefs(newCondition);
+ // if the new IS NOT NULL has no input ref, there is redundancy here,
bail out
+ if (inputRefs.isEmpty()) {
+ return;
Review comment:
As discussed offline, `HiveReduceExpressionsRule` is applied already but
it cannot simplify those cases, for instance:
```
[ReduceExpressionsRule(Filter)] to
[rel#75:HiveFilter.HIVE.[].any(input=HepRelVertex#21,condition=AND(IS NOT
NULL($0), IS NOT NULL(CAST(TRUNC($0, _UTF-16LE'MONTH':VARCHAR(2147483647)
CHARACTER SET "UTF-16LE")):DATE), IS NOT NULL(CAST(TRUNC(CAST(TRUNC($0,
_UTF-16LE'MONTH':VARCHAR(2147483647) CHARACTER SET "UTF-16LE")):DATE,
_UTF-16LE'MONTH':VARCHAR(2147483647) CHARACTER SET "UTF-16LE")):DATE)))]
```
does not get simplified and the expression keeps growing.
[HIVE-25758](https://issues.apache.org/jira/browse/HIVE-25758)) is affected
by the same issue, but since the expression has no input ref in it, this fix
does not help. Once we get a more general solution for this "loop" that is
created, we will probably be able to remove or scope down what's provided for
[HIVE-25275](https://issues.apache.org/jira/browse/HIVE-25275).
Since the tickets are linked in Jira, I think we are good for now, we can
revisit it later upon finding a solution to the general problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 690131)
Time Spent: 1h 40m (was: 1.5h)
> java.util.NoSuchElementException in HiveFilterProjectTransposeRule if
> predicate has no InputRef
> -----------------------------------------------------------------------------------------------
>
> Key: HIVE-25766
> URL: https://issues.apache.org/jira/browse/HIVE-25766
> Project: Hive
> Issue Type: Bug
> Components: CBO, Query Planning
> Affects Versions: 4.0.0
> Reporter: Alessandro Solimando
> Assignee: Alessandro Solimando
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> The issue can be reproduced with the following query:
> {code:java}
> create table test1 (s string);
> create table test2 (m string);
> EXPLAIN CBO SELECT c.m
> FROM (
> SELECT substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) as m
> FROM test1
> WHERE substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) = '2') c
> JOIN test2 d ON c.m = d.m;
> {code}
> It fails with the following exception:
> {noformat}
> java.util.NoSuchElementException
> at java.util.HashMap$HashIterator.nextNode(HashMap.java:1447)
> at java.util.HashMap$KeyIterator.next(HashMap.java:1469)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.check(HiveFilterProjectTransposeRule.java:348)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:306)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:303)
> at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72)
> at org.apache.calcite.rel.RelVisitor.visit(RelVisitor.java:44)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:316)
> at org.apache.calcite.rel.RelVisitor.go(RelVisitor.java:61)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.isRedundantIsNotNull(HiveFilterProjectTransposeRule.java:276)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.onMatch(HiveFilterProjectTransposeRule.java:191){noformat}
> The current implementation, while checking if the predicate to be transposed
> is redundant or not, it expects at least one InputRef, but the predicate can
> have none as in this case.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
