[ https://issues.apache.org/jira/browse/HIVE-25766?focusedWorklogId=689891&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-689891 ]
ASF GitHub Bot logged work on HIVE-25766: ----------------------------------------- Author: ASF GitHub Bot Created on: 03/Dec/21 10:33 Start Date: 03/Dec/21 10:33 Worklog Time Spent: 10m Work Description: kgyrtkirk commented on a change in pull request #2839: URL: https://github.com/apache/hive/pull/2839#discussion_r761820492 ########## File path: ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFilterProjectTransposeRule.java ########## @@ -339,6 +339,11 @@ private void check(Filter filter) { final RexNode filterCondition = simplify.simplify(filter.getCondition()); final Set<Integer> inputRefs = HiveCalciteUtil.getInputRefs(newCondition); + // if the new IS NOT NULL has no input ref, there is redundancy here, bail out + if (inputRefs.isEmpty()) { Review comment: there are a couple of ways to fix things like this: * we could call `RexSimplify` everywhere - but I don't really like that idea as it exposes unneccessary complexity - at places where a higher level objective is targeted * use the `RelBuilder` which is usually calling the simplifier here and there * use a rule which runs simplifiaction - it could be that these rules should be accompanied by `ReduceExpressionsRule` ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 689891) Time Spent: 1h 10m (was: 1h) > java.util.NoSuchElementException in HiveFilterProjectTransposeRule if > predicate has no InputRef > ----------------------------------------------------------------------------------------------- > > Key: HIVE-25766 > URL: https://issues.apache.org/jira/browse/HIVE-25766 > Project: Hive > Issue Type: Bug > Components: CBO, Query Planning > Affects Versions: 4.0.0 > Reporter: Alessandro Solimando > Assignee: Alessandro Solimando > Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > The issue can be reproduced with the following query: > {code:java} > create table test1 (s string); > create table test2 (m string); > EXPLAIN CBO SELECT c.m > FROM ( > SELECT substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) as m > FROM test1 > WHERE substr(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), 1, 1) = '2') c > JOIN test2 d ON c.m = d.m; > {code} > It fails with the following exception: > {noformat} > java.util.NoSuchElementException > at java.util.HashMap$HashIterator.nextNode(HashMap.java:1447) > at java.util.HashMap$KeyIterator.next(HashMap.java:1469) > at > org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.check(HiveFilterProjectTransposeRule.java:348) > at > org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:306) > at > org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:303) > at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72) > at org.apache.calcite.rel.RelVisitor.visit(RelVisitor.java:44) > at > org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule$RedundancyChecker.visit(HiveFilterProjectTransposeRule.java:316) > at org.apache.calcite.rel.RelVisitor.go(RelVisitor.java:61) > at > org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.isRedundantIsNotNull(HiveFilterProjectTransposeRule.java:276) > at > org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveFilterProjectTransposeRule.onMatch(HiveFilterProjectTransposeRule.java:191){noformat} > The current implementation, while checking if the predicate to be transposed > is redundant or not, it expects at least one InputRef, but the predicate can > have none as in this case. -- This message was sent by Atlassian Jira (v8.20.1#820001)