[
https://issues.apache.org/jira/browse/HIVE-29688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18092682#comment-18092682
]
Stamatis Zampetakis commented on HIVE-29688:
--------------------------------------------
The proposed fix in PR#6572 led to some changes in subquery_in.q file for the
following query:
{code:sql}
select p_partkey from part where p_name in
(select p.p_name from part p left outer join part pp on p.p_type =
pp.p_type where pp.p_size = part.p_size and p.p_size=part.p_size)
{code}
The new query plan is less efficient since it has some additional joins but
remains correct since it returns the same result set. Avoiding crashes is more
important than performance so we decided to leave potential plan improvements
as a follow-up.
> IndexOutOfBoundsException when WHERE clause contains IN/EXISTS subqueries AND
> correlated conditions
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-29688
> URL: https://issues.apache.org/jira/browse/HIVE-29688
> Project: Hive
> Issue Type: Bug
> Reporter: Thomas Rebele
> Assignee: Thomas Rebele
> Priority: Major
> Labels: pull-request-available
>
> The following q file test fails with an exception:
> {code:java}
> drop table if exists `table1`;
> CREATE EXTERNAL TABLE `table1`(
> `f1` string,
> `f2` string,
> `f3` string,
> `f4` string,
> `f5` string,
> `f6` string)
> ;
> SELECT 1
> FROM table1 a
> WHERE a.f4 IN ('1', '2')
> AND EXISTS (
> SELECT 1
> FROM table1 b
> WHERE a.f6 = b.f1 AND b.f3 IN (SELECT 1)
> )
> ;
> {code}
> Steps to reproduce:
> {code}
> mvn clean install -DskipTests -Denforcer.skip=true -T 1C
> mvn test -pl ql,itests/qtest,itests/test-serde,itests/util -Pitests
> -Dtest=TestMiniLlapLocalCliDriver -Dqfile=decorrelate-semi-join.q
> -Dtest.output.overwrite=true -Denforcer.skip=true
> {code}
> The exception was:
> {code:java}
> java.lang.IndexOutOfBoundsException: Index 3 out of bounds for length 1
> at
> java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:100)
> at
> java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:106)
> at
> java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:302)
> at java.base/java.util.Objects.checkIndex(Objects.java:385)
> at java.base/java.util.ArrayList.get(ArrayList.java:427)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter$RexVisitor.visitInputRef(ASTConverter.java:853)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter$RexVisitor.visitInputRef(ASTConverter.java:808)
> at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:113)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter$RexVisitor.visitCall(ASTConverter.java:1107)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter$RexVisitor.visitCall(ASTConverter.java:808)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:189)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter.convert(ASTConverter.java:283)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter.convert(ASTConverter.java:136)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:605)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13230)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:476)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:359)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:109)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:499)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:451)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:415)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:409)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:234)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:203)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:129)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:430)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:358)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:790)
> at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:760)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:115)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:139)
> at
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:62)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
