[
https://issues.apache.org/jira/browse/PHOENIX-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15416498#comment-15416498
]
Saurabh Seth edited comment on PHOENIX-2976 at 8/11/16 4:00 AM:
----------------------------------------------------------------
In ScanUtil's getKey method, maxLength computation is incorrect. The issue
occurs when the the filter is being pushed down to the scan on the index table.
It can occur for other tables with a similar row key schema e.g. where there
are at least 2 fixed width data types followed by a non fixed width data type
in the row key.
Here's a slightly simpler test case without an index:
{code}
CREATE TABLE SCAN_UTIL_TEST (id1 SMALLINT NOT NULL, id2 SMALLINT NOT NULL, id3
DECIMAL NOT NULL CONSTRAINT pk PRIMARY KEY (id1, id2, id3));
SELECT id1, id2, id3 FROM SCAN_UTIL_TEST WHERE ((id1,id2) IN ((1,1), (2,2),
(3,3))) AND id3 > 10 limit 1;
{code}
I am attaching a patch that fixes this issue. There is some scope for
refactoring some of the code here since the same logic is replicated in getKey
and setKey methods and they both need to be in sync.
was (Author: saurabh.s...@gmail.com):
In ScanUtil's getKey method, maxLength computation is incorrect. The issue
occurs when the the filter is being pushed down to the scan on the index table.
It can occur for other tables with a similar row key schema e.g. where there
are at least 2 fixed width data types followed by a non fixed width data type
in the row key.
Here's a slightly simpler test case without an index:
CREATE TABLE SCAN_UTIL_TEST (id1 SMALLINT NOT NULL, id2 SMALLINT NOT NULL, id3
DECIMAL NOT NULL CONSTRAINT pk PRIMARY KEY (id1, id2, id3));
SELECT id1, id2, id3 FROM SCAN_UTIL_TEST WHERE ((id1,id2) IN ((1,1), (2,2),
(3,3))) AND id3 > 10 limit 1;
I am attaching a patch that fixes this issue. There is some scope for
refactoring some of the code here since the same logic is replicated in getKey
and setKey methods and they both need to be in sync.
> PKs included in index cause ArrayIndexOutOfBoundsException on ScanUtil
> ----------------------------------------------------------------------
>
> Key: PHOENIX-2976
> URL: https://issues.apache.org/jira/browse/PHOENIX-2976
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.8.0
> Reporter: Hoonmin Kim
> Assignee: Saurabh Seth
> Attachments: PHOENIX-2976.patch
>
>
> ScanUtil seems to throw an ArrayIndexOutOfBoundsException iif :
> - There are more than one PK column INCLUDED in the index.
> - And there's IN clause with PKs (indexed columns).
> - And there's a non-PK column comparison.
> I could avoid the exception by modifying the ScanUtil.java to check the array
> length and offset, but it's just a temp work.
> *To reproduce my problem:*
> Table & Index
> {code}
> CREATE TABLE INCLUDED_PK (id1 SMALLINT NOT NULL, id2 SMALLINT NOT NULL, tid1
> SMALLINT NOT NULL, liked SMALLINT CONSTRAINT pk PRIMARY KEY (id1, id2, tid1))
> CREATE INDEX INCLUDED_PK_INDEX ON INCLUDED_PK(id1 ASC, id2 ASC, liked ASC)
> INCLUDE (tid1)
> {code}
> Query throwing an exception
> {code}
> SELECT id1, id2, tid1, liked FROM INCLUDED_PK WHERE ((id1,id2) IN ((1,1),
> (2,2), (3,3))) AND liked > 10 limit 10
> {code}
> Testing the query with QueryPlanTest
> {code}
> 2016-06-09 14:18:44,589 DEBUG [main]
> org.apache.phoenix.util.ReadOnlyProps(317): Creating new ReadOnlyProps due to
> phoenix.query.dateFormat with yyyy-MM-dd HH:mm:ss.SSS!=yyyy-MM-dd
> 2016-06-09 14:18:44,597 DEBUG [main]
> org.apache.phoenix.jdbc.PhoenixStatement(1402): Execute query: EXPLAIN SELECT
> id1, id2, tid1, liked FROM INCLUDED_PK WHERE ((id1,id2) IN ((1,1), (2,2),
> (3,3))) AND liked > 10 limit 10
> 2016-06-09 14:18:44,601 DEBUG [main]
> org.apache.phoenix.compile.FromCompiler$BaseColumnResolver(568): Re-resolved
> stale table INCLUDED_PK with seqNum 0 at timestamp 0 with 4 columns: [ID1,
> ID2, TID1, 0.LIKED]
> 2016-06-09 14:18:44,683 DEBUG [main]
> org.apache.phoenix.compile.FromCompiler$BaseColumnResolver(568): Re-resolved
> stale table INCLUDED_PK_INDEX with seqNum 0 at timestamp 0 with 4 columns:
> [:ID1, :ID2, 0:LIKED, :TID1]
> 2016-06-09 14:18:44,684 DEBUG [main]
> org.apache.phoenix.compile.FromCompiler$BaseColumnResolver(568): Re-resolved
> stale table INCLUDED_PK with seqNum 0 at timestamp 0 with 4 columns: [ID1,
> ID2, TID1, 0.LIKED]
> java.lang.ArrayIndexOutOfBoundsException: 6
> at org.apache.phoenix.util.ScanUtil.setKey(ScanUtil.java:417)
> at org.apache.phoenix.util.ScanUtil.setKey(ScanUtil.java:348)
> at org.apache.phoenix.util.ScanUtil.getKey(ScanUtil.java:320)
> at org.apache.phoenix.util.ScanUtil.getMinKey(ScanUtil.java:293)
> at org.apache.phoenix.compile.ScanRanges.create(ScanRanges.java:130)
> at
> org.apache.phoenix.compile.WhereOptimizer.pushKeyExpressionsToScan(WhereOptimizer.java:302)
> at
> org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:149)
> at
> org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:100)
> at
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:559)
> at
> org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:510)
> at
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:205)
> at
> org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
> at
> org.apache.phoenix.optimize.QueryOptimizer.addPlan(QueryOptimizer.java:236)
> at
> org.apache.phoenix.optimize.QueryOptimizer.getApplicablePlans(QueryOptimizer.java:146)
> at
> org.apache.phoenix.optimize.QueryOptimizer.optimize(QueryOptimizer.java:94)
> at
> org.apache.phoenix.optimize.QueryOptimizer.optimize(QueryOptimizer.java:80)
> at
> org.apache.phoenix.execute.BaseQueryPlan.getExplainPlan(BaseQueryPlan.java:484)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableExplainStatement.compilePlan(PhoenixStatement.java:464)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableExplainStatement.compilePlan(PhoenixStatement.java:443)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:271)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:266)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:265)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:1408)
> at
> org.apache.phoenix.query.QueryPlanTest.testExplainPlan(QueryPlanTest.java:190)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
