[
https://issues.apache.org/jira/browse/CALCITE-7574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18085091#comment-18085091
]
leishp commented on CALCITE-7574:
---------------------------------
I'd like to work on this
> RelDecorrelator.isFieldNotNullRecursive throws IndexOutOfBoundsException when
> decorrelating correlated scalar subquery with Aggregate
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-7574
> URL: https://issues.apache.org/jira/browse/CALCITE-7574
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.41.0, 1.42.0
> Reporter: leishp
> Priority: Major
> Labels: pull-request-available
>
> `RelDecorrelator.isFieldNotNullRecursive` throws `IndexOutOfBoundsException`
> when decorrelating a correlated scalar subquery that involves an Aggregate.
> The bug was introduced in CALCITE-6962 which added the
> `isFieldNotNull`/`isFieldNotNullRecursive` methods.
> h3. Root Cause
> In `RelDecorrelator.isFieldNotNullRecursive`, the Aggregate branch at line
> 3856:
>
> {code:java}
> } else if (rel instanceof Aggregate) {
> Aggregate agg = (Aggregate) rel;
> ImmutableBitSet groupSet = agg.getGroupSet();
> if (index >= groupSet.size())
> { // BUG: should be agg.getGroupCount() return false; }
> return isFieldNotNullRecursive(agg.getInput(),
> groupSet.asList().get(index));
> }
> {code}
>
> - `ImmutableBitSet.size()` returns the bitset {*}{{*}}capacity{{*}}{*}
> (`words.length * 64`), typically 64 or more
> - `groupSet.asList().get(index)` internally calls `nth(index)`, which
> requires `index < cardinality()`
> - `Aggregate.getGroupCount()` returns `groupSet.cardinality()`, the actual
> number of group keys
> When a field index corresponds to an aggregate result field (not a group
> field), the bounds check `index >= groupSet.size()` is too lenient (e.g., `1
> >= 64` is false), allowing execution to proceed to `nth(1)` which throws
> `IndexOutOfBoundsException` because the bitset has only 1 set bit.
> h3. h3. Reproduction SQL
> {code:sql}
> select t1.deptno, t1.total_sal,
> (select count(distinct t2.total_sal) + 1
> from (select deptno, sum(sal) as total_sal
> from emp where deptno is not null
> group by deptno) t2
> where t2.deptno = t1.deptno
> and t2.total_sal > t1.total_sal) as rank_sal
> from (select deptno, sum(sal) as total_sal
> from emp where deptno is not null
> group by deptno) t1
> order by t1.deptno, t1.total_sal desc
> {code}
> h3. h3. Exception
> {code:java}
> Unable to find source-code formatter for language: text. Available languages
> are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go,
> groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc,
> perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml,
> yamljava.lang.IndexOutOfBoundsException: index out of range: 1
> at org.apache.calcite.util.ImmutableBitSet.nth(ImmutableBitSet.java:903)
> at org.apache.calcite.util.ImmutableBitSet$2.get(ImmutableBitSet.java:681)
> at org.apache.calcite.util.ImmutableBitSet$2.get(ImmutableBitSet.java:679)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3859)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3851)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3859)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3872)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNullRecursive(RelDecorrelator.java:3851)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.isFieldNotNull(RelDecorrelator.java:3840)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1902)
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
