[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835683#comment-17835683 ] Ruben Q L commented on CALCITE-5390: Recently I stumbled upon this bug, in my case it's a LogicalFilter with 2 RexSubquery using the same $cor0 which triggers the NPE when calling the Decorrelator. It's a far more complicated query that the ones already mentioned previously (so far I haven't been able to reproduce my issue with a minimalistic test), but I think the principle is the same. I'm not sure trimming plays a role here (at least on my case I get the error without trimming the plan). I pretty much agree with [~libenchao]'s analysis here, it seems strange that the SubQueryRemoveRule generates a plan with two LogicalCorrelate using the same corr variable (it's exactly the same on my case). The plan looks odd, but... if I am not mistaken, if we skip the decorrelation, that odd plan seems to run fine (at least on my case, where we run the decorrelator within a try-catch, given that it's been traditionally a problematic module), and it case of exception, we simply go with the original (non-decorrelated) plan, and everything is fine. Actually, it seems that this "bizarre plan structure" of having 2 LogicalCorrelate with the same variable is not that uncommon: if we look inside {{RelOptRulesTest.xml}} we can see around 10 coincidences of said pattern. Moreover, it'd seem that this pattern is a necessary condition for the Decorrelator NPE, but not sufficient (according to my tests, some of these plans can be correctly decorrelated; it probably depends on which fields are accessed on each "instance" of the corr variable, as [~julianhyde] mentioned here). To sum up, I agree with [~libenchao] that one potential solution could be modifying SubQueryRemoveRule to introduce different correlating variables instead of one in these cases (and that will probably make the Decorralator happy). But, given that the current plan generated by SubQueryRemoveRule seems "odd by correct" (it gets executed correctly if we skip the Decorrelator and its potential NPE); I wonder if perhaps our efforts should rather focus on fixing the Decorrelator and make it work with this type of plans. I'm not sure which approach would be simpler, but I just wanted to mention it. PS: I haven't looked in detail, but I feel that on Decorrelator side, the problem (or at least one of them) might originate because its {{CorelMap}} contains just a Map for {{}} , and it would seems that in case like this, a MultiMap would be required? (two different LogicalCorrelate values having the same CorrelationId key?) > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at >
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819898#comment-17819898 ] Dan Zou commented on CALCITE-5390: -- [~hanu.ncr] Yes, feel free to do it. > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Dan Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1187) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1169) > at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at >
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819894#comment-17819894 ] Hanumath Rao Maduri commented on CALCITE-5390: -- [~FrankZou] Is it fine if I assign this Jira to myself. > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Dan Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1187) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1169) > at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at >
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723271#comment-17723271 ] Sergey Nuyanzin commented on CALCITE-5390: -- I noticed that for the first query it considers correlation as a trivial one at {{org.apache.calcite.sql2rel.CorrelateProjectExtractor#visit}} {code:java} Set callsWithCorrelationInRight = findCorrelationDependentCalls(correlate.getCorrelationId(), right); boolean isTrivialCorrelation = callsWithCorrelationInRight.stream().allMatch(exp -> exp instanceof RexFieldAccess); // Early exit condition if (isTrivialCorrelation) { ... } {code} at the same time it is still questionable whether it could be trivial since there is no condition for {{t2}} column... also disable of {{early exit}} for this query makes the query working > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Dan Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at >
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716357#comment-17716357 ] Stamatis Zampetakis commented on CALCITE-5390: -- I will also check this more in detail once I get the chance; sorry if I broke something in the process. A good measure of progress is the number of successful unit tests and code coverage metric. As long as we increase those I think we are moving in the right direction. > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Dan Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1187) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1169) > at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716354#comment-17716354 ] Julian Hyde commented on CALCITE-5390: -- Ok, I'll look into trimming. It seems that there are a lot of bugs in this area. I fear that one or more of the efforts to "improve" things - e.g. de-duplicate correlation variables, and your effort to move expressions - have made some cases better but some cases worse. The bug density is high enough that an attempt to fix one bug triggers other bugs. Therefore it's difficult to tell whether we are making progress. > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Dan Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1187) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1169) > at
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715923#comment-17715923 ] Stamatis Zampetakis commented on CALCITE-5390: -- [~julianhyde] I didn't debug the example you provided yet but it is likely that the problem is somewhat related to field trimming. I remember some [discussion|https://github.com/apache/calcite/pull/2623#discussion_r823790744] with [~korlov] where he pointed out a problem in RelFieldTrimmer. In PR#2623, there was also a [part|https://github.com/apache/calcite/pull/2623/commits/8fc0f2bc687142141ab04d63a67952b9022f2cf0#diff-a57af2470b3215be0ce7d94a226f1997d9417c613fd7f32097fd07543b30633a)] adding logic in {{SubQueryRemoveRule}} to do something similar to what [~libenchao] suggested above (rewrite to different variables when removing the sub-query) but was abandoned given that the root cause seemed to be in the trimmer. > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Dan Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) >
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715365#comment-17715365 ] Benchao Li commented on CALCITE-5390: - Variables deduplication was done in CALCITE-4913, and I think it's outcome is correct because the two sub-queries is indeed referencing the same {{RelNode}}. For the {{SQL}} pasted above by Julian, the plan after converting to {{RelNode}} is below. {code:xml} LogicalProject(variablesSet=[[$cor0]], T1=[$SCALAR_QUERY({ LogicalSort(fetch=[1]) LogicalProject(EXPR$0=[1]) LogicalFilter(condition=[=($2, $cor0.JOB)]) LogicalTableScan(table=[[scott, EMP]]) })], T2=[$SCALAR_QUERY({ LogicalSort(fetch=[1]) LogicalProject(EXPR$0=[=($cor0.JOB, 'PRESIDENT')]) LogicalTableScan(table=[[scott, EMP]]) })]) LogicalTableScan(table=[[scott, EMP]]) {code} After {{SubQueryRemoveRule}}, it would be transformed to below, which I think it's not correct since {{cor0}} is not clear. It may should be rewritten to different variables again when we are doing the sub-query removing. {code:xml|} LogicalProject(T1=[$8], T2=[$9]) LogicalCorrelate(correlation=[$cor0], joinType=[left], requiredColumns=[{2}]) LogicalCorrelate(correlation=[$cor0], joinType=[left], requiredColumns=[{2}]) LogicalTableScan(table=[[scott, EMP]]) LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) LogicalSort(fetch=[1]) LogicalProject(EXPR$0=[1]) LogicalFilter(condition=[=($2, $cor0.JOB)]) LogicalTableScan(table=[[scott, EMP]]) LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) LogicalSort(fetch=[1]) LogicalProject(EXPR$0=[=($cor0.JOB, 'PRESIDENT')]) LogicalTableScan(table=[[scott, EMP]]) {code} > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Dan Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715343#comment-17715343 ] Julian Hyde commented on CALCITE-5390: -- [~zabetak], I'm investigating the NullPointerException caused by the first query. It seems related to the changes that you made in CALCITE-4560 (pushing correlation expressions) and also the logic to deduplicate correlation variables. Here's the intermediate plan: {noformat} LogicalProject(T1=[$8], T2=[$9]) LogicalCorrelate(correlation=[$cor0], joinType=[left], requiredColumns=[{2}]) LogicalCorrelate(correlation=[$cor0], joinType=[left], requiredColumns=[{2}]) LogicalTableScan(table=[[scott, EMP]]) LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) LogicalSort(fetch=[1]) LogicalProject(EXPR$0=[1]) LogicalFilter(condition=[=($2, $cor0.JOB)]) LogicalTableScan(table=[[scott, EMP]]) LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) LogicalSort(fetch=[1]) LogicalProject(EXPR$0=[=($cor0.JOB, 'PRESIDENT')]) LogicalTableScan(table=[[scott, EMP]]) {noformat} Note that $cor0 occurs on lines 2 and 3. But I believe it has different types on these lines - the occurrence on line 2 has more fields than line 3. When field #9 is referenced in a location that has only 7 fields, that's when the NPE occurs. Do you agree that the different types are a problem? By the way, you can reproduce by pasting the following into {{dummy.iq}}: {noformat} !use scott # Throws: java.lang.NullPointerException # at java.base/java.util.Objects.requireNonNull(Objects.java:208) # at org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) SELECT (SELECT 1 FROM emp AS d WHERE d.job = a.job LIMIT 1) AS t1, (SELECT a.job = 'PRESIDENT' FROM emp AS s LIMIT 1) as t2 FROM emp AS a; !ok {noformat} > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Dan Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714269#comment-17714269 ] Julian Hyde commented on CALCITE-5390: -- The latter query (with "CASE") throws {noformat} java.lang.AssertionError: Required columns {3} not subset of left columns {0, 1} at org.apache.calcite.util.Litmus$1.fail(Litmus.java:32) at org.apache.calcite.util.Litmus$1.check(Litmus.java:44) at org.apache.calcite.rel.core.Correlate.isValid(Correlate.java:145) at org.apache.calcite.rel.core.Correlate.(Correlate.java:109) at org.apache.calcite.rel.logical.LogicalCorrelate.(LogicalCorrelate.java:72) at org.apache.calcite.rel.logical.LogicalCorrelate.create(LogicalCorrelate.java:115) at org.apache.calcite.rel.core.RelFactories$CorrelateFactoryImpl.createCorrelate(RelFactories.java:440) at org.apache.calcite.tools.RelBuilder.join(RelBuilder.java:2861) at org.apache.calcite.rel.rules.SubQueryRemoveRule.rewriteIn(SubQueryRemoveRule.java:698) {noformat} and is therefore similar to CALCITE-5638 and CALCITE-5429. > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Dan Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at >
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697487#comment-17697487 ] Sergey Nuyanzin commented on CALCITE-5390: -- Thanks for the quick response. I will try to see how hard is to handle this... Here are some more findings: there are 2 more queries failing with same exception (I put them in description as well) {code:sql} SELECT * FROM emps e WHERE e.name NOT IN (SELECT d.name FROM depts d WHERE e.deptno = d.deptno OR e.gender ='M'); SELECT city FROM emps e WHERE ((CASE WHEN name NOT IN (SELECT name FROM sdepts s WHERE s.deptno = e.deptno) THEN '1' ELSE '2' END) NOT IN (SELECT name FROM depts d WHERE e.deptno = d.deptno)); {code} Also I noticed that these queries starts failing with NPE with this change https://issues.apache.org/jira/browse/CALCITE-4560. In a commit before that change there is no such issue > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at >
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697401#comment-17697401 ] Zou commented on CALCITE-5390: -- [~Sergey Nuyanzin] I am sorry that I haven't started this work yet as I am busy recently. You are free to take it if you are interested in it. > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1187) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1169) > at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at >
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697394#comment-17697394 ] Sergey Nuyanzin commented on CALCITE-5390: -- Hi [~FrankZou] , do you have any plans to work on this issue or do you need some help? I'm asking since also interested in this issue as faced it in Flink > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1187) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1169) > at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645268#comment-17645268 ] Dmitry Sysolyatin commented on CALCITE-5390: [~FrankZou] Hi! Do you have any progress with this ticket ? > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1187) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1169) > at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at >
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640020#comment-17640020 ] Benchao Li commented on CALCITE-5390: - [~FrankZou] Thanks for taking this, looking forward to your PR! > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Zou >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1187) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1169) > at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) >
[jira] [Commented] (CALCITE-5390) RelDecorrelator throws NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640011#comment-17640011 ] Zou commented on CALCITE-5390: -- I would like to tack this issue > RelDecorrelator throws NullPointerException > --- > > Key: CALCITE-5390 > URL: https://issues.apache.org/jira/browse/CALCITE-5390 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Dmitry Sysolyatin >Priority: Major > > The current query throws NullPointerException > {code:java} > SELECT > (SELECT 1 FROM emp d WHERE d.job = a.job LIMIT 1) AS t1, > (SELECT a.job = 'PRESIDENT' FROM emp s LIMIT 1) as t2 > FROM emp a; > {code} > Test case - > [https://github.com/apache/calcite/commit/46fe9bc456f2d34cf7dccd29829c9e85abe69d5f] > Logical plan before it fails: > {code:java} > LogicalProject(T1=[$8], T2=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f09=[$9]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], > SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f00=[$10]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{9}]) > LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], > HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], $f0=[$8], $f9=[=($2, > 'PRESIDENT')]) > LogicalCorrelate(correlation=[$cor0], joinType=[left], > requiredColumns=[{2}]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[1]) > LogicalFilter(condition=[=($2, $cor0.JOB)]) > LogicalTableScan(table=[[scott, EMP]]) > LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)]) > LogicalSort(fetch=[1]) > LogicalProject(EXPR$0=[$cor0.$f9]) > LogicalTableScan(table=[[scott, EMP]]) {code} > Stack trace: > {code:java} > Caused by: java.lang.NullPointerException > at java.util.Objects.requireNonNull(Objects.java:203) > at > org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:833) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:764) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738) > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:464) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495) > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1187) > at > org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1169) > at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531) > at > org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707) > at >