[
https://issues.apache.org/jira/browse/CALCITE-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750750#comment-16750750
]
pengzhiwei edited comment on CALCITE-2792 at 1/24/19 6:17 AM:
--------------------------------------------------------------
Hi [~julianhyde],the StackOverflowError comes from the recursive call in the
SqlValidatorImpl#registerSubQueries,even if the OR RexCall has more than 2
arguments.
Here is the code for this:
{code:java}
private void registerOperandSubQueries(
SqlValidatorScope parentScope,
SqlCall call,
int operandOrdinal) {
......
else if (node instanceof SqlCall) {
validateNodeFeature(node);
SqlCall call = (SqlCall) node;
for (int i = 0; i < call.operandCount(); i++) {
registerOperandSubQueries(parentScope, call, i);
}
.......
} {code}
Except the StackOverflowError, the code generated for "In" may exceed the limit
of 64KB for java method when the IN constants list is to large as the case
shown in this issue.
was (Author: pzw2018):
Hi [~julianhyde],the StackOverflowError comes from the recursive call in the
SqlValidatorImpl#registerSubQueries,even if the OR RexCall has more than 2
arguments.
Here is the code for this:
{code:java}
private void registerOperandSubQueries(
SqlValidatorScope parentScope,
SqlCall call,
int operandOrdinal) {
......
else if (node instanceof SqlCall) {
validateNodeFeature(node);
SqlCall call = (SqlCall) node;
for (int i = 0; i < call.operandCount(); i++) {
registerOperandSubQueries(parentScope, call, i);
}
.......
} {code}
> Stackoverflow while evaluating filter with large number of OR conditions
> ------------------------------------------------------------------------
>
> Key: CALCITE-2792
> URL: https://issues.apache.org/jira/browse/CALCITE-2792
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.18.0
> Reporter: Dirk Mahler
> Assignee: Julian Hyde
> Priority: Major
> Attachments: calcite-stackoverflow.zip
>
>
> As a workaround for CALCITE-2696 we're currently using OR conditions for
> filtering values, e.g. instead of
> {noformat}
> ... WHERE value2 IN (1,2,3)
> {noformat}
> {noformat}
> ... WHERE value2=1 OR value2=2 OR value2=3
> {noformat}
> We're now hitting a StackOverflowError because the number of values in the
> filter grows quite large (i.e. 1000-3000) and obviously the evaluation
> recursive:
> {noformat}
> java.lang.StackOverflowError
> at java.util.AbstractCollection.toArray(AbstractCollection.java:176)
> at
> org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.<init>(SqlShuttle.java:111)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visitScoped(SqlValidatorImpl.java:5699)
> at
> org.apache.calcite.sql.validate.SqlScopedShuttle.visit(SqlScopedShuttle.java:50)
> at
> org.apache.calcite.sql.validate.SqlScopedShuttle.visit(SqlScopedShuttle.java:33)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138)
> at
> org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:134)
> at
> org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:101)
> at org.apache.calcite.sql.SqlOperator.acceptCall(SqlOperator.java:865)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visitScoped(SqlValidatorImpl.java:5701)
> at
> org.apache.calcite.sql.validate.SqlScopedShuttle.visit(SqlScopedShuttle.java:50)
> at
> org.apache.calcite.sql.validate.SqlScopedShuttle.visit(SqlScopedShuttle.java:33)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138)
> at
> org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:134)
> at
> org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:101)
> at org.apache.calcite.sql.SqlOperator.acceptCall(SqlOperator.java:865)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visitScoped(SqlValidatorImpl.java:5701)
> at
> org.apache.calcite.sql.validate.SqlScopedShuttle.visit(SqlScopedShuttle.java:50)
> at
> org.apache.calcite.sql.validate.SqlScopedShuttle.visit(SqlScopedShuttle.java:33)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138)
> ...
> {noformat}
> We tried to increase the stack size of the virtual machine (-Xss8000k) but
> this leads to a problem with Janino:
> {noformat}
> java.sql.SQLException: exception while executing query: Compiling "Buzz":
> Code of method
> "execute(Lorg/apache/calcite/interpreter/Context;[Ljava/lang/Object;)V" of
> class "Buzz" grows beyond 64 KB
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:577)
> at
> org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:137)
> at
> org.apache.calcite.issue.StackOverflowTest.stackOverflow(StackOverflowTest.java:45)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: org.codehaus.janino.InternalCompilerException: Compiling "Buzz":
> Code of method
> "execute(Lorg/apache/calcite/interpreter/Context;[Ljava/lang/Object;)V" of
> class "Buzz" grows beyond 64 KB
> at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:382)
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237)
> at
> org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:465)
> at
> org.codehaus.janino.ClassBodyEvaluator.compileToClass(ClassBodyEvaluator.java:313)
> at
> org.codehaus.janino.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:235)
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207)
> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:50)
> at
> org.codehaus.janino.ClassBodyEvaluator.createInstance(ClassBodyEvaluator.java:347)
> at
> org.apache.calcite.interpreter.JaninoRexCompiler.getScalar(JaninoRexCompiler.java:176)
> at
> org.apache.calcite.interpreter.JaninoRexCompiler.baz(JaninoRexCompiler.java:153)
> at
> org.apache.calcite.interpreter.JaninoRexCompiler.compile(JaninoRexCompiler.java:111)
> at
> org.apache.calcite.interpreter.Interpreter$CompilerImpl.compile(Interpreter.java:487)
> at
> org.apache.calcite.interpreter.Nodes$CoreCompiler.compile(Nodes.java:42)
> at
> org.apache.calcite.interpreter.TableScanNode.createEnumerable(TableScanNode.java:266)
> at
> org.apache.calcite.interpreter.TableScanNode.createProjectableFilterable(TableScanNode.java:233)
> at
> org.apache.calcite.interpreter.TableScanNode.create(TableScanNode.java:81)
> at
> org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:69)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.calcite.util.ReflectUtil.invokeVisitorInternal(ReflectUtil.java:257)
> at
> org.apache.calcite.util.ReflectUtil.invokeVisitor(ReflectUtil.java:214)
> at
> org.apache.calcite.util.ReflectUtil$1.invokeVisitor(ReflectUtil.java:464)
> at
> org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:451)
> at
> org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:42)
> at
> org.apache.calcite.interpreter.Interpreter$CompilerImpl.visitRoot(Interpreter.java:405)
> at
> org.apache.calcite.interpreter.Interpreter.<init>(Interpreter.java:88)
> at Baz.bind(Unknown Source)
> at
> org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:355)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:309)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:506)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:497)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:182)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:64)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:43)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:573)
> ... 26 more
> Caused by: org.codehaus.janino.InternalCompilerException: Code of method
> "execute(Lorg/apache/calcite/interpreter/Context;[Ljava/lang/Object;)V" of
> class "Buzz" grows beyond 64 KB
> at org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:1048)
> at org.codehaus.janino.CodeContext.write(CodeContext.java:925)
> at org.codehaus.janino.UnitCompiler.writeByte(UnitCompiler.java:12275)
> at org.codehaus.janino.UnitCompiler.load(UnitCompiler.java:11936)
> at org.codehaus.janino.UnitCompiler.load(UnitCompiler.java:11926)
> at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4465)
> at org.codehaus.janino.UnitCompiler.access$8000(UnitCompiler.java:215)
> at
> org.codehaus.janino.UnitCompiler$16$1.visitLocalVariableAccess(UnitCompiler.java:4408)
> at
> org.codehaus.janino.UnitCompiler$16$1.visitLocalVariableAccess(UnitCompiler.java:4400)
> at org.codehaus.janino.Java$LocalVariableAccess.accept(Java.java:4274)
> at
> org.codehaus.janino.UnitCompiler$16.visitLvalue(UnitCompiler.java:4400)
> at
> org.codehaus.janino.UnitCompiler$16.visitLvalue(UnitCompiler.java:4396)
> at org.codehaus.janino.Java$Lvalue.accept(Java.java:4148)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
> at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4461)
> at org.codehaus.janino.UnitCompiler.access$7500(UnitCompiler.java:215)
> at
> org.codehaus.janino.UnitCompiler$16$1.visitAmbiguousName(UnitCompiler.java:4403)
> at
> org.codehaus.janino.UnitCompiler$16$1.visitAmbiguousName(UnitCompiler.java:4400)
> at org.codehaus.janino.Java$AmbiguousName.accept(Java.java:4224)
> at
> org.codehaus.janino.UnitCompiler$16.visitLvalue(UnitCompiler.java:4400)
> at
> org.codehaus.janino.UnitCompiler$16.visitLvalue(UnitCompiler.java:4396)
> at org.codehaus.janino.Java$Lvalue.accept(Java.java:4148)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
> at
> org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5662)
> at
> org.codehaus.janino.UnitCompiler.compileBoolean2(UnitCompiler.java:4120)
> at org.codehaus.janino.UnitCompiler.access$6600(UnitCompiler.java:215)
> at
> org.codehaus.janino.UnitCompiler$14.visitBinaryOperation(UnitCompiler.java:3957)
> at
> org.codehaus.janino.UnitCompiler$14.visitBinaryOperation(UnitCompiler.java:3935)
> at org.codehaus.janino.Java$BinaryOperation.accept(Java.java:4864)
> at
> org.codehaus.janino.UnitCompiler.compileBoolean(UnitCompiler.java:3935)
> at
> org.codehaus.janino.UnitCompiler.compileBoolean2(UnitCompiler.java:4078)
> at org.codehaus.janino.UnitCompiler.access$6600(UnitCompiler.java:215)
> at
> org.codehaus.janino.UnitCompiler$14.visitBinaryOperation(UnitCompiler.java:3957)
> at
> org.codehaus.janino.UnitCompiler$14.visitBinaryOperation(UnitCompiler.java:3935)
> at org.codehaus.janino.Java$BinaryOperation.accept(Java.java:4864)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
