[
https://issues.apache.org/jira/browse/FLINK-19363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-19363:
----------------------------
Fix Version/s: (was: 1.13.0)
1.14.0
> Code of split method grows beyond 64 KB
> ---------------------------------------
>
> Key: FLINK-19363
> URL: https://issues.apache.org/jira/browse/FLINK-19363
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Affects Versions: 1.11.0
> Reporter: hailong wang
> Priority: Major
> Fix For: 1.14.0
>
>
> For now, If the total size beyond 64kb, we will split one field into one
> field. But when sql is complicated, one field size will grow 64kb.
> {code:java}
> Caused by: org.codehaus.janino.InternalCompilerException: Compiling
> "StreamExecCalc$4436": Code of method "split$4435$(LStreamExecCalc$4436;)V"
> of class "StreamExecCalc$4436" grows beyond 64 KBCaused by:
> org.codehaus.janino.InternalCompilerException: Compiling
> "StreamExecCalc$4436": Code of method "split$4435$(LStreamExecCalc$4436;)V"
> of class "StreamExecCalc$4436" grows beyond 64 KB at
> org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:382) at
> org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237) at
> org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:465)
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216) at
> org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207) at
> org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80) at
> org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75) at
> org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:78)
> ... 24 moreCaused by: org.codehaus.janino.InternalCompilerException: Code of
> method "split$4435$(LStreamExecCalc$4436;)V" of class "StreamExecCalc$4436"
> grows beyond 64 KB at
> org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:1009) at
> org.codehaus.janino.CodeContext.write(CodeContext.java:918) at
> org.codehaus.janino.CodeContext.writeBranch(CodeContext.java:1038)
> {code}
> By this case, we should split one field to multi functions.
> My solution is as follow,
> {code:java}
> override def visitCall(call: RexCall): GeneratedExpression = {
> val resultType = FlinkTypeFactory.toLogicalType(call.getType)
> // convert operands and help giving untyped NULL literals a type
> val operands = call.getOperands.zipWithIndex.map {
> // this helps e.g. for AS(null)
> // we might need to extend this logic in case some rules do not create
> typed NULLs
> case (operandLiteral: RexLiteral, 0) if
> operandLiteral.getType.getSqlTypeName == SqlTypeName.NULL &&
> call.getOperator.getReturnTypeInference == ReturnTypes.ARG0 =>
> generateNullLiteral(resultType, ctx.nullCheck)
> case (o@_, _) => o.accept(this)
> }
> // when function is too large, we split it.
> val exp = generateCallExpression(ctx, call.getOperator, operands,
> resultType)
> if (exp.code.length > maxGeneratedCodeLength) {
> ctx.setCodeSplit()
> val methodName = newName("callSplit")
> val resultTypeTerm = boxedTypeTermForType(exp.resultType)
> val callResultTerm = ctx.addReusableLocalVariable(resultTypeTerm,
> "callSplitResultTerm")
> val callNullResultTerm = ctx.addReusableLocalVariable("boolean",
> "callSplitNullResultTerm")
> val method =
> s"""
> |private void $methodName() throws Exception {
> | ${exp.code}
> | ${callResultTerm}= ${exp.resultTerm};
> | ${callNullResultTerm}= ${exp.nullTerm};
> |}
> |""".stripMargin
> val methodCode =
> s"""
> |$methodName();
> |""".stripMargin
> ctx.addReusableMember(method)
> return new GeneratedExpression(callResultTerm, callNullResultTerm,
> methodCode, exp.resultType)
> }
> exp
> }
> {code}
> When function beyong maxGeneratedCodeLength, we split it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
