[ https://issues.apache.org/jira/browse/FLINK-27606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Anderson updated FLINK-27606: ----------------------------------- Summary: CompileException when using UDAF with merge() method (was: Fix CompileException when using UDAF with merge() method) > CompileException when using UDAF with merge() method > ---------------------------------------------------- > > Key: FLINK-27606 > URL: https://issues.apache.org/jira/browse/FLINK-27606 > Project: Flink > Issue Type: Bug > Components: Table SQL / Runtime > Affects Versions: 1.15.0, 1.16.0 > Reporter: dalongliu > Assignee: dalongliu > Priority: Major > Labels: pull-request-available > Fix For: 1.16.0, 1.15.1 > > > If the SQL contains a UDAF with {{merge()}} method, the job will throw an > exception like following in cluster mode. > Flink generates code to use {{SingleElementIterator}} class to call merge() > of UDAF. However, the {{SingleElementIterator}} is in flink-table-planner > module and planner scala-free was introduced since release 1.15. Therefore, > user classloader can't find {{SingleElementIterator}} class. > We can move {{SingleElementIterator}} to flink-table-runtime to fix this > problem. > {code} > 2022-05-10 17:41:46,621 WARN > org.apache.flink.table.runtime.generated.GeneratedClass [] - Failed to > compile split code, falling back to original code > org.apache.flink.util.FlinkRuntimeException: > org.apache.flink.api.common.InvalidProgramException: Table program cannot be > compiled. This is a bug. Please file an issue. > at > org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:94) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.table.runtime.generated.GeneratedClass.compile(GeneratedClass.java:97) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:68) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.table.runtime.operators.aggregate.MiniBatchGlobalGroupAggFunction.open(MiniBatchGlobalGroupAggFunction.java:142) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.table.runtime.operators.bundle.AbstractMapBundleOperator.open(AbstractMapBundleOperator.java:86) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:107) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:700) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:676) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:643) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:954) > [flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:923) > [flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746) > [flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:568) > [flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at java.lang.Thread.run(Thread.java:834) [?:?] > Caused by: > org.apache.flink.shaded.guava30.com.google.common.util.concurrent.UncheckedExecutionException: > org.apache.flink.api.common.InvalidProgramException: Table program cannot be > compiled. This is a bug. Please file an issue. > at > org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.get(LocalCache.java:3962) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4859) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:92) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > ... 14 more > Caused by: org.apache.flink.api.common.InvalidProgramException: Table > program cannot be compiled. This is a bug. Please file an issue. > at > org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:107) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.table.runtime.generated.CompileUtils.lambda$compile$0(CompileUtils.java:92) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4864) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.get(LocalCache.java:3962) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4859) > ~[flink-dist-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:92) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > ... 14 more > Caused by: org.codehaus.commons.compiler.CompileException: Line 11, Column > 28: Cannot determine simple type name "org" > at > org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6833) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6594) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6573) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at org.codehaus.janino.UnitCompiler.access$13900(UnitCompiler.java:215) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(UnitCompiler.java:6481) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(UnitCompiler.java:6476) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3928) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:6476) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at > org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:6469) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3927) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at org.codehaus.janino.UnitCompiler.access$1300(UnitCompiler.java:215) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at org.codehaus.janino.UnitCompiler$25.getType(UnitCompiler.java:8271) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6873) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > at org.codehaus.janino.UnitCompiler.access$14400(UnitCompiler.java:215) > ~[flink-table-runtime-1.15-vvr-6.0-SNAPSHOT.jar:1.15-vvr-6.0-SNAPSHOT] > {code} -- This message was sent by Atlassian Jira (v8.20.7#820007)