Henning Schmiedehausen created FLINK-33363:
----------------------------------------------
Summary: docker bases images can't run the java compiler
Key: FLINK-33363
URL: https://issues.apache.org/jira/browse/FLINK-33363
Project: Flink
Issue Type: Bug
Components: flink-docker
Affects Versions: 1.17.1
Reporter: Henning Schmiedehausen
I have set up a small cluster (Job Manager + 2 Task managers) using docker
compose. When submitting a flink job that needs Calcite planning, it crashes
with
org.apache.flink.client.program.ProgramInvocationException: The main method
caused an error: Unable to instantiate java compiler
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
~[flink-dist-1.17.1.jar:1.17.1]
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
~[flink-dist-1.17.1.jar:1.17.1]
at
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:105)
~[flink-dist-1.17.1.jar:1.17.1]
at
org.apache.flink.client.deployment.application.DetachedApplicationRunner.tryExecuteJobs(DetachedApplicationRunner.java:84)
~[flink-dist-1.17.1.jar:1.17.1]
at
org.apache.flink.client.deployment.application.DetachedApplicationRunner.run(DetachedApplicationRunner.java:70)
~[flink-dist-1.17.1.jar:1.17.1]
at
org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$0(JarRunHandler.java:108)
~[flink-dist-1.17.1.jar:1.17.1]
at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown
Source) [?:?]
at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: java.lang.IllegalStateException: Unable to instantiate java compiler
at
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:163)
~[?:?]
at
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.generateCompileAndInstantiate(JaninoRelMetadataProvider.java:141)
~[?:?]
at
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.lambda$static$0(JaninoRelMetadataProvider.java:73)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:165)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.get(LocalCache.java:3951)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
~[?:?]
at
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.revise(JaninoRelMetadataProvider.java:197)
~[?:?]
at
org.apache.calcite.rel.metadata.RelMetadataQueryBase.revise(RelMetadataQueryBase.java:118)
~[?:?]
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:844)
~[?:?]
at
org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:307)
~[?:?]
at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:337)
~[?:?]
at
org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:565) ~[?:?]
at
org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:428) ~[?:?]
at
org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:251)
~[?:?]
at
org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:130)
~[?:?]
at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:208)
~[?:?]
at
org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:195) ~[?:?]
at
org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:64)
~[?:?]
at
org.apache.flink.table.planner.plan.optimize.program.FlinkHepRuleSetProgram.optimize(FlinkHepRuleSetProgram.scala:78)
~[?:?]
at
org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.$anonfun$optimize$1(FlinkChainedProgram.scala:59)
~[?:?]
at
scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:156)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at
scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:156)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.Iterator.foreach(Iterator.scala:937)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.Iterator.foreach$(Iterator.scala:937)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.AbstractIterator.foreach(Iterator.scala:1425)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.IterableLike.foreach(IterableLike.scala:70)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.IterableLike.foreach$(IterableLike.scala:69)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:156)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at
scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:154)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at
org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.optimize(FlinkChainedProgram.scala:55)
~[?:?]
at
org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.optimizeTree(StreamCommonSubGraphBasedOptimizer.scala:176)
~[?:?]
at
org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.doOptimize(StreamCommonSubGraphBasedOptimizer.scala:83)
~[?:?]
at
org.apache.flink.table.planner.plan.optimize.CommonSubGraphBasedOptimizer.optimize(CommonSubGraphBasedOptimizer.scala:87)
~[?:?]
at
org.apache.flink.table.planner.delegation.PlannerBase.optimize(PlannerBase.scala:329)
~[?:?]
at
org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:195)
~[?:?]
at
org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:224)
~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
at
org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:219)
~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
at
org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:253)
~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
at
org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:226)
~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
at
com.appfolio.data.kafka.consumer.PropertyListings.getDataStream(PropertyListings.java:198)
~[?:?]
at
com.appfolio.data.kafka.consumer.KafkaIcebergConsumer.run(KafkaIcebergConsumer.java:80)
~[?:?]
at
com.appfolio.data.kafka.consumer.KafkaIcebergConsumer.main(KafkaIcebergConsumer.java:32)
~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source) ~[?:?]
at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
~[flink-dist-1.17.1.jar:1.17.1]
... 7 more
Caused by: java.lang.ClassCastException: class
org.codehaus.janino.CompilerFactory cannot be cast to class
org.codehaus.commons.compiler.ICompilerFactory
(org.codehaus.janino.CompilerFactory is in unnamed module of loader
org.apache.flink.util.ChildFirstClassLoader @252ce819;
org.codehaus.commons.compiler.ICompilerFactory is in unnamed module of loader
'app')
at
org.codehaus.commons.compiler.CompilerFactoryFactory.getCompilerFactory(CompilerFactoryFactory.java:129)
~[flink-table-runtime-1.17.1.jar:1.17.1]
at
org.codehaus.commons.compiler.CompilerFactoryFactory.getDefaultCompilerFactory(CompilerFactoryFactory.java:79)
~[flink-table-runtime-1.17.1.jar:1.17.1]
at
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:161)
~[?:?]
at
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.generateCompileAndInstantiate(JaninoRelMetadataProvider.java:141)
~[?:?]
at
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.lambda$static$0(JaninoRelMetadataProvider.java:73)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:165)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.get(LocalCache.java:3951)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
~[?:?]
at
org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
~[?:?]
at
org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.revise(JaninoRelMetadataProvider.java:197)
~[?:?]
at
org.apache.calcite.rel.metadata.RelMetadataQueryBase.revise(RelMetadataQueryBase.java:118)
~[?:?]
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:844)
~[?:?]
at
org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:307)
~[?:?]
at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:337)
~[?:?]
at
org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:565) ~[?:?]
at
org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:428) ~[?:?]
at
org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:251)
~[?:?]
at
org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:130)
~[?:?]
at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:208)
~[?:?]
at
org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:195) ~[?:?]
at
org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:64)
~[?:?]
at
org.apache.flink.table.planner.plan.optimize.program.FlinkHepRuleSetProgram.optimize(FlinkHepRuleSetProgram.scala:78)
~[?:?]
at
org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.$anonfun$optimize$1(FlinkChainedProgram.scala:59)
~[?:?]
at
scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:156)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at
scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:156)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.Iterator.foreach(Iterator.scala:937)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.Iterator.foreach$(Iterator.scala:937)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.AbstractIterator.foreach(Iterator.scala:1425)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.IterableLike.foreach(IterableLike.scala:70)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.IterableLike.foreach$(IterableLike.scala:69)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:156)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at
scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:154)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)
~[flink-scala_2.12-1.17.1.jar:1.17.1]
at
org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.optimize(FlinkChainedProgram.scala:55)
~[?:?]
at
org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.optimizeTree(StreamCommonSubGraphBasedOptimizer.scala:176)
~[?:?]
at
org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.doOptimize(StreamCommonSubGraphBasedOptimizer.scala:83)
~[?:?]
at
org.apache.flink.table.planner.plan.optimize.CommonSubGraphBasedOptimizer.optimize(CommonSubGraphBasedOptimizer.scala:87)
~[?:?]
at
org.apache.flink.table.planner.delegation.PlannerBase.optimize(PlannerBase.scala:329)
~[?:?]
at
org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:195)
~[?:?]
at
org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:224)
~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
at
org.apache.flink.table.api.bridge.internal.AbstractStreamTableEnvironmentImpl.toStreamInternal(AbstractStreamTableEnvironmentImpl.java:219)
~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
at
org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:253)
~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
at
org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.java:226)
~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
at
com.appfolio.data.kafka.consumer.PropertyListings.getDataStream(PropertyListings.java:198)
~[?:?]
at
com.appfolio.data.kafka.consumer.KafkaIcebergConsumer.run(KafkaIcebergConsumer.java:80)
~[?:?]
at
com.appfolio.data.kafka.consumer.KafkaIcebergConsumer.main(KafkaIcebergConsumer.java:32)
~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source) ~[?:?]
at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
~[flink-dist-1.17.1.jar:1.17.1]
... 7 more
2023-10-25 23:23:02,347 ERROR
org.apache.flink.runtime.webmonitor.handlers.JarRunHandler [] - Exception
occurred in REST handler: Could not execute application.
As this image builds off eclipse-temurin:11-jre-jammy, this may be the problem
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
