Hi Avi, apparently the maximum version that Flink supports for scala is 2.12.7 [1]. Do you have a specific reason to use a higher version?
[1] https://issues.apache.org/jira/browse/FLINK-12461 On Thu, Jan 14, 2021 at 5:11 AM Avi Levi <a...@neosec.com> wrote: > Hi Arvid, > Please find attached full build.gradle file > > On Tue, Jan 12, 2021 at 8:18 PM Arvid Heise <ar...@ververica.com> wrote: > >> Can you post the full dependencies of sbt/maven/gradle whatever? >> >> On Tue, Jan 12, 2021 at 3:54 AM Avi Levi <a...@neosec.com> wrote: >> >>> Hi Arvid, >>> using : >>> >>> flinkVersion = '1.12.0' >>> scalaBinaryVersion = '2.12' >>> >>> I simplified the example to (same exception) : >>> >>> object Flinktest extends App { >>> private val env = StreamExecutionEnvironment.getExecutionEnvironment >>> env.fromElements("A", "B","c") >>> .windowAll(TumblingEventTimeWindows.of(Time.seconds(5))) >>> .process{new ProcessAllWindowFunction[String, List[String], TimeWindow] >>> { >>> override def process(context: Context, elements: Iterable[String], >>> out: Collector[List[String]]): Unit = { >>> out.collect(elements.toList) >>> } >>> } >>> } >>> .print() >>> >>> env.execute("Sample") >>> } >>> >>> >>> >>> >>> On Tue, Jan 5, 2021 at 1:53 PM Arvid Heise <ar...@ververica.com> wrote: >>> >>>> Hi Avi, >>>> >>>> without being a scala-guy, I'm guessing that you are mixing scala >>>> versions. Could you check that your user code uses the same scala version >>>> as Flink (1.11 or 1.12)? I have also heard of issues with different minor >>>> versions of scala, so make sure to use the exact same version (e.g. >>>> 2.11.12). >>>> >>>> On Mon, Dec 28, 2020 at 3:54 PM Avi Levi <a...@neosec.com> wrote: >>>> >>>>> I am trying to aggregate all records in a time window. This is my >>>>> ProcessAllWindowFunction : >>>>> >>>>> case class SimpleAggregate(elms: List[String]) >>>>> >>>>> class AggregateLogs extends ProcessAllWindowFunction[String, >>>>> SimpleAggregate, TimeWindow ] { >>>>> override def process(context: Context, elements: Iterable[String], out: >>>>> Collector[SimpleAggregate]): Unit = { >>>>> val es: List[String] = elements.toList >>>>> val record = SimpleAggregate(es) >>>>> out.collect(record) >>>>> } >>>>> } >>>>> >>>>> But I am getting this exception why ? >>>>> >>>>> Exception in thread "main" java.util.concurrent.ExecutionException: >>>>> scala.tools.reflect.ToolBoxError: reflective compilation has failed: >>>>> cannot >>>>> initialize the compiler due to java.lang.BootstrapMethodError: >>>>> java.lang.NoSuchMethodError: >>>>> scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object; >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299) >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286) >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:137) >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2348) >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2320) >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282) >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197) >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache.get(LocalCache.java:3937) >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4739) >>>>> at >>>>> org.apache.flink.api.scala.typeutils.TraversableSerializer$.compileCbf(TraversableSerializer.scala:184) >>>>> at >>>>> org.apache.flink.api.scala.typeutils.TraversableSerializer.compileCbf(TraversableSerializer.scala:51) >>>>> at >>>>> org.apache.flink.api.scala.typeutils.TraversableSerializer.<init>(TraversableSerializer.scala:41) >>>>> at >>>>> com.neosec.handlefinancial.HandleFinancialJob$$anon$1$$anon$2$$anon$3.<init>(HandleFinancialJob.scala:52) >>>>> at >>>>> com.neosec.handlefinancial.HandleFinancialJob$$anon$1$$anon$2.createSerializer(HandleFinancialJob.scala:52) >>>>> at >>>>> com.neosec.handlefinancial.HandleFinancialJob$$anon$1$$anon$2.createSerializer(HandleFinancialJob.scala:52) >>>>> at >>>>> com.neosec.handlefinancial.HandleFinancialJob$$anon$1.$anonfun$createSerializer$1(HandleFinancialJob.scala:52) >>>>> at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158) >>>>> at >>>>> com.neosec.handlefinancial.HandleFinancialJob$$anon$1.createSerializer(HandleFinancialJob.scala:52) >>>>> at >>>>> org.apache.flink.streaming.api.graph.StreamGraph.createSerializer(StreamGraph.java:864) >>>>> at >>>>> org.apache.flink.streaming.api.graph.StreamGraph.addOperator(StreamGraph.java:308) >>>>> at >>>>> org.apache.flink.streaming.api.graph.StreamGraph.addOperator(StreamGraph.java:293) >>>>> at >>>>> org.apache.flink.streaming.api.graph.StreamGraphGenerator.transformOneInputTransform(StreamGraphGenerator.java:680) >>>>> at >>>>> org.apache.flink.streaming.api.graph.StreamGraphGenerator.transform(StreamGraphGenerator.java:253) >>>>> at >>>>> org.apache.flink.streaming.api.graph.StreamGraphGenerator.generate(StreamGraphGenerator.java:212) >>>>> at >>>>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:1863) >>>>> at >>>>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:1848) >>>>> at >>>>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1699) >>>>> at >>>>> org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.scala:699) >>>>> at >>>>> com.neosec.handlefinancial.HandleFinancialJob$.delayedEndpoint$com$neosec$handlefinancial$HandleFinancialJob$1(HandleFinancialJob.scala:60) >>>>> at >>>>> com.neosec.handlefinancial.HandleFinancialJob$delayedInit$body.apply(HandleFinancialJob.scala:20) >>>>> at scala.Function0.apply$mcV$sp(Function0.scala:39) >>>>> at scala.Function0.apply$mcV$sp$(Function0.scala:39) >>>>> at >>>>> scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17) >>>>> at scala.App.$anonfun$main$1$adapted(App.scala:80) >>>>> at scala.collection.immutable.List.foreach(List.scala:431) >>>>> at scala.App.main(App.scala:80) >>>>> at scala.App.main$(App.scala:78) >>>>> at >>>>> com.neosec.handlefinancial.HandleFinancialJob$.main(HandleFinancialJob.scala:20) >>>>> at >>>>> com.neosec.handlefinancial.HandleFinancialJob.main(HandleFinancialJob.scala) >>>>> Caused by: scala.tools.reflect.ToolBoxError: reflective compilation >>>>> has failed: cannot initialize the compiler due to >>>>> java.lang.BootstrapMethodError: java.lang.NoSuchMethodError: >>>>> scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object; >>>>> Caused by: scala.tools.reflect.ToolBoxError: reflective compilation >>>>> has failed: cannot initialize the compiler due to >>>>> java.lang.BootstrapMethodError: java.lang.NoSuchMethodError: >>>>> scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object; >>>>> >>>>> at >>>>> scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$withCompilerApi$api$.liftedTree1$1(ToolBoxFactory.scala:360) >>>>> at >>>>> scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$withCompilerApi$api$.compiler$lzycompute(ToolBoxFactory.scala:346) >>>>> at >>>>> scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$withCompilerApi$api$.compiler(ToolBoxFactory.scala:345) >>>>> at >>>>> scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$withCompilerApi$.apply(ToolBoxFactory.scala:372) >>>>> at >>>>> scala.tools.reflect.ToolBoxFactory$ToolBoxImpl.parse(ToolBoxFactory.scala:429) >>>>> at >>>>> org.apache.flink.api.scala.typeutils.TraversableSerializer$LazyRuntimeCompiler.compileCbfInternal(TraversableSerializer.scala:229) >>>>> at >>>>> org.apache.flink.api.scala.typeutils.TraversableSerializer$LazyRuntimeCompiler.call(TraversableSerializer.scala:220) >>>>> at >>>>> org.apache.flink.api.scala.typeutils.TraversableSerializer$LazyRuntimeCompiler.call(TraversableSerializer.scala:216) >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4742) >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527) >>>>> at >>>>> org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319) >>>>> ... 34 more >>>>> Caused by: java.lang.BootstrapMethodError: >>>>> java.lang.NoSuchMethodError: >>>>> scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object; >>>>> at >>>>> scala.tools.nsc.PhaseAssembly$DependencyGraph.compilerPhaseList(PhaseAssembly.scala:102) >>>>> at >>>>> scala.tools.nsc.PhaseAssembly.computePhaseAssembly(PhaseAssembly.scala:244) >>>>> at >>>>> scala.tools.nsc.PhaseAssembly.computePhaseAssembly$(PhaseAssembly.scala:216) >>>>> Caused by: java.lang.BootstrapMethodError: >>>>> java.lang.NoSuchMethodError: >>>>> scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object; >>>>> >>>>> at scala.tools.nsc.Global.computePhaseAssembly(Global.scala:44) >>>>> at scala.tools.nsc.Global.computePhaseDescriptors(Global.scala:713) >>>>> at scala.tools.nsc.Global.phaseDescriptors$lzycompute(Global.scala:717) >>>>> at scala.tools.nsc.Global.phaseDescriptors(Global.scala:717) >>>>> at scala.tools.nsc.Global$Run.<init>(Global.scala:1219) >>>>> at scala.tools.reflect.ReflectSetup.$init$(ReflectSetup.scala:21) >>>>> at scala.tools.reflect.ReflectGlobal.<init>(ReflectGlobal.scala:26) >>>>> at >>>>> scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.<init>(ToolBoxFactory.scala:52) >>>>> at >>>>> scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$withCompilerApi$api$.liftedTree1$1(ToolBoxFactory.scala:350) >>>>> ... 44 more >>>>> Caused by: java.lang.NoSuchMethodError: >>>>> scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object; >>>>> ... 56 more >>>>> Caused by: java.lang.NoSuchMethodError: >>>>> scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object; >>>>> >>>> >>>> >>>> -- >>>> >>>> Arvid Heise | Senior Java Developer >>>> >>>> <https://www.ververica.com/> >>>> >>>> Follow us @VervericaData >>>> >>>> -- >>>> >>>> Join Flink Forward <https://flink-forward.org/> - The Apache Flink >>>> Conference >>>> >>>> Stream Processing | Event Driven | Real Time >>>> >>>> -- >>>> >>>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >>>> >>>> -- >>>> Ververica GmbH >>>> Registered at Amtsgericht Charlottenburg: HRB 158244 B >>>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji >>>> (Toni) Cheng >>>> >>> >> >> -- >> >> Arvid Heise | Senior Java Developer >> >> <https://www.ververica.com/> >> >> Follow us @VervericaData >> >> -- >> >> Join Flink Forward <https://flink-forward.org/> - The Apache Flink >> Conference >> >> Stream Processing | Event Driven | Real Time >> >> -- >> >> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >> >> -- >> Ververica GmbH >> Registered at Amtsgericht Charlottenburg: HRB 158244 B >> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji >> (Toni) Cheng >> > -- Arvid Heise | Senior Java Developer <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
