[
https://issues.apache.org/jira/browse/FLINK-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176007#comment-15176007
]
Maximilian Michels commented on FLINK-3565:
-------------------------------------------
Thanks to [~till.rohrmann], I've come up with another approach which is also
used in other projects. The idea is to add a dependency to the root pom which
is then explicitly shaded from all modules. That way, the Shade plugin always
generates the so called "effective" poms. Effective poms have their properties
resolved. Thus, we can configure the Scala version like before in the root pom.
When Maven packages the Jars, the Shade plugin will ensure that all poms are
made "effective".
I don't know whether we stick to this approach. It is possible to move to
another solution later on. Most importantly, this change is transparent to the
user.
In contrast to this, I've created an independent module in our source. So
everything is self-contained and we don't rely on an external Maven module:
https://issues.apache.org/jira/browse/SPARK-3812
> FlinkKafkaConsumer does not work with Scala 2.11
> -------------------------------------------------
>
> Key: FLINK-3565
> URL: https://issues.apache.org/jira/browse/FLINK-3565
> Project: Flink
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.0.0
> Reporter: Aljoscha Krettek
> Assignee: Maximilian Michels
> Priority: Blocker
>
> Running a program built against Flink_2.11 dependencies fails on a Flink_2.11
> cluster with this exception:
> java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class
> at kafka.utils.Pool.<init>(Pool.scala:28)
> at
> kafka.consumer.FetchRequestAndResponseStatsRegistry$.<init>(FetchRequestAndResponseStats.scala:60)
> at
> kafka.consumer.FetchRequestAndResponseStatsRegistry$.<clinit>(FetchRequestAndResponseStats.scala)
> at kafka.consumer.SimpleConsumer.<init>(SimpleConsumer.scala:39)
> at kafka.javaapi.consumer.SimpleConsumer.<init>(SimpleConsumer.scala:34)
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.getPartitionsForTopic(FlinkKafkaConsumer08.java:518)
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:218)
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:193)
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:160)
> at com.dataartisans.querywindow.WindowJob.main(WindowJob.java:93)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:505)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:403)
> at org.apache.flink.client.program.Client.runBlocking(Client.java:248)
> at
> org.apache.flink.client.CliFrontend.executeProgramBlocking(CliFrontend.java:866)
> at org.apache.flink.client.CliFrontend.run(CliFrontend.java:333)
> at
> org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1189)
> at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1239)
> Caused by: java.lang.ClassNotFoundException:
> scala.collection.GenTraversableOnce$class
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 21 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
