Hi all,
I've been building for years via gradle. The dependency management is
probably a bit different from that of maven, but it seems that dataflow now
requires Confluent Kafka dependencies. They are not available in Maven
Central. This feels wrong for an Apache project.
-
file:/Users/alex.vanboxel/.m2/repository/io/confluent/kafka-avro-serializer/5.3.2/kafka-avro-serializer-5.3.2.pom
-
https://repo.maven.apache.org/maven2/io/confluent/kafka-avro-serializer/5.3.2/kafka-avro-serializer-5.3.2.pom
-
https://repository.apache.org/content/repositories/releases/io/confluent/kafka-avro-serializer/5.3.2/kafka-avro-serializer-5.3.2.pom
Excluding the dependencies "exclude module: '*beam-sdks-java-io-kafka*'"
doesn't work with:
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/beam/sdk/io/kafka/KafkaIO$Read
at
org.apache.beam.runners.dataflow.DataflowRunner.getOverrides(DataflowRunner.java:522)
at
org.apache.beam.runners.dataflow.DataflowRunner.replaceV1Transforms(DataflowRunner.java:1337)
at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:967)
at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:196)
This happens from version 2.30 onwards. Is this intended?!
_/
_/ Alex Van Boxel
