Hi, I think this is the classic problem of dependency version conflicts. Beam has a dependency on protobuf 3.0.0-beta1 while Flink has a dependency on protobuf 2.5.0 (through Akka). I think when running your program through the bin/flink command the order in the classpath might be different and you're getting the wrong version.
As an immediate fix, I think you could try having your own dependency on protobuf and shade that, so that you don't have version conflicts. Cheers, Aljoscha On Thu, 1 Sep 2016 at 16:15 Pawel Szczur <[email protected]> wrote: > Hi, > > I'm trying to run a simple pipeline on local cluster using Protocol Buffer > to pass data between Beam functions. > > Everything works fine if I run it through: > java -jar target/dataflow-test-1.0-SNAPSHOT.jar > --runner=org.apache.beam.runners.flink.FlinkRunner > --input=/tmp/kinglear.txt --output=/tmp/wordcounts.txt > > But it fails when trying to run on flink cluster: > flink run target/dataflow-test-1.0-SNAPSHOT.jar > --runner=org.apache.beam.runners.flink.FlinkRunner > --input=/tmp/kinglear.txt --output=/tmp/wordcounts.txt > > The ready to run project: > https://github.com/orian/beam-flink-local-cluster > > Any clues? > > Cheers, Pawel > > ------------------------------------------------------------ > The program finished with the following exception: > > java.lang.NoSuchMethodError: > com.google.protobuf.ExtensionRegistry.getAllImmutableExtensionsByExtendedType(Ljava/lang/String;)Ljava/util/Set; > at com.mycompany.dataflow.WordCount.main(WordCount.java:109) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:505) > at > org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:403) > at org.apache.flink.client.program.Client.runBlocking(Client.java:248) > at > org.apache.flink.client.CliFrontend.executeProgramBlocking(CliFrontend.java:866) > at org.apache.flink.client.CliFrontend.run(CliFrontend.java:333) > at > org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1192) > at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1243) >
