Hi Filip, Have you tried putting protobuf-java 3.7.1 into the Flink's lib/ folder? Or maybe re-writing the dependencies you mentioned to be loaded as plugins? [1]
I don't see any other ways to solve this problem. Probably Chesnay or Seth will suggest a better solution. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/filesystems/plugins/ Regards, Roman On Fri, Mar 4, 2022 at 9:54 AM Filip Karnicki <filip.karni...@gmail.com> wrote: > > Hi All! > > We're running a statefun uber jar on a shared cloudera flink cluster, the > latter of which launches with some ancient protobuf dependencies because of > reasons[1]. > > Setting the following flink-config settings on the entire cluster > > classloader.parent-first-patterns.additional: > org.apache.flink.statefun;org.apache.kafka;com.google.protobuf > > causes these old protobuf dependencies to get loaded over statefun's > protobuf-java 3.7.1, and NoSuchMethod exceptions occur. > > We hacked together a version of statefun that doesn't perform the check > whether the classloader settings contain the three patterns from above, and > as long as our job uses protobouf-java 3.7.1 and the com.google.protobuf > pattern is not present in the classloader.parent-first-patterns.additional > setting, then all is well. > > Aside from removing old hadoop from the classpath, which may not be possible > given that it's a shared cluster, is there anything we can do other than > adding a configurable override not to perform the config check in > StatefulFunctionsConfigValidator to an upcoming statefun core release? > > Many thanks > Fil > > > [1] We're still trying to find out if it's absolutely necessary to have these > on the classpath.
