Hi everyone! We have been time and time again struck by the problem that Hadoop bundles many dependencies in certain versions, that conflict either with versions of the dependencies we use, or with versions that users use.
The most prominent examples are Guava and Protobuf. One way to solve this is to create ourselves a "custom" Hadoop dependency: We build a fat jar from Hadoop (with all its dependencies) and in this fat jar shade Guava and Protobuf. We can use the flink-shading project for this - it is already used to build a custom (shaded) version of Guava for Flink. Any opinions on this? Greetings, Stephan
