[FLINK-7420] [avro] (follow-up) Do not shade or exclude avro in Hadoop Excluding Avro in Hadoop will make basic classes like sequence file writers fail to load, even if Avro is not used.
Shading Avro is not possible because it is part of the API for users that use Avro types. Project: http://git-wip-us.apache.org/repos/asf/flink/repo Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/db7c70fa Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/db7c70fa Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/db7c70fa Branch: refs/heads/master Commit: db7c70faa0b5aed652b525e8091cdc0e265362bc Parents: 537a10e Author: Stephan Ewen <[email protected]> Authored: Fri Nov 3 12:42:21 2017 +0100 Committer: Stephan Ewen <[email protected]> Committed: Fri Nov 3 16:40:34 2017 +0100 ---------------------------------------------------------------------- flink-formats/flink-avro/pom.xml | 2 +- flink-shaded-hadoop/flink-shaded-hadoop2/pom.xml | 19 +++++++------------ pom.xml | 7 +++++++ 3 files changed, 15 insertions(+), 13 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/flink/blob/db7c70fa/flink-formats/flink-avro/pom.xml ---------------------------------------------------------------------- diff --git a/flink-formats/flink-avro/pom.xml b/flink-formats/flink-avro/pom.xml index 19d9129..a0f63be 100644 --- a/flink-formats/flink-avro/pom.xml +++ b/flink-formats/flink-avro/pom.xml @@ -55,7 +55,7 @@ under the License. <dependency> <groupId>org.apache.avro</groupId> <artifactId>avro</artifactId> - <version>1.8.2</version> + <!-- managed version --> </dependency> <!-- test dependencies --> http://git-wip-us.apache.org/repos/asf/flink/blob/db7c70fa/flink-shaded-hadoop/flink-shaded-hadoop2/pom.xml ---------------------------------------------------------------------- diff --git a/flink-shaded-hadoop/flink-shaded-hadoop2/pom.xml b/flink-shaded-hadoop/flink-shaded-hadoop2/pom.xml index 864c257..66d8549 100644 --- a/flink-shaded-hadoop/flink-shaded-hadoop2/pom.xml +++ b/flink-shaded-hadoop/flink-shaded-hadoop2/pom.xml @@ -43,6 +43,13 @@ under the License. <!-- managed version --> </dependency> + <!-- explicitly include avro to force Flink's managed version --> + <dependency> + <groupId>org.apache.avro</groupId> + <artifactId>avro</artifactId> + <!-- managed version --> + </dependency> + <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> @@ -161,12 +168,6 @@ under the License. <groupId>commons-beanutils</groupId> <artifactId>commons-beanutils</artifactId> </exclusion> - - <!-- we don't want Hadoop's Avro dependency, since Flink adds its own Avro support --> - <exclusion> - <groupId>org.apache.avro</groupId> - <artifactId>avro</artifactId> - </exclusion> </exclusions> </dependency> @@ -418,12 +419,6 @@ under the License. <groupId>com.sun.jersey.contribs</groupId> <artifactId>jersey-guice</artifactId> </exclusion> - - <!-- we don't want Hadoop's Avro dependency, since Flink adds its own Avro support --> - <exclusion> - <groupId>org.apache.avro</groupId> - <artifactId>avro</artifactId> - </exclusion> </exclusions> </dependency> http://git-wip-us.apache.org/repos/asf/flink/blob/db7c70fa/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 5302c5c..773dc34 100644 --- a/pom.xml +++ b/pom.xml @@ -282,6 +282,13 @@ under the License. <version>1.1.1.3</version> </dependency> + <!-- Make sure we use a consistent avro version between Flink and Hadoop --> + <dependency> + <groupId>org.apache.avro</groupId> + <artifactId>avro</artifactId> + <version>1.8.2</version> + </dependency> + <!-- Make sure we use a consistent commons-cli version throughout the project --> <dependency> <groupId>commons-cli</groupId>
