Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/4939#discussion_r149980302
--- Diff: flink-yarn/pom.xml ---
@@ -153,6 +159,63 @@ under the License.
</plugins>
</build>
</profile>
+
+ <profile>
+ <!-- Hadoop >= 2.6 moved the S3 file systems from
hadoop-common into hadoop-aws artifact
+ (see
https://issues.apache.org/jira/browse/HADOOP-11074)
+ We can add the (test) dependency per default
once 2.6 is the minimum required version.
+ -->
+ <id>include_hadoop_aws</id>
+ <activation>
+ <property>
+ <name>include_hadoop_aws</name>
+ </property>
+ </activation>
+ <dependencies>
+ <!-- for the S3 tests of
YarnFileStageTestS3ITCase -->
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-aws</artifactId>
+ <version>${hadoop.version}</version>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+
<groupId>org.apache.avro</groupId>
+
<artifactId>avro</artifactId>
+ </exclusion>
+ <!-- The aws-java-sdk-core
requires jackson 2.6, but
+ hadoop pulls in 2.3 -->
+ <exclusion>
+
<groupId>com.fasterxml.jackson.core</groupId>
+
<artifactId>jackson-annotations</artifactId>
+ </exclusion>
+ <exclusion>
+
<groupId>com.fasterxml.jackson.core</groupId>
+
<artifactId>jackson-core</artifactId>
+ </exclusion>
+ <exclusion>
+
<groupId>com.fasterxml.jackson.core</groupId>
+
<artifactId>jackson-databind</artifactId>
+ </exclusion>
--- End diff --
Can't we enforce jackson 2.6 via dependency management? I think this would
be cleaner than excluding the dependencies here and assume that
`aws-java-sdk-s3` pulls in the missing dependencies.
---
