[ https://issues.apache.org/jira/browse/FLINK-16400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050298#comment-17050298 ]
Robert Metzger commented on FLINK-16400: ---------------------------------------- The same error also occurs in the {{YarnFileStageTestS3ITCase}}: {code:java} 17:16:23.508 [INFO] Running org.apache.flink.yarn.YarnFileStageTestS3ITCase 17:16:29.337 [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.826 s <<< FAILURE! - in org.apache.flink.yarn.YarnFileStageTestS3ITCase 17:16:29.337 [ERROR] testRecursiveUploadForYarnS3a(org.apache.flink.yarn.YarnFileStageTestS3ITCase) Time elapsed: 0.071 s <<< ERROR! org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find a file system implementation for scheme 's3a'. The scheme is directly supported by Flink through the following plugin: flink-s3-fs-hadoop. Please ensure that each plugin resides within its own subfolder within the plugins directory. See https://ci.apache.org/projects/flink/flink-docs-stable/ops/plugins.html for more information. If you want to use a Hadoop file system for that scheme, please add the scheme to the configuration fs.allowed-fallback-filesystems. For a full list of supported file systems, please see https://ci.apache.org/projects/flink/flink-docs-stable/ops/filesystems/. at org.apache.flink.yarn.YarnFileStageTestS3ITCase.testRecursiveUploadForYarn(YarnFileStageTestS3ITCase.java:157) at org.apache.flink.yarn.YarnFileStageTestS3ITCase.testRecursiveUploadForYarnS3a(YarnFileStageTestS3ITCase.java:197) 17:16:29.368 [INFO] 17:16:29.368 [INFO] Results: {code} In this run: https://travis-ci.org/apache/flink/jobs/657296271 Interestingly, it does not surface on the same run on AZP: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=5843&view=logs&j=c2f345e3-6738-50c0-333e-11265e9cd7e4&t=bfc49226-e770-5168-1d5a-8fe08e0d5386 It logs {code} 2020-03-03T01:32:52.8933937Z [INFO] T E S T S 2020-03-03T01:32:52.8934558Z [INFO] ------------------------------------------------------- 2020-03-03T01:32:53.1954466Z [INFO] Running org.apache.flink.yarn.YarnFileStageTestS3ITCase 2020-03-03T01:32:53.6854001Z [WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.488 s - in org.apache.flink.yarn.YarnFileStageTestS3ITCase 2020-03-03T01:32:54.0205161Z [INFO] 2020-03-03T01:32:54.0206010Z [INFO] Results: {code} ... so it seems the test was skipped because the {{NativeS3FileSystem}} was not in the classpath. [~chesnay] Do you have an idea why this is happening? The only difference I can see between these two tests is that on Travis, we are using {{PROFILE="-Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.12 -Phive-1.2.1"}}, while on azure, it is {{PROFILE="-Dinclude-hadoop -Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.12 -Phive-1.2.1"}} (-Dinclude-hadoop is set on AZP). > HdfsKindTest.testS3Kind fails in Hadoop 2.4.1 nightly test > ---------------------------------------------------------- > > Key: FLINK-16400 > URL: https://issues.apache.org/jira/browse/FLINK-16400 > Project: Flink > Issue Type: Bug > Components: FileSystems, Tests > Reporter: Robert Metzger > Priority: Major > Labels: test-stability > > Log: > [https://dev.azure.com/rmetzger/Flink/_build/results?buildId=5843&view=logs&j=f8cdcc9b-111a-5332-0026-209cb3eb5d15&t=57d35dc9-027e-5d4a-fbeb-1c24315e6ffb] > and: [https://travis-ci.org/apache/flink/jobs/657296261] > {code:java} > 15:57:21.539 [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time > elapsed: 0.291 s <<< FAILURE! - in > org.apache.flink.runtime.fs.hdfs.HdfsKindTest > 15:57:21.552 [ERROR] > testS3Kind(org.apache.flink.runtime.fs.hdfs.HdfsKindTest) Time elapsed: > 0.032 s <<< ERROR! > org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find > a file system implementation for scheme 's3'. The scheme is directly > supported by Flink through the following plugins: flink-s3-fs-hadoop, > flink-s3-fs-presto. Please ensure that each plugin resides within its own > subfolder within the plugins directory. See > https://ci.apache.org/projects/flink/flink-docs-stable/ops/plugins.html for > more information. If you want to use a Hadoop file system for that scheme, > please add the scheme to the configuration fs.allowed-fallback-filesystems. > For a full list of supported file systems, please see > https://ci.apache.org/projects/flink/flink-docs-stable/ops/filesystems/. > at > org.apache.flink.runtime.fs.hdfs.HdfsKindTest.testS3Kind(HdfsKindTest.java:57) > 15:57:21.574 [INFO] Running > org.apache.flink.runtime.fs.hdfs.HadoopRecoverableWriterOldHadoopWithNoTruncateSupportTest > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)