Generally, any test that uses HDFS will fail on Windows. We've disabled
most of them, but some slip through from time to time.
Note that we do not provide any guarantees for all tests passing on Windows.
On 10.07.2018 16:28, NEKRASSOV, ALEXEI wrote:
I'm running 'mvn clean verify' on Windows with no Hadoop libraries installed,
and the build fails (see below).
What's the solution? Is there a switch to skip Hadoop-related tests?
Or I need to install Hadoop libraries?
Thanks,
Alex
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.726 sec <<<
FAILURE! - in org.apache.flink.runtime.fs.hdfs.HdfsBehaviorTest
org.apache.flink.runtime.fs.hdfs.HdfsBehaviorTest Time elapsed: 1.726 sec <<<
ERROR!
java.lang.UnsatisfiedLinkError:
org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native
Method)
at
org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:570)
at org.apache.hadoop.fs.FileUtil.canWrite(FileUtil.java:996)
at
org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:484)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:293)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:202)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:891)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:638)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:503)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:559)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:724)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:708)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1358)
at
org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCluster.java:996)
at
org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:867)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:702)
at
org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:374)
at
org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:355)
at
org.apache.flink.runtime.fs.hdfs.HdfsBehaviorTest.createHDFS(HdfsBehaviorTest.java:65)
Running org.apache.flink.runtime.fs.hdfs.HdfsKindTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.081 sec - in
org.apache.flink.runtime.fs.hdfs.HdfsKindTest
Running org.apache.flink.runtime.fs.hdfs.LimitedConnectionsConfigurationTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.017 sec - in
org.apache.flink.runtime.fs.hdfs.LimitedConnectionsConfigurationTest
Results :
Tests in error:
HdfsBehaviorTest.createHDFS:65 ▒ UnsatisfiedLink
org.apache.hadoop.io.nativeio...
Tests run: 24, Failures: 0, Errors: 1, Skipped: 1
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] force-shading ...................................... SUCCESS [ 2.335 s]
[INFO] flink .............................................. SUCCESS [ 29.794 s]
[INFO] flink-annotations .................................. SUCCESS [ 2.198 s]
[INFO] flink-shaded-hadoop ................................ SUCCESS [ 0.226 s]
[INFO] flink-shaded-hadoop2 ............................... SUCCESS [ 11.015 s]
[INFO] flink-shaded-hadoop2-uber .......................... SUCCESS [ 16.343 s]
[INFO] flink-shaded-yarn-tests ............................ SUCCESS [ 13.653 s]
[INFO] flink-shaded-curator ............................... SUCCESS [ 1.386 s]
[INFO] flink-test-utils-parent ............................ SUCCESS [ 0.191 s]
[INFO] flink-test-utils-junit ............................. SUCCESS [ 3.318 s]
[INFO] flink-metrics ...................................... SUCCESS [ 0.212 s]
[INFO] flink-metrics-core ................................. SUCCESS [ 3.502 s]
[INFO] flink-core ......................................... SUCCESS [01:30 min]
[INFO] flink-java ......................................... SUCCESS [01:31 min]
[INFO] flink-queryable-state .............................. SUCCESS [ 0.186 s]
[INFO] flink-queryable-state-client-java .................. SUCCESS [ 4.099 s]
[INFO] flink-filesystems .................................. SUCCESS [ 0.198 s]
[INFO] flink-hadoop-fs .................................... FAILURE [ 8.672 s]
