I added lines below to flink-hadoop-fs/pom.xml, and that allowed me to turn off
the tests that were failing for me.
Do we want to add this change to master?
If so, do I need to document this new switch somewhere?
(
the build then hang for me at flink-runtime, but that's a different issue
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.969 sec - in
org.apache.flink.runtime.taskmanager.TaskManagerRegistrationTest
)
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>${skipHdfsTests}</skipTests>
</configuration>
</plugin>
</plugins>
</build>
-----Original Message-----
From: Chesnay Schepler [mailto:ches...@apache.org]
Sent: Tuesday, July 10, 2018 10:36 AM
To: dev@flink.apache.org; NEKRASSOV, ALEXEI <an4...@att.com>
Subject: Re: 'mvn verify' fails at flink-hadoop-fs
There's currently no workaround except going in and manually disabling them.
On 10.07.2018 16:32, Chesnay Schepler wrote:
> Generally, any test that uses HDFS will fail on Windows. We've
> disabled most of them, but some slip through from time to time.
>
> Note that we do not provide any guarantees for all tests passing on
> Windows.
>
> On 10.07.2018 16:28, NEKRASSOV, ALEXEI wrote:
>> I'm running 'mvn clean verify' on Windows with no Hadoop libraries
>> installed, and the build fails (see below).
>> What's the solution? Is there a switch to skip Hadoop-related tests?
>> Or I need to install Hadoop libraries?
>>
>> Thanks,
>> Alex
>>
>>
>> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.726
>> sec <<< FAILURE! - in
>> org.apache.flink.runtime.fs.hdfs.HdfsBehaviorTest
>> org.apache.flink.runtime.fs.hdfs.HdfsBehaviorTest Time elapsed:
>> 1.726 sec <<< ERROR!
>> java.lang.UnsatisfiedLinkError:
>> org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
>> at
>> org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method)
>> at
>> org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:570)
>> at org.apache.hadoop.fs.FileUtil.canWrite(FileUtil.java:996)
>> at
>> org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:484)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:293)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:202)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:891)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:638)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:503)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:559)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:724)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:708)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1358)
>> at
>> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCluster.java:996)
>> at
>> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:867)
>> at
>> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:702)
>> at
>> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:374)
>> at
>> org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:355)
>> at
>> org.apache.flink.runtime.fs.hdfs.HdfsBehaviorTest.createHDFS(HdfsBeha
>> viorTest.java:65)
>>
>> Running org.apache.flink.runtime.fs.hdfs.HdfsKindTest
>> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.081
>> sec - in org.apache.flink.runtime.fs.hdfs.HdfsKindTest
>> Running
>> org.apache.flink.runtime.fs.hdfs.LimitedConnectionsConfigurationTest
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.017
>> sec - in
>> org.apache.flink.runtime.fs.hdfs.LimitedConnectionsConfigurationTest
>>
>> Results :
>>
>> Tests in error:
>> HdfsBehaviorTest.createHDFS:65 ▒ UnsatisfiedLink
>> org.apache.hadoop.io.nativeio...
>>
>> Tests run: 24, Failures: 0, Errors: 1, Skipped: 1
>>
>> [INFO]
>> ---------------------------------------------------------------------
>> ---
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] force-shading ...................................... SUCCESS [
>> 2.335 s] [INFO] flink ..............................................
>> SUCCESS [
>> 29.794 s]
>> [INFO] flink-annotations .................................. SUCCESS [
>> 2.198 s] [INFO] flink-shaded-hadoop ................................
>> SUCCESS [ 0.226 s] [INFO] flink-shaded-hadoop2
>> ............................... SUCCESS [
>> 11.015 s]
>> [INFO] flink-shaded-hadoop2-uber .......................... SUCCESS [
>> 16.343 s]
>> [INFO] flink-shaded-yarn-tests ............................ SUCCESS [
>> 13.653 s]
>> [INFO] flink-shaded-curator ............................... SUCCESS [
>> 1.386 s] [INFO] flink-test-utils-parent ............................
>> SUCCESS [ 0.191 s] [INFO] flink-test-utils-junit
>> ............................. SUCCESS [ 3.318 s] [INFO]
>> flink-metrics ...................................... SUCCESS [ 0.212
>> s] [INFO] flink-metrics-core .................................
>> SUCCESS [ 3.502 s] [INFO] flink-core
>> ......................................... SUCCESS
>> [01:30 min]
>> [INFO] flink-java ......................................... SUCCESS
>> [01:31 min]
>> [INFO] flink-queryable-state .............................. SUCCESS [
>> 0.186 s] [INFO] flink-queryable-state-client-java ..................
>> SUCCESS [ 4.099 s] [INFO] flink-filesystems
>> .................................. SUCCESS [ 0.198 s] [INFO]
>> flink-hadoop-fs .................................... FAILURE [ 8.672
>> s]
>>
>>
>>
>>
>
>
