See <https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/556/changes>
Changes: [kihwal] HDFS-4937. ReplicationMonitor can infinite-loop in [kihwal] fix CHANGES.txt [jlowe] Creating 2.6.3 entries in CHANGES.txt files. [jlowe] Update CHANGES.txt to reflect commit of MR-6273 to branch-2.6 [jlowe] MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary(). ------------------------------------------ [...truncated 12972 lines...] TestReplicationPolicy.testChooseTarget5:413 null TestReplicationPolicy.testRereplicate1:745 null TestReplicationPolicy.testRereplicate3:800 null TestReplicationPolicy.testChooseTargetWithHalfStaleNodes:602 null TestBlockTokenWithDFS.testRead:355->doTestRead:475 null TestBlockManager.testTwoOfThreeNodesDecommissioned:180->doTestTwoOfThreeNodesDecommissioned:197 Should have three targets expected:<3> but was:<2> TestBlockManager.testAllNodesHoldingReplicasDecommissioned:224->doTestAllNodesHoldingReplicasDecommissioned:241 Should have three targets expected:<4> but was:<2> TestBlockManager.testOneOfTwoRacksDecommissioned:277->doTestOneOfTwoRacksDecommissioned:321->scheduleSingleReplication:464 computeBlockRecoveryWork should indicate replication is needed expected:<1> but was:<0> TestBlockManager.testSufficientlyReplBlocksUsesNewRack:337->doTestSufficientlyReplBlocksUsesNewRack:345->scheduleSingleReplication:464 computeBlockRecoveryWork should indicate replication is needed expected:<1> but was:<0> TestDFSStripedOutputStreamWithFailure160>TestDFSStripedOutputStreamWithFailure$TestBase.test3:492->TestDFSStripedOutputStreamWithFailure$TestBase.run:486 failed, dn=0, length=3538943java.io.IOException: Failed at i=1967615 at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.write(TestDFSStripedOutputStreamWithFailure.java:402) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTest(TestDFSStripedOutputStreamWithFailure.java:378) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTest(TestDFSStripedOutputStreamWithFailure.java:281) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure$TestBase.run(TestDFSStripedOutputStreamWithFailure.java:486) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure$TestBase.test3(TestDFSStripedOutputStreamWithFailure.java:492) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) Caused by: java.io.IOException: Failed: the number of failed blocks = 4 > the number of data blocks = 3 at org.apache.hadoop.hdfs.DFSStripedOutputStream.checkStreamers(DFSStripedOutputStream.java:359) at org.apache.hadoop.hdfs.DFSStripedOutputStream.checkStreamerFailures(DFSStripedOutputStream.java:580) at org.apache.hadoop.hdfs.DFSStripedOutputStream.writeChunk(DFSStripedOutputStream.java:523) at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:217) at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:164) at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:145) at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:79) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:48) at java.io.DataOutputStream.write(DataOutputStream.java:88) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.write(TestDFSStripedOutputStreamWithFailure.java:400) ... 13 more TestDFSStripedOutputStreamWithFailure160>TestDFSStripedOutputStreamWithFailure$TestBase.test5:494->TestDFSStripedOutputStreamWithFailure$TestBase.run:486 failed, dn=0, length=3538945java.io.IOException: Failed at i=1967615 at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.write(TestDFSStripedOutputStreamWithFailure.java:402) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTest(TestDFSStripedOutputStreamWithFailure.java:378) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTest(TestDFSStripedOutputStreamWithFailure.java:281) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure$TestBase.run(TestDFSStripedOutputStreamWithFailure.java:486) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure$TestBase.test5(TestDFSStripedOutputStreamWithFailure.java:494) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) Caused by: java.io.IOException: Failed: the number of failed blocks = 4 > the number of data blocks = 3 at org.apache.hadoop.hdfs.DFSStripedOutputStream.checkStreamers(DFSStripedOutputStream.java:359) at org.apache.hadoop.hdfs.DFSStripedOutputStream.checkStreamerFailures(DFSStripedOutputStream.java:580) at org.apache.hadoop.hdfs.DFSStripedOutputStream.writeChunk(DFSStripedOutputStream.java:523) at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:217) at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:164) at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:145) at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:79) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:48) at java.io.DataOutputStream.write(DataOutputStream.java:88) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.write(TestDFSStripedOutputStreamWithFailure.java:400) ... 13 more TestDFSStripedOutputStreamWithFailure190>TestDFSStripedOutputStreamWithFailure$TestBase.test6:495->TestDFSStripedOutputStreamWithFailure$TestBase.run:486 failed, dn=0, length=4259839java.lang.AssertionError: expected:<1003> but was:<1004> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.junit.Assert.assertEquals(Assert.java:542) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTest(TestDFSStripedOutputStreamWithFailure.java:362) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTest(TestDFSStripedOutputStreamWithFailure.java:281) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure$TestBase.run(TestDFSStripedOutputStreamWithFailure.java:486) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure$TestBase.test6(TestDFSStripedOutputStreamWithFailure.java:495) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) TestSaslDataTransfer.testAuthentication:80->doTest:185 expected:<3> but was:<2> TestSaslDataTransfer.testPrivacy:100->doTest:185 expected:<3> but was:<2> TestSaslDataTransfer.testIntegrity:90->doTest:185 expected:<3> but was:<2> TestReplication.testNoExtraReplicationWhenBlockReceivedIsLate:665 timed out while waiting for no pending replication. TestDFSStripedOutputStreamWithFailure.testAddBlockWhenNoSufficientDataBlockNumOfNodes:238 Expected to find 'Failed to get 6 nodes from namenode: blockGroupSize= 9, blocks.length= 5' but got unexpected exception:java.io.IOException: Failed to get 6 nodes from namenode: blockGroupSize= 9, blocks.length= 3 at org.apache.hadoop.hdfs.DFSStripedOutputStream.allocateNewBlock(DFSStripedOutputStream.java:443) at org.apache.hadoop.hdfs.DFSStripedOutputStream.writeChunk(DFSStripedOutputStream.java:482) at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:217) at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:164) at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:145) at org.apache.hadoop.hdfs.DFSStripedOutputStream.closeImpl(DFSStripedOutputStream.java:922) at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:753) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:101) at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.testAddBlockWhenNoSufficientDataBlockNumOfNodes(TestDFSStripedOutputStreamWithFailure.java:234) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) TestDFSStripedOutputStreamWithFailure.testBlockTokenExpired:199->runTest:362 expected:<1001> but was:<1002> TestEncryptedTransfer.testEncryptedAppendRequiringBlockTransfer:583 expected:<3> but was:<2> Tests in error: TestDecommission.testNodeUsageAfterDecommissioned » Remote File /user/jenkins/... TestDecommission.testPendingNodes:1111 » IO Unable to close file because the l... TestFileCreation.testFsCloseAfterClusterShutdown » Remote File /TestFileCreati... TestDFSStripedOutputStream.testFileSmallerThanOneCell1:80->testOneFile:155 » Timeout TestDFSStripedOutputStream.testFileSmallerThanOneCell2:85->testOneFile:155 » Timeout TestDFSStripedOutputStream.testFileMoreThanOneStripe1:110->testOneFile:155 » Timeout TestDFSStripedOutputStream.testFileSmallerThanOneStripe1:95->testOneFile:155 » Timeout TestDFSStripedOutputStream.testFileSmallerThanOneStripe2:100->testOneFile:155 » Timeout TestDFSStripedOutputStream.testFileMoreThanABlockGroup1:132->testOneFile:155 » Timeout TestDFSStripedOutputStream.testFileMoreThanABlockGroup2:137->testOneFile:155 » Timeout TestDFSStripedOutputStream.testFileMoreThanABlockGroup3:144->testOneFile:155 » Timeout TestDFSStripedOutputStream.testFileFullBlockGroup:127->testOneFile:155 » Timeout TestDFSStripedOutputStream.testFileLessThanFullBlockGroup:121->testOneFile:155 » Timeout TestPipelines.pipeline_01 » IO Failed to replace a bad datanode on the existin... TestBalancer.testTwoReplicaShouldNotInSameDN:1500->createFile:176 » Timeout Ti... TestBalancer.testBalancerWithStripedFile:1713->doTestBalancerWithStripedFile:1746 » TestHAAppend.testMultipleAppendsDuringCatchupTailing » IO Failed to replace a ... TestDiskspaceQuotaUpdate.testUpdateQuotaForAppend » IO Failed to replace a bad... TestSnapshotDiffReport.testDiffReportWithRenameAndAppend » IO Failed to replac... TestAddBlock.testAddBlockUC » IO Failed to replace a bad datanode on the exist... TestAddStripedBlocks.testAllocateBlockId:103 » IO Failed to get 6 nodes from n... TestAddStripedBlocks.testAddStripedBlock:124->writeAndFlushStripedOutputStream:113 » IO TestAddStripedBlocks.testGetLocatedStripedBlocks:202->writeAndFlushStripedOutputStream:113 » IO TestFileTruncate.testSnapshotWithAppendTruncate » IO Failed to replace a bad d... TestFsck.testFsckOpenECFiles:669 » IO Failed to get 6 nodes from namenode: blo... TestFSImageWithSnapshot.testSaveLoadImageWithAppending » IO Failed to replace ... TestDataNodeVolumeFailureReporting.testSuccessiveVolumeFailures » Remote File ... TestDataNodeVolumeFailureReporting.testVolFailureStatsPreservedOnNNRestart » Remote TestDataNodeMetrics.testTimeoutMetric » Remote File /test could only be replic... TestDataNodeVolumeFailure.testUnderReplicationAfterVolFailure » Remote File /t... TestDataNodeVolumeFailure.testVolumeFailure:146 » Timeout Timed out waiting fo... TestDiskError.testShutdown:110 » Timeout Timed out waiting for /test.txt0 to r... TestDataNodeHotSwapVolumes.testDirectlyReloadAfterCheckDiskError:740->createFile:140->createFile:156 » TestRBWBlockInvalidation.testBlockInvalidationWhenRBWReplicaMissedInDN » IO Al... TestRBWBlockInvalidation.testRWRInvalidation » IO All datanodes [DatanodeInfoW... TestUnderReplicatedBlocks.testNumberOfBlocksToBeReplicated:122 » Timeout Timed... TestNodeCount.testNodeCount:130->checkTimeout:146->checkTimeout:156 Timeout Ti... TestOverReplicatedBlocks.testChooseReplicaToDelete:166 » Timeout Timed out wai... TestReplicationPolicyWithNodeGroup.testRereplicateOnBoundaryTopology:631 ArrayIndexOutOfBounds TestBlockTokenWithDFSStriped.testEnd2End:89 » IO Failed to get 6 nodes from na... TestBlockTokenWithDFSStriped.testRead:62->TestBlockTokenWithDFS.doTestRead:407->isBlockTokenExpired:109->TestBlockTokenWithDFS.isBlockTokenExpired:628 » NullPointer TestWriteReadStripedFile.testFileMoreThanABlockGroup2:175->testOneFileUsingDFSStripedInputStream:199 » IO TestQuota.testMultipleFilesSmallerThanOneBlock:972 » Timeout Timed out waiting... TestDFSStripedOutputStreamWithFailure.testAddBlockWhenNoSufficientParityNumOfNodes:267 » IO TestDFSStripedOutputStreamWithFailure.testMultipleDatanodeFailure56:172->runTestWithMultipleFailure:301->runTest:378->write:402 IO TestEncryptedTransfer.testEncryptedAppend[0] » IO Failed to replace a bad data... TestEncryptedTransfer.testEncryptedAppendRequiringBlockTransfer[0] » IO Failed... Tests run: 3397, Failures: 73, Errors: 45, Skipped: 17 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Skipping Apache Hadoop HDFS Native Client [INFO] This project has been banned from the build due to previous failures. [INFO] ------------------------------------------------------------------------ [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Skipping Apache Hadoop HttpFS [INFO] This project has been banned from the build due to previous failures. [INFO] ------------------------------------------------------------------------ [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Skipping Apache Hadoop HDFS BookKeeper Journal [INFO] This project has been banned from the build due to previous failures. [INFO] ------------------------------------------------------------------------ [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Skipping Apache Hadoop HDFS-NFS [INFO] This project has been banned from the build due to previous failures. [INFO] ------------------------------------------------------------------------ [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Apache Hadoop HDFS Project 3.0.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hadoop-hdfs-project --- [INFO] Deleting <https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/ws/hadoop-hdfs-project/target> [INFO] [INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-hdfs-project --- [INFO] Executing tasks main: [mkdir] Created dir: <https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/ws/hadoop-hdfs-project/target/test-dir> [INFO] Executed tasks [INFO] [INFO] --- maven-source-plugin:2.3:jar-no-fork (hadoop-java-sources) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-source-plugin:2.3:test-jar-no-fork (hadoop-java-sources) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ hadoop-hdfs-project --- [INFO] Not executing Javadoc as the project is not a Java classpath-capable package [INFO] [INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project --- [INFO] [INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ hadoop-hdfs-project --- [INFO] [INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ hadoop-hdfs-project --- [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache Hadoop HDFS Client ......................... SUCCESS [03:58 min] [INFO] Apache Hadoop HDFS ................................ FAILURE [ 03:20 h] [INFO] Apache Hadoop HDFS Native Client .................. SKIPPED [INFO] Apache Hadoop HttpFS .............................. SKIPPED [INFO] Apache Hadoop HDFS BookKeeper Journal ............. SKIPPED [INFO] Apache Hadoop HDFS-NFS ............................ SKIPPED [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [ 0.066 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:24 h [INFO] Finished at: 2015-10-30T19:24:04+00:00 [INFO] Final Memory: 73M/670M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project hadoop-hdfs: ExecutionException: java.lang.RuntimeException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called? [ERROR] Command was /bin/sh -c cd <https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/ws/hadoop-hdfs-project/hadoop-hdfs> && /home/jenkins/tools/java/jdk1.8.0/jre/bin/java -Xmx2048m -XX:MaxPermSize=768m -XX:+HeapDumpOnOutOfMemoryError -jar <https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/ws/hadoop-hdfs-project/hadoop-hdfs/target/surefire/surefirebooter3881843624911135906.jar> <https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/ws/hadoop-hdfs-project/hadoop-hdfs/target/surefire/surefire6652811198879516247tmp> <https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/ws/hadoop-hdfs-project/hadoop-hdfs/target/surefire/surefire_3886736626828458841961tmp> [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hadoop-hdfs Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results Updating HDFS-4937 Updating MAPREDUCE-6528