[jira] [Created] (HDFS-10423) Increase default value of httpfs maxHttpHeaderSize

[Nicolae Popa (JIRA)]

Nicolae Popa created HDFS-10423:
---

 Summary: Increase default value of httpfs maxHttpHeaderSize
 Key: HDFS-10423
 URL: https://issues.apache.org/jira/browse/HDFS-10423
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs
Reporter: Nicolae Popa
Priority: Minor


The Tomcat default value of maxHttpHeaderSize is 8k, which is too low for 
certain Hadoop workloads in kerberos enabled environments. This JIRA will to 
change it to 65536 in server.xml



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Hadoop-Hdfs-trunk-Java8 - Build # 1232 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1232/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 5425 lines...]
[INFO] --- maven-source-plugin:2.3:jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-source-plugin:2.3:test-jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [04:08 min]
[INFO] Apache Hadoop HDFS . FAILURE [  01:14 h]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.085 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:18 h
[INFO] Finished at: 2016-05-18T04:04:18+00:00
[INFO] Final Memory: 110M/3693M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There was a timeout or other error in the fork -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
2 tests failed.
FAILED:  
org.apache.hadoop.hdfs.server.datanode.TestDataNodeMXBean.testDataNodeMXBeanBlockCount

Error Message:
After delete one file expected:<4> but was:<5>

Stack Trace:
java.lang.AssertionError: After delete one file expected:<4> but was:<5>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at 
org.apache.hadoop.hdfs.server.datanode.TestDataNodeMXBean.testDataNodeMXBeanBlockCount(TestDataNodeMXBean.java:120)


FAILED:  org.apache.hadoop.hdfs.TestRollingUpgrade.testRollback

Error Message:
Test resulted in an unexpected exit

Stack Trace:
java.lang.AssertionError: Test resulted in an unexpected exit
at 
org.a

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1232

[Apache Jenkins Server]

See 

Changes:

[Arun Suresh] YARN-5090. Add End-to-End test-cases for DistributedScheduling 
using

--
[...truncated 5228 lines...]
Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 84.137 sec - 
in org.apache.hadoop.hdfs.TestEncryptionZones
Running org.apache.hadoop.hdfs.protocolPB.TestPBHelper
Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.8 sec - in 
org.apache.hadoop.hdfs.protocolPB.TestPBHelper
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.936 sec - in 
org.apache.hadoop.hdfs.TestDFSShellGenericOptions
Running org.apache.hadoop.hdfs.TestReconstructStripedFile
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 60.18 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStream
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 102.278 sec - 
in org.apache.hadoop.hdfs.TestReconstructStripedFile
Running org.apache.hadoop.hdfs.TestDFSInputStream
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 152.012 sec - 
in org.apache.hadoop.hdfs.TestWriteReadStripedFile
Running org.apache.hadoop.hdfs.TestDFSRemove
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.564 sec - in 
org.apache.hadoop.hdfs.TestDFSInputStream
Running org.apache.hadoop.hdfs.TestFileAppend4
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.801 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStream
Running org.apache.hadoop.hdfs.TestParallelRead
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.478 sec - in 
org.apache.hadoop.hdfs.TestDFSRemove
Running org.apache.hadoop.hdfs.TestClose
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.947 sec - in 
org.apache.hadoop.hdfs.TestClose
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.487 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040
Running org.apache.hadoop.hdfs.TestDFSAddressConfig
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.029 sec - in 
org.apache.hadoop.hdfs.TestDFSAddressConfig
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.366 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
Running org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.725 sec - in 
org.apache.hadoop.hdfs.TestFileAppend4
Running org.apache.hadoop.hdfs.TestLargeBlock
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.194 sec - in 
org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead
Running org.apache.hadoop.hdfs.TestHDFSTrash
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.085 sec - in 
org.apache.hadoop.hdfs.TestParallelRead
Running org.apache.hadoop.hdfs.TestClientReportBadBlock
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.734 sec - in 
org.apache.hadoop.hdfs.TestLargeBlock
Running org.apache.hadoop.hdfs.TestWriteRead
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.018 sec - in 
org.apache.hadoop.hdfs.TestHDFSTrash
Running org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.694 sec - in 
org.apache.hadoop.hdfs.TestClientReportBadBlock
Running org.apache.hadoop.hdfs.TestBalancerBandwidth
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.481 sec - in 
org.apache.hadoop.hdfs.TestWriteRead
Running org.apache.hadoop.hdfs.TestDFSUpgrade
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.664 sec - in 
org.apache.hadoop.hdfs.TestBalancerBandwidth
Running org.apache.hadoop.hdfs.TestBlockStoragePolicy
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.463 sec - in 
org.apache.hadoop.hdfs.TestDFSUpgrade
Running org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.511 sec - in 
org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
Running org.apache.hadoop.hdfs.TestLeaseRecoveryStriped
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 62.717 sec - in 
org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocal
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.603 sec - 
in org.apache.hadoop.hdfs.TestBlockStoragePolicy
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.951 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy
Running or

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1231

[Apache Jenkins Server]

See 

Changes:

[wang] HADOOP-13157. Follow-on improvements to hadoop credential commands.

--
[...truncated 51645 lines...]
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 42.798 sec - in 
org.apache.hadoop.hdfs.TestParallelRead
Running org.apache.hadoop.hdfs.TestClientReportBadBlock
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.484 sec - in 
org.apache.hadoop.hdfs.TestHDFSTrash
Running org.apache.hadoop.hdfs.TestWriteRead
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.043 sec - in 
org.apache.hadoop.hdfs.TestLargeBlock
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.88 sec - in 
org.apache.hadoop.hdfs.TestClientReportBadBlock
Running org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Running org.apache.hadoop.hdfs.TestBalancerBandwidth
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.879 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040
Running org.apache.hadoop.hdfs.TestDFSUpgrade
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.993 sec - in 
org.apache.hadoop.hdfs.TestBalancerBandwidth
Running org.apache.hadoop.hdfs.TestBlockStoragePolicy
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 29.419 sec - in 
org.apache.hadoop.hdfs.TestWriteRead
Running org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.969 sec - in 
org.apache.hadoop.hdfs.TestDFSUpgrade
Running org.apache.hadoop.hdfs.TestLeaseRecoveryStriped
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.722 sec - in 
org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocal
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.823 sec - in 
org.apache.hadoop.hdfs.TestLeaseRecoveryStriped
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 64.674 sec - in 
org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote2
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.783 sec - 
in org.apache.hadoop.hdfs.TestBlockStoragePolicy
Running org.apache.hadoop.hdfs.client.impl.TestClientBlockVerification
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.887 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote2
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.344 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderFactory
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.705 sec - in 
org.apache.hadoop.hdfs.client.impl.TestClientBlockVerification
Running org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.089 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote
Running org.apache.hadoop.hdfs.qjournal.TestNNWithQJM
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.627 sec - in 
org.apache.hadoop.hdfs.qjournal.TestNNWithQJM
Running org.apache.hadoop.hdfs.qjournal.server.TestJournalNodeMXBean
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.649 sec - in 
org.apache.hadoop.hdfs.qjournal.server.TestJournalNodeMXBean
Running org.apache.hadoop.hdfs.qjournal.server.TestJournal
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.699 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderFactory
Running org.apache.hadoop.hdfs.qjournal.server.TestJournalNode
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.891 sec - in 
org.apache.hadoop.hdfs.qjournal.server.TestJournal
Running org.apache.hadoop.hdfs.qjournal.client.TestIPCLoggerChannel
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.948 sec - in 
org.apache.hadoop.hdfs.qjournal.client.TestIPCLoggerChannel
Running org.apache.hadoop.hdfs.qjournal.client.TestQJMWithFaults
Tests run: 37, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 76.242 sec - 
in org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocal
Running org.apache.hadoop.hdfs.qjournal.client.TestEpochsAreUnique
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.246 sec - in 
org.apache.hadoop.hdfs.qjournal.server.TestJournalNode
Running org.apache.hadoop.hdfs.qjournal.client.TestQuorumJournalManagerUnit
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.241 sec - in 
org.apache.hadoop.hdfs.qjournal.client.TestEpochsAreUnique
Running org.apache.hadoop.hdfs.qjournal.client.TestSegmentRecoveryComparator
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.245 sec - in 
org.

Hadoop-Hdfs-trunk-Java8 - Build # 1231 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1231/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 51842 lines...]
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [04:15 min]
[INFO] Apache Hadoop HDFS . FAILURE [  01:11 h]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.666 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:15 h
[INFO] Finished at: 2016-05-18T02:16:22+00:00
[INFO] Final Memory: 120M/4010M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/source/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
3 tests failed.
FAILED:  org.apache.hadoop.hdfs.TestRollingUpgrade.testRollback

Error Message:
Test resulted in an unexpected exit

Stack Trace:
java.lang.AssertionError: Test resulted in an unexpected exit
at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1908)
at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1895)
at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1888)
at 
org.apache.hadoop.hdfs.TestRollingUpgrade.testRollback(TestRollingUpgrade.java:351)


FAILED:  
org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten

Error Message:
Failed to replace a bad datanode on the existing pipeline due to no more good 
datanodes being available to try. (Nodes: 
current=[DatanodeInfoWithStorage[127.0.0.1:3474

[jira] [Created] (HDFS-10422) Do not throw AccessControlException for {{security.hdfs.unreadable.by.superuser}} ops

[Tianyin Xu (JIRA)]

Tianyin Xu created HDFS-10422:
-

 Summary: Do not throw AccessControlException for 
{{security.hdfs.unreadable.by.superuser}} ops
 Key: HDFS-10422
 URL: https://issues.apache.org/jira/browse/HDFS-10422
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: namenode
Affects Versions: 2.7.2
Reporter: Tianyin Xu


The xattr {{security.hdfs.unreadable.by.superuser}} has certain syntax rules: 
"this xattr is also write-once, and cannot be removed once set. This xattr does 
not allow a value to be set."
[https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/ExtendedAttributes.html]

Upon violation, the system should throw {{IllegalArgumentException}}, the same 
behaviour like the other xattrs such as {{raw.hdfs.crypto.encryption.zone}}. 

Currently, it always throws {{AccessControlException}} which is supposed to be 
something related to access control. The only case {{AccessControlException}} 
should be thrown is when the superuser tries to access the file, but *not* 
anything else. 

If the user set any value, it violates the syntax of the xattr (this xattr does 
not need values) which has nothing to do with access control:
{code:title=XAttrPermissionFilter.java|borderStyle=solid}
if (XAttrHelper.getPrefixedName(xAttr).
equals(SECURITY_XATTR_UNREADABLE_BY_SUPERUSER)) {
  if (xAttr.getValue() != null) {
throw new AccessControlException("Attempt to set a value for '" +
SECURITY_XATTR_UNREADABLE_BY_SUPERUSER +
"'. Values are not allowed for this xattr.");
  }   
  return;
}   
{code} 
\\

Similarly, if the user wants to remove the xattr, it should throw 
{{IllegalArgumentException}} just like the encryption xattr in the following 
code. Basically I don't understand why the encryption xattr throws an 
{{IllegalArgumentException}} (via the {{checkArgument}} call) but the 
{{security.hdfs.unreadable.by.superuser}} xattr throws an 
{{AccessControlException}}. It's the same thing here: both of them cannot be 
removed...
{code:title=XAttrPermissionFilter.java|borderStyle=solid}
Preconditions.checkArgument(
!KEYID_XATTR.equalsIgnoreValue(filter),
"The encryption zone xattr should never be deleted.");
if (UNREADABLE_BY_SUPERUSER_XATTR.equalsIgnoreValue(filter)) {
  throw new AccessControlException("The xattr '" +
  SECURITY_XATTR_UNREADABLE_BY_SUPERUSER + "' can not be deleted.");
}   
{code} 





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Hadoop-Hdfs-trunk-Java8 - Build # 1230 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1230/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 8503 lines...]
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [04:14 min]
[INFO] Apache Hadoop HDFS . FAILURE [55:54 min]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.082 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:00 h
[INFO] Finished at: 2016-05-18T00:55:20+00:00
[INFO] Final Memory: 96M/3878M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
ERROR: Step ?Publish JUnit test result report? failed: No test report files 
were found. Configuration error?
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
No tests ran.

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1230

[Apache Jenkins Server]

See 

Changes:

[xyao] HADOOP-13159. Fix potential NPE in Metrics2 source for

[jing9] HDFS-10397. Distcp should ignore -delete option if -diff option is

[yzhang] HDFS-10381, DataStreamer DataNode exclusion log message should be

[xyao] HDFS-10383. Safely close resources in DFSTestUtil. Contributed by

--
[...truncated 8306 lines...]
Running org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.182 sec - in 
org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
Running org.apache.hadoop.hdfs.web.TestAuthFilter
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.649 sec - in 
org.apache.hadoop.hdfs.web.TestAuthFilter
Running org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 138.426 sec - 
in org.apache.hadoop.hdfs.TestDFSClientRetries
Running org.apache.hadoop.hdfs.web.TestWebHdfsTokens
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.208 sec - in 
org.apache.hadoop.hdfs.web.TestWebHdfsTokens
Running org.apache.hadoop.hdfs.web.TestJsonUtil
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.61 sec - in 
org.apache.hadoop.hdfs.web.TestJsonUtil
Running org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs
Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 82.56 sec - in 
org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
Running org.apache.hadoop.hdfs.web.TestWebHDFSAcl
Tests run: 52, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.026 sec - 
in org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs
Running org.apache.hadoop.hdfs.web.TestWebHdfsUrl
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.791 sec - in 
org.apache.hadoop.hdfs.web.TestWebHdfsUrl
Running org.apache.hadoop.hdfs.web.TestWebHDFSForHA
Tests run: 64, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 28.857 sec - 
in org.apache.hadoop.hdfs.web.TestWebHDFSAcl
Running org.apache.hadoop.hdfs.web.TestHttpsFileSystem
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.495 sec - in 
org.apache.hadoop.hdfs.web.TestWebHDFSForHA
Running org.apache.hadoop.hdfs.web.resources.TestParam
Tests run: 29, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.659 sec - in 
org.apache.hadoop.hdfs.web.resources.TestParam
Running org.apache.hadoop.hdfs.web.TestWebHdfsWithAuthenticationFilter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.903 sec - in 
org.apache.hadoop.hdfs.web.TestHttpsFileSystem
Running org.apache.hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.58 sec - in 
org.apache.hadoop.hdfs.web.TestWebHdfsWithAuthenticationFilter
Running org.apache.hadoop.hdfs.web.TestWebHDFSXAttr
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.633 sec - 
in org.apache.hadoop.hdfs.web.TestWebHDFSXAttr
Running org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.152 sec - in 
org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.363 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130
Running org.apache.hadoop.hdfs.TestHDFSTrash
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 129.072 sec - 
in org.apache.hadoop.hdfs.web.TestWebHDFS
Running org.apache.hadoop.net.TestNetworkTopology
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.726 sec - in 
org.apache.hadoop.hdfs.TestHDFSTrash
Running org.apache.hadoop.security.TestPermission
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.573 sec - in 
org.apache.hadoop.net.TestNetworkTopology
Running org.apache.hadoop.security.TestRefreshUserMappings
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.527 sec - in 
org.apache.hadoop.security.TestPermission
Running org.apache.hadoop.security.TestPermissionSymlinks
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.293 sec - in 
org.apache.hadoop.security.TestRefreshUserMappings
Running org.apache.hadoop.cli.TestDeleteCLI
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.11 sec - in 
org.apache.hadoop.security.TestPermissionSymlinks
Running org.apache.hadoop.cli.TestCacheAdminCLI
Tests run: 32, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 43.421 sec - 
in org.apache.hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter
Running org.apache.hadoop.cli.TestAclCLI
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.233 sec - in 
org.apache.hadoop.cli.TestCacheAdminCLI
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.319 sec - in 
org.apache.hadoop.cli.TestDeleteCLI
Tests run: 1, Failures: 0

[jira] [Created] (HDFS-10421) The nameservice for a Hadoop Namenode HA should be discoverable across clusters

[Mahipal Jupalli (JIRA)]

Mahipal Jupalli created HDFS-10421:
--

 Summary: The nameservice for a Hadoop Namenode HA should be 
discoverable across clusters
 Key: HDFS-10421
 URL: https://issues.apache.org/jira/browse/HDFS-10421
 Project: Hadoop HDFS
  Issue Type: New Feature
  Components: ha, hdfs-client
Reporter: Mahipal Jupalli
Priority: Minor


Name service configured in NameNode HA mode, should be discoverable across 
clusters without having to restart all the clusters every time a new cluster is 
deployed.

Solution #1: One solution would be to add the nameservice configurations to all 
the hdfs-site.xml files in the clusters involved.
Cons: We cannot restart all clusters every time we deploy a new cluster.

We are developing a custom class to extend 
org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider, this 
custom provider will make a REST call to fetch any additional nameservice 
configurations and cache them locally (expires on a configured time limit). 
This will fix the issue to resolve the name service across clusters and also 
eliminate the need to restart all the clusters whenever a new cluster is 
configured.

Please provide your inputs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Hadoop-Hdfs-trunk-Java8 - Build # 1229 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1229/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 5502 lines...]
[INFO] --- maven-source-plugin:2.3:jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-source-plugin:2.3:test-jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [04:05 min]
[INFO] Apache Hadoop HDFS . FAILURE [  01:10 h]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.127 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:14 h
[INFO] Finished at: 2016-05-17T23:43:34+00:00
[INFO] Final Memory: 96M/3823M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: ExecutionException: java.lang.RuntimeException: 
java.lang.RuntimeException: java.io.IOException: Stream Closed -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
6 tests failed.
FAILED:  
org.apache.hadoop.hdfs.TestDecommissionWithStriped.testDecommissionWithURBlockForSameBlockGroup

Error Message:
Problem binding to [localhost:33980] java.net.BindException: Address already in 
use; For more details see:  http://wiki.apache.org/hadoop/BindException

Stack Trace:
java.net.BindException: Problem binding to [localhost:33980] 
java.net.BindException: Address already in use; For more details see:  
http://wiki.apache.org/hadoop/BindException
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at 
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSoc

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1229

[Apache Jenkins Server]

See 

Changes:

[yzhang] HDFS-9732, Improve DelegationTokenIdentifier.toString() for better

[junping_du] MAPREDUCE-6657. Job history server can fail on startup when 
NameNode is

[jianhe] Addendum patch for YARN-4832. Contributed by Junping Du

--
[...truncated 5305 lines...]
Running org.apache.hadoop.tracing.TestTracingShortCircuitLocalRead
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.16 sec - in 
org.apache.hadoop.hdfs.TestFileCreationDelete
Running org.apache.hadoop.net.TestNetworkTopology
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.523 sec - in 
org.apache.hadoop.tracing.TestTracingShortCircuitLocalRead
Running org.apache.hadoop.TestGenericRefresh
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.321 sec - in 
org.apache.hadoop.TestGenericRefresh
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.954 sec - 
in org.apache.hadoop.net.TestNetworkTopology
Running org.apache.hadoop.tools.TestHdfsConfigFields
Running org.apache.hadoop.tools.TestTools
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.049 sec - in 
org.apache.hadoop.tools.TestHdfsConfigFields
Running org.apache.hadoop.tools.TestJMXGet
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.706 sec - in 
org.apache.hadoop.tools.TestTools
Running org.apache.hadoop.cli.TestErasureCodingCLI
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.879 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure020
Running org.apache.hadoop.cli.TestHDFSCLI
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 102.269 sec - 
in org.apache.hadoop.hdfs.TestReconstructStripedFile
Running org.apache.hadoop.cli.TestCryptoAdminCLI
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.08 sec - in 
org.apache.hadoop.tools.TestJMXGet
Running org.apache.hadoop.cli.TestAclCLI
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.808 sec - in 
org.apache.hadoop.cli.TestErasureCodingCLI
Running org.apache.hadoop.cli.TestDeleteCLI
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.921 sec - in 
org.apache.hadoop.cli.TestCryptoAdminCLI
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.027 sec - in 
org.apache.hadoop.cli.TestAclCLI
Running org.apache.hadoop.cli.TestXAttrCLI
Running org.apache.hadoop.cli.TestCacheAdminCLI
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.324 sec - in 
org.apache.hadoop.cli.TestDeleteCLI
Running org.apache.hadoop.TestRefreshCallQueue
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.396 sec - in 
org.apache.hadoop.cli.TestXAttrCLI
Running org.apache.hadoop.fs.TestWebHdfsFileContextMainOperations
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.628 sec - in 
org.apache.hadoop.TestRefreshCallQueue
Running org.apache.hadoop.fs.TestFcHdfsSetUMask
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.082 sec - in 
org.apache.hadoop.cli.TestCacheAdminCLI
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractConcat
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.586 sec - in 
org.apache.hadoop.fs.TestFcHdfsSetUMask
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.464 sec - in 
org.apache.hadoop.fs.contract.hdfs.TestHDFSContractConcat
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractRename
Tests run: 61, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.622 sec - 
in org.apache.hadoop.fs.TestWebHdfsFileContextMainOperations
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractSetTimes
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.634 sec - in 
org.apache.hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractOpen
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.377 sec - in 
org.apache.hadoop.fs.contract.hdfs.TestHDFSContractSetTimes
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.352 sec - in 
org.apache.hadoop.fs.contract.hdfs.TestHDFSContractRename
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractAppend
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractCreate
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.476 sec - in 
org.apache.hadoop.fs.contract.hdfs.TestHDFSContractOpen
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.597 sec - in 
org.apache.hadoop.fs.contract.hdfs.TestHDFSContractCreate
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.935 sec - in 
org.apache.hadoop.fs.contract.hdfs.TestHDFSContractAppend
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractSeek
Running org.apache.hadoop.fs

[jira] [Created] (HDFS-10420) Fix Ozone unit tests to use MiniOzoneCluster

[Arpit Agarwal (JIRA)]

Arpit Agarwal created HDFS-10420:


 Summary: Fix Ozone unit tests to use MiniOzoneCluster
 Key: HDFS-10420
 URL: https://issues.apache.org/jira/browse/HDFS-10420
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: ozone, test
Reporter: Arpit Agarwal






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Hadoop-Hdfs-trunk-Java8 - Build # 1228 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1228/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 5420 lines...]
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [04:08 min]
[INFO] Apache Hadoop HDFS . FAILURE [  01:00 h]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.104 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:04 h
[INFO] Finished at: 2016-05-17T22:03:10+00:00
[INFO] Final Memory: 96M/3746M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
1 tests failed.
FAILED:  
org.apache.hadoop.hdfs.TestDFSUpgradeFromImage.testUpgradeFromRel1BBWImage

Error Message:
Cannot obtain block length for 
LocatedBlock{BP-814998555-67.195.81.148-1463519669945:blk_7162739548153522810_1020;
 getBlockSize()=1024; corrupt=false; offset=0; 
locs=[DatanodeInfoWithStorage[127.0.0.1:45141,DS-8defc790-3d8b-4e0c-9fb9-40be0e5c445f,DISK]]}

Stack Trace:
java.io.IOException: Cannot obtain block length for 
LocatedBlock{BP-814998555-67.195.81.148-1463519669945:blk_7162739548153522810_1020;
 getBlockSize()=1024; corrupt=false; offset=0; 
locs=[DatanodeInfoWithStorage[127.0.0.1:45141,DS-8defc790-3d8b-4e0c-9fb9-40be0e5c445f,DISK]]}
at 
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:435)
at 
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStr

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1228

[Apache Jenkins Server]

See 

Changes:

[jianhe] YARN-4832. NM side resource value should get updated if change applied

[kihwal] HDFS-10417. Improve error message from checkBlockLocalPathAccess.

[kihwal] Revert "HDFS-10417. Improve error message from

--
[...truncated 5223 lines...]
Running org.apache.hadoop.hdfs.TestDatanodeReport
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.6 sec - in 
org.apache.hadoop.hdfs.TestDatanodeReport
Running org.apache.hadoop.hdfs.protocolPB.TestPBHelper
Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.77 sec - in 
org.apache.hadoop.hdfs.protocolPB.TestPBHelper
Running org.apache.hadoop.hdfs.TestEncryptedTransfer
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 92.457 sec - 
in org.apache.hadoop.hdfs.server.mover.TestMover
Running org.apache.hadoop.hdfs.TestPipelines
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.19 sec - in 
org.apache.hadoop.hdfs.TestPipelines
Running org.apache.hadoop.hdfs.TestHttpPolicy
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.594 sec - in 
org.apache.hadoop.hdfs.TestHttpPolicy
Running org.apache.hadoop.hdfs.TestEncryptionZonesWithHA
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.201 sec - in 
org.apache.hadoop.hdfs.TestEncryptionZonesWithHA
Running org.apache.hadoop.hdfs.TestDFSClientSocketSize
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.194 sec - in 
org.apache.hadoop.hdfs.TestDFSClientSocketSize
Running org.apache.hadoop.hdfs.TestWriteRead
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.79 sec - in 
org.apache.hadoop.hdfs.TestWriteRead
Running org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.168 sec - in 
org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream
Running org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 190.196 sec - 
in org.apache.hadoop.hdfs.server.mover.TestStorageMover
Running org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 116.094 sec - 
in org.apache.hadoop.hdfs.TestEncryptedTransfer
Running org.apache.hadoop.hdfs.TestPersistBlocks
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.97 sec - in 
org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
Running org.apache.hadoop.hdfs.TestFSInputChecker
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.107 sec - in 
org.apache.hadoop.hdfs.TestFSInputChecker
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.602 sec - in 
org.apache.hadoop.hdfs.TestPersistBlocks
Running org.apache.hadoop.fs.TestFcHdfsSetUMask
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.963 sec - in 
org.apache.hadoop.fs.TestFcHdfsSetUMask
Running org.apache.hadoop.fs.TestFcHdfsPermission
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.731 sec - in 
org.apache.hadoop.fs.TestFcHdfsPermission
Running org.apache.hadoop.fs.TestGlobPaths
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 56.406 sec - in 
org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Tests run: 37, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 7.052 sec - in 
org.apache.hadoop.fs.TestGlobPaths
Running org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
Running org.apache.hadoop.fs.TestSymlinkHdfsDisable
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.211 sec - in 
org.apache.hadoop.fs.TestSymlinkHdfsDisable
Running org.apache.hadoop.fs.TestSymlinkHdfsFileSystem
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.797 sec - in 
org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
Running org.apache.hadoop.fs.TestUrlStreamHandler
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.157 sec - in 
org.apache.hadoop.fs.TestUrlStreamHandler
Running org.apache.hadoop.fs.TestUnbuffer
Tests run: 74, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 21.194 sec - 
in org.apache.hadoop.fs.TestSymlinkHdfsFileSystem
Running org.apache.hadoop.fs.TestFcHdfsCreateMkdir
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.563 sec - in 
org.apache.hadoop.fs.TestUnbuffer
Running org.apache.hadoop.fs.TestEnhancedByteBufferAccess
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.53 sec - in 
org.apache.hadoop.fs.TestFcHdfsCreateMkdir
Running org.apache.hadoop.fs.TestHdfsNativeCodeLoader
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.362 sec - in 
org.apache.hadoop.fs.TestHdfsNativeCodeLoader
Running org.apache.hadoop.fs.TestHDFSFileContextMainOperations
Tests run: 10, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 18.243 sec - 
in org.apache.hadoop.fs.Te

[jira] [Reopened] (HDFS-8661) DataNode should filter the set of NameSpaceInfos passed to Datasets

[Arpit Agarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-8661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpit Agarwal reopened HDFS-8661:
-

> DataNode should filter the set of NameSpaceInfos passed to Datasets
> ---
>
> Key: HDFS-8661
> URL: https://issues.apache.org/jira/browse/HDFS-8661
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: datanode
>Affects Versions: HDFS-7240
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
> Attachments: HDFS-8661-HDFS-7240.01.patch, 
> HDFS-8661-HDFS-7240.02.patch, HDFS-8661-HDFS-7240.03.patch, 
> HDFS-8661-HDFS-7240.04.patch, v03-v04.diff
>
>
> {{DataNode#refreshVolumes}} passes the list of NamespaceInfos to each dataset 
> when adding new volumes.
> This list should be filtered by the correct NodeType(s) for each dataset. 
> e.g. in a shared HDFS+Ozone cluster, FsDatasets would be notified of NN block 
> pools and Ozone datasets would be notified of Ozone block pool(s).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-8677) Ozone: Introduce KeyValueContainerDatasetSpi

[Arpit Agarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpit Agarwal resolved HDFS-8677.
-
Resolution: Later

> Ozone: Introduce KeyValueContainerDatasetSpi
> 
>
> Key: HDFS-8677
> URL: https://issues.apache.org/jira/browse/HDFS-8677
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
> Attachments: HDFS-8677-HDFS-7240.01.patch, 
> HDFS-8677-HDFS-7240.02.patch, HDFS-8677-HDFS-7240.03.patch, 
> HDFS-8677-HDFS-7240.04.patch, HDFS-8677-HDFS-7240.05.patch
>
>
> KeyValueContainerDatasetSpi will be a new interface for Ozone containers, 
> just as FsDatasetSpi is an interface for manipulating HDFS block files.
> The interface will have support for both key-value containers for storing 
> Ozone metadata and blobs for storing user data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-8661) DataNode should filter the set of NameSpaceInfos passed to Datasets

[Arpit Agarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-8661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpit Agarwal resolved HDFS-8661.
-
Resolution: Later

> DataNode should filter the set of NameSpaceInfos passed to Datasets
> ---
>
> Key: HDFS-8661
> URL: https://issues.apache.org/jira/browse/HDFS-8661
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: datanode
>Affects Versions: HDFS-7240
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
> Attachments: HDFS-8661-HDFS-7240.01.patch, 
> HDFS-8661-HDFS-7240.02.patch, HDFS-8661-HDFS-7240.03.patch, 
> HDFS-8661-HDFS-7240.04.patch, v03-v04.diff
>
>
> {{DataNode#refreshVolumes}} passes the list of NamespaceInfos to each dataset 
> when adding new volumes.
> This list should be filtered by the correct NodeType(s) for each dataset. 
> e.g. in a shared HDFS+Ozone cluster, FsDatasets would be notified of NN block 
> pools and Ozone datasets would be notified of Ozone block pool(s).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Reopened] (HDFS-8677) Ozone: Introduce KeyValueContainerDatasetSpi

[Arpit Agarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpit Agarwal reopened HDFS-8677:
-

> Ozone: Introduce KeyValueContainerDatasetSpi
> 
>
> Key: HDFS-8677
> URL: https://issues.apache.org/jira/browse/HDFS-8677
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
> Attachments: HDFS-8677-HDFS-7240.01.patch, 
> HDFS-8677-HDFS-7240.02.patch, HDFS-8677-HDFS-7240.03.patch, 
> HDFS-8677-HDFS-7240.04.patch, HDFS-8677-HDFS-7240.05.patch
>
>
> KeyValueContainerDatasetSpi will be a new interface for Ozone containers, 
> just as FsDatasetSpi is an interface for manipulating HDFS block files.
> The interface will have support for both key-value containers for storing 
> Ozone metadata and blobs for storing user data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Reopened] (HDFS-8679) Move DatasetSpi to new package

[Arpit Agarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-8679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpit Agarwal reopened HDFS-8679:
-

> Move DatasetSpi to new package
> --
>
> Key: HDFS-8679
> URL: https://issues.apache.org/jira/browse/HDFS-8679
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
> Attachments: HDFS-8679-HDFS-7240.01.patch, 
> HDFS-8679-HDFS-7240.02.patch
>
>
> The DatasetSpi and VolumeSpi interfaces are currently in 
> {{org.apache.hadoop.hdfs.server.datanode.fsdataset}}. They can be moved to a 
> new package {{org.apache.hadoop.hdfs.server.datanode.dataset}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-8679) Move DatasetSpi to new package

[Arpit Agarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-8679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpit Agarwal resolved HDFS-8679.
-
Resolution: Later

These changes were discarded in the HDFS-7240 branch due to merge resolution 
complexity.

We may consider reintroducing them later.

> Move DatasetSpi to new package
> --
>
> Key: HDFS-8679
> URL: https://issues.apache.org/jira/browse/HDFS-8679
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
> Attachments: HDFS-8679-HDFS-7240.01.patch, 
> HDFS-8679-HDFS-7240.02.patch
>
>
> The DatasetSpi and VolumeSpi interfaces are currently in 
> {{org.apache.hadoop.hdfs.server.datanode.fsdataset}}. They can be moved to a 
> new package {{org.apache.hadoop.hdfs.server.datanode.dataset}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-8457) Ozone: Refactor FsDatasetSpi to pull up HDFS-agnostic functionality into parent interface

[Arpit Agarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpit Agarwal resolved HDFS-8457.
-
Resolution: Later

These changes were discarded in the HDFS-7240 branch due to merge resolution 
complexity.

We may consider reintroducing them later.

> Ozone: Refactor FsDatasetSpi to pull up HDFS-agnostic functionality into 
> parent interface
> -
>
> Key: HDFS-8457
> URL: https://issues.apache.org/jira/browse/HDFS-8457
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
> Attachments: HDFS-8457-HDFS-7240.01.patch, 
> HDFS-8457-HDFS-7240.02.patch, HDFS-8457-HDFS-7240.03.patch, 
> HDFS-8457-HDFS-7240.04.patch, HDFS-8457-HDFS-7240.05.patch, 
> HDFS-8457-HDFS-7240.06.patch, HDFS-8457-HDFS-7240.07.patch
>
>
> FsDatasetSpi can be split up into HDFS-specific and HDFS-agnostic parts. The 
> HDFS-specific parts can continue to be retained in FsDataSpi while those 
> relating to volume management, block pools and upgrade can be moved to a 
> parent interface.
> There will be no change to implementations of FsDatasetSpi.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Reopened] (HDFS-8457) Ozone: Refactor FsDatasetSpi to pull up HDFS-agnostic functionality into parent interface

[Arpit Agarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpit Agarwal reopened HDFS-8457:
-

> Ozone: Refactor FsDatasetSpi to pull up HDFS-agnostic functionality into 
> parent interface
> -
>
> Key: HDFS-8457
> URL: https://issues.apache.org/jira/browse/HDFS-8457
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
> Attachments: HDFS-8457-HDFS-7240.01.patch, 
> HDFS-8457-HDFS-7240.02.patch, HDFS-8457-HDFS-7240.03.patch, 
> HDFS-8457-HDFS-7240.04.patch, HDFS-8457-HDFS-7240.05.patch, 
> HDFS-8457-HDFS-7240.06.patch, HDFS-8457-HDFS-7240.07.patch
>
>
> FsDatasetSpi can be split up into HDFS-specific and HDFS-agnostic parts. The 
> HDFS-specific parts can continue to be retained in FsDataSpi while those 
> relating to volume management, block pools and upgrade can be moved to a 
> parent interface.
> There will be no change to implementations of FsDatasetSpi.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1227

[Apache Jenkins Server]

See 

Changes:

[cmccabe] HDFS-10404. Fix formatting of CacheAdmin command usage help text 
(Yiqun

--
[...truncated 5278 lines...]
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.973 sec - in 
org.apache.hadoop.hdfs.TestDatanodeReport
Running org.apache.hadoop.hdfs.protocolPB.TestPBHelper
Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.73 sec - in 
org.apache.hadoop.hdfs.protocolPB.TestPBHelper
Running org.apache.hadoop.hdfs.TestEncryptedTransfer
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 91.094 sec - 
in org.apache.hadoop.hdfs.server.mover.TestMover
Running org.apache.hadoop.hdfs.TestPipelines
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.67 sec - in 
org.apache.hadoop.hdfs.TestPipelines
Running org.apache.hadoop.hdfs.TestHttpPolicy
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.546 sec - in 
org.apache.hadoop.hdfs.TestHttpPolicy
Running org.apache.hadoop.hdfs.TestEncryptionZonesWithHA
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.526 sec - in 
org.apache.hadoop.hdfs.TestEncryptionZonesWithHA
Running org.apache.hadoop.hdfs.TestDFSClientSocketSize
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.769 sec - in 
org.apache.hadoop.hdfs.TestDFSClientSocketSize
Running org.apache.hadoop.hdfs.TestWriteRead
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.119 sec - in 
org.apache.hadoop.hdfs.TestWriteRead
Running org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream
Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 109.278 sec - 
in org.apache.hadoop.hdfs.TestEncryptedTransfer
Running org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.602 sec - in 
org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream
Running org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.681 sec - in 
org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
Running org.apache.hadoop.hdfs.TestPersistBlocks
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 186.164 sec - 
in org.apache.hadoop.hdfs.server.mover.TestStorageMover
Running org.apache.hadoop.hdfs.TestFSInputChecker
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.889 sec - in 
org.apache.hadoop.hdfs.TestFSInputChecker
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.359 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
Running org.apache.hadoop.fs.TestFcHdfsSetUMask
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.052 sec - in 
org.apache.hadoop.fs.TestFcHdfsSetUMask
Running org.apache.hadoop.fs.TestFcHdfsPermission
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.838 sec - in 
org.apache.hadoop.fs.TestFcHdfsPermission
Running org.apache.hadoop.fs.TestGlobPaths
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.819 sec - in 
org.apache.hadoop.hdfs.TestPersistBlocks
Running org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
Tests run: 37, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 5.493 sec - in 
org.apache.hadoop.fs.TestGlobPaths
Running org.apache.hadoop.fs.TestSymlinkHdfsDisable
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.725 sec - in 
org.apache.hadoop.fs.TestSymlinkHdfsDisable
Running org.apache.hadoop.fs.TestSymlinkHdfsFileSystem
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 54.029 sec - in 
org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Running org.apache.hadoop.fs.TestUrlStreamHandler
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.472 sec - in 
org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
Running org.apache.hadoop.fs.TestUnbuffer
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.455 sec - in 
org.apache.hadoop.fs.TestUrlStreamHandler
Running org.apache.hadoop.fs.TestFcHdfsCreateMkdir
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.666 sec - in 
org.apache.hadoop.fs.TestUnbuffer
Running org.apache.hadoop.fs.TestEnhancedByteBufferAccess
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.717 sec - in 
org.apache.hadoop.fs.TestFcHdfsCreateMkdir
Running org.apache.hadoop.fs.TestHdfsNativeCodeLoader
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.176 sec - in 
org.apache.hadoop.fs.TestHdfsNativeCodeLoader
Running org.apache.hadoop.fs.TestHDFSFileContextMainOperations
Tests run: 74, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 18.444 sec - 
in org.apache.hadoop.fs.TestSymlinkHdfsFileSystem
Running org.apache.hadoop.fs.shell.TestHdfsTextCommand
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.402 sec - in 

Hadoop-Hdfs-trunk-Java8 - Build # 1227 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1227/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 5475 lines...]
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [04:18 min]
[INFO] Apache Hadoop HDFS . FAILURE [59:04 min]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.089 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:03 h
[INFO] Finished at: 2016-05-17T20:55:00+00:00
[INFO] Final Memory: 100M/3963M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
2 tests failed.
FAILED:  
org.apache.hadoop.hdfs.security.TestDelegationToken.testDTManagerInSafeMode

Error Message:
Port in use: localhost:54632

Stack Trace:
java.net.BindException: Port in use: localhost:54632
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at 
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at 
org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
at 
org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:911)
at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:853)
at 
org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer

[jira] [Reopened] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

[Kihwal Lee (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kihwal Lee reopened HDFS-10417:
---

> Improve error message from checkBlockLocalPathAccess
> 
>
> Key: HDFS-10417
> URL: https://issues.apache.org/jira/browse/HDFS-10417
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.7.2
>Reporter: Tianyin Xu
>Assignee: Tianyin Xu
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HDFS-10417.000.patch
>
>
> The exception msg thrown by {{checkBlockLocalPathAccess}} is very specific to 
> the implementation detail. It's really hard for users to understand it unless 
> she reads and understands the code. 
> The code is shown as follows:
> {code:title=org.apache.hadoop.hdfs.server.datanode.DataNode|borderStyle=solid}
>   private void checkBlockLocalPathAccess() throws IOException {
> checkKerberosAuthMethod("getBlockLocalPathInfo()");
> String currentUser = 
> UserGroupInformation.getCurrentUser().getShortUserName();
> if (!usersWithLocalPathAccess.contains(currentUser)) {
>   throw new AccessControlException(
>   "Can't continue with getBlockLocalPathInfo() "
>   + "authorization. The user " + currentUser
>   + " is not allowed to call getBlockLocalPathInfo");
> }
>   }
> {code}
> (basically she needs to understand the code logic of getBlockLocalPathInfo)
> \\
> Note that {{usersWithLocalPathAccess}} is a *private final* purely coming 
> from the configuration settings of {{dfs.block.local-path-access.user}},
> {code:title=org.apache.hadoop.hdfs.server.datanode.DataNode|borderStyle=solid}
> private final List usersWithLocalPathAccess;
> 
> this.usersWithLocalPathAccess = Arrays.asList(
> 
> conf.getTrimmedStrings(DFSConfigKeys.DFS_BLOCK_LOCAL_PATH_ACCESS_USER_KEY));
> {code}
> In other word, the checking fails simply because the current user is not 
> specified in the configuration setting of 
> {{dfs.block.local-path-access.user}}. The log message should be much more 
> clearer to make it easy for users to take actions, as demonstrated in the 
> attached patch. 
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-10419) Building HDFS on top of Ozone's storage containers

[Jing Zhao (JIRA)]

Jing Zhao created HDFS-10419:


 Summary: Building HDFS on top of Ozone's storage containers
 Key: HDFS-10419
 URL: https://issues.apache.org/jira/browse/HDFS-10419
 Project: Hadoop HDFS
  Issue Type: New Feature
Reporter: Jing Zhao
Assignee: Jing Zhao


In HDFS-7240, Ozone defines storage containers to store both the data and the 
metadata. The storage container layer provides an object storage interface and 
aims to manage data/metadata in a distributed manner. More details about 
storage containers can be found in the design doc in HDFS-7240.

HDFS can adopt the storage containers to store and manage blocks. The general 
idea is:
# Each block can be treated as an object and the block ID is the object's key.
# Blocks will still be stored in DataNodes but as objects in storage containers.
# The block management work can be separated out of the NameNode and will be 
handled by the storage container layer in a more distributed way. The NameNode 
will only manage the namespace (i.e., files and directories).
# For each file, the NameNode only needs to record a list of block IDs which 
are used as keys to obtain real data from storage containers.
# A new DFSClient implementation talks to both NameNode and the storage
container layer to read/write.

HDFS, especially the NameNode, can get much better scalability from this
design. Currently the NameNode's heaviest workload comes from the block 
management, which includes maintaining the block-DataNode mapping, receiving 
full/incremental block reports, tracking block states (under/over/miss 
replicated), and joining every writing pipeline protocol to guarantee the data 
consistency. These work bring high memory footprint
and make NameNode suffer from GC. HDFS-5477 already proposes to convert 
BlockManager as a service. If we can build HDFS on top of the storage container 
layer, we not only separate out the BlockManager from the NameNode, but also 
replace it with a new distributed management scheme.

The storage container work is currently in progress in HDFS-7240, and the work 
proposed here is still in an experimental/exploring stage. We can do this 
experiment in a feature branch so that people with interests can be involved.

A design doc will be uploaded later explaining more details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1226

[Apache Jenkins Server]

See 

Changes:

[jlowe] MAPREDUCE-6701. application master log can not be available when

[jlowe] MAPREDUCE-6698. Increase timeout on

--
[...truncated 5228 lines...]
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.344 sec - in 
org.apache.hadoop.hdfs.TestClose
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.44 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
Running org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.872 sec - in 
org.apache.hadoop.hdfs.TestDFSAddressConfig
Running org.apache.hadoop.hdfs.TestLargeBlock
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 40.578 sec - in 
org.apache.hadoop.hdfs.TestFileAppend4
Running org.apache.hadoop.hdfs.TestHDFSTrash
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 43.292 sec - in 
org.apache.hadoop.hdfs.TestParallelRead
Running org.apache.hadoop.hdfs.TestClientReportBadBlock
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.765 sec - in 
org.apache.hadoop.hdfs.TestHDFSTrash
Running org.apache.hadoop.hdfs.TestWriteRead
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.671 sec - in 
org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead
Running org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.333 sec - in 
org.apache.hadoop.hdfs.TestLargeBlock
Running org.apache.hadoop.hdfs.TestBalancerBandwidth
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.692 sec - in 
org.apache.hadoop.hdfs.TestClientReportBadBlock
Running org.apache.hadoop.hdfs.TestDFSUpgrade
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.936 sec - in 
org.apache.hadoop.hdfs.TestWriteRead
Running org.apache.hadoop.hdfs.TestBlockStoragePolicy
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.431 sec - in 
org.apache.hadoop.hdfs.TestBalancerBandwidth
Running org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.191 sec - in 
org.apache.hadoop.hdfs.TestDFSUpgrade
Running org.apache.hadoop.hdfs.TestLeaseRecoveryStriped
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.63 sec - in 
org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocal
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 79.502 sec - in 
org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.016 sec - in 
org.apache.hadoop.hdfs.TestLeaseRecoveryStriped
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote2
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.119 sec - 
in org.apache.hadoop.hdfs.TestBlockStoragePolicy
Running org.apache.hadoop.hdfs.client.impl.TestClientBlockVerification
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.173 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote2
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.825 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderFactory
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.403 sec - in 
org.apache.hadoop.hdfs.client.impl.TestClientBlockVerification
Running org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.333 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote
Running org.apache.hadoop.hdfs.qjournal.TestNNWithQJM
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.499 sec - in 
org.apache.hadoop.hdfs.qjournal.TestNNWithQJM
Running org.apache.hadoop.hdfs.qjournal.server.TestJournalNodeMXBean
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.388 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderFactory
Running org.apache.hadoop.hdfs.qjournal.server.TestJournal
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.508 sec - in 
org.apache.hadoop.hdfs.qjournal.server.TestJournalNodeMXBean
Running org.apache.hadoop.hdfs.qjournal.server.TestJournalNode
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.219 sec - in 
org.apache.hadoop.hdfs.qjournal.server.TestJournal
Running org.apache.hadoop.hdfs.qjournal.client.TestIPCLoggerChannel
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.954 sec - in 
org.apache.hadoop.hdfs.qjournal.client.TestIPCLoggerChannel
Running org.apache

Hadoop-Hdfs-trunk-Java8 - Build # 1226 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1226/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 5425 lines...]
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [04:05 min]
[INFO] Apache Hadoop HDFS . FAILURE [  01:12 h]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.072 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:16 h
[INFO] Finished at: 2016-05-17T17:29:16+00:00
[INFO] Final Memory: 96M/3875M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
1 tests failed.
FAILED:  
org.apache.hadoop.hdfs.TestDecommissionWithStriped.testDecommissionWithURBlockForSameBlockGroup

Error Message:
Problem binding to [localhost:50178] java.net.BindException: Address already in 
use; For more details see:  http://wiki.apache.org/hadoop/BindException

Stack Trace:
java.net.BindException: Problem binding to [localhost:50178] 
java.net.BindException: Address already in use; For more details see:  
http://wiki.apache.org/hadoop/BindException
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at 
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.apache.hadoop.ipc.Server.bind(Server.java:530)
at org.a

[jira] [Resolved] (HDFS-10418) NPE in TestDistributedFileSystem.testDFSCloseOrdering

[Steve Loughran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-10418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran resolved HDFS-10418.
---
Resolution: Duplicate

you're right: closing

> NPE in TestDistributedFileSystem.testDFSCloseOrdering
> -
>
> Key: HDFS-10418
> URL: https://issues.apache.org/jira/browse/HDFS-10418
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs, test
>Affects Versions: 2.8.0
> Environment: Jenkins
>Reporter: Steve Loughran
>Priority: Critical
>
> Jenkins is failing with an NPE in close() —the close op assumes there's 
> always a StorageStatistics. instance. If there isn"t, you get a stack trace



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1225

[Apache Jenkins Server]

See 

Changes:

[kihwal] HDFS-10303. DataStreamer#ResponseProcessor calculates packet ack 
latency

--
[...truncated 7034 lines...]
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160
Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.781 sec - in 
org.apache.hadoop.hdfs.protocolPB.TestPBHelper
Running org.apache.hadoop.hdfs.TestReconstructStripedFile
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 59.128 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStream
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 162.896 sec - 
in org.apache.hadoop.hdfs.TestWriteReadStripedFile
Running org.apache.hadoop.hdfs.TestDFSInputStream
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 120.504 sec - 
in org.apache.hadoop.hdfs.TestReconstructStripedFile
Running org.apache.hadoop.hdfs.TestDFSRemove
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 125.867 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160
Running org.apache.hadoop.hdfs.TestFileAppend4
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 83.357 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStream
Running org.apache.hadoop.hdfs.TestParallelRead
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.747 sec - in 
org.apache.hadoop.hdfs.TestDFSInputStream
Running org.apache.hadoop.hdfs.TestClose
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.702 sec - in 
org.apache.hadoop.hdfs.TestClose
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.364 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040
Running org.apache.hadoop.hdfs.TestDFSAddressConfig
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.888 sec - in 
org.apache.hadoop.hdfs.TestDFSRemove
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.432 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
Running org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.712 sec - in 
org.apache.hadoop.hdfs.TestDFSAddressConfig
Running org.apache.hadoop.hdfs.TestLargeBlock
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.586 sec - in 
org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead
Running org.apache.hadoop.hdfs.TestHDFSTrash
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 40.601 sec - in 
org.apache.hadoop.hdfs.TestFileAppend4
Running org.apache.hadoop.hdfs.TestClientReportBadBlock
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.55 sec - in 
org.apache.hadoop.hdfs.TestLargeBlock
Running org.apache.hadoop.hdfs.TestWriteRead
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 42.753 sec - in 
org.apache.hadoop.hdfs.TestParallelRead
Running org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.739 sec - in 
org.apache.hadoop.hdfs.TestHDFSTrash
Running org.apache.hadoop.hdfs.TestBalancerBandwidth
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.601 sec - in 
org.apache.hadoop.hdfs.TestClientReportBadBlock
Running org.apache.hadoop.hdfs.TestDFSUpgrade
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.803 sec - in 
org.apache.hadoop.hdfs.TestBalancerBandwidth
Running org.apache.hadoop.hdfs.TestBlockStoragePolicy
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 41.285 sec - in 
org.apache.hadoop.hdfs.TestWriteRead
Running org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.535 sec - in 
org.apache.hadoop.hdfs.TestDFSUpgrade
Running org.apache.hadoop.hdfs.TestLeaseRecoveryStriped
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.826 sec - in 
org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocal
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 71.405 sec - in 
org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.31 sec - in 
org.apache.hadoop.hdfs.TestLeaseRecoveryStriped
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.76 sec - in 
org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderRemote2
Running org.apache.hadoop.hdfs.client.impl.TestClientBlockVerification
Tests run: 20, Failures: 0, Errors:

Hadoop-Hdfs-trunk-Java8 - Build # 1225 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1225/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 7231 lines...]
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client .. SUCCESS [04:11 min]
[INFO] Apache Hadoop HDFS . FAILURE [  01:13 h]
[INFO] Apache Hadoop HDFS Native Client ... SKIPPED
[INFO] Apache Hadoop HttpFS ... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .. SKIPPED
[INFO] Apache Hadoop HDFS-NFS . SKIPPED
[INFO] Apache Hadoop HDFS Project . SUCCESS [  0.078 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:17 h
[INFO] Finished at: 2016-05-17T16:09:42+00:00
[INFO] Final Memory: 99M/3896M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Recording test results
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3
Setting 
LATEST1_8_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
Setting 
MAVEN_3_3_3_HOME=/home/jenkins/jenkins-slave/tools/hudson.tasks.Maven_MavenInstallation/maven-3.3.3



###
## FAILED TESTS (if any) 
##
1 tests failed.
FAILED:  
org.apache.hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics.testEcTasks

Error Message:
Bad value for metric EcReconstructionTasks expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: Bad value for metric EcReconstructionTasks 
expected:<1> but was:<0>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at 
org.apache.hadoop.test.MetricsAsserts.assertCounter(MetricsAsserts.java:228)
at 
org.apache.hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics.testEcTasks(TestDataNodeErasureCodingMetrics.java:92)




-
To unsubscribe, e-mail: hdfs-dev

Jenkins build is back to normal : Hadoop-Hdfs-trunk-Java8 #1224

[Apache Jenkins Server]

See 


-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)

[Allen Wittenauer]

Thanks to Karthik’s +1, HADOOP-13161 was committed last night and appears to be 
working.

I’ve got almost the entire list of stuff finished.  I’m now going to go 
through the java8 jobs and actually sync them up, since they all do slightly 
different things. :/


> On May 16, 2016, at 8:02 PM, Allen Wittenauer  wrote:
> 
>   OK, it looks like if someone commits HADOOP-13161, then trunk only uses 
> JDK8 and branch-2 will use JDK7 and JDK8 during precommit with no changes 
> required to Apache Yetus. :D
> 
> 
>> On May 16, 2016, at 5:38 PM, Allen Wittenauer 
>>  wrote:
>> 
>> 
>> There’s a bunch of stuff that needs to happen at the Jenkins level:
>> 
>> * Kill off the JDK7 trunk builds for HADOOP, HDFS, MAPRED, YARN
>> * Remove JDK7 from pre-commit for HADOOP, HDFS, MAPRED, YARN
>> 
>> One thing that needs to happen in the Apache Yetus project:
>> * Wait until YETUS-369 has been written and committed to re-enable JDK7 for 
>> pre-commit  (This effectively means that *ALL* JDK7 testing will *ONLY* be 
>> happening in the regularly scheduled builds)
>> 
>> One thing that really should happen in the Apache Hadoop project:
>> * Remove JDK7 from trunk Dockerfile
>> 
>> I’ll start banging on this stuff over the next few days.
>> 
>> 
>>> On May 16, 2016, at 3:58 PM, Andrew Wang  wrote:
>>> 
>>> Very happy to announce that we've committed HADOOP-11858. I'm looking
>>> forward to writing my first lambda in Java. I also attached a video to the
>>> JIRA so we can all relive this moment in Hadoop development history.
>>> 
>>> It sounds like there's some precommit work to align test-patch with this
>>> change. I'm hoping Allen will take point on this, but ping me if I can be
>>> of any assistance.
>>> 
>>> On Thu, May 12, 2016 at 11:53 AM, Li Lu  wrote:
>>> 
 I’d like to bring YARN-4977 into attention for using Java 8. HADOOP-13083
 does the maven change and in yarn-api there are ~5000 javadoc warnings.
 
 Li Lu
 
> On May 10, 2016, at 08:32, Akira AJISAKA 
 wrote:
> 
> Hi developers,
> 
> Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk.
> Given this is a critical change, I'm thinking we should get the
 consensus first.
> 
> One concern I think is, when the minimum version is set to JDK8, we need
 to configure Jenkins to disable multi JDK test only in trunk.
> 
> Any thoughts?
> 
> Thanks,
> Akira
> 
> -
> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
> 
> 
 
 
>> 
>> 
>> -
>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>> 
> 
> 
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> 


-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Hadoop-Hdfs-trunk-Java8 - Build # 1223 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1223/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 5483 lines...]
[INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-hdfs-project 
---
[INFO] Executing tasks

main:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/hadoop-hdfs-project/target/test-dir
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-source-plugin:2.3:jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-source-plugin:2.3:test-jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client . SUCCESS [04:07 min]
[INFO] Apache Hadoop HDFS  FAILURE [  01:13 h]
[INFO] Apache Hadoop HDFS Native Client .. SKIPPED
[INFO] Apache Hadoop HttpFS .. SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal . SKIPPED
[INFO] Apache Hadoop HDFS-NFS  SKIPPED
[INFO] Apache Hadoop HDFS Project  SUCCESS [  0.089 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:18 h
[INFO] Finished at: 2016-05-17T13:04:43+00:00
[INFO] Final Memory: 59M/709M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
2 tests failed.
FAILED:  
org.apache.hadoop.hdfs.tools.TestDFSZKFailoverController.testThreadDumpCaptureAfterNNStateChange

Error Message:
Deferred

Stack Trace:
java.lang.RuntimeException: Deferred
at 
org.apache.hadoop.test.MultithreadedTestUtil$TestContext.checkException(MultithreadedTestUtil.java:130)
at 
org.apache.hadoop.ha.ZKFCTestUtil.waitForHealthState(ZKFCTestUtil.java:29)
at 
org.apache.hadoop.hdfs.tools.TestDFSZKFailoverController.setup(TestDFSZKFailoverController.java:106)
Caused by: java.net.BindException: Problem binding to [localhost:10024] 
java.net.BindException: Address already in use; For more details see:  
http://wiki.apache.org/hadoop/BindException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:815)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:735)
at org.apache.hadoop.ipc.Server.bind(Server.java:547)
at org.apache.hadoop.ipc.Server$Listener.(Server.java:793)
  

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1223

[Apache Jenkins Server]

See 

Changes:

[aajisaka] MAPREDUCE-6693. ArrayIndexOutOfBoundsException occurs when the 
length of

--
[...truncated 5290 lines...]
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.306 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050
Running org.apache.hadoop.hdfs.TestDFSShellGenericOptions
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.176 sec - in 
org.apache.hadoop.hdfs.TestDFSStartupVersions
Running org.apache.hadoop.hdfs.protocolPB.TestPBHelper
Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.798 sec - in 
org.apache.hadoop.hdfs.protocolPB.TestPBHelper
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.996 sec - in 
org.apache.hadoop.hdfs.TestDFSShellGenericOptions
Running org.apache.hadoop.hdfs.TestReconstructStripedFile
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 88.592 sec - in 
org.apache.hadoop.hdfs.TestLeaseRecovery2
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStream
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.021 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160
Running org.apache.hadoop.hdfs.TestDFSInputStream
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.2 sec - in 
org.apache.hadoop.hdfs.TestDFSInputStream
Running org.apache.hadoop.hdfs.TestDFSRemove
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 81.719 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStream
Running org.apache.hadoop.hdfs.TestFileAppend4
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.635 sec - in 
org.apache.hadoop.hdfs.TestDFSRemove
Running org.apache.hadoop.hdfs.TestParallelRead
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 116.556 sec - 
in org.apache.hadoop.hdfs.TestReconstructStripedFile
Running org.apache.hadoop.hdfs.TestClose
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 168.574 sec - 
in org.apache.hadoop.hdfs.TestWriteReadStripedFile
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.417 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040
Running org.apache.hadoop.hdfs.TestDFSAddressConfig
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.641 sec - in 
org.apache.hadoop.hdfs.TestClose
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 41.373 sec - in 
org.apache.hadoop.hdfs.TestFileAppend4
Running org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.305 sec - in 
org.apache.hadoop.hdfs.TestDFSAddressConfig
Running org.apache.hadoop.hdfs.TestLargeBlock
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 45.293 sec - in 
org.apache.hadoop.hdfs.TestParallelRead
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.351 sec - in 
org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead
Running org.apache.hadoop.hdfs.TestHDFSTrash
Running org.apache.hadoop.hdfs.TestClientReportBadBlock
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.562 sec - in 
org.apache.hadoop.hdfs.TestHDFSTrash
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.77 sec - in 
org.apache.hadoop.hdfs.TestLargeBlock
Running org.apache.hadoop.hdfs.TestWriteRead
Running org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.113 sec - in 
org.apache.hadoop.hdfs.TestClientReportBadBlock
Running org.apache.hadoop.hdfs.TestBalancerBandwidth
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.267 sec - in 
org.apache.hadoop.hdfs.TestBalancerBandwidth
Running org.apache.hadoop.hdfs.TestDFSUpgrade
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.519 sec - in 
org.apache.hadoop.hdfs.TestWriteRead
Running org.apache.hadoop.hdfs.TestBlockStoragePolicy
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.542 sec - in 
org.apache.hadoop.hdfs.TestDFSUpgrade
Running org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.47 sec - in 
org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
Running org.apache.hadoop.hdfs.TestLeaseRecoveryStriped
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.347 sec - in 
org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Running org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocal
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 59.312 sec - 
in org.apache.hadoop.hdfs.TestBlockStoragePolicy
Running org.apache.hadoop.hdfs.client.im

Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)

[Steve Loughran]

> On 16 May 2016, at 23:58, Andrew Wang  wrote:
> 
> Very happy to announce that we've committed HADOOP-11858. I'm looking
> forward to writing my first lambda in Java. I also attached a video to the
> JIRA so we can all relive this moment in Hadoop development history.
> 
> It sounds like there's some precommit work to align test-patch with this
> change. I'm hoping Allen will take point on this, but ping me if I can be
> of any assistance.
> 

Everyone who worked on this deserves a lot of credit, especially anyone working 
on library incompatibilities


What does it mean?

1. it means that if you are working on trunk, you only need to worry about Java 
8. 

2. You can use java 8 language features: time to learn your lambda expressions. 
 Apparently this it a good tutorial 
http://www.cse.chalmers.se/research/group/logic/TypesSS05/Extra/geuvers.pdf

3. You can use java 8 library features.


It also means: you can no longer backport stuff from trunk to branch-2 and 
expect it to always compile

If you are working on code which you hope to get into branch-2, you need to 
show it will work in Java 7

Tactics

1. work on branch-2; forward port to trunk
2. before backporting from trunk, submit a HADOOP-1234-branch-2-001.patch ; 
yetus will see the branch-2 marker and test against branch2.

I would recommend always doing #2, irrespective of which branch you work on. 
Why? It means you can just switch the java version on your path to 1.8, with 
branch-2 still being in java 7 language —but the java library picked up will be 
the 1.8 lib. The only way to be confident that you haven't picked up java 8 
features is to build and test against JDK7, which Yetus does if you name your 
patch right.

I know this will be a bit of extra work, and I expect we will all make mistakes 
here. If anyone does accidentally commit JDK8+ code to branch-2, then it should 
be bounced out as soon as it's noticed, then a new patch nurtured in via Yetus  
& Jenkins. We want to keep branch-2 building.

On that topic, Jenkins isn't happy these days...I think we need to spend some 
monitoring it and fixing. At least now we can cull the java7 trunk builds, 
leaving less to ignore.


 There's one test, TestDNS which fails intermittently (always), but more and 
more we are learning to ignore. That's dangerous. 



-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Reopened] (HDFS-8914) Document HA support in the HDFS HdfsDesign.md

[Ravindra Babu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDFS-8914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravindra Babu reopened HDFS-8914:
-

Still the documentation page contains "Single Point of Failure comment" in 

http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html



> Document HA support in the HDFS HdfsDesign.md
> -
>
> Key: HDFS-8914
> URL: https://issues.apache.org/jira/browse/HDFS-8914
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 2.7.1
> Environment: Documentation page in live
>Reporter: Ravindra Babu
>Assignee: Lars Francke
> Fix For: 2.8.0, 2.7.3
>
> Attachments: HDFS-8914.1.patch, HDFS-8914.2.patch
>
>
> Please refer to these two links and correct one of them.
> http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html
> The NameNode machine is a single point of failure for an HDFS cluster. If the 
> NameNode machine fails, manual intervention is necessary. Currently, 
> automatic restart and failover of the NameNode software to another machine is 
> not supported.
> http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html
> The HDFS High Availability feature addresses the above problems by providing 
> the option of running two redundant NameNodes in the same cluster in an 
> Active/Passive configuration with a hot standby. This allows a fast failover 
> to a new NameNode in the case that a machine crashes, or a graceful 
> administrator-initiated failover for the purpose of planned maintenance.
> Please update hdfsDesign article with same facts to avoid confusion in 
> Reader's mind..



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-10418) NPE in TestDistributedFileSystem.testDFSCloseOrdering

[Steve Loughran (JIRA)]

Steve Loughran created HDFS-10418:
-

 Summary: NPE in TestDistributedFileSystem.testDFSCloseOrdering
 Key: HDFS-10418
 URL: https://issues.apache.org/jira/browse/HDFS-10418
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs, test
Affects Versions: 2.8.0
 Environment: Jenkins
Reporter: Steve Loughran
Priority: Critical


Jenkins is failing with an NPE in close() —the close op assumes there's always 
a StorageStatistics. instance. If there isn"t, you get a stack trace



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1222

[Apache Jenkins Server]

See 

Changes:

[aajisaka] HDFS-10242. Cannot create space quota of zero. Contributed by Takashi

--
[...truncated 6191 lines...]
java.lang.Thread.State: TIMED_WAITING
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at 
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
at 
org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:218)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2387)
"nioEventLoopGroup-6-1"  prio=10 tid=147 runnable
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:621)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:309)
at 
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:703)
at 
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:744)
"pool-9-thread-1"  prio=5 tid=94 timed_waiting
java.lang.Thread.State: TIMED_WAITING
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:744)
"java.util.concurrent.ThreadPoolExecutor$Worker@14312ad8[State = -1, empty 
queue]" daemon prio=5 tid=197 timed_waiting
java.lang.Thread.State: TIMED_WAITING
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:744)
"1053556764@qtp-1245630814-1 - Acceptor0 
SelectChannelConnector@localhost:50373" daemon prio=5 tid=6217 runnable
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at 
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:498)
at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
at 
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
at 
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
at 
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
"IPC Server handler 7 on 41872" daemon prio=5 tid=164 timed_waiting
java.lang.Thread.State: TIMED_WAITING
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at 
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
at 
org.apache.hadoop.ipc.CallQueueManager.take(CallQueueMana

Hadoop-Hdfs-trunk-Java8 - Build # 1222 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1222/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 6384 lines...]
[INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-hdfs-project 
---
[INFO] Executing tasks

main:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/hadoop-hdfs-project/target/test-dir
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-source-plugin:2.3:jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-source-plugin:2.3:test-jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client . SUCCESS [04:04 min]
[INFO] Apache Hadoop HDFS  FAILURE [  01:15 h]
[INFO] Apache Hadoop HDFS Native Client .. SKIPPED
[INFO] Apache Hadoop HttpFS .. SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal . SKIPPED
[INFO] Apache Hadoop HDFS-NFS  SKIPPED
[INFO] Apache Hadoop HDFS Project  SUCCESS [  0.096 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:19 h
[INFO] Finished at: 2016-05-17T09:18:35+00:00
[INFO] Final Memory: 59M/743M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
2 tests failed.
FAILED:  
org.apache.hadoop.hdfs.TestAsyncDFSRename.testAggressiveConcurrentAsyncAPI

Error Message:
test timed out after 6 milliseconds

Stack Trace:
java.lang.Exception: test timed out after 6 milliseconds
at java.lang.Thread.sleep(Native Method)
at 
org.apache.hadoop.hdfs.MiniDFSCluster.waitActive(MiniDFSCluster.java:2485)
at 
org.apache.hadoop.hdfs.MiniDFSCluster.waitActive(MiniDFSCluster.java:2525)
at 
org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNodes(MiniDFSCluster.java:1990)
at 
org.apache.hadoop.hdfs.TestAsyncDFSRename.internalTestConcurrentAsyncAPI(TestAsyncDFSRename.java:365)
at 
org.apache.hadoop.hdfs.TestAsyncDFSRename.testAggressiveConcurrentAsyncAPI(TestAsyncDFSRename.java:289)


FAILED:  
org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten

Error Message:
expected:<0> but was:<387591854>

Stack Trace:
java.lang.AssertionError: expected:<0> but was:<387591854>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at 
org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolume

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #1221

[Apache Jenkins Server]

See 

Changes:

[Arun Suresh] YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. 
(asuresh)

--
[...truncated 7081 lines...]
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.376 sec - in 
org.apache.hadoop.hdfs.TestSetrepDecreasing
Running org.apache.hadoop.hdfs.TestBlocksScheduledCounter
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.656 sec - in 
org.apache.hadoop.hdfs.TestLocalDFS
Running org.apache.hadoop.hdfs.TestApplyingStoragePolicy
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.245 sec - in 
org.apache.hadoop.hdfs.TestBlocksScheduledCounter
Running org.apache.hadoop.hdfs.TestSetrepIncreasing
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.098 sec - in 
org.apache.hadoop.hdfs.TestApplyingStoragePolicy
Running org.apache.hadoop.hdfs.TestDecommission
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 43.921 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070
Running org.apache.hadoop.hdfs.TestMultiThreadedHflush
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.453 sec - in 
org.apache.hadoop.hdfs.TestMultiThreadedHflush
Running org.apache.hadoop.hdfs.TestMissingBlocksAlert
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.159 sec - in 
org.apache.hadoop.hdfs.TestMissingBlocksAlert
Running org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.07 sec - in 
org.apache.hadoop.hdfs.TestSetrepIncreasing
Running org.apache.hadoop.hdfs.TestFileStatus
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.071 sec - in 
org.apache.hadoop.hdfs.TestFileStatus
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.315 sec - in 
org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150
Running org.apache.hadoop.hdfs.TestBalancerBandwidth
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.726 sec - in 
org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060
Tests run: 44, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 135.321 sec - 
in org.apache.hadoop.hdfs.TestHDFSFileSystemContract
Running org.apache.hadoop.hdfs.TestSetTimes
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.901 sec - in 
org.apache.hadoop.hdfs.TestBalancerBandwidth
Running org.apache.hadoop.TestGenericRefresh
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.025 sec - in 
org.apache.hadoop.TestGenericRefresh
Running org.apache.hadoop.tracing.TestTracing
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.647 sec - in 
org.apache.hadoop.hdfs.TestSetTimes
Running org.apache.hadoop.tracing.TestTracingShortCircuitLocalRead
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.047 sec - in 
org.apache.hadoop.tracing.TestTracing
Running org.apache.hadoop.tracing.TestTraceAdmin
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.114 sec - in 
org.apache.hadoop.tracing.TestTracingShortCircuitLocalRead
Running org.apache.hadoop.security.TestPermission
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.028 sec - in 
org.apache.hadoop.tracing.TestTraceAdmin
Running org.apache.hadoop.security.TestPermissionSymlinks
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.918 sec - in 
org.apache.hadoop.security.TestPermission
Running org.apache.hadoop.security.TestRefreshUserMappings
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.826 sec - in 
org.apache.hadoop.security.TestPermissionSymlinks
Running org.apache.hadoop.fs.TestFcHdfsSetUMask
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.046 sec - in 
org.apache.hadoop.security.TestRefreshUserMappings
Running org.apache.hadoop.fs.TestSymlinkHdfsFileSystem
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 42.623 sec - 
in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.859 sec - in 
org.apache.hadoop.fs.TestFcHdfsSetUMask
Running org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractRename
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.525 sec - in 
org.apache.hadoop.fs.contract.hdfs.TestHDFSContractRename
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractDelete
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.407 sec - in 
org.apache.hadoop.fs.contract.hdfs.TestHDFSContractDelete
Running org.apache.hadoop.fs.contract.hdfs.TestHDFSContractAppend
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.978 sec - in 
org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
Runni

Hadoop-Hdfs-trunk-Java8 - Build # 1221 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/1221/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 7274 lines...]
[INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-hdfs-project 
---
[INFO] Executing tasks

main:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/hadoop-hdfs-project/target/test-dir
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-source-plugin:2.3:jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-source-plugin:2.3:test-jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.5:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.15:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.0:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS Client . SUCCESS [04:11 min]
[INFO] Apache Hadoop HDFS  FAILURE [  01:04 h]
[INFO] Apache Hadoop HDFS Native Client .. SKIPPED
[INFO] Apache Hadoop HttpFS .. SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal . SKIPPED
[INFO] Apache Hadoop HDFS-NFS  SKIPPED
[INFO] Apache Hadoop HDFS Project  SUCCESS [  0.082 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:08 h
[INFO] Finished at: 2016-05-17T07:55:28+00:00
[INFO] Final Memory: 59M/1037M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk-Java8/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-hdfs
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
6 tests failed.
FAILED:  
org.apache.hadoop.hdfs.TestDatanodeLayoutUpgrade.testUpgradeToIdBasedLayout

Error Message:
Timed out waiting for Mini HDFS Cluster to start

Stack Trace:
java.io.IOException: Timed out waiting for Mini HDFS Cluster to start
at 
org.apache.hadoop.hdfs.MiniDFSCluster.waitClusterUp(MiniDFSCluster.java:1358)
at 
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:861)
at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:490)
at 
org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:449)
at 
org.apache.hadoop.hdfs.TestDFSUpgradeFromImage.upgradeAndVerify(TestDFSUpgradeFromImage.java:594)
at 
org.apache.hadoop.hdfs.TestDatanodeLayoutUpgrade.testUpgradeToIdBasedLayout(TestDatanodeLayoutUpgrade.java:51)


FAILED:  
org.apache.hadoop.hdfs.TestDatanodeLayoutUpgrade.testUpgradeFrom256To32Layout

Error Message:
Timed out waiting for Mini HDFS Cluster to start

Stack Trace:
java.io.IOException: Timed out waiting for Mini HDFS Cluster to start
at 
org.apache.hadoop.hdfs.MiniDFSCluster.waitClusterUp(MiniDFSCluster.java:1358)
at 
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:861)
at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:490)
at 
org.apache.hadoop.hdfs