Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/59/ No changes -1 overall The following subsystems voted -1: asflicense unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Unreaped Processes : hadoop-common:1 hadoop-hdfs:23 bkjournal:5 hadoop-yarn-common:1 hadoop-yarn-server-nodemanager:3 hadoop-yarn-server-timelineservice:1 hadoop-yarn-client:4 hadoop-yarn-applications-distributedshell:1 hadoop-mapreduce-client-app:1 hadoop-mapreduce-client-jobclient:5 hadoop-distcp:3 hadoop-extras:1 Failed junit tests : hadoop.fs.sftp.TestSFTPFileSystem hadoop.hdfs.TestDatanodeDeath hadoop.hdfs.TestSetrepIncreasing hadoop.hdfs.TestDataTransferProtocol hadoop.hdfs.TestDFSFinalize hadoop.yarn.server.nodemanager.webapp.TestNMWebServer hadoop.yarn.server.nodemanager.containermanager.TestContainerManagerRecovery hadoop.yarn.server.nodemanager.containermanager.TestContainerManager hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels hadoop.yarn.server.nodemanager.TestNodeStatusUpdater hadoop.yarn.applications.distributedshell.TestDistributedShellWithNodeLabels hadoop.tools.TestIntegration hadoop.tools.TestDistCpViewFs hadoop.resourceestimator.solver.impl.TestLpSolver hadoop.resourceestimator.service.TestResourceEstimatorService Timed out junit tests : org.apache.hadoop.log.TestLogLevel org.apache.hadoop.hdfs.TestLeaseRecovery2 org.apache.hadoop.hdfs.TestDatanodeRegistration org.apache.hadoop.hdfs.TestBlocksScheduledCounter org.apache.hadoop.hdfs.TestDFSClientFailover org.apache.hadoop.hdfs.TestDFSClientRetries org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream org.apache.hadoop.hdfs.TestFileAppendRestart org.apache.hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter org.apache.hadoop.hdfs.TestSeekBug org.apache.hadoop.hdfs.TestDatanodeReport org.apache.hadoop.hdfs.web.TestWebHDFS org.apache.hadoop.hdfs.web.TestWebHDFSXAttr org.apache.hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs org.apache.hadoop.hdfs.TestDFSRollback org.apache.hadoop.hdfs.TestMiniDFSCluster org.apache.hadoop.hdfs.TestDistributedFileSystem org.apache.hadoop.hdfs.web.TestWebHDFSForHA org.apache.hadoop.hdfs.TestTrashWithEncryptionZones org.apache.hadoop.hdfs.TestSetTimes org.apache.hadoop.hdfs.TestDFSShell org.apache.hadoop.hdfs.web.TestWebHDFSAcl org.apache.hadoop.contrib.bkjournal.TestBootstrapStandbyWithBKJM org.apache.hadoop.contrib.bkjournal.TestBookKeeperJournalManager org.apache.hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints org.apache.hadoop.contrib.bkjournal.TestBookKeeperAsHASharedDir org.apache.hadoop.contrib.bkjournal.TestBookKeeperSpeculativeRead org.apache.hadoop.yarn.webapp.TestWebApp org.apache.hadoop.yarn.server.nodemanager.TestNodeManagerResync org.apache.hadoop.yarn.server.nodemanager.TestNodeManagerReboot org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServices org.apache.hadoop.yarn.client.TestRMFailover org.apache.hadoop.yarn.client.TestApplicationClientProtocolOnHA org.apache.hadoop.yarn.client.api.impl.TestYarnClientWithReservation org.apache.hadoop.yarn.client.api.impl.TestAMRMClient org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell org.apache.hadoop.mapreduce.v2.app.TestStagingCleanup org.apache.hadoop.mapred.TestMRCJCFileInputFormat org.apache.hadoop.mapred.TestClusterMapReduceTestCase org.apache.hadoop.mapred.TestJobName org.apache.hadoop.mapred.TestMiniMRBringup org.apache.hadoop.mapred.TestNetworkedJob org.apache.hadoop.tools.TestDistCpSync org.apache.hadoop.tools.TestDistCpSyncReverseFromTarget org.apache.hadoop.tools.TestDistCpSyncReverseFromSource org.apache.hadoop.tools.TestCopyFiles cc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/59/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/59/artifact/out/diff-compile-javac-root.txt [324K] checkstyle: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/59/artifact/out/diff-checkstyle-root.txt [16M] pylint:
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/ [Dec 2, 2017 6:34:30 AM] (cdouglas) HDFS-11576. Block recovery will fail indefinitely if recovery time > -1 overall The following subsystems voted -1: asflicense findbugs unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api org.apache.hadoop.yarn.api.records.Resource.getResources() may expose internal representation by returning Resource.resources At Resource.java:by returning Resource.resources At Resource.java:[line 234] Failed junit tests : hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 hadoop.hdfs.TestDFSStripedOutputStreamWithFailure hadoop.hdfs.TestDFSStripedOutputStreamWithFailure030 hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.fs.TestUnbuffer hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup hadoop.hdfs.server.balancer.TestBalancerRPCDelay hadoop.hdfs.TestErasureCodingPolicies hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200 hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140 hadoop.yarn.server.nodemanager.webapp.TestNMWebServices hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart hadoop.mapreduce.v2.app.rm.TestRMContainerAllocator hadoop.mapreduce.v2.TestUberAM cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/diff-compile-javac-root.txt [280K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/diff-checkstyle-root.txt [17M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/diff-patch-pylint.txt [20K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/whitespace-eol.txt [8.8M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/whitespace-tabs.txt [288K] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-warnings.html [8.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/diff-javadoc-javadoc-root.txt [760K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [360K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt [44K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [80K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt [20K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt [104K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt [84K] asflicense: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/612/artifact/out/patch-asflicense-problems.txt [4.0K] Powered by Apache Yetus 0.7.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[DISCUSS] Merge HDFS-9806 to trunk
Hey hdfs-dev@- The HDFS-9806 dev branch is getting ready to merge to trunk. This branch adds a new storage type (PROVIDED) to support reading remote data as HDFS blocks. Design documentation and discussion are available on JIRA [1]. Over the next week or so, we'll work through the remaining issues, write the site documentation, and generally clean up the branch. The current code has been stressed in experimental settings, including a large production benchmark. When this feature is not enabled and active, it should have no effect on HDFS. The merge will checkpoint progress on reading remote data through HDFS, setting up work on the write path [2] and improving usability. Please look over the branch and raise issues that should be addressed before the merge. -C [1]: https://issues.apache.org/jira/browse/HDFS-9806 [2]: https://issues.apache.org/jira/browse/HDFS-12090
[jira] [Created] (HDFS-12882) Support full open(PathHandle) contract in HDFS
Chris Douglas created HDFS-12882:
Summary: Support full open(PathHandle) contract in HDFS
Key: HDFS-12882
URL: https://issues.apache.org/jira/browse/HDFS-12882
Project: Hadoop HDFS
Issue Type: Improvement
Components: hdfs-client
Reporter: Chris Douglas
Assignee: Chris Douglas
HDFS-7878 added support for {{open(PathHandle)}}, but it only partially
implemented the semantics specified in the contract (i.e., open-by-inodeID).
HDFS should implement all permutations of the default options for
{{PathHandle}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/58/ [Dec 1, 2017 8:15:25 PM] (jlowe) MAPREDUCE-5124. AM lacks flow control for task events. Contributed by [Dec 1, 2017 8:21:18 PM] (rkanter) YARN-4813. TestRMWebServicesDelegationTokenAuthentication.testDoAs fails [Dec 1, 2017 9:51:49 PM] (jlowe) YARN-7455. quote_and_append_arg can overflow buffer. Contributed by Jim -1 overall The following subsystems voted -1: asflicense unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Unreaped Processes : hadoop-common:2 hadoop-hdfs:21 hadoop-hdfs-httpfs:1 bkjournal:5 hadoop-yarn-server-timelineservice:1 hadoop-yarn-server-resourcemanager:2 hadoop-yarn-client:4 hadoop-yarn-applications-distributedshell:1 hadoop-mapreduce-client-app:1 hadoop-mapreduce-client-jobclient:21 hadoop-distcp:3 Failed junit tests : hadoop.hdfs.web.TestWebHdfsUrl hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter hadoop.hdfs.TestDatanodeRegistration hadoop.hdfs.TestDatanodeReport hadoop.hdfs.web.TestHftpFileSystem hadoop.hdfs.web.TestFSMainOperationsWebHdfs hadoop.hdfs.web.TestWebHDFS hadoop.yarn.applications.distributedshell.TestDistributedShellWithNodeLabels hadoop.mapreduce.lib.join.TestJoinProperties hadoop.mapreduce.lib.join.TestJoinDatamerge hadoop.tools.TestIntegration hadoop.tools.TestDistCpViewFs hadoop.resourceestimator.solver.impl.TestLpSolver hadoop.resourceestimator.service.TestResourceEstimatorService Timed out junit tests : org.apache.hadoop.http.TestHttpServer org.apache.hadoop.log.TestLogLevel org.apache.hadoop.hdfs.TestLeaseRecovery2 org.apache.hadoop.hdfs.TestRead org.apache.hadoop.security.TestPermission org.apache.hadoop.hdfs.web.TestWebHdfsTokens org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream org.apache.hadoop.hdfs.TestDatanodeLayoutUpgrade org.apache.hadoop.hdfs.TestFileAppendRestart org.apache.hadoop.hdfs.security.TestDelegationToken org.apache.hadoop.hdfs.TestDFSMkdirs org.apache.hadoop.hdfs.TestDFSOutputStream org.apache.hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs org.apache.hadoop.hdfs.web.TestWebHDFSXAttr org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes org.apache.hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs org.apache.hadoop.hdfs.TestDistributedFileSystem org.apache.hadoop.hdfs.TestReplaceDatanodeFailureReplication org.apache.hadoop.hdfs.TestDFSShell org.apache.hadoop.fs.http.client.TestHttpFSWithHttpFSFileSystem org.apache.hadoop.contrib.bkjournal.TestBootstrapStandbyWithBKJM org.apache.hadoop.contrib.bkjournal.TestBookKeeperJournalManager org.apache.hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints org.apache.hadoop.contrib.bkjournal.TestBookKeeperAsHASharedDir org.apache.hadoop.contrib.bkjournal.TestBookKeeperSpeculativeRead org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServices org.apache.hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore org.apache.hadoop.yarn.server.resourcemanager.reservation.TestFairSchedulerPlanFollower org.apache.hadoop.yarn.client.TestRMFailover org.apache.hadoop.yarn.client.TestApplicationClientProtocolOnHA org.apache.hadoop.yarn.client.api.impl.TestYarnClientWithReservation org.apache.hadoop.yarn.client.api.impl.TestAMRMClient org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell org.apache.hadoop.mapreduce.v2.app.TestStagingCleanup org.apache.hadoop.mapreduce.TestNewCombinerGrouping org.apache.hadoop.mapreduce.TestMRJobClient org.apache.hadoop.mapred.TestMiniMRClasspath org.apache.hadoop.mapreduce.TestMapCollection org.apache.hadoop.mapreduce.TestMROutputFormat org.apache.hadoop.mapred.TestMRIntermediateDataEncryption org.apache.hadoop.mapreduce.lib.aggregate.TestMapReduceAggregates org.apache.hadoop.mapred.TestJobSysDirWithDFS org.apache.hadoop.mapreduce.security.TestBinaryTokenFile org.apache.hadoop.mapred.TestMRTimelineEventHandling org.apache.hadoop.mapreduce.lib.chain.TestSingleElementChain org.apache.hadoop.mapred.join.TestDatamerge org.apache.hadoop.mapreduce.lib.chain.TestMapReduceChain org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl
