Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64
For more details, see https://builds.apache.org/job/hadoop-trunk-win/467/ [May 14, 2018 6:24:01 AM] (littlezhou) Add 2.9.1 release notes and changes documents [May 14, 2018 6:38:40 AM] (sammichen) Revert "Add 2.9.1 release notes and changes documents" [May 14, 2018 7:14:02 AM] (sammi.chen) Add 2.9.1 release notes and changes documents -1 overall The following subsystems voted -1: compile mvninstall unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 00m 00s) unit Specific tests: Failed junit tests : hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec hadoop.fs.contract.rawlocal.TestRawlocalContractAppend hadoop.fs.TestFileUtil hadoop.fs.TestFsShellCopy hadoop.fs.TestFsShellList hadoop.fs.TestLocalFileSystem hadoop.fs.TestRawLocalFileSystemContract hadoop.fs.TestTrash hadoop.http.TestHttpServer hadoop.http.TestHttpServerLogs hadoop.io.nativeio.TestNativeIO hadoop.ipc.TestSocketFactory hadoop.metrics2.impl.TestStatsDMetrics hadoop.security.TestSecurityUtil hadoop.security.TestShellBasedUnixGroupsMapping hadoop.security.token.TestDtUtilShell hadoop.util.TestNativeCodeLoader hadoop.fs.TestResolveHdfsSymlink hadoop.hdfs.client.impl.TestBlockReaderLocal hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy hadoop.hdfs.crypto.TestHdfsCryptoStreams hadoop.hdfs.qjournal.client.TestQuorumJournalManager hadoop.hdfs.qjournal.server.TestJournalNode hadoop.hdfs.qjournal.server.TestJournalNodeSync hadoop.hdfs.security.TestDelegationTokenForProxyUser hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter hadoop.hdfs.server.datanode.fsdataset.impl.TestProvidedImpl hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation hadoop.hdfs.server.datanode.TestBlockPoolSliceStorage hadoop.hdfs.server.datanode.TestBlockRecovery hadoop.hdfs.server.datanode.TestBlockScanner hadoop.hdfs.server.datanode.TestDataNodeFaultInjector hadoop.hdfs.server.datanode.TestDataNodeMetrics hadoop.hdfs.server.datanode.TestDataNodeUUID hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure hadoop.hdfs.server.datanode.TestDirectoryScanner hadoop.hdfs.server.datanode.TestHSync hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport hadoop.hdfs.server.datanode.web.TestDatanodeHttpXFrame hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC hadoop.hdfs.server.mover.TestStorageMover hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots hadoop.hdfs.server.namenode.snapshot.TestSnapRootDescendantDiff hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport hadoop.hdfs.server.namenode.TestAddBlock hadoop.hdfs.server.namenode.TestAuditLoggerWithCommands hadoop.hdfs.server.namenode.TestCheckpoint hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate hadoop.hdfs.server.namenode.TestEditLogRace hadoop.hdfs.server.namenode.TestFSImage hadoop.hdfs.server.namenode.TestFSImageWithSnapshot hadoop.hdfs.server.namenode.TestNamenodeCapacityReport hadoop.hdfs.server.namenode.TestNameNodeMXBean hadoop.hdfs.server.namenode.TestNestedEncryptionZones hadoop.hdfs.server.namenode.TestQuotaByStorageType hadoop.hdfs.server.namenode.TestReencryptionHandler hadoop.hdfs.server.namenode.TestStartup hadoop.hdfs.TestDatanodeRegistration hadoop.hdfs.TestDatanodeReport hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs hadoop.hdfs.TestDecommission hadoop.
RE: [VOTE] Release Apache Hadoop 2.8.4 (RC0)
Thanks Junping for driving this release. +1 (binding) -- Build successfully from the source code -- Start HA cluster -- Verified basic shell operations -- Ran pi,wordcount -- Browsed the NN and RM UI -Brahma Reddy Battula -Original Message- From: 俊平堵 [mailto:junping...@apache.org] Sent: 09 May 2018 01:41 To: Hadoop Common ; Hdfs-dev ; mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: [VOTE] Release Apache Hadoop 2.8.4 (RC0) Hi all, I've created the first release candidate (RC0) for Apache Hadoop 2.8.4. This is our next maint release to follow up 2.8.3. It includes 77 important fixes and improvements. The RC artifacts are available at: http://home.apache.org/~junping_du/hadoop-2.8.4-RC0 The RC tag in git is: release-2.8.4-RC0 The maven artifacts are available via repository.apache.org< http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1118 Please try the release and vote; the vote will run for the usual 5 working days, ending on 5/14/2018 PST time. Thanks, Junping - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15468) The setDeprecatedProperties method for the Configuration in Hadoop3.0.2.
Wenming He created HADOOP-15468: --- Summary: The setDeprecatedProperties method for the Configuration in Hadoop3.0.2. Key: HADOOP-15468 URL: https://issues.apache.org/jira/browse/HADOOP-15468 Project: Hadoop Common Issue Type: Bug Components: common Affects Versions: 3.0.2 Reporter: Wenming He Fix For: 3.0.2 在判断overlay变量是否存在弃用的键时，为什么他是直接判断overlay中的值 ，而不是去判断overlay中存在相同的键。这是个什么逻辑？ -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/225/ [May 14, 2018 6:00:00 AM] (rohithsharmaks) YARN-8247 Incorrect HTTP status code returned by ATSv2 for [May 14, 2018 7:36:30 AM] (sammi.chen) Add 2.9.1 release notes and changes documents [May 14, 2018 5:21:59 PM] (hanishakoneru) HDFS-13544. Improve logging for JournalNode in federated cluster. -1 overall The following subsystems voted -1: docker Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15466) Correct units in adl.http.timeout
Sean Mackrory created HADOOP-15466: -- Summary: Correct units in adl.http.timeout Key: HADOOP-15466 URL: https://issues.apache.org/jira/browse/HADOOP-15466 Project: Hadoop Common Issue Type: Bug Components: fs/adl Reporter: Sean Mackrory Assignee: Sean Mackrory Comment in core-default.xml says seconds, but according to the SDK docs it's getting interpreted as milliseconds ([https://github.com/Azure/azure-data-lake-store-java/blob/master/src/main/java/com/microsoft/azure/datalake/store/ADLStoreOptions.java#L139-L144).] Pinging [~ASikaria] for any additional comment. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15465) Use native java code for symlinks
Íñigo Goiri created HADOOP-15465: Summary: Use native java code for symlinks Key: HADOOP-15465 URL: https://issues.apache.org/jira/browse/HADOOP-15465 Project: Hadoop Common Issue Type: Sub-task Reporter: Íñigo Goiri Hadoop uses the shell to create symbolic links. Now that Hadoop relies on Java 7+, we can deprecate all the shell code and rely on the Java APIs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15461) Improvements over the Hadoop support with Windows
Giovanni Matteo Fumarola created HADOOP-15461: - Summary: Improvements over the Hadoop support with Windows Key: HADOOP-15461 URL: https://issues.apache.org/jira/browse/HADOOP-15461 Project: Hadoop Common Issue Type: Bug Reporter: Giovanni Matteo Fumarola -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.4 (RC0)
+1 (binding) - Build from source. - Ran sanity jobs successfully. Thanks, Wangda On Mon, May 14, 2018 at 5:44 AM, Sunil G wrote: > +1 (binding) > > 1. Build package from src > 2. Ran few MR jobs and verified checked App Priority cases > 3. Node Label basic functions are ok. > > Thanks > Sunil > > > On Tue, May 8, 2018 at 11:11 PM 俊平堵 wrote: > > > Hi all, > > I've created the first release candidate (RC0) for Apache Hadoop > > 2.8.4. This is our next maint release to follow up 2.8.3. It includes 77 > > important fixes and improvements. > > > > The RC artifacts are available at: > > http://home.apache.org/~junping_du/hadoop-2.8.4-RC0 > > > > The RC tag in git is: release-2.8.4-RC0 > > > > The maven artifacts are available via repository.apache.org< > > http://repository.apache.org> at: > > https://repository.apache.org/content/repositories/orgapachehadoop-1118 > > > > Please try the release and vote; the vote will run for the usual 5 > > working days, ending on 5/14/2018 PST time. > > > > Thanks, > > > > Junping > > >
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/781/ No changes -1 overall The following subsystems voted -1: asflicense findbugs unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: FindBugs : module:hadoop-hdds/common Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$CloseContainerRequestProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 17815] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$CloseContainerResponseProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 18363] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$CopyContainerRequestProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 34624] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$CopyContainerResponseProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 35479] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$CreateContainerResponseProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 12991] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$DatanodeBlockID$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 1112] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$DeleteChunkResponseProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 30029] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$DeleteContainerRequestProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 15580] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$DeleteContainerResponseProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 16042] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$DeleteKeyResponseProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 23085] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$KeyValue$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 1739] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$ListContainerRequestProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 16530] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$ListKeyRequestProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 23608] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$PutKeyResponseProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 20936] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$PutSmallFileResponseProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 32916] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$ReadContainerRequestProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 13417] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$UpdateContainerResponseProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 15107] Useless control flow in org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$WriteChunkResponseProto$Builder.maybeForceBuilderInitialization() At ContainerProtos.java: At ContainerProtos.java:[line 27144] Found reliance on default encoding in org.apache.hadoop.utils.MetadataKeyFilters$KeyPrefixFilter.filterKey(byte[], byte[], byte[]):in org.apache.hadoop.utils.MetadataKeyFilters$KeyPrefixFilter.filterKey(byte[], byte[], byte[]): String.getBytes() At MetadataKeyFilters.java:[line 97] Failed junit tests : hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.yarn.applications.distributedshell.TestDistributedShell
Re: [VOTE] Release Apache Hadoop 2.8.4 (RC0)
+1 (binding) 1. Build package from src 2. Ran few MR jobs and verified checked App Priority cases 3. Node Label basic functions are ok. Thanks Sunil On Tue, May 8, 2018 at 11:11 PM 俊平堵 wrote: > Hi all, > I've created the first release candidate (RC0) for Apache Hadoop > 2.8.4. This is our next maint release to follow up 2.8.3. It includes 77 > important fixes and improvements. > > The RC artifacts are available at: > http://home.apache.org/~junping_du/hadoop-2.8.4-RC0 > > The RC tag in git is: release-2.8.4-RC0 > > The maven artifacts are available via repository.apache.org< > http://repository.apache.org> at: > https://repository.apache.org/content/repositories/orgapachehadoop-1118 > > Please try the release and vote; the vote will run for the usual 5 > working days, ending on 5/14/2018 PST time. > > Thanks, > > Junping >
Re: [VOTE] Release Apache Hadoop 2.8.4 (RC0)
+1 (binding) - Downloaded source and built from it. Installed 2 node RM HA cluster. - Verified for RM HA, RM Restart, work preserving restart. - Ran sample MR jobs and Distributed shell with HA scenario. -Rohith Sharma K S On 8 May 2018 at 23:11, 俊平堵 wrote: > Hi all, > I've created the first release candidate (RC0) for Apache Hadoop > 2.8.4. This is our next maint release to follow up 2.8.3. It includes 77 > important fixes and improvements. > > The RC artifacts are available at: > http://home.apache.org/~junping_du/hadoop-2.8.4-RC0 > > The RC tag in git is: release-2.8.4-RC0 > > The maven artifacts are available via repository.apache.org< > http://repository.apache.org> at: > https://repository.apache.org/content/repositories/orgapachehadoop-1118 > > Please try the release and vote; the vote will run for the usual 5 > working days, ending on 5/14/2018 PST time. > > Thanks, > > Junping >
[jira] [Created] (HADOOP-15460) S3A FS to add "s3a:no-existence-checks" to the builder file creation option set
Steve Loughran created HADOOP-15460: --- Summary: S3A FS to add "s3a:no-existence-checks" to the builder file creation option set Key: HADOOP-15460 URL: https://issues.apache.org/jira/browse/HADOOP-15460 Project: Hadoop Common Issue Type: Sub-task Components: fs/s3 Affects Versions: 3.1.0 Reporter: Steve Loughran As promised to [~StephanEwen]: add and s3a-specific option to the builder-API to create files for all existence checks to be skipped. This # eliminates a few hundred milliseconds # avoids any caching of negative HEAD/GET responses in the S3 load balancers. Callers will be expected to know what what they are doing. FWIW, we are doing some PUT calls in the committer which bypass this stuff, for the same reason. If you've just created a directory, you know there's nothing underneath, so no need to check. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
