Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64
For more details, see https://builds.apache.org/job/hadoop-trunk-win/440/
[Apr 16, 2018 5:15:37 PM] (xyao) HADOOP-15369. Avoid usage of
${project.version} in parent poms.
[Apr 16, 2018 6:24:26 PM] (eyang) YARN-8060. Added default readiness check for
service components.
[Apr 16, 2018 9:16:59 PM] (arp) HDFS-13439. Add test case for read block
operation when it is moved.
[Apr 16, 2018 10:05:04 PM] (haibochen) YARN-8162. Remove Method
DirectoryCollection#verifyDirUsingMkdir. (Yufei
[Apr 16, 2018 10:55:00 PM] (virajith) HDFS-13311. RBF:
TestRouterAdminCLI#testCreateInvalidEntry fails on
[Apr 16, 2018 11:26:54 PM] (inigoiri) YARN-7773. YARN Federation used Mysql as
state store throw exception,
[Apr 16, 2018 11:52:17 PM] (inigoiri) YARN-8165. Incorrect queue name logging
in AbstractContainerAllocator.
[Apr 17, 2018 12:32:19 AM] (inigoiri) YARN-8164. Fix a potential NPE in
AbstractSchedulerPlanFollower.
[Apr 17, 2018 3:23:22 AM] (yqlin) HDFS-13435. RBF: Improve the error loggings
for printing the stack
[Apr 17, 2018 9:38:31 AM] (sunilg) YARN-8126. Support auto-spawning of admin
configured services during
-1 overall
The following subsystems voted -1:
compile mvninstall unit
The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac
The following subsystems are considered long running:
(runtime bigger than 1h 00m 00s)
unit
Specific tests:
Failed junit tests :
hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec
hadoop.fs.contract.rawlocal.TestRawlocalContractAppend
hadoop.fs.TestFileUtil
hadoop.fs.TestFsShellCopy
hadoop.fs.TestFsShellList
hadoop.fs.TestLocalFileSystem
hadoop.fs.TestRawLocalFileSystemContract
hadoop.fs.TestSymlinkLocalFSFileContext
hadoop.fs.TestTrash
hadoop.http.TestHttpServer
hadoop.http.TestHttpServerLogs
hadoop.io.nativeio.TestNativeIO
hadoop.ipc.TestSocketFactory
hadoop.metrics2.impl.TestStatsDMetrics
hadoop.metrics2.sink.TestRollingFileSystemSinkWithLocal
hadoop.security.TestSecurityUtil
hadoop.security.TestShellBasedUnixGroupsMapping
hadoop.security.token.TestDtUtilShell
hadoop.util.TestNativeCodeLoader
hadoop.util.TestNodeHealthScriptRunner
hadoop.fs.TestResolveHdfsSymlink
hadoop.hdfs.crypto.TestHdfsCryptoStreams
hadoop.hdfs.qjournal.client.TestQuorumJournalManager
hadoop.hdfs.qjournal.server.TestJournalNode
hadoop.hdfs.qjournal.server.TestJournalNodeSync
hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages
hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter
hadoop.hdfs.server.datanode.fsdataset.impl.TestProvidedImpl
hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation
hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica
hadoop.hdfs.server.datanode.TestBlockPoolSliceStorage
hadoop.hdfs.server.datanode.TestBlockRecovery
hadoop.hdfs.server.datanode.TestBlockScanner
hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics
hadoop.hdfs.server.datanode.TestDataNodeFaultInjector
hadoop.hdfs.server.datanode.TestDataNodeMetrics
hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
hadoop.hdfs.server.datanode.TestDirectoryScanner
hadoop.hdfs.server.datanode.TestHSync
hadoop.hdfs.server.datanode.web.TestDatanodeHttpXFrame
hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand
hadoop.hdfs.server.diskbalancer.TestDiskBalancer
hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC
hadoop.hdfs.server.mover.TestStorageMover
hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot
hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
hadoop.hdfs.server.namenode.snapshot.TestSnapRootDescendantDiff
hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
hadoop.hdfs.server.namenode.TestAddBlock
hadoop.hdfs.server.namenode.TestAuditLoggerWithCommands
hadoop.hdfs.server.namenode.
[jira] [Created] (YARN-8172) Unit test coverage for C code of winutils
Xiao Liang created YARN-8172: Summary: Unit test coverage for C code of winutils Key: YARN-8172 URL: https://issues.apache.org/jira/browse/YARN-8172 Project: Hadoop YARN Issue Type: Test Reporter: Xiao Liang Currently there's no UT coverage for the C code of winutils, which should be added. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[RESULT] [VOTE] Release Apache Hadoop 2.7.6 (RC0)
Hi everybody, With 4 binding and 4 non-binding +1s and no -1s the vote for Apache Release 2.7.6 passes. Thank you everybody for contributing to the release, testing, and voting. Binding +1s Zhe Zhang Brahma Reddy Battula Jason Lowe Konstantin Shvachko Non-binding +1s Chen Liang Erik Krogen Takanobu Asanuma Ajay Kumar On Mon, Apr 9, 2018 at 4:14 PM, Konstantin Shvachko wrote: > Hi everybody, > > This is the next dot release of Apache Hadoop 2.7 line. The previous one > 2.7.5 was released on December 14, 2017. > Release 2.7.6 includes critical bug fixes and optimizations. See more > details in Release Note: > http://home.apache.org/~shv/hadoop-2.7.6-RC0/releasenotes.html > > The RC0 is available at: http://home.apache.org/~shv/hadoop-2.7.6-RC0/ > > Please give it a try and vote on this thread. The vote will run for 5 > days ending 04/16/2018. > > My up to date public key is available from: > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > Thanks, > --Konstantin >
[jira] [Resolved] (YARN-7088) Add application launch time to Resource Manager REST API
[ https://issues.apache.org/jira/browse/YARN-7088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen resolved YARN-7088. -- Resolution: Fixed > Add application launch time to Resource Manager REST API > > > Key: YARN-7088 > URL: https://issues.apache.org/jira/browse/YARN-7088 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0-alpha4 >Reporter: Abdullah Yousufi >Assignee: Kanwaljeet Sachdev >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7088.001.patch, YARN-7088.002.patch, > YARN-7088.003.patch, YARN-7088.004.patch, YARN-7088.005.patch, > YARN-7088.006.patch, YARN-7088.007.patch, YARN-7088.008.patch, > YARN-7088.009.patch, YARN-7088.010.patch, YARN-7088.011.patch, > YARN-7088.012.patch, YARN-7088.013.patch, YARN-7088.014.patch, > YARN-7088.015.patch, YARN-7088.016.patch, YARN-7088.017.patch > > > Currently, the start time in the old and new UI actually shows the app > submission time. There should actually be two different fields; one for the > app's submission and one for its launch, as well as the elapsed pending time > between the two. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.7.6 (RC0)
My formal +1 for 2.7.6 RC0 Thanks, --Konstantin On Mon, Apr 9, 2018 at 4:14 PM, Konstantin Shvachko wrote: > Hi everybody, > > This is the next dot release of Apache Hadoop 2.7 line. The previous one > 2.7.5 was released on December 14, 2017. > Release 2.7.6 includes critical bug fixes and optimizations. See more > details in Release Note: > http://home.apache.org/~shv/hadoop-2.7.6-RC0/releasenotes.html > > The RC0 is available at: http://home.apache.org/~shv/hadoop-2.7.6-RC0/ > > Please give it a try and vote on this thread. The vote will run for 5 > days ending 04/16/2018. > > My up to date public key is available from: > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > Thanks, > --Konstantin >
[jira] [Created] (YARN-8171) [UI2] AM Node link from attempt page should not redirect to new tab
Sunil G created YARN-8171: - Summary: [UI2] AM Node link from attempt page should not redirect to new tab Key: YARN-8171 URL: https://issues.apache.org/jira/browse/YARN-8171 Project: Hadoop YARN Issue Type: Bug Components: yarn-ui-v2 Reporter: Sunil G Assignee: Sunil G Avoid node link redirection to old UI -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8170) Caching Node Rack Location
BELUGA BEHR created YARN-8170:
-
Summary: Caching Node Rack Location
Key: YARN-8170
URL: https://issues.apache.org/jira/browse/YARN-8170
Project: Hadoop YARN
Issue Type: New Feature
Components: applications, nodemanager
Affects Versions: 3.0.1
Reporter: BELUGA BEHR
When the MapReduce ApplicationMaster is trying to assign Mappers to Nodes, it
loops all of the queued Mappers and looks up the ideal rack location of each
Mapper.
Under the covers, the rack awareness script is being called, once per Mapper.
The results do get cached, but for only as long as the ApplicationMaster
exists. That means that the script gets called N times each time a new
ApplicationMaster is launched. If the rack awareness script is complex or
requires an external lookup, this can be a slow process and can even DDOS the
external lookup source.
There are at least a couple of ways to tackle this...
# Add a DNSToSwitchMapping implementation that caches in an external cache
(i.e., memcached) instead of memory so that all ApplicationMasters can share
the same cache and would rarely call the rack awareness script.
# Like the shuffle service, add a new NodeManager auxiliary which exposes a
rack lookup API so that the NodeManagers are responsible for caching the rack
locations. This would also require a DNSToSwitchMapping implementation that
interacts with this new service.
# Other?
{code:java}
String host = allocated.getNodeId().getHost();
String rack = RackResolver.resolve(host).getNetworkLocation();
{code}
[https://github.com/apache/hadoop/blob/453d48bdfbb67ed3e66c33c4aef239c3d7bdd3bc/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java#L1435-L1464]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8169) Review RackResolver.java
BELUGA BEHR created YARN-8169: - Summary: Review RackResolver.java Key: YARN-8169 URL: https://issues.apache.org/jira/browse/YARN-8169 Project: Hadoop YARN Issue Type: Improvement Components: yarn Affects Versions: 3.0.1 Reporter: BELUGA BEHR Attachments: YARN-8169.1.patch # Use SLF4J # Fix some checkstyle warnings # Minor clean up -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/
[Apr 16, 2018 5:15:37 PM] (xyao) HADOOP-15369. Avoid usage of
${project.version} in parent poms.
[Apr 16, 2018 6:24:26 PM] (eyang) YARN-8060. Added default readiness check for
service components.
[Apr 16, 2018 9:16:59 PM] (arp) HDFS-13439. Add test case for read block
operation when it is moved.
[Apr 16, 2018 10:05:04 PM] (haibochen) YARN-8162. Remove Method
DirectoryCollection#verifyDirUsingMkdir. (Yufei
[Apr 16, 2018 10:55:00 PM] (virajith) HDFS-13311. RBF:
TestRouterAdminCLI#testCreateInvalidEntry fails on
[Apr 16, 2018 11:26:54 PM] (inigoiri) YARN-7773. YARN Federation used Mysql as
state store throw exception,
[Apr 16, 2018 11:52:17 PM] (inigoiri) YARN-8165. Incorrect queue name logging
in AbstractContainerAllocator.
[Apr 17, 2018 12:32:19 AM] (inigoiri) YARN-8164. Fix a potential NPE in
AbstractSchedulerPlanFollower.
-1 overall
The following subsystems voted -1:
asflicense unit xml
The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace
The following subsystems are considered long running:
(runtime bigger than 1h 0m 0s)
unit
Specific tests:
Failed junit tests :
hadoop.util.TestBasicDiskValidator
hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy
hadoop.hdfs.web.TestWebHdfsTimeouts
hadoop.yarn.sls.TestReservationSystemInvariants
cc:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/diff-compile-cc-root.txt
[4.0K]
javac:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/diff-compile-javac-root.txt
[288K]
checkstyle:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/diff-checkstyle-root.txt
[17M]
pylint:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/diff-patch-pylint.txt
[24K]
shellcheck:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/diff-patch-shellcheck.txt
[20K]
shelldocs:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/diff-patch-shelldocs.txt
[12K]
whitespace:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/whitespace-eol.txt
[9.4M]
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/whitespace-tabs.txt
[1.1M]
xml:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/xml.txt
[4.0K]
javadoc:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/diff-javadoc-javadoc-root.txt
[760K]
unit:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
[184K]
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
[296K]
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt
[8.0K]
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
[84K]
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/patch-unit-hadoop-tools_hadoop-sls.txt
[12K]
asflicense:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/754/artifact/out/patch-asflicense-problems.txt
[4.0K]
Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
