This is an automated email from the ASF dual-hosted git repository.
keepromise pushed a change to branch HDFS-17531
in repository https://gitbox.apache.org/repos/asf/hadoop.git
omit 8efd978eae5 HDFS-17651.[ARR] Async handler executor isolation (#7244).
Contributed by hfutatzhanghb.
omit 273673cf2a8 HDFS-17650. [ARR] The router server-side rpc protocol PB
supports asynchrony. (#7139). Contributed by hfutatzhanghb.
omit c48db62c193 HDFS-17640.[ARR] RouterClientProtocol supports
asynchronous rpc. (#7188)
omit be06adcde0a HADOOP-19361. RPC DeferredMetrics bugfix. (#7220).
Contributed by hfutatzhanghb.
omit d4a6a271385 HDFS-17672. [ARR] Move asynchronous related classes to the
async package. (#7184). Contributed by Jian Zhang.
omit d8768cff46c HDFS-17659. [ARR]Router Quota supports asynchronous rpc.
(#7157). Contributed by hfutatzhanghb.
omit 33c661bd180 HDFS-17656. [ARR] RouterNamenodeProtocol and
RouterUserProtocol supports asynchronous rpc. (#7159). Contributed by Jian
Zhang.
omit ea3c4c84ba2 HDFS-17596. [ARR] RouterStoragePolicy supports
asynchronous rpc. (#6988). Contributed by hfutatzhanghb.
omit 118061c4435 HDFS-17601. [ARR] RouterRpcServer supports asynchronous
rpc. (#7108). Contributed by hfutatzhanghb.
omit 2aaa1176563 HDFS-17595. [ARR] ErasureCoding supports asynchronous rpc.
(#6983). Contributed by hfutatzhanghb.
omit 1152442a98e HDFS-17597. [ARR] RouterSnapshot supports asynchronous
rpc. (#6994). Contributed by Wenqi Li.
omit 724c7d3e1bb HDFS-17594. [ARR] RouterCacheAdmin supports asynchronous
rpc. (#6986). Contributed by Archie73.
omit 71c74660b61 HDFS-17545. [ARR] router async rpc client. (#6871).
Contributed by Jian Zhang.
omit 92173176713 HDFS-17544. [ARR] The router client rpc protocol PB
supports asynchrony. (#6870). Contributed by Jian Zhang.
omit ff2a574cd72 HADOOP-19235. IPC client uses CompletableFuture to support
asynchronous operations. (#6888)
omit a3e5258a4ce HDFS-17543. [ARR] AsyncUtil makes asynchronous code more
concise and easier. (#6868). Contributed by Jian Zhang.
add cd2cffe73f9 HADOOP-19342. SaslRpcServer.AuthMethod print INFO messages
in client side. (#7174)
add 31b8b4a8bd5 HADOOP-19336: S3A: Test failures after CSE support added
(#7164)
add b9060fc00df YARN-11738 Modernize SecretManager config (#7144)
add 2ab33c672ea HDFS-17671 Suppress callstack when adding a datanode to
deadnodes (#7180)
add 919bd18c06f HDFS-17641. Add badly distributed blocks metric (#7123)
add 65a5bf3b20c HADOOP-19226: [ABFS][FNSOverBlob] Implementing Azure Rest
APIs on Blob Endpoint for AbfsBlobClient (#6944)
add 964e0897d4a HADOOP-19346. ViewFileSystem.InnerCache: Replaced
ReentrantReadWriteLock with ConcurrentHashMap/putIfAbsent() (#7187)
add 464d7d97f9f HDFS-17668 Treat null SASL negotiated QOP as auth in
DataTransferSasl… (#7171)
add b55c9c7495a HADOOP-19349: S3A : Improve Client Side Encryption
Documentation (#7191)
add 2c2beb715d0 HADOOP-19358. Update command usage of appendToFile. (#7209)
add 86d8fa6d518 HDFS-17679 Use saslClient#hasInitialResponse() instead of
heuristics in SaslParticipant#createFirstMessage() (#7201)
add 2b2acb028c7 HDFS-17682. Fix incorrect command of fs2img tool. (#7206).
Contributed by fuchaohong.
add 93a4dd02cf2 HDFS-17648. Fix BalancerMetrics duplicate registration
issue. (#7133). Contributed by Zhaobo Huang.
add de8b5061d89 HADOOP-19337. Fix ZKFailoverController NPE issue due to
integer overflow in parseInt when initHM. (#7165). Contributed by ConfX.
add 58896f0e135 HADOOP-19339. OutofBounds Exception due to assumption
about buffer size in BlockCompressorStream. Contributed by ConfX.
add aa5fe6f550c YARN-10058. Handle uncaught exception for async-scheduling
threads to prevent scheduler hangs (#7129). Contributed by Tao Yang.
add efb83ecbc34 HADOOP-19360. Disable releases for apache.snapshots repo
(#7218)
add fc42da70cea HADOOP-19357: [ABFS] Optimizations for Retry Handling and
Client Side Throttling (#7216)
add 626455653f5 YARN-11744. Tackle flaky test
testGetRunningContainersToKill (#7227) Contributed by Cheng Pan.
add 5fe4f134afe HADOOP-19366. Install OpenJDK 17 in default ubuntu build
container (#7226) Contributed by Cheng Pan.
add 7f491908020 YARN-7327: Enable asynchronous scheduling by default for
capacity scheduler (#7138)
add 6cb2e866a6c HADOOP-19367. Fix setting final field value on Java 17
(#7228)
add f32a937511e HADOOP-19362. RPC metrics should be updated correctly when
call is defered. (#7224). Contributed by hfutatzhanghb.
add 3bf43b41441 HADOOP-19370. Fix error links of huaweicloud in site
index. (#7229). Contributed by fuchaohong.
add ecc4e00145f HADOOP-19371. JVM GC Metrics supports ZGC pause time and
count (#7230) Contributed by cxzl25.
add 71b3a6477b5 HADOOP-19335. Bump netty to 4.1.116 due to CVE-2024-47535
(#7158). Contributed by PJ Fanning
add d5b836bc34e HDFS-17655. Cannot run HDFS balancer with
BlockPlacementPolicyWithNodeGroup (#7141). Contributed by YUBI LEE.
add 7e673584a12 HADOOP-19207: [ABFS][FNSOverBlob] Response Handling of
Blob Endpoint APIs and Metadata APIs (#7210)
add 305e3e73a91 HADOOP-19380. Update the year to 2025 (#7262). Contributed
by Ayush Saxena.
add 94d6a77c394 HDFS-17496. DataNode supports more fine-grained dataset
lock based on blockid. (#6764). Contributed by farmmamba.
add fd653fa9885 HADOOP-19376. Add fs.hdfs.impl.disable.cache to
core-default.xml. (#7249)
add fd4aa2a0d41 HDFS-17683. Add metrics for acquiring dataset read/write
lock. (#7211). Contributed by farmmamba.
add f65747d4596 YARN-11751. Remove TestAppLevelTimelineCollector. (#7271)
Contributed by Shilun Fan.
add 815ca41c69b HDFS-17080. fix ec connection leak. (#5807)
add 22a9287e4a6 HDFS-17696. Optimize isBlockReplicatedOk method when
scheduleReconStruction parameter is false (#7243)
add fb1bb6429df HDFS-17695. Fix javadoc for FSDirectory#resolvePath
method. (#7238)
add a4872099908 HDFS-17602. RBF: Fix mount point with SPACE order can not
find the available namespace. (#6991). Contributed by Zhongkun Wu.
add d44ac28e326 HADOOP-19278. S3A: Remove option to delete directory
markers (#7052)
add f0ab1e69723 Revert "HDFS-17496. DataNode supports more fine-grained
dataset lock based on…" (#7279)
add 37bdf945d1a YARN-11743. Cgroup v2 support should fall back to v1 when
there are no v2 controllers (#7222)
add 1bd4835a3d1 HADOOP-19305: Fix ProcessEnvironment ClassCastException in
Shell.java (#7106) Contributed by Butao Zhang.
add d2095fae38e HADOOP-19382. [ABFS][FnsOverBlob] Test Fix for
ITestAzureBlobFileSystemInitAndCreate failure (#7269)
add c24d569a6e6 YARN-11752: Global Scheduler: Improve the container
allocation time (#7277) Contributed by Syed Shameerur Rahman.
add 266dad1617d HDFS-17706. TestBlockTokenWithDFSStriped fails due to
closed streams. (#7275). Contribtued by Felix N.
add c3e322860f7 HADOOP-19229. S3A/ABFS: Vector IO on cloud storage:
increase threshold for range merging (#7281)
add f38d7072566 HADOOP-19351. S3A: Add config option to skip test with
performance mode (#7223)
add 06d36f5a6a1 HADOOP-15984. Update jersey from 1.19 to 2.x. (#7019)
add c12cbea7760 HDFS-17543. [ARR] AsyncUtil makes asynchronous code more
concise and easier. (#6868). Contributed by Jian Zhang.
add 846b113df5a HADOOP-19235. IPC client uses CompletableFuture to support
asynchronous operations. (#6888)
add 1e6b302f858 HDFS-17544. [ARR] The router client rpc protocol PB
supports asynchrony. (#6870). Contributed by Jian Zhang.
add fd64eb4f419 HDFS-17545. [ARR] router async rpc client. (#6871).
Contributed by Jian Zhang.
add abb9346a69a HDFS-17594. [ARR] RouterCacheAdmin supports asynchronous
rpc. (#6986). Contributed by Archie73.
add 3083d443449 HDFS-17597. [ARR] RouterSnapshot supports asynchronous
rpc. (#6994). Contributed by Wenqi Li.
add 27847675cec HDFS-17595. [ARR] ErasureCoding supports asynchronous rpc.
(#6983). Contributed by hfutatzhanghb.
add 720e000e3ce HDFS-17601. [ARR] RouterRpcServer supports asynchronous
rpc. (#7108). Contributed by hfutatzhanghb.
add d076822c47e HDFS-17596. [ARR] RouterStoragePolicy supports
asynchronous rpc. (#6988). Contributed by hfutatzhanghb.
add 5f92a3df801 HDFS-17656. [ARR] RouterNamenodeProtocol and
RouterUserProtocol supports asynchronous rpc. (#7159). Contributed by Jian
Zhang.
add 2fe85fbca41 HDFS-17659. [ARR]Router Quota supports asynchronous rpc.
(#7157). Contributed by hfutatzhanghb.
add bb014e64a7a HDFS-17672. [ARR] Move asynchronous related classes to the
async package. (#7184). Contributed by Jian Zhang.
add 1f1d7c1ef8e HADOOP-19361. RPC DeferredMetrics bugfix. (#7220).
Contributed by hfutatzhanghb.
add 1e0dabc5946 HDFS-17640.[ARR] RouterClientProtocol supports
asynchronous rpc. (#7188)
add d83dae4cae8 HDFS-17650. [ARR] The router server-side rpc protocol PB
supports asynchrony. (#7139). Contributed by hfutatzhanghb.
add 616d707bcdc HDFS-17651.[ARR] Async handler executor isolation (#7244).
Contributed by hfutatzhanghb.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (8efd978eae5)
\
N -- N -- N refs/heads/HDFS-17531 (616d707bcdc)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
LICENSE-binary | 104 +-
dev-support/docker/pkg-resolver/packages.json | 6 +-
.../hadoop-client-check-invariants/pom.xml | 2 +
.../resources/ensure-jars-have-correct-contents.sh | 6 +-
.../hadoop-client-check-test-invariants/pom.xml | 1 +
.../hadoop-client-integration-tests/pom.xml | 10 +-
.../hadoop-client-minicluster/pom.xml | 106 +-
.../hadoop-client-runtime/pom.xml | 26 +-
hadoop-client-modules/hadoop-client/pom.xml | 76 -
.../hadoop-cloud-storage/pom.xml | 12 -
.../hadoop-huaweicloud/pom.xml | 6 +
.../apache/hadoop/fs/obs/OBSBlockOutputStream.java | 2 +-
.../org/apache/hadoop/fs/obs/OBSInputStream.java | 2 +-
hadoop-common-project/hadoop-auth-examples/pom.xml | 4 +-
hadoop-common-project/hadoop-auth/pom.xml | 4 +-
hadoop-common-project/hadoop-common/pom.xml | 97 +-
.../hadoop/fs/CommonConfigurationKeysPublic.java | 9 +
.../org/apache/hadoop/fs/PositionedReadable.java | 7 +-
.../apache/hadoop/fs/viewfs/ViewFileSystem.java | 50 +-
.../org/apache/hadoop/ha/ZKFailoverController.java | 7 +-
.../java/org/apache/hadoop/http/HttpServer2.java | 34 +-
.../src/main/java/org/apache/hadoop/io/Sizes.java | 94 +
.../hadoop/io/compress/BlockCompressorStream.java | 6 +-
.../org/apache/hadoop/ipc/ProtobufRpcEngine.java | 25 +-
.../org/apache/hadoop/ipc/ProtobufRpcEngine2.java | 24 +-
.../main/java/org/apache/hadoop/ipc/Server.java | 55 +-
.../apache/hadoop/metrics2/source/JvmMetrics.java | 7 +
.../hadoop/security/SaslMechanismFactory.java | 28 +-
.../org/apache/hadoop/security/SaslRpcServer.java | 15 +-
.../hadoop/security/token/SecretManager.java | 46 +-
.../main/java/org/apache/hadoop/util/Shell.java | 4 +-
.../src/main/resources/core-default.xml | 33 +
.../src/site/markdown/FileSystemShell.md | 8 +-
.../hadoop-common/src/site/markdown/Metrics.md | 5 +
.../hadoop/conf/TestCommonConfigurationFields.java | 1 +
.../org/apache/hadoop/http/TestHttpServer.java | 18 +-
.../org/apache/hadoop/http/TestHttpServerLogs.java | 9 +-
.../hadoop/ipc/TestProtoBufRpcServerHandoff.java | 58 +-
.../org/apache/hadoop/test/ReflectionUtils.java | 25 +
hadoop-common-project/hadoop-kms/pom.xml | 15 +-
.../hadoop-kms/src/main/conf/kms-log4j.properties | 1 -
.../apache/hadoop/crypto/key/kms/server/KMS.java | 14 +-
.../key/kms/server/KMSExceptionsProvider.java | 2 +-
.../src/main/resources/webapps/kms/WEB-INF/web.xml | 16 +-
.../hadoop/crypto/key/kms/server/TestKMS.java | 4 +-
.../hadoop-kms/src/test/resources/log4j.properties | 2 -
hadoop-common-project/hadoop-nfs/pom.xml | 4 +-
hadoop-hdfs-project/hadoop-hdfs-client/pom.xml | 5 +
.../org/apache/hadoop/hdfs/DFSInputStream.java | 6 +-
.../java/org/apache/hadoop/hdfs/StripeReader.java | 40 +-
.../hadoop/hdfs/protocol/ECBlockGroupStats.java | 26 +-
.../hadoop/hdfs/protocol/ReplicatedBlockStats.java | 23 +-
.../datatransfer/sasl/DataTransferSaslUtil.java | 9 +-
.../datatransfer/sasl/SaslParticipant.java | 9 +-
.../hadoop/hdfs/protocolPB/PBHelperClient.java | 13 +-
.../src/main/proto/ClientNamenodeProtocol.proto | 2 +
hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml | 11 +-
.../fs/http/server/HttpFSExceptionProvider.java | 2 +-
.../fs/http/server/HttpFSParametersProvider.java | 2 -
.../apache/hadoop/fs/http/server/HttpFSServer.java | 45 +-
.../apache/hadoop/lib/wsrs/JSONMapProvider.java | 14 +-
.../apache/hadoop/lib/wsrs/ParametersProvider.java | 67 +-
.../main/resources/webapps/webhdfs/WEB-INF/web.xml | 18 +-
.../src/main/webapp/WEB-INF/web.xml | 18 +-
.../hadoop/fs/http/server/TestHttpFSServer.java | 3 +-
.../src/test/resources/default-log4j.properties | 1 -
hadoop-hdfs-project/hadoop-hdfs-nfs/pom.xml | 10 +-
hadoop-hdfs-project/hadoop-hdfs-rbf/pom.xml | 5 +
.../server/federation/metrics/FederationMBean.java | 7 +
.../federation/metrics/NamenodeBeanMetrics.java | 10 +
.../hdfs/server/federation/metrics/RBFMetrics.java | 6 +
.../resolver/MembershipNamenodeResolver.java | 2 +
.../federation/resolver/NamenodeStatusReport.java | 15 +
.../resolver/order/AvailableSpaceResolver.java | 7 +-
.../router/NamenodeHeartbeatService.java | 5 +-
.../federation/router/RouterWebHdfsMethods.java | 3 -
.../federation/store/records/MembershipStats.java | 7 +-
.../records/impl/pb/MembershipStatsPBImpl.java | 14 +-
.../src/main/proto/FederationProtocol.proto | 3 +-
.../server/federation/metrics/TestRBFMetrics.java | 4 +
.../resolver/order/TestAvailableSpaceResolver.java | 8 +
hadoop-hdfs-project/hadoop-hdfs/pom.xml | 24 +-
.../hadoop/hdfs/server/balancer/Balancer.java | 6 +-
.../hdfs/server/blockmanagement/BlockManager.java | 12 +-
.../DatanodeAdminBackoffMonitor.java | 17 +-
.../DatanodeAdminDefaultMonitor.java | 15 +-
.../blockmanagement/DatanodeAdminMonitorBase.java | 18 +
.../blockmanagement/LowRedundancyBlocks.java | 20 +-
.../hadoop/hdfs/server/datanode/DataNode.java | 4 +-
.../hdfs/server/datanode/DataSetLockManager.java | 21 +-
.../server/datanode/metrics/DataNodeMetrics.java | 12 +
.../datanode/web/webhdfs/ExceptionHandler.java | 4 +-
.../hadoop/hdfs/server/namenode/FSDirectory.java | 2 +-
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 22 +-
.../hdfs/server/namenode/NameNodeHttpServer.java | 27 +-
.../hdfs/server/namenode/NameNodeMXBean.java | 8 +
.../namenode/metrics/ReplicatedBlocksMBean.java | 5 +
.../web/resources/NamenodeWebHdfsMethods.java | 285 ++-
.../org/apache/hadoop/hdfs/tools/DFSAdmin.java | 6 +-
.../org/apache/hadoop/hdfs/web/ParamFilter.java | 103 +-
.../hdfs/web/resources/ExceptionHandler.java | 11 +-
.../hdfs/web/resources/TokenServiceParam.java | 2 -
.../hadoop/hdfs/web/resources/UserProvider.java | 35 +-
.../src/site/markdown/HdfsProvidedStorage.md | 2 +-
.../org/apache/hadoop/hdfs/TestFileCreation.java | 20 +-
.../hdfs/TestStoragePolicyPermissionSettings.java | 24 +-
.../hadoop/hdfs/server/balancer/TestBalancer.java | 19 +
.../balancer/TestBalancerLongRunningTasks.java | 60 +
.../blockmanagement/TestBlockTokenWithDFS.java | 3 +
.../TestLowRedundancyBlockQueues.java | 42 +-
.../hdfs/server/datanode/TestDataNodeMetrics.java | 20 +
.../namenode/metrics/TestNameNodeMetrics.java | 7 +
.../hdfs/web/TestWebHdfsFileSystemContract.java | 10 +-
.../src/test/resources/log4j.properties | 4 -
.../hadoop-mapreduce-client-app/pom.xml | 44 +
.../jobhistory/JobHistoryEventHandler.java | 8 +-
.../mapreduce/v2/app/client/MRClientService.java | 28 +-
.../hadoop/mapreduce/v2/app/webapp/AMWebApp.java | 19 +-
.../mapreduce/v2/app/webapp/AMWebServices.java | 11 +-
.../apache/hadoop/mapreduce/v2/app/webapp/App.java | 7 +-
.../v2/app/webapp/JAXBContextResolver.java | 15 +-
.../mapreduce/v2/app/webapp/TestAMWebApp.java | 45 +-
.../mapreduce/v2/app/webapp/TestAMWebServices.java | 245 ++-
.../v2/app/webapp/TestAMWebServicesAttempt.java | 145 +-
.../v2/app/webapp/TestAMWebServicesAttempts.java | 246 ++-
.../v2/app/webapp/TestAMWebServicesJobConf.java | 124 +-
.../v2/app/webapp/TestAMWebServicesJobs.java | 434 ++--
.../v2/app/webapp/TestAMWebServicesTasks.java | 427 ++--
.../hadoop-mapreduce-client-common/pom.xml | 5 +
.../hadoop-mapreduce-client-core/pom.xml | 20 +
.../org/apache/hadoop/mapred/LineRecordReader.java | 7 +-
.../hadoop-mapreduce-client-hs/pom.xml | 30 +
.../mapreduce/v2/hs/HistoryClientService.java | 38 +
.../hadoop/mapreduce/v2/hs/webapp/HsWebApp.java | 4 -
.../mapreduce/v2/hs/webapp/HsWebServices.java | 34 +-
.../v2/hs/webapp/JAXBContextResolver.java | 10 +-
.../mapreduce/v2/hs/webapp/TestHsWebServices.java | 253 ++-
.../v2/hs/webapp/TestHsWebServicesAttempts.java | 257 ++-
.../v2/hs/webapp/TestHsWebServicesJobConf.java | 120 +-
.../v2/hs/webapp/TestHsWebServicesJobs.java | 468 ++---
.../v2/hs/webapp/TestHsWebServicesJobsQuery.java | 471 +++--
.../v2/hs/webapp/TestHsWebServicesLogs.java | 398 ++--
.../v2/hs/webapp/TestHsWebServicesLogsExtend.java | 399 ++++
.../v2/hs/webapp/TestHsWebServicesTasks.java | 446 ++---
.../reader/ContainerLogsInfoMessageBodyReader.java | 103 +
.../reader/RemoteLogPathsMessageBodyReader.java | 88 +
.../hadoop-mapreduce-client-jobclient/pom.xml | 21 +
.../hadoop-mapreduce-client-nativetask/pom.xml | 20 +
.../hadoop-mapreduce-client-shuffle/pom.xml | 5 +
.../hadoop-mapreduce-client/pom.xml | 9 +-
.../hadoop-mapreduce-examples/pom.xml | 9 +-
hadoop-project/pom.xml | 193 +-
hadoop-project/src/site/site.xml | 2 +-
hadoop-tools/hadoop-archive-logs/pom.xml | 20 +
hadoop-tools/hadoop-archives/pom.xml | 20 +
hadoop-tools/hadoop-aws/pom.xml | 73 +-
.../java/org/apache/hadoop/fs/s3a/Constants.java | 40 +-
.../java/org/apache/hadoop/fs/s3a/Listing.java | 49 +-
.../org/apache/hadoop/fs/s3a/S3AFileSystem.java | 194 +-
.../java/org/apache/hadoop/fs/s3a/S3AUtils.java | 1 -
.../apache/hadoop/fs/s3a/WriteOperationHelper.java | 25 +-
.../fs/s3a/auth/CredentialProviderListFactory.java | 4 -
.../fs/s3a/commit/impl/CommitOperations.java | 2 +-
.../fs/s3a/commit/magic/S3MagicCommitTracker.java | 4 +-
.../hadoop/fs/s3a/impl/DirMarkerTracker.java | 26 -
.../apache/hadoop/fs/s3a/impl/DirectoryPolicy.java | 110 --
.../hadoop/fs/s3a/impl/DirectoryPolicyImpl.java | 212 --
.../fs/s3a/impl/EncryptionS3ClientFactory.java | 2 +
.../apache/hadoop/fs/s3a/impl/MkdirOperation.java | 15 +-
.../hadoop/fs/s3a/impl/PutObjectOptions.java | 38 +-
.../hadoop/fs/s3a/impl/S3AMultipartUploader.java | 2 +-
.../org/apache/hadoop/fs/s3a/s3guard/S3Guard.java | 6 -
.../apache/hadoop/fs/s3a/s3guard/S3GuardTool.java | 67 +-
.../org/apache/hadoop/fs/s3a/tools/MarkerTool.java | 93 +-
.../hadoop/fs/s3native/NativeS3FileSystem.java | 148 --
.../apache/hadoop/fs/s3native/S3xLoginHelper.java | 87 +-
.../services/org.apache.hadoop.fs.FileSystem | 16 -
.../markdown/tools/hadoop-aws/assumed_roles.md | 11 +-
.../markdown/tools/hadoop-aws/directory_markers.md | 466 +----
.../site/markdown/tools/hadoop-aws/encryption.md | 247 ++-
.../src/site/markdown/tools/hadoop-aws/index.md | 8 +-
.../site/markdown/tools/hadoop-aws/performance.md | 10 +-
.../src/site/markdown/tools/hadoop-aws/s3guard.md | 7 +-
.../src/site/markdown/tools/hadoop-aws/s3n.md | 52 -
.../src/site/markdown/tools/hadoop-aws/testing.md | 63 +-
.../tools/hadoop-aws/troubleshooting_s3a.md | 158 --
.../fs/contract/s3a/ITestS3AContractCreate.java | 5 +
.../s3a/ITestS3AContractMkdirWithCreatePerf.java | 4 +
.../contract/s3a/ITestS3AContractVectoredRead.java | 29 +-
.../apache/hadoop/fs/s3a/AbstractS3ATestBase.java | 51 -
.../hadoop/fs/s3a/ITestDowngradeSyncable.java | 5 -
.../fs/s3a/ITestS3AClientSideEncryption.java | 10 +-
.../fs/s3a/ITestS3AClientSideEncryptionCustom.java | 2 -
.../hadoop/fs/s3a/ITestS3AFileOperationCost.java | 31 +-
.../hadoop/fs/s3a/ITestS3AMiscOperations.java | 2 +-
.../fs/s3a/ITestS3APrefetchingCacheFiles.java | 4 -
.../fs/s3a/ITestS3APrefetchingInputStream.java | 4 -
.../fs/s3a/ITestS3APrefetchingLruEviction.java | 1 -
.../apache/hadoop/fs/s3a/MockS3AFileSystem.java | 8 -
.../apache/hadoop/fs/s3a/MultipartTestUtils.java | 2 +-
.../org/apache/hadoop/fs/s3a/S3ATestConstants.java | 11 +-
.../org/apache/hadoop/fs/s3a/S3ATestUtils.java | 16 +-
.../hadoop/fs/s3a/TestS3ABlockOutputStream.java | 2 +-
.../fs/s3a/audit/ITestAuditAccessChecks.java | 4 -
.../hadoop/fs/s3a/audit/ITestAuditManager.java | 4 -
.../fs/s3a/audit/ITestAuditManagerDisabled.java | 4 -
.../fs/s3a/commit/ITestCommitOperationCost.java | 45 +-
.../hadoop/fs/s3a/commit/ITestUploadRecovery.java | 31 +-
.../hadoop/fs/s3a/impl/ITestAwsSdkWorkarounds.java | 1 +
.../fs/s3a/impl/ITestPartialRenamesDeletes.java | 4 -
.../hadoop/fs/s3a/impl/ITestRenameDeleteRace.java | 23 +-
.../apache/hadoop/fs/s3a/impl/ITestXAttrCost.java | 4 -
.../fs/s3a/impl/TestDirectoryMarkerPolicy.java | 163 --
.../hadoop/fs/s3a/impl/TestRequestFactory.java | 6 +-
.../fs/s3a/performance/AbstractS3ACostTest.java | 91 +-
.../fs/s3a/performance/ITestCreateFileCost.java | 6 +-
.../performance/ITestDirectoryMarkerListing.java | 82 +-
.../fs/s3a/performance/ITestS3ADeleteCost.java | 105 +-
.../s3a/performance/ITestS3AMiscOperationCost.java | 7 +-
.../fs/s3a/performance/ITestS3AMkdirCost.java | 23 -
.../fs/s3a/performance/ITestS3AOpenCost.java | 4 -
.../fs/s3a/performance/ITestS3ARenameCost.java | 52 +-
.../fs/s3a/performance/ITestUnbufferDraining.java | 7 -
.../s3a/s3guard/AbstractS3GuardToolTestBase.java | 6 +-
.../fs/s3a/s3guard/TestAuthoritativePath.java | 132 --
.../fs/s3a/scale/ITestS3ADirectoryPerformance.java | 9 +-
.../test/MinimalWriteOperationHelperCallbacks.java | 7 -
.../fs/s3a/tools/AbstractMarkerToolTest.java | 95 +-
.../hadoop/fs/s3a/tools/ITestMarkerTool.java | 172 +-
.../hadoop/fs/s3native/TestS3xLoginHelper.java | 205 --
hadoop-tools/hadoop-azure-datalake/pom.xml | 21 +
hadoop-tools/hadoop-azure/pom.xml | 25 +-
.../src/config/checkstyle-suppressions.xml | 4 +
.../hadoop/fs/azurebfs/AbfsConfiguration.java | 47 +-
.../hadoop/fs/azurebfs/AzureBlobFileSystem.java | 54 +-
.../fs/azurebfs/AzureBlobFileSystemStore.java | 224 ++-
.../fs/azurebfs/constants/AbfsHttpConstants.java | 80 +-
.../constants/FileSystemConfigurations.java | 8 +-
.../azurebfs/constants/FileSystemUriSchemes.java | 4 +-
.../constants/HttpHeaderConfigurations.java | 31 +-
.../fs/azurebfs/constants/HttpQueryParams.java | 31 +
.../services/AppendRequestParameters.java | 59 +
.../contracts/services/AzureServiceErrorCode.java | 2 +
.../services/BlobAppendRequestParameters.java | 38 +-
.../services/BlobListResultEntrySchema.java | 238 +++
.../contracts/services/BlobListResultSchema.java | 71 +
.../contracts/services/BlobListXmlParser.java | 304 +++
...rySchema.java => DfsListResultEntrySchema.java} | 48 +-
...tResultSchema.java => DfsListResultSchema.java} | 21 +-
.../contracts/services/ListResultEntrySchema.java | 218 +--
.../contracts/services/ListResultSchema.java | 29 +-
.../services/StorageErrorResponseSchema.java | 62 +
.../fs/azurebfs/services/AbfsAHCHttpOperation.java | 32 +-
.../fs/azurebfs/services/AbfsBlobClient.java | 1472 ++++++++++++++
.../hadoop/fs/azurebfs/services/AbfsClient.java | 69 +-
.../fs/azurebfs/services/AbfsClientHandler.java | 62 +-
.../services/AbfsClientThrottlingIntercept.java | 2 +
.../hadoop/fs/azurebfs/services/AbfsDfsClient.java | 195 +-
.../fs/azurebfs/services/AbfsHttpOperation.java | 283 ++-
.../fs/azurebfs/services/AbfsInputStream.java | 14 +
.../fs/azurebfs/services/AbfsJdkHttpOperation.java | 12 +-
.../fs/azurebfs/services/AbfsRestOperation.java | 35 +-
.../azurebfs/services/AbfsRestOperationType.java | 15 +
.../fs/azurebfs/services/ListingSupport.java | 2 +-
.../apache/hadoop/fs/azurebfs/utils/UriUtils.java | 29 +-
.../hadoop-azure/src/site/markdown/blobEndpoint.md | 102 +
.../hadoop-azure/src/site/markdown/fns_blob.md | 8 +-
.../apache/hadoop/fs/azurebfs/ITestAbfsClient.java | 6 +-
.../azurebfs/ITestAzureBlobFileSystemCreate.java | 22 +-
.../azurebfs/ITestAzureBlobFileSystemDelete.java | 10 +-
.../ITestAzureBlobFileSystemInitAndCreate.java | 13 +-
.../ITestAzureBlobFileSystemListStatus.java | 10 +-
.../extensions/MockDelegationSASTokenProvider.java | 2 +-
.../fs/azurebfs/services/ITestAbfsClient.java | 111 +-
.../azurebfs/services/ITestAbfsClientHandler.java | 49 +
.../services/ITestAbfsPaginatedDelete.java | 3 +-
.../services/ITestExponentialRetryPolicy.java | 18 +
.../fs/azurebfs/services/TestAbfsPerfTracker.java | 18 +-
.../hadoop/fs/azurebfs/utils/TestMockHelpers.java | 59 -
.../hadoop/fs/azurebfs/utils/TestUriUtils.java | 60 +
hadoop-tools/hadoop-datajoin/pom.xml | 20 +
hadoop-tools/hadoop-distcp/pom.xml | 20 +
.../hadoop-dynamometer-blockgen/pom.xml | 20 +
.../hadoop-dynamometer-infra/pom.xml | 20 +
.../hadoop-dynamometer-workload/pom.xml | 20 +
hadoop-tools/hadoop-extras/pom.xml | 20 +
hadoop-tools/hadoop-federation-balance/pom.xml | 20 +
hadoop-tools/hadoop-fs2img/pom.xml | 20 +
hadoop-tools/hadoop-gridmix/pom.xml | 20 +
hadoop-tools/hadoop-resourceestimator/pom.xml | 53 +-
.../service/ResourceEstimatorService.java | 33 +-
.../service/GuiceServletConfig.java | 42 -
.../service/TestResourceEstimatorService.java | 69 +-
hadoop-tools/hadoop-rumen/pom.xml | 20 +
hadoop-tools/hadoop-sls/pom.xml | 20 +
hadoop-tools/hadoop-streaming/pom.xml | 20 +
.../hadoop-yarn/hadoop-yarn-api/pom.xml | 4 +-
.../api/records/timeline/TimelineEntities.java | 2 +
.../yarn/api/records/timeline/TimelineEntity.java | 11 +-
.../yarn/api/records/timeline/TimelineEvent.java | 2 +
.../yarn/api/records/timeline/TimelineEvents.java | 5 +-
.../api/records/timeline/TimelinePutResponse.java | 7 +
.../timeline/reader/TimelineDomainReader.java | 57 +
.../timeline/reader/TimelineEntitiesReader.java | 57 +
.../timeline/reader/TimelinePutResponseReader.java | 58 +
.../api/records/timeline/reader/package-info.java} | 10 +-
.../timeline/writer/TimelineDomainWriter.java | 65 +
.../timeline/writer/TimelineDomainsWriter.java | 65 +
.../timeline/writer/TimelineEntitiesWriter.java | 65 +
.../timeline/writer/TimelineEntityWriter.java | 65 +
.../timeline/writer/TimelineEventsWriter.java | 66 +
.../timeline/writer/TimelinePutResponseWriter.java | 66 +
.../api/records/timeline/writer/package-info.java} | 10 +-
.../records/timelineservice/TimelineEntities.java | 2 +
.../records/timelineservice/TimelineEntity.java | 6 +
.../api/records/timelineservice/TimelineEvent.java | 3 +
.../records/timelineservice/TimelineMetric.java | 12 +-
.../reader/TimelineDomainReader.java | 57 +
.../reader/TimelineEntitiesReader.java | 57 +
.../reader/TimelineEntityReader.java | 57 +
.../timelineservice/reader/package-info.java} | 10 +-
.../writer/TimelineDomainWriter.java | 65 +
.../writer/TimelineEntitiesWriter.java | 65 +
.../writer/TimelineEntitySetWriter.java | 67 +
.../writer/TimelineEntityWriter.java | 67 +
.../writer/TimelineHealthWriter.java | 66 +
.../timelineservice/writer/package-info.java} | 10 +-
.../pom.xml | 23 -
.../appcatalog/application/YarnServiceClient.java | 80 +-
.../src/main/webapp/WEB-INF/web.xml | 8 +-
.../pom.xml | 17 +
.../distributedshell/ApplicationMaster.java | 10 +-
.../distributedshell/TestDSTimelineV10.java | 37 +-
.../hadoop-yarn-services-api/pom.xml | 10 +-
.../yarn/service/client/ApiServiceClient.java | 110 +-
.../src/main/webapp/WEB-INF/web.xml | 8 +-
.../apache/hadoop/yarn/service/TestApiServer.java | 53 +-
.../hadoop-yarn-services-core/pom.xml | 20 +-
.../hadoop/yarn/service/ServiceScheduler.java | 14 +-
.../apache/hadoop/yarn/service/utils/HttpUtil.java | 13 +-
.../hadoop/yarn/service/ServiceTestUtils.java | 21 +-
.../hadoop-yarn/hadoop-yarn-client/pom.xml | 17 +-
.../org/apache/hadoop/yarn/client/cli/LogsCLI.java | 268 ++-
.../hadoop/yarn/client/cli/SchedConfCLI.java | 138 +-
.../apache/hadoop/yarn/client/cli/TestLogsCLI.java | 23 +-
.../hadoop/yarn/client/cli/TestSchedConfCLI.java | 114 +-
.../hadoop-yarn/hadoop-yarn-common/pom.xml | 74 +-
.../yarn/client/api/impl/DirectTimelineWriter.java | 9 +-
.../client/api/impl/FileSystemTimelineWriter.java | 8 +-
.../yarn/client/api/impl/TimelineClientImpl.java | 12 +-
.../yarn/client/api/impl/TimelineConnector.java | 223 +--
.../client/api/impl/TimelineReaderClientImpl.java | 90 +-
.../yarn/client/api/impl/TimelineV2ClientImpl.java | 85 +-
.../yarn/client/api/impl/TimelineWriter.java | 64 +-
.../hadoop/yarn/logaggregation/LogToolUtils.java | 16 +-
.../hadoop/yarn/webapp/ConflictException.java} | 37 +-
.../yarn/webapp/GenericExceptionHandler.java | 29 +-
.../java/org/apache/hadoop/yarn/webapp/WebApp.java | 32 +-
.../org/apache/hadoop/yarn/webapp/WebApps.java | 39 +-
.../yarn/webapp/YarnJacksonJaxbJsonProvider.java | 2 +-
.../hadoop/yarn/webapp/util/WebAppUtils.java | 16 +
.../hadoop/yarn/webapp/util/WebServiceClient.java | 52 +-
.../yarn/webapp/util/YarnWebServiceUtils.java | 75 +-
.../yarn/client/api/impl/TestTimelineClient.java | 80 +-
.../api/impl/TestTimelineClientForATS1_5.java | 18 +-
.../api/impl/TestTimelineReaderClientImpl.java | 34 +-
.../hadoop/yarn/event/TestAsyncDispatcher.java | 5 +-
.../apache/hadoop/yarn/webapp/JerseyTestBase.java | 37 +-
.../yarn/webapp/MyTestJAXBContextResolver.java | 11 +-
.../hadoop/yarn/webapp/MyTestWebService.java | 2 +-
.../org/apache/hadoop/yarn/webapp/TestWebApp.java | 110 +-
.../yarn/webapp/util/TestWebServiceClient.java | 7 +-
.../hadoop-yarn/hadoop-yarn-csi/pom.xml | 9 +-
.../pom.xml | 66 +-
.../ApplicationHistoryServer.java | 47 +-
.../webapp/AHSWebApp.java | 12 +-
.../webapp/AHSWebServices.java | 20 +-
.../webapp/ContextFactory.java | 2 +-
.../webapp/JAXBContextResolver.java | 11 +-
.../timeline/webapp/TimelineWebServices.java | 35 +-
.../webapp/TestAHSWebServices.java | 461 +++--
.../reader/ContainerLogsInfoListReader.java | 117 ++
.../timeline/reader/TimelineAboutReader.java | 82 +
.../timeline/reader/TimelineDomainReader.java | 57 +
.../timeline/reader/TimelineDomainsReader.java | 57 +
.../timeline/reader/TimelineEntitiesReader.java | 57 +
.../timeline/reader/TimelineEntityReader.java | 69 +
.../timeline/reader/TimelineEventsReader.java | 80 +
.../timeline/reader/TimelinePutResponseReader.java | 57 +
.../TestTimelineAuthenticationFilterForV1.java | 4 +-
.../timeline/webapp/TestTimelineWebServices.java | 929 +++++----
.../webapp/TestTimelineWebServicesWithSSL.java | 17 +-
.../hadoop-yarn-server-common/pom.xml | 24 +
.../policies/dao/WeightedPolicyInfo.java | 57 +-
.../federation/store/records/SubClusterIdInfo.java | 2 +-
.../hadoop/yarn/server/webapp/LogServlet.java | 4 +-
.../hadoop/yarn/server/webapp/LogWebService.java | 70 +-
.../yarn/server/webapp/LogWebServiceUtils.java | 9 +-
.../yarn/server/webapp/dao/ContainerLogsInfo.java | 29 +-
.../pom.xml | 27 +-
.../server/globalpolicygenerator/GPGUtils.java | 29 +-
.../GlobalPolicyGenerator.java | 2 +-
.../globalpolicygenerator/webapp/GPGWebApp.java | 23 +-
.../webapp/GPGWebServices.java | 10 +-
.../policygenerator/TestPolicyGenerator.java | 23 +-
.../webapp/TestGPGWebServices.java | 76 +-
.../src/test/resources/schedulerInfo1.json | 250 +--
.../src/test/resources/schedulerInfo2.json | 374 ++--
.../hadoop-yarn-server-nodemanager/pom.xml | 96 +-
.../nodemanager/api/deviceplugin/Device.java | 13 +-
.../linux/resources/ResourceHandlerModule.java | 22 +-
.../deviceframework/AssignedDevice.java | 4 +-
.../resourceplugin/gpu/AssignedGpuDevice.java | 3 +
.../resourceplugin/gpu/GpuDevice.java | 3 +
.../nodemanager/webapp/JAXBContextResolver.java | 15 +-
.../server/nodemanager/webapp/NMWebAppFilter.java | 37 +-
.../server/nodemanager/webapp/NMWebServices.java | 50 +-
.../yarn/server/nodemanager/webapp/WebServer.java | 79 +-
.../server/nodemanager/webapp/dao/AppsInfo.java | 3 +-
.../webapp/dao/NMDeviceResourceInfo.java | 8 +
.../nodemanager/webapp/dao/NMResourceInfo.java | 13 +
.../webapp/dao/gpu/NMGpuResourceInfo.java | 8 +
.../security/TestNMTokenSecretManagerInNM.java | 2 +
.../server/nodemanager/webapp/TestNMWebFilter.java | 17 +-
.../server/nodemanager/webapp/TestNMWebServer.java | 20 +-
.../nodemanager/webapp/TestNMWebServices.java | 492 ++---
.../nodemanager/webapp/TestNMWebServicesApps.java | 404 ++--
.../webapp/TestNMWebServicesAuxServices.java | 155 +-
.../webapp/TestNMWebServicesContainers.java | 251 ++-
.../nodemanager/webapp/TestNMWebTerminal.java | 19 +-
.../hadoop-yarn-server-resourcemanager/pom.xml | 92 +-
.../server/resourcemanager/ResourceManager.java | 27 +-
.../federation/FederationStateStoreHeartbeat.java | 42 +-
.../scheduler/capacity/CapacityScheduler.java | 16 +-
.../capacity/CapacitySchedulerConfiguration.java | 2 +-
.../resourcemanager/scheduler/fair/FSQueue.java | 2 +-
.../fair/converter/FSYarnSiteConverter.java | 2 +-
.../webapp/JAXBContextResolver.java | 71 +-
.../server/resourcemanager/webapp/RMWebApp.java | 34 +-
.../resourcemanager/webapp/RMWebAppFilter.java | 33 +-
.../resourcemanager/webapp/RMWebServices.java | 15 +-
.../resourcemanager/webapp/dao/AppsInfo.java | 2 +-
.../webapp/dao/CapacitySchedulerInfo.java | 2 +-
.../webapp/dao/CapacitySchedulerQueueInfoList.java | 2 +-
.../yarn/server/resourcemanager/TestRMHA.java | 27 +-
.../resourcemanager/TestRMHAForAsyncScheduler.java | 52 +
.../TestFederationRMStateStoreService.java | 22 +-
.../recovery/TestFSRMStateStore.java | 23 +-
.../scheduler/TestAbstractYarnScheduler.java | 8 +-
.../TestCapacitySchedulerAsyncScheduling.java | 39 +-
.../fair/converter/TestFSYarnSiteConverter.java | 4 +-
.../TestWeightToPercentageConverter.java | 10 +-
.../TestWeightToWeightConverter.java | 8 +-
.../webapp/ActivitiesTestUtils.java | 26 +-
.../resourcemanager/webapp/TestRMWebApp.java | 29 +-
.../webapp/TestRMWebAppFairScheduler.java | 38 +-
.../webapp/TestRMWebServiceAppsNodelabel.java | 94 +-
.../resourcemanager/webapp/TestRMWebServices.java | 369 ++--
.../webapp/TestRMWebServicesAppAttempts.java | 258 +--
.../TestRMWebServicesAppCustomResourceTypes.java | 85 +-
.../webapp/TestRMWebServicesApps.java | 1012 +++++-----
.../TestRMWebServicesAppsCustomResourceTypes.java | 97 +-
.../webapp/TestRMWebServicesAppsModification.java | 669 ++++---
.../webapp/TestRMWebServicesCapacitySched.java | 178 +-
...TestRMWebServicesCapacitySchedDefaultLabel.java | 139 ++
...estRMWebServicesCapacitySchedDynamicConfig.java | 185 +-
...icesCapacitySchedDynamicConfigAbsoluteMode.java | 140 ++
...rvicesCapacitySchedDynamicConfigWeightMode.java | 138 ++
...esCapacitySchedDynamicConfigWeightModeDQC.java} | 172 +-
...ebServicesCapacitySchedLegacyQueueCreation.java | 114 +-
...pacitySchedLegacyQueueCreationAbsoluteMode.java | 140 ++
...WebServicesCapacitySchedulerConfigMutation.java | 138 +-
...estRMWebServicesCapacitySchedulerMixedMode.java | 216 +-
...itySchedulerMixedModeAbsoluteAndPercentage.java | 121 ++
...lerMixedModeAbsoluteAndPercentageAndWeight.java | 120 ++
...edModeAbsoluteAndPercentageAndWeightVector.java | 123 ++
...edulerMixedModeAbsoluteAndPercentageVector.java | 121 ++
...apacitySchedulerMixedModeAbsoluteAndWeight.java | 120 ++
...ySchedulerMixedModeAbsoluteAndWeightVector.java | 121 ++
...acitySchedulerMixedModePercentageAndWeight.java | 117 ++
...chedulerMixedModePercentageAndWeightVector.java | 121 ++
.../TestRMWebServicesConfigurationMutation.java | 412 ++--
.../webapp/TestRMWebServicesContainers.java | 113 +-
...estRMWebServicesCustomResourceTypesCommons.java | 16 +-
...RMWebServicesDelegationTokenAuthentication.java | 54 +-
.../webapp/TestRMWebServicesDelegationTokens.java | 471 +++--
.../TestRMWebServicesForCSWithPartitions.java | 218 ++-
...TestRMWebServicesHttpStaticUserPermissions.java | 8 +-
.../webapp/TestRMWebServicesNodeLabels.java | 266 +--
.../webapp/TestRMWebServicesNodes.java | 573 +++---
.../webapp/TestRMWebServicesReservation.java | 543 ++---
.../TestRMWebServicesSchedulerActivities.java | 393 ++--
...esSchedulerActivitiesWithMultiNodesEnabled.java | 149 +-
.../webapp/TestRMWebappAuthentication.java | 18 +-
.../resourcemanager/webapp/TestWebServiceUtil.java | 104 +-
.../TestRMWebServicesFairScheduler.java | 117 +-
...ebServicesFairSchedulerCustomResourceTypes.java | 137 +-
.../webapp/helper/AppInfoJsonVerifications.java | 2 +-
.../webapp/helper/BufferedClientResponse.java | 23 +-
.../helper/JsonCustomResourceTypeTestcase.java | 29 +-
.../helper/XmlCustomResourceTypeTestCase.java | 23 +-
.../webapp/reader/AppStateReader.java | 71 +
.../ApplicationSubmissionContextInfoReader.java | 75 +
.../webapp/reader/LabelsToNodesInfoReader.java | 74 +
.../webapp/reader/NodeLabelsInfoReader.java | 74 +
.../webapp/reader/NodeToLabelsInfoReader.java | 74 +
.../webapp/reader/ResourceOptionInfoReader.java | 74 +
.../ApplicationSubmissionContextInfoWriter.java | 89 +
.../webapp/writer/ResourceOptionInfoWriter.java | 87 +
.../webapp/writer/SchedConfUpdateInfoWriter.java | 87 +
.../hadoop/yarn/webapp/TestRMWithCSRFFilter.java | 117 +-
.../src/test/resources/capacity-scheduler.xml | 255 +++
.../src/test/resources/delete-reservation.json | 4 +-
.../src/test/resources/submit-reservation.json | 62 +-
.../src/test/resources/update-reservation.json | 61 +-
...igmutation-absolute-hierarchy-after-update.json | 552 +++---
...gmutation-absolute-hierarchy-before-update.json | 560 +++---
...ion-absolute-hierarchy-legacy-after-update.json | 552 +++---
...on-absolute-hierarchy-legacy-before-update.json | 560 +++---
.../webapp/dynamic-testAbsoluteMode-0.json | 832 ++++----
.../webapp/dynamic-testAbsoluteMode-16.json | 796 ++++----
.../webapp/dynamic-testAbsoluteMode-32.json | 796 ++++----
.../webapp/dynamic-testAbsoluteMode-legacy-0.json | 824 ++++----
.../webapp/dynamic-testPercentageMode-0.json | 808 ++++----
.../webapp/dynamic-testPercentageMode-16.json | 772 ++++----
.../webapp/dynamic-testPercentageMode-32.json | 772 ++++----
.../dynamic-testPercentageMode-legacy-0.json | 772 ++++----
.../resources/webapp/dynamic-testWeightMode-0.json | 808 ++++----
.../webapp/dynamic-testWeightMode-16.json | 772 ++++----
.../webapp/dynamic-testWeightMode-32.json | 772 ++++----
.../webapp/dynamic-testWeightMode-after-aqc.json | 2068 ++++++++++----------
.../webapp/dynamic-testWeightMode-before-aqc.json | 742 +++----
.../webapp/dynamic-testWeightMode-legacy-0.json | 764 ++++----
.../webapp/dynamic-testWeightMode-legacy-16.json | 764 ++++----
.../webapp/dynamic-testWeightMode-legacy-32.json | 764 ++++----
.../dynamic-testWeightMode-legacy-after-aqc.json | 2052 +++++++++----------
.../dynamic-testWeightMode-legacy-before-aqc.json | 734 +++----
...mixed-testSchedulerAbsoluteAndPercentage-0.json | 820 ++++----
...ixed-testSchedulerAbsoluteAndPercentage-16.json | 804 ++++----
...ixed-testSchedulerAbsoluteAndPercentage-32.json | 784 ++++----
...tSchedulerAbsoluteAndPercentageAndWeight-0.json | 816 ++++----
...SchedulerAbsoluteAndPercentageAndWeight-16.json | 800 ++++----
...SchedulerAbsoluteAndPercentageAndWeight-32.json | 780 ++++----
...dulerAbsoluteAndPercentageAndWeightMixed-0.json | 818 ++++----
...ulerAbsoluteAndPercentageAndWeightMixed-16.json | 802 ++++----
...ulerAbsoluteAndPercentageAndWeightMixed-32.json | 782 ++++----
.../mixed-testSchedulerAbsoluteAndWeight-0.json | 820 ++++----
.../mixed-testSchedulerAbsoluteAndWeight-16.json | 804 ++++----
.../mixed-testSchedulerAbsoluteAndWeight-32.json | 784 ++++----
.../mixed-testSchedulerPercentageAndWeight-0.json | 808 ++++----
.../mixed-testSchedulerPercentageAndWeight-16.json | 772 ++++----
.../mixed-testSchedulerPercentageAndWeight-32.json | 772 ++++----
...er-response-AbsoluteModeLegacyAutoCreation.json | 418 ++--
.../scheduler-response-PerUserResources.json | 578 +++---
...-response-PercentageModeLegacyAutoCreation.json | 298 +--
.../test/resources/webapp/scheduler-response.json | 490 ++---
.../hadoop-yarn-server-router/pom.xml | 6 +
.../apache/hadoop/yarn/server/router/Router.java | 6 +-
.../yarn/server/router/webapp/AppsBlock.java | 4 +-
.../webapp/DefaultRequestInterceptorREST.java | 2 +-
.../yarn/server/router/webapp/FederationBlock.java | 17 +-
.../router/webapp/FederationInterceptorREST.java | 10 +-
.../server/router/webapp/MetricsOverviewTable.java | 4 +-
.../yarn/server/router/webapp/NodeLabelsBlock.java | 5 +-
.../yarn/server/router/webapp/NodesBlock.java | 4 +-
.../yarn/server/router/webapp/RouterBlock.java | 19 +-
.../yarn/server/router/webapp/RouterWebApp.java | 34 +-
.../server/router/webapp/RouterWebServiceUtil.java | 159 +-
.../server/router/webapp/RouterWebServices.java | 10 +-
.../subcluster/TestFederationSubCluster.java | 133 +-
.../TestYarnFederationWithCapacityScheduler.java | 64 +-
.../fair/TestYarnFederationWithFairScheduler.java | 85 +-
.../router/webapp/TestRouterWebServiceUtil.java | 28 +-
.../router/webapp/TestRouterWebServicesREST.java | 322 ++-
.../TestTimelineServiceClientIntegration.java | 3 +
.../pom.xml | 28 +
.../pom.xml | 60 +-
.../pom.xml | 4 +
.../pom.xml | 4 +
.../storage/common/HBaseTimelineServerUtils.java | 3 +-
.../hadoop-yarn-server-timelineservice/pom.xml | 21 +-
.../collector/NodeTimelineCollectorManager.java | 24 +-
.../collector/TimelineCollectorWebService.java | 7 +-
.../reader/TimelineReaderManager.java | 1 +
.../reader/TimelineReaderServer.java | 26 +-
.../reader/TimelineReaderWebServices.java | 11 +-
.../storage/FileSystemTimelineWriterImpl.java | 4 +
.../reader/TestTimelineReaderWebServices.java | 405 ++--
.../reader/TestTimelineReaderWebServicesACL.java | 50 +-
.../reader/TimelineAboutReader.java | 82 +
.../reader/TimelineEntityReader.java | 58 +
.../reader/TimelineEntitySetReader.java | 94 +
.../reader/TimelineHealthReader.java | 57 +
.../hadoop-yarn-server-web-proxy/pom.xml | 8 +-
.../src/site/markdown/CapacityScheduler.md | 37 +
pom.xml | 3 +
596 files changed, 38326 insertions(+), 31105 deletions(-)
create mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Sizes.java
create mode 100644
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesLogsExtend.java
create mode 100644
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/reader/ContainerLogsInfoMessageBodyReader.java
create mode 100644
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/reader/RemoteLogPathsMessageBodyReader.java
delete mode 100644
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/DirectoryPolicy.java
delete mode 100644
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/DirectoryPolicyImpl.java
delete mode 100644
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
delete mode 100644
hadoop-tools/hadoop-aws/src/main/resources/META-INF/services/org.apache.hadoop.fs.FileSystem
delete mode 100644
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/s3n.md
delete mode 100644
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/TestDirectoryMarkerPolicy.java
delete mode 100644
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/TestAuthoritativePath.java
delete mode 100644
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/TestS3xLoginHelper.java
copy
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/TokenServiceParam.java
=>
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/BlobAppendRequestParameters.java
(54%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/BlobListResultEntrySchema.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/BlobListResultSchema.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/BlobListXmlParser.java
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/{ListResultEntrySchema.java
=> DfsListResultEntrySchema.java} (85%)
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/{ListResultSchema.java
=> DfsListResultSchema.java} (71%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/StorageErrorResponseSchema.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java
create mode 100644 hadoop-tools/hadoop-azure/src/site/markdown/blobEndpoint.md
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsClientHandler.java
delete mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/TestMockHelpers.java
delete mode 100644
hadoop-tools/hadoop-resourceestimator/src/test/java/org/apache/hadoop/resourceestimator/service/GuiceServletConfig.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/reader/TimelineDomainReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/reader/TimelineEntitiesReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/reader/TimelinePutResponseReader.java
copy
hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/collector/TestAppLevelTimelineCollector.java
=>
hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/reader/package-info.java}
(77%)
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineDomainWriter.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineDomainsWriter.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineEntitiesWriter.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineEntityWriter.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineEventsWriter.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelinePutResponseWriter.java
copy
hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/collector/TestAppLevelTimelineCollector.java
=>
hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/package-info.java}
(77%)
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/reader/TimelineDomainReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/reader/TimelineEntitiesReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/reader/TimelineEntityReader.java
copy
hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/collector/TestAppLevelTimelineCollector.java
=>
hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/reader/package-info.java}
(77%)
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineDomainWriter.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineEntitiesWriter.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineEntitySetWriter.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineEntityWriter.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineHealthWriter.java
rename
hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/collector/TestAppLevelTimelineCollector.java
=>
hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/package-info.java}
(77%)
copy
hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/AppsInfo.java
=>
hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/ConflictException.java}
(53%)
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/ContainerLogsInfoListReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineAboutReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineDomainReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineDomainsReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineEntitiesReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineEntityReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineEventsReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelinePutResponseReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedDefaultLabel.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedDynamicConfigAbsoluteMode.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedDynamicConfigWeightMode.java
copy
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/{TestRMWebServicesCapacitySchedDynamicConfig.java
=> TestRMWebServicesCapacitySchedDynamicConfigWeightModeDQC.java} (54%)
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedLegacyQueueCreationAbsoluteMode.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentage.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentageAndWeight.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentageAndWeightVector.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentageVector.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndWeight.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndWeightVector.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModePercentageAndWeight.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModePercentageAndWeightVector.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/AppStateReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/ApplicationSubmissionContextInfoReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/LabelsToNodesInfoReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/NodeLabelsInfoReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/NodeToLabelsInfoReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/ResourceOptionInfoReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/writer/ApplicationSubmissionContextInfoWriter.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/writer/ResourceOptionInfoWriter.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/writer/SchedConfUpdateInfoWriter.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/capacity-scheduler.xml
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineAboutReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineEntityReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineEntitySetReader.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineHealthReader.java
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-commits-h...@hadoop.apache.org
