This is an automated email from the ASF dual-hosted git repository. ahmar pushed a change to branch feature-HADOOP-19363-analytics-accelerator-s3 in repository https://gitbox.apache.org/repos/asf/hadoop.git
omit 2a8e9210c18 cleaning up prod code omit 4344d290f0c fix typos omit 771ab2a417f updates to skipped tests omit dc2dc636a4f enable MPU omit d45beae8e59 Integrate analytics-accelerator with factory (#7332) omit 63daf564351 HADOOP-19348. Add initial support for Analytics Accelerator Library for Amazon S3 (#7192) omit eadf0ddd71d HADOOP-19354. S3AInputStream to be created by factory under S3AStore omit 9c8e753209d HADOOP-19354. S3A: Create S3 InputStreams via a factory add 06d36f5a6a1 HADOOP-15984. Update jersey from 1.19 to 2.x. (#7019) add 9bf5e38c483 YARN-11745: Fix TimSort contract violation in PriorityQueueComparator Class (#7278) add 0d72896db16 YARN-11753. Ensure NM is marked unhealthy if the ProcessBuilder reports an issue with the container-executor (#7290) add 0432761ac84 HDFS-17657. The balancer service supports httpserver. (#7242) Contribtued by Zhaobo Huang. add e41f2ba1552 HADOOP-19392: Upgrade ftpserver to resolve maven/pom issue (#7311) Contributed by YanivKunda. add 8afbfbc2f57 HDFS-17394. [FGL] Remove unused WriteHoldCount of FSNamesystemLock (#6571) add 9b45c66e34f HDFS-17387. [FGL] Abstract the configuration locking mode (#6572) add 16c25b277da HDFS-17398. [FGL] Implement the FGL lock for FSNLockManager (#6599) add f0368bb2372 HDFS-17405. [FGL] Using different metric name to trace performance for FGL and Global lock (#6600) add a1ab911896f HDFS-17390. [FGL] FSDirectory supports this fine-grained locking (#6573) add 7158b9ee194 HDFS-17395. [FGL] Use FSLock to protect ErasureCodingPolicy related operations (#6579) add d39f034f982 HDFS-17413. [FGL] CacheReplicationMonitor supports fine-grained lock (#6641) add 7764b39e4ab HDFS-17414. [FGL] RPCs in DatanodeProtocol support fine-grained lock (#6649) add ac0605db419 HDFS-17416. [FGL] Monitor threads in BlockManager.class support fine-grained lock (#6647) add 3b5e28a87ec HDFS-17423. [FGL] BlockManagerSafeMode supports fine-grained lock (#6645) add 778e819b41a HDFS-17410. [FGL] Client RPCs that changes file attributes supports fine-grained lock (#6634) add 7d37995f46e HDFS-17415. [FGL] RPCs in NamenodeProtocol support fine-grained lock (#6654) add 40d54ebb66a HDFS-17417. [FGL] HeartbeatManager and DatanodeAdminMonitor support fine-grained locking (#6656) add 03603efd74b HDFS-17412. [FGL] Client RPCs involving maintenance supports fine-grained lock (#6667) add ed302ae3aad HDFS-17420. [FGL] EditLogTailer and FSEditLogLoader support FGL (#6679) add 7577f3ad619 HDFS-17388. [FGL] Client RPCs involving write process supports fine-grained lock (#6589) add 4c86601cba5 HDFS-17389. [FGL] Client RPCs involving read process supports fine-grained lock (#6590) add 6d888d599f4 HDFS-17411. [FGL] Client RPCs involving snapshot support fine-grained lock (#6714) add 02b149da67e HDFS-17445. [FGL] some operations support fine-grained locking (#6715) add e4885b02efc HDFS-17424. [FGL] DelegationTokenSecretManager supports fine-grained lock (#6696) add 0d626f516f4 HDFS-17472. [FGL] gcDeletedSnapshot and getDelegationToken support FGL (#6743) add e5b1c09b8a0 HDFS-17459. [FGL] Add documentation (#6737) add be0267313bd HDFS-17457. [FGL] UTs support fine-grained locking (#6741) add e8807726b8d HDFS-17506. [FGL] Performance for phase 1 add b289f9abd33 HDFS-17691. [FGL] Move FSNamesystemLockMode to org.apache.hadoop.hdfs.util package (#7232) add 16552b6fbdc HDFS-17692. [FGL] fix bug in getContentSummary method (#7233) add 8ab53982a56 HDFS-17697. [FGL] hasWriteLock and hasReadLock in FineGrainedFSNamesystemLock shouldn't throw assert error (#7250) add 2343bd71a93 HDFS-17701. [FGL] Fix some javadocs (#7256) add 7ce4ea57807 HDFS-17711: Change fsimage loading progress percentage discontinuous to continuous add 053afb7063f HDFS-17704. Fix TestDecommission and TestDecommissionWithBackoffMonitor often run timeout. (#7266). Contributed by hfutatzhanghb. add e18d0a4d26d HADOOP-19348. Add initial support for Analytics Accelerator Library for Amazon S3 (#7192) This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (2a8e9210c18) \ N -- N -- N refs/heads/feature-HADOOP-19363-analytics-accelerator-s3 (e18d0a4d26d) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: LICENSE-binary | 28 +- .../hadoop-client-check-invariants/pom.xml | 2 + .../resources/ensure-jars-have-correct-contents.sh | 6 +- .../hadoop-client-check-test-invariants/pom.xml | 1 + .../hadoop-client-integration-tests/pom.xml | 10 +- .../hadoop-client-minicluster/pom.xml | 106 +- .../hadoop-client-runtime/pom.xml | 26 +- hadoop-client-modules/hadoop-client/pom.xml | 76 - .../hadoop-cloud-storage/pom.xml | 12 - .../hadoop-huaweicloud/pom.xml | 6 + .../apache/hadoop/fs/obs/OBSBlockOutputStream.java | 2 +- .../org/apache/hadoop/fs/obs/OBSInputStream.java | 2 +- hadoop-common-project/hadoop-auth-examples/pom.xml | 4 +- hadoop-common-project/hadoop-auth/pom.xml | 4 +- hadoop-common-project/hadoop-common/pom.xml | 97 +- .../java/org/apache/hadoop/http/HttpServer2.java | 34 +- .../apache/hadoop/util/ConfigurationHelper.java | 36 - .../apache/hadoop/fs/ftp/TestFTPFileSystem.java | 8 +- .../org/apache/hadoop/http/TestHttpServer.java | 18 +- .../org/apache/hadoop/http/TestHttpServerLogs.java | 9 +- hadoop-common-project/hadoop-kms/pom.xml | 15 +- .../hadoop-kms/src/main/conf/kms-log4j.properties | 1 - .../apache/hadoop/crypto/key/kms/server/KMS.java | 14 +- .../key/kms/server/KMSExceptionsProvider.java | 2 +- .../src/main/resources/webapps/kms/WEB-INF/web.xml | 16 +- .../hadoop/crypto/key/kms/server/TestKMS.java | 4 +- .../hadoop-kms/src/test/resources/log4j.properties | 2 - hadoop-common-project/hadoop-nfs/pom.xml | 4 +- hadoop-hdfs-project/hadoop-hdfs-client/pom.xml | 5 + hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml | 11 +- .../fs/http/server/HttpFSExceptionProvider.java | 2 +- .../fs/http/server/HttpFSParametersProvider.java | 2 - .../apache/hadoop/fs/http/server/HttpFSServer.java | 45 +- .../apache/hadoop/lib/wsrs/JSONMapProvider.java | 14 +- .../apache/hadoop/lib/wsrs/ParametersProvider.java | 67 +- .../main/resources/webapps/webhdfs/WEB-INF/web.xml | 18 +- .../src/main/webapp/WEB-INF/web.xml | 18 +- .../hadoop/fs/http/server/TestHttpFSServer.java | 3 +- .../src/test/resources/default-log4j.properties | 1 - hadoop-hdfs-project/hadoop-hdfs-nfs/pom.xml | 10 +- .../hadoop/hdfs/nfs/nfs3/Nfs3HttpServer.java | 2 +- hadoop-hdfs-project/hadoop-hdfs-rbf/pom.xml | 5 + .../server/federation/router/RouterHttpServer.java | 2 +- .../federation/router/RouterWebHdfsMethods.java | 3 - .../server/federation/router/TestRouterRpc.java | 5 +- hadoop-hdfs-project/hadoop-hdfs/pom.xml | 28 +- .../java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 21 + .../main/java/org/apache/hadoop/hdfs/DFSUtil.java | 4 +- .../qjournal/server/JournalNodeHttpServer.java | 2 +- .../delegation/DelegationTokenSecretManager.java | 9 +- .../hadoop/hdfs/server/balancer/Balancer.java | 55 +- .../hdfs/server/balancer/BalancerHttpServer.java | 114 ++ .../hdfs/server/balancer/BalancerMXBean.java | 29 +- .../hadoop/hdfs/server/balancer}/package-info.java | 18 +- .../hdfs/server/blockmanagement/BlockManager.java | 122 +- .../blockmanagement/BlockManagerSafeMode.java | 33 +- .../blockmanagement/CacheReplicationMonitor.java | 13 +- .../DatanodeAdminBackoffMonitor.java | 38 +- .../DatanodeAdminDefaultMonitor.java | 11 +- .../server/blockmanagement/DatanodeManager.java | 12 +- .../server/blockmanagement/HeartbeatManager.java | 9 +- .../server/blockmanagement/ProvidedStorageMap.java | 5 +- .../datanode/web/webhdfs/ExceptionHandler.java | 4 +- .../hadoop/hdfs/server/namenode/BackupImage.java | 5 +- .../hadoop/hdfs/server/namenode/CacheManager.java | 33 +- .../hadoop/hdfs/server/namenode/Checkpointer.java | 10 +- .../namenode/ContentSummaryComputationContext.java | 12 +- .../server/namenode/EncryptionZoneManager.java | 14 +- .../hadoop/hdfs/server/namenode/FSDirAppendOp.java | 5 +- .../hadoop/hdfs/server/namenode/FSDirDeleteOp.java | 4 +- .../server/namenode/FSDirEncryptionZoneOp.java | 15 +- .../hdfs/server/namenode/FSDirErasureCodingOp.java | 17 +- .../namenode/FSDirSatisfyStoragePolicyOp.java | 3 +- .../server/namenode/FSDirStatAndListingOp.java | 25 +- .../hdfs/server/namenode/FSDirTruncateOp.java | 9 +- .../hdfs/server/namenode/FSDirWriteFileOp.java | 46 +- .../hadoop/hdfs/server/namenode/FSDirectory.java | 29 +- .../hdfs/server/namenode/FSEditLogLoader.java | 9 +- .../hdfs/server/namenode/FSImageFormatPBINode.java | 5 +- .../hadoop/hdfs/server/namenode/FSNamesystem.java | 672 ++++--- .../hdfs/server/namenode/FSNamesystemLock.java | 44 +- .../hdfs/server/namenode/FSPermissionChecker.java | 15 +- .../hdfs/server/namenode/FSTreeTraverser.java | 7 +- .../hdfs/server/namenode/FsImageValidation.java | 5 +- .../hadoop/hdfs/server/namenode/INodeFile.java | 6 + .../hadoop/hdfs/server/namenode/LeaseManager.java | 15 +- .../hadoop/hdfs/server/namenode/NameNode.java | 25 +- .../hdfs/server/namenode/NameNodeHttpServer.java | 29 +- .../hadoop/hdfs/server/namenode/NamenodeFsck.java | 10 +- .../hdfs/server/namenode/ReencryptionHandler.java | 11 +- .../hdfs/server/namenode/ReencryptionUpdater.java | 7 +- .../hdfs/server/namenode/SecondaryNameNode.java | 7 +- .../hdfs/server/namenode/fgl/FSNLockManager.java | 187 ++ .../namenode/fgl/FineGrainedFSNamesystemLock.java | 285 +++ .../namenode/fgl/GlobalFSNamesystemLock.java | 150 ++ .../hdfs/server/namenode/fgl}/package-info.java | 15 +- .../hdfs/server/namenode/ha/EditLogTailer.java | 5 +- .../namenode/snapshot/SnapshotDeletionGc.java | 5 +- .../web/resources/NamenodeWebHdfsMethods.java | 285 ++- .../java/org/apache/hadoop/hdfs/util/RwLock.java | 106 +- .../org/apache/hadoop/hdfs/util/RwLockMode.java | 18 +- .../org/apache/hadoop/hdfs/web/ParamFilter.java | 103 +- .../hdfs/web/resources/ExceptionHandler.java | 11 +- .../hdfs/web/resources/TokenServiceParam.java | 2 - .../hadoop/hdfs/web/resources/UserProvider.java | 35 +- .../src/main/resources/hdfs-default.xml | 63 + .../src/main/webapps/balancer/balancer.html | 114 ++ .../src/main/webapps/balancer/balancer.js | 95 + .../src/main/webapps/balancer/index.html | 24 + .../main/webapps/{datanode => balancer}/robots.txt | 0 .../hadoop-hdfs/src/site/markdown/NamenodeFGL.md | 210 ++ .../apache/hadoop/hdfs/AdminStatesBaseTest.java | 4 + .../hadoop/hdfs/TestBlocksScheduledCounter.java | 11 +- .../org/apache/hadoop/hdfs/TestDecommission.java | 51 +- .../org/apache/hadoop/hdfs/TestFileCorruption.java | 13 +- .../apache/hadoop/hdfs/TestMaintenanceState.java | 50 +- .../server/balancer/TestBalancerHttpServer.java | 100 + .../hdfs/server/balancer/TestBalancerService.java | 51 + .../blockmanagement/BlockManagerTestUtil.java | 22 +- .../server/blockmanagement/TestBlockManager.java | 15 +- .../blockmanagement/TestBlockManagerSafeMode.java | 3 + .../TestBlocksWithNotEnoughRacks.java | 5 +- .../blockmanagement/TestComputeInvalidateWork.java | 25 +- .../blockmanagement/TestDatanodeManager.java | 15 +- .../blockmanagement/TestHeartbeatHandling.java | 9 +- .../TestNameNodePrunesMissingStorages.java | 10 +- .../hdfs/server/blockmanagement/TestNodeCount.java | 5 +- .../blockmanagement/TestOverReplicatedBlocks.java | 9 +- .../blockmanagement/TestPendingReconstruction.java | 17 +- .../blockmanagement/TestProvidedStorageMap.java | 3 +- ...tReconstructStripedBlocksWithRackAwareness.java | 10 +- .../blockmanagement/TestReplicationPolicy.java | 8 + .../TestReplicationPolicyConsiderLoad.java | 10 +- .../TestReplicationPolicyExcludeSlowNodes.java | 11 +- ...licationPolicyRatioConsiderLoadWithStorage.java | 6 +- .../hdfs/server/namenode/NameNodeAdapter.java | 17 +- .../hdfs/server/namenode/TestAddBlockRetry.java | 14 +- .../TestAddOverReplicatedStripedBlocks.java | 6 +- .../TestBlockPlacementPolicyRackFaultTolerant.java | 6 +- .../hdfs/server/namenode/TestCacheDirectives.java | 17 +- .../hdfs/server/namenode/TestDeleteRace.java | 5 +- .../server/namenode/TestDiskspaceQuotaUpdate.java | 5 +- .../hdfs/server/namenode/TestEditLogRace.java | 6 +- .../server/namenode/TestFSImageWithSnapshot.java | 9 +- .../hdfs/server/namenode/TestFSNamesystem.java | 5 +- .../hdfs/server/namenode/TestFSNamesystemLock.java | 20 +- .../server/namenode/TestFSNamesystemMBean.java | 7 +- .../hdfs/server/namenode/TestFileTruncate.java | 9 +- .../hadoop/hdfs/server/namenode/TestFsck.java | 9 +- .../server/namenode/TestGetBlockLocations.java | 14 +- .../server/namenode/TestLargeDirectoryDelete.java | 5 +- .../hdfs/server/namenode/TestLeaseManager.java | 5 + .../hdfs/server/namenode/TestListOpenFiles.java | 5 +- .../namenode/TestNameNodeMetadataConsistency.java | 11 +- .../namenode/TestReconstructStripedBlocks.java | 5 +- .../server/namenode/TestSecurityTokenEditLog.java | 5 +- .../namenode/fgl/FSNLockBenchmarkThroughput.java | 322 +++ .../fgl/TestFSNLockBenchmarkThroughput.java | 104 + .../fgl/TestFineGrainedFSNamesystemLock.java | 276 +++ .../hdfs/server/namenode/ha/TestDNFencing.java | 5 +- .../namenode/metrics/TestNameNodeMetrics.java | 13 +- ...TestINodeFileUnderConstructionWithSnapshot.java | 5 +- .../org/apache/hadoop/hdfs/tools/TestDFSAdmin.java | 5 +- .../hdfs/web/TestWebHdfsFileSystemContract.java | 10 +- .../src/test/resources/log4j.properties | 4 - .../hadoop-mapreduce-client-app/pom.xml | 44 + .../jobhistory/JobHistoryEventHandler.java | 8 +- .../mapreduce/v2/app/client/MRClientService.java | 28 +- .../hadoop/mapreduce/v2/app/webapp/AMWebApp.java | 19 +- .../mapreduce/v2/app/webapp/AMWebServices.java | 11 +- .../apache/hadoop/mapreduce/v2/app/webapp/App.java | 7 +- .../v2/app/webapp/JAXBContextResolver.java | 15 +- .../mapreduce/v2/app/webapp/TestAMWebApp.java | 45 +- .../mapreduce/v2/app/webapp/TestAMWebServices.java | 245 ++- .../v2/app/webapp/TestAMWebServicesAttempt.java | 145 +- .../v2/app/webapp/TestAMWebServicesAttempts.java | 246 ++- .../v2/app/webapp/TestAMWebServicesJobConf.java | 124 +- .../v2/app/webapp/TestAMWebServicesJobs.java | 434 ++-- .../v2/app/webapp/TestAMWebServicesTasks.java | 427 ++-- .../hadoop-mapreduce-client-common/pom.xml | 5 + .../hadoop-mapreduce-client-core/pom.xml | 20 + .../hadoop-mapreduce-client-hs/pom.xml | 30 + .../mapreduce/v2/hs/HistoryClientService.java | 38 + .../hadoop/mapreduce/v2/hs/webapp/HsWebApp.java | 4 - .../mapreduce/v2/hs/webapp/HsWebServices.java | 34 +- .../v2/hs/webapp/JAXBContextResolver.java | 10 +- .../mapreduce/v2/hs/webapp/TestHsWebServices.java | 253 ++- .../v2/hs/webapp/TestHsWebServicesAttempts.java | 257 ++- .../v2/hs/webapp/TestHsWebServicesJobConf.java | 120 +- .../v2/hs/webapp/TestHsWebServicesJobs.java | 468 ++--- .../v2/hs/webapp/TestHsWebServicesJobsQuery.java | 471 +++-- .../v2/hs/webapp/TestHsWebServicesLogs.java | 398 ++-- .../v2/hs/webapp/TestHsWebServicesLogsExtend.java | 399 ++++ .../v2/hs/webapp/TestHsWebServicesTasks.java | 446 ++--- .../reader/ContainerLogsInfoMessageBodyReader.java | 103 + .../reader/RemoteLogPathsMessageBodyReader.java | 88 + .../hadoop-mapreduce-client-jobclient/pom.xml | 21 + .../hadoop-mapreduce-client-nativetask/pom.xml | 20 + .../hadoop-mapreduce-client-shuffle/pom.xml | 5 + .../hadoop-mapreduce-client/pom.xml | 9 +- .../hadoop-mapreduce-examples/pom.xml | 9 +- hadoop-project/pom.xml | 199 +- hadoop-tools/hadoop-archive-logs/pom.xml | 20 + hadoop-tools/hadoop-archives/pom.xml | 20 + .../hadoop-aws/dev-support/findbugs-exclude.xml | 2 +- hadoop-tools/hadoop-aws/pom.xml | 26 + .../java/org/apache/hadoop/fs/s3a/Constants.java | 73 +- .../org/apache/hadoop/fs/s3a/S3AFileSystem.java | 379 ++-- .../org/apache/hadoop/fs/s3a/S3AInputStream.java | 388 +++- .../org/apache/hadoop/fs/s3a/S3AReadOpContext.java | 44 +- ...AnalyticsStream.java => S3ASeekableStream.java} | 52 +- .../java/org/apache/hadoop/fs/s3a/S3AStore.java | 38 +- .../apache/hadoop/fs/s3a/VectoredIOContext.java | 17 - .../apache/hadoop/fs/s3a/impl/ClientManager.java | 9 +- .../hadoop/fs/s3a/impl/ClientManagerImpl.java | 56 +- .../fs/s3a/impl/InputStreamCallbacksImpl.java | 123 -- .../apache/hadoop/fs/s3a/impl/S3AStoreImpl.java | 208 +- .../streams/AbstractObjectInputStreamFactory.java | 94 - .../s3a/impl/streams/AnalyticsStreamFactory.java | 100 - .../streams/ClassicObjectInputStreamFactory.java | 73 - .../s3a/impl/streams/FactoryBindingParameters.java | 46 - .../fs/s3a/impl/streams/InputStreamType.java | 123 -- .../fs/s3a/impl/streams/ObjectInputStream.java | 378 ---- .../impl/streams/ObjectInputStreamCallbacks.java | 64 - .../s3a/impl/streams/ObjectInputStreamFactory.java | 89 - .../fs/s3a/impl/streams/ObjectReadParameters.java | 186 -- .../impl/streams/StreamFactoryRequirements.java | 92 - .../fs/s3a/impl/streams/StreamIntegration.java | 213 -- .../hadoop/fs/s3a/prefetch/PrefetchOptions.java | 67 - .../prefetch/PrefetchingInputStreamFactory.java | 108 - .../fs/s3a/prefetch/S3ACachingInputStream.java | 16 +- .../fs/s3a/prefetch/S3AInMemoryInputStream.java | 7 +- .../fs/s3a/prefetch/S3APrefetchingInputStream.java | 67 +- .../fs/s3a/prefetch/S3ARemoteInputStream.java | 14 +- .../hadoop/fs/s3a/prefetch/S3ARemoteObject.java | 5 +- .../site/markdown/tools/hadoop-aws/prefetching.md | 14 +- .../src/site/markdown/tools/hadoop-aws/reading.md | 112 -- .../fs/contract/s3a/ITestS3AContractCreate.java | 7 +- .../fs/contract/s3a/ITestS3AContractRename.java | 4 + .../fs/contract/s3a/ITestS3AContractSeek.java | 2 +- .../hadoop/fs/s3a/ITestS3AConfiguration.java | 18 +- .../apache/hadoop/fs/s3a/ITestS3ADelayedFNF.java | 2 + .../hadoop/fs/s3a/ITestS3AEncryptionSSEC.java | 2 +- .../hadoop/fs/s3a/ITestS3AFileSystemContract.java | 7 +- .../hadoop/fs/s3a/ITestS3AIOStatisticsContext.java | 5 +- .../hadoop/fs/s3a/ITestS3AInputStreamLeakage.java | 47 +- .../org/apache/hadoop/fs/s3a/ITestS3AMetrics.java | 5 +- .../fs/s3a/ITestS3APrefetchingCacheFiles.java | 12 +- .../fs/s3a/ITestS3APrefetchingInputStream.java | 10 +- .../fs/s3a/ITestS3APrefetchingLruEviction.java | 16 +- .../hadoop/fs/s3a/ITestS3ARequesterPays.java | 5 +- .../hadoop/fs/s3a/ITestS3AS3SeekableStream.java | 55 +- .../org/apache/hadoop/fs/s3a/S3ATestUtils.java | 38 +- .../hadoop/fs/s3a/TestS3AInputStreamRetry.java | 27 +- .../fs/s3a/commit/ITestCommitOperationCost.java | 11 +- .../fs/s3a/commit/ITestS3ACommitterFactory.java | 3 +- .../s3a/commit/magic/ITestMagicCommitProtocol.java | 2 + .../integration/ITestDirectoryCommitProtocol.java | 3 + .../ITestPartitionedCommitProtocol.java | 3 + .../integration/ITestStagingCommitProtocol.java | 4 + .../ITestStagingCommitProtocolFailure.java | 12 +- .../fileContext/ITestS3AFileContextStatistics.java | 4 +- .../fs/s3a/impl/ITestConnectionTimeouts.java | 16 +- .../fs/s3a/impl/streams/TestStreamFactories.java | 240 --- .../fs/s3a/performance/ITestS3AOpenCost.java | 12 +- .../fs/s3a/performance/ITestUnbufferDraining.java | 9 +- .../fs/s3a/prefetch/MockS3ARemoteObject.java | 8 +- .../hadoop/fs/s3a/prefetch/S3APrefetchFakes.java | 43 +- .../fs/s3a/prefetch/TestS3ARemoteInputStream.java | 19 +- .../fs/s3a/prefetch/TestS3ARemoteObject.java | 6 +- .../s3a/scale/ITestS3AHugeFilesSSECDiskBlocks.java | 3 - .../s3a/scale/ITestS3AInputStreamPerformance.java | 47 +- .../ITestS3AContractStreamIOStatistics.java | 9 +- .../statistics/ITestS3AFileSystemStatistic.java | 8 +- .../hadoop-aws/src/test/resources/log4j.properties | 4 - hadoop-tools/hadoop-azure-datalake/pom.xml | 21 + hadoop-tools/hadoop-azure/pom.xml | 25 +- hadoop-tools/hadoop-datajoin/pom.xml | 20 + hadoop-tools/hadoop-distcp/pom.xml | 20 + .../hadoop-dynamometer-blockgen/pom.xml | 20 + .../hadoop-dynamometer-infra/pom.xml | 20 + .../hadoop-dynamometer-workload/pom.xml | 20 + hadoop-tools/hadoop-extras/pom.xml | 20 + hadoop-tools/hadoop-federation-balance/pom.xml | 20 + hadoop-tools/hadoop-fs2img/pom.xml | 20 + .../namenode/ITestProvidedImplementation.java | 13 +- hadoop-tools/hadoop-gridmix/pom.xml | 20 + hadoop-tools/hadoop-resourceestimator/pom.xml | 53 +- .../service/ResourceEstimatorService.java | 33 +- .../service/GuiceServletConfig.java | 42 - .../service/TestResourceEstimatorService.java | 69 +- hadoop-tools/hadoop-rumen/pom.xml | 20 + hadoop-tools/hadoop-sls/pom.xml | 20 + hadoop-tools/hadoop-streaming/pom.xml | 20 + .../hadoop-yarn/hadoop-yarn-api/pom.xml | 4 +- .../api/records/timeline/TimelineEntities.java | 2 + .../yarn/api/records/timeline/TimelineEntity.java | 11 +- .../yarn/api/records/timeline/TimelineEvent.java | 2 + .../yarn/api/records/timeline/TimelineEvents.java | 5 +- .../api/records/timeline/TimelinePutResponse.java | 7 + .../timeline/reader/TimelineDomainReader.java | 57 + .../timeline/reader/TimelineEntitiesReader.java | 57 + .../timeline/reader/TimelinePutResponseReader.java | 58 + .../api/records/timeline/reader}/package-info.java | 18 +- .../timeline/writer/TimelineDomainWriter.java | 65 + .../timeline/writer/TimelineDomainsWriter.java | 65 + .../timeline/writer/TimelineEntitiesWriter.java | 65 + .../timeline/writer/TimelineEntityWriter.java | 65 + .../timeline/writer/TimelineEventsWriter.java | 66 + .../timeline/writer/TimelinePutResponseWriter.java | 66 + .../api/records/timeline/writer}/package-info.java | 18 +- .../records/timelineservice/TimelineEntities.java | 2 + .../records/timelineservice/TimelineEntity.java | 6 + .../api/records/timelineservice/TimelineEvent.java | 3 + .../records/timelineservice/TimelineMetric.java | 12 +- .../reader/TimelineDomainReader.java | 57 + .../reader/TimelineEntitiesReader.java | 57 + .../reader/TimelineEntityReader.java | 57 + .../timelineservice/reader}/package-info.java | 18 +- .../writer/TimelineDomainWriter.java | 65 + .../writer/TimelineEntitiesWriter.java | 65 + .../writer/TimelineEntitySetWriter.java | 67 + .../writer/TimelineEntityWriter.java | 67 + .../writer/TimelineHealthWriter.java | 66 + .../timelineservice/writer}/package-info.java | 18 +- .../pom.xml | 23 - .../appcatalog/application/YarnServiceClient.java | 80 +- .../src/main/webapp/WEB-INF/web.xml | 8 +- .../pom.xml | 17 + .../distributedshell/ApplicationMaster.java | 10 +- .../distributedshell/TestDSTimelineV10.java | 37 +- .../hadoop-yarn-services-api/pom.xml | 10 +- .../yarn/service/client/ApiServiceClient.java | 110 +- .../src/main/webapp/WEB-INF/web.xml | 8 +- .../apache/hadoop/yarn/service/TestApiServer.java | 53 +- .../hadoop-yarn-services-core/pom.xml | 20 +- .../hadoop/yarn/service/ServiceScheduler.java | 14 +- .../apache/hadoop/yarn/service/utils/HttpUtil.java | 13 +- .../hadoop/yarn/service/ServiceTestUtils.java | 21 +- .../hadoop-yarn/hadoop-yarn-client/pom.xml | 17 +- .../org/apache/hadoop/yarn/client/cli/LogsCLI.java | 268 ++- .../hadoop/yarn/client/cli/SchedConfCLI.java | 138 +- .../apache/hadoop/yarn/client/cli/TestLogsCLI.java | 23 +- .../hadoop/yarn/client/cli/TestSchedConfCLI.java | 114 +- .../hadoop-yarn/hadoop-yarn-common/pom.xml | 74 +- .../yarn/client/api/impl/DirectTimelineWriter.java | 9 +- .../client/api/impl/FileSystemTimelineWriter.java | 8 +- .../yarn/client/api/impl/TimelineClientImpl.java | 12 +- .../yarn/client/api/impl/TimelineConnector.java | 223 +-- .../client/api/impl/TimelineReaderClientImpl.java | 90 +- .../yarn/client/api/impl/TimelineV2ClientImpl.java | 85 +- .../yarn/client/api/impl/TimelineWriter.java | 64 +- .../hadoop/yarn/logaggregation/LogToolUtils.java | 16 +- .../hadoop/yarn/webapp/ConflictException.java} | 37 +- .../yarn/webapp/GenericExceptionHandler.java | 29 +- .../java/org/apache/hadoop/yarn/webapp/WebApp.java | 32 +- .../org/apache/hadoop/yarn/webapp/WebApps.java | 39 +- .../yarn/webapp/YarnJacksonJaxbJsonProvider.java | 2 +- .../hadoop/yarn/webapp/util/WebAppUtils.java | 16 + .../hadoop/yarn/webapp/util/WebServiceClient.java | 52 +- .../yarn/webapp/util/YarnWebServiceUtils.java | 75 +- .../yarn/client/api/impl/TestTimelineClient.java | 80 +- .../api/impl/TestTimelineClientForATS1_5.java | 18 +- .../api/impl/TestTimelineReaderClientImpl.java | 34 +- .../apache/hadoop/yarn/webapp/JerseyTestBase.java | 37 +- .../yarn/webapp/MyTestJAXBContextResolver.java | 11 +- .../hadoop/yarn/webapp/MyTestWebService.java | 2 +- .../org/apache/hadoop/yarn/webapp/TestWebApp.java | 110 +- .../yarn/webapp/util/TestWebServiceClient.java | 7 +- .../hadoop-yarn/hadoop-yarn-csi/pom.xml | 5 - .../pom.xml | 66 +- .../ApplicationHistoryServer.java | 47 +- .../webapp/AHSWebApp.java | 12 +- .../webapp/AHSWebServices.java | 20 +- .../webapp/ContextFactory.java | 2 +- .../webapp/JAXBContextResolver.java | 11 +- .../timeline/webapp/TimelineWebServices.java | 35 +- .../webapp/TestAHSWebServices.java | 461 +++-- .../reader/ContainerLogsInfoListReader.java | 117 ++ .../timeline/reader/TimelineAboutReader.java | 82 + .../timeline/reader/TimelineDomainReader.java | 57 + .../timeline/reader/TimelineDomainsReader.java | 57 + .../timeline/reader/TimelineEntitiesReader.java | 57 + .../timeline/reader/TimelineEntityReader.java | 69 + .../timeline/reader/TimelineEventsReader.java | 80 + .../timeline/reader/TimelinePutResponseReader.java | 57 + .../TestTimelineAuthenticationFilterForV1.java | 4 +- .../timeline/webapp/TestTimelineWebServices.java | 929 +++++---- .../webapp/TestTimelineWebServicesWithSSL.java | 17 +- .../hadoop-yarn-server-common/pom.xml | 24 + .../policies/dao/WeightedPolicyInfo.java | 57 +- .../federation/store/records/SubClusterIdInfo.java | 2 +- .../hadoop/yarn/server/webapp/LogServlet.java | 4 +- .../hadoop/yarn/server/webapp/LogWebService.java | 70 +- .../yarn/server/webapp/LogWebServiceUtils.java | 9 +- .../yarn/server/webapp/dao/ContainerLogsInfo.java | 29 +- .../pom.xml | 27 +- .../server/globalpolicygenerator/GPGUtils.java | 29 +- .../GlobalPolicyGenerator.java | 2 +- .../globalpolicygenerator/webapp/GPGWebApp.java | 23 +- .../webapp/GPGWebServices.java | 10 +- .../policygenerator/TestPolicyGenerator.java | 23 +- .../webapp/TestGPGWebServices.java | 76 +- .../src/test/resources/schedulerInfo1.json | 250 +-- .../src/test/resources/schedulerInfo2.json | 374 ++-- .../hadoop-yarn-server-nodemanager/pom.xml | 96 +- .../server/nodemanager/LinuxContainerExecutor.java | 6 +- .../nodemanager/api/deviceplugin/Device.java | 13 +- .../deviceframework/AssignedDevice.java | 4 +- .../resourceplugin/gpu/AssignedGpuDevice.java | 3 + .../resourceplugin/gpu/GpuDevice.java | 3 + .../nodemanager/webapp/JAXBContextResolver.java | 15 +- .../server/nodemanager/webapp/NMWebAppFilter.java | 37 +- .../server/nodemanager/webapp/NMWebServices.java | 50 +- .../yarn/server/nodemanager/webapp/WebServer.java | 79 +- .../server/nodemanager/webapp/dao/AppsInfo.java | 3 +- .../webapp/dao/NMDeviceResourceInfo.java | 8 + .../nodemanager/webapp/dao/NMResourceInfo.java | 13 + .../webapp/dao/gpu/NMGpuResourceInfo.java | 8 + .../TestLinuxContainerExecutorWithMocks.java | 62 +- .../server/nodemanager/webapp/TestNMWebFilter.java | 17 +- .../server/nodemanager/webapp/TestNMWebServer.java | 20 +- .../nodemanager/webapp/TestNMWebServices.java | 492 ++--- .../nodemanager/webapp/TestNMWebServicesApps.java | 404 ++-- .../webapp/TestNMWebServicesAuxServices.java | 155 +- .../webapp/TestNMWebServicesContainers.java | 251 ++- .../nodemanager/webapp/TestNMWebTerminal.java | 19 +- .../hadoop-yarn-server-resourcemanager/pom.xml | 92 +- .../server/resourcemanager/ResourceManager.java | 27 +- .../federation/FederationStateStoreHeartbeat.java | 42 +- .../PriorityUtilizationQueueOrderingPolicy.java | 6 +- .../resourcemanager/scheduler/fair/FSQueue.java | 2 +- .../webapp/JAXBContextResolver.java | 71 +- .../server/resourcemanager/webapp/RMWebApp.java | 34 +- .../resourcemanager/webapp/RMWebAppFilter.java | 33 +- .../resourcemanager/webapp/RMWebServices.java | 15 +- .../resourcemanager/webapp/dao/AppsInfo.java | 2 +- .../webapp/dao/CapacitySchedulerInfo.java | 2 +- .../webapp/dao/CapacitySchedulerQueueInfoList.java | 2 +- .../yarn/server/resourcemanager/TestRMHA.java | 27 +- .../TestFederationRMStateStoreService.java | 22 +- .../recovery/TestFSRMStateStore.java | 23 +- ...TestPriorityUtilizationQueueOrderingPolicy.java | 54 + .../webapp/ActivitiesTestUtils.java | 26 +- .../resourcemanager/webapp/TestRMWebApp.java | 29 +- .../webapp/TestRMWebAppFairScheduler.java | 38 +- .../webapp/TestRMWebServiceAppsNodelabel.java | 94 +- .../resourcemanager/webapp/TestRMWebServices.java | 369 ++-- .../webapp/TestRMWebServicesAppAttempts.java | 258 +-- .../TestRMWebServicesAppCustomResourceTypes.java | 85 +- .../webapp/TestRMWebServicesApps.java | 1012 +++++----- .../TestRMWebServicesAppsCustomResourceTypes.java | 97 +- .../webapp/TestRMWebServicesAppsModification.java | 669 ++++--- .../webapp/TestRMWebServicesCapacitySched.java | 177 +- ...TestRMWebServicesCapacitySchedDefaultLabel.java | 139 ++ ...estRMWebServicesCapacitySchedDynamicConfig.java | 185 +- ...icesCapacitySchedDynamicConfigAbsoluteMode.java | 140 ++ ...rvicesCapacitySchedDynamicConfigWeightMode.java | 138 ++ ...esCapacitySchedDynamicConfigWeightModeDQC.java} | 172 +- ...ebServicesCapacitySchedLegacyQueueCreation.java | 114 +- ...pacitySchedLegacyQueueCreationAbsoluteMode.java | 140 ++ ...WebServicesCapacitySchedulerConfigMutation.java | 138 +- ...estRMWebServicesCapacitySchedulerMixedMode.java | 216 +- ...itySchedulerMixedModeAbsoluteAndPercentage.java | 121 ++ ...lerMixedModeAbsoluteAndPercentageAndWeight.java | 120 ++ ...edModeAbsoluteAndPercentageAndWeightVector.java | 123 ++ ...edulerMixedModeAbsoluteAndPercentageVector.java | 121 ++ ...apacitySchedulerMixedModeAbsoluteAndWeight.java | 120 ++ ...ySchedulerMixedModeAbsoluteAndWeightVector.java | 121 ++ ...acitySchedulerMixedModePercentageAndWeight.java | 117 ++ ...chedulerMixedModePercentageAndWeightVector.java | 121 ++ .../TestRMWebServicesConfigurationMutation.java | 412 ++-- .../webapp/TestRMWebServicesContainers.java | 113 +- ...estRMWebServicesCustomResourceTypesCommons.java | 16 +- ...RMWebServicesDelegationTokenAuthentication.java | 54 +- .../webapp/TestRMWebServicesDelegationTokens.java | 471 +++-- .../TestRMWebServicesForCSWithPartitions.java | 218 ++- ...TestRMWebServicesHttpStaticUserPermissions.java | 8 +- .../webapp/TestRMWebServicesNodeLabels.java | 266 +-- .../webapp/TestRMWebServicesNodes.java | 573 +++--- .../webapp/TestRMWebServicesReservation.java | 543 ++--- .../TestRMWebServicesSchedulerActivities.java | 393 ++-- ...esSchedulerActivitiesWithMultiNodesEnabled.java | 149 +- .../webapp/TestRMWebappAuthentication.java | 18 +- .../resourcemanager/webapp/TestWebServiceUtil.java | 104 +- .../TestRMWebServicesFairScheduler.java | 117 +- ...ebServicesFairSchedulerCustomResourceTypes.java | 137 +- .../webapp/helper/AppInfoJsonVerifications.java | 2 +- .../webapp/helper/BufferedClientResponse.java | 23 +- .../helper/JsonCustomResourceTypeTestcase.java | 29 +- .../helper/XmlCustomResourceTypeTestCase.java | 23 +- .../webapp/reader/AppStateReader.java | 71 + .../ApplicationSubmissionContextInfoReader.java | 75 + .../webapp/reader/LabelsToNodesInfoReader.java | 74 + .../webapp/reader/NodeLabelsInfoReader.java | 74 + .../webapp/reader/NodeToLabelsInfoReader.java | 74 + .../webapp/reader/ResourceOptionInfoReader.java | 74 + .../ApplicationSubmissionContextInfoWriter.java | 89 + .../webapp/writer/ResourceOptionInfoWriter.java | 87 + .../webapp/writer/SchedConfUpdateInfoWriter.java | 87 + .../hadoop/yarn/webapp/TestRMWithCSRFFilter.java | 117 +- .../src/test/resources/delete-reservation.json | 4 +- .../src/test/resources/submit-reservation.json | 62 +- .../src/test/resources/update-reservation.json | 61 +- ...igmutation-absolute-hierarchy-after-update.json | 552 +++--- ...gmutation-absolute-hierarchy-before-update.json | 560 +++--- ...ion-absolute-hierarchy-legacy-after-update.json | 552 +++--- ...on-absolute-hierarchy-legacy-before-update.json | 560 +++--- .../webapp/dynamic-testAbsoluteMode-0.json | 832 ++++---- .../webapp/dynamic-testAbsoluteMode-16.json | 796 ++++---- .../webapp/dynamic-testAbsoluteMode-32.json | 796 ++++---- .../webapp/dynamic-testAbsoluteMode-legacy-0.json | 824 ++++---- .../webapp/dynamic-testPercentageMode-0.json | 808 ++++---- .../webapp/dynamic-testPercentageMode-16.json | 772 ++++---- .../webapp/dynamic-testPercentageMode-32.json | 772 ++++---- .../dynamic-testPercentageMode-legacy-0.json | 772 ++++---- .../resources/webapp/dynamic-testWeightMode-0.json | 808 ++++---- .../webapp/dynamic-testWeightMode-16.json | 772 ++++---- .../webapp/dynamic-testWeightMode-32.json | 772 ++++---- .../webapp/dynamic-testWeightMode-after-aqc.json | 2068 ++++++++++---------- .../webapp/dynamic-testWeightMode-before-aqc.json | 742 +++---- .../webapp/dynamic-testWeightMode-legacy-0.json | 764 ++++---- .../webapp/dynamic-testWeightMode-legacy-16.json | 764 ++++---- .../webapp/dynamic-testWeightMode-legacy-32.json | 764 ++++---- .../dynamic-testWeightMode-legacy-after-aqc.json | 2052 +++++++++---------- .../dynamic-testWeightMode-legacy-before-aqc.json | 734 +++---- ...mixed-testSchedulerAbsoluteAndPercentage-0.json | 820 ++++---- ...ixed-testSchedulerAbsoluteAndPercentage-16.json | 804 ++++---- ...ixed-testSchedulerAbsoluteAndPercentage-32.json | 784 ++++---- ...tSchedulerAbsoluteAndPercentageAndWeight-0.json | 816 ++++---- ...SchedulerAbsoluteAndPercentageAndWeight-16.json | 800 ++++---- ...SchedulerAbsoluteAndPercentageAndWeight-32.json | 780 ++++---- ...dulerAbsoluteAndPercentageAndWeightMixed-0.json | 818 ++++---- ...ulerAbsoluteAndPercentageAndWeightMixed-16.json | 802 ++++---- ...ulerAbsoluteAndPercentageAndWeightMixed-32.json | 782 ++++---- .../mixed-testSchedulerAbsoluteAndWeight-0.json | 820 ++++---- .../mixed-testSchedulerAbsoluteAndWeight-16.json | 804 ++++---- .../mixed-testSchedulerAbsoluteAndWeight-32.json | 784 ++++---- .../mixed-testSchedulerPercentageAndWeight-0.json | 808 ++++---- .../mixed-testSchedulerPercentageAndWeight-16.json | 772 ++++---- .../mixed-testSchedulerPercentageAndWeight-32.json | 772 ++++---- ...er-response-AbsoluteModeLegacyAutoCreation.json | 418 ++-- .../scheduler-response-PerUserResources.json | 578 +++--- ...-response-PercentageModeLegacyAutoCreation.json | 298 +-- .../test/resources/webapp/scheduler-response.json | 490 ++--- .../hadoop-yarn-server-router/pom.xml | 6 + .../apache/hadoop/yarn/server/router/Router.java | 6 +- .../yarn/server/router/webapp/AppsBlock.java | 4 +- .../webapp/DefaultRequestInterceptorREST.java | 2 +- .../yarn/server/router/webapp/FederationBlock.java | 17 +- .../router/webapp/FederationInterceptorREST.java | 10 +- .../server/router/webapp/MetricsOverviewTable.java | 4 +- .../yarn/server/router/webapp/NodeLabelsBlock.java | 5 +- .../yarn/server/router/webapp/NodesBlock.java | 4 +- .../yarn/server/router/webapp/RouterBlock.java | 19 +- .../yarn/server/router/webapp/RouterWebApp.java | 34 +- .../server/router/webapp/RouterWebServiceUtil.java | 159 +- .../server/router/webapp/RouterWebServices.java | 10 +- .../subcluster/TestFederationSubCluster.java | 133 +- .../TestYarnFederationWithCapacityScheduler.java | 64 +- .../fair/TestYarnFederationWithFairScheduler.java | 85 +- .../router/webapp/TestRouterWebServiceUtil.java | 28 +- .../router/webapp/TestRouterWebServicesREST.java | 322 ++- .../TestTimelineServiceClientIntegration.java | 3 + .../pom.xml | 28 + .../pom.xml | 60 +- .../pom.xml | 4 + .../pom.xml | 4 + .../storage/common/HBaseTimelineServerUtils.java | 3 +- .../hadoop-yarn-server-timelineservice/pom.xml | 21 +- .../collector/NodeTimelineCollectorManager.java | 24 +- .../collector/TimelineCollectorWebService.java | 7 +- .../reader/TimelineReaderManager.java | 1 + .../reader/TimelineReaderServer.java | 26 +- .../reader/TimelineReaderWebServices.java | 11 +- .../storage/FileSystemTimelineWriterImpl.java | 4 + .../reader/TestTimelineReaderWebServices.java | 405 ++-- .../reader/TestTimelineReaderWebServicesACL.java | 50 +- .../reader/TimelineAboutReader.java | 82 + .../reader/TimelineEntityReader.java | 58 + .../reader/TimelineEntitySetReader.java | 94 + .../reader/TimelineHealthReader.java | 57 + .../hadoop-yarn-server-web-proxy/pom.xml | 8 +- 583 files changed, 37155 insertions(+), 31090 deletions(-) create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/BalancerHttpServer.java copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/NMResourceInfo.java => hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/BalancerMXBean.java (60%) copy {hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams => hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer}/package-info.java (69%) create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/FSNLockManager.java create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/FineGrainedFSNamesystemLock.java create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/GlobalFSNamesystemLock.java copy {hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams => hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl}/package-info.java (73%) copy hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/package-info.java => hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/RwLockMode.java (73%) create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/balancer/balancer.html create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/balancer/balancer.js create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/balancer/index.html copy hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/{datanode => balancer}/robots.txt (100%) create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/NamenodeFGL.md create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerHttpServer.java create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/fgl/FSNLockBenchmarkThroughput.java create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/fgl/TestFSNLockBenchmarkThroughput.java create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/fgl/TestFineGrainedFSNamesystemLock.java create mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesLogsExtend.java create mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/reader/ContainerLogsInfoMessageBodyReader.java create mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/reader/RemoteLogPathsMessageBodyReader.java rename hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/{impl/streams/AnalyticsStream.java => S3ASeekableStream.java} (76%) delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/InputStreamCallbacksImpl.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/AbstractObjectInputStreamFactory.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/AnalyticsStreamFactory.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/ClassicObjectInputStreamFactory.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/FactoryBindingParameters.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/InputStreamType.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/ObjectInputStream.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/ObjectInputStreamCallbacks.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/ObjectInputStreamFactory.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/ObjectReadParameters.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/StreamFactoryRequirements.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/StreamIntegration.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/prefetch/PrefetchOptions.java delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/prefetch/PrefetchingInputStreamFactory.java delete mode 100644 hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/reading.md delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/streams/TestStreamFactories.java delete mode 100644 hadoop-tools/hadoop-resourceestimator/src/test/java/org/apache/hadoop/resourceestimator/service/GuiceServletConfig.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/reader/TimelineDomainReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/reader/TimelineEntitiesReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/reader/TimelinePutResponseReader.java copy {hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams => hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/reader}/package-info.java (73%) create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineDomainWriter.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineDomainsWriter.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineEntitiesWriter.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineEntityWriter.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineEventsWriter.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelinePutResponseWriter.java copy {hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams => hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer}/package-info.java (73%) create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/reader/TimelineDomainReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/reader/TimelineEntitiesReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/reader/TimelineEntityReader.java copy {hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams => hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/reader}/package-info.java (73%) create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineDomainWriter.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineEntitiesWriter.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineEntitySetWriter.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineEntityWriter.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineHealthWriter.java rename {hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams => hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer}/package-info.java (73%) copy hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/AppsInfo.java => hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/ConflictException.java} (53%) create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/ContainerLogsInfoListReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineAboutReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineDomainReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineDomainsReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineEntitiesReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineEntityReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineEventsReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelinePutResponseReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedDefaultLabel.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedDynamicConfigAbsoluteMode.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedDynamicConfigWeightMode.java copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/{TestRMWebServicesCapacitySchedDynamicConfig.java => TestRMWebServicesCapacitySchedDynamicConfigWeightModeDQC.java} (54%) create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedLegacyQueueCreationAbsoluteMode.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentage.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentageAndWeight.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentageAndWeightVector.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentageVector.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndWeight.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndWeightVector.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModePercentageAndWeight.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModePercentageAndWeightVector.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/AppStateReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/ApplicationSubmissionContextInfoReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/LabelsToNodesInfoReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/NodeLabelsInfoReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/NodeToLabelsInfoReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/ResourceOptionInfoReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/writer/ApplicationSubmissionContextInfoWriter.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/writer/ResourceOptionInfoWriter.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/writer/SchedConfUpdateInfoWriter.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineAboutReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineEntityReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineEntitySetReader.java create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineHealthReader.java --------------------------------------------------------------------- To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org