This is an automated email from the ASF dual-hosted git repository. ahmar pushed a change to branch feature-HADOOP-19363-analytics-accelerator-s3 in repository https://gitbox.apache.org/repos/asf/hadoop.git
omit e18d0a4d26d HADOOP-19348. Add initial support for Analytics Accelerator Library for Amazon S3 (#7192) omit 053afb7063f HDFS-17704. Fix TestDecommission and TestDecommissionWithBackoffMonitor often run timeout. (#7266). Contributed by hfutatzhanghb. omit 7ce4ea57807 HDFS-17711: Change fsimage loading progress percentage discontinuous to continuous omit 2343bd71a93 HDFS-17701. [FGL] Fix some javadocs (#7256) omit 8ab53982a56 HDFS-17697. [FGL] hasWriteLock and hasReadLock in FineGrainedFSNamesystemLock shouldn't throw assert error (#7250) omit 16552b6fbdc HDFS-17692. [FGL] fix bug in getContentSummary method (#7233) omit b289f9abd33 HDFS-17691. [FGL] Move FSNamesystemLockMode to org.apache.hadoop.hdfs.util package (#7232) omit e8807726b8d HDFS-17506. [FGL] Performance for phase 1 omit be0267313bd HDFS-17457. [FGL] UTs support fine-grained locking (#6741) omit e5b1c09b8a0 HDFS-17459. [FGL] Add documentation (#6737) omit 0d626f516f4 HDFS-17472. [FGL] gcDeletedSnapshot and getDelegationToken support FGL (#6743) omit e4885b02efc HDFS-17424. [FGL] DelegationTokenSecretManager supports fine-grained lock (#6696) omit 02b149da67e HDFS-17445. [FGL] some operations support fine-grained locking (#6715) omit 6d888d599f4 HDFS-17411. [FGL] Client RPCs involving snapshot support fine-grained lock (#6714) omit 4c86601cba5 HDFS-17389. [FGL] Client RPCs involving read process supports fine-grained lock (#6590) omit 7577f3ad619 HDFS-17388. [FGL] Client RPCs involving write process supports fine-grained lock (#6589) omit ed302ae3aad HDFS-17420. [FGL] EditLogTailer and FSEditLogLoader support FGL (#6679) omit 03603efd74b HDFS-17412. [FGL] Client RPCs involving maintenance supports fine-grained lock (#6667) omit 40d54ebb66a HDFS-17417. [FGL] HeartbeatManager and DatanodeAdminMonitor support fine-grained locking (#6656) omit 7d37995f46e HDFS-17415. [FGL] RPCs in NamenodeProtocol support fine-grained lock (#6654) omit 778e819b41a HDFS-17410. [FGL] Client RPCs that changes file attributes supports fine-grained lock (#6634) omit 3b5e28a87ec HDFS-17423. [FGL] BlockManagerSafeMode supports fine-grained lock (#6645) omit ac0605db419 HDFS-17416. [FGL] Monitor threads in BlockManager.class support fine-grained lock (#6647) omit 7764b39e4ab HDFS-17414. [FGL] RPCs in DatanodeProtocol support fine-grained lock (#6649) omit d39f034f982 HDFS-17413. [FGL] CacheReplicationMonitor supports fine-grained lock (#6641) omit 7158b9ee194 HDFS-17395. [FGL] Use FSLock to protect ErasureCodingPolicy related operations (#6579) omit a1ab911896f HDFS-17390. [FGL] FSDirectory supports this fine-grained locking (#6573) omit f0368bb2372 HDFS-17405. [FGL] Using different metric name to trace performance for FGL and Global lock (#6600) omit 16c25b277da HDFS-17398. [FGL] Implement the FGL lock for FSNLockManager (#6599) omit 9b45c66e34f HDFS-17387. [FGL] Abstract the configuration locking mode (#6572) omit 8afbfbc2f57 HDFS-17394. [FGL] Remove unused WriteHoldCount of FSNamesystemLock (#6571) omit e41f2ba1552 HADOOP-19392: Upgrade ftpserver to resolve maven/pom issue (#7311) Contributed by YanivKunda. omit 0432761ac84 HDFS-17657. The balancer service supports httpserver. (#7242) Contribtued by Zhaobo Huang. omit 0d72896db16 YARN-11753. Ensure NM is marked unhealthy if the ProcessBuilder reports an issue with the container-executor (#7290) omit 9bf5e38c483 YARN-11745: Fix TimSort contract violation in PriorityQueueComparator Class (#7278) omit 06d36f5a6a1 HADOOP-15984. Update jersey from 1.19 to 2.x. (#7019) add 9c8e753209d HADOOP-19354. S3A: Create S3 InputStreams via a factory add eadf0ddd71d HADOOP-19354. S3AInputStream to be created by factory under S3AStore add 0d1f291db1b Adds in integration for AWS analytics accelerator library This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (e18d0a4d26d) \ N -- N -- N refs/heads/feature-HADOOP-19363-analytics-accelerator-s3 (0d1f291db1b) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: LICENSE-binary | 28 +- .../hadoop-client-check-invariants/pom.xml | 2 - .../resources/ensure-jars-have-correct-contents.sh | 6 +- .../hadoop-client-check-test-invariants/pom.xml | 1 - .../hadoop-client-integration-tests/pom.xml | 10 +- .../hadoop-client-minicluster/pom.xml | 106 +- .../hadoop-client-runtime/pom.xml | 26 +- hadoop-client-modules/hadoop-client/pom.xml | 76 + .../hadoop-cloud-storage/pom.xml | 12 + .../hadoop-huaweicloud/pom.xml | 6 - .../apache/hadoop/fs/obs/OBSBlockOutputStream.java | 2 +- .../org/apache/hadoop/fs/obs/OBSInputStream.java | 2 +- hadoop-common-project/hadoop-auth-examples/pom.xml | 4 +- hadoop-common-project/hadoop-auth/pom.xml | 4 +- hadoop-common-project/hadoop-common/pom.xml | 97 +- .../java/org/apache/hadoop/http/HttpServer2.java | 34 +- .../apache/hadoop/util/ConfigurationHelper.java | 36 + .../apache/hadoop/fs/ftp/TestFTPFileSystem.java | 8 +- .../org/apache/hadoop/http/TestHttpServer.java | 18 +- .../org/apache/hadoop/http/TestHttpServerLogs.java | 9 +- hadoop-common-project/hadoop-kms/pom.xml | 15 +- .../hadoop-kms/src/main/conf/kms-log4j.properties | 1 + .../apache/hadoop/crypto/key/kms/server/KMS.java | 14 +- .../key/kms/server/KMSExceptionsProvider.java | 2 +- .../src/main/resources/webapps/kms/WEB-INF/web.xml | 16 +- .../hadoop/crypto/key/kms/server/TestKMS.java | 4 +- .../hadoop-kms/src/test/resources/log4j.properties | 2 + hadoop-common-project/hadoop-nfs/pom.xml | 4 +- hadoop-hdfs-project/hadoop-hdfs-client/pom.xml | 5 - hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml | 11 +- .../fs/http/server/HttpFSExceptionProvider.java | 2 +- .../fs/http/server/HttpFSParametersProvider.java | 2 + .../apache/hadoop/fs/http/server/HttpFSServer.java | 45 +- .../apache/hadoop/lib/wsrs/JSONMapProvider.java | 14 +- .../apache/hadoop/lib/wsrs/ParametersProvider.java | 67 +- .../main/resources/webapps/webhdfs/WEB-INF/web.xml | 18 +- .../src/main/webapp/WEB-INF/web.xml | 18 +- .../hadoop/fs/http/server/TestHttpFSServer.java | 3 +- .../src/test/resources/default-log4j.properties | 1 + hadoop-hdfs-project/hadoop-hdfs-nfs/pom.xml | 10 +- .../hadoop/hdfs/nfs/nfs3/Nfs3HttpServer.java | 2 +- hadoop-hdfs-project/hadoop-hdfs-rbf/pom.xml | 5 - .../server/federation/router/RouterHttpServer.java | 2 +- .../federation/router/RouterWebHdfsMethods.java | 3 + .../server/federation/router/TestRouterRpc.java | 5 +- hadoop-hdfs-project/hadoop-hdfs/pom.xml | 28 +- .../java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 21 - .../main/java/org/apache/hadoop/hdfs/DFSUtil.java | 4 +- .../qjournal/server/JournalNodeHttpServer.java | 2 +- .../delegation/DelegationTokenSecretManager.java | 9 +- .../hadoop/hdfs/server/balancer/Balancer.java | 55 +- .../hdfs/server/balancer/BalancerHttpServer.java | 114 -- .../hdfs/server/balancer/BalancerMXBean.java | 43 - .../hadoop/hdfs/server/balancer/package-info.java | 25 - .../hdfs/server/blockmanagement/BlockManager.java | 122 +- .../blockmanagement/BlockManagerSafeMode.java | 33 +- .../blockmanagement/CacheReplicationMonitor.java | 13 +- .../DatanodeAdminBackoffMonitor.java | 38 +- .../DatanodeAdminDefaultMonitor.java | 11 +- .../server/blockmanagement/DatanodeManager.java | 12 +- .../server/blockmanagement/HeartbeatManager.java | 9 +- .../server/blockmanagement/ProvidedStorageMap.java | 5 +- .../datanode/web/webhdfs/ExceptionHandler.java | 4 +- .../hadoop/hdfs/server/namenode/BackupImage.java | 5 +- .../hadoop/hdfs/server/namenode/CacheManager.java | 33 +- .../hadoop/hdfs/server/namenode/Checkpointer.java | 10 +- .../namenode/ContentSummaryComputationContext.java | 12 +- .../server/namenode/EncryptionZoneManager.java | 14 +- .../hadoop/hdfs/server/namenode/FSDirAppendOp.java | 5 +- .../hadoop/hdfs/server/namenode/FSDirDeleteOp.java | 4 +- .../server/namenode/FSDirEncryptionZoneOp.java | 15 +- .../hdfs/server/namenode/FSDirErasureCodingOp.java | 17 +- .../namenode/FSDirSatisfyStoragePolicyOp.java | 3 +- .../server/namenode/FSDirStatAndListingOp.java | 25 +- .../hdfs/server/namenode/FSDirTruncateOp.java | 9 +- .../hdfs/server/namenode/FSDirWriteFileOp.java | 46 +- .../hadoop/hdfs/server/namenode/FSDirectory.java | 29 +- .../hdfs/server/namenode/FSEditLogLoader.java | 9 +- .../hdfs/server/namenode/FSImageFormatPBINode.java | 5 +- .../hadoop/hdfs/server/namenode/FSNamesystem.java | 672 +++---- .../hdfs/server/namenode/FSNamesystemLock.java | 44 +- .../hdfs/server/namenode/FSPermissionChecker.java | 15 +- .../hdfs/server/namenode/FSTreeTraverser.java | 7 +- .../hdfs/server/namenode/FsImageValidation.java | 5 +- .../hadoop/hdfs/server/namenode/INodeFile.java | 6 - .../hadoop/hdfs/server/namenode/LeaseManager.java | 15 +- .../hadoop/hdfs/server/namenode/NameNode.java | 25 +- .../hdfs/server/namenode/NameNodeHttpServer.java | 29 +- .../hadoop/hdfs/server/namenode/NamenodeFsck.java | 10 +- .../hdfs/server/namenode/ReencryptionHandler.java | 11 +- .../hdfs/server/namenode/ReencryptionUpdater.java | 7 +- .../hdfs/server/namenode/SecondaryNameNode.java | 7 +- .../hdfs/server/namenode/fgl/FSNLockManager.java | 187 -- .../namenode/fgl/FineGrainedFSNamesystemLock.java | 285 --- .../namenode/fgl/GlobalFSNamesystemLock.java | 150 -- .../hdfs/server/namenode/fgl/package-info.java | 18 - .../hdfs/server/namenode/ha/EditLogTailer.java | 5 +- .../namenode/snapshot/SnapshotDeletionGc.java | 5 +- .../web/resources/NamenodeWebHdfsMethods.java | 285 +-- .../java/org/apache/hadoop/hdfs/util/RwLock.java | 106 +- .../org/apache/hadoop/hdfs/util/RwLockMode.java | 27 - .../org/apache/hadoop/hdfs/web/ParamFilter.java | 103 +- .../hdfs/web/resources/ExceptionHandler.java | 11 +- .../hdfs/web/resources/TokenServiceParam.java | 2 + .../hadoop/hdfs/web/resources/UserProvider.java | 35 +- .../src/main/resources/hdfs-default.xml | 63 - .../src/main/webapps/balancer/balancer.html | 114 -- .../src/main/webapps/balancer/balancer.js | 95 - .../src/main/webapps/balancer/index.html | 24 - .../src/main/webapps/balancer/robots.txt | 2 - .../hadoop-hdfs/src/site/markdown/NamenodeFGL.md | 210 -- .../apache/hadoop/hdfs/AdminStatesBaseTest.java | 4 - .../hadoop/hdfs/TestBlocksScheduledCounter.java | 11 +- .../org/apache/hadoop/hdfs/TestDecommission.java | 51 +- .../org/apache/hadoop/hdfs/TestFileCorruption.java | 13 +- .../apache/hadoop/hdfs/TestMaintenanceState.java | 50 +- .../server/balancer/TestBalancerHttpServer.java | 100 - .../hdfs/server/balancer/TestBalancerService.java | 51 - .../blockmanagement/BlockManagerTestUtil.java | 22 +- .../server/blockmanagement/TestBlockManager.java | 15 +- .../blockmanagement/TestBlockManagerSafeMode.java | 3 - .../TestBlocksWithNotEnoughRacks.java | 5 +- .../blockmanagement/TestComputeInvalidateWork.java | 25 +- .../blockmanagement/TestDatanodeManager.java | 15 +- .../blockmanagement/TestHeartbeatHandling.java | 9 +- .../TestNameNodePrunesMissingStorages.java | 10 +- .../hdfs/server/blockmanagement/TestNodeCount.java | 5 +- .../blockmanagement/TestOverReplicatedBlocks.java | 9 +- .../blockmanagement/TestPendingReconstruction.java | 17 +- .../blockmanagement/TestProvidedStorageMap.java | 3 +- ...tReconstructStripedBlocksWithRackAwareness.java | 10 +- .../blockmanagement/TestReplicationPolicy.java | 8 - .../TestReplicationPolicyConsiderLoad.java | 10 +- .../TestReplicationPolicyExcludeSlowNodes.java | 11 +- ...licationPolicyRatioConsiderLoadWithStorage.java | 6 +- .../hdfs/server/namenode/NameNodeAdapter.java | 17 +- .../hdfs/server/namenode/TestAddBlockRetry.java | 14 +- .../TestAddOverReplicatedStripedBlocks.java | 6 +- .../TestBlockPlacementPolicyRackFaultTolerant.java | 6 +- .../hdfs/server/namenode/TestCacheDirectives.java | 17 +- .../hdfs/server/namenode/TestDeleteRace.java | 5 +- .../server/namenode/TestDiskspaceQuotaUpdate.java | 5 +- .../hdfs/server/namenode/TestEditLogRace.java | 6 +- .../server/namenode/TestFSImageWithSnapshot.java | 9 +- .../hdfs/server/namenode/TestFSNamesystem.java | 5 +- .../hdfs/server/namenode/TestFSNamesystemLock.java | 20 +- .../server/namenode/TestFSNamesystemMBean.java | 7 +- .../hdfs/server/namenode/TestFileTruncate.java | 9 +- .../hadoop/hdfs/server/namenode/TestFsck.java | 9 +- .../server/namenode/TestGetBlockLocations.java | 14 +- .../server/namenode/TestLargeDirectoryDelete.java | 5 +- .../hdfs/server/namenode/TestLeaseManager.java | 5 - .../hdfs/server/namenode/TestListOpenFiles.java | 5 +- .../namenode/TestNameNodeMetadataConsistency.java | 11 +- .../namenode/TestReconstructStripedBlocks.java | 5 +- .../server/namenode/TestSecurityTokenEditLog.java | 5 +- .../namenode/fgl/FSNLockBenchmarkThroughput.java | 322 --- .../fgl/TestFSNLockBenchmarkThroughput.java | 104 - .../fgl/TestFineGrainedFSNamesystemLock.java | 276 --- .../hdfs/server/namenode/ha/TestDNFencing.java | 5 +- .../namenode/metrics/TestNameNodeMetrics.java | 13 +- ...TestINodeFileUnderConstructionWithSnapshot.java | 5 +- .../org/apache/hadoop/hdfs/tools/TestDFSAdmin.java | 5 +- .../hdfs/web/TestWebHdfsFileSystemContract.java | 10 +- .../src/test/resources/log4j.properties | 4 + .../hadoop-mapreduce-client-app/pom.xml | 44 - .../jobhistory/JobHistoryEventHandler.java | 8 +- .../mapreduce/v2/app/client/MRClientService.java | 28 +- .../hadoop/mapreduce/v2/app/webapp/AMWebApp.java | 19 +- .../mapreduce/v2/app/webapp/AMWebServices.java | 11 +- .../apache/hadoop/mapreduce/v2/app/webapp/App.java | 7 +- .../v2/app/webapp/JAXBContextResolver.java | 15 +- .../mapreduce/v2/app/webapp/TestAMWebApp.java | 45 +- .../mapreduce/v2/app/webapp/TestAMWebServices.java | 245 +-- .../v2/app/webapp/TestAMWebServicesAttempt.java | 145 +- .../v2/app/webapp/TestAMWebServicesAttempts.java | 246 +-- .../v2/app/webapp/TestAMWebServicesJobConf.java | 124 +- .../v2/app/webapp/TestAMWebServicesJobs.java | 434 ++-- .../v2/app/webapp/TestAMWebServicesTasks.java | 427 ++-- .../hadoop-mapreduce-client-common/pom.xml | 5 - .../hadoop-mapreduce-client-core/pom.xml | 20 - .../hadoop-mapreduce-client-hs/pom.xml | 30 - .../mapreduce/v2/hs/HistoryClientService.java | 38 - .../hadoop/mapreduce/v2/hs/webapp/HsWebApp.java | 4 + .../mapreduce/v2/hs/webapp/HsWebServices.java | 34 +- .../v2/hs/webapp/JAXBContextResolver.java | 10 +- .../mapreduce/v2/hs/webapp/TestHsWebServices.java | 253 +-- .../v2/hs/webapp/TestHsWebServicesAttempts.java | 257 +-- .../v2/hs/webapp/TestHsWebServicesJobConf.java | 120 +- .../v2/hs/webapp/TestHsWebServicesJobs.java | 468 +++-- .../v2/hs/webapp/TestHsWebServicesJobsQuery.java | 471 ++--- .../v2/hs/webapp/TestHsWebServicesLogs.java | 398 ++-- .../v2/hs/webapp/TestHsWebServicesLogsExtend.java | 399 ---- .../v2/hs/webapp/TestHsWebServicesTasks.java | 446 +++-- .../reader/ContainerLogsInfoMessageBodyReader.java | 103 - .../reader/RemoteLogPathsMessageBodyReader.java | 88 - .../hadoop-mapreduce-client-jobclient/pom.xml | 21 - .../hadoop-mapreduce-client-nativetask/pom.xml | 20 - .../hadoop-mapreduce-client-shuffle/pom.xml | 5 - .../hadoop-mapreduce-client/pom.xml | 9 +- .../hadoop-mapreduce-examples/pom.xml | 9 +- hadoop-project/pom.xml | 199 +- hadoop-tools/hadoop-archive-logs/pom.xml | 20 - hadoop-tools/hadoop-archives/pom.xml | 20 - .../hadoop-aws/dev-support/findbugs-exclude.xml | 2 +- hadoop-tools/hadoop-aws/pom.xml | 26 - .../java/org/apache/hadoop/fs/s3a/Constants.java | 73 +- .../org/apache/hadoop/fs/s3a/S3AFileSystem.java | 380 ++-- .../org/apache/hadoop/fs/s3a/S3AInputStream.java | 388 +--- .../org/apache/hadoop/fs/s3a/S3AReadOpContext.java | 44 +- .../java/org/apache/hadoop/fs/s3a/S3AStore.java | 38 +- .../apache/hadoop/fs/s3a/VectoredIOContext.java | 17 + .../apache/hadoop/fs/s3a/impl/ClientManager.java | 9 +- .../hadoop/fs/s3a/impl/ClientManagerImpl.java | 56 +- .../fs/s3a/impl/InputStreamCallbacksImpl.java | 123 ++ .../apache/hadoop/fs/s3a/impl/S3AStoreImpl.java | 208 +- .../streams/AbstractObjectInputStreamFactory.java | 94 + .../streams/AnalyticsStream.java} | 52 +- .../s3a/impl/streams/AnalyticsStreamFactory.java | 100 + .../streams/ClassicObjectInputStreamFactory.java | 73 + .../streams/FactoryBindingParameters.java} | 28 +- .../fs/s3a/impl/streams/InputStreamType.java | 123 ++ .../fs/s3a/impl/streams/ObjectInputStream.java | 378 ++++ .../impl/streams/ObjectInputStreamCallbacks.java | 64 + .../s3a/impl/streams/ObjectInputStreamFactory.java | 89 + .../fs/s3a/impl/streams/ObjectReadParameters.java | 186 ++ .../impl/streams/StreamFactoryRequirements.java | 92 + .../fs/s3a/impl/streams/StreamIntegration.java | 213 ++ .../hadoop/fs/s3a/impl/streams}/package-info.java | 11 +- .../hadoop/fs/s3a/prefetch/PrefetchOptions.java | 67 + .../prefetch/PrefetchingInputStreamFactory.java | 108 + .../fs/s3a/prefetch/S3ACachingInputStream.java | 16 +- .../fs/s3a/prefetch/S3AInMemoryInputStream.java | 7 +- .../fs/s3a/prefetch/S3APrefetchingInputStream.java | 67 +- .../fs/s3a/prefetch/S3ARemoteInputStream.java | 14 +- .../hadoop/fs/s3a/prefetch/S3ARemoteObject.java | 5 +- .../site/markdown/tools/hadoop-aws/prefetching.md | 14 +- .../src/site/markdown/tools/hadoop-aws/reading.md | 112 ++ .../fs/contract/s3a/ITestS3AContractCreate.java | 7 +- .../fs/contract/s3a/ITestS3AContractRename.java | 8 - .../fs/contract/s3a/ITestS3AContractSeek.java | 2 +- .../hadoop/fs/s3a/ITestS3AConfiguration.java | 18 +- .../apache/hadoop/fs/s3a/ITestS3ADelayedFNF.java | 3 - .../hadoop/fs/s3a/ITestS3AEncryptionSSEC.java | 2 +- .../hadoop/fs/s3a/ITestS3AFSMainOperations.java | 16 +- .../hadoop/fs/s3a/ITestS3AFileSystemContract.java | 7 +- .../hadoop/fs/s3a/ITestS3AIOStatisticsContext.java | 5 +- .../hadoop/fs/s3a/ITestS3AInputStreamLeakage.java | 47 +- .../org/apache/hadoop/fs/s3a/ITestS3AMetrics.java | 5 +- .../fs/s3a/ITestS3APrefetchingCacheFiles.java | 11 +- .../fs/s3a/ITestS3APrefetchingInputStream.java | 10 +- .../fs/s3a/ITestS3APrefetchingLruEviction.java | 15 +- .../hadoop/fs/s3a/ITestS3ARequesterPays.java | 5 +- .../hadoop/fs/s3a/ITestS3AS3SeekableStream.java | 55 +- .../org/apache/hadoop/fs/s3a/S3ATestUtils.java | 38 +- .../hadoop/fs/s3a/TestS3AInputStreamRetry.java | 27 +- .../fs/s3a/commit/ITestCommitOperationCost.java | 11 +- .../fs/s3a/commit/ITestS3ACommitterFactory.java | 3 - .../s3a/commit/magic/ITestMagicCommitProtocol.java | 3 - .../integration/ITestDirectoryCommitProtocol.java | 9 - .../ITestPartitionedCommitProtocol.java | 9 - .../integration/ITestStagingCommitProtocol.java | 4 - .../ITestStagingCommitProtocolFailure.java | 12 +- .../fileContext/ITestS3AFileContextStatistics.java | 4 +- .../fs/s3a/impl/ITestConnectionTimeouts.java | 18 +- .../fs/s3a/impl/streams/TestStreamFactories.java | 240 +++ .../fs/s3a/performance/ITestS3AOpenCost.java | 12 +- .../fs/s3a/performance/ITestUnbufferDraining.java | 10 +- .../fs/s3a/prefetch/MockS3ARemoteObject.java | 8 +- .../hadoop/fs/s3a/prefetch/S3APrefetchFakes.java | 43 +- .../fs/s3a/prefetch/TestS3ARemoteInputStream.java | 19 +- .../fs/s3a/prefetch/TestS3ARemoteObject.java | 6 +- .../s3a/scale/ITestS3AHugeFilesSSECDiskBlocks.java | 3 + .../s3a/scale/ITestS3AInputStreamPerformance.java | 47 +- .../ITestS3AContractStreamIOStatistics.java | 9 +- .../statistics/ITestS3AFileSystemStatistic.java | 8 +- .../hadoop-aws/src/test/resources/log4j.properties | 4 + hadoop-tools/hadoop-azure-datalake/pom.xml | 21 - hadoop-tools/hadoop-azure/pom.xml | 25 +- hadoop-tools/hadoop-datajoin/pom.xml | 20 - hadoop-tools/hadoop-distcp/pom.xml | 20 - .../hadoop-dynamometer-blockgen/pom.xml | 20 - .../hadoop-dynamometer-infra/pom.xml | 20 - .../hadoop-dynamometer-workload/pom.xml | 20 - hadoop-tools/hadoop-extras/pom.xml | 20 - hadoop-tools/hadoop-federation-balance/pom.xml | 20 - hadoop-tools/hadoop-fs2img/pom.xml | 20 - .../namenode/ITestProvidedImplementation.java | 13 +- hadoop-tools/hadoop-gridmix/pom.xml | 20 - hadoop-tools/hadoop-resourceestimator/pom.xml | 53 +- .../service/ResourceEstimatorService.java | 33 +- .../service}/GuiceServletConfig.java | 11 +- .../service/TestResourceEstimatorService.java | 69 +- hadoop-tools/hadoop-rumen/pom.xml | 20 - hadoop-tools/hadoop-sls/pom.xml | 20 - hadoop-tools/hadoop-streaming/pom.xml | 20 - .../hadoop-yarn/hadoop-yarn-api/pom.xml | 4 +- .../api/records/timeline/TimelineEntities.java | 2 - .../yarn/api/records/timeline/TimelineEntity.java | 11 +- .../yarn/api/records/timeline/TimelineEvent.java | 2 - .../yarn/api/records/timeline/TimelineEvents.java | 5 +- .../api/records/timeline/TimelinePutResponse.java | 7 - .../timeline/reader/TimelineDomainReader.java | 57 - .../timeline/reader/TimelineEntitiesReader.java | 57 - .../timeline/reader/TimelinePutResponseReader.java | 58 - .../api/records/timeline/reader/package-info.java | 23 - .../timeline/writer/TimelineDomainWriter.java | 65 - .../timeline/writer/TimelineDomainsWriter.java | 65 - .../timeline/writer/TimelineEntitiesWriter.java | 65 - .../timeline/writer/TimelineEntityWriter.java | 65 - .../timeline/writer/TimelineEventsWriter.java | 66 - .../timeline/writer/TimelinePutResponseWriter.java | 66 - .../api/records/timeline/writer/package-info.java | 23 - .../records/timelineservice/TimelineEntities.java | 2 - .../records/timelineservice/TimelineEntity.java | 6 - .../api/records/timelineservice/TimelineEvent.java | 3 - .../records/timelineservice/TimelineMetric.java | 12 +- .../reader/TimelineDomainReader.java | 57 - .../reader/TimelineEntitiesReader.java | 57 - .../reader/TimelineEntityReader.java | 57 - .../timelineservice/reader/package-info.java | 23 - .../writer/TimelineDomainWriter.java | 65 - .../writer/TimelineEntitiesWriter.java | 65 - .../writer/TimelineEntitySetWriter.java | 67 - .../writer/TimelineEntityWriter.java | 67 - .../writer/TimelineHealthWriter.java | 66 - .../timelineservice/writer/package-info.java | 23 - .../pom.xml | 23 + .../appcatalog/application/YarnServiceClient.java | 80 +- .../src/main/webapp/WEB-INF/web.xml | 8 +- .../pom.xml | 17 - .../distributedshell/ApplicationMaster.java | 10 +- .../distributedshell/TestDSTimelineV10.java | 37 +- .../hadoop-yarn-services-api/pom.xml | 10 +- .../yarn/service/client/ApiServiceClient.java | 110 +- .../src/main/webapp/WEB-INF/web.xml | 8 +- .../apache/hadoop/yarn/service/TestApiServer.java | 53 +- .../hadoop-yarn-services-core/pom.xml | 20 +- .../hadoop/yarn/service/ServiceScheduler.java | 14 +- .../apache/hadoop/yarn/service/utils/HttpUtil.java | 13 +- .../hadoop/yarn/service/ServiceTestUtils.java | 21 +- .../hadoop-yarn/hadoop-yarn-client/pom.xml | 17 +- .../org/apache/hadoop/yarn/client/cli/LogsCLI.java | 268 +-- .../hadoop/yarn/client/cli/SchedConfCLI.java | 138 +- .../apache/hadoop/yarn/client/cli/TestLogsCLI.java | 23 +- .../hadoop/yarn/client/cli/TestSchedConfCLI.java | 114 +- .../hadoop-yarn/hadoop-yarn-common/pom.xml | 74 +- .../yarn/client/api/impl/DirectTimelineWriter.java | 9 +- .../client/api/impl/FileSystemTimelineWriter.java | 8 +- .../yarn/client/api/impl/TimelineClientImpl.java | 12 +- .../yarn/client/api/impl/TimelineConnector.java | 223 ++- .../client/api/impl/TimelineReaderClientImpl.java | 90 +- .../yarn/client/api/impl/TimelineV2ClientImpl.java | 85 +- .../yarn/client/api/impl/TimelineWriter.java | 64 +- .../hadoop/yarn/logaggregation/LogToolUtils.java | 16 +- .../hadoop/yarn/webapp/ConflictException.java | 40 - .../yarn/webapp/GenericExceptionHandler.java | 29 +- .../java/org/apache/hadoop/yarn/webapp/WebApp.java | 32 +- .../org/apache/hadoop/yarn/webapp/WebApps.java | 39 +- .../yarn/webapp/YarnJacksonJaxbJsonProvider.java | 2 +- .../hadoop/yarn/webapp/util/WebAppUtils.java | 16 - .../hadoop/yarn/webapp/util/WebServiceClient.java | 52 +- .../yarn/webapp/util/YarnWebServiceUtils.java | 75 +- .../yarn/client/api/impl/TestTimelineClient.java | 80 +- .../api/impl/TestTimelineClientForATS1_5.java | 18 +- .../api/impl/TestTimelineReaderClientImpl.java | 34 +- .../apache/hadoop/yarn/webapp/JerseyTestBase.java | 37 +- .../yarn/webapp/MyTestJAXBContextResolver.java | 11 +- .../hadoop/yarn/webapp/MyTestWebService.java | 2 +- .../org/apache/hadoop/yarn/webapp/TestWebApp.java | 110 +- .../yarn/webapp/util/TestWebServiceClient.java | 7 +- .../hadoop-yarn/hadoop-yarn-csi/pom.xml | 5 + .../pom.xml | 66 +- .../ApplicationHistoryServer.java | 47 +- .../webapp/AHSWebApp.java | 12 +- .../webapp/AHSWebServices.java | 20 +- .../webapp/ContextFactory.java | 2 +- .../webapp/JAXBContextResolver.java | 11 +- .../timeline/webapp/TimelineWebServices.java | 35 +- .../webapp/TestAHSWebServices.java | 461 ++--- .../reader/ContainerLogsInfoListReader.java | 117 -- .../timeline/reader/TimelineAboutReader.java | 82 - .../timeline/reader/TimelineDomainReader.java | 57 - .../timeline/reader/TimelineDomainsReader.java | 57 - .../timeline/reader/TimelineEntitiesReader.java | 57 - .../timeline/reader/TimelineEntityReader.java | 69 - .../timeline/reader/TimelineEventsReader.java | 80 - .../timeline/reader/TimelinePutResponseReader.java | 57 - .../TestTimelineAuthenticationFilterForV1.java | 4 +- .../timeline/webapp/TestTimelineWebServices.java | 929 ++++----- .../webapp/TestTimelineWebServicesWithSSL.java | 17 +- .../hadoop-yarn-server-common/pom.xml | 24 - .../policies/dao/WeightedPolicyInfo.java | 57 +- .../federation/store/records/SubClusterIdInfo.java | 2 +- .../hadoop/yarn/server/webapp/LogServlet.java | 4 +- .../hadoop/yarn/server/webapp/LogWebService.java | 70 +- .../yarn/server/webapp/LogWebServiceUtils.java | 9 +- .../yarn/server/webapp/dao/ContainerLogsInfo.java | 29 +- .../pom.xml | 27 +- .../server/globalpolicygenerator/GPGUtils.java | 29 +- .../GlobalPolicyGenerator.java | 2 +- .../globalpolicygenerator/webapp/GPGWebApp.java | 23 +- .../webapp/GPGWebServices.java | 10 +- .../policygenerator/TestPolicyGenerator.java | 23 +- .../webapp/TestGPGWebServices.java | 76 +- .../src/test/resources/schedulerInfo1.json | 250 ++- .../src/test/resources/schedulerInfo2.json | 374 ++-- .../hadoop-yarn-server-nodemanager/pom.xml | 96 +- .../server/nodemanager/LinuxContainerExecutor.java | 6 +- .../nodemanager/api/deviceplugin/Device.java | 13 +- .../deviceframework/AssignedDevice.java | 4 +- .../resourceplugin/gpu/AssignedGpuDevice.java | 3 - .../resourceplugin/gpu/GpuDevice.java | 3 - .../nodemanager/webapp/JAXBContextResolver.java | 15 +- .../server/nodemanager/webapp/NMWebAppFilter.java | 37 +- .../server/nodemanager/webapp/NMWebServices.java | 50 +- .../yarn/server/nodemanager/webapp/WebServer.java | 79 +- .../server/nodemanager/webapp/dao/AppsInfo.java | 3 +- .../webapp/dao/NMDeviceResourceInfo.java | 8 - .../nodemanager/webapp/dao/NMResourceInfo.java | 13 - .../webapp/dao/gpu/NMGpuResourceInfo.java | 8 - .../TestLinuxContainerExecutorWithMocks.java | 62 +- .../server/nodemanager/webapp/TestNMWebFilter.java | 17 +- .../server/nodemanager/webapp/TestNMWebServer.java | 20 +- .../nodemanager/webapp/TestNMWebServices.java | 492 +++-- .../nodemanager/webapp/TestNMWebServicesApps.java | 404 ++-- .../webapp/TestNMWebServicesAuxServices.java | 155 +- .../webapp/TestNMWebServicesContainers.java | 251 +-- .../nodemanager/webapp/TestNMWebTerminal.java | 19 +- .../hadoop-yarn-server-resourcemanager/pom.xml | 92 +- .../server/resourcemanager/ResourceManager.java | 27 +- .../federation/FederationStateStoreHeartbeat.java | 42 +- .../PriorityUtilizationQueueOrderingPolicy.java | 6 +- .../resourcemanager/scheduler/fair/FSQueue.java | 2 +- .../webapp/JAXBContextResolver.java | 71 +- .../server/resourcemanager/webapp/RMWebApp.java | 34 +- .../resourcemanager/webapp/RMWebAppFilter.java | 33 +- .../resourcemanager/webapp/RMWebServices.java | 15 +- .../resourcemanager/webapp/dao/AppsInfo.java | 2 +- .../webapp/dao/CapacitySchedulerInfo.java | 2 +- .../webapp/dao/CapacitySchedulerQueueInfoList.java | 2 +- .../yarn/server/resourcemanager/TestRMHA.java | 27 +- .../TestFederationRMStateStoreService.java | 22 +- .../recovery/TestFSRMStateStore.java | 23 +- ...TestPriorityUtilizationQueueOrderingPolicy.java | 54 - .../webapp/ActivitiesTestUtils.java | 26 +- .../resourcemanager/webapp/TestRMWebApp.java | 29 +- .../webapp/TestRMWebAppFairScheduler.java | 38 +- .../webapp/TestRMWebServiceAppsNodelabel.java | 94 +- .../resourcemanager/webapp/TestRMWebServices.java | 369 ++-- .../webapp/TestRMWebServicesAppAttempts.java | 258 ++- .../TestRMWebServicesAppCustomResourceTypes.java | 85 +- .../webapp/TestRMWebServicesApps.java | 1012 +++++----- .../TestRMWebServicesAppsCustomResourceTypes.java | 97 +- .../webapp/TestRMWebServicesAppsModification.java | 669 +++---- .../webapp/TestRMWebServicesCapacitySched.java | 177 +- ...TestRMWebServicesCapacitySchedDefaultLabel.java | 139 -- ...estRMWebServicesCapacitySchedDynamicConfig.java | 185 +- ...icesCapacitySchedDynamicConfigAbsoluteMode.java | 140 -- ...rvicesCapacitySchedDynamicConfigWeightMode.java | 138 -- ...cesCapacitySchedDynamicConfigWeightModeDQC.java | 195 -- ...ebServicesCapacitySchedLegacyQueueCreation.java | 114 +- ...pacitySchedLegacyQueueCreationAbsoluteMode.java | 140 -- ...WebServicesCapacitySchedulerConfigMutation.java | 138 +- ...estRMWebServicesCapacitySchedulerMixedMode.java | 216 +- ...itySchedulerMixedModeAbsoluteAndPercentage.java | 121 -- ...lerMixedModeAbsoluteAndPercentageAndWeight.java | 120 -- ...edModeAbsoluteAndPercentageAndWeightVector.java | 123 -- ...edulerMixedModeAbsoluteAndPercentageVector.java | 121 -- ...apacitySchedulerMixedModeAbsoluteAndWeight.java | 120 -- ...ySchedulerMixedModeAbsoluteAndWeightVector.java | 121 -- ...acitySchedulerMixedModePercentageAndWeight.java | 117 -- ...chedulerMixedModePercentageAndWeightVector.java | 121 -- .../TestRMWebServicesConfigurationMutation.java | 412 ++-- .../webapp/TestRMWebServicesContainers.java | 113 +- ...estRMWebServicesCustomResourceTypesCommons.java | 16 +- ...RMWebServicesDelegationTokenAuthentication.java | 54 +- .../webapp/TestRMWebServicesDelegationTokens.java | 471 ++--- .../TestRMWebServicesForCSWithPartitions.java | 218 +-- ...TestRMWebServicesHttpStaticUserPermissions.java | 8 +- .../webapp/TestRMWebServicesNodeLabels.java | 266 ++- .../webapp/TestRMWebServicesNodes.java | 573 +++--- .../webapp/TestRMWebServicesReservation.java | 543 +++-- .../TestRMWebServicesSchedulerActivities.java | 393 ++-- ...esSchedulerActivitiesWithMultiNodesEnabled.java | 149 +- .../webapp/TestRMWebappAuthentication.java | 18 +- .../resourcemanager/webapp/TestWebServiceUtil.java | 104 +- .../TestRMWebServicesFairScheduler.java | 117 +- ...ebServicesFairSchedulerCustomResourceTypes.java | 137 +- .../webapp/helper/AppInfoJsonVerifications.java | 2 +- .../webapp/helper/BufferedClientResponse.java | 23 +- .../helper/JsonCustomResourceTypeTestcase.java | 29 +- .../helper/XmlCustomResourceTypeTestCase.java | 23 +- .../webapp/reader/AppStateReader.java | 71 - .../ApplicationSubmissionContextInfoReader.java | 75 - .../webapp/reader/LabelsToNodesInfoReader.java | 74 - .../webapp/reader/NodeLabelsInfoReader.java | 74 - .../webapp/reader/NodeToLabelsInfoReader.java | 74 - .../webapp/reader/ResourceOptionInfoReader.java | 74 - .../ApplicationSubmissionContextInfoWriter.java | 89 - .../webapp/writer/ResourceOptionInfoWriter.java | 87 - .../webapp/writer/SchedConfUpdateInfoWriter.java | 87 - .../hadoop/yarn/webapp/TestRMWithCSRFFilter.java | 117 +- .../src/test/resources/delete-reservation.json | 4 +- .../src/test/resources/submit-reservation.json | 62 +- .../src/test/resources/update-reservation.json | 61 +- ...igmutation-absolute-hierarchy-after-update.json | 552 +++--- ...gmutation-absolute-hierarchy-before-update.json | 560 +++--- ...ion-absolute-hierarchy-legacy-after-update.json | 552 +++--- ...on-absolute-hierarchy-legacy-before-update.json | 560 +++--- .../webapp/dynamic-testAbsoluteMode-0.json | 832 ++++---- .../webapp/dynamic-testAbsoluteMode-16.json | 796 ++++---- .../webapp/dynamic-testAbsoluteMode-32.json | 796 ++++---- .../webapp/dynamic-testAbsoluteMode-legacy-0.json | 824 ++++---- .../webapp/dynamic-testPercentageMode-0.json | 808 ++++---- .../webapp/dynamic-testPercentageMode-16.json | 772 ++++---- .../webapp/dynamic-testPercentageMode-32.json | 772 ++++---- .../dynamic-testPercentageMode-legacy-0.json | 772 ++++---- .../resources/webapp/dynamic-testWeightMode-0.json | 808 ++++---- .../webapp/dynamic-testWeightMode-16.json | 772 ++++---- .../webapp/dynamic-testWeightMode-32.json | 772 ++++---- .../webapp/dynamic-testWeightMode-after-aqc.json | 2068 ++++++++++---------- .../webapp/dynamic-testWeightMode-before-aqc.json | 742 +++---- .../webapp/dynamic-testWeightMode-legacy-0.json | 764 ++++---- .../webapp/dynamic-testWeightMode-legacy-16.json | 764 ++++---- .../webapp/dynamic-testWeightMode-legacy-32.json | 764 ++++---- .../dynamic-testWeightMode-legacy-after-aqc.json | 2052 +++++++++---------- .../dynamic-testWeightMode-legacy-before-aqc.json | 734 +++---- ...mixed-testSchedulerAbsoluteAndPercentage-0.json | 820 ++++---- ...ixed-testSchedulerAbsoluteAndPercentage-16.json | 804 ++++---- ...ixed-testSchedulerAbsoluteAndPercentage-32.json | 784 ++++---- ...tSchedulerAbsoluteAndPercentageAndWeight-0.json | 816 ++++---- ...SchedulerAbsoluteAndPercentageAndWeight-16.json | 800 ++++---- ...SchedulerAbsoluteAndPercentageAndWeight-32.json | 780 ++++---- ...dulerAbsoluteAndPercentageAndWeightMixed-0.json | 818 ++++---- ...ulerAbsoluteAndPercentageAndWeightMixed-16.json | 802 ++++---- ...ulerAbsoluteAndPercentageAndWeightMixed-32.json | 782 ++++---- .../mixed-testSchedulerAbsoluteAndWeight-0.json | 820 ++++---- .../mixed-testSchedulerAbsoluteAndWeight-16.json | 804 ++++---- .../mixed-testSchedulerAbsoluteAndWeight-32.json | 784 ++++---- .../mixed-testSchedulerPercentageAndWeight-0.json | 808 ++++---- .../mixed-testSchedulerPercentageAndWeight-16.json | 772 ++++---- .../mixed-testSchedulerPercentageAndWeight-32.json | 772 ++++---- ...er-response-AbsoluteModeLegacyAutoCreation.json | 418 ++-- .../scheduler-response-PerUserResources.json | 578 +++--- ...-response-PercentageModeLegacyAutoCreation.json | 298 +-- .../test/resources/webapp/scheduler-response.json | 490 ++--- .../hadoop-yarn-server-router/pom.xml | 6 - .../apache/hadoop/yarn/server/router/Router.java | 6 +- .../yarn/server/router/webapp/AppsBlock.java | 4 +- .../webapp/DefaultRequestInterceptorREST.java | 2 +- .../yarn/server/router/webapp/FederationBlock.java | 17 +- .../router/webapp/FederationInterceptorREST.java | 10 +- .../server/router/webapp/MetricsOverviewTable.java | 4 +- .../yarn/server/router/webapp/NodeLabelsBlock.java | 5 +- .../yarn/server/router/webapp/NodesBlock.java | 4 +- .../yarn/server/router/webapp/RouterBlock.java | 19 +- .../yarn/server/router/webapp/RouterWebApp.java | 34 +- .../server/router/webapp/RouterWebServiceUtil.java | 159 +- .../server/router/webapp/RouterWebServices.java | 10 +- .../subcluster/TestFederationSubCluster.java | 133 +- .../TestYarnFederationWithCapacityScheduler.java | 64 +- .../fair/TestYarnFederationWithFairScheduler.java | 85 +- .../router/webapp/TestRouterWebServiceUtil.java | 28 +- .../router/webapp/TestRouterWebServicesREST.java | 322 +-- .../TestTimelineServiceClientIntegration.java | 3 - .../pom.xml | 28 - .../pom.xml | 60 +- .../pom.xml | 4 - .../pom.xml | 4 - .../storage/common/HBaseTimelineServerUtils.java | 3 +- .../hadoop-yarn-server-timelineservice/pom.xml | 21 +- .../collector/NodeTimelineCollectorManager.java | 24 +- .../collector/TimelineCollectorWebService.java | 7 +- .../reader/TimelineReaderManager.java | 1 - .../reader/TimelineReaderServer.java | 26 +- .../reader/TimelineReaderWebServices.java | 11 +- .../storage/FileSystemTimelineWriterImpl.java | 4 - .../reader/TestTimelineReaderWebServices.java | 405 ++-- .../reader/TestTimelineReaderWebServicesACL.java | 50 +- .../reader/TimelineAboutReader.java | 82 - .../reader/TimelineEntityReader.java | 58 - .../reader/TimelineEntitySetReader.java | 94 - .../reader/TimelineHealthReader.java | 57 - .../hadoop-yarn-server-web-proxy/pom.xml | 8 +- 585 files changed, 30834 insertions(+), 37490 deletions(-) delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/BalancerHttpServer.java delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/BalancerMXBean.java delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/package-info.java delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/FSNLockManager.java delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/FineGrainedFSNamesystemLock.java delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/GlobalFSNamesystemLock.java delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/package-info.java delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/RwLockMode.java delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/balancer/balancer.html delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/balancer/balancer.js delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/balancer/index.html delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/balancer/robots.txt delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/NamenodeFGL.md delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerHttpServer.java delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/fgl/FSNLockBenchmarkThroughput.java delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/fgl/TestFSNLockBenchmarkThroughput.java delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/fgl/TestFineGrainedFSNamesystemLock.java delete mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesLogsExtend.java delete mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/reader/ContainerLogsInfoMessageBodyReader.java delete mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/reader/RemoteLogPathsMessageBodyReader.java create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/InputStreamCallbacksImpl.java create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/AbstractObjectInputStreamFactory.java rename hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/{S3ASeekableStream.java => impl/streams/AnalyticsStream.java} (76%) create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/AnalyticsStreamFactory.java create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/ClassicObjectInputStreamFactory.java copy hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/{auth/delegation/DelegationBindingInfo.java => impl/streams/FactoryBindingParameters.java} (55%) create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/InputStreamType.java create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/ObjectInputStream.java create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/ObjectInputStreamCallbacks.java create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/ObjectInputStreamFactory.java create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/ObjectReadParameters.java create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/StreamFactoryRequirements.java create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams/StreamIntegration.java copy {hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch => hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/streams}/package-info.java (76%) create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/prefetch/PrefetchOptions.java create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/prefetch/PrefetchingInputStreamFactory.java create mode 100644 hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/reading.md create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/streams/TestStreamFactories.java copy {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/webapp => hadoop-tools/hadoop-resourceestimator/src/test/java/org/apache/hadoop/resourceestimator/service}/GuiceServletConfig.java (89%) delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/reader/TimelineDomainReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/reader/TimelineEntitiesReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/reader/TimelinePutResponseReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/reader/package-info.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineDomainWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineDomainsWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineEntitiesWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineEntityWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelineEventsWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/TimelinePutResponseWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/writer/package-info.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/reader/TimelineDomainReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/reader/TimelineEntitiesReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/reader/TimelineEntityReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/reader/package-info.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineDomainWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineEntitiesWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineEntitySetWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineEntityWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/TimelineHealthWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/writer/package-info.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/ConflictException.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/ContainerLogsInfoListReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineAboutReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineDomainReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineDomainsReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineEntitiesReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineEntityReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelineEventsReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/reader/TimelinePutResponseReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedDefaultLabel.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedDynamicConfigAbsoluteMode.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedDynamicConfigWeightMode.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedDynamicConfigWeightModeDQC.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedLegacyQueueCreationAbsoluteMode.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentage.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentageAndWeight.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentageAndWeightVector.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentageVector.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndWeight.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndWeightVector.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModePercentageAndWeight.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCapacitySchedulerMixedModePercentageAndWeightVector.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/AppStateReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/ApplicationSubmissionContextInfoReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/LabelsToNodesInfoReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/NodeLabelsInfoReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/NodeToLabelsInfoReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/reader/ResourceOptionInfoReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/writer/ApplicationSubmissionContextInfoWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/writer/ResourceOptionInfoWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/writer/SchedConfUpdateInfoWriter.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineAboutReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineEntityReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineEntitySetReader.java delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineHealthReader.java --------------------------------------------------------------------- To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org