This is an automated email from the ASF dual-hosted git repository.
keepromise pushed a change to branch HDFS-17531
in repository https://gitbox.apache.org/repos/asf/hadoop.git
omit b47e11e67db HDFS-17715. [ARR] Add documentation for asynchronous
router. (#7318). Contributed by Jian Zhang.
omit 15cc381414d HDFS-17651.[ARR] Async handler executor isolation (#7244).
Contributed by hfutatzhanghb.
omit 35690cfc8cc HDFS-17650. [ARR] The router server-side rpc protocol PB
supports asynchrony. (#7139). Contributed by hfutatzhanghb.
omit d7779c22f7a HDFS-17640.[ARR] RouterClientProtocol supports
asynchronous rpc. (#7188)
omit 2df1e2a57ca HADOOP-19361. RPC DeferredMetrics bugfix. (#7220).
Contributed by hfutatzhanghb.
omit bb014e64a7a HDFS-17672. [ARR] Move asynchronous related classes to the
async package. (#7184). Contributed by Jian Zhang.
omit 2fe85fbca41 HDFS-17659. [ARR]Router Quota supports asynchronous rpc.
(#7157). Contributed by hfutatzhanghb.
omit 5f92a3df801 HDFS-17656. [ARR] RouterNamenodeProtocol and
RouterUserProtocol supports asynchronous rpc. (#7159). Contributed by Jian
Zhang.
omit d076822c47e HDFS-17596. [ARR] RouterStoragePolicy supports
asynchronous rpc. (#6988). Contributed by hfutatzhanghb.
omit 720e000e3ce HDFS-17601. [ARR] RouterRpcServer supports asynchronous
rpc. (#7108). Contributed by hfutatzhanghb.
omit 27847675cec HDFS-17595. [ARR] ErasureCoding supports asynchronous rpc.
(#6983). Contributed by hfutatzhanghb.
omit 3083d443449 HDFS-17597. [ARR] RouterSnapshot supports asynchronous
rpc. (#6994). Contributed by Wenqi Li.
omit abb9346a69a HDFS-17594. [ARR] RouterCacheAdmin supports asynchronous
rpc. (#6986). Contributed by Archie73.
omit fd64eb4f419 HDFS-17545. [ARR] router async rpc client. (#6871).
Contributed by Jian Zhang.
omit 1e6b302f858 HDFS-17544. [ARR] The router client rpc protocol PB
supports asynchrony. (#6870). Contributed by Jian Zhang.
omit 846b113df5a HADOOP-19235. IPC client uses CompletableFuture to support
asynchronous operations. (#6888)
omit c12cbea7760 HDFS-17543. [ARR] AsyncUtil makes asynchronous code more
concise and easier. (#6868). Contributed by Jian Zhang.
add 9bf5e38c483 YARN-11745: Fix TimSort contract violation in
PriorityQueueComparator Class (#7278)
add 0d72896db16 YARN-11753. Ensure NM is marked unhealthy if the
ProcessBuilder reports an issue with the container-executor (#7290)
add 0432761ac84 HDFS-17657. The balancer service supports httpserver.
(#7242) Contribtued by Zhaobo Huang.
add e41f2ba1552 HADOOP-19392: Upgrade ftpserver to resolve maven/pom issue
(#7311) Contributed by YanivKunda.
add 8afbfbc2f57 HDFS-17394. [FGL] Remove unused WriteHoldCount of
FSNamesystemLock (#6571)
add 9b45c66e34f HDFS-17387. [FGL] Abstract the configuration locking mode
(#6572)
add 16c25b277da HDFS-17398. [FGL] Implement the FGL lock for
FSNLockManager (#6599)
add f0368bb2372 HDFS-17405. [FGL] Using different metric name to trace
performance for FGL and Global lock (#6600)
add a1ab911896f HDFS-17390. [FGL] FSDirectory supports this fine-grained
locking (#6573)
add 7158b9ee194 HDFS-17395. [FGL] Use FSLock to protect
ErasureCodingPolicy related operations (#6579)
add d39f034f982 HDFS-17413. [FGL] CacheReplicationMonitor supports
fine-grained lock (#6641)
add 7764b39e4ab HDFS-17414. [FGL] RPCs in DatanodeProtocol support
fine-grained lock (#6649)
add ac0605db419 HDFS-17416. [FGL] Monitor threads in BlockManager.class
support fine-grained lock (#6647)
add 3b5e28a87ec HDFS-17423. [FGL] BlockManagerSafeMode supports
fine-grained lock (#6645)
add 778e819b41a HDFS-17410. [FGL] Client RPCs that changes file attributes
supports fine-grained lock (#6634)
add 7d37995f46e HDFS-17415. [FGL] RPCs in NamenodeProtocol support
fine-grained lock (#6654)
add 40d54ebb66a HDFS-17417. [FGL] HeartbeatManager and
DatanodeAdminMonitor support fine-grained locking (#6656)
add 03603efd74b HDFS-17412. [FGL] Client RPCs involving maintenance
supports fine-grained lock (#6667)
add ed302ae3aad HDFS-17420. [FGL] EditLogTailer and FSEditLogLoader
support FGL (#6679)
add 7577f3ad619 HDFS-17388. [FGL] Client RPCs involving write process
supports fine-grained lock (#6589)
add 4c86601cba5 HDFS-17389. [FGL] Client RPCs involving read process
supports fine-grained lock (#6590)
add 6d888d599f4 HDFS-17411. [FGL] Client RPCs involving snapshot support
fine-grained lock (#6714)
add 02b149da67e HDFS-17445. [FGL] some operations support fine-grained
locking (#6715)
add e4885b02efc HDFS-17424. [FGL] DelegationTokenSecretManager supports
fine-grained lock (#6696)
add 0d626f516f4 HDFS-17472. [FGL] gcDeletedSnapshot and getDelegationToken
support FGL (#6743)
add e5b1c09b8a0 HDFS-17459. [FGL] Add documentation (#6737)
add be0267313bd HDFS-17457. [FGL] UTs support fine-grained locking (#6741)
add e8807726b8d HDFS-17506. [FGL] Performance for phase 1
add b289f9abd33 HDFS-17691. [FGL] Move FSNamesystemLockMode to
org.apache.hadoop.hdfs.util package (#7232)
add 16552b6fbdc HDFS-17692. [FGL] fix bug in getContentSummary method
(#7233)
add 8ab53982a56 HDFS-17697. [FGL] hasWriteLock and hasReadLock in
FineGrainedFSNamesystemLock shouldn't throw assert error (#7250)
add 2343bd71a93 HDFS-17701. [FGL] Fix some javadocs (#7256)
add 7ce4ea57807 HDFS-17711: Change fsimage loading progress percentage
discontinuous to continuous
add 053afb7063f HDFS-17704. Fix TestDecommission and
TestDecommissionWithBackoffMonitor often run timeout. (#7266). Contributed by
hfutatzhanghb.
add 4ad3f1f579a HADOOP-19389: Optimize shell -text command I/O with
multi-byte read. (#7291)
add 6cc743eac6f HADOOP-19397. Update LICENSE-binary with jersey 2 details
(#7315). Contributed by PJ Fanning
add 0698e8a2c40 HADOOP-19236. Incorporate VolcanoEngine Cloud TOS File
System Implementation. (#7294)
add 6370cd268c6 Revert "HADOOP-19236. Incorporate VolcanoEngine Cloud TOS
File System Impleme…" (#7327)
add b6916ebe905 HADOOP-19401: Improve error message when OS can't identify
the current user.
add d9bcee929e7 HADOOP-19289. Upgrade to protobuf-java 3.25.5 (#7072)
add 91535fa7b7d YARN-11759: Fix log statement in
RMAppImpl#processNodeUpdate
add a5926668682 HADOOP-19225. Upgrade Jetty to 9.4.57.v20241219 due to
CVE-2024-8184 (#7116). Contributed by PJ Fanning.
add c7e1b660879 HADOOP-19232: [ABFS][FNSOverBlob] Implementing Ingress
Support with various Fallback Handling (#7272)
add 1ba30d6ca63 YARN-11754. [JDK17] Fix SpotBugs Issues in YARN. (#7317)
Contributed by Shilun Fan.
add 44a5cba78ac HADOOP-19407: Update 3.4.1 docs landing page to highlight
changes shipped in the release.
add 741bdd636b4 HDFS-17721. RBF: Allow routers to declare IP for admin
addr (#7342) Contributed by Felix Nguyen.
add 6d20de1ceda HADOOP-19233: ABFS: [FnsOverBlob] Implementing Rename and
Delete APIs over Blob Endpoint (#7265)
add 7459a1f215a HADOOP-19405. hadoop-aws and hadoop-azure tests have
stopped running. (#7335)
add eae63ec1827 HADOOP-19443: [ABFS][FnsOverBlob] Updating hadoop-azure
Test Scripts and Adding Tests For Metadata APIs (#7344)
add 950b3eb431d MAPREDUCE-7497. mapreduce tests have stopped running.
(#7343)
add f1ee35b8a18 HADOOP-19415. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-common Part1. (#7339)
add f900339a210 HADOOP-19404: [ABFS][FNSOverBlob] Update documentation for
FNS Blob Onboard (#7341)
add b5cbca37dfc HADOOP-19415. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-common Part2. (#7347)
add 872ebda81d4 MAPREDUCE-7415. [JDK17] Upgrade Junit 4 to 5 in
hadoop-mapreduce-client-nativetask. (#7349)
add 57d0979d383 HADOOP-19311: [ABFS] Implement Backoff and Read Footer
metrics using IOStatistics Class (#7122)
add acb7a800601 MAPREDUCE-7414. [JDK17] Upgrade Junit 4 to 5 in
hadoop-mapreduce-client-hs. (#7354)
add 851b4c38a54 MAPREDUCE-7416. [JDK17] Upgrade Junit 4 to 5 in
hadoop-mapreduce-client-shuffle. (#7351)
add d552670b86d MAPREDUCE-7418. [JDK17] Upgrade Junit 4 to 5 in
hadoop-mapreduce-client-app. (#7350)
add 73ac0b9e39c HADOOP-19448: [ABFS][FNSOverBlob][Optimizations] Reduce
Network Calls In Create and Mkdir Flow (#7353)
add cfe76e45cb6 HADOOP-19415. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-common Part3. (#7355)
add 755fa7482bf HDFS-17724: Set recover.lease.on.close.exception as an
instance member in the DfsClientConf.java
add 2550344f4f0 HADOOP-19415. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-common Part4. (#7356)
add 81146fed2f8 HADOOP-19415. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-common Part5. (#7357)
add c5fdc3ba6c2 HADOOP-19431. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-distcp. (#7368)
add 8424c155f11 MAPREDUCE-7420. [JDK17] Upgrade Junit 4 to 5 in
hadoop-mapreduce-client-core Part1. (#7363)
add d7a0bfd3234 HDFS-17543. [ARR] AsyncUtil makes asynchronous code more
concise and easier. (#6868). Contributed by Jian Zhang.
add 4cbadefd054 HADOOP-19235. IPC client uses CompletableFuture to support
asynchronous operations. (#6888)
add 58b02672925 HDFS-17544. [ARR] The router client rpc protocol PB
supports asynchrony. (#6870). Contributed by Jian Zhang.
add 4a9e44cdf95 HDFS-17545. [ARR] router async rpc client. (#6871).
Contributed by Jian Zhang.
add dd1192f0478 HDFS-17594. [ARR] RouterCacheAdmin supports asynchronous
rpc. (#6986). Contributed by Archie73.
add 7be68eb61b3 HDFS-17597. [ARR] RouterSnapshot supports asynchronous
rpc. (#6994). Contributed by Wenqi Li.
add 24baed0ecbb HDFS-17595. [ARR] ErasureCoding supports asynchronous rpc.
(#6983). Contributed by hfutatzhanghb.
add 4bf866a18ed HDFS-17601. [ARR] RouterRpcServer supports asynchronous
rpc. (#7108). Contributed by hfutatzhanghb.
add 1d7471f4def HDFS-17596. [ARR] RouterStoragePolicy supports
asynchronous rpc. (#6988). Contributed by hfutatzhanghb.
add 6f038f3e57f HDFS-17656. [ARR] RouterNamenodeProtocol and
RouterUserProtocol supports asynchronous rpc. (#7159). Contributed by Jian
Zhang.
add 2ef69623685 HDFS-17659. [ARR]Router Quota supports asynchronous rpc.
(#7157). Contributed by hfutatzhanghb.
add 824224270ce HDFS-17672. [ARR] Move asynchronous related classes to the
async package. (#7184). Contributed by Jian Zhang.
add 240efd49caa HADOOP-19361. RPC DeferredMetrics bugfix. (#7220).
Contributed by hfutatzhanghb.
add e350b83942d HDFS-17640.[ARR] RouterClientProtocol supports
asynchronous rpc. (#7188)
add 5b7bc50dcda HDFS-17650. [ARR] The router server-side rpc protocol PB
supports asynchrony. (#7139). Contributed by hfutatzhanghb.
add 58b13914f55 HDFS-17651.[ARR] Async handler executor isolation (#7244).
Contributed by hfutatzhanghb.
add 82bccee6538 HDFS-17715. [ARR] Add documentation for asynchronous
router. (#7318). Contributed by Jian Zhang.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b47e11e67db)
\
N -- N -- N refs/heads/HDFS-17531 (82bccee6538)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
LICENSE-binary | 58 +-
hadoop-common-project/hadoop-common/pom.xml | 25 +
.../java/org/apache/hadoop/fs/shell/Display.java | 167 +-
.../hadoop/security/UGIExceptionMessages.java | 1 +
.../hadoop/security/UserGroupInformation.java | 6 +
.../java/org/apache/hadoop/cli/CLITestHelper.java | 19 +-
.../test/java/org/apache/hadoop/cli/TestCLI.java | 10 +-
.../org/apache/hadoop/conf/TestConfServlet.java | 26 +-
.../org/apache/hadoop/conf/TestConfigRedactor.java | 13 +-
.../org/apache/hadoop/conf/TestConfiguration.java | 138 +-
.../hadoop/conf/TestConfigurationDeprecation.java | 41 +-
.../hadoop/conf/TestConfigurationFieldsBase.java | 42 +-
.../hadoop/conf/TestConfigurationSubclass.java | 14 +-
.../org/apache/hadoop/conf/TestDeprecatedKeys.java | 13 +-
.../org/apache/hadoop/conf/TestGetInstances.java | 6 +-
.../apache/hadoop/conf/TestReconfiguration.java | 132 +-
.../org/apache/hadoop/conf/TestStorageUnit.java | 234 ++-
.../hadoop/crypto/CryptoStreamsTestBase.java | 187 +-
.../org/apache/hadoop/crypto/TestCryptoCodec.java | 72 +-
.../crypto/TestCryptoOutputStreamClosing.java | 12 +-
.../apache/hadoop/crypto/TestCryptoStreams.java | 14 +-
.../hadoop/crypto/TestCryptoStreamsForLocalFS.java | 65 +-
.../hadoop/crypto/TestCryptoStreamsNormal.java | 70 +-
.../TestCryptoStreamsWithJceAesCtrCryptoCodec.java | 4 +-
.../TestCryptoStreamsWithJceSm4CtrCryptoCodec.java | 4 +-
...tCryptoStreamsWithOpensslAesCtrCryptoCodec.java | 23 +-
...tCryptoStreamsWithOpensslSm4CtrCryptoCodec.java | 32 +-
.../org/apache/hadoop/crypto/TestCryptoUtils.java | 36 +-
.../apache/hadoop/crypto/TestOpensslCipher.java | 60 +-
.../hadoop/crypto/key/TestCachingKeyProvider.java | 163 +-
.../apache/hadoop/crypto/key/TestKeyProvider.java | 36 +-
.../crypto/key/TestKeyProviderCryptoExtension.java | 149 +-
.../TestKeyProviderDelegationTokenExtension.java | 24 +-
.../hadoop/crypto/key/TestKeyProviderFactory.java | 98 +-
.../org/apache/hadoop/crypto/key/TestKeyShell.java | 20 +-
.../apache/hadoop/crypto/key/TestValueQueue.java | 132 +-
.../crypto/key/kms/TestKMSClientProvider.java | 20 +-
.../kms/TestLoadBalancingKMSClientProvider.java | 27 +-
.../crypto/random/TestOpensslSecureRandom.java | 18 +-
.../hadoop/crypto/random/TestOsSecureRandom.java | 26 +-
.../org/apache/hadoop/fs/TestAfsCheckPath.java | 14 +-
.../java/org/apache/hadoop/fs/TestAvroFSInput.java | 4 +-
.../org/apache/hadoop/fs/TestBlockLocation.java | 15 +-
.../apache/hadoop/fs/TestChecksumFileSystem.java | 30 +-
.../org/apache/hadoop/fs/TestCommandFormat.java | 9 +-
.../org/apache/hadoop/fs/TestContentSummary.java | 63 +-
.../hadoop/fs/TestDFCachingGetSpaceUsed.java | 12 +-
.../org/apache/hadoop/fs/TestDFVariations.java | 51 +-
.../src/test/java/org/apache/hadoop/fs/TestDU.java | 35 +-
.../java/org/apache/hadoop/fs/TestDefaultUri.java | 2 +-
.../apache/hadoop/fs/TestDelegateToFileSystem.java | 7 +-
.../hadoop/fs/TestDelegateToFsCheckPath.java | 2 +-
.../hadoop/fs/TestDelegationTokenRenewer.java | 31 +-
.../java/org/apache/hadoop/fs/TestFileContext.java | 24 +-
.../hadoop/fs/TestFileContextDeleteOnExit.java | 30 +-
.../hadoop/fs/TestFileContextResolveAfs.java | 15 +-
.../java/org/apache/hadoop/fs/TestFileStatus.java | 9 +-
.../hadoop/fs/TestFileSystemCanonicalization.java | 12 +-
.../hadoop/fs/TestFileSystemInitialization.java | 5 +-
.../hadoop/fs/TestFileSystemStorageStatistics.java | 20 +-
.../org/apache/hadoop/fs/TestFileSystemTokens.java | 13 +-
.../org/apache/hadoop/fs/TestFilterFileSystem.java | 31 +-
.../java/org/apache/hadoop/fs/TestFilterFs.java | 2 +-
.../java/org/apache/hadoop/fs/TestFsOptions.java | 4 +-
.../java/org/apache/hadoop/fs/TestFsShell.java | 27 +-
.../java/org/apache/hadoop/fs/TestFsShellCopy.java | 18 +-
.../java/org/apache/hadoop/fs/TestFsShellList.java | 28 +-
.../apache/hadoop/fs/TestFsShellReturnCode.java | 62 +-
.../org/apache/hadoop/fs/TestFsShellTouch.java | 10 +-
.../apache/hadoop/fs/TestFsUrlConnectionPath.java | 21 +-
.../hadoop/fs/TestGetFileBlockLocations.java | 13 +-
.../org/apache/hadoop/fs/TestGetSpaceUsed.java | 15 +-
.../org/apache/hadoop/fs/TestGlobExpander.java | 12 +-
.../java/org/apache/hadoop/fs/TestGlobPattern.java | 13 +-
.../org/apache/hadoop/fs/TestHarFileSystem.java | 6 +-
.../apache/hadoop/fs/TestHarFileSystemBasics.java | 58 +-
.../java/org/apache/hadoop/fs/TestHardLink.java | 18 +-
.../java/org/apache/hadoop/fs/TestListFiles.java | 22 +-
.../apache/hadoop/fs/TestLocalDirAllocator.java | 205 +-
.../org/apache/hadoop/fs/TestLocalFileSystem.java | 99 +-
.../hadoop/fs/TestLocalFileSystemPermission.java | 21 +-
.../apache/hadoop/fs/TestLocatedFileStatus.java | 2 +-
.../java/org/apache/hadoop/fs/TestQuotaUsage.java | 34 +-
.../test/java/org/apache/hadoop/fs/TestTrash.java | 145 +-
.../apache/hadoop/fs/TestTruncatedInputBug.java | 4 +-
.../apache/hadoop/fs/ftp/TestFTPFileSystem.java | 36 +-
.../apache/hadoop/fs/http/TestHttpFileSystem.java | 8 +-
.../org/apache/hadoop/fs/permission/TestAcl.java | 10 +-
.../hadoop/fs/permission/TestFsPermission.java | 10 +-
.../hadoop/fs/protocolPB/TestFSSerialization.java | 4 +-
.../apache/hadoop/fs/sftp/TestSFTPFileSystem.java | 112 +-
.../apache/hadoop/fs/shell/TestAclCommands.java | 81 +-
.../apache/hadoop/fs/shell/TestCommandFactory.java | 11 +-
.../java/org/apache/hadoop/fs/shell/TestCopy.java | 26 +-
.../apache/hadoop/fs/shell/TestCopyFromLocal.java | 36 +-
.../hadoop/fs/shell/TestCopyPreserveFlag.java | 69 +-
.../apache/hadoop/fs/shell/TestCopyToLocal.java | 47 +-
.../java/org/apache/hadoop/fs/shell/TestCount.java | 34 +-
.../org/apache/hadoop/fs/shell/TestCpCommand.java | 47 +-
.../java/org/apache/hadoop/fs/shell/TestLs.java | 74 +-
.../java/org/apache/hadoop/fs/shell/TestMove.java | 28 +-
.../org/apache/hadoop/fs/shell/TestPathData.java | 65 +-
.../apache/hadoop/fs/shell/TestPathExceptions.java | 6 +-
.../hadoop/fs/shell/TestPrintableString.java | 2 +-
.../java/org/apache/hadoop/fs/shell/TestTail.java | 4 +-
.../apache/hadoop/fs/shell/TestTextCommand.java | 580 ++++--
.../apache/hadoop/fs/shell/TestXAttrCommands.java | 42 +-
.../org/apache/hadoop/fs/shell/find/TestAnd.java | 17 +-
.../hadoop/fs/shell/find/TestFilterExpression.java | 25 +-
.../org/apache/hadoop/fs/shell/find/TestFind.java | 29 +-
.../org/apache/hadoop/fs/shell/find/TestIname.java | 16 +-
.../org/apache/hadoop/fs/shell/find/TestName.java | 16 +-
.../org/apache/hadoop/fs/shell/find/TestPrint.java | 20 +-
.../apache/hadoop/fs/shell/find/TestPrint0.java | 21 +-
.../apache/hadoop/fs/shell/find/TestResult.java | 4 +-
.../org/apache/hadoop/fs/store/TestDataBlocks.java | 21 +-
.../apache/hadoop/fs/store/TestEtagChecksum.java | 6 +-
.../apache/hadoop/fs/viewfs/TestChRootedFs.java | 155 +-
.../TestRegexMountPointInterceptorFactory.java | 9 +-
...ountPointResolvedDstPathReplaceInterceptor.java | 21 +-
...iewFSOverloadSchemeCentralMountTableConfig.java | 4 +-
.../fs/viewfs/TestViewFileSystemDelegation.java | 9 +-
.../TestViewFileSystemDelegationTokenSupport.java | 8 +-
...iewFileSystemOverloadSchemeLocalFileSystem.java | 43 +-
.../apache/hadoop/fs/viewfs/TestViewFsConfig.java | 54 +-
.../apache/hadoop/fs/viewfs/TestViewFsTrash.java | 11 +-
.../apache/hadoop/fs/viewfs/TestViewFsURIs.java | 2 +-
.../hadoop/fs/viewfs/TestViewfsFileStatus.java | 24 +-
.../org/apache/hadoop/ha/DummySharedResource.java | 4 +-
.../java/org/apache/hadoop/ha/MiniZKFCCluster.java | 4 +-
.../apache/hadoop/ha/TestActiveStandbyElector.java | 116 +-
.../apache/hadoop/ha/TestFailoverController.java | 7 +-
.../java/org/apache/hadoop/ha/TestHAAdmin.java | 9 +-
.../org/apache/hadoop/ha/TestHealthMonitor.java | 19 +-
.../java/org/apache/hadoop/ha/TestNodeFencer.java | 20 +-
.../apache/hadoop/ha/TestShellCommandFencer.java | 83 +-
.../apache/hadoop/ha/TestSshFenceByTcpPort.java | 18 +-
.../hadoop/http/HttpServerFunctionalTest.java | 8 +-
.../http/TestAuthenticationSessionCookie.java | 22 +-
.../hadoop/http/TestDisabledProfileServlet.java | 26 +-
.../org/apache/hadoop/http/TestGlobalFilter.java | 2 +-
.../org/apache/hadoop/http/TestHtmlQuoting.java | 22 +-
.../org/apache/hadoop/http/TestHttpCookieFlag.java | 27 +-
.../org/apache/hadoop/http/TestHttpRequestLog.java | 17 +-
.../org/apache/hadoop/http/TestHttpServer.java | 51 +-
.../hadoop/http/TestHttpServerLifecycle.java | 12 +-
.../org/apache/hadoop/http/TestHttpServerLogs.java | 10 +-
.../apache/hadoop/http/TestHttpServerWebapps.java | 2 +-
.../hadoop/http/TestHttpServerWithSpnego.java | 29 +-
.../apache/hadoop/http/TestIsActiveServlet.java | 8 +-
.../org/apache/hadoop/http/TestProfileServlet.java | 10 +-
.../org/apache/hadoop/http/TestSSLHttpServer.java | 17 +-
.../hadoop/http/TestSSLHttpServerConfigs.java | 52 +-
.../org/apache/hadoop/http/TestServletFilter.java | 2 +-
.../hadoop/http/lib/TestStaticUserWebFilter.java | 4 +-
.../org/apache/hadoop/ipc/MiniRPCBenchmark.java | 5 +-
.../java/org/apache/hadoop/ipc/TestAsyncIPC.java | 72 +-
.../apache/hadoop/ipc/TestCallQueueManager.java | 35 +-
.../org/apache/hadoop/ipc/TestCallerContext.java | 41 +-
.../apache/hadoop/ipc/TestDecayRpcScheduler.java | 38 +-
.../org/apache/hadoop/ipc/TestFairCallQueue.java | 36 +-
.../test/java/org/apache/hadoop/ipc/TestIPC.java | 318 ++--
.../apache/hadoop/ipc/TestIPCServerResponder.java | 28 +-
.../apache/hadoop/ipc/TestIdentityProviders.java | 6 +-
.../apache/hadoop/ipc/TestMiniRPCBenchmark.java | 2 +-
.../hadoop/ipc/TestMultipleProtocolServer.java | 10 +-
.../apache/hadoop/ipc/TestProcessingDetails.java | 4 +-
.../hadoop/ipc/TestProtoBufRPCCompatibility.java | 27 +-
.../org/apache/hadoop/ipc/TestProtoBufRpc.java | 77 +-
.../hadoop/ipc/TestProtoBufRpcServerHandoff.java | 23 +-
.../test/java/org/apache/hadoop/ipc/TestRPC.java | 186 +-
.../apache/hadoop/ipc/TestRPCCallBenchmark.java | 8 +-
.../apache/hadoop/ipc/TestRPCCompatibility.java | 14 +-
.../apache/hadoop/ipc/TestRPCServerShutdown.java | 22 +-
.../org/apache/hadoop/ipc/TestRPCWaitForProxy.java | 22 +-
.../org/apache/hadoop/ipc/TestResponseBuffer.java | 4 +-
.../java/org/apache/hadoop/ipc/TestRetryCache.java | 25 +-
.../apache/hadoop/ipc/TestRetryCacheMetrics.java | 5 +-
.../apache/hadoop/ipc/TestReuseRpcConnections.java | 21 +-
.../java/org/apache/hadoop/ipc/TestRpcBase.java | 8 +-
.../apache/hadoop/ipc/TestRpcServerHandoff.java | 24 +-
.../org/apache/hadoop/ipc/TestRpcWritable.java | 60 +-
.../java/org/apache/hadoop/ipc/TestSaslRPC.java | 207 +-
.../java/org/apache/hadoop/ipc/TestServer.java | 21 +-
.../org/apache/hadoop/ipc/TestSocketFactory.java | 24 +-
.../ipc/TestWeightedRoundRobinMultiplexer.java | 30 +-
.../hadoop/ipc/TestWeightedTimeCostProvider.java | 15 +-
.../TestDecayRpcSchedulerDetailedMetrics.java | 6 +-
.../apache/hadoop/ipc/metrics/TestRpcMetrics.java | 6 +-
.../org/apache/hadoop/jmx/TestJMXJsonServlet.java | 18 +-
.../hadoop/jmx/TestJMXJsonServletNaNFiltered.java | 14 +-
.../apache/hadoop/log/TestLogThrottlingHelper.java | 23 +-
.../hadoop/metrics2/filter/TestPatternFilter.java | 34 +-
.../apache/hadoop/metrics2/impl/ConfigUtil.java | 11 +-
.../hadoop/metrics2/impl/TestGangliaMetrics.java | 11 +-
.../metrics2/impl/TestMetricsCollectorImpl.java | 21 +-
.../hadoop/metrics2/impl/TestMetricsConfig.java | 35 +-
.../metrics2/impl/TestMetricsSourceAdapter.java | 13 +-
.../metrics2/impl/TestMetricsSystemImpl.java | 78 +-
.../hadoop/metrics2/impl/TestMetricsVisitor.java | 39 +-
.../apache/hadoop/metrics2/impl/TestSinkQueue.java | 81 +-
.../apache/hadoop/metrics2/lib/TestInterns.java | 25 +-
.../metrics2/lib/TestMetricsAnnotations.java | 35 +-
.../hadoop/metrics2/lib/TestMetricsRegistry.java | 30 +-
.../hadoop/metrics2/lib/TestMutableMetrics.java | 34 +-
.../metrics2/lib/TestMutableRollingAverages.java | 24 +-
.../apache/hadoop/metrics2/lib/TestUniqNames.java | 4 +-
.../apache/hadoop/metrics2/sink/TestFileSink.java | 12 +-
.../hadoop/metrics2/sink/TestGraphiteMetrics.java | 4 +-
.../metrics2/sink/TestPrometheusMetricsSink.java | 57 +-
.../metrics2/sink/TestRollingFileSystemSink.java | 87 +-
.../hadoop/metrics2/sink/TestStatsDMetrics.java | 24 +-
.../metrics2/sink/ganglia/TestGangliaSink.java | 28 +-
.../hadoop/metrics2/source/TestJvmMetrics.java | 50 +-
.../apache/hadoop/metrics2/util/TestMBeans.java | 13 +-
.../hadoop/metrics2/util/TestMetricsCache.java | 45 +-
.../hadoop/metrics2/util/TestSampleQuantiles.java | 6 +-
.../hadoop/metrics2/util/TestSampleStat.java | 52 +-
.../org/apache/hadoop/net/TestClusterTopology.java | 66 +-
.../test/java/org/apache/hadoop/net/TestDNS.java | 39 +-
.../hadoop/net/TestDNSDomainNameResolver.java | 13 +-
.../hadoop/net/TestMockDomainNameResolver.java | 10 +-
.../java/org/apache/hadoop/net/TestNetUtils.java | 54 +-
.../net/TestNetworkTopologyWithNodeGroup.java | 18 +-
.../apache/hadoop/net/TestScriptBasedMapping.java | 21 +-
.../net/TestScriptBasedMappingWithDependency.java | 23 +-
.../apache/hadoop/net/TestSocketIOWithTimeout.java | 8 +-
.../org/apache/hadoop/net/TestStaticMapping.java | 33 +-
.../org/apache/hadoop/net/TestSwitchMapping.java | 32 +-
.../org/apache/hadoop/net/TestTableMapping.java | 8 +-
.../apache/hadoop/net/unix/TestDomainSocket.java | 117 +-
.../hadoop/net/unix/TestDomainSocketWatcher.java | 36 +-
.../org/apache/hadoop/oncrpc/TestFrameDecoder.java | 19 +-
.../apache/hadoop/oncrpc/TestRpcAcceptedReply.java | 10 +-
.../java/org/apache/hadoop/oncrpc/TestRpcCall.java | 22 +-
.../org/apache/hadoop/oncrpc/TestRpcCallCache.java | 30 +-
.../apache/hadoop/oncrpc/TestRpcDeniedReply.java | 23 +-
.../org/apache/hadoop/oncrpc/TestRpcMessage.java | 18 +-
.../org/apache/hadoop/oncrpc/TestRpcReply.java | 20 +-
.../java/org/apache/hadoop/oncrpc/TestXDR.java | 10 +-
.../hadoop/oncrpc/security/TestCredentialsSys.java | 4 +-
.../hadoop/oncrpc/security/TestRpcAuthInfo.java | 10 +-
.../org/apache/hadoop/portmap/TestPortmap.java | 29 +-
.../hadoop/security/ManualTestKeytabLogins.java | 2 +-
.../hadoop/security/TestAuthenticationFilter.java | 6 +-
.../hadoop/security/TestCompositeGroupMapping.java | 4 +-
.../apache/hadoop/security/TestCredentials.java | 90 +-
.../hadoop/security/TestDoAsEffectiveUser.java | 39 +-
.../apache/hadoop/security/TestGroupFallback.java | 4 +-
.../apache/hadoop/security/TestGroupsCaching.java | 32 +-
.../TestHttpCrossOriginFilterInitializer.java | 15 +-
.../security/TestIngressPortBasedResolver.java | 5 +-
.../hadoop/security/TestJNIGroupsMapping.java | 8 +-
.../java/org/apache/hadoop/security/TestKDiag.java | 34 +-
.../org/apache/hadoop/security/TestKDiagNoKDC.java | 23 +-
.../hadoop/security/TestLdapGroupsMapping.java | 46 +-
.../hadoop/security/TestLdapGroupsMappingBase.java | 6 +-
.../TestLdapGroupsMappingWithBindUserSwitch.java | 10 +-
.../TestLdapGroupsMappingWithFailover.java | 4 +-
.../TestLdapGroupsMappingWithOneQuery.java | 18 +-
.../TestLdapGroupsMappingWithPosixGroup.java | 13 +-
.../apache/hadoop/security/TestNetgroupCache.java | 10 +-
.../hadoop/security/TestNullGroupsMapping.java | 17 +-
.../hadoop/security/TestProxyUserFromEnv.java | 6 +-
.../security/TestRuleBasedLdapGroupsMapping.java | 29 +-
.../apache/hadoop/security/TestSecurityUtil.java | 18 +-
.../hadoop/security/TestShellBasedIdMapping.java | 6 +-
.../security/TestShellBasedUnixGroupsMapping.java | 77 +-
.../hadoop/security/TestUGILoginFromKeytab.java | 176 +-
.../hadoop/security/TestUGIWithExternalKdc.java | 22 +-
.../apache/hadoop/security/TestUGIWithMiniKdc.java | 10 +-
.../apache/hadoop/security/TestUserFromEnv.java | 7 +-
.../security/TestWhitelistBasedResolver.java | 4 +-
.../hadoop/security/alias/TestCredShell.java | 40 +-
.../security/alias/TestCredentialProvider.java | 6 +-
.../alias/TestCredentialProviderFactory.java | 50 +-
.../server/TestProxyUserAuthenticationFilter.java | 6 +-
.../security/authorize/TestAccessControlList.java | 23 +-
.../TestDefaultImpersonationProvider.java | 10 +-
.../security/authorize/TestProxyServers.java | 6 +-
.../hadoop/security/authorize/TestProxyUsers.java | 82 +-
.../authorize/TestServiceAuthorization.java | 6 +-
.../security/http/TestCrossOriginFilter.java | 177 +-
.../http/TestRestCsrfPreventionFilter.java | 2 +-
.../security/http/TestXFrameOptionsFilter.java | 30 +-
.../ssl/TestDelegatingSSLSocketFactory.java | 2 +-
.../security/ssl/TestReloadingX509KeyManager.java | 337 ++--
.../ssl/TestReloadingX509TrustManager.java | 46 +-
.../apache/hadoop/security/ssl/TestSSLFactory.java | 110 +-
.../hadoop/security/token/TestDtUtilShell.java | 178 +-
.../apache/hadoop/security/token/TestToken.java | 4 +-
.../token/delegation/TestDelegationToken.java | 50 +-
.../delegation/web/TestDelegationTokenManager.java | 29 +-
.../delegation/web/TestWebDelegationToken.java | 189 +-
.../hadoop/service/TestCompositeService.java | 248 +--
.../hadoop/service/TestServiceOperations.java | 8 +-
.../launcher/AbstractServiceLauncherTestBase.java | 36 +-
.../hadoop/service/launcher/TestServiceConf.java | 13 +-
.../launcher/TestServiceInterruptHandling.java | 14 +-
.../service/launcher/TestServiceLauncher.java | 2 +-
.../TestServiceLauncherCreationFailures.java | 2 +-
.../launcher/TestServiceLauncherInnerMethods.java | 6 +-
.../InitInConstructorLaunchableService.java | 14 +-
.../testservices/LaunchableRunningService.java | 4 +-
.../apache/hadoop/test/TestGenericTestUtils.java | 15 +-
.../org/apache/hadoop/test/TestJUnitSetup.java | 7 +-
.../apache/hadoop/test/TestLambdaTestUtils.java | 18 +-
.../hadoop/test/TestMultithreadedTestUtil.java | 23 +-
.../hadoop/test/TestTimedOutTestsListener.java | 21 +-
.../apache/hadoop/test/TimedOutTestsListener.java | 7 +-
.../apache/hadoop/test/UnitTestcaseTimeLimit.java | 9 +-
.../org/apache/hadoop/tools/TestCommandShell.java | 28 +-
.../hadoop/util/TestApplicationClassLoader.java | 32 +-
.../apache/hadoop/util/TestAsyncDiskService.java | 10 +-
.../apache/hadoop/util/TestAutoCloseableLock.java | 8 +-
.../apache/hadoop/util/TestBasicDiskValidator.java | 4 +-
.../apache/hadoop/util/TestCacheableIPList.java | 53 +-
.../apache/hadoop/util/TestChunkedArrayList.java | 49 +-
.../java/org/apache/hadoop/util/TestClassUtil.java | 29 +-
.../java/org/apache/hadoop/util/TestClasspath.java | 16 +-
.../java/org/apache/hadoop/util/TestConfTest.java | 5 +-
.../hadoop/util/TestConfigurationHelper.java | 2 +-
.../org/apache/hadoop/util/TestCpuTimeTracker.java | 16 +-
.../org/apache/hadoop/util/TestCrcComposer.java | 17 +-
.../java/org/apache/hadoop/util/TestCrcUtil.java | 22 +-
.../org/apache/hadoop/util/TestDataChecksum.java | 7 +-
.../apache/hadoop/util/TestDirectBufferPool.java | 8 +-
.../org/apache/hadoop/util/TestDiskChecker.java | 63 +-
.../hadoop/util/TestDiskCheckerWithDiskIo.java | 42 +-
.../hadoop/util/TestDiskValidatorFactory.java | 23 +-
.../org/apache/hadoop/util/TestDurationInfo.java | 25 +-
.../java/org/apache/hadoop/util/TestExitUtil.java | 102 +-
.../apache/hadoop/util/TestFastNumberFormat.java | 13 +-
.../apache/hadoop/util/TestFileBasedIPList.java | 79 +-
.../java/org/apache/hadoop/util/TestFindClass.java | 6 +-
.../test/java/org/apache/hadoop/util/TestGSet.java | 66 +-
.../hadoop/util/TestGenericOptionsParser.java | 79 +-
.../org/apache/hadoop/util/TestGenericsUtil.java | 36 +-
.../apache/hadoop/util/TestHostsFileReader.java | 65 +-
.../apache/hadoop/util/TestHttpExceptionUtils.java | 81 +-
.../apache/hadoop/util/TestIdentityHashStore.java | 64 +-
.../org/apache/hadoop/util/TestIndexedSort.java | 44 +-
.../apache/hadoop/util/TestInstrumentedLock.java | 43 +-
.../hadoop/util/TestInstrumentedReadWriteLock.java | 57 +-
.../java/org/apache/hadoop/util/TestJarFinder.java | 14 +-
.../apache/hadoop/util/TestLightWeightCache.java | 32 +-
.../apache/hadoop/util/TestLightWeightGSet.java | 17 +-
.../hadoop/util/TestLightWeightResizableGSet.java | 9 +-
.../org/apache/hadoop/util/TestLineReader.java | 27 +-
.../java/org/apache/hadoop/util/TestLists.java | 61 +-
.../org/apache/hadoop/util/TestMachineList.java | 27 +-
.../apache/hadoop/util/TestNativeCodeLoader.java | 5 +-
.../org/apache/hadoop/util/TestNativeCrc32.java | 103 +-
.../hadoop/util/TestNativeLibraryChecker.java | 5 +-
.../java/org/apache/hadoop/util/TestOptions.java | 19 +-
.../org/apache/hadoop/util/TestPreconditions.java | 2 +-
.../java/org/apache/hadoop/util/TestProgress.java | 15 +-
.../java/org/apache/hadoop/util/TestProtoUtil.java | 6 +-
.../org/apache/hadoop/util/TestPureJavaCrc32.java | 7 +-
.../org/apache/hadoop/util/TestPureJavaCrc32C.java | 6 +-
.../hadoop/util/TestReadWriteDiskValidator.java | 30 +-
.../apache/hadoop/util/TestReflectionUtils.java | 20 +-
.../java/org/apache/hadoop/util/TestRunJar.java | 50 +-
.../java/org/apache/hadoop/util/TestShell.java | 59 +-
.../hadoop/util/TestShutdownHookManager.java | 82 +-
.../hadoop/util/TestShutdownThreadsHelper.java | 18 +-
.../org/apache/hadoop/util/TestSignalLogger.java | 16 +-
.../java/org/apache/hadoop/util/TestStopWatch.java | 20 +-
.../org/apache/hadoop/util/TestStringInterner.java | 5 +-
.../org/apache/hadoop/util/TestStringUtils.java | 125 +-
.../org/apache/hadoop/util/TestSysInfoLinux.java | 6 +-
.../org/apache/hadoop/util/TestSysInfoWindows.java | 17 +-
.../test/java/org/apache/hadoop/util/TestTime.java | 4 +-
.../org/apache/hadoop/util/TestVersionUtil.java | 5 +-
.../apache/hadoop/util/TestWeakReferenceMap.java | 38 +-
.../java/org/apache/hadoop/util/TestWinUtils.java | 95 +-
.../java/org/apache/hadoop/util/TestXMLUtils.java | 107 +-
.../java/org/apache/hadoop/util/TestZKUtil.java | 9 +-
.../hadoop/util/bloom/BloomFilterCommonTester.java | 108 +-
.../apache/hadoop/util/bloom/TestBloomFilters.java | 39 +-
.../util/curator/TestSecureZKCuratorManager.java | 75 +-
.../hadoop/util/curator/TestZKCuratorManager.java | 31 +-
.../hadoop/util/dynamic/TestDynConstructors.java | 31 +-
.../apache/hadoop/util/dynamic/TestDynMethods.java | 117 +-
.../hadoop/util/functional/TestFunctionalIO.java | 2 +-
.../hadoop/util/functional/TestLazyReferences.java | 2 +-
.../util/functional/TestRemoteIterators.java | 2 +-
.../java/org/apache/hadoop/util/hash/TestHash.java | 61 +-
.../apache/hadoop/hdfs/DFSStripedOutputStream.java | 7 +-
.../hadoop/hdfs/client/impl/DfsClientConf.java | 10 +
.../hadoop/hdfs/nfs/nfs3/Nfs3HttpServer.java | 2 +-
.../router/MountTableRefresherService.java | 16 +-
.../server/federation/router/RBFConfigKeys.java | 3 +
.../federation/router/RouterHeartbeatService.java | 9 +-
.../server/federation/router/RouterHttpServer.java | 2 +-
.../server/federation/store/StateStoreUtils.java | 16 +
.../src/main/resources/hdfs-rbf-default.xml | 8 +
.../router/TestRouterMountTableCacheRefresh.java | 32 +-
.../server/federation/router/TestRouterRpc.java | 5 +-
hadoop-hdfs-project/hadoop-hdfs/pom.xml | 4 +
.../java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 21 +
.../main/java/org/apache/hadoop/hdfs/DFSUtil.java | 4 +-
.../qjournal/server/JournalNodeHttpServer.java | 2 +-
.../delegation/DelegationTokenSecretManager.java | 9 +-
.../hadoop/hdfs/server/balancer/Balancer.java | 55 +-
.../hdfs/server/balancer/BalancerHttpServer.java | 114 ++
.../hdfs/server/balancer/BalancerMXBean.java | 28 +-
.../hadoop/hdfs/server/balancer}/package-info.java | 12 +-
.../hdfs/server/blockmanagement/BlockManager.java | 122 +-
.../blockmanagement/BlockManagerSafeMode.java | 33 +-
.../blockmanagement/CacheReplicationMonitor.java | 13 +-
.../DatanodeAdminBackoffMonitor.java | 38 +-
.../DatanodeAdminDefaultMonitor.java | 11 +-
.../server/blockmanagement/DatanodeManager.java | 12 +-
.../server/blockmanagement/HeartbeatManager.java | 9 +-
.../server/blockmanagement/ProvidedStorageMap.java | 5 +-
.../hadoop/hdfs/server/namenode/BackupImage.java | 5 +-
.../hadoop/hdfs/server/namenode/CacheManager.java | 33 +-
.../hadoop/hdfs/server/namenode/Checkpointer.java | 10 +-
.../namenode/ContentSummaryComputationContext.java | 12 +-
.../server/namenode/EncryptionZoneManager.java | 14 +-
.../hadoop/hdfs/server/namenode/FSDirAppendOp.java | 5 +-
.../hadoop/hdfs/server/namenode/FSDirDeleteOp.java | 4 +-
.../server/namenode/FSDirEncryptionZoneOp.java | 15 +-
.../hdfs/server/namenode/FSDirErasureCodingOp.java | 17 +-
.../namenode/FSDirSatisfyStoragePolicyOp.java | 3 +-
.../server/namenode/FSDirStatAndListingOp.java | 25 +-
.../hdfs/server/namenode/FSDirTruncateOp.java | 9 +-
.../hdfs/server/namenode/FSDirWriteFileOp.java | 46 +-
.../hadoop/hdfs/server/namenode/FSDirectory.java | 29 +-
.../hdfs/server/namenode/FSEditLogLoader.java | 9 +-
.../hdfs/server/namenode/FSImageFormatPBINode.java | 5 +-
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 672 ++++---
.../hdfs/server/namenode/FSNamesystemLock.java | 44 +-
.../hdfs/server/namenode/FSPermissionChecker.java | 15 +-
.../hdfs/server/namenode/FSTreeTraverser.java | 7 +-
.../hdfs/server/namenode/FsImageValidation.java | 5 +-
.../hadoop/hdfs/server/namenode/INodeFile.java | 6 +
.../hadoop/hdfs/server/namenode/LeaseManager.java | 15 +-
.../hadoop/hdfs/server/namenode/NameNode.java | 25 +-
.../hdfs/server/namenode/NameNodeHttpServer.java | 2 +-
.../hadoop/hdfs/server/namenode/NamenodeFsck.java | 10 +-
.../hdfs/server/namenode/ReencryptionHandler.java | 11 +-
.../hdfs/server/namenode/ReencryptionUpdater.java | 7 +-
.../hdfs/server/namenode/SecondaryNameNode.java | 7 +-
.../hdfs/server/namenode/fgl/FSNLockManager.java | 187 ++
.../namenode/fgl/FineGrainedFSNamesystemLock.java | 285 +++
.../namenode/fgl/GlobalFSNamesystemLock.java | 150 ++
.../hdfs/server/namenode/fgl}/package-info.java | 2 +-
.../hdfs/server/namenode/ha/EditLogTailer.java | 5 +-
.../namenode/snapshot/SnapshotDeletionGc.java | 5 +-
.../java/org/apache/hadoop/hdfs/util/RwLock.java | 106 +-
.../org/apache/hadoop/hdfs/util/RwLockMode.java} | 9 +
.../src/main/resources/hdfs-default.xml | 63 +
.../journalnode.html => balancer/balancer.html} | 51 +-
.../src/main/webapps/balancer/balancer.js | 95 +
.../webapps/{secondary => balancer}/index.html | 2 +-
.../main/webapps/{datanode => balancer}/robots.txt | 0
.../hadoop-hdfs/src/site/markdown/NamenodeFGL.md | 210 +++
.../apache/hadoop/hdfs/AdminStatesBaseTest.java | 4 +
.../hadoop/hdfs/TestBlocksScheduledCounter.java | 11 +-
.../org/apache/hadoop/hdfs/TestDecommission.java | 51 +-
.../org/apache/hadoop/hdfs/TestFileCorruption.java | 13 +-
.../apache/hadoop/hdfs/TestMaintenanceState.java | 50 +-
.../balancer/TestBalancerHttpServer.java} | 99 +-
.../hdfs/server/balancer/TestBalancerService.java | 51 +
.../blockmanagement/BlockManagerTestUtil.java | 22 +-
.../server/blockmanagement/TestBlockManager.java | 15 +-
.../blockmanagement/TestBlockManagerSafeMode.java | 3 +
.../TestBlocksWithNotEnoughRacks.java | 5 +-
.../blockmanagement/TestComputeInvalidateWork.java | 25 +-
.../blockmanagement/TestDatanodeManager.java | 15 +-
.../blockmanagement/TestHeartbeatHandling.java | 9 +-
.../TestNameNodePrunesMissingStorages.java | 10 +-
.../hdfs/server/blockmanagement/TestNodeCount.java | 5 +-
.../blockmanagement/TestOverReplicatedBlocks.java | 9 +-
.../blockmanagement/TestPendingReconstruction.java | 17 +-
.../blockmanagement/TestProvidedStorageMap.java | 3 +-
...tReconstructStripedBlocksWithRackAwareness.java | 10 +-
.../blockmanagement/TestReplicationPolicy.java | 8 +
.../TestReplicationPolicyConsiderLoad.java | 10 +-
.../TestReplicationPolicyExcludeSlowNodes.java | 11 +-
...licationPolicyRatioConsiderLoadWithStorage.java | 6 +-
.../hdfs/server/namenode/NameNodeAdapter.java | 17 +-
.../hdfs/server/namenode/TestAddBlockRetry.java | 14 +-
.../TestAddOverReplicatedStripedBlocks.java | 6 +-
.../TestBlockPlacementPolicyRackFaultTolerant.java | 6 +-
.../hdfs/server/namenode/TestCacheDirectives.java | 17 +-
.../hdfs/server/namenode/TestDeleteRace.java | 5 +-
.../server/namenode/TestDiskspaceQuotaUpdate.java | 5 +-
.../hdfs/server/namenode/TestEditLogRace.java | 6 +-
.../server/namenode/TestFSImageWithSnapshot.java | 9 +-
.../hdfs/server/namenode/TestFSNamesystem.java | 5 +-
.../hdfs/server/namenode/TestFSNamesystemLock.java | 20 +-
.../server/namenode/TestFSNamesystemMBean.java | 7 +-
.../hdfs/server/namenode/TestFileTruncate.java | 9 +-
.../hadoop/hdfs/server/namenode/TestFsck.java | 9 +-
.../server/namenode/TestGetBlockLocations.java | 14 +-
.../server/namenode/TestLargeDirectoryDelete.java | 5 +-
.../hdfs/server/namenode/TestLeaseManager.java | 5 +
.../hdfs/server/namenode/TestListOpenFiles.java | 5 +-
.../namenode/TestNameNodeMetadataConsistency.java | 11 +-
.../namenode/TestReconstructStripedBlocks.java | 5 +-
.../server/namenode/TestSecurityTokenEditLog.java | 5 +-
.../namenode/fgl/FSNLockBenchmarkThroughput.java | 322 ++++
.../fgl/TestFSNLockBenchmarkThroughput.java | 104 +
.../fgl/TestFineGrainedFSNamesystemLock.java | 276 +++
.../hdfs/server/namenode/ha/TestDNFencing.java | 5 +-
.../namenode/metrics/TestNameNodeMetrics.java | 13 +-
...TestINodeFileUnderConstructionWithSnapshot.java | 5 +-
.../org/apache/hadoop/hdfs/tools/TestDFSAdmin.java | 5 +-
.../hadoop-mapreduce-client-app/pom.xml | 9 +
.../hadoop/mapred/TestLocalContainerLauncher.java | 21 +-
.../mapred/TestTaskAttemptFinishingMonitor.java | 6 +-
.../hadoop/mapred/TestTaskAttemptListenerImpl.java | 44 +-
.../org/apache/hadoop/mapred/TestYarnChild.java | 13 +-
.../hadoop/mapreduce/jobhistory/TestEvents.java | 22 +-
.../jobhistory/TestJobHistoryEventHandler.java | 268 ++-
.../mapreduce/jobhistory/TestJobSummary.java | 15 +-
.../v2/api/records/TestTaskAttemptReport.java | 8 +-
.../mapreduce/v2/api/records/TestTaskReport.java | 8 +-
.../org/apache/hadoop/mapreduce/v2/app/MRApp.java | 73 +-
.../hadoop/mapreduce/v2/app/MRAppBenchmark.java | 13 +-
.../hadoop/mapreduce/v2/app/TestAMInfos.java | 14 +-
.../v2/app/TestCheckpointPreemptionPolicy.java | 11 +-
.../apache/hadoop/mapreduce/v2/app/TestFail.java | 49 +-
.../hadoop/mapreduce/v2/app/TestFetchFailure.java | 160 +-
.../mapreduce/v2/app/TestJobEndNotifier.java | 99 +-
.../apache/hadoop/mapreduce/v2/app/TestKill.java | 85 +-
.../v2/app/TestKillAMPreemptionPolicy.java | 2 +-
.../apache/hadoop/mapreduce/v2/app/TestMRApp.java | 123 +-
.../v2/app/TestMRAppComponentDependencies.java | 16 +-
.../hadoop/mapreduce/v2/app/TestMRAppMaster.java | 92 +-
.../mapreduce/v2/app/TestMRClientService.java | 96 +-
.../hadoop/mapreduce/v2/app/TestRecovery.java | 203 +-
.../mapreduce/v2/app/TestRuntimeEstimators.java | 41 +-
.../mapreduce/v2/app/TestStagingCleanup.java | 44 +-
.../mapreduce/v2/app/TestTaskHeartbeatHandler.java | 25 +-
.../v2/app/commit/TestCommitterEventHandler.java | 41 +-
.../mapreduce/v2/app/job/impl/TestJobImpl.java | 170 +-
.../v2/app/job/impl/TestMapReduceChildJVM.java | 92 +-
.../v2/app/job/impl/TestShuffleProvider.java | 14 +-
.../mapreduce/v2/app/job/impl/TestTaskAttempt.java | 315 ++--
.../job/impl/TestTaskAttemptContainerRequest.java | 30 +-
.../mapreduce/v2/app/job/impl/TestTaskImpl.java | 28 +-
.../v2/app/launcher/TestContainerLauncher.java | 65 +-
.../v2/app/launcher/TestContainerLauncherImpl.java | 19 +-
.../v2/app/local/TestLocalContainerAllocator.java | 31 +-
.../mapreduce/v2/app/metrics/TestMRAppMetrics.java | 9 +-
.../mapreduce/v2/app/rm/TestRMCommunicator.java | 9 +-
.../v2/app/rm/TestRMContainerAllocator.java | 511 ++---
.../v2/app/rm/TestResourceCalculatorUtils.java | 21 +-
.../v2/app/speculate/TestDataStatistics.java | 49 +-
.../forecast/TestSimpleExponentialForecast.java | 14 +-
.../mapreduce/v2/app/webapp/TestAMWebApp.java | 56 +-
.../mapreduce/v2/app/webapp/TestAMWebServices.java | 47 +-
.../v2/app/webapp/TestAMWebServicesAttempt.java | 13 +-
.../v2/app/webapp/TestAMWebServicesAttempts.java | 88 +-
.../v2/app/webapp/TestAMWebServicesJobConf.java | 41 +-
.../v2/app/webapp/TestAMWebServicesJobs.java | 161 +-
.../v2/app/webapp/TestAMWebServicesTasks.java | 92 +-
.../mapreduce/v2/app/webapp/TestAppController.java | 18 +-
.../hadoop/mapreduce/v2/app/webapp/TestBlocks.java | 10 +-
.../hadoop-mapreduce-client-core/pom.xml | 10 +
.../java/org/apache/hadoop/mapred/TestClock.java | 21 +-
.../apache/hadoop/mapred/TestClusterStatus.java | 20 +-
.../org/apache/hadoop/mapred/TestCounters.java | 90 +-
.../apache/hadoop/mapred/TestFileInputFormat.java | 116 +-
.../hadoop/mapred/TestFileOutputCommitter.java | 43 +-
.../org/apache/hadoop/mapred/TestIndexCache.java | 10 +-
.../apache/hadoop/mapred/TestJobAclsManager.java | 22 +-
.../java/org/apache/hadoop/mapred/TestJobConf.java | 91 +-
.../apache/hadoop/mapred/TestJobEndNotifier.java | 21 +-
.../java/org/apache/hadoop/mapred/TestJobInfo.java | 15 +-
.../apache/hadoop/mapred/TestJobQueueClient.java | 9 +-
.../apache/hadoop/mapred/TestLineRecordReader.java | 131 +-
.../mapred/TestLocatedFileStatusFetcher.java | 19 +-
.../hadoop/mapred/TestMapFileOutputFormat.java | 10 +-
.../java/org/apache/hadoop/mapred/TestMapTask.java | 71 +-
.../java/org/apache/hadoop/mapred/TestMaster.java | 4 +-
.../apache/hadoop/mapred/TestOldMethodsJobID.java | 32 +-
.../java/org/apache/hadoop/mapred/TestQueue.java | 28 +-
.../apache/hadoop/mapred/TestSkipBadRecords.java | 12 +-
.../java/org/apache/hadoop/mapred/TestTask.java | 22 +-
.../java/org/apache/hadoop/mapred/TestTaskLog.java | 15 +-
.../hadoop/mapred/TestTaskProgressReporter.java | 29 +-
.../mapred/lib/TestCombineFileRecordReader.java | 6 +-
.../hadoop/mapred/lib/db/TestDBInputFormat.java | 29 +-
.../org/apache/hadoop/mapreduce/TestCluster.java | 10 +-
.../hadoop/mapreduce/TestContextFactory.java | 8 +-
.../java/org/apache/hadoop/mapreduce/TestJob.java | 23 +-
.../hadoop/mapreduce/TestJobMonitorAndPrint.java | 15 +-
.../hadoop/mapreduce/TestJobResourceUploader.java | 73 +-
.../TestJobResourceUploaderWithSharedCache.java | 16 +-
.../hadoop/mapreduce/TestJobSubmissionFiles.java | 39 +-
.../apache/hadoop/mapreduce/TestShufflePlugin.java | 17 +-
.../org/apache/hadoop/mapreduce/TestTaskID.java | 218 ++-
.../mapreduce/checkpoint/TestFSCheckpointID.java | 2 +-
.../checkpoint/TestFSCheckpointService.java | 16 +-
.../TestClientDistributedCacheManager.java | 79 +-
.../mapreduce/filecache/TestDistributedCache.java | 78 +-
.../jobhistory/TestHistoryViewerPrinter.java | 32 +-
.../hadoop/mapreduce/lib/db/DriverForTest.java | 5 +-
.../hadoop/mapreduce/lib/db/TestDbClasses.java | 26 +-
.../hadoop/mapreduce/lib/db/TestSplitters.java | 32 +-
.../lib/input/BaseTestLineRecordReaderBZip2.java | 14 +-
.../lib/input/TestCombineFileRecordReader.java | 6 +-
.../mapreduce/lib/input/TestFileInputFormat.java | 179 +-
.../mapreduce/lib/input/TestLineRecordReader.java | 133 +-
.../mapreduce/lib/jobcontrol/TestJobControl.java | 4 +-
.../lib/output/TestFileOutputCommitter.java | 75 +-
.../mapreduce/lib/output/TestFileOutputFormat.java | 5 +-
.../lib/output/TestMapFileOutputFormat.java | 8 +-
.../lib/output/TestPathOutputCommitter.java | 6 +-
.../lib/output/TestPathOutputCommitterFactory.java | 30 +-
.../output/TestPreemptableFileOutputCommitter.java | 12 +-
.../lib/partition/TestRehashPartitioner.java | 6 +-
.../hadoop/mapreduce/security/TestTokenCache.java | 19 +-
.../hadoop/mapreduce/split/TestJobSplitWriter.java | 16 +-
.../mapreduce/split/TestJobSplitWriterWithEC.java | 17 +-
.../mapreduce/task/reduce/TestEventFetcher.java | 2 +-
.../hadoop/mapreduce/task/reduce/TestFetcher.java | 86 +-
.../mapreduce/task/reduce/TestMergeManager.java | 74 +-
.../hadoop/mapreduce/task/reduce/TestMerger.java | 73 +-
.../task/reduce/TestShuffleClientMetrics.java | 4 +-
.../task/reduce/TestShuffleScheduler.java | 36 +-
.../org/apache/hadoop/mapreduce/tools/TestCLI.java | 40 +-
.../hadoop-mapreduce-client-hs/pom.xml | 5 +
.../hadoop/mapreduce/v2/hs/TestCompletedTask.java | 39 +-
.../mapreduce/v2/hs/TestHistoryFileManager.java | 136 +-
...estHistoryFileManagerInitWithNonRunningDFS.java | 15 +-
...stHistoryServerFileSystemStateStoreService.java | 87 +-
.../TestHistoryServerLeveldbStateStoreService.java | 84 +-
.../v2/hs/TestJHSDelegationTokenSecretManager.java | 52 +-
.../hadoop/mapreduce/v2/hs/TestJobHistory.java | 16 +-
.../mapreduce/v2/hs/TestJobHistoryEntities.java | 67 +-
.../mapreduce/v2/hs/TestJobHistoryEvents.java | 74 +-
.../mapreduce/v2/hs/TestJobHistoryParsing.java | 279 ++-
.../mapreduce/v2/hs/TestJobHistoryServer.java | 31 +-
.../v2/hs/TestJobIdHistoryFileInfoMap.java | 30 +-
.../hadoop/mapreduce/v2/hs/TestJobListCache.java | 19 +-
.../TestUnnecessaryBlockingOnHistoryFileInfo.java | 22 +-
.../mapreduce/v2/hs/server/TestHSAdminServer.java | 89 +-
.../hadoop/mapreduce/v2/hs/webapp/TestBlocks.java | 27 +-
.../mapreduce/v2/hs/webapp/TestHSWebApp.java | 18 +-
.../mapreduce/v2/hs/webapp/TestHsJobBlock.java | 32 +-
.../mapreduce/v2/hs/webapp/TestHsWebServices.java | 37 +-
.../v2/hs/webapp/TestHsWebServicesAcls.java | 10 +-
.../v2/hs/webapp/TestHsWebServicesAttempts.java | 85 +-
.../v2/hs/webapp/TestHsWebServicesJobConf.java | 34 +-
.../v2/hs/webapp/TestHsWebServicesJobs.java | 104 +-
.../v2/hs/webapp/TestHsWebServicesJobsQuery.java | 88 +-
.../v2/hs/webapp/TestHsWebServicesLogs.java | 12 +-
.../v2/hs/webapp/TestHsWebServicesLogsExtend.java | 16 +-
.../v2/hs/webapp/TestHsWebServicesTasks.java | 91 +-
.../mapreduce/v2/hs/webapp/VerifyJobsUtils.java | 50 +-
.../mapreduce/v2/hs/webapp/dao/TestJobInfo.java | 26 +-
.../hadoop-mapreduce-client-jobclient/pom.xml | 5 +
.../hadoop-mapreduce-client-nativetask/pom.xml | 5 +
.../hadoop/mapred/nativetask/TestTaskContext.java | 13 +-
.../nativetask/buffer/TestBufferPushPull.java | 17 +-
.../nativetask/buffer/TestByteBufferReadWrite.java | 2 +-
.../mapred/nativetask/buffer/TestInputBuffer.java | 2 +-
.../mapred/nativetask/buffer/TestOutputBuffer.java | 2 +-
.../nativetask/combinertest/CombinerTest.java | 18 +-
.../combinertest/LargeKVCombinerTest.java | 17 +-
.../combinertest/OldAPICombinerTest.java | 16 +-
.../nativetask/compresstest/CompressTest.java | 16 +-
.../nativetask/handlers/TestCombineHandler.java | 6 +-
.../handlers/TestNativeCollectorOnlyHandler.java | 23 +-
.../hadoop/mapred/nativetask/kvtest/KVTest.java | 38 +-
.../mapred/nativetask/kvtest/LargeKVTest.java | 25 +-
.../mapred/nativetask/nonsorttest/NonSortTest.java | 16 +-
.../mapred/nativetask/serde/TestKVSerializer.java | 23 +-
.../nativetask/serde/TestNativeSerialization.java | 12 +-
.../mapred/nativetask/testutil/ResultVerifier.java | 20 +-
.../mapred/nativetask/utils/TestBytesUtil.java | 17 +-
.../nativetask/utils/TestReadWriteBuffer.java | 2 +-
.../mapred/nativetask/utils/TestSizedWritable.java | 10 +-
.../hadoop-mapreduce-client-shuffle/pom.xml | 5 +
.../hadoop/mapred/TestFadvisedChunkedFile.java | 12 +-
.../hadoop/mapred/TestFadvisedFileRegion.java | 28 +-
.../hadoop/mapred/TestShuffleChannelHandler.java | 36 +-
.../apache/hadoop/mapred/TestShuffleHandler.java | 69 +-
.../hadoop/mapred/TestShuffleHandlerBase.java | 14 +-
hadoop-project/pom.xml | 22 +-
hadoop-project/src/site/markdown/index.md.vm | 162 +-
hadoop-tools/hadoop-archive-logs/pom.xml | 5 +
hadoop-tools/hadoop-archives/pom.xml | 5 +
hadoop-tools/hadoop-aws/pom.xml | 5 +
hadoop-tools/hadoop-azure-datalake/pom.xml | 6 +-
.../dev-support/testrun-scripts/runtests.sh | 123 +-
.../dev-support/testrun-scripts/testsupport.sh | 19 +
hadoop-tools/hadoop-azure/pom.xml | 5 +
.../hadoop/fs/azurebfs/AbfsBackoffMetrics.java | 312 ---
.../hadoop/fs/azurebfs/AbfsConfiguration.java | 98 +-
.../hadoop/fs/azurebfs/AbfsCountersImpl.java | 12 +-
.../hadoop/fs/azurebfs/AzureBlobFileSystem.java | 155 +-
.../fs/azurebfs/AzureBlobFileSystemStore.java | 277 ++-
.../fs/azurebfs/constants/AbfsHttpConstants.java | 37 +-
.../fs/azurebfs/constants/ConfigurationKeys.java | 25 +
.../constants/FileSystemConfigurations.java | 21 +
.../constants/HttpHeaderConfigurations.java | 28 +
.../fs/azurebfs/constants/MetricsConstants.java | 121 ++
.../ConcurrentWriteOperationDetectedException.java | 17 +
...on.java => InvalidIngressServiceException.java} | 32 +-
.../services/AppendRequestParameters.java | 54 +-
.../contracts/services/AzureServiceErrorCode.java | 7 +
.../services/BlobAppendRequestParameters.java | 8 +
.../fs/azurebfs/enums/AbfsBackoffMetricsEnum.java | 110 ++
.../azurebfs/enums/AbfsReadFooterMetricsEnum.java | 97 +
.../fs/azurebfs/enums/BlobCopyProgress.java} | 29 +-
.../enums/{package-info.java => FileType.java} | 24 +-
.../hadoop/fs/azurebfs/enums/RetryValue.java | 83 +
.../{package-info.java => StatisticTypeEnum.java} | 25 +-
.../fs/azurebfs/extensions/SASTokenProvider.java | 2 +
.../fs/azurebfs/services/AbfsBackoffMetrics.java | 323 ++++
.../hadoop/fs/azurebfs/services/AbfsBlobBlock.java | 72 +
.../fs/azurebfs/services/AbfsBlobClient.java | 904 ++++++++-
.../hadoop/fs/azurebfs/services/AbfsBlock.java | 160 ++
...ryPolicyConstants.java => AbfsBlockStatus.java} | 30 +-
.../hadoop/fs/azurebfs/services/AbfsClient.java | 114 +-
.../fs/azurebfs/services/AbfsClientHandler.java | 29 +
.../hadoop/fs/azurebfs/services/AbfsCounters.java | 1 -
.../hadoop/fs/azurebfs/services/AbfsDfsClient.java | 114 +-
.../hadoop/fs/azurebfs/services/AbfsErrors.java | 25 +-
.../fs/azurebfs/services/AbfsHttpOperation.java | 4 +-
.../fs/azurebfs/services/AbfsInputStream.java | 2 +-
.../hadoop/fs/azurebfs/services/AbfsLease.java | 165 +-
.../fs/azurebfs/services/AbfsOutputStream.java | 614 ++++--
.../azurebfs/services/AbfsOutputStreamContext.java | 67 +-
.../fs/azurebfs/services/AbfsPermission.java | 5 +-
.../azurebfs/services/AbfsReadFooterMetrics.java | 984 +++++-----
.../fs/azurebfs/services/AbfsRestOperation.java | 83 +-
.../azurebfs/services/AbfsRestOperationType.java | 3 +-
.../services/AbstractAbfsStatisticsSource.java | 153 ++
.../azurebfs/services/AzureBlobBlockManager.java | 194 ++
.../azurebfs/services/AzureBlobIngressHandler.java | 348 ++++
.../fs/azurebfs/services/AzureBlockManager.java | 170 ++
.../fs/azurebfs/services/AzureDFSBlockManager.java | 89 +
.../azurebfs/services/AzureDFSIngressHandler.java | 280 +++
.../AzureDfsToBlobIngressFallbackHandler.java | 265 +++
.../fs/azurebfs/services/AzureIngressHandler.java | 209 +++
.../fs/azurebfs/services/BlobDeleteHandler.java | 201 ++
.../fs/azurebfs/services/BlobRenameHandler.java | 637 +++++++
.../hadoop/fs/azurebfs/services/BlockEntry.java | 86 +
.../fs/azurebfs/services/ListActionTaker.java | 270 +++
.../hadoop/fs/azurebfs/services/ListBlobQueue.java | 197 ++
.../fs/azurebfs/services/PathInformation.java | 108 ++
.../fs/azurebfs/services/RenameAtomicity.java | 336 ++++
.../RenamePendingJsonFormat.java} | 48 +-
.../apache/hadoop/fs/azurebfs/utils/Listener.java | 2 +
.../hadoop/fs/azurebfs/utils/TracingContext.java | 65 +
.../apache/hadoop/fs/azurebfs/utils/UriUtils.java | 38 +
.../hadoop-azure/src/site/markdown/blobEndpoint.md | 7 +-
.../hadoop-azure/src/site/markdown/fns_blob.md | 80 +-
.../fs/azurebfs/AbstractAbfsIntegrationTest.java | 166 +-
.../fs/azurebfs/ITestAbfsCustomEncryption.java | 48 +-
.../ITestAbfsListStatusRemoteIterator.java | 1 +
.../fs/azurebfs/ITestAbfsNetworkStatistics.java | 59 +-
.../fs/azurebfs/ITestAbfsReadFooterMetrics.java | 39 +-
.../azurebfs/ITestAbfsRestOperationException.java | 133 +-
.../hadoop/fs/azurebfs/ITestAbfsStatistics.java | 2 -
.../azurebfs/ITestAzureBlobFileSystemAppend.java | 1052 ++++++++++-
.../ITestAzureBlobFileSystemAttributes.java | 184 +-
.../azurebfs/ITestAzureBlobFileSystemChecksum.java | 48 +-
.../ITestAzureBlobFileSystemChooseSAS.java | 2 +
.../azurebfs/ITestAzureBlobFileSystemCreate.java | 1978 +++++++++++++++++---
.../ITestAzureBlobFileSystemDelegationSAS.java | 8 +-
.../azurebfs/ITestAzureBlobFileSystemDelete.java | 328 +++-
.../fs/azurebfs/ITestAzureBlobFileSystemE2E.java | 19 +-
.../ITestAzureBlobFileSystemFileStatus.java | 125 +-
.../fs/azurebfs/ITestAzureBlobFileSystemFlush.java | 5 +
.../ITestAzureBlobFileSystemInitAndCreate.java | 25 +-
.../fs/azurebfs/ITestAzureBlobFileSystemLease.java | 75 +-
.../ITestAzureBlobFileSystemListStatus.java | 1 +
.../fs/azurebfs/ITestAzureBlobFileSystemMkDir.java | 29 +-
.../ITestAzureBlobFileSystemPermission.java | 3 +
.../ITestAzureBlobFileSystemRandomRead.java | 1 +
.../azurebfs/ITestAzureBlobFileSystemRename.java | 1453 +++++++++++++-
.../ITestAzureBlobFileSystemRenameUnicode.java | 23 +-
.../fs/azurebfs/ITestAzureBlobFilesystemAcl.java | 5 +
.../fs/azurebfs/ITestGetNameSpaceEnabled.java | 22 +-
.../hadoop/fs/azurebfs/ITestSharedKeyAuth.java | 15 +-
.../fs/azurebfs/ITestSmallWriteOptimization.java | 21 +-
.../fs/azurebfs/ITestWasbAbfsCompatibility.java | 12 +-
.../hadoop/fs/azurebfs/TestTracingContext.java | 2 +-
.../azurebfs/constants/TestConfigurationKeys.java | 1 +
.../azurebfs/contract/TestBlobListXmlParser.java | 154 ++
...tSchemaTest.java => TestDfsListJsonParser.java} | 12 +-
.../fs/azurebfs/services/AbfsClientTestUtil.java | 136 ++
.../fs/azurebfs/services/ITestAbfsClient.java | 9 +-
.../azurebfs/services/ITestAbfsClientHandler.java | 12 +-
.../ITestAbfsHttpClientRequestExecutor.java | 59 +-
.../azurebfs/services/ITestAbfsOutputStream.java | 187 +-
.../services/ITestAbfsPaginatedDelete.java | 12 +-
.../azurebfs/services/ITestAbfsRestOperation.java | 5 +-
.../services/ITestApacheClientConnectionPool.java | 3 +-
.../services/RenameAtomicityTestUtils.java | 79 +
.../azurebfs/services/TestAbfsBackoffMetrics.java | 115 ++
.../fs/azurebfs/services/TestAbfsOutputStream.java | 56 +-
.../services/TestAbfsReadFooterMetrics.java | 98 +
.../services/TestAbfsRenameRetryRecovery.java | 51 +-
.../azurebfs/services/TestAbfsRestOperation.java | 5 +-
.../fs/azurebfs/services/TestListActionTaker.java | 142 ++
.../azurebfs/utils/AzcopyExecutionException.java} | 28 +-
.../hadoop/fs/azurebfs/utils/AzcopyToolHelper.java | 319 ++++
.../fs/azurebfs/utils/DirectoryStateHelper.java | 118 ++
.../fs/azurebfs/utils/TestAzcopyToolHelper.java | 78 +
.../fs/azurebfs/utils/TracingHeaderValidator.java | 45 +-
.../accountName_settings.xml.template | 24 +
hadoop-tools/hadoop-datajoin/pom.xml | 5 +
hadoop-tools/hadoop-distcp/pom.xml | 5 +
.../org/apache/hadoop/tools/SimpleCopyListing.java | 15 +-
.../org/apache/hadoop/tools/TestCopyFilter.java | 16 +-
.../org/apache/hadoop/tools/TestCopyListing.java | 175 +-
.../hadoop/tools/TestCopyListingFileStatus.java | 4 +-
.../org/apache/hadoop/tools/TestDistCpOptions.java | 214 ++-
.../org/apache/hadoop/tools/TestDistCpSync.java | 92 +-
.../hadoop/tools/TestDistCpSyncReverseBase.java | 60 +-
.../org/apache/hadoop/tools/TestDistCpSystem.java | 52 +-
.../org/apache/hadoop/tools/TestDistCpViewFs.java | 16 +-
.../apache/hadoop/tools/TestDistCpWithAcls.java | 15 +-
.../hadoop/tools/TestDistCpWithRawXAttrs.java | 63 +-
.../apache/hadoop/tools/TestDistCpWithXAttrs.java | 10 +-
.../org/apache/hadoop/tools/TestExternalCall.java | 67 +-
.../hadoop/tools/TestFileBasedCopyListing.java | 48 +-
.../hadoop/tools/TestGlobbedCopyListing.java | 17 +-
.../org/apache/hadoop/tools/TestIntegration.java | 199 +-
.../org/apache/hadoop/tools/TestOptionsParser.java | 312 +--
.../apache/hadoop/tools/TestRegexCopyFilter.java | 15 +-
.../tools/TestRegexpInConfigurationFilter.java | 22 +-
.../apache/hadoop/tools/TestTrueCopyFilter.java | 9 +-
.../apache/hadoop/tools/mapred/TestCopyMapper.java | 197 +-
.../tools/mapred/TestCopyMapperCompositeCrc.java | 4 +-
.../hadoop/tools/mapred/TestCopyOutputFormat.java | 43 +-
.../hadoop/tools/mapred/TestDeletedDirTracker.java | 43 +-
.../tools/mapred/TestRetriableFileCopyCommand.java | 31 +-
.../tools/mapred/TestUniformSizeInputFormat.java | 22 +-
.../tools/mapred/lib/TestDynamicInputFormat.java | 59 +-
.../apache/hadoop/tools/util/DistCpTestUtils.java | 13 +-
.../apache/hadoop/tools/util/TestDistCpUtils.java | 472 +++--
.../tools/util/TestDistCpUtilsWithCombineMode.java | 37 +-
.../hadoop/tools/util/TestProducerConsumer.java | 30 +-
.../hadoop/tools/util/TestRetriableCommand.java | 17 +-
.../tools/util/TestThrottledInputStream.java | 23 +-
.../hadoop-dynamometer-blockgen/pom.xml | 5 +
.../hadoop-dynamometer-infra/pom.xml | 5 +
.../hadoop-dynamometer-workload/pom.xml | 5 +
hadoop-tools/hadoop-extras/pom.xml | 5 +
hadoop-tools/hadoop-federation-balance/pom.xml | 5 +
hadoop-tools/hadoop-fs2img/pom.xml | 5 +
.../namenode/ITestProvidedImplementation.java | 13 +-
hadoop-tools/hadoop-gridmix/pom.xml | 5 +
hadoop-tools/hadoop-resourceestimator/pom.xml | 5 +
hadoop-tools/hadoop-rumen/pom.xml | 5 +
hadoop-tools/hadoop-sls/pom.xml | 5 +
hadoop-tools/hadoop-streaming/pom.xml | 5 +
.../records/timelineservice/ApplicationEntity.java | 7 +-
.../org/apache/hadoop/yarn/webapp/WebApps.java | 12 +
.../server/nodemanager/LinuxContainerExecutor.java | 6 +-
.../localizer/ResourceLocalizationService.java | 4 +-
.../TestLinuxContainerExecutorWithMocks.java | 62 +-
.../server/resourcemanager/rmapp/RMAppImpl.java | 2 +-
.../PriorityUtilizationQueueOrderingPolicy.java | 6 +-
...TestPriorityUtilizationQueueOrderingPolicy.java | 54 +
.../storage/common/ColumnRWHelper.java | 2 +-
867 files changed, 31450 insertions(+), 14772 deletions(-)
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/BalancerHttpServer.java
copy
hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/BasicSessionCredential.java
=>
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/BalancerMXBean.java
(64%)
copy
{hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/oncrpc/security
=>
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer}/package-info.java
(73%)
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/FSNLockManager.java
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/FineGrainedFSNamesystemLock.java
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/GlobalFSNamesystemLock.java
copy
hadoop-hdfs-project/{hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol
=>
hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl}/package-info.java
(94%)
copy
hadoop-hdfs-project/{hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/package-info.java
=> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/RwLockMode.java} (90%)
copy
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/{journal/journalnode.html =>
balancer/balancer.html} (77%)
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/balancer/balancer.js
copy hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/{secondary =>
balancer}/index.html (93%)
copy hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/{datanode =>
balancer}/robots.txt (100%)
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/NamenodeFGL.md
copy
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/{web/TestHttpsFileSystem.java
=> server/balancer/TestBalancerHttpServer.java} (52%)
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/fgl/FSNLockBenchmarkThroughput.java
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/fgl/TestFSNLockBenchmarkThroughput.java
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/fgl/TestFineGrainedFSNamesystemLock.java
delete mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsBackoffMetrics.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/MetricsConstants.java
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/{InvalidConfigurationValueException.java
=> InvalidIngressServiceException.java} (53%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/enums/AbfsBackoffMetricsEnum.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/enums/AbfsReadFooterMetricsEnum.java
copy
hadoop-tools/{hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/statistics/StatisticTypeEnum.java
=>
hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/enums/BlobCopyProgress.java}
(77%)
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/enums/{package-info.java
=> FileType.java} (72%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/enums/RetryValue.java
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/enums/{package-info.java
=> StatisticTypeEnum.java} (79%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBackoffMetrics.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobBlock.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlock.java
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/{RetryPolicyConstants.java
=> AbfsBlockStatus.java} (58%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbstractAbfsStatisticsSource.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AzureBlobBlockManager.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AzureBlobIngressHandler.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AzureBlockManager.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AzureDFSBlockManager.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AzureDFSIngressHandler.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AzureDfsToBlobIngressFallbackHandler.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AzureIngressHandler.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/BlobDeleteHandler.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/BlobRenameHandler.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/BlockEntry.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ListActionTaker.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ListBlobQueue.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/PathInformation.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/RenameAtomicity.java
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/{contracts/services/BlobAppendRequestParameters.java
=> services/RenamePendingJsonFormat.java} (56%)
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/TestBlobListXmlParser.java
rename
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/{ListResultSchemaTest.java
=> TestDfsListJsonParser.java} (93%)
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/RenameAtomicityTestUtils.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsBackoffMetrics.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsReadFooterMetrics.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestListActionTaker.java
copy
hadoop-tools/hadoop-azure/src/{main/java/org/apache/hadoop/fs/azurebfs/utils/IdentityHandler.java
=>
test/java/org/apache/hadoop/fs/azurebfs/utils/AzcopyExecutionException.java}
(56%)
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/AzcopyToolHelper.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/DirectoryStateHelper.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/TestAzcopyToolHelper.java
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-commits-h...@hadoop.apache.org
