This is an automated email from the ASF dual-hosted git repository.
avijayan pushed a change to branch HDDS-3698-upgrade
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.
discard b07927e HDDS-4253. Add LayoutVersion request/response for DN
registration. (#1457)
omit a2788b7 HDDS-4227. Implement a 'Prepare For Upgrade' step in OM that
applies all committed Ratis transactions. (#1430)
omit c6f51ee3 HDDS-4252. Add the current layout versions to DN - SCM proto
payload. (#1432)
omit 86c4c9b HDDS-4143. Implement a factory for OM Requests that returns
an instance based on layout version. (#1405)
omit 75ea5bf HDDS-4174. Add current HDDS layout version to Datanode
heartbeat/registration (#1421)
omit 54e5982 HDDS-4141. Implement Finalize command in Ozone Manager
client. (#1400)
omit 0731098 HDDS-4173. Implement HDDS Version management using the
LayoutVersionManager interface. (#1392)
omit d65827c HDDS-3829. Introduce Layout Feature interface in Ozone.
(#1322)
add ceeca92 HDDS-4201. Improve the performance of OmKeyLocationInfoGroup
(#1381)
add 642d660 HDDS-4186: Adjust RetryPolicy of SCMConnectionManager for
SCM/Recon (#1373)
add f254183 HDDS-3725. Ozone sh volume client support quota option.
(#1233)
add acfef2d HDDS-4205. Disable coverage upload to codecov for pull
requests (#1394)
add 549a1a0 HDDS-4197. Failed to load existing service definition files:
...SubcommandWithParent (#1386)
add 7bf205c Removing an archaic reference to Skaffold in the README and
other little improvements (#1360)
add 157864a HDDS-4208. Fix table rendering and logo display in docs
(#1391)
add b12f6b6 HDDS-4161. Set fs.defaultFS in docker compose cluster config
to OFS (#1362)
add dc49daa HDDS-4198. Compile Ozone with multiple Java versions (#1387)
add 4b325a8 HDDS-4193. Range used by S3 MultipartUpload copy-from-source
should be incusive (#1384)
add ce02172 HDDS-4202. Upgrade ratis to 1.1.0-ea949f1-SNAPSHOT (#1382)
add 53353c0 HDDS-4204. upgrade docker environment does not work with
KEEP_RUNNING=true (#1388)
add 0a490cb HDDS-3441. Enable TestKeyManagerImpl test cases. (#1326)
add d2c0470 HDDS-4213. Log when a datanode has become dead in the
DeadNodeHandler (#1402)
add 9a4cb9e HDDS-3151. TestCloseContainerHandlingByClient
Enable-testMultiBlockW… (#1333)
add 49e3a1a HDDS-4064. Show container verbose info with verbose option
(#1290)
add 22e4288 HDDS-4170 - Fix typo in method description. (#1406)
add 971a36e HDDS-4150. recon.api.TestEndpoints test is flaky (#1396)
add bfa2801 HDDS-4211. [OFS] Better owner and group display for listing
Ozone volumes and buckets (#1397)
add a7c72fb HDDS-4196. Add an endpoint in Recon to query Prometheus
(#1390)
add 4b96d83 HDDS-4228: add field 'num' to ALLOCATE_BLOCK of scm audit
log. (#1413)
add 04ac1ef HDDS-4129. change MAX_QUOTA_IN_BYTES to Long.MAX_VALUE.
(#1337)
add 2134c2e HDDS-4218.Remove test TestRatisManager (#1409)
add ab7481d HDDS-4217.Remove test TestOzoneContainerRatis (#1408)
add 72e3215 HDDS-4119. Improve performance of the BufferPool management
of Ozone client (#1336)
add 48e8e50 HDDS-3927. Rename Ozone OM,DN,SCM runtime options to conform
to naming conventions (#1401)
add 68869d1 HDDS-4155. Directory and filename can end up with same name
in a path. (#1361)
add def697f HDDS-4039. Reduce the number of fields in hdds.proto to
improve performance (#1289)
add 8e71e81 HDDS-4166. Documentation index page redirects to the wrong
address (#1372)
add 045aa71 HDDS-4075. Retry request on different OM on
AccessControlException (#1303)
add e84f5ce HDDS-4210. ResolveBucket during checkAcls fails. (#1398)
add 7beb2d0 HDDS-4053. Volume space: add quotaUsageInBytes and update it
when write and delete key. (#1296)
add 570d34c HDDS-4244. Container deleted wrong replica cause
mis-replicated. (#1423)
add 69c3e0e HDDS-4250. Fix wrong logger name (#1429)
add 079ee7f HDDS-4104. Provide a way to get the default value and key of
java-based-configuration easily (#1369)
add 241de5a HDDS-4241. Support HADOOP_TOKEN_FILE_LOCATION for Ozone token
CLI. (#1422)
add 0da6cfd HDDS-4247. Fixed log4j usage in some places (#1426)
add 410a246 HDDS-4255. Remove unused Ant and Jdiff dependency versions
(#1433)
add 68d1ab0 HDDS-3981. Add more debug level log to XceiverClientGrpc for
debug purpose (#1214)
add ce0c072 HDDS-3102. ozone getconf command should use the GenericCli
parent class (#1410)
add 1e9ff6c HDDS-3947: Sort DNs for client when the key is a file for
#getFileStatus #listStatus APIs (#1385)
add f3a60dc HDDS-4233. Interrupted exeception printed out from
DatanodeStateMachine (#1416)
add b281d62 HDDS-4206. Attempt pipeline creation more frequently in
acceptance tests (#1389)
add 0eceb4c HDDS-2766. security/SecuringDataNodes.md (#1175)
add a78a4b7 HDDS-4254. Bucket space: add usedBytes and update it when
create and delete key. (#1431)
add 8ca694a HDDS-4236. Move "Om*Codec.java" to new project
hadoop-ozone/interface-storage (#1424)
add 8899ff7 HDDS-4324. Add important comment to ListVolumes logic (#1417)
add 261d34d HDDS-3297. Enable TestOzoneClientKeyGenerator. (#1442)
add 64026dd HDDS-2660. Create insight point for datanode container
protocol (#1272)
add 5f1900a HDDS-4270. Add more reusable byteman scripts to debug
ofs/o3fs performance (#1443)
add c955729 HDDS-4194. Create a script to check AWS S3 compatibility
(#1383)
add 7d0d330 HDDS-4282. Improve the emptyDir syntax (#1450)
add f1cdbe7 HDDS-4263. ReplicatiomManager shouldn't consider origin node
Id for CLOSED containers. (#1438)
add 004dd3f HDDS-4102. Normalize Keypath for lookupKey. (#1328)
add 525ecbb HDDS-3727. Volume space: check quotaUsageInBytes when write
key. (#1434)
add 6267a39 HDDS-4231. Background Service blocks on task results. (#1414)
add 34f3b91 HDDS-4232. Use single thread for KeyDeletingService. (#1415)
add 68642c2 HDDS-4023. Delete closed container after all blocks have been
deleted. (#1338)
add 3ad1034 HDDS-4215. Update Freon doc in source tree. (#1403)
add 2420ee8 HDDS-4288. the icon of hadoop-ozone is bigger than ever
(#1452)
add fdcc696 HDDS-4287: Exclude protobuff classes from
ozone-filesystem-hadoop3 jars (#1455). Contributed by Uma Maheswara Rao G.
add d1ac423 HDDS-3751. Ozone sh client support bucket quota option.
(#1412)
add be25991 HDDS-4292. Ozone Client not working with Hadoop Version < 3.2
(#1463)
add d6a1836 HDDS-4251. Update Ratis version to latest snapshot (#1462)
add 275653e HDDS-3869. Use different column families for datanode block
and metadata (#1298)
add 819b455 HDDS-4302 : Shade the org.apache.common.lang3 package as this
is coming from other hadoop packages as well. (#1469). Contributed by Uma
Maheswara Rao G.
add 7216e3c HDDS-3966. Enable TestOMRatisSnapshots. (#1441)
add 55c9df8 HDDS-4290. Enable insight point for SCM heartbeat protocol
(#1453)
add 60d2bcc HDDS-4274. Change the log level of the SCM Delete block to
improve performance. (#1446)
add f8a62d6 HDDS-3810. Add the logic to distribute open containers among
the pipelines of a datanode. (#1274)
add 5719615 HDDS-4304. Close Container event can fail if pipeline is
removed first. (#1471)
add 8cd86a6 HDDS-4299. Display Ratis version with ozone version (#1464)
add cfff097 HDDS-4271. Avoid logging chunk content in Ozone Insight
(#1466)
add 4ad0318 HDDS-4264. Uniform naming conventions of Ozone Shell Options.
(#1447)
add d6d27e4 HDDS-4242. Copy PrefixInfo proto to new project
hadoop-ozone/interface-storage (#1444)
add 19cb481 HDDS-4156. add hierarchical layout to Chinese doc (#1368)
add b6efb95 HDDS-4280. Document notable configurations for Recon. (#1448)
add 0d7d1e2 HDDS-4298. Use an interface in Ozone client instead of
XceiverClientManager (#1460)
add f9b1ca4 HDDS-4310: Ozone getconf broke the compatibility (#1475)
new 3e21d75 HDDS-3829. Introduce Layout Feature interface in Ozone.
(#1322)
new 3fe83ce HDDS-4173. Implement HDDS Version management using the
LayoutVersionManager interface. (#1392)
new 9b55f69 HDDS-4141. Implement Finalize command in Ozone Manager
client. (#1400)
new 95b8712 HDDS-4174. Add current HDDS layout version to Datanode
heartbeat/registration (#1421)
new 6fa5fa1 HDDS-4143. Implement a factory for OM Requests that returns
an instance based on layout version. (#1405)
new b96834f HDDS-4252. Add the current layout versions to DN - SCM proto
payload. (#1432)
new 08d8d36 HDDS-4227. Implement a 'Prepare For Upgrade' step in OM that
applies all committed Ratis transactions. (#1430)
new 8fe8a1f HDDS-4253. Add LayoutVersion request/response for DN
registration. (#1457)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b07927e)
\
N -- N -- N refs/heads/HDDS-3698-upgrade (8fe8a1f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 8 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/post-commit.yml | 20 +-
.../byteman/appendlog.btm | 21 +-
dev-support/byteman/hcfs-read.btm | 67 +++
.../byteman/ratis-flush.btm | 27 +-
.../byteman/ratis-no-flush.btm | 16 +-
.../byteman/watchforcommit.btm | 28 +-
dev-support/byteman/watchforcommit_all.btm | 47 ++
hadoop-hdds/client/pom.xml | 11 +
.../hadoop/hdds/scm/XceiverClientFactory.java | 23 +-
.../apache/hadoop/hdds/scm/XceiverClientGrpc.java | 18 +-
.../hadoop/hdds/scm/XceiverClientManager.java | 40 +-
.../apache/hadoop/hdds/scm/XceiverClientRatis.java | 4 +-
.../hadoop/hdds/scm/client/HddsClientUtils.java | 8 +-
.../hadoop/hdds/scm/storage/BlockInputStream.java | 54 +-
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 143 +++--
.../apache/hadoop/hdds/scm/storage/BufferPool.java | 49 +-
.../hadoop/hdds/scm/storage/CommitWatcher.java | 37 +-
.../storage/TestBlockOutputStreamCorrectness.java | 224 ++++++++
.../hadoop/hdds/scm/storage/TestBufferPool.java} | 37 +-
hadoop-hdds/common/pom.xml | 5 +
hadoop-hdds/common/src/main/conf/hadoop-env.sh | 13 +-
.../org/apache/hadoop/hdds/client/OzoneQuota.java | 240 +++++---
.../hadoop/hdds/protocol/DatanodeDetails.java | 83 ++-
.../RequestTypeDependentRetryPolicyCreator.java | 8 +-
.../apache/hadoop/hdds/recon/ReconConfigKeys.java | 4 +
.../hadoop/hdds/scm/ByteStringConversion.java | 18 +-
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 9 +
.../apache/hadoop/hdds/scm/XceiverClientSpi.java | 4 +-
.../hadoop/hdds/scm/net/NetworkTopologyImpl.java | 2 +-
.../hdds/scm/storage/ContainerProtocolCalls.java | 80 ++-
.../x509/certificate/utils/CertificateCodec.java | 2 +-
.../hadoop/hdds/utils/BackgroundService.java | 73 +--
.../apache/hadoop/hdds/utils/BackgroundTask.java | 4 +-
.../hadoop/hdds/utils/BackgroundTaskQueue.java | 5 +-
.../{VersionInfo.java => RatisVersionInfo.java} | 66 +--
.../org/apache/hadoop/hdds/utils/Scheduler.java | 2 +-
.../org/apache/hadoop/hdds/utils/VersionInfo.java | 9 +-
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 50 +-
.../apache/hadoop/ozone/common/ChunkBuffer.java | 14 +-
.../common/ChunkBufferImplWithByteBuffer.java | 10 +-
.../container/common/helpers/ChunkInfoList.java | 56 ++
.../common/src/main/resources/ozone-default.xml | 17 +
.../hadoop/hdds/protocol/MockDatanodeDetails.java | 2 +-
.../hadoop/hdds/scm/pipeline/MockPipeline.java | 29 +-
.../hadoop/ozone/common/TestChunkBuffer.java | 16 +-
.../hdds/conf/ConfigurationReflectionUtil.java | 46 ++
.../hdds/conf/TestConfigurationReflectionUtil.java | 111 ++++
.../container/common/helpers/ContainerUtils.java | 66 ++-
.../container/common/impl/ContainerDataYaml.java | 3 +
.../container/common/impl/HddsDispatcher.java | 61 +-
.../container/common/interfaces/BlockIterator.java | 5 +-
.../container/common/interfaces/Container.java | 7 -
.../common/statemachine/DatanodeStateMachine.java | 7 +-
.../common/statemachine/SCMConnectionManager.java | 9 +-
.../CloseContainerCommandHandler.java | 2 +-
.../CreatePipelineCommandHandler.java | 2 +-
.../commandhandler/DeleteBlocksCommandHandler.java | 69 ++-
.../states/endpoint/RegisterEndpointTask.java | 6 +-
.../server/ratis/ContainerStateMachine.java | 2 +-
.../transport/server/ratis/XceiverServerRatis.java | 6 +-
.../container/common/utils/ContainerCache.java | 34 +-
.../container/common/utils/HddsVolumeUtil.java | 2 +-
.../container/common/utils/ReferenceCountedDB.java | 10 +-
.../container/keyvalue/KeyValueBlockIterator.java | 156 -----
.../container/keyvalue/KeyValueContainer.java | 29 +-
.../container/keyvalue/KeyValueContainerCheck.java | 12 +-
.../container/keyvalue/KeyValueContainerData.java | 48 +-
.../ozone/container/keyvalue/KeyValueHandler.java | 14 +-
.../container/keyvalue/helpers/BlockUtils.java | 2 +-
.../keyvalue/helpers/KeyValueContainerUtil.java | 194 ++++---
.../container/keyvalue/impl/BlockManagerImpl.java | 65 ++-
.../background/BlockDeletingService.java | 56 +-
.../metadata/AbstractDatanodeDBDefinition.java | 74 +++
.../container/metadata/AbstractDatanodeStore.java | 297 ++++++++++
.../ozone/container/metadata/BlockDataCodec.java | 47 ++
.../container/metadata/ChunkInfoListCodec.java | 45 ++
.../metadata/DatanodeSchemaOneDBDefinition.java | 91 +++
.../metadata/DatanodeSchemaTwoDBDefinition.java | 81 +++
.../ozone/container/metadata/DatanodeStore.java | 94 ++++
.../metadata/DatanodeStoreSchemaOneImpl.java | 49 ++
.../metadata/DatanodeStoreSchemaTwoImpl.java | 44 ++
.../ozone/container/metadata/DatanodeTable.java | 130 +++++
.../metadata/SchemaOneChunkInfoListCodec.java | 68 +++
.../metadata/SchemaOneDeletedBlocksTable.java | 180 ++++++
.../container/metadata/SchemaOneKeyCodec.java | 106 ++++
.../ozone/container/metadata}/package-info.java | 16 +-
.../protocol/StorageContainerDatanodeProtocol.java | 8 +-
...inerDatanodeProtocolClientSideTranslatorPB.java | 16 +-
...inerDatanodeProtocolServerSideTranslatorPB.java | 5 +-
.../hadoop/ozone/container/common/ScmTestMock.java | 12 +-
.../container/common/TestBlockDeletingService.java | 113 +++-
.../ozone/container/common/TestContainerCache.java | 25 +-
.../common/TestKeyValueContainerData.java | 3 +
.../TestSchemaOneBackwardsCompatibility.java | 626 +++++++++++++++++++++
.../common/impl/TestContainerDataYaml.java | 3 +
.../keyvalue/TestKeyValueBlockIterator.java | 340 +++++++----
.../container/keyvalue/TestKeyValueContainer.java | 75 +--
.../keyvalue/TestKeyValueContainerCheck.java | 18 +-
.../container/ozoneimpl/TestContainerReader.java | 43 +-
.../container/ozoneimpl/TestOzoneContainer.java | 14 +-
.../test/resources/123-dn-container.db/000024.sst | Bin 0 -> 1022 bytes
.../test/resources/123-dn-container.db/000026.sst | Bin 0 -> 827 bytes
.../test/resources/123-dn-container.db/000032.sst | Bin 0 -> 896 bytes
.../test/resources/123-dn-container.db/000034.log | 0
.../src/test/resources/123-dn-container.db/CURRENT | 1 +
.../test/resources/123-dn-container.db/IDENTITY | 1 +
.../resources/123-dn-container.db/MANIFEST-000033 | Bin 0 -> 297 bytes
.../resources/123-dn-container.db/OPTIONS-000033 | 165 ++++++
.../resources/123-dn-container.db/OPTIONS-000036 | 165 ++++++
.../src/test/resources/123.container | 10 +
hadoop-hdds/docs/content/_index.md | 2 +-
hadoop-hdds/docs/content/concept/Datanodes.zh.md | 3 +
hadoop-hdds/docs/content/concept/Overview.zh.md | 7 +-
hadoop-hdds/docs/content/concept/OzoneManager.md | 8 +-
.../docs/content/concept/OzoneManager.zh.md | 3 +
.../content/concept/StorageContainerManager.md | 6 +-
.../content/concept/StorageContainerManager.zh.md | 3 +
hadoop-hdds/docs/content/concept/_index.zh.md | 2 +-
hadoop-hdds/docs/content/feature/Observability.md | 4 +-
hadoop-hdds/docs/content/feature/Recon.md | 18 +-
hadoop-hdds/docs/content/interface/CSI.zh.md | 3 +
hadoop-hdds/docs/content/interface/JavaApi.zh.md | 3 +
hadoop-hdds/docs/content/interface/O3fs.zh.md | 8 +-
hadoop-hdds/docs/content/interface/S3.zh.md | 3 +
.../docs/content/security/SecureOzone.zh.md | 6 +-
.../docs/content/security/SecuringDatanodes.md | 34 +-
.../docs/content/security/SecuringDatanodes.zh.md | 53 ++
hadoop-hdds/docs/content/security/SecuringS3.zh.md | 3 +
.../docs/content/security/SecuringTDE.zh.md | 3 +
.../docs/content/security/SecurityAcls.zh.md | 3 +
.../docs/content/security/SecurityWithRanger.zh.md | 3 +
hadoop-hdds/docs/content/tools/TestTools.md | 14 +-
hadoop-hdds/docs/content/tools/TestTools.zh.md | 14 +-
.../themes/ozonedoc/layouts/_default/single.html | 2 +-
.../ozonedoc/layouts/partials/languages.html | 3 +-
.../themes/ozonedoc/layouts/partials/navbar.html | 6 +-
.../themes/ozonedoc/layouts/partials/sidebar.html | 4 +-
.../docs/themes/ozonedoc/static/css/ozonedoc.css | 20 +-
.../x509/certificate/authority/BaseApprover.java | 2 +-
.../server/OzoneProtocolMessageDispatcher.java | 41 +-
.../apache/hadoop/hdds/utils/HddsServerUtil.java | 13 +
.../hadoop/hdds/utils/MetadataKeyFilters.java | 42 +-
.../apache/hadoop/hdds/utils/db/DBDefinition.java | 14 +-
.../org/apache/hadoop/hdds/utils/db/DBStore.java | 8 +-
.../hadoop/hdds/utils/db/DBStoreBuilder.java | 41 +-
.../org/apache/hadoop/hdds/utils/db/RDBStore.java | 17 +-
.../org/apache/hadoop/hdds/utils/db/RDBTable.java | 92 +++
.../org/apache/hadoop/hdds/utils/db/Table.java | 56 ++
.../apache/hadoop/hdds/utils/db/TypedTable.java | 46 ++
.../apache/hadoop/hdds/server/TestJsonUtils.java | 5 +-
.../src/main/proto/DatanodeClientProtocol.proto | 4 +
.../interface-client/src/main/proto/hdds.proto | 17 +-
.../interface-client/src/main/resources/proto.lock | 40 +-
.../proto/ScmServerDatanodeHeartbeatProtocol.proto | 2 +-
.../interface-server/src/main/resources/proto.lock | 4 +-
hadoop-hdds/pom.xml | 2 +
.../hdds/scm/block/SCMBlockDeletingService.java | 3 +-
.../container/AbstractContainerReportHandler.java | 51 +-
.../hdds/scm/container/ContainerReportHandler.java | 14 +-
.../IncrementalContainerReportHandler.java | 2 +-
.../hdds/scm/container/ReplicationManager.java | 131 ++++-
.../hdds/scm/container/SCMContainerManager.java | 33 +-
.../apache/hadoop/hdds/scm/node/DatanodeInfo.java | 33 ++
.../hadoop/hdds/scm/node/DeadNodeHandler.java | 2 +-
.../apache/hadoop/hdds/scm/node/NodeManager.java | 6 +-
.../hadoop/hdds/scm/node/NodeStateManager.java | 30 +-
.../hadoop/hdds/scm/node/SCMNodeManager.java | 53 +-
.../scm/pipeline/BackgroundPipelineCreator.java | 2 +-
.../hadoop/hdds/scm/pipeline/PipelineManager.java | 4 +-
.../hdds/scm/pipeline/PipelinePlacementPolicy.java | 7 +-
.../hdds/scm/pipeline/PipelineReportHandler.java | 2 +-
.../hdds/scm/pipeline/RatisPipelineProvider.java | 6 +-
.../hdds/scm/pipeline/RatisPipelineUtils.java | 2 +-
.../hdds/scm/pipeline/SCMPipelineManager.java | 16 +-
.../SCMSecurityProtocolServerSideTranslatorPB.java | 17 +-
...lockLocationProtocolServerSideTranslatorPB.java | 2 +-
...inerLocationProtocolServerSideTranslatorPB.java | 17 +-
.../hdds/scm/safemode/ContainerSafeModeRule.java | 4 +-
.../hdds/scm/server/SCMBlockProtocolServer.java | 7 +-
.../hdds/scm/server/SCMDatanodeProtocolServer.java | 4 +-
.../java/org/apache/hadoop/hdds/scm/TestUtils.java | 1 +
.../hadoop/hdds/scm/block/TestBlockManager.java | 69 +++
.../hadoop/hdds/scm/container/MockNodeManager.java | 28 +-
.../scm/container/TestContainerReportHandler.java | 33 ++
.../hdds/scm/container/TestReplicationManager.java | 92 +++
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 4 +-
.../scm/pipeline/TestPipelinePlacementPolicy.java | 4 +-
.../hdds/scm/pipeline/TestSCMPipelineManager.java | 74 +++
.../ozone/container/common/TestEndPoint.java | 2 +-
.../testutils/ReplicationNodeManagerMock.java | 12 +-
.../hdds/scm/cli/container/InfoSubcommand.java | 14 +-
.../scm/cli/pipeline/CreatePipelineSubcommand.java | 10 +-
.../org/apache/hadoop/ozone/client/BucketArgs.java | 43 +-
.../apache/hadoop/ozone/client/ObjectStore.java | 3 +
.../apache/hadoop/ozone/client/OzoneBucket.java | 93 +++
.../apache/hadoop/ozone/client/OzoneVolume.java | 103 +++-
.../org/apache/hadoop/ozone/client/VolumeArgs.java | 45 +-
.../ozone/client/io/BlockOutputStreamEntry.java | 22 +-
.../client/io/BlockOutputStreamEntryPool.java | 29 +-
.../hadoop/ozone/client/io/KeyInputStream.java | 36 +-
.../hadoop/ozone/client/io/KeyOutputStream.java | 60 +-
.../ozone/client/protocol/ClientProtocol.java | 17 +-
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 97 +++-
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 26 +
.../apache/hadoop/ozone/freon/OzoneGetConf.java | 278 ---------
.../hadoop/ozone/om/exceptions/OMException.java | 4 +-
.../ozone/om/exceptions/OMNotLeaderException.java | 2 +-
.../ozone/om/ha/OMFailoverProxyProvider.java | 237 ++++++--
.../hadoop/ozone/om/helpers/OmBucketArgs.java | 50 +-
.../hadoop/ozone/om/helpers/OmBucketInfo.java | 78 ++-
.../ozone/om/helpers/OmKeyLocationInfoGroup.java | 8 +-
.../hadoop/ozone/om/helpers/OmVolumeArgs.java | 64 ++-
.../ozone/om/protocol/OzoneManagerProtocol.java | 6 +-
.../ozone/om/protocolPB/Hadoop3OmTransport.java | 175 +-----
...OzoneManagerProtocolClientSideTranslatorPB.java | 12 +-
.../ozone/security/OzoneTokenIdentifier.java | 5 +-
.../apache/hadoop/ozone/util/OzoneVersionInfo.java | 15 +-
.../hadoop/ozone/om/helpers/TestOmVolumeArgs.java | 4 +-
hadoop-ozone/dev-support/checks/build.sh | 2 +-
hadoop-ozone/dist/README.md | 52 +-
.../dist/src/main/compose/ozone-csi/docker-config | 3 +
.../dist/src/main/compose/ozone-ha/docker-config | 2 +-
.../dist/src/main/compose/ozone-mr/common-config | 3 +-
.../src/main/compose/ozone-om-ha-s3/docker-config | 4 +-
.../dist/src/main/compose/ozone-om-ha-s3/test.sh | 2 +
.../src/main/compose/ozone-om-ha/docker-config | 3 +-
.../src/main/compose/ozone-topology/docker-config | 7 +-
.../dist/src/main/compose/ozone/docker-config | 3 +
.../src/main/compose/ozoneblockade/docker-config | 2 +
.../src/main/compose/ozones3-haproxy/docker-config | 3 +
.../src/main/compose/ozonesecure-mr/docker-config | 3 +-
.../main/compose/ozonesecure-om-ha/docker-config | 6 +-
.../src/main/compose/ozonesecure-om-ha/test.sh | 2 +
.../src/main/compose/ozonesecure/docker-config | 3 +-
hadoop-ozone/dist/src/main/compose/testlib.sh | 8 +-
.../dist/src/main/compose/upgrade/docker-config | 1 +
hadoop-ozone/dist/src/main/compose/upgrade/test.sh | 3 +-
.../main/k8s/definitions/ozone/freon/freon.yaml | 2 +-
.../getting-started/freon/freon-deployment.yaml | 2 +-
.../examples/minikube/freon/freon-deployment.yaml | 2 +-
.../ozone-dev/csi/csi-provisioner-deployment.yaml | 2 +-
.../examples/ozone-dev/freon/freon-deployment.yaml | 2 +-
.../ozone/csi/csi-provisioner-deployment.yaml | 2 +-
.../k8s/examples/ozone/freon/freon-deployment.yaml | 2 +-
.../src/main/smoketest/admincli/container.robot | 5 +
.../main/smoketest/auditparser/auditparser.robot | 2 +-
.../dist/src/main/smoketest/basic/basic.robot | 2 +-
.../loaddata.robot => basic/getconf.robot} | 20 +-
.../src/main/smoketest/basic/ozone-shell-lib.robot | 53 +-
.../main/smoketest/basic/ozone-shell-single.robot | 2 +-
.../src/main/smoketest/basic/ozone-shell.robot | 2 +-
.../dist/src/main/smoketest/createbucketenv.robot | 2 +-
.../dist/src/main/smoketest/createmrenv.robot | 2 +-
.../src/main/smoketest/debug/ozone-debug.robot | 2 +-
.../dist/src/main/smoketest/freon/freon.robot | 2 +-
.../dist/src/main/smoketest/gdpr/gdpr.robot | 2 +-
.../dist/src/main/smoketest/ozonefs/setup.robot | 8 +-
.../dist/src/main/smoketest/recon/recon-api.robot | 2 +-
.../src/main/smoketest/s3/MultipartUpload.robot | 130 ++---
.../dist/src/main/smoketest/s3/commonawslib.robot | 5 +
.../dist/src/main/smoketest/s3/objectcopy.robot | 23 +-
.../dist/src/main/smoketest/s3/objectdelete.robot | 28 +-
.../src/main/smoketest/s3/objectmultidelete.robot | 24 +-
.../dist/src/main/smoketest/s3/objectputget.robot | 40 +-
.../src/main/smoketest/s3/s3_compatbility_check.sh | 47 ++
.../smoketest/security/ozone-secure-token.robot | 16 +-
.../dist/src/main/smoketest/spnego/web.robot | 2 +-
.../src/main/smoketest/topology/loaddata.robot | 2 +-
hadoop-ozone/dist/src/shell/ozone/ozone | 15 +-
hadoop-ozone/dist/src/shell/ozone/stop-ozone.sh | 8 +-
.../hadoop/ozone/TestMiniChaosOzoneCluster.java | 40 +-
.../src/test/blockade/ozone/client.py | 10 +-
.../hadoop/ozone/insight/BaseInsightPoint.java | 34 +-
.../ozone/insight/BaseInsightSubCommand.java | 6 +
.../apache/hadoop/ozone/insight/InsightPoint.java | 4 +-
.../hadoop/ozone/insight/MetricGroupDisplay.java | 4 +-
.../hadoop/ozone/insight/MetricsSubCommand.java | 33 +-
.../datanode/DatanodeDispatcherInsight.java | 107 ++++
.../insight/datanode/PipelineComponentUtil.java | 78 +++
.../ozone/insight/datanode/RatisInsight.java | 45 +-
.../hadoop/ozone/insight/om/KeyManagerInsight.java | 2 +-
.../hadoop/ozone/insight/om/OmProtocolInsight.java | 2 +-
.../ozone/insight/scm/NodeManagerInsight.java | 2 +-
.../ozone/insight/scm/ReplicaManagerInsight.java | 2 +-
.../scm/ScmProtocolBlockLocationInsight.java | 2 +-
.../scm/ScmProtocolContainerLocationInsight.java | 2 +-
.../insight/scm/ScmProtocolDatanodeInsight.java | 2 +-
.../insight/scm/ScmProtocolSecurityInsight.java | 2 +-
.../fs/ozone/TestOzoneFSWithObjectStoreCreate.java | 160 ++++++
.../hadoop/fs/ozone/TestOzoneFileSystem.java | 6 +-
.../hadoop/fs/ozone/TestRootedOzoneFileSystem.java | 18 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 4 +-
.../ozone/TestStorageContainerManagerHelper.java | 31 +-
.../rpc/TestBlockOutputStreamWithFailures.java | 4 +-
...estBlockOutputStreamWithFailuresFlushDelay.java | 4 +-
.../rpc/TestCloseContainerHandlingByClient.java | 55 --
.../hadoop/ozone/client/rpc/TestCommitWatcher.java | 39 +-
.../rpc/TestContainerStateMachineFailures.java | 20 +-
.../client/rpc/TestDiscardPreallocatedBlocks.java | 186 ++++++
.../rpc/TestOzoneClientRetriesOnException.java | 2 +-
...estOzoneClientRetriesOnExceptionFlushDelay.java | 2 +-
.../client/rpc/TestOzoneRpcClientAbstract.java | 560 +++++++++++++++++-
.../hadoop/ozone/client/rpc/TestReadRetries.java | 40 +-
.../client/rpc/TestValidateBCSIDOnRestart.java | 8 +-
.../ozone/client/rpc/TestWatchForCommit.java | 2 +-
.../commandhandler/TestBlockDeletion.java | 92 ++-
.../ozoneimpl/TestOzoneContainerRatis.java | 138 -----
.../container/ozoneimpl/TestRatisManager.java | 124 ----
.../ozone/freon/TestOzoneClientKeyGenerator.java | 2 -
.../hadoop/ozone/fsck/TestContainerMapper.java | 3 +
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 18 +-
.../hadoop/ozone/om/TestOMRatisSnapshots.java | 2 -
.../src/main/proto/OmClientProtocol.proto | 11 +
hadoop-ozone/interface-storage/pom.xml | 96 ++++
.../apache/hadoop/ozone/om/OMMetadataManager.java | 0
.../ozone/om/codec/OMTransactionInfoCodec.java | 0
.../hadoop/ozone/om/codec/OmBucketInfoCodec.java | 0
.../hadoop/ozone/om/codec/OmKeyInfoCodec.java | 0
.../ozone/om/codec/OmMultipartKeyInfoCodec.java | 0
.../hadoop/ozone/om/codec/OmPrefixInfoCodec.java | 5 +-
.../hadoop/ozone/om/codec/OmVolumeArgsCodec.java | 0
.../ozone/om/codec/RepeatedOmKeyInfoCodec.java | 0
.../hadoop/ozone/om/codec/S3SecretValueCodec.java | 0
.../ozone/om/codec/TokenIdentifierCodec.java | 0
.../hadoop/ozone/om/codec/UserVolumeInfoCodec.java | 0
.../apache/hadoop/ozone/om/codec/package-info.java | 2 +-
.../hadoop/ozone/om/helpers/OmPrefixInfo.java | 13 +-
.../hadoop/ozone/om/helpers/OzoneAclStorage.java | 63 +++
.../ozone/om/helpers/OzoneAclStorageUtil.java | 62 ++
.../hadoop/ozone/om/helpers}/package-info.java | 4 +-
.../org/apache/hadoop/ozone/om}/package-info.java | 4 +-
.../hadoop/ozone/om/ratis/OMTransactionInfo.java | 2 +-
.../hadoop/ozone/om/ratis}/package-info.java | 4 +-
.../src/main/proto/OmStorageProtocol.proto | 60 ++
.../ozone/om/codec/TestOMTransactionInfoCodec.java | 0
.../hadoop/ozone/om/codec/TestOmKeyInfoCodec.java | 0
.../om/codec/TestOmMultipartKeyInfoCodec.java | 0
.../ozone/om/codec/TestOmPrefixInfoCodec.java | 0
.../ozone/om/codec/TestRepeatedOmKeyInfoCodec.java | 0
.../ozone/om/codec/TestS3SecretValueCodec.java | 0
.../apache/hadoop/ozone/om/codec/package-info.java | 0
.../hadoop/ozone/om/helpers/TestOmPrefixInfo.java | 0
.../hadoop/ozone/om/helpers}/package-info.java | 4 +-
hadoop-ozone/ozone-manager/pom.xml | 5 +
.../apache/hadoop/ozone/om/BucketManagerImpl.java | 4 +-
.../apache/hadoop/ozone/om/KeyDeletingService.java | 9 +-
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 81 ++-
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 3 +-
.../hadoop/ozone/om/OpenKeyCleanupService.java | 3 +-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 83 ++-
.../org/apache/hadoop/ozone/om/VolumeManager.java | 9 -
.../apache/hadoop/ozone/om/VolumeManagerImpl.java | 39 --
.../apache/hadoop/ozone/om/codec/package-info.java | 3 +
.../apache/hadoop/ozone/om/fs/OzoneManagerFS.java | 53 +-
.../ozone/om/ratis/OzoneManagerRatisServer.java | 6 +-
.../ozone/om/ratis/OzoneManagerStateMachine.java | 2 +-
.../hadoop/ozone/om/request/OMClientRequest.java | 18 +-
.../om/request/bucket/OMBucketCreateRequest.java | 37 ++
.../request/bucket/OMBucketSetPropertyRequest.java | 64 +++
.../bucket/acl/OMBucketRemoveAclRequest.java | 2 +-
.../request/bucket/acl/OMBucketSetAclRequest.java | 2 +-
.../ozone/om/request/file/OMFileCreateRequest.java | 43 +-
.../om/request/key/OMAllocateBlockRequest.java | 45 +-
.../ozone/om/request/key/OMKeyCommitRequest.java | 48 +-
.../ozone/om/request/key/OMKeyCreateRequest.java | 48 +-
.../ozone/om/request/key/OMKeyDeleteRequest.java | 33 +-
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 76 ++-
.../ozone/om/request/key/OMKeysDeleteRequest.java | 26 +-
.../ozone/om/request/key/OMKeysRenameRequest.java | 2 +-
.../om/request/key/OMTrashRecoverRequest.java | 2 +-
.../om/request/key/acl/OMKeyRemoveAclRequest.java | 2 +-
.../om/request/key/acl/OMKeySetAclRequest.java | 2 +-
.../key/acl/prefix/OMPrefixRemoveAclRequest.java | 2 +-
.../key/acl/prefix/OMPrefixSetAclRequest.java | 2 +-
.../multipart/S3MultipartUploadAbortRequest.java | 41 +-
.../S3MultipartUploadCommitPartRequest.java | 22 +-
.../S3MultipartUploadCompleteRequest.java | 10 +
.../om/request/volume/OMVolumeSetQuotaRequest.java | 54 +-
.../om/response/file/OMFileCreateResponse.java | 10 +-
.../om/response/key/OMAllocateBlockResponse.java | 18 +-
.../ozone/om/response/key/OMKeyCommitResponse.java | 18 +-
.../ozone/om/response/key/OMKeyCreateResponse.java | 20 +-
.../ozone/om/response/key/OMKeyDeleteResponse.java | 18 +-
.../om/response/key/OMKeysDeleteResponse.java | 20 +-
.../multipart/S3MultipartUploadAbortResponse.java | 20 +-
.../S3MultipartUploadCommitPartResponse.java | 18 +-
...OzoneManagerProtocolServerSideTranslatorPB.java | 4 +-
.../OzoneDelegationTokenSecretManager.java | 2 +-
.../hadoop/ozone/om/failover/TestOMFailovers.java | 151 +++++
.../ozone/om/request/TestOMRequestUtils.java | 57 +-
.../bucket/TestOMBucketSetPropertyRequest.java | 41 +-
.../request/file/TestOMDirectoryCreateRequest.java | 4 +-
.../ozone/om/request/key/TestOMKeyRequest.java | 7 +-
.../s3/multipart/TestS3MultipartRequest.java | 4 +-
.../volume/TestOMVolumeSetOwnerRequest.java | 2 +-
.../volume/TestOMVolumeSetQuotaRequest.java | 69 ++-
.../response/key/TestOMAllocateBlockResponse.java | 27 +-
.../om/response/key/TestOMKeyCommitResponse.java | 19 +-
.../om/response/key/TestOMKeyCreateResponse.java | 23 +-
.../om/response/key/TestOMKeyDeleteResponse.java | 27 +-
.../om/response/key/TestOMKeysDeleteResponse.java | 30 +-
.../s3/multipart/TestS3MultipartResponse.java | 7 +-
.../TestS3MultipartUploadAbortResponse.java | 21 +-
.../ozone/BasicRootedOzoneClientAdapterImpl.java | 41 +-
hadoop-ozone/ozonefs-shaded/pom.xml | 5 +
hadoop-ozone/pom.xml | 9 +-
.../ozone/recon/MetricsServiceProviderFactory.java | 86 +++
.../hadoop/ozone/recon/ReconControllerModule.java | 1 +
.../hadoop/ozone/recon/ReconSchemaManager.java | 2 +-
.../hadoop/ozone/recon/ReconServerConfigKeys.java | 10 +
.../org/apache/hadoop/ozone/recon/ReconUtils.java | 15 +-
.../ozone/recon/api/MetricsProxyEndpoint.java | 118 ++++
.../hadoop/ozone/recon/api/PipelineEndpoint.java | 59 +-
.../ozone/recon/codec/DatanodeDetailsCodec.java | 5 +-
.../apache/hadoop/ozone/recon/metrics/Metric.java | 51 ++
.../ReconIncrementalContainerReportHandler.java | 4 +-
.../ozone/recon/spi/MetricsServiceProvider.java | 60 ++
.../spi/impl/OzoneManagerServiceProviderImpl.java | 2 +-
.../spi/impl/PrometheusServiceProviderImpl.java | 212 +++++++
.../hadoop/ozone/recon/ReconTestInjector.java | 3 +
.../apache/hadoop/ozone/recon/TestReconUtils.java | 6 +-
.../hadoop/ozone/recon/api/TestEndpoints.java | 119 +++-
.../impl/TestOzoneManagerServiceProviderImpl.java | 9 +-
.../test/resources/prometheus-test-response.txt | 21 +
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 24 +-
.../hadoop/ozone/client/ObjectStoreStub.java | 5 +-
.../hadoop/ozone/client/OzoneVolumeStub.java | 6 +-
.../s3/endpoint/TestMultipartUploadWithCopy.java | 15 +-
.../org/apache/hadoop/ozone/conf/OzoneGetConf.java | 86 +++
.../ozone/conf/OzoneManagersCommandHandler.java | 53 ++
.../ozone/conf/PrintConfKeyCommandHandler.java | 51 ++
.../StorageContainerManagersCommandHandler.java | 52 ++
.../org/apache/hadoop/ozone/conf/package-info.java | 11 +-
.../hadoop/ozone/freon/HadoopDirTreeGenerator.java | 15 +-
.../ozone/freon/HadoopNestedDirGenerator.java | 5 +-
.../hadoop/ozone/freon/RandomKeyGenerator.java | 53 +-
.../ozone/shell/ClearSpaceQuotaOptions.java} | 29 +-
.../hadoop/ozone/shell/SetSpaceQuotaOptions.java} | 30 +-
.../hadoop/ozone/shell/bucket/BucketCommands.java | 4 +-
.../ClearQuotaHandler.java} | 32 +-
.../ozone/shell/bucket/CreateBucketHandler.java | 13 +
.../hadoop/ozone/shell/bucket/SetQuotaHandler.java | 62 ++
.../hadoop/ozone/shell/token/GetTokenHandler.java | 8 +-
.../ozone/shell/token/PrintTokenHandler.java | 3 +-
.../ozone/shell/token/RenewTokenHandler.java | 4 +-
.../hadoop/ozone/shell/token/TokenOption.java | 38 +-
.../ClearQuotaHandler.java} | 30 +-
.../ozone/shell/volume/CreateVolumeHandler.java | 18 +-
...dateVolumeHandler.java => SetQuotaHandler.java} | 47 +-
.../ozone/shell/volume/UpdateVolumeHandler.java | 11 -
.../hadoop/ozone/shell/volume/VolumeCommands.java | 4 +-
.../hadoop/ozone/conf/TestGetConfOptions.java | 90 +++
pom.xml | 22 +-
453 files changed, 11235 insertions(+), 3415 deletions(-)
copy hadoop-ozone/dist/src/main/smoketest/basic/ozone-shell-single.robot =>
dev-support/byteman/appendlog.btm (66%)
create mode 100644 dev-support/byteman/hcfs-read.btm
copy hadoop-ozone/dist/src/main/smoketest/basic/ozone-shell-single.robot =>
dev-support/byteman/ratis-flush.btm (57%)
copy hadoop-ozone/dev-support/checks/build.sh =>
dev-support/byteman/ratis-no-flush.btm (77%)
mode change 100755 => 100644
copy hadoop-ozone/dist/src/main/smoketest/basic/ozone-shell-single.robot =>
dev-support/byteman/watchforcommit.btm (56%)
create mode 100644 dev-support/byteman/watchforcommit_all.btm
rename
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/freon/package-info.java
=>
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientFactory.java
(60%)
create mode 100644
hadoop-hdds/client/src/test/java/org/apache/hadoop/hdds/scm/storage/TestBlockOutputStreamCorrectness.java
copy
hadoop-hdds/{framework/src/test/java/org/apache/hadoop/hdds/server/TestJsonUtils.java
=>
client/src/test/java/org/apache/hadoop/hdds/scm/storage/TestBufferPool.java}
(55%)
copy
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/{VersionInfo.java
=> RatisVersionInfo.java} (50%)
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ChunkInfoList.java
create mode 100644
hadoop-hdds/config/src/test/java/org/apache/hadoop/hdds/conf/TestConfigurationReflectionUtil.java
delete mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueBlockIterator.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/AbstractDatanodeDBDefinition.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/AbstractDatanodeStore.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/BlockDataCodec.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/ChunkInfoListCodec.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeSchemaOneDBDefinition.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeSchemaTwoDBDefinition.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeStore.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeStoreSchemaOneImpl.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeStoreSchemaTwoImpl.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeTable.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/SchemaOneChunkInfoListCodec.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/SchemaOneDeletedBlocksTable.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/SchemaOneKeyCodec.java
copy
{hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/codec =>
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata}/package-info.java
(76%)
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestSchemaOneBackwardsCompatibility.java
create mode 100644
hadoop-hdds/container-service/src/test/resources/123-dn-container.db/000024.sst
create mode 100644
hadoop-hdds/container-service/src/test/resources/123-dn-container.db/000026.sst
create mode 100644
hadoop-hdds/container-service/src/test/resources/123-dn-container.db/000032.sst
create mode 100644
hadoop-hdds/container-service/src/test/resources/123-dn-container.db/000034.log
create mode 100644
hadoop-hdds/container-service/src/test/resources/123-dn-container.db/CURRENT
create mode 100644
hadoop-hdds/container-service/src/test/resources/123-dn-container.db/IDENTITY
create mode 100644
hadoop-hdds/container-service/src/test/resources/123-dn-container.db/MANIFEST-000033
create mode 100644
hadoop-hdds/container-service/src/test/resources/123-dn-container.db/OPTIONS-000033
create mode 100644
hadoop-hdds/container-service/src/test/resources/123-dn-container.db/OPTIONS-000036
create mode 100644
hadoop-hdds/container-service/src/test/resources/123.container
create mode 100644 hadoop-hdds/docs/content/security/SecuringDatanodes.zh.md
delete mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/freon/OzoneGetConf.java
copy hadoop-ozone/dist/src/main/smoketest/{topology/loaddata.robot =>
basic/getconf.robot} (64%)
create mode 100755
hadoop-ozone/dist/src/main/smoketest/s3/s3_compatbility_check.sh
create mode 100644
hadoop-ozone/insight/src/main/java/org/apache/hadoop/ozone/insight/datanode/DatanodeDispatcherInsight.java
create mode 100644
hadoop-ozone/insight/src/main/java/org/apache/hadoop/ozone/insight/datanode/PipelineComponentUtil.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestDiscardPreallocatedBlocks.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerRatis.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestRatisManager.java
create mode 100644 hadoop-ozone/interface-storage/pom.xml
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/OMMetadataManager.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/codec/OMTransactionInfoCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/codec/OmBucketInfoCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/codec/OmKeyInfoCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/codec/OmMultipartKeyInfoCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/codec/OmPrefixInfoCodec.java
(91%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/codec/OmVolumeArgsCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/codec/RepeatedOmKeyInfoCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/codec/S3SecretValueCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/codec/TokenIdentifierCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/codec/UserVolumeInfoCodec.java
(100%)
copy hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/codec/package-info.java
(95%)
rename hadoop-ozone/{common =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/helpers/OmPrefixInfo.java
(92%)
create mode 100644
hadoop-ozone/interface-storage/src/main/java/org/apache/hadoop/ozone/om/helpers/OzoneAclStorage.java
create mode 100644
hadoop-ozone/interface-storage/src/main/java/org/apache/hadoop/ozone/om/helpers/OzoneAclStorageUtil.java
copy
hadoop-ozone/{ozone-manager/src/test/java/org/apache/hadoop/ozone/om/codec =>
interface-storage/src/main/java/org/apache/hadoop/ozone/om/helpers}/package-info.java
(91%)
copy
hadoop-ozone/{ozone-manager/src/test/java/org/apache/hadoop/ozone/om/codec =>
interface-storage/src/main/java/org/apache/hadoop/ozone/om}/package-info.java
(92%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/main/java/org/apache/hadoop/ozone/om/ratis/OMTransactionInfo.java
(100%)
copy
hadoop-ozone/{ozone-manager/src/test/java/org/apache/hadoop/ozone/om/codec =>
interface-storage/src/main/java/org/apache/hadoop/ozone/om/ratis}/package-info.java
(92%)
create mode 100644
hadoop-ozone/interface-storage/src/main/proto/OmStorageProtocol.proto
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/test/java/org/apache/hadoop/ozone/om/codec/TestOMTransactionInfoCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/test/java/org/apache/hadoop/ozone/om/codec/TestOmKeyInfoCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/test/java/org/apache/hadoop/ozone/om/codec/TestOmMultipartKeyInfoCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/test/java/org/apache/hadoop/ozone/om/codec/TestOmPrefixInfoCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/test/java/org/apache/hadoop/ozone/om/codec/TestRepeatedOmKeyInfoCodec.java
(100%)
rename hadoop-ozone/{ozone-manager =>
interface-storage}/src/test/java/org/apache/hadoop/ozone/om/codec/TestS3SecretValueCodec.java
(100%)
copy hadoop-ozone/{ozone-manager =>
interface-storage}/src/test/java/org/apache/hadoop/ozone/om/codec/package-info.java
(100%)
rename hadoop-ozone/{common =>
interface-storage}/src/test/java/org/apache/hadoop/ozone/om/helpers/TestOmPrefixInfo.java
(100%)
copy
hadoop-ozone/{ozone-manager/src/test/java/org/apache/hadoop/ozone/om/codec =>
interface-storage/src/test/java/org/apache/hadoop/ozone/om/helpers}/package-info.java
(92%)
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/failover/TestOMFailovers.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/MetricsServiceProviderFactory.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/MetricsProxyEndpoint.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/metrics/Metric.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/MetricsServiceProvider.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/PrometheusServiceProviderImpl.java
create mode 100644
hadoop-ozone/recon/src/test/resources/prometheus-test-response.txt
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/conf/OzoneGetConf.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/conf/OzoneManagersCommandHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/conf/PrintConfKeyCommandHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/conf/StorageContainerManagersCommandHandler.java
copy
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/BackgroundTask.java
=>
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/conf/package-info.java
(79%)
rename
hadoop-ozone/{ozone-manager/src/test/java/org/apache/hadoop/ozone/om/codec/package-info.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/ClearSpaceQuotaOptions.java}
(60%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/om/codec/package-info.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/SetSpaceQuotaOptions.java}
(55%)
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/{token/RenewTokenHandler.java
=> bucket/ClearQuotaHandler.java} (57%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/SetQuotaHandler.java
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/{token/RenewTokenHandler.java
=> volume/ClearQuotaHandler.java} (59%)
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/volume/{UpdateVolumeHandler.java
=> SetQuotaHandler.java} (58%)
create mode 100644
hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/conf/TestGetConfOptions.java
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
