This is an automated email from the ASF dual-hosted git repository.
szetszwo pushed a change to branch HDDS-4454
in repository https://gitbox.apache.org/repos/asf/ozone.git.
discard cd5bfd1 HDDS-5705. [Ozone-Streaming] Change ByteBufStreamOutput to
ByteBufferStreamOutput (#2603)
omit 6e60a16 HDDS-5599. [Ozone-Streaming]drop BufferPool and ChunkBuffer
to avoid buffer copying (#2557)
omit 3c719a0 HDDS-5488. [Ozone-Streaming] Add a new
BlockOutputStream/KeyOutputStream to support streaming api (#2495)
omit bbfc4c0 HDDS-5480. [Ozone-Streaming] Client and server should support
stream setup. (#2452)
omit 2972386 HDDS-5481. Fix stream() and link() method in
ContainerStateMachine. (#2451)
omit 76990b4 HDDS-5452. Add link method to ContainerStateMachine for Ratis
streaming (#2422)
omit 06e193b HDDS-5366. [Ozone-Streaming] Implement stream method to
ContainerStateMachine. (#2358). Contributed by mingchao zhao
add 30859fe HDDS-5687. support Optional<T> as parameters of commandLine
(#2588)
add 7dcaad1 HDDS-5681. Speed up TestOzoneManagerHAWithACL (#2587)
add b593fa8 HDDS-5688. Rpc should not retry if exception is
ContainerNotFoundException. (#2589)
add db75046 HDDS-5662. Fix blank traceId for FsShell command (#2568)
add 2958ab3 HDDS-5535. SCM-HA documents configuration error (#2492)
add 33b3f9e HDDS-5669. Fix NSSummaryEndpoint's NPE bug during OM
initialization (#2592)
add aab19a6 HDDS-5368. Add CLI command: ozone admin namespace summary
(#2565)
add 9baca7c HDDS-5692. Combine some tests in TestOzoneManagerHAWithData
to reuse mini-Clusters (#2593)
add 758b8ca HDDS-5689. chooseRandom in NetworkTopology should exclude the
affinity node too (#2590)
add 194de5b HDDS-5680. Fix bug for incomplete SCM ratis log (#2586)
add cdac8bc HDDS-5644. Speed up decommission tests using a background
Mini Cluster provider (#2554)
add fe42640 HDDS-5667. documentation page layout (#2604)
add 730cb66 HDDS-5706. Fix ReplicationManager zero metrics for inflight
actions. (#2605)
add ea1ff1e HDDS-5690. Speed up TestContainerReplication by removing
testSkipDemmissionAndMaintenanceNode (#2591)
add f71474a HDDS-5700. Improve LOG message of decommission progress.
(#2598)
add aa8cfcd HDDS-5709. do not call removeTransactionsFromDB if nothing to
remove (#2608)
add 7e2cdf0 HDDS-5432. Enable downgrade testing after 1.1.0 release.
(#2484)
new f89e1f5 HDDS-5366. [Ozone-Streaming] Implement stream method to
ContainerStateMachine. (#2358). Contributed by mingchao zhao
new ecff9ab HDDS-5452. Add link method to ContainerStateMachine for Ratis
streaming (#2422)
new 1924810 HDDS-5481. Fix stream() and link() method in
ContainerStateMachine. (#2451)
new 2ef30bb HDDS-5480. [Ozone-Streaming] Client and server should support
stream setup. (#2452)
new 04b4c17 HDDS-5488. [Ozone-Streaming] Add a new
BlockOutputStream/KeyOutputStream to support streaming api (#2495)
new 9cf9c4d HDDS-5599. [Ozone-Streaming]drop BufferPool and ChunkBuffer
to avoid buffer copying (#2557)
new 4156bf7 HDDS-5705. [Ozone-Streaming] Change ByteBufStreamOutput to
ByteBufferStreamOutput (#2603)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (cd5bfd1)
\
N -- N -- N refs/heads/HDDS-4454 (4156bf7)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/hadoop/hdds/recon/ReconConfigKeys.java | 10 +
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 5 -
.../org/apache/hadoop/hdds/scm/ha/SCMHAUtils.java | 2 +
.../hadoop/hdds/scm/net/NetworkTopologyImpl.java | 24 +-
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 2 -
.../upgrade/AbstractLayoutVersionManager.java | 158 +++--
.../common/src/main/resources/ozone-default.xml | 9 -
.../hdds/scm/net/TestNetworkTopologyImpl.java | 33 +-
.../apache/hadoop/ozone/HddsDatanodeService.java | 5 +-
.../common/interfaces/ContainerDispatcher.java | 7 +-
.../common/statemachine/DatanodeStateMachine.java | 6 +-
.../states/endpoint/VersionEndpointTask.java | 5 +-
.../container/common/utils/HddsVolumeUtil.java | 72 +-
.../ozone/container/common/volume/HddsVolume.java | 2 +-
.../container/common/volume/MutableVolumeSet.java | 13 -
.../container/keyvalue/KeyValueContainer.java | 12 +-
.../ozone/container/keyvalue/KeyValueHandler.java | 5 +-
.../keyvalue/helpers/KeyValueContainerUtil.java | 7 +-
.../ozone/container/ozoneimpl/ContainerReader.java | 161 ++---
.../ozone/container/ozoneimpl/OzoneContainer.java | 5 +-
.../upgrade/DataNodeUpgradeFinalizer.java | 5 +-
.../upgrade/DatanodeMetadataFeatures.java | 53 --
.../ScmHAFinalizeUpgradeActionDatanode.java | 129 ++++
.../ScmHAFirstUpgradeLayoutChangeAction.java | 45 --
.../upgrade/VersionedDatanodeFeatures.java | 145 ++++
.../hadoop/ozone/container/common/ScmTestMock.java | 9 +
.../ozone/container/common/TestContainerCache.java | 23 +-
.../common/TestKeyValueContainerData.java | 8 +-
.../common/impl/TestContainerDataYaml.java | 6 +-
.../upgrade/TestDatanodeUpgradeToScmHA.java | 741 +++++++++++++++++++++
hadoop-hdds/docs/content/feature/SCM-HA.md | 4 +-
.../docs/themes/ozonedoc/layouts/index.html | 2 +
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 7 +-
.../replication/ReplicationManagerMetrics.java | 44 +-
.../hdds/scm/node/DatanodeAdminMonitorImpl.java | 18 +
.../hdds/scm/server/StorageContainerManager.java | 5 +-
.../TestSCMContainerPlacementRackAware.java | 27 +
.../ozone/container/common/TestEndPoint.java | 4 +-
.../org/apache/hadoop/hdds/cli/OzoneAdmin.java | 7 +
.../dist/src/main/compose/ozone/docker-config | 2 +
hadoop-ozone/dist/src/main/compose/testlib.sh | 16 -
.../dist/src/main/compose/upgrade/README.md | 35 +-
.../compose/upgrade/delete-and-regenerate-data.sh | 53 --
.../dist/src/main/compose/upgrade/delete-data.sh | 36 -
hadoop-ozone/dist/src/main/compose/upgrade/test.sh | 9 +-
.../manual-upgrade/0.5.0-1.1.0/callback.sh | 40 --
.../upgrades/manual-upgrade}/README.md | 9 +-
.../{1.0.0-1.1.0 => 1.1.0-1.2.0}/callback.sh | 17 +-
.../upgrade/upgrades/non-rolling-upgrade/driver.sh | 31 +-
.../dist/src/main/compose/versions/0.5.0.sh | 26 -
.../dist/src/main/compose/versions/1.0.0.sh | 26 -
.../dist/src/main/compose/versions/1.1.0.sh | 26 -
.../dist/src/main/compose/versions/README.md | 20 -
hadoop-ozone/dist/src/main/compose/xcompat/test.sh | 4 -
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 7 +-
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 2 +-
.../hadoop/ozone/MiniOzoneClusterProvider.java | 283 ++++++++
.../hadoop/ozone/TestOzoneConfigurationFields.java | 8 +-
.../ozone/container/TestContainerReplication.java | 47 --
.../container/ozoneimpl/TestOzoneContainer.java | 8 +-
.../ozone/dn/TestDatanodeLayoutUpgradeTool.java | 131 ----
.../hadoop/ozone/om/TestOzoneManagerHAWithACL.java | 24 +-
.../ozone/om/TestOzoneManagerHAWithData.java | 32 +-
.../scm/node/TestDecommissionAndMaintenance.java | 36 +-
...TestScmAdminHA.java => TestNSSummaryAdmin.java} | 58 +-
.../ozone/om/upgrade/OMLayoutVersionManager.java | 3 +-
.../org/apache/hadoop/fs/ozone/OzoneFsShell.java | 4 +-
.../apache/hadoop/ozone/recon/ReconHttpServer.java | 6 +-
.../hadoop/ozone/recon/ReconServerConfigKeys.java | 4 -
.../hadoop/ozone/recon/api/NSSummaryEndpoint.java | 121 ++--
.../hadoop/ozone/recon/api/types/DUResponse.java | 1 +
.../ozone/recon/api/types/ResponseStatus.java | 1 +
.../src/views/diskUsage/diskUsage.tsx | 17 +-
.../ozone/admin/nssummary/DiskUsageSubCommand.java | 214 ++++++
.../admin/nssummary/FileSizeDistSubCommand.java | 129 ++++
.../ozone/admin/nssummary/NSSummaryAdmin.java | 137 ++++
.../ozone/admin/nssummary/NSSummaryCLIUtils.java | 174 +++++
.../admin/nssummary/QuotaUsageSubCommand.java | 117 ++++
.../ozone/admin/nssummary/SummarySubCommand.java | 115 ++++
.../admin/{om => nssummary}/package-info.java | 6 +-
.../apache/hadoop/ozone/debug/DatanodeLayout.java | 110 ---
pom.xml | 2 +-
82 files changed, 2842 insertions(+), 1130 deletions(-)
delete mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/DatanodeMetadataFeatures.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/ScmHAFinalizeUpgradeActionDatanode.java
delete mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/ScmHAFirstUpgradeLayoutChangeAction.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/VersionedDatanodeFeatures.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/upgrade/TestDatanodeUpgradeToScmHA.java
delete mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/delete-and-regenerate-data.sh
delete mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/delete-data.sh
delete mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/0.5.0-1.1.0/callback.sh
copy hadoop-ozone/dist/src/main/compose/{ozonesecure =>
upgrade/upgrades/manual-upgrade}/README.md (66%)
rename
hadoop-ozone/dist/src/main/compose/upgrade/upgrades/non-rolling-upgrade/{1.0.0-1.1.0
=> 1.1.0-1.2.0}/callback.sh (86%)
delete mode 100644 hadoop-ozone/dist/src/main/compose/versions/0.5.0.sh
delete mode 100644 hadoop-ozone/dist/src/main/compose/versions/1.0.0.sh
delete mode 100644 hadoop-ozone/dist/src/main/compose/versions/1.1.0.sh
delete mode 100644 hadoop-ozone/dist/src/main/compose/versions/README.md
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneClusterProvider.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/dn/TestDatanodeLayoutUpgradeTool.java
copy
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/{TestScmAdminHA.java
=> TestNSSummaryAdmin.java} (56%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/nssummary/DiskUsageSubCommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/nssummary/FileSizeDistSubCommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/nssummary/NSSummaryAdmin.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/nssummary/NSSummaryCLIUtils.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/nssummary/QuotaUsageSubCommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/nssummary/SummarySubCommand.java
copy hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/{om =>
nssummary}/package-info.java (88%)
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/DatanodeLayout.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
