[
https://issues.apache.org/jira/browse/HDDS-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen O'Donnell resolved HDDS-4131.
-------------------------------------
Fix Version/s: 1.1.0
Resolution: Fixed
> Container report should update container key count and bytes used if they
> differ in SCM
> ---------------------------------------------------------------------------------------
>
> Key: HDDS-4131
> URL: https://issues.apache.org/jira/browse/HDDS-4131
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: SCM
> Affects Versions: 1.1.0
> Reporter: Stephen O'Donnell
> Assignee: Stephen O'Donnell
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> In HDDS-4037 it was noted that when blocks are deleted from closed
> containers, the bytesUsed and Key Count metrics on the SCM container are not
> updated correctly.
> These stats should be updated via the container reports issued by the DNs to
> SCM periodically. However, in
> `AbstractContainerReportHandler#updateContainerStats`, the code assumes the
> values are always increasing and it will not update them if they are
> decreasing:
> {code}
> private void updateContainerStats(final ContainerID containerId,
> final ContainerReplicaProto replicaProto)
> throws ContainerNotFoundException {
> if (isHealthy(replicaProto::getState)) {
> final ContainerInfo containerInfo = containerManager
> .getContainer(containerId);
> if (containerInfo.getSequenceId() <
> replicaProto.getBlockCommitSequenceId()) {
> containerInfo.updateSequenceId(
> replicaProto.getBlockCommitSequenceId());
> }
> if (containerInfo.getUsedBytes() < replicaProto.getUsed()) {
> containerInfo.setUsedBytes(replicaProto.getUsed());
> }
> if (containerInfo.getNumberOfKeys() < replicaProto.getKeyCount()) {
> containerInfo.setNumberOfKeys(replicaProto.getKeyCount());
> }
> }
> }
> {code}
> In HDDS-4037 a change was made to the Replication Manager, so it updates the
> stats. However I don't believe that is the correct place to perform this
> check, and the issue is caused by the logic shared above.
> In this Jira, I have removed the changes to Replication Manager in HDDS-4037
> (but retained the other changes), ensuring the problem statistics are only
> updated via the containers reports if they are different in SCM from what is
> reported.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
