Stephen O'Donnell created HDDS-4131:
---------------------------------------
Summary: Container report should update container key count and
bytes used if they differ in SCM
Key: HDDS-4131
URL: https://issues.apache.org/jira/browse/HDDS-4131
Project: Hadoop Distributed Data Store
Issue Type: Improvement
Components: SCM
Affects Versions: 0.7.0
Reporter: Stephen O'Donnell
Assignee: Stephen O'Donnell
In HDDS-4037 it was noted that when blocks are deleted from closed containers,
the bytesUsed and Key Count metrics on the SCM container are not updated
correctly.
These stats should be updated via the container reports issued by the DNs to
SCM periodically. However, in
`AbstractContainerReportHandler#updateContainerStats`, the code assumes the
values are always increasing and it will not update them if they are decreasing:
{code}
private void updateContainerStats(final ContainerID containerId,
final ContainerReplicaProto replicaProto)
throws ContainerNotFoundException {
if (isHealthy(replicaProto::getState)) {
final ContainerInfo containerInfo = containerManager
.getContainer(containerId);
if (containerInfo.getSequenceId() <
replicaProto.getBlockCommitSequenceId()) {
containerInfo.updateSequenceId(
replicaProto.getBlockCommitSequenceId());
}
if (containerInfo.getUsedBytes() < replicaProto.getUsed()) {
containerInfo.setUsedBytes(replicaProto.getUsed());
}
if (containerInfo.getNumberOfKeys() < replicaProto.getKeyCount()) {
containerInfo.setNumberOfKeys(replicaProto.getKeyCount());
}
}
}
{code}
In HDDS-4037 a change was made to the Replication Manager, so it updates the
stats. However I don't believe that is the correct place to perform this check,
and the issue is caused by the logic shared above.
In this Jira, I have removed the changes to Replication Manager in HDDS-4037
(but retained the other changes), ensuring the problem statistics are only
updated via the containers reports if they are different in SCM from what is
reported.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
