[
https://issues.apache.org/jira/browse/HDDS-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Rose updated HDDS-6234:
-----------------------------
Description:
HDDS-5359 shows that the bytes used and block count metadata of a container is
tracked incorrectly. Empty containers with negative values in these fields will
not be deleted per HDDS-4023 and will remain tracked by SCM unnecessarily.
This Jira aims to add a config flag that when enabled will repair the metadata
information in the datanodes on restart. Datanodes already iterate all their
containers on startup, but the process is kept light to not slow down startup
too much. Repairing these values will slow down datanode startup, since it
requires writing to RocksDB (currently opened in readonly mode on startup) and
iterating the block keys stored there. For this reason the plan is to put this
behind a config flag.
I have set the affected versions to 1.2.0 since it is after this release that
HDDS-5359 was reported, but it is possible earlier releases are affected.
was:
HDDS-5359 shows that the bytes used and block count metadata of a container is
tracked incorrectly. Empty containers with negative values in these fields will
not be deleted per HDDS-4023 and will remain tracked by SCM unnecessarily.
This Jira aims to add a config flag that when enabled will repair the metadata
information in the datanodes on restart. Datanodes already iterate all their
containers on startup, but the process is kept light to not slow down startup
too much. Repairing these values will require iterating all block files on disk
and checking their size, which will slow down startup significantly. For this
reason the plan is to put this behind a config flag.
I have set the affected versions to 1.2.0 since it is after this release that
HDDS-5359 was reported, but it is possible earlier releases are affected.
> Repair containers affected by incorrect used bytes and block count
> ------------------------------------------------------------------
>
> Key: HDDS-6234
> URL: https://issues.apache.org/jira/browse/HDDS-6234
> Project: Apache Ozone
> Issue Type: New Feature
> Components: Ozone Datanode, SCM
> Affects Versions: 1.2.0
> Reporter: Ethan Rose
> Assignee: Ethan Rose
> Priority: Major
>
> HDDS-5359 shows that the bytes used and block count metadata of a container
> is tracked incorrectly. Empty containers with negative values in these fields
> will not be deleted per HDDS-4023 and will remain tracked by SCM
> unnecessarily.
> This Jira aims to add a config flag that when enabled will repair the
> metadata information in the datanodes on restart. Datanodes already iterate
> all their containers on startup, but the process is kept light to not slow
> down startup too much. Repairing these values will slow down datanode
> startup, since it requires writing to RocksDB (currently opened in readonly
> mode on startup) and iterating the block keys stored there. For this reason
> the plan is to put this behind a config flag.
> I have set the affected versions to 1.2.0 since it is after this release that
> HDDS-5359 was reported, but it is possible earlier releases are affected.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org
For additional commands, e-mail: issues-h...@ozone.apache.org
