[
https://issues.apache.org/jira/browse/HDDS-15077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDDS-15077:
-----------------------------------
Status: Patch Available (was: Open)
> Explain "exclusiveSize" and "exclusiveSizeDeltaFromDirDeepCleaning"
> -------------------------------------------------------------------
>
> Key: HDDS-15077
> URL: https://issues.apache.org/jira/browse/HDDS-15077
> Project: Apache Ozone
> Issue Type: Task
> Reporter: Wei-Chiu Chuang
> Assignee: Wei-Chiu Chuang
> Priority: Major
> Labels: pull-request-available
>
> HDDS-13022 changed the way that exclusiveSize is tracked internally. Here's
> the explanation. We should add them into the code and maybe System Internals
> doc.
> h1. exclusiveSize
> Set by: KeyDeletingService (key cleanup background service)
> Meaning: The amount of data (in bytes, unreplicated) that is exclusively
> referenced by this snapshot, as determined during key-level deep cleaning —
> i.e., when deleted keys are processed and it's determined that only this
> snapshot holds a reference to certain data blocks.
> h1. exclusiveReplicatedSize
> Set by: KeyDeletingService
> Meaning: Same as exclusiveSize, but accounts for replication factor. For
> example, if a block is 1 MB stored with RATIS/3 replication, exclusiveSize =
> 1 MB but exclusiveReplicatedSize = 3 MB.
> h1. exclusiveSizeDeltaFromDirDeepCleaning
> Set by: SnapshotDirectoryCleaningService (directory cleanup background
> service)
> Meaning: The additional exclusive size (unreplicated) discovered during
> directory-level deep cleaning — i.e., when deleted directories are
> recursively processed and their contents are found to be exclusively held by
> this snapshot. This is kept separate from exclusiveSize to avoid the two
> services overwriting each other.
> h1. exclusiveReplicatedSizeDeltaFromDirDeepCleaning
> Set by: SnapshotDirectoryCleaningService
> Meaning: Same as exclusiveSizeDeltaFromDirDeepCleaning, but scaled by the
> replication factor.
>
>
> h1. Why Are They Split?
> Before this PR, both services wrote to the same exclusiveSize field. Since
> they run independently and asynchronously, one service could overwrite the
> other's result, causing the exclusive size to be understated (a correctness
> bug).
> By using separate fields, each service owns its own variable. The true total
> exclusive size is then computed as:
> {code}
> totalExclusiveSize = exclusiveSize + exclusiveSizeDeltaFromDirDeepCleaning
> totalExclusiveReplicatedSize = exclusiveReplicatedSize +
> exclusiveReplicatedSizeDeltaFromDirDeepCleaning
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
