Wei-Chiu Chuang created HDDS-14985:
--------------------------------------
Summary: [Docs] Explain "exclusive size" and "referenced size" of
a snapshot
Key: HDDS-14985
URL: https://issues.apache.org/jira/browse/HDDS-14985
Project: Apache Ozone
Issue Type: Task
Components: documentation, Snapshot
Reporter: Wei-Chiu Chuang
It's not all that clear what "exclusive size" and "referenced size" mean.
Elaborate it in the Ozone Snapshot overview doc:
[https://ozone.apache.org/docs/administrator-guide/operations/snapshots/overview/]
In the context of Apache Ozone, the exclusiveSize of a snapshot represents the
amount of data that is held only by that specific snapshot and no other
entity in the system.
To understand it better, it helps to compare it with referencedSize:
1. Exclusive Size vs. Referenced Size
* Referenced Size: This is the "Total View" size. It is the sum of all files
and directories visible when you look into that snapshot. If your bucket has
100GB of data and you take a snapshot, the referencedSize is 100GB.
* Exclusive Size: This is the "Unique" size. It is the data that exists in
the system exclusively because this snapshot is keeping it alive. If you just
took a snapshot and haven't deleted anything from the main bucket yet, the
exclusiveSize is 0 bytes.
2. How exclusiveSize grows
A snapshot's exclusive size increases when data is removed from the "newer"
parts of the chain. Specifically, a file's size moves into a snapshot's
exclusiveSize when:
1. The file is deleted or overwritten in the active bucket (AOS).
2. The file is not present in any newer snapshots in the same path chain.
Example Scenario:
1. Time T1: You have File-A (1GB) in your bucket.
2. Time T2: You take Snapshot-1.
* Snapshot-1 Referenced: 1GB
* Snapshot-1 Exclusive: 0GB (because File-A still exists in the active
bucket).
3. Time T3: You delete File-A from the active bucket.
* Snapshot-1 Referenced: 1GB
* Snapshot-1 Exclusive: 1GB (because Snapshot-1 is now the only thing in
the system that "owns" File-A).
3. Why is it important?
The exclusiveSize is the most critical metric for storage management because
it tells you exactly how much space will be reclaimed if you delete that
snapshot.
When the SnapshotDeletingService (which you looked at earlier) runs, it
identifies this exclusive data and marks it for deletion in the Datanodes. If a
snapshot has an exclusiveSize of 50GB, deleting it will eventually free up
50GB of disk space.
4. Calculation in the Background
Ozone calculates these sizes using a background service called the
SnapshotSstFilteringService and the SnapshotDeletingService. They compare the
"Deleted
Table" of a snapshot with newer snapshots in the chain to determine if a
block is truly exclusive or if it's still being "referenced" by a neighbor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]