Wei-Chiu Chuang created HDDS-14985:
--------------------------------------

             Summary: [Docs] Explain "exclusive size" and "referenced size" of 
a snapshot
                 Key: HDDS-14985
                 URL: https://issues.apache.org/jira/browse/HDDS-14985
             Project: Apache Ozone
          Issue Type: Task
          Components: documentation, Snapshot
            Reporter: Wei-Chiu Chuang


It's not all that clear what "exclusive size" and "referenced size" mean. 
Elaborate it in the Ozone Snapshot overview doc: 
[https://ozone.apache.org/docs/administrator-guide/operations/snapshots/overview/]

 

In the context of Apache Ozone, the exclusiveSize of a snapshot represents the 
amount of data that is held only by that specific snapshot and no other
  entity in the system.

  To understand it better, it helps to compare it with referencedSize:

  1. Exclusive Size vs. Referenced Size
   * Referenced Size: This is the "Total View" size. It is the sum of all files 
and directories visible when you look into that snapshot. If your bucket has
     100GB of data and you take a snapshot, the referencedSize is 100GB.
   * Exclusive Size: This is the "Unique" size. It is the data that exists in 
the system exclusively because this snapshot is keeping it alive. If you just
     took a snapshot and haven't deleted anything from the main bucket yet, the 
exclusiveSize is 0 bytes.

  2. How exclusiveSize grows
  A snapshot's exclusive size increases when data is removed from the "newer" 
parts of the chain. Specifically, a file's size moves into a snapshot's
  exclusiveSize when:
   1. The file is deleted or overwritten in the active bucket (AOS).
   2. The file is not present in any newer snapshots in the same path chain.

  Example Scenario:
   1. Time T1: You have File-A (1GB) in your bucket.
   2. Time T2: You take Snapshot-1.
       * Snapshot-1 Referenced: 1GB
       * Snapshot-1 Exclusive: 0GB (because File-A still exists in the active 
bucket).
   3. Time T3: You delete File-A from the active bucket.
       * Snapshot-1 Referenced: 1GB
       * Snapshot-1 Exclusive: 1GB (because Snapshot-1 is now the only thing in 
the system that "owns" File-A).

  3. Why is it important?
  The exclusiveSize is the most critical metric for storage management because 
it tells you exactly how much space will be reclaimed if you delete that
  snapshot.

  When the SnapshotDeletingService (which you looked at earlier) runs, it 
identifies this exclusive data and marks it for deletion in the Datanodes. If a
  snapshot has an exclusiveSize of 50GB, deleting it will eventually free up 
50GB of disk space.

  4. Calculation in the Background
  Ozone calculates these sizes using a background service called the 
SnapshotSstFilteringService and the SnapshotDeletingService. They compare the 
"Deleted
  Table" of a snapshot with newer snapshots in the chain to determine if a 
block is truly exclusive or if it's still being "referenced" by a neighbor.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to