anuragp010 opened a new pull request, #11951:
URL: https://github.com/apache/ozone/pull/11951

   ## What changes were proposed in this pull request?
   A Key is initially written with a `StoragePolicy` and when the container is 
created, the `StorageTier` is chosen. This is then propagated to a datanode 
which places this container on a volume with a `StorageType`. Currently, 
nothing tells an operator whether a key's data is really on the intended 
storage type. A write can fallback to another tier, or a volume can be 
relabelled amongst other scenarios.
   This PR adds the command `ozone admin storagepolicy check 
<volume/bucket/key> [--show-replicas]`. The command returns YES only if each 
the key's container replica's `storageType` and `volumeStorageType` match the 
expected `storageType`. 
   1. The command first gets the `OzoneKeyDetails` of the key.
   2. For each unique container in the list of containers that the key's data 
is present in, a call is made to the SCM to get the container replicas.
   3. The expected storageType for a key is derived from it's StoragePolicy 
(`OzoneStoragePolicy` -> `StorageTier` -> `StorageType`)
   
   ## What is the link to the Apache JIRA
   https://issues.apache.org/jira/browse/HDDS-17710
   
   ## How was this patch tested?
   1. Unit Tests and Integration Tests.
   2. 
   **Docker compose cluster with output:**
   ```shell
   bash-5.1$ ozone admin storagepolicy check vol1/bucket1/key1 --show-replicas
   Storage Policy for key 'vol1/bucket1/key1':
     HOT{Creation Tier: [SSD x 3], Fallback Tier: [DISK x 3]}
     Key Match Storage Policy: YES
   Container ID: 1
     Datanode: 5e7318f9-14bc-4db8-b3b7-bc78737746a6 
(ozone-datanode-ssd-3.ozone_default, /default-rack)
     Replica Storage Type: SSD
     Volume Storage Type: SSD
     Replica Match: YES
     Datanode: 5b53ac10-d16b-4a36-a4df-269cc737964c 
(ozone-datanode-ssd-1.ozone_default, /default-rack)
     Replica Storage Type: SSD
     Volume Storage Type: SSD
     Replica Match: YES
     Datanode: d675d25c-8ea3-4c36-85d7-df57524e1ee7 
(ozone-datanode-ssd-2.ozone_default, /default-rack)
     Replica Storage Type: SSD
     Volume Storage Type: SSD
     Replica Match: YES
     Container Match Storage Policy: YES
   ```
   
   **Sample Output for the volume relabelling scenario:**
   ```shell
   Storage Policy for key 'vol1/bucket1/key1':
     HOT{Creation Tier: [SSD x 3], Fallback Tier: [DISK x 3]}
     Key Match Storage Policy: NO
   Container ID: 12
     Datanode: 1b6e0c52-... (dn1.example.com, /rack1)
     Replica Storage Type: SSD
     Volume Storage Type: SSD
     Replica Match: YES
     Datanode: 7d41a9f3-... (dn2.example.com, /rack2)
     Replica Storage Type: SSD
     Volume Storage Type: DISK
     Replica Match: NO (volumeStorageType DISK != expected SSD)
     Datanode: c93f5e18-... (dn3.example.com, /rack3)
     Replica Storage Type: SSD
     Volume Storage Type: SSD
     Replica Match: YES
     Container Match Storage Policy: NO
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to