SaketaChalamchala commented on code in PR #11229:
URL: https://github.com/apache/ozone/pull/11229#discussion_r4264999394
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/S3Consts.java:
##########
@@ -131,6 +131,11 @@ public final class S3Consts {
/** Request header carrying the list of object attributes to return. */
public static final String OBJECT_ATTRIBUTES_HEADER =
"x-amz-object-attributes";
+ // S3 storage class values passed via the `x-amz-storage-class` request
header.
+ public static final String S3_STORAGE_CLASS_STANDARD = "STANDARD";
+ public static final String S3_STORAGE_CLASS_STANDARD_IA = "STANDARD_IA";
+ public static final String S3_STORAGE_CLASS_GLACIER = "GLACIER";
+
Review Comment:
Agree with @Gargi-jais11. To maintain backward compatibility and to enable
future extension of existing S3 storage class definitions we should continue to
base existing definitions on replication config and extend current definitions
with optional additional specs i.e., :
```
standard: (RATIS, THREE, null)
reduced_redundancy: (RATIS, ONE, null)
standard_IA: (EC, replConf, null)
```
For the newly supported glacier family storage class however, it may be safe
have the extended definition :
```
glacier: (RATIS, THREE, COLD)
glacier_deep(optional): (EC, replConf, COLD)
```
This way Forward mapping will be:
```
standard.toReplConf --> RATIS/THREE (same as now)
reduced_redunancy.toReplConf --> RATIS/ONE (same as now)
standard_IA.toReplConf --> EC (same as now)
glacier.toReplConf --> RATIS/THREE (new keys)
glacier_deep.toReplConf --> EC (new keys)
standard.toStoragePolicy --> HOT (new keys)
reduced_redunancy.toStoragePolicy --> HOT (new keys)
standard_IA.toStoragePolicy --> WARM (new keys)
glacier.toStoragePolicy --> COLD (new keys)
glacier_deep.toStoragePolicy --> COLD (new keys)
```
Reverse mapping:
```
storageClass.fromSpec(replicationConf, storagePolicy) :
RATIS/THREE, (HOT/WARM(default)/null) --> standard (covers existing
keys with null/default policy, new keys with HOT/default policy)
RATIS/ONE, (HOT/WARM(default)/null) --> standard (covers existing
keys with null/default policy, new keys with HOT/default policy)
EC, (WARM/null) --> standard_IA (covers existing
keys with null/default policy, new keys with WARM(default) policy)
RATIS/THREE, COLD --> glacier (new)
EC, COLD --> glacier_deep (new)
```
@devmadhuu **Question: Can keys created without storage class header inherit
EC repl and HOT storage policy from bucket? What is the resolution order for
storage policy?**
It looks like the safe way to upgrade existing storage class defs to include
specific storage policies (ex. standard: (RATIS, THREE, HOT)) would be to
first, apply the appropriate storage policy to the existing key space
covered by the class and
then to upgrade the storage class definition.
FYI, **Both existing forward and reverse mapping of standard_IA accommodate
user defined EC replication configs in `S3Utils.toReplicationConfig` and
`S3StorageType.fromReplicationConfig`**. Forward mapping accepts,
`x-amz-meta-storage-config` for EC replication config along with the
`STANDARD_IA` storage class and only defaults to EC-3-2 in it's absence.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
