[
https://issues.apache.org/jira/browse/HDDS-6279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kaijie Chen updated HDDS-6279:
------------------------------
Description:
The usedBytes of EC keys is incorrect.
For an EC key with d+p blocks, usedBytes is dataSize * (d + p) now.
It should be (dataSize + padding) * (d + p) / d instead.
For example, writing a 3:2 EC key with 3,000,000 bytes, usedBytes should be
5,000,000 instead of 15,000,000.
{code:java}
$ bin/ozone sh bucket create /vol1/bucket1 -t EC -r rs-3-2-1024k
$ bin/ozone sh key put /vol1/bucket1/key1 ~/keys/key1
$ bin/ozone sh key list /vol1/bucket1
[ {
"volumeName" : "vol1",
"bucketName" : "bucket1",
"name" : "key1",
"dataSize" : 3000000,
"creationTime" : "2022-02-08T08:35:30.945Z",
"modificationTime" : "2022-02-08T08:35:32.029Z",
"replicationConfig" : {
"data" : 3,
"parity" : 2,
"ecChunkSize" : 1048576,
"codec" : "RS",
"requiredNodes" : 5,
"replicationType" : "EC"
},
"replicationType" : "EC",
"replicationFactor" : 5
} ]
$ bin/ozone sh bucket list /vol1
[ {
"metadata" : { },
"volumeName" : "vol1",
"name" : "bucket1",
"storageType" : "DISK",
"versioning" : false,
"usedBytes" : 15000000,
"usedNamespace" : 1,
"creationTime" : "2022-02-08T08:12:22.897Z",
"modificationTime" : "2022-02-08T08:12:22.897Z",
"quotaInBytes" : -1,
"quotaInNamespace" : -1,
"bucketLayout" : "OBJECT_STORE",
"owner" : "ckj",
"link" : false,
"replicationConfig" : {
"data" : 3,
"parity" : 2,
"ecChunkSize" : 1048576,
"codec" : "RS",
"replicationType" : "EC",
"requiredNodes" : 5
}
} ]{code}
was:
The usedBytes of EC keys is incorrect.
For an EC key with d+p blocks, usedBytes is dataSize * (d + p) now.
It should be (dataSize + padding) * (d + p) / d instead.
For example,
{code:java}
$ bin/ozone sh bucket create /vol1/bucket1 -t EC -r rs-3-2-1024k
$ bin/ozone sh key put /vol1/bucket1/key1 ~/keys/key1
$ bin/ozone sh key list /vol1/bucket1
[ {
"volumeName" : "vol1",
"bucketName" : "bucket1",
"name" : "key1",
"dataSize" : 3000000,
"creationTime" : "2022-02-08T08:35:30.945Z",
"modificationTime" : "2022-02-08T08:35:32.029Z",
"replicationConfig" : {
"data" : 3,
"parity" : 2,
"ecChunkSize" : 1048576,
"codec" : "RS",
"requiredNodes" : 5,
"replicationType" : "EC"
},
"replicationType" : "EC",
"replicationFactor" : 5
} ]
$ bin/ozone sh bucket list /vol1
[ {
"metadata" : { },
"volumeName" : "vol1",
"name" : "bucket1",
"storageType" : "DISK",
"versioning" : false,
"usedBytes" : 15000000,
"usedNamespace" : 1,
"creationTime" : "2022-02-08T08:12:22.897Z",
"modificationTime" : "2022-02-08T08:12:22.897Z",
"quotaInBytes" : -1,
"quotaInNamespace" : -1,
"bucketLayout" : "OBJECT_STORE",
"owner" : "ckj",
"link" : false,
"replicationConfig" : {
"data" : 3,
"parity" : 2,
"ecChunkSize" : 1048576,
"codec" : "RS",
"replicationType" : "EC",
"requiredNodes" : 5
}
} ]{code}
> EC: Quota for EC keys
> ---------------------
>
> Key: HDDS-6279
> URL: https://issues.apache.org/jira/browse/HDDS-6279
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Kaijie Chen
> Priority: Major
>
> The usedBytes of EC keys is incorrect.
> For an EC key with d+p blocks, usedBytes is dataSize * (d + p) now.
> It should be (dataSize + padding) * (d + p) / d instead.
>
> For example, writing a 3:2 EC key with 3,000,000 bytes, usedBytes should be
> 5,000,000 instead of 15,000,000.
> {code:java}
> $ bin/ozone sh bucket create /vol1/bucket1 -t EC -r rs-3-2-1024k
> $ bin/ozone sh key put /vol1/bucket1/key1 ~/keys/key1
> $ bin/ozone sh key list /vol1/bucket1
> [ {
> "volumeName" : "vol1",
> "bucketName" : "bucket1",
> "name" : "key1",
> "dataSize" : 3000000,
> "creationTime" : "2022-02-08T08:35:30.945Z",
> "modificationTime" : "2022-02-08T08:35:32.029Z",
> "replicationConfig" : {
> "data" : 3,
> "parity" : 2,
> "ecChunkSize" : 1048576,
> "codec" : "RS",
> "requiredNodes" : 5,
> "replicationType" : "EC"
> },
> "replicationType" : "EC",
> "replicationFactor" : 5
> } ]
> $ bin/ozone sh bucket list /vol1
> [ {
> "metadata" : { },
> "volumeName" : "vol1",
> "name" : "bucket1",
> "storageType" : "DISK",
> "versioning" : false,
> "usedBytes" : 15000000,
> "usedNamespace" : 1,
> "creationTime" : "2022-02-08T08:12:22.897Z",
> "modificationTime" : "2022-02-08T08:12:22.897Z",
> "quotaInBytes" : -1,
> "quotaInNamespace" : -1,
> "bucketLayout" : "OBJECT_STORE",
> "owner" : "ckj",
> "link" : false,
> "replicationConfig" : {
> "data" : 3,
> "parity" : 2,
> "ecChunkSize" : 1048576,
> "codec" : "RS",
> "replicationType" : "EC",
> "requiredNodes" : 5
> }
> } ]{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org
For additional commands, e-mail: issues-h...@ozone.apache.org
