[
https://issues.apache.org/jira/browse/HDDS-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17701002#comment-17701002
]
Tsz-wo Sze commented on HDDS-8128:
----------------------------------
We may simply have a cache in RDBBatchOperation for caching the large ops (say
> 16KB). Â When there are multiple ops with the same key, it will overwrite the
previous op in the cache so the previous op won't be written into RocksDB.
Just have tried it. The cache can reduce the a single upload of 1000 parts
from 755MB to 23MB.
- Without cache
{code}
755M
./MiniOzoneClusterImpl-450e1fd2-6a54-43fc-9f5d-badda40cb494/ozone-meta/om.db
{code}
- With cache
{code}
23M
./MiniOzoneClusterImpl-99d523fe-4d0d-4c94-95a1-598cc1212c46/ozone-meta/om.db
{code}
> OM rocksdb uses a lot of space
> ------------------------------
>
> Key: HDDS-8128
> URL: https://issues.apache.org/jira/browse/HDDS-8128
> Project: Apache Ozone
> Issue Type: Improvement
> Components: OM
> Reporter: Tsz-wo Sze
> Assignee: Tsz-wo Sze
> Priority: Blocker
>
> In a multipart upload test, the key "testKey" had 1000-parts with 8KB each.
> The same key was uploaded 10 times sequentially (i.e. it overwrote the
> previous upload) in a newly formatted cluster. The replication was 3, so the
> total raw size of the key is ~ 24 MB. After the test has completed, OM rocks
> db uses ~ 7.5 GB.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
