Ivan Andika created HDDS-12903:
----------------------------------
Summary: Separate namespace and blockspace CF in OM
Key: HDDS-12903
URL: https://issues.apache.org/jira/browse/HDDS-12903
Project: Apache Ozone
Issue Type: Wish
Components: Ozone Manager
Reporter: Ivan Andika
Assignee: Ivan Andika
Currently when OM list keys, it needs to create a RocksDB iterator which will
load both the RocksDB key and value.
For a key with a lot of blocks (for example MPU key with few hundred parts),
this can take a considerable amount of OM heap memory. These blocks is even
loaded to OM memory even when we use listKeysLight which will remove the key
block info.
One possible way to handle this is to separate the keyspace and blockspace into
two separate column families. The keyspace CF will only store the basic key
info (similar to BasicOmKeyInfo), while the blockspace CF stores the blocks
associated with the key. Therefore, during list, we will only load the
BasicOmKeyInfo which will result in lower memory overhead.
The downside is that now we have two CF to update or query for key get,
creation, update, and deletion. For get, we can use RocksDB multiget to get
both the keyspace and blockspace CF.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
