[
https://issues.apache.org/jira/browse/HDDS-8874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Andika updated HDDS-8874:
------------------------------
Description:
Currently in the OM passes OzoneManager#getOmNodeId in the
OMKeyRequest#allocateBlock call. This means that the created container's owner
is based on the OM HA leader's Node ID specified in the ozone configuration
(i.e. ozone.om.node.id).
In the case where there are multiple OMs using the same SCM. If the Node IDs
configured are not unique across the SCM cluster, there might be a situation
where a single container contains blocks owned by multiple OMs. This should be
fine for now, but in the future if we want to separate the containers based on
the Storage Type, this collision might raise tricky situations
For example for configuration (omitting om adresses)
{code:java}
<property>
<name>ozone.om.service.ids</name>
<value>service1,service2</value>
</property>
<property>
<name>ozone.om.nodes.service1</name>
<value>om1,om2,om3</value>
</property>
<property>
<name>ozone.om.nodes.service2</name>
<value>om1,om2,om3</value>
</property>{code}
If the OM leader of both service1 and service2 has the same node ID (e.g. om1),
both OM will use the configured node ID as the owner when allocating block, and
SCM will assume that the request comes from the same owner and allocate the
same container for the blocks.
We should find a way to pick a unique ID to pass to the allocateBlock call.
Currently a valid option is to use omUuid since the chance of collisions would
be smaller.
However, a better option in OM HA setup is to pick a unique ID for the entire
OM HA service (instead of node ID). This has possible additional advantages:
* When the OM leader change, the SCM does not need to create a new container
for the new blocks
* Fewer number of OPEN containers and all containers in SCM overall
Unfortunately, I have not found such an ID available.
Another consideration is to migrate the existing containers's owner to the new
naming conventions. This might be able to be done by using some container task.
This issue can be handled in another.
> Use unique OM ID as SCM container owner
> ---------------------------------------
>
> Key: HDDS-8874
> URL: https://issues.apache.org/jira/browse/HDDS-8874
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
>
> Currently in the OM passes OzoneManager#getOmNodeId in the
> OMKeyRequest#allocateBlock call. This means that the created container's
> owner is based on the OM HA leader's Node ID specified in the ozone
> configuration (i.e. ozone.om.node.id).
> In the case where there are multiple OMs using the same SCM. If the Node IDs
> configured are not unique across the SCM cluster, there might be a situation
> where a single container contains blocks owned by multiple OMs. This should
> be fine for now, but in the future if we want to separate the containers
> based on the Storage Type, this collision might raise tricky situations
> For example for configuration (omitting om adresses)
> {code:java}
> <property>
> <name>ozone.om.service.ids</name>
> <value>service1,service2</value>
> </property>
> <property>
> <name>ozone.om.nodes.service1</name>
> <value>om1,om2,om3</value>
> </property>
> <property>
> <name>ozone.om.nodes.service2</name>
> <value>om1,om2,om3</value>
> </property>{code}
>
> If the OM leader of both service1 and service2 has the same node ID (e.g.
> om1), both OM will use the configured node ID as the owner when allocating
> block, and SCM will assume that the request comes from the same owner and
> allocate the same container for the blocks.
> We should find a way to pick a unique ID to pass to the allocateBlock call.
> Currently a valid option is to use omUuid since the chance of collisions
> would be smaller.
> However, a better option in OM HA setup is to pick a unique ID for the entire
> OM HA service (instead of node ID). This has possible additional advantages:
> * When the OM leader change, the SCM does not need to create a new container
> for the new blocks
> * Fewer number of OPEN containers and all containers in SCM overall
> Unfortunately, I have not found such an ID available.
> Another consideration is to migrate the existing containers's owner to the
> new naming conventions. This might be able to be done by using some container
> task. This issue can be handled in another.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
