[
https://issues.apache.org/jira/browse/HDDS-5526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddhant Sangwan updated HDDS-5526:
-----------------------------------
Description:
ContainerBalancer#checkConditionsForBalancing is used to check whether the max
size to move or max datanodes to involve limits have been reached.
Currently, ContainerBalancer#doIteration uses this method before finding a
ContainerMoveSelection for a source datanode. Since the check is performed
before selecting the next container to move, the configuration
{color:#172b4d}OZONE_SCM_CONTAINER_SIZE {color}(default is 5GB) is used to
predetermine if:
sizeMovedPerIteration + configured container size >= maxSizeToMovePerIteration.
This check will cause the current iteration to stop in cases like:
maxSizeToMovePerIteration = 100GB, sizeMovedPerIteration = 96GB, and
{color:#172b4d}OZONE_SCM_CONTAINER_SIZE{color} {color:#172b4d}= 5GB. {color}
{color:#172b4d}On one hand the current implementation can avoid extra work by
not letting balancer continue when we are very close to the limit. On the other
hand, because of this implementation, we will never reach the limit. This Jira
tracks if it's better to check conditions using container size in
ContainerBalancer#checkConditionsForBalancing after having found a
ContainerMoveSelection in ContainerBalancer#doIteration.{color}
Another solution is to allow one container move before checking the limit. The
downside is that if the limits are really low (size lesser than 5GB, number of
datanodes lesser than 2), we will fail to respect them. However, I can't think
of any use-case for setting such low limits.
was:
ContainerBalancer#checkConditionsForBalancing is used to check whether the max
size to move or max datanodes to involve limits have been reached.
Currently, ContainerBalancer#doIteration uses this method before finding a
ContainerMoveSelection for a source datanode. Since the check is performed
before selecting the next container to move, the configuration
{color:#172b4d}OZONE_SCM_CONTAINER_SIZE {color}(default is 5GB) is used to
predetermine if:
sizeMovedPerIteration + configured container size >= maxSizeToMovePerIteration.
This check will cause the current iteration to stop in cases like:
maxSizeToMovePerIteration = 100GB, sizeMovedPerIteration = 96GB, and
{color:#172b4d}OZONE_SCM_CONTAINER_SIZE{color} {color:#172b4d}= 5GB. {color}
{color:#172b4d}On one hand the current implementation can avoid extra work by
not letting balancer continue when we are very close to the limit. On the other
hand, because of this implementation, we will never reach the limit. This Jira
tracks if it's better to check conditions using container size in
ContainerBalancer#checkConditionsForBalancing after having found a
ContainerMoveSelection in ContainerBalancer#doIteration.{color}
> ContainerBalancer#checkConditionsForBalancing can use container size for
> checking iteration limits.
> ---------------------------------------------------------------------------------------------------
>
> Key: HDDS-5526
> URL: https://issues.apache.org/jira/browse/HDDS-5526
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: SCM
> Reporter: Siddhant Sangwan
> Priority: Minor
>
> ContainerBalancer#checkConditionsForBalancing is used to check whether the
> max size to move or max datanodes to involve limits have been reached.
> Currently, ContainerBalancer#doIteration uses this method before finding a
> ContainerMoveSelection for a source datanode. Since the check is performed
> before selecting the next container to move, the configuration
> {color:#172b4d}OZONE_SCM_CONTAINER_SIZE {color}(default is 5GB) is used to
> predetermine if:
> sizeMovedPerIteration + configured container size >=
> maxSizeToMovePerIteration.
> This check will cause the current iteration to stop in cases like:
> maxSizeToMovePerIteration = 100GB, sizeMovedPerIteration = 96GB, and
> {color:#172b4d}OZONE_SCM_CONTAINER_SIZE{color} {color:#172b4d}= 5GB. {color}
> {color:#172b4d}On one hand the current implementation can avoid extra work by
> not letting balancer continue when we are very close to the limit. On the
> other hand, because of this implementation, we will never reach the limit.
> This Jira tracks if it's better to check conditions using container size in
> ContainerBalancer#checkConditionsForBalancing after having found a
> ContainerMoveSelection in ContainerBalancer#doIteration.{color}
> Another solution is to allow one container move before checking the limit.
> The downside is that if the limits are really low (size lesser than 5GB,
> number of datanodes lesser than 2), we will fail to respect them. However, I
> can't think of any use-case for setting such low limits.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
