[
https://issues.apache.org/jira/browse/HDFS-16502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani updated HDFS-16502:
--------------------------------
Description:
Based on the cluster load, it would be helpful to consider tuning block
invalidate limit (dfs.block.invalidate.limit). The only way we can do this
without restarting Namenode as of today is by reconfiguring heartbeat interval
{code:java}
Math.max(heartbeatInt*20, blockInvalidateLimit){code}
, this logic is not straightforward and operators are usually not aware of it
(lack of documentation), also updating heartbeat interval is not desired in all
the cases.
We should provide the ability to alter block invalidation limit without
affecting heartbeat interval on the live cluster to adjust some load at
Datanode level.
We should also take this opportunity to keep (heartbeatInterval * 20)
computation logic in a common method.
was:
Based on the cluster load, it would be helpful to consider tuning block
invalidate limit (dfs.block.invalidate.limit). The only way we can do this
without restarting Namenode as of today is by reconfiguring heartbeat interval
{code:java}
Math.max(heartbeatInt*20, blockInvalidateLimit){code}
, this logic is not straightforward and operators are usually not aware of it
(lack of documentation), also updating heartbeat interval is not desired in all
the cases.
We should provide the ability to alter block invalidation limit without
affecting heartbeat interval on the live cluster to adjust some load at
Datanode level.
> Reconfigure Block Invalidate limit
> ----------------------------------
>
> Key: HDFS-16502
> URL: https://issues.apache.org/jira/browse/HDFS-16502
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: Viraj Jasani
> Assignee: Viraj Jasani
> Priority: Major
>
> Based on the cluster load, it would be helpful to consider tuning block
> invalidate limit (dfs.block.invalidate.limit). The only way we can do this
> without restarting Namenode as of today is by reconfiguring heartbeat
> interval
> {code:java}
> Math.max(heartbeatInt*20, blockInvalidateLimit){code}
> , this logic is not straightforward and operators are usually not aware of it
> (lack of documentation), also updating heartbeat interval is not desired in
> all the cases.
> We should provide the ability to alter block invalidation limit without
> affecting heartbeat interval on the live cluster to adjust some load at
> Datanode level.
> We should also take this opportunity to keep (heartbeatInterval * 20)
> computation logic in a common method.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
