[
https://issues.apache.org/jira/browse/HDFS-14789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haibin Huang updated HDFS-14789:
--------------------------------
Description:
With HDFS-11194 and HDFS-11551, namenode can show SlowPeersReport and
SlowDisksReport in jmx. I think namenode can avoid these slow node while
chooseTarget in BlockPlacementPolicyDefault. Because if there is a slow node in
pipeline, client might write very slowly.
I use a invalidityTime to let namnode not choose slow node before invalid time
finish. After the invalidityTime, if slow node return to normal, namenode can
choose it again, or it's still very slow, the invalidityTime will update and
keep not choosing it.
Also i consider the fallback, if namenode can't choose any normal node,
chooseTarget will throw NotEnoughReplicasException and retry, this time not
avoiding slow nodes.
!image-2020-12-22-22-15-17-703.png|width=969,height=322!
was:
With HDFS-11194 and HDFS-11551, namenode can show SlowPeersReport and
SlowDisksReport in jmx. I think namenode can avoid these slow node while
chooseTarget in BlockPlacementPolicyDefault. Because if there is a slow node in
pipeline, client might write very slowly.
I use a invalidityTime to let namnode not choose slow node before invalid time
finish. After the invalidityTime, if slow node return to normal, namenode can
choose it again, or it's still very slow, the invalidityTime will update and
keep not choosing it.
Also i consider the fallback, if namenode can't choose any normal node,
chooseTarget will throw NotEnoughReplicasException and retry, this time not
avoiding slow nodes.
> namenode should avoid slow node while chooseTarget in
> BlockPlacementPolicyDefault
> ---------------------------------------------------------------------------------
>
> Key: HDFS-14789
> URL: https://issues.apache.org/jira/browse/HDFS-14789
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Haibin Huang
> Assignee: Haibin Huang
> Priority: Major
> Attachments: HDFS-14789, HDFS-14789-001.patch,
> image-2020-12-22-22-15-17-703.png
>
>
> With HDFS-11194 and HDFS-11551, namenode can show SlowPeersReport and
> SlowDisksReport in jmx. I think namenode can avoid these slow node while
> chooseTarget in BlockPlacementPolicyDefault. Because if there is a slow node
> in pipeline, client might write very slowly.
> I use a invalidityTime to let namnode not choose slow node before invalid
> time finish. After the invalidityTime, if slow node return to normal,
> namenode can choose it again, or it's still very slow, the invalidityTime
> will update and keep not choosing it.
> Also i consider the fallback, if namenode can't choose any normal node,
> chooseTarget will throw NotEnoughReplicasException and retry, this time not
> avoiding slow nodes.
>
> !image-2020-12-22-22-15-17-703.png|width=969,height=322!
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
