[
https://issues.apache.org/jira/browse/HDFS-12008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059747#comment-16059747
]
Chen Liang commented on HDFS-12008:
-----------------------------------
Hi [~kihwal] seem the
{{TestAvailableSpaceBlockPlacementPolicy#testChooseTarget}} has a very specific
assertion check that, among the two selected node, the one with higher
availability will be chosen with probability in range 0.52 to 0.55. Namely,
when two nodes selected, there is still a fair chance that the one with lower
availability gets choson. And the probability in the assertion can be easily
violated when making change to {{AvailableSpaceBlockPlacementPolicy}}. I
haven't digged into it, but HDFS-8131 seems to have some mathematical proof on
this.
The change in this JIRA makes sense to me though, just want to make sure we
don't lose anything important we get from this two-node-based-selection
introduced in HDFS-8131.
> Improve the available-space block placement policy
> --------------------------------------------------
>
> Key: HDFS-12008
> URL: https://issues.apache.org/jira/browse/HDFS-12008
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: block placement
> Affects Versions: 2.8.1
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Attachments: HDFS-12008.patch
>
>
> AvailableSpaceBlockPlacementPolicy currently picks two nodes unconditionally,
> then picks one node. It could avoid picking the second node when not
> necessary.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
