[ http://issues.apache.org/jira/browse/HADOOP-26?page=comments#action_12365457 ]
Mike Cafarella commented on HADOOP-26: -------------------------------------- OK, the above comment is a little misleading. We used to have a difference space-management system in place, which would set certain nodes off-limits if they didn't have enough space. Now we choose nodes probabilistically, with more weight given to nodes with more space. That's probably a little too cute than is necessary. We should just have hard limits on how much a given node will take before it complains and never receives further node allocations. > DFS node choice doesn't take available space into account effectively > --------------------------------------------------------------------- > > Key: HADOOP-26 > URL: http://issues.apache.org/jira/browse/HADOOP-26 > Project: Hadoop > Type: Bug > Components: dfs > Reporter: Mike Cafarella > Assignee: Mike Cafarella > > We used to have node-allocation be sensitive to available disk space. It > turned out that > this was a little accident-prone and hard to debug on then-available > machines, so I removed it. > DFS is mature enough now that this needs to come back and work properly. A > node with less > than X bytes should never accept a new block for allocation. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
