Have the dfs upload done by a server not running a datanode and your blocks will be randomly distributed among active datanodes.
On Fri, Oct 8, 2010 at 2:39 PM, shangan <[email protected]> wrote: > is there any way to change the default storage policy ? for example: don't > store the first copy of a block on the local node but distribute the copies > randomly instread > > > 2010-10-08 > > > > > 发件人： Raj V > 发送时间： 2010-09-28 22:28:12 > 收件人： common-user > 抄送： > 主题： Re: how to make hadoop balance automatically > > The first copy of a block is always stored on the local node. If you want a > balanced distribution, do the data moving from the name node and don't make > the name node into a data node. > Raj > ________________________________ > From: Neil Xu <[email protected]> > To: [email protected] > Sent: Tue, September 28, 2010 3:13:01 AM > Subject: Re: how to make hadoop balance automatically > Hi, Shangan > you can find something useful at > https://issues.apache.org/jira/browse/HADOOP-1652 > and the document > https://issues.apache.org/jira/secure/attachment/12370966/BalancerUserGuide2.pdf > shows how to rebalance. > I think you can try to set more mappers (much larger than the number of > nodes), and see if it will be improved. > Neil > 在 2010年9月28日 下午4:09，shangan <[email protected]>写道： >> I have a cluster of 30 nodes, and I put data into the cluster on one node I >> called "NodeA" here. The consequence is that now this node always stores >> more data than other node, for example other nodes store 10G to 15G,while >> NodeA will store 50G to 60G . >> >> do anyone know what cause such consequence and how to avoid it ? >> btw: I know there a balancer tool can do balance >> >> 2010-09-28 >> >> >> >> shangan >> > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 5484 (20100927) __________ > The message was checked by ESET NOD32 Antivirus. > http://www.eset.com >
