is there any way to change the default storage policy ? for example: don't store the first copy of a block on the local node but distribute the copies randomly instread
2010-10-08 发件人： Raj V 发送时间： 2010-09-28 22:28:12 收件人： common-user 抄送： 主题： Re: how to make hadoop balance automatically The first copy of a block is always stored on the local node. If you want a balanced distribution, do the data moving from the name node and don't make the name node into a data node. Raj ________________________________ From: Neil Xu <[email protected]> To: [email protected] Sent: Tue, September 28, 2010 3:13:01 AM Subject: Re: how to make hadoop balance automatically Hi, Shangan you can find something useful at https://issues.apache.org/jira/browse/HADOOP-1652 and the document https://issues.apache.org/jira/secure/attachment/12370966/BalancerUserGuide2.pdf shows how to rebalance. I think you can try to set more mappers (much larger than the number of nodes), and see if it will be improved. Neil 在 2010年9月28日 下午4:09，shangan <[email protected]>写道： > I have a cluster of 30 nodes, and I put data into the cluster on one node I > called "NodeA" here. The consequence is that now this node always stores > more data than other node, for example other nodes store 10G to 15G,while > NodeA will store 50G to 60G . > > do anyone know what cause such consequence and how to avoid it ? > btw: I know there a balancer tool can do balance > > 2010-09-28 > > > > shangan > __________ Information from ESET NOD32 Antivirus, version of virus signature database 5484 (20100927) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com
