Even though hbase runs on 'commodity' hardware, it's important to remember that to achieve scale you need to do a bit better than 1 cpu 1 gb ram type things.
I tend to think in per-core specs, that way you don't have to worry about 2 core vs 4 core vs 8 core - you buy whatever is most economical at the time. I'd match 1 core with 2-4gb ram. You'll want to dedicate 4 gb of ram to hbase, it'll make life easier. You should also be getting 64-bit systems and running a 64 bit distro on it and a jvm that has -d64 available. A word about master... For hbase, the master is (a) important and (b) very lightweight. Meaning the master doesn't use much ram. For hadoop, things are different, because the HDFS master is relatively light weight, but needs lots of ram (every file takes up memory space). On my cluster, the master is the same node-type as the rest. I've heard recommendations to buy better hardware for your master - if you lose a disk, your whole cluster goes down. I can't say i disagree with that sentiment. Good luck! On Fri, Mar 27, 2009 at 10:43 PM, Yabo-Arber Xu <arber.resea...@gmail.com>wrote: > Hi Amandeep, > > I just did the same investigation not long ago, and I was recommended to > get > Amazon EC2 X-Large > equivalent< > http://www.google.com/url?q=http%3A%2F%2Faws.amazon.com%2Fec2%2F%23pricing&sa=D&sntz=1&usg=AFrqEzc1z8IB5p0hIR7SGe-mRVRZXW7Lvg > >nodes: > , 8 > EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 15 GB > memory, 1690 GB of instance storage, 64-bit platform. One EC2 Compute Unit > (ECU) is equivalent to CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 > Xeon processor. > > For more details, you may want to refer to Daniel Leffel's experience on > setting up HBase< > http://www.google.com/url?q=http%3A%2F%2Fmail-archives.apache.org%2Fmod_mbox%2Fhadoop-hbase-user%2F200805.mbox%2F%253C25e5a0c00805072129w3b54599r286940f134c6f235%40mail.gmail.com%253E&sa=D&sntz=1&usg=AFrqEzcmU5_eMlrfoBJwCTxOg9I8NeJ2JQ > > > > Hope it helps. > > Best, > Arber > > On Fri, Mar 27, 2009 at 10:07 PM, Amandeep Khurana <ama...@gmail.com> > wrote: > > > What are the typical hardware config for a node that people are using for > > Hadoop and HBase? I am setting up a new 10 node cluster which will have > > HBase running as well that will be feeding my front end directly. > > Currently, > > I had a 3 node cluster with 2 GB of RAM on the slaves and 4 GB of RAM on > > the > > master. This didnt work very well due to the RAM being a little low. > > > > I got some config details from the powered by page on the Hadoop wiki, > but > > nothing like that for Hbase. > > > > > > Amandeep Khurana > > Computer Science Graduate Student > > University of California, Santa Cruz > > >
