Re: Big machines or (relatively) small machines?

2010-06-10 Thread Todd Lipcon
;t have to guess or reconstruct good settings from 10-100 emails on the > list? If I understood the issues I would be happy to write it up, but I am > afraid I don't. > > Thanks, > Dave > > -Original Message- > From: Ryan Rawson [mailto:ryano...@gmail.com] > Sent

Re: Big machines or (relatively) small machines?

2010-06-10 Thread Sean Bigdatafun
00 emails on the > list? If I understood the issues I would be happy to write it up, but I am > afraid I don't. > > Thanks, > Dave > > -Original Message- > From: Ryan Rawson [mailto:ryano...@gmail.com] > Sent: Monday, June 07, 2010 10:51 PM > To: user@hba

Re: Big machines or (relatively) small machines?

2010-06-10 Thread Edward Capriolo
ess or reconstruct good settings from 10-100 emails on the > list? If I understood the issues I would be happy to write it up, but I am > afraid I don't. > > Thanks, > Dave > > -Original Message- > From: Ryan Rawson [mailto:ryano...@gmail.com] > Sent:

RE: Big machines or (relatively) small machines?

2010-06-10 Thread Buttler, David
, but I am afraid I don't. Thanks, Dave -Original Message- From: Ryan Rawson [mailto:ryano...@gmail.com] Sent: Monday, June 07, 2010 10:51 PM To: user@hbase.apache.org Subject: Re: Big machines or (relatively) small machines? I would take it one notch smaller, 32GB ram per node i

Re: Big machines or (relatively) small machines?

2010-06-08 Thread Tim Robertson
> - Do you plan to serve data out of HBase or will you just use it for > MapReduce? Or will it be a mix (not recommended)? I am also curious what would be the recommended deployment when you have this need (e.g. building multiple Lucene indexes which hold only the Row ID, so building is MR intens

Re: Big machines or (relatively) small machines?

2010-06-07 Thread Sean Bigdatafun
On Mon, Jun 7, 2010 at 10:46 AM, Jean-Daniel Cryans wrote: > It really depends on your usage pattern, but there's a balance wrt > cost VS hardware you must achieve. At StumbleUpon we run with 2xi7, > 24GB, 4x 1TB and it works like a charm. The only thing I would change > is maybe more disks/node b

Re: Big machines or (relatively) small machines?

2010-06-07 Thread Ryan Rawson
I would take it one notch smaller, 32GB ram per node is probably more than enough... It would be hard to get full utilization of 128GB ram, and maybe even 64GB. With 32GB you might even be able to get 2GB dimms (much cheaper). -ryan On Mon, Jun 7, 2010 at 10:48 PM, Sean Bigdatafun wrote: > On

Re: Big machines or (relatively) small machines?

2010-06-07 Thread Sean Bigdatafun
On Mon, Jun 7, 2010 at 1:13 PM, Todd Lipcon wrote: > If those are your actual specs, I would definitely go with 16 of the > smaller > ones. 128G heaps are not going to work well in a JVM, you're better off > running with more nodes with a more common configuration. > I am not using one JVM on a

Re: Big machines or (relatively) small machines?

2010-06-07 Thread Todd Lipcon
If those are your actual specs, I would definitely go with 16 of the smaller ones. 128G heaps are not going to work well in a JVM, you're better off running with more nodes with a more common configuration. -Todd On Mon, Jun 7, 2010 at 1:46 PM, Jean-Daniel Cryans wrote: > It really depends on yo

Re: Big machines or (relatively) small machines?

2010-06-07 Thread Jean-Daniel Cryans
It really depends on your usage pattern, but there's a balance wrt cost VS hardware you must achieve. At StumbleUpon we run with 2xi7, 24GB, 4x 1TB and it works like a charm. The only thing I would change is maybe more disks/node but that's pretty much it. Some relevant questions: - Do you have a

Big machines or (relatively) small machines?

2010-06-02 Thread Sean Bigdatafun
I am thinking of the following problem lately. I started thinking of this problem in the following context. I have a predefined budget and I can either -- A) purchase 8 more powerful servers (4cpu x 4 cores/cpu + 128GB mem + 16 x 1TB disk) or -- B) purchase 16 less powerful servers(2cpu x 4 c