HW infrastructure for Hadoop

2013-04-16 Thread Tadas Makčinskas
We are thinking to distribute like 50 node cluster. And trying to figure out what would be a good HW infrastructure (Disks – I/O‘s, RAM, CPUs, network). I cannot actually come around any examples that people ran and found it working well and cost effectively. If anybody could share their best c

steps to fix data block corruption after server failure

2012-12-20 Thread Tadas Makčinskas
Having situation here. Some of our servers went away for a while. As we attached them back to the cluster it appeared that as a result we have multiple Missing/Corrupt blocks and some Mis-replicated blocks. Can't figure out how to solve the issue of restoring the system to a normal working state.

Re: Any way to recover CORRUPT/MISSING blocks? (was: HELP NEEDED: What to do after crash and fsck says that .2% Blocks missing. Namenode in safemode)

2012-12-20 Thread Tadas Makčinskas
Robert J Berger writes: > > Just want to follow up to first thank QwertyM aka Harsh Chouraria for helping me out on the IRC channel. Well > beyond the call of duty! Its people like Harsh that make the HBase/Hadoop community what it is and one of the > joys of working with this technology. And

RE: #of disk partitions on a phisical machine

2012-12-18 Thread Tadas Makčinskas
will try to configure my machines in that manner thanks, Tadas From: Mohammad Tariq [mailto:donta...@gmail.com] Sent: 2012.12.18 13:08 To: user@hadoop.apache.org Subject: Re: #of disk partitions on a phisical machine Hello Tadas, It is actually a good option to have multiple disks attac

#of disk partitions on a phisical machine

2012-12-18 Thread Tadas Makčinskas
I have several physical machines with two mounted hard disks each. Now going through partitioning process and not sure should I leave partitions according to the disk size or it’s better to merge them as one? As I will have just one JT, DN per server not sure how multiple partitions will be ha