Re: High Availability hadoop cluster.

2014-11-06 Thread Alexander Pivovarov
2 boxes for 2 NNs (better dedicated boxes) min 3 JNs min 3 ZKs JNs and ZKs can share boxes with other services On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets oruchov...@gmail.com wrote: Hello. We are using hortonwork distribution and want to evaluate HA capabilities. Can community please

Re: High Availability hadoop cluster.

2014-11-06 Thread Oleg Ruchovets
Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of hortonworks distributions. Sorry for dummy question - what is JN is? Can you please please point me on some manual wiki for installation / configuration. Thanks Oleg. On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov

Re: High Availability hadoop cluster.

2014-11-06 Thread Andrej
JN is the journalnode. You have to use it for HA. Like zookeeper, you need odd numbers of JN. Anyway ZKFC is neccessary too for automatic failover. On 2014.11.06. 9:45, Oleg Ruchovets wrote: Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of hortonworks distributions.

Re: High Availability hadoop cluster.

2014-11-06 Thread Oleg Ruchovets
Great. Thank you for the link. Just to be sure - JN can be installed on data nodes like zookeeper? If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK and JN on datanodes machines? Thanks Oleg. On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov apivova...@gmail.com wrote:

Re: High Availability hadoop cluster.

2014-11-06 Thread Leonid Fedotov
Oleg, why don’t you talk to me? Put 1 ZK and 1 JN on NN boxes, third one on one of the data nodes. HA wizard will automatically remove SNN from your cluster and put second NN in it’s place instead. If something does not work as you expected, give me a call. Thank you! Leonid Fedotov Systems

Re: High Availability hadoop cluster.

2014-11-06 Thread Alexander Pivovarov
For 17 box cluster it's probably good to run 5 ZKs and 5 JNs So, run 2 ZKs on 2 NNs 3 ZKs on 3 DNs same for JNs you can start additional ZKs and JNs after you are done with initial Enabling HA in Ambari On Thu, Nov 6, 2014 at 3:01 AM, Oleg Ruchovets oruchov...@gmail.com wrote: Great.

High Availability hadoop cluster.

2014-11-05 Thread Oleg Ruchovets
Hello. We are using hortonwork distribution and want to evaluate HA capabilities. Can community please share the best practices and potential problems? Is it required additional hardware? Thanks Oleg.