Hi all,
I have a doubt about how to deploy the Zookeeper in a NN HA cluster,
As far as I know, I need at least three nodes to run three ZooKeeper
FailOver Controller (ZKFC). I plan to put these 3 daemons this way:
- Active NameNode + 1 ZKFC daemon
- Standby NameNode + 1 ZKFC daemon
- JobTracker
Hi,
There are two different things here: Automatic Failover and Quorum
Journal Manager. The former, used via a ZooKeeper Failover Controller,
is to manage failovers automatically (based on health checks of NNs).
The latter, used via a set of Journal Nodes, is a medium of shared
storage for namesys
Unsubscribe
On Dec 27, 2012 8:04 PM, "ESGLinux" wrote:
> Hi all,
>
> I have a doubt about how to deploy the Zookeeper in a NN HA cluster,
>
> As far as I know, I need at least three nodes to run three ZooKeeper
> FailOver Controller (ZKFC). I plan to put these 3 daemons this way:
>
> - Active Na
Hi,
well, If I have understand you I can configure my NN HA cluster this way:
- Active NameNode + 1 ZKFC daemon + Journal Node
- Standby NameNode + 1 ZKFC daemon + Journal Node
- JobTracker node + 1 ZKFC daemon + Journal Node,
Is this right?
Thanks in advance,
ESGLinux,
2012/12/27 Harsh J
>
You need the following:
- active namenode + zkfc
- standby namenode + zkfc
- pool of journal nodes (odd number, 3 or more)
- pool of zookeeper nodes (odd number, 3 or more)
As the journal nodes hold the namesystem transactions they should not be
co-located with the namenodes in case of failure.
Hi Craig,
I´m a bit confused, I have read this from cloudera:
https://ccp.cloudera.com/display/CDH4DOC/Hardware+Configuration+for+Quorum-based+Storage
The JournalNode daemon is relatively lightweight, so these daemons can
reasonably be collocated on machines with other Hadoop daemons, for example
OK, I have reliable storage on my datanodes so not an issue for me. If
that's what Cloudera recommends then I'm sure it's fine.
On Dec 28, 2012 10:38 AM, "ESGLinux" wrote:
> Hi Craig,
>
> I´m a bit confused, I have read this from cloudera:
> https://ccp.cloudera.com/display/CDH4DOC/Hardware+Conf
Thank you for your answer Craig,
I´m planning my cluster and for now I´m not sure how many machines I need;-)
If I have doubt i´ll what clouder say and If have a problem I have where to
ask for explications :-)
ESGLinux
2012/12/28 Craig Munro
> OK, I have reliable storage on my datanodes so
Hi ESGLinux,
In production, you need to run QJM on at least 3 nodes. You also need
to run ZKFC on at least 3 nodes. You can run them on the same nodes
if you like, though.
Of course, none of this is "needed" to set up an example cluster. If
you just want to try something out, you can run every
On Mon, Jan 14, 2013 at 11:49 AM, Colin McCabe wrote:
> Hi ESGLinux,
>
> In production, you need to run QJM on at least 3 nodes. You also need
> to run ZKFC on at least 3 nodes. You can run them on the same nodes
> if you like, though.
Er, this should read "You also need to run ZooKeeper on at
Hi all,
I´m only testing the new HA feature. I´m not in a production system,
Well, let´s talk about the number of nodes and the ZKFC daemons.
In this url:
https://ccp.cloudera.com/display/CDH4DOC/HDFS+High+Availability+Initial+Deployment#HDFSHighAvailabilityInitialDeployment-DeployingAutomaticFa
Hi,
I fail to see your confusion.
ZKFC != ZK
ZK is a quorum software, like QJM is. The ZK peers are to be run odd in
numbers, such as JNs are to be.
ZKFC is something the NN needs for its Automatic Failover capability. It is
a client to ZK and thereby demands ZK's presence; for which the odd #
Hi Harsh,
Now I´m confussed at all :-
as you pointed ZKFC runs only in the NN. That´s looks right.
So, what are ZK peers (the odd number I´m looking for) and where I have to
run them? on another 3 nodes?
As I can read from the previous url:
In a typical deployment, ZooKeeper daemons are co
No, ZooKeeper daemons == http://zookeeper.apache.org.
On Tue, Jan 15, 2013 at 3:38 PM, ESGLinux wrote:
> Hi Harsh,
>
> Now I´m confussed at all :-
>
> as you pointed ZKFC runs only in the NN. That´s looks right.
>
> So, what are ZK peers (the odd number I´m looking for) and where I have to
ok,
Thats the origin of my confussion, I thought they were the same.
I´m going to read this doc to bring me a bit of light about ZooKeeper..
thank you very much for your help,
ESGLinux,
2013/1/15 Harsh J
> No, ZooKeeper daemons == http://zookeeper.apache.org.
>
>
> On Tue, Jan 15, 2013 at 3
15 matches
Mail list logo
