Re: why not zookeeper for the namenode

2010-02-23 Thread Steve Loughran
Eli Collins wrote: From what I read, I thought, that bookkeeper would be the ideal enhancement for the namenode, to make it distributed and therefor finaly highly available. Being distributed doesn't imply high availability. Availability is about minimizing downtime. For example, a primary that

Re: why not zookeeper for the namenode

2010-02-23 Thread Steve Loughran
E. Sammer wrote: On 2/22/10 12:53 PM, zlatin.balev...@barclayscapital.com wrote: My 2 cents: If the NN stores all state behind a javax.Cache façade it will be possible to use all kinds of products (commercial, open source, facades to ZK) for redundancy, load balancing, etc. This would be pret

Re: why not zookeeper for the namenode

2010-02-23 Thread Eli Collins
> From what I read, I thought, that bookkeeper would be the ideal enhancement > for the namenode, to make it distributed and therefor finaly highly available. Being distributed doesn't imply high availability. Availability is about minimizing downtime. For example, a primary that can fail over to

Re: why not zookeeper for the namenode

2010-02-22 Thread E. Sammer
On 2/22/10 12:53 PM, zlatin.balev...@barclayscapital.com wrote: My 2 cents: If the NN stores all state behind a javax.Cache façade it will be possible to use all kinds of products (commercial, open source, facades to ZK) for redundancy, load balancing, etc. This would be pretty interesting fr

RE: why not zookeeper for the namenode

2010-02-22 Thread Zlatin.Balevsky
, February 22, 2010 12:47 PM To: t...@cloudera.com; tho...@koch.ro; common-user@hadoop.apache.org Subject: Re: why not zookeeper for the namenode You can see the work Flavio and Luca did here integrating the NN with BookKeeper. http://bit.ly/aELMbH This addresses some issues but not all of them (and I&#

Re: why not zookeeper for the namenode

2010-02-22 Thread Patrick Hunt
You can see the work Flavio and Luca did here integrating the NN with BookKeeper. http://bit.ly/aELMbH This addresses some issues but not all of them (and I'm not an expert on NN to be able to explain them all). I believe it would not address "availability" in the sense that bk alone does not m

Re: why not zookeeper for the namenode

2010-02-19 Thread Todd Lipcon
On Fri, Feb 19, 2010 at 12:41 AM, Thomas Koch wrote: > Hi, > > yesterday I read the documentation of zookeeper and the zk contrib bookkeeper. > From what I read, I thought, that bookkeeper would be the ideal enhancement > for the namenode, to make it distributed and therefor finaly highly availabl