Re: HDFS + ZooKeeper

2011-04-23 Thread Mag Gam
Will the avatar node be part of 0.22? On Sat, Apr 23, 2011 at 6:29 PM, Dhruba Borthakur wrote: > The AvatarNode does use zookeeper (but since this is not directly related to > Apache HDFS code, if u have more questions, please send it to me directly). > The latest AvatarNode code is in > https:/

Re: HDFS + ZooKeeper

2011-04-23 Thread Dhruba Borthakur
The AvatarNode does use zookeeper (but since this is not directly related to Apache HDFS code, if u have more questions, please send it to me directly). The latest AvatarNode code is in https://github.com/facebook/hadoop-20-append/tree/master/src/contrib/highavailability/src/java/org/apache/hadoop

Re: HDFS + ZooKeeper

2011-04-23 Thread Jason Rutherglen
This is an interesting question. AvatarNode doesn't use Zookeeper? Is there a drawback to implementing ZK for the NameNode other than the potential bugs from a newly engineered feature? Perhaps this issue is the main one for implementing the NameNode in ZK? HDFS-1623 On Fri, Apr 22, 2011 at 10:

Re: distcp problems going from hadoop-0.20.1 to -0.20.2

2011-04-23 Thread Jean-Daniel Cryans
Errr really? Well shipping a bunch of hard drives should be faster. J-D On Apr 23, 2011 12:17 AM, "Jonathan Disher" wrote: > Aha, that works. > > Any ideas what kind of throughput I can expect, or suggestions for making this run as fast as possible? Obviously exact numbers will depend on cluster

Re: distcp problems going from hadoop-0.20.1 to -0.20.2

2011-04-23 Thread Jonathan Disher
Aha, that works. Any ideas what kind of throughput I can expect, or suggestions for making this run as fast as possible? Obviously exact numbers will depend on cluster config, I won't bore you with the details, but... 10mbit? 100mbit? A gigabit? I've got about 112TB of data to move from the