Re: distcp problems going from hadoop-0.20.1 to -0.20.2

2011-04-23 Thread Jean-Daniel Cryans
Errr really? Well shipping a bunch of hard drives should be faster. J-D On Apr 23, 2011 12:17 AM, "Jonathan Disher" wrote: > Aha, that works. > > Any ideas what kind of throughput I can expect, or suggestions for making this run as fast as possible? Obviously exact numbers will depend on cluster

Re: distcp problems going from hadoop-0.20.1 to -0.20.2

2011-04-23 Thread Jonathan Disher
Aha, that works. Any ideas what kind of throughput I can expect, or suggestions for making this run as fast as possible? Obviously exact numbers will depend on cluster config, I won't bore you with the details, but... 10mbit? 100mbit? A gigabit? I've got about 112TB of data to move from the

Re: distcp problems going from hadoop-0.20.1 to -0.20.2

2011-04-22 Thread Jean-Daniel Cryans
See "Copying between versions of HDFS": http://hadoop.apache.org/common/docs/r0.20.2/distcp.html#cpver J-D On Fri, Apr 22, 2011 at 10:37 PM, Jonathan Disher wrote: > I have an existing cluster running hadoop-0.20.1, and I am migrating most of > the data to a new cluster running -0.20.2.  I am s

distcp problems going from hadoop-0.20.1 to -0.20.2

2011-04-22 Thread Jonathan Disher
I have an existing cluster running hadoop-0.20.1, and I am migrating most of the data to a new cluster running -0.20.2. I am seeing this in the namenode logs when I try to run a distcp: @40004db263bf29c77134 WARN ipc.Server: Incorrect header or version mismatch from newNN:46111 got version