Aha, that works. Any ideas what kind of throughput I can expect, or suggestions for making this run as fast as possible? Obviously exact numbers will depend on cluster config, I won't bore you with the details, but... 10mbit? 100mbit? A gigabit? I've got about 112TB of data to move from the East coast to the West coast, and sooner would be better than later :)
-j On Apr 22, 2011, at 10:38 PM, Jean-Daniel Cryans wrote: > See "Copying between versions of HDFS": > http://hadoop.apache.org/common/docs/r0.20.2/distcp.html#cpver > > J-D > > On Fri, Apr 22, 2011 at 10:37 PM, Jonathan Disher <jdis...@parad.net> wrote: >> I have an existing cluster running hadoop-0.20.1, and I am migrating most of >> the data to a new cluster running -0.20.2. I am seeing this in the namenode >> logs when I try to run a distcp: >> >> @400000004db263bf29c77134 WARN ipc.Server: Incorrect header or version >> mismatch from newNN:46111 got version 4 expected version 3 >> 2011-04-23 05:30:55,999 WARN org.apache.hadoop.ipc.Server: Incorrect header >> or version mismatch from oldNN:48750 got version 3 expected version 4 >> >> When I run my distcp, on either side, it dies with a >> java.io.IOException/java.io.EOFException. >> >> Ideas? Am I screwed? I really don't want to drop my new cluster down to >> 0.20.1. >> >> -j
