Re: DistCP from Hadoop 2.X to 3.X - where to compute

2023-10-18 Thread 杨光
Hi PA, We just did the same work recently, copying data from hadoop 2 to hadoop 3, to be precise, src hadoop version was CDH hadoop-2.6 (5 hdfs nameservices federation), dst hadoop version was hadoop 3.3.4. Both clusters are protected with Kerberos, and of course, two realms have been trusted

DistCP from Hadoop 2.X to 3.X - where to compute

2023-10-18 Thread Paul-Adrien Cordonnier
Hey team, We're planning to migrate some of our data from an obsolete Hadoop 2.7 to a more recent Hadoop 3. There is approximately 60 Datanodes on the old one and approximately 10 on the new ones. It will get bigger over the next months but since some of the use cases are migrating out of