How to clear EXPIRED routers?

2023-10-18 Thread 杨光
Hi everyone! I'm using hadoop 3.3.4, and started 5 hdfs routers on servers. Now I have to delete two of them using: hdfs --daemon stop dfsrouter. The commend executed successfully, but on router WebUI (http://url-to-router-webui:50071), it shows 5 routers but 2 of them are on EXPIRED status. How

Re: DistCP from Hadoop 2.X to 3.X - where to compute

2023-10-18 Thread 杨光
Hi PA, We just did the same work recently, copying data from hadoop 2 to hadoop 3, to be precise, src hadoop version was CDH hadoop-2.6 (5 hdfs nameservices federation), dst hadoop version was hadoop 3.3.4. Both clusters are protected with Kerberos, and of course, two realms have been trusted with

DistCP from Hadoop 2.X to 3.X - where to compute

2023-10-18 Thread Paul-Adrien Cordonnier
Hey team, We're planning to migrate some of our data from an obsolete Hadoop 2.7 to a more recent Hadoop 3. There is approximately 60 Datanodes on the old one and approximately 10 on the new ones. It will get bigger over the next months but since some of the use cases are migrating out of ha