JiaLiangC commented on PR #3782: URL: https://github.com/apache/ambari/pull/3782#issuecomment-2080308226
@AnanyaSingh2121 I believe decoupling is necessary, and I'm not clear how you enabled federation, whether modifying the Hadoop configuration or restarting the cluster is required. However, by making the router standalone, testing does not require any changes to the current Hadoop cluster configuration to enable federation. 1. First, we can enable federation on an existing cluster without altering any Hadoop cluster configurations, making federation implementation impact-free for this cluster. It only requires modifying the router to monitor this Hadoop. Therefore, to enable federation for a cluster, one just needs to deploy a router, monitor the NameNode address, set up a common zookeeper and namespace, and then mount it. If we want to separate this cluster from the federated cluster for independent use, only uninstalling this router is needed, likewise without any modifications or restarts required for this existing Hadoop cluster. I think we should distinguish between client and server configurations. Ambari lacks a feature, which is differentiating Hadoop's client configurations from server configurations. After enabling federation, only client configurations need to be modified without conflicting, as the server-side configurations remain unchanged. Client configurations are provided to other big data components for reading. Alternatively, maintain a separate set of federated client configurations manually for other components. After enabling federation, server-side configurations remain unchanged: I provide a client configuration: dfs.nameservices=ns-fed dfs.namenode.rpc-address.ns-fed.r1=your router1 address dfs.namenode.rpc-address.ns-fed.r2=your router2 address dfs.namenode.rpc-address.ns-fed.rn=your routern address fs.defaultFS=hdfs://ns-fed This way, all users utilizing this client configuration will access the federation. If enabling or disabling federation requires restarting the entire cluster, it would be intolerable for an existing cluster running a significant volume of business. It seems you have developed a more comprehensive router solution. You can open a new issue and submit a PR. I will consider closing this PR accordingly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
