Reading the HDFS Federation documentation, it seems that it provides support for multiple NameNodes in a single cluster. The DataNodes are shared across all NameNodes. It seems one can take a single hadoop cluster, and add HDFS Federation, but I do not see any way to take multiple hadoop clusters and make the composite cluster federated; and then, undo/change the definition of the composite cluster back to individual clusters as needed.
Am I reading this correctly? Or is there a way to define a federated hadoop cluster from multiple clusters? It seems the DataNodes would need to be separate by original cluster to accomplish this. My interest stems from a desire to allow a compute service to issue map/reduce jobs to the multiple hadoop cluster, without say needing internally managed catalogs, etc. Ideally the composite hadoop cluster could be dynamically changed to include those hadoop clusters of interest, with the ease analogous to changing /etc/fstab. Appreciate any thoughts on this. LAT