Hi Genius, I am currently involved in a project that will create/start multiple namenodes(It is different with Federation that: We want to partition the metadata not only by directory, and may support other partition schemes, and we want to support the distributed operations that cross multiple namenodes), It would be great if I can get some suggestions:
(1). How to create/start multiple namenodes? Suppose I want to create 2 Namenodes, and one in machine A and the other in machien B, I should start the Namenode in both machines(For example, each machine will call the initialize() method in NameNode.java), correct? Yes, I definitely need to change the related code like partition the metadata into different namenode and change the data structure etc, just want to make sure that I can use this way to simply start multiple namenodes. Or do you have any suggestions to do so? (2). Once I have multiple Namenodes running, do you think what is the best/simple way to change the HDFS client code to let the clients send the requests to a random Namenode? (3). I need to support the communication between the Namenodes, my current plan is to create one more protocol that supports the communication between the Namenodes, something like the clientProtocol and ClientDataNodeProtocol. Do you think is it easy to do so? Or do you have other suggestions to support the communication between the Namenodes? Thanks so much. Kun
