[ https://issues.apache.org/jira/browse/HDFS-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14249207#comment-14249207 ]
Jesse Yates commented on HDFS-6440: ----------------------------------- I'll post the updated patch somewhere, if you like. However, for the meantime, responses! I think some stuff got a little messed up with the trunk port... these are all great catches! bq. I guess the default value of isPrimaryCheckPointer might be a typo, which should be false. Yup and bq. is there a case that SNN switches from primary check pointer to non-primary check pointer Not that I can find either :) Should be that we track success in the transfer result from the upload and then update the primary checkpoint status based on the success therein (so if no upload is valid, no longer the primary). bq. 2. Is the following condition correct? I think only sendRequest is needed. Kinda. I think it should actually be: {code} if (needCheckpoint) { doCheckpoint(sendRequest); {code} and then make and save the checkpoint, but only send it if we need to (sendRequest == true). bq. If it is the case, are these duplicated conditions? The quiet period should be larger than the usual checking period (multiplier is 1.5), so its the separation of the sending the request vs. taking the checkpoint that comes into conflict here. I think this logic makes more sense with the above change for separating the use of needCheckpoint and sendCheckpoint. bq. might be easier to let ANN calculate the above conditions... It could be a nice optimization later. Definitely! Was trying to keep the change footprint down. bq. When it uploads fsimage, are SC_CONFLICT and SC_EXPECTATION_FAILED not handled in the SNN in the current patch They somewhat are - they don't throw an exception back out, but are marked as 'failures'. Either way, in the new version of the patch (coming), in keeping with the changes for setting isPrimaryCheckpointer described above, the primaryCheckpointStatus is set to the correct value. Either, it got a NOT_ACTIVE_NAMENODE_FAILURE on the other SNN or it tried to upload an old transaction to the ANN (OLD_TRANSACTION_ID_FAILURE). If its the first, the other NN could succeed (making this pSNN) or its an older transaction, so it shouldn't be the pSNN. With the caveat you mentioned in your last comment about both SNN thinking they are pSNN. bq. Could you set EditLogTailer#maxRetries to private final? That wasn't part of my change set - the code was already there. It looks like that its used to set the edit log in testing. bq. Do we need to enforce an acceptable value range for maxRetries An interesting idea! I didn't want to spin forever there and instead surface the issue to the user by bringing down the NN. My question back is, is there another process that will bring down the NN if it cannot reach the other NNs? Otherwise, it can get hopelessly out of date and look like a valid standby when it really isn't. bq. NN when nextNN = nns.size() - 1 and maxRetries = 1 Oh, yeah - that's a problem, regardless of the above. Pending patch should fix that. Coming patch should also fix the remainder of the formatting issues. > Support more than 2 NameNodes > ----------------------------- > > Key: HDFS-6440 > URL: https://issues.apache.org/jira/browse/HDFS-6440 > Project: Hadoop HDFS > Issue Type: New Feature > Components: auto-failover, ha, namenode > Affects Versions: 2.4.0 > Reporter: Jesse Yates > Assignee: Jesse Yates > Attachments: Multiple-Standby-NameNodes_V1.pdf, > hdfs-6440-cdh-4.5-full.patch, hdfs-multiple-snn-trunk-v0.patch > > > Most of the work is already done to support more than 2 NameNodes (one > active, one standby). This would be the last bit to support running multiple > _standby_ NameNodes; one of the standbys should be available for fail-over. > Mostly, this is a matter of updating how we parse configurations, some > complexity around managing the checkpointing, and updating a whole lot of > tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)