errose28 commented on code in PR #10084:
URL: https://github.com/apache/ozone/pull/10084#discussion_r3119223448


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/SCMUpgradeFinalizer.java:
##########
@@ -177,4 +146,76 @@ private void 
waitForDatanodesToFinalize(SCMUpgradeFinalizationContext context)
       }
     }
   }
+
+  public static DatanodeFinalizationCounts getNumFinalizedDatanodes(

Review Comment:
   This should be an instance method in `NodeManager`, not a static method that 
takes an instance as its first/only argument. `DatanodeFinalizationCounts` can 
be moved there as well, or  moved to a standalone class.



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java:
##########
@@ -1153,13 +1155,16 @@ public HddsProtos.UpgradeStatus queryUpgradeStatus() 
throws IOException {
     try {
       getScm().checkAdminAccess(getRemoteUser(), true);
 
-      // Returning a placeholder for now.
-      HddsProtos.UpgradeStatus result = HddsProtos.UpgradeStatus.newBuilder()
-          .setScmFinalized(true)
-          .setNumDatanodesFinalized(10)
-          .setNumDatanodesTotal(10)
-          .setShouldFinalize(true)
-          .build();
+      UpgradeFinalization.Status scmUpgradeStatus = 
scm.getLayoutVersionManager().getUpgradeState();

Review Comment:
   This upgrade status enum is going to be removed in the new version manager. 
We will only use the proto facing version for compatibility with the older 
APIs. This call can be changed to 
`scm.getLayoutVersionManager().needsFinalization()` which easily maps to the 
same call in the [new version 
manager](https://github.com/apache/ozone/blob/50f407e2f61ad0d9ca959fe4bd9d8911ea61f330/hadoop-hdds/framework/src/main/java/org/apache/hadoop/ozone/upgrade/ComponentVersionManager.java#L87)
 when we switch. From here we can:
   - Remove the `isScmFinalized` helper method
   - Add a `DatanodeFinalizationCounts#allNodesFinalized` method as suggested 
[here](https://github.com/apache/ozone/pull/10084/changes#r3119214490)
   - Inline the `shouldFinalize` method to just a variable: `shouldFinalize = 
scmFinalized && datanodeFinalizationCounts.allNodesFinalized()`
   - Inline `buildUpgradeStatus` to invoke the builder directly in this method 
since the labelled builder setters are clearer than the unlabeled method 
parameters and it is now just a passthrough.
   
   This minimizes the amount of upgrade specific logic in the server side 
translator.



##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/server/TestSCMClientProtocolServer.java:
##########
@@ -138,6 +140,80 @@ public void testScmListContainer() throws Exception {
         HddsProtos.ReplicationFactor.THREE).getContainerInfoList().size());
   }
 
+  @Test

Review Comment:
   With the simplifications mentioned above I don't think we need all these 
tests, it looks like the AI went overboard here. These aren't actually testing 
the `queryUpgradeStatus` method either. Looking at the other methods in this 
class it looks like we usually only cover them with integration tests. So in 
this PR I would switch all test usage of `queryUpgradeFinalizationProgress` to 
`queryUpgradeStatus` for test coverage.



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/SCMUpgradeFinalizer.java:
##########
@@ -177,4 +146,76 @@ private void 
waitForDatanodesToFinalize(SCMUpgradeFinalizationContext context)
       }
     }
   }
+
+  public static DatanodeFinalizationCounts getNumFinalizedDatanodes(
+      NodeManager nodeManager) {
+    int finalizedNodes = 0;
+    int totalHealthyNodes = 0;
+    int unfinalizedNodes = 0;
+
+    for (DatanodeDetails dn : nodeManager.getAllNodes()) {
+      try {
+        // Only check HEALTHY nodes. STALE/DEAD nodes will be told to
+        // finalize when they recover.
+        if (!nodeManager.getNodeStatus(dn).isHealthy()) {

Review Comment:
   This doesn't address the [previous 
comment](https://github.com/apache/ozone/pull/10084#discussion_r3110465695) 
because it is only checking the health state returned by `getNodeStatus`, not 
the operational state. We can probably just loop over 
`nodeManager.getNodes(NodeStatus.inServiceHealthy())` then not need to skip any 
entries in the loop.



##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java:
##########
@@ -737,6 +738,71 @@ public void testProcessLayoutVersion() throws IOException {
     testProcessLayoutVersionReportHigherMlv();
   }
 
+  @Test
+  public void testDatanodeFinalizedCounterTracksLayoutVersionReports()
+      throws IOException, AuthenticationException {
+    try (SCMNodeManager nodeManager = createNodeManager(getConf())) {
+      DatanodeDetails node =
+          HddsTestUtils.createRandomDatanodeAndRegister(nodeManager);
+      assertEquals(1, SCMUpgradeFinalizer.getNumFinalizedDatanodes(nodeManager)
+              .getNumFinalizedDatanodes(),
+          "Initial datanode should be counted as finalized");
+
+      int softwareVersion =
+          nodeManager.getLayoutVersionManager().getSoftwareLayoutVersion();
+      int metadataVersion =
+          nodeManager.getLayoutVersionManager().getMetadataLayoutVersion();
+      nodeManager.processLayoutVersionReport(node,
+          LayoutVersionProto.newBuilder()
+              .setMetadataLayoutVersion(metadataVersion - 1)
+              .setSoftwareLayoutVersion(softwareVersion)
+              .build());
+      assertEquals(0, SCMUpgradeFinalizer.getNumFinalizedDatanodes(nodeManager)
+              .getNumFinalizedDatanodes(),
+          "Lower metadata layout version should decrement finalized count");
+
+      nodeManager.processLayoutVersionReport(node,
+          LayoutVersionProto.newBuilder()
+              .setMetadataLayoutVersion(metadataVersion)
+              .setSoftwareLayoutVersion(softwareVersion)
+              .build());
+      assertEquals(1, SCMUpgradeFinalizer.getNumFinalizedDatanodes(nodeManager)
+              .getNumFinalizedDatanodes(),
+          "Restored metadata layout version should restore finalized count");
+    }
+  }
+
+  @Test
+  public void testDatanodeFinalizedCounterTracksRegistrationAndRemoveNode()
+      throws IOException, AuthenticationException, NodeNotFoundException {
+    try (SCMNodeManager nodeManager = createNodeManager(getConf())) {
+      DatanodeDetails finalizedNode =
+          registerWithCapacity(nodeManager, CORRECT_LAYOUT_PROTO, success);
+      assertEquals(1, SCMUpgradeFinalizer.getNumFinalizedDatanodes(nodeManager)
+              .getNumFinalizedDatanodes(),
+          "Finalized registration should increment finalized count");
+
+      DatanodeDetails nonFinalizedNode =
+          registerWithCapacity(nodeManager, SMALLER_MLV_LAYOUT_PROTO, success);
+      assertEquals(1, SCMUpgradeFinalizer.getNumFinalizedDatanodes(nodeManager)
+              .getNumFinalizedDatanodes(),
+          "Non-finalized registration should not increment finalized count");
+
+      nonFinalizedNode.setPersistedOpState(
+          HddsProtos.NodeOperationalState.DECOMMISSIONED);

Review Comment:
   We should have test coverage for all ineligible health and op states to make 
the intent clear. It should be fast to enumerate all of them.



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/SCMUpgradeFinalizer.java:
##########
@@ -177,4 +146,76 @@ private void 
waitForDatanodesToFinalize(SCMUpgradeFinalizationContext context)
       }
     }
   }
+
+  public static DatanodeFinalizationCounts getNumFinalizedDatanodes(
+      NodeManager nodeManager) {
+    int finalizedNodes = 0;
+    int totalHealthyNodes = 0;
+    int unfinalizedNodes = 0;
+
+    for (DatanodeDetails dn : nodeManager.getAllNodes()) {
+      try {
+        // Only check HEALTHY nodes. STALE/DEAD nodes will be told to
+        // finalize when they recover.
+        if (!nodeManager.getNodeStatus(dn).isHealthy()) {
+          continue;
+        }
+        totalHealthyNodes++;
+        DatanodeInfo datanodeInfo = nodeManager.getDatanodeInfo(dn);
+        if (datanodeInfo == null) {
+          LOG.warn("Could not get DatanodeInfo for {}, skipping in " +
+              "finalization wait.", dn.getHostName());
+          continue;
+        }
+
+        LayoutVersionProto dnLayout = datanodeInfo.getLastKnownLayoutVersion();
+        int dnMlv = dnLayout.getMetadataLayoutVersion();
+        int dnSlv = dnLayout.getSoftwareLayoutVersion();
+
+        if (dnMlv < dnSlv) {
+          // Datanode has not yet finalized
+          unfinalizedNodes++;
+          LOG.debug("Datanode {} has not yet finalized: MLV={}, SLV={}",
+              dn.getHostName(), dnMlv, dnSlv);
+        } else {
+          finalizedNodes++;
+        }
+      } catch (NodeNotFoundException e) {
+        // Node was removed while we were iterating. This is OK, skip it.
+        LOG.debug("Node {} not found while waiting for finalization, " +
+            "skipping.", dn);
+      }
+    }
+
+    return new DatanodeFinalizationCounts(finalizedNodes, totalHealthyNodes, 
unfinalizedNodes);
+  }
+
+  /**
+   * Class to store the number finalized, unfinalized and healthy datanodes.
+   */
+  public static final class DatanodeFinalizationCounts {
+    private final int numFinalizedDatanodes;
+    private final int totalHealthyDatanodes;
+    private final int numUnfinalizedDatanodes;

Review Comment:
   Do we actually need three fields here? I think we can just track total 
healthy in-service nodes and number of finalized nodes. The number of 
unfinalized nodes can be derived from that. These are the only two fields the 
upgrade status API uses.
   
   The finalization wait code can be changed to check that the healthy node 
count equals the finalized count. We could also wrap that in an 
`allNodesFinalized` method inside `DatanodeFinalizationCount`.
   
   `getNumUnfinalizedDatanodes` has no test coverage, but I think it should be 
removed. `getTotalHealthyDatanodes` also has no test coverage, which should be 
added.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to