sadanand48 commented on code in PR #11962:
URL: https://github.com/apache/ozone/pull/11962#discussion_r4510086407


##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeQueueMetrics.java:
##########
@@ -60,6 +61,16 @@ public final class DatanodeQueueMetrics implements 
MetricsSource {
       "ContainerActionQueue";
   public static final String PIPELINE_ACTION_QUEUE_PREFIX =
       "PipelineActionQueue";
+  public static final String COMMAND_STATUS_REPORT_QUEUE_PREFIX =
+      "CommandStatusReportQueue";
+  public static final String COMMAND_STATUS_REPORT_SENT_PREFIX =
+      "CommandStatusReportSent";
+  public static final String COMMAND_STATUS_REPORT_DEFERRED_PREFIX =

Review Comment:
   Agreed. removed the deferred-bytes metric and its associated map



##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/StateContext.java:
##########
@@ -392,6 +395,71 @@ public List<Message> getAllAvailableReports(
     return getAllAvailableReportsUpToLimit(endpoint, maxLimit);
   }
 
+  /**
+   * Returns available reports other than command status reports. Full reports
+   * are selected before incremental reports, and the total number returned is
+   * capped by {@code maxLimit}.
+   */
+  public List<Message> getAvailableNonCommandStatusReports(
+      HostAndPort endpoint, int maxLimit) {
+    List<Message> reports = getFullReports(endpoint, maxLimit);
+    reports.addAll(getIncrementalReportsExcludingCommandStatus(
+        endpoint, maxLimit - reports.size()));
+    return reports;
+  }
+
+  /**
+   * Selects command status reports within both the report count and serialized
+   * byte limits. An aggregate report may be split between complete command
+   * statuses, but an individual command status is never split.
+   */
+  public CommandStatusReportSelection getCommandStatusReports(
+      HostAndPort endpoint, int maxLimit, long maxBytes,
+      long maxConfiguredBytes) {
+    List<CommandStatusReportsProto> selected = new LinkedList<>();
+    long selectedBytes = 0;
+    long oversizedBytes = 0;
+    synchronized (incrementalReportsQueue) {
+      List<Message> reportsForEndpoint =
+          incrementalReportsQueue.get(endpoint);
+      if (reportsForEndpoint != null) {
+        ListIterator<Message> iterator = reportsForEndpoint.listIterator();
+        while (iterator.hasNext() && selected.size() < maxLimit) {
+          Message message = iterator.next();
+          if (!(message instanceof CommandStatusReportsProto)) {
+            continue;
+          }
+
+          CommandStatusReportsProto report =
+              (CommandStatusReportsProto) message;
+          long remainingBytes = maxBytes - selectedBytes;
+          long reportBytes = getCommandStatusReportWireSize(report);
+          if (reportBytes <= remainingBytes) {
+            selected.add(report);
+            selectedBytes += reportBytes;
+            iterator.remove();
+            continue;
+          }
+
+          CommandStatusReportSplit split =
+              splitCommandStatusReport(report, remainingBytes);
+          if (split.getSelected() != null) {
+            selected.add(split.getSelected());
+            selectedBytes +=
+                getCommandStatusReportWireSize(split.getSelected());
+            iterator.set(split.getRemainder());
+          } else if (getFirstCommandStatusWireSize(report)

Review Comment:
   Good catch. I added a progress exception: when no command status has been 
selected, one complete status may exceed the configured budget if it still fits 
within the remaining safe RPC payload. It is sent as the only command status 
report and its ACK results are never split. Oversized detection now uses 
min(configured limit, safe heartbeat size).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to