szetszwo commented on code in PR #10124:
URL: https://github.com/apache/ozone/pull/10124#discussion_r3142269598
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DatanodeInfo.java:
##########
@@ -362,4 +388,82 @@ public int hashCode() {
public boolean equals(Object obj) {
return super.equals(obj);
}
+
+ /**
+ * Two-window tumbling bucket for a single DataNode.
+ *
+ * <p>New allocations go into {@code currentWindow}. Every {@code
rollIntervalMs}, the current
+ * window shifts to {@code previousWindow} and a fresh empty set becomes
current. After two
+ * intervals without confirmation a container ID is automatically discarded
(aged out).
+ * Pending count is the <em>union</em> of both windows.
+ */
+ static class TwoWindowBucket {
Review Comment:
Let's keep TwoWindowBucket and the other code in PendingContainerTracker
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NodeStateManager.java:
##########
@@ -134,6 +134,8 @@ public class NodeStateManager implements Runnable,
Closeable {
*/
private boolean checkPaused;
+ private final ConfigurationSource conf;
Review Comment:
Change it to containerRollIntervalMs (never store conf since it could be a
big object.)
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ScmConfigKeys.java:
##########
@@ -269,6 +269,11 @@ public final class ScmConfigKeys {
public static final String OZONE_SCM_STALENODE_INTERVAL_DEFAULT =
"5m";
+ public static final String OZONE_SCM_PENDING_CONTAINER_ROLL_INTERVAL =
+ "ozone.scm.pending.container.roll.interval";
+ public static final String OZONE_SCM_PENDING_CONTAINER_ROLL_INTERVAL_DEFAULT
=
+ "5m";
Review Comment:
Let make this a pure refactoring but not adding new conf/logic.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DatanodeInfo.java:
##########
@@ -353,6 +372,13 @@ public int getCommandCount(SCMCommandProto.Type cmd) {
}
}
+ /**
+ * Returns the {@link TwoWindowBucket} for this datanode.
+ */
+ public TwoWindowBucket getTwoWindowBucket() {
+ return twoWindowBucket;
Review Comment:
Call `twoWindowBucket.rollIfNeeded();` before return.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DatanodeInfo.java:
##########
@@ -61,10 +74,12 @@ public class DatanodeInfo extends DatanodeDetails {
* Constructs DatanodeInfo from DatanodeDetails.
*
* @param datanodeDetails Details about the datanode
+ * @param nodeStatus initial node status
* @param layoutInfo Details about the LayoutVersionProto
+ * @param conf configuration source
*/
public DatanodeInfo(DatanodeDetails datanodeDetails, NodeStatus nodeStatus,
- LayoutVersionProto layoutInfo) {
+ LayoutVersionProto layoutInfo, ConfigurationSource conf) {
Review Comment:
Pass containerRollInterval
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DatanodeInfo.java:
##########
@@ -50,6 +58,11 @@ public class DatanodeInfo extends DatanodeDetails {
private long lastStatsUpdatedTime;
private int failedVolumeCount;
+ /**
+ * Two-window tumbling bucket for tracking pending container allocations on
this datanode.
+ */
+ private final TwoWindowBucket twoWindowBucket;
Review Comment:
Let's name it `pendingContainerAllocations`.
(TwoWindowBucket is a data structure. E.g. We use HashMap all the time but
seldomly use hashMap as the name.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]