avijayanhwx commented on a change in pull request #2321:
URL: https://github.com/apache/ozone/pull/2321#discussion_r649454623
##########
File path:
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/DatanodeMetadata.java
##########
@@ -56,6 +56,12 @@
@XmlElement(name = "containers")
private int containers;
+ @XmlElement(name = "openContainers")
+ private int openContainers;
+
+ @XmlElement(name = "closedContainers")
Review comment:
Since we have other transient states from OPEN to CLOSED, maybe we can
just track openContainers & totalContainers. It is the 'OPEN' container count
per node which is a significant data point for writes.
##########
File path:
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/NodeEndpoint.java
##########
@@ -117,11 +123,26 @@ public Response getDatanodes() {
}
});
try {
- int containers = nodeManager.getContainers(datanode).size();
+ Set<ContainerID> allContainers = nodeManager.getContainers(datanode);
+
+ int openContainers = 0;
+ for (ContainerID containerID: allContainers) {
+ ContainerInfo containerInfo =
reconContainerManager.getContainer(containerID);
Review comment:
When a container goes from OPEN to CLOSED it is meant to be an
irreversible change, and it stays CLOSED (or DELETED). Over the lifetime of
the cluster, we expect the number of OPEN containers (which are newly created)
to remain stable, while the CLOSED containers will keep increasing. If we
iterate over all the containers in the system for every call, then most of the
computation will remain unchanged (since CLOSED containers will remain that
way). Given the above, can we think of a way to optimize this using cached
information?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
