slfan1989 commented on code in PR #6016:
URL: https://github.com/apache/hadoop/pull/6016#discussion_r1329590819
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##########
@@ -234,6 +272,23 @@ public void init(Configuration conf) throws YarnException {
reservationsZNode = getNodePath(baseZNode, ROOT_ZNODE_NAME_RESERVATION);
versionNode = getNodePath(baseZNode, ROOT_ZNODE_NAME_VERSION);
+ String hierarchiesPath = getNodePath(appsZNode,
ROUTER_APP_ROOT_HIERARCHIES);
+ routerAppRootHierarchies = new HashMap<>();
+ routerAppRootHierarchies.put(0, appsZNode);
+ for (int splitIndex = 1; splitIndex <= 4; splitIndex++) {
Review Comment:
Thank you very much for your help in reviewing the code!
While implementing this part of the code, we referred to the RM's
implementation in YARN-2962. The purpose of YARN-2962 is to prevent an
excessive number of child nodes under the App's ZNode. This JIRA introduces a
hierarchical structure with 5 levels (0-4).
The basic rules are as follows:
hierarchical 0: Represents no hierarchical partitioning, following the
original structure where all application information is stored under /app.
hierarchical 1: Represents partitioning based on the last 1 character of the
application_id.
hierarchical 2: Represents partitioning based on the last 2 characters of
the application_id.
hierarchical 3: Represents partitioning based on the last 3 characters of
the application_id.
hierarchical 4: Represents partitioning based on the last 4 characters of
the application_id.
Let’s take `application_1_0001` as an example:
- Hierarchical1
```
--/app
---/application_1_000
----/1 (store data)
```
- Hierarchical2
```
--/app
---/application_1_00
----/01 (store data)
```
- Hierarchical3
```
--/app
---/application_1_0
----/001 (store data)
```
- Hierarchical4
```
--/app
---/application_1_
----/0001 (store data)
```
While implementing this functionality, I aimed to maintain consistency with
the RM's implementation, so I chose to adopt the five hierarchicals of
partitioning: 0 ~ 4.
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##########
@@ -234,6 +272,23 @@ public void init(Configuration conf) throws YarnException {
reservationsZNode = getNodePath(baseZNode, ROOT_ZNODE_NAME_RESERVATION);
versionNode = getNodePath(baseZNode, ROOT_ZNODE_NAME_VERSION);
+ String hierarchiesPath = getNodePath(appsZNode,
ROUTER_APP_ROOT_HIERARCHIES);
+ routerAppRootHierarchies = new HashMap<>();
+ routerAppRootHierarchies.put(0, appsZNode);
+ for (int splitIndex = 1; splitIndex <= 4; splitIndex++) {
Review Comment:
Thank you very much for your help in reviewing the code!
While implementing this part of the code, we referred to the RM's
implementation in YARN-2962. The purpose of YARN-2962 is to prevent an
excessive number of child nodes under the App's ZNode. This JIRA introduces a
hierarchical structure with 5 levels (0-4).
The basic rules are as follows:
hierarchical 0: Represents no hierarchical partitioning, following the
original structure where all application information is stored under /app.
hierarchical 1: Represents partitioning based on the last 1 character of the
application_id.
hierarchical 2: Represents partitioning based on the last 2 characters of
the application_id.
hierarchical 3: Represents partitioning based on the last 3 characters of
the application_id.
hierarchical 4: Represents partitioning based on the last 4 characters of
the application_id.
Let’s take `application_1_0001` as an example:
- Hierarchical1
```
--/app
---/application_1_000
----/1 (store data)
```
- Hierarchical2
```
--/app
---/application_1_00
----/01 (store data)
```
- Hierarchical3
```
--/app
---/application_1_0
----/001 (store data)
```
- Hierarchical4
```
--/app
---/application_1_
----/0001 (store data)
```
While implementing this functionality, I aimed to maintain consistency with
the RM's implementation, so I chose to adopt the five hierarchicals of
partitioning: 0 ~ 4.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
