[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17604824#comment-17604824
]
ASF GitHub Bot commented on YARN-11290:
---------------------------------------
goiri commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r971038757
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##########
@@ -255,24 +263,54 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
- long start = clock.getTime();
- List<ApplicationHomeSubCluster> result = new ArrayList<>();
+
+ if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+ }
try {
- for (String child : zkManager.getChildren(appsZNode)) {
- ApplicationId appId = ApplicationId.fromString(child);
- SubClusterId homeSubCluster = getApp(appId);
- ApplicationHomeSubCluster app =
- ApplicationHomeSubCluster.newInstance(appId, homeSubCluster);
- result.add(app);
- }
+ long start = clock.getTime();
+ SubClusterId requestSC = request.getSubClusterId();
+ List<String> children = zkManager.getChildren(appsZNode);
+ List<ApplicationHomeSubCluster> result =
+ children.stream().map(child -> generateAppHomeSC(child))
+
.sorted(Comparator.comparing(ApplicationHomeSubCluster::getCreateTime).reversed())
+ .filter(appHomeSC -> filterHomeSubCluster(requestSC,
appHomeSC.getHomeSubCluster()))
+ .limit(maxAppsInStateStore)
+ .collect(Collectors.toList());
+ long end = clock.getTime();
+ opDurations.addGetAppsHomeSubClusterDuration(start, end);
+ LOG.info("filterSubClusterId = {}, appCount = {}.", requestSC,
result.size());
+ return GetApplicationsHomeSubClusterResponse.newInstance(result);
} catch (Exception e) {
String errMsg = "Cannot get apps: " + e.getMessage();
FederationStateStoreUtils.logAndThrowStoreException(LOG, errMsg);
}
- long end = clock.getTime();
- opDurations.addGetAppsHomeSubClusterDuration(start, end);
- return GetApplicationsHomeSubClusterResponse.newInstance(result);
+
+ throw new YarnException("Cannot get app by request");
+ }
+
+ private ApplicationHomeSubCluster generateAppHomeSC(String appId) {
+ try {
+ ApplicationId applicationId = ApplicationId.fromString(appId);
+ SubClusterId homeSubCluster = getApp(applicationId);
+ ApplicationHomeSubCluster app =
+ ApplicationHomeSubCluster.newInstance(applicationId, Time.now(),
homeSubCluster);
+ return app;
+ } catch (Exception ex) {
+ LOG.error("get homeSubCluster by appId = {}.", appId);
+ }
+ return null;
+ }
+
+ private boolean filterHomeSubCluster(SubClusterId filterSubCluster,
Review Comment:
We have this function defined three times in three places.
It should also be static.
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##########
@@ -255,24 +263,54 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
- long start = clock.getTime();
- List<ApplicationHomeSubCluster> result = new ArrayList<>();
+
+ if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+ }
try {
- for (String child : zkManager.getChildren(appsZNode)) {
- ApplicationId appId = ApplicationId.fromString(child);
- SubClusterId homeSubCluster = getApp(appId);
- ApplicationHomeSubCluster app =
- ApplicationHomeSubCluster.newInstance(appId, homeSubCluster);
- result.add(app);
- }
+ long start = clock.getTime();
+ SubClusterId requestSC = request.getSubClusterId();
+ List<String> children = zkManager.getChildren(appsZNode);
+ List<ApplicationHomeSubCluster> result =
+ children.stream().map(child -> generateAppHomeSC(child))
+
.sorted(Comparator.comparing(ApplicationHomeSubCluster::getCreateTime).reversed())
Review Comment:
I don't think indentation is correct.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -----------------------------------------------------------------------------
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
> Affects Versions: 3.4.0
> Reporter: fanshilun
> Assignee: fanshilun
> Priority: Minor
> Labels: pull-request-available
> Attachments: SqlServer-StoredProc.png
>
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
