[jira] [Commented] (YARN-8482) [Router] Add cache for fast answers to getApps
[ https://issues.apache.org/jira/browse/YARN-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582509#comment-17582509 ] ASF GitHub Bot commented on YARN-8482: -- slfan1989 commented on PR #4769: URL: https://github.com/apache/hadoop/pull/4769#issuecomment-1221501038 @ashutoshcipher Thank you very much for your suggestion, but the verification of the number of parameters of the constructor, I think it can be ignored. > [Router] Add cache for fast answers to getApps > -- > > Key: YARN-8482 > URL: https://issues.apache.org/jira/browse/YARN-8482 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Giovanni Matteo Fumarola >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7615) [RESERVATION] Federation StateStore: support storage/retrieval of reservations
[ https://issues.apache.org/jira/browse/YARN-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-7615: Parent: (was: YARN-7402) Issue Type: New Feature (was: Sub-task) > [RESERVATION] Federation StateStore: support storage/retrieval of reservations > -- > > Key: YARN-7615 > URL: https://issues.apache.org/jira/browse/YARN-7615 > Project: Hadoop YARN > Issue Type: New Feature > Components: federation >Reporter: Carlo Curino >Assignee: fanshilun >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11272) [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With Zk
fanshilun created YARN-11272: Summary: [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With Zk Key: YARN-11272 URL: https://issues.apache.org/jira/browse/YARN-11272 Project: Hadoop YARN Issue Type: Sub-task Components: federation, reservation system Affects Versions: 3.4.0 Reporter: fanshilun Assignee: fanshilun -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11273) [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With SQL
fanshilun created YARN-11273: Summary: [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With SQL Key: YARN-11273 URL: https://issues.apache.org/jira/browse/YARN-11273 Project: Hadoop YARN Issue Type: Sub-task Components: federation Affects Versions: 3.4.0 Reporter: fanshilun Assignee: fanshilun -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8482) [Router] Add cache for fast answers to getApps
[
https://issues.apache.org/jira/browse/YARN-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582541#comment-17582541
]
ASF GitHub Bot commented on YARN-8482:
--
hadoop-yetus commented on PR #4769:
URL: https://github.com/apache/hadoop/pull/4769#issuecomment-1221530594
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 4s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 29s | | trunk passed |
| +1 :green_heart: | compile | 10m 33s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 9m 11s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 29s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 21s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 11s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 58s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 53s | | the patch passed |
| +1 :green_heart: | compile | 9m 59s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 9m 59s | | the patch passed |
| +1 :green_heart: | compile | 9m 12s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 9m 12s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 0s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 6s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 55s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 47s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 25s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 5s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 56s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 8s | | The patch does not
generate ASF License warnings. |
| | | 185m 2s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4769/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4769 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux e94e998dd765 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4b6a474c69daa86b1b8d2d1e6e2c2d8d16c90cc9 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4769/2/testReport/ |
[jira] [Commented] (YARN-11272) [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With Zk
[ https://issues.apache.org/jira/browse/YARN-11272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582545#comment-17582545 ] ASF GitHub Bot commented on YARN-11272: --- slfan1989 opened a new pull request, #4781: URL: https://github.com/apache/hadoop/pull/4781 JIRA: YARN-11272. [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With Zk > [RESERVATION] Federation StateStore: Support storage/retrieval of > Reservations With Zk > -- > > Key: YARN-11272 > URL: https://issues.apache.org/jira/browse/YARN-11272 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation, reservation system >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11272) [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With Zk
[ https://issues.apache.org/jira/browse/YARN-11272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-11272: -- Labels: pull-request-available (was: ) > [RESERVATION] Federation StateStore: Support storage/retrieval of > Reservations With Zk > -- > > Key: YARN-11272 > URL: https://issues.apache.org/jira/browse/YARN-11272 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation, reservation system >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11177) Support getNewReservation, submitReservation, updateReservation, deleteReservation API's for Federation
[
https://issues.apache.org/jira/browse/YARN-11177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582558#comment-17582558
]
ASF GitHub Bot commented on YARN-11177:
---
hadoop-yetus commented on PR #4764:
URL: https://github.com/apache/hadoop/pull/4764#issuecomment-1221548568
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 13s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 14s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 13s | | trunk passed |
| +1 :green_heart: | compile | 4m 12s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 32s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 30s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 18s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 51s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 4s | | the patch passed |
| +1 :green_heart: | compile | 4m 1s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 4m 1s | | the patch passed |
| +1 :green_heart: | compile | 3m 24s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 3m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 14s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 18s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 53s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 46s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 15s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 50s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 117m 10s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| -1 :x: | unit | 3m 41s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4764/6/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt)
| hadoop-yarn-server-router in the patch passed. |
| +1 :green_heart: | asflicense | 0m 43s | | The patch does not
generate ASF License warnings. |
| | | 261m 54s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.router.rmadmin.TestRouterRMAdminService |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4764/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4764 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 6c0990c5f0a2 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk
[jira] [Commented] (YARN-11153) Make proxy server support YARN federation.
[
https://issues.apache.org/jira/browse/YARN-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582560#comment-17582560
]
ASF GitHub Bot commented on YARN-11153:
---
slfan1989 commented on code in PR #4314:
URL: https://github.com/apache/hadoop/pull/4314#discussion_r950851516
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/FedAppReportFetcher.java:
##
@@ -0,0 +1,110 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.yarn.server.webproxy;
+
+import java.io.IOException;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.apache.commons.lang3.tuple.Pair;
+import org.apache.hadoop.classification.VisibleForTesting;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.ipc.RPC;
+import org.apache.hadoop.yarn.api.ApplicationClientProtocol;
+import org.apache.hadoop.yarn.api.records.ApplicationId;
+import org.apache.hadoop.yarn.client.ClientRMProxy;
+import org.apache.hadoop.yarn.exceptions.YarnException;
+import
org.apache.hadoop.yarn.server.federation.failover.FederationProxyProviderUtil;
+import org.apache.hadoop.yarn.server.federation.store.records.SubClusterId;
+import org.apache.hadoop.yarn.server.federation.store.records.SubClusterInfo;
+import
org.apache.hadoop.yarn.server.federation.utils.FederationStateStoreFacade;
+import org.apache.hadoop.yarn.util.StringHelper;
+import org.apache.hadoop.yarn.webapp.util.WebAppUtils;
+
+public class FedAppReportFetcher extends AppReportFetcher {
+
+ private final Map> subClusters;
+ private FederationStateStoreFacade federationFacade;
+
+ /**
+ * Create a new Connection to the RM/Application History Server to fetch
+ * Application reports.
+ *
+ * @param conf the conf to use to know where the RM is.
+ */
+ public FedAppReportFetcher(Configuration conf) {
+super(conf);
+subClusters = new ConcurrentHashMap<>();
+federationFacade = FederationStateStoreFacade.getInstance();
+ }
+
+ /**
+ * Get an application report for the specified application id from the RM and
+ * fall back to the Application History Server if not found in RM.
+ *
+ * @param appId id of the application to get.
+ * @return the ApplicationReport for the appId.
+ * @throws YarnException on any error.
+ * @throws IOException connection exception.
+ */
+ @Override
+ public FetchedAppReport getApplicationReport(ApplicationId appId)
+ throws YarnException, IOException {
+SubClusterId scid = federationFacade.getApplicationHomeSubCluster(appId);
+createSubclusterIfAbsent(scid);
+ApplicationClientProtocol applicationsManager =
subClusters.get(scid).getRight();
+
+return super.getApplicationReport(applicationsManager, appId);
+ }
+
+ @Override
+ public String getRmAppPageUrlBase(ApplicationId appId)
+ throws IOException, YarnException {
+SubClusterId scid = federationFacade.getApplicationHomeSubCluster(appId);
+createSubclusterIfAbsent(scid);
+
+SubClusterInfo subClusterInfo = subClusters.get(scid).getLeft();
+String scheme = WebAppUtils.getHttpSchemePrefix(getConf());
+return StringHelper.pjoin(scheme +
subClusterInfo.getRMWebServiceAddress(), "cluster", "app");
+ }
+
+ private void createSubclusterIfAbsent(SubClusterId scId) throws
YarnException, IOException {
+if (subClusters.containsKey(scId)) {
+ return;
+}
+SubClusterInfo subClusterInfo = federationFacade.getSubCluster(scId);
+Configuration subClusterConf = new Configuration(getConf());
+FederationProxyProviderUtil
+.updateConfForFederation(subClusterConf,
subClusterInfo.getSubClusterId().toString());
Review Comment:
Is toString necessary?
> Make proxy server support YARN federation.
> --
>
> Key: YARN-11153
> URL: https://issues.apache.org/jira/browse/YARN-11153
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Affects Versions: 3.2.1
>Reporter: zhengchenyu
[jira] [Commented] (YARN-11153) Make proxy server support YARN federation.
[
https://issues.apache.org/jira/browse/YARN-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582561#comment-17582561
]
ASF GitHub Bot commented on YARN-11153:
---
slfan1989 commented on code in PR #4314:
URL: https://github.com/apache/hadoop/pull/4314#discussion_r950851572
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/FedAppReportFetcher.java:
##
@@ -0,0 +1,110 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.yarn.server.webproxy;
+
+import java.io.IOException;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.apache.commons.lang3.tuple.Pair;
+import org.apache.hadoop.classification.VisibleForTesting;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.ipc.RPC;
+import org.apache.hadoop.yarn.api.ApplicationClientProtocol;
+import org.apache.hadoop.yarn.api.records.ApplicationId;
+import org.apache.hadoop.yarn.client.ClientRMProxy;
+import org.apache.hadoop.yarn.exceptions.YarnException;
+import
org.apache.hadoop.yarn.server.federation.failover.FederationProxyProviderUtil;
+import org.apache.hadoop.yarn.server.federation.store.records.SubClusterId;
+import org.apache.hadoop.yarn.server.federation.store.records.SubClusterInfo;
+import
org.apache.hadoop.yarn.server.federation.utils.FederationStateStoreFacade;
+import org.apache.hadoop.yarn.util.StringHelper;
+import org.apache.hadoop.yarn.webapp.util.WebAppUtils;
+
+public class FedAppReportFetcher extends AppReportFetcher {
+
+ private final Map> subClusters;
+ private FederationStateStoreFacade federationFacade;
+
+ /**
+ * Create a new Connection to the RM/Application History Server to fetch
+ * Application reports.
+ *
+ * @param conf the conf to use to know where the RM is.
+ */
+ public FedAppReportFetcher(Configuration conf) {
+super(conf);
+subClusters = new ConcurrentHashMap<>();
+federationFacade = FederationStateStoreFacade.getInstance();
+ }
+
+ /**
+ * Get an application report for the specified application id from the RM and
+ * fall back to the Application History Server if not found in RM.
+ *
+ * @param appId id of the application to get.
+ * @return the ApplicationReport for the appId.
+ * @throws YarnException on any error.
+ * @throws IOException connection exception.
+ */
+ @Override
+ public FetchedAppReport getApplicationReport(ApplicationId appId)
+ throws YarnException, IOException {
+SubClusterId scid = federationFacade.getApplicationHomeSubCluster(appId);
+createSubclusterIfAbsent(scid);
+ApplicationClientProtocol applicationsManager =
subClusters.get(scid).getRight();
+
+return super.getApplicationReport(applicationsManager, appId);
+ }
+
+ @Override
+ public String getRmAppPageUrlBase(ApplicationId appId)
+ throws IOException, YarnException {
+SubClusterId scid = federationFacade.getApplicationHomeSubCluster(appId);
+createSubclusterIfAbsent(scid);
+
+SubClusterInfo subClusterInfo = subClusters.get(scid).getLeft();
+String scheme = WebAppUtils.getHttpSchemePrefix(getConf());
+return StringHelper.pjoin(scheme +
subClusterInfo.getRMWebServiceAddress(), "cluster", "app");
+ }
+
+ private void createSubclusterIfAbsent(SubClusterId scId) throws
YarnException, IOException {
+if (subClusters.containsKey(scId)) {
+ return;
+}
+SubClusterInfo subClusterInfo = federationFacade.getSubCluster(scId);
+Configuration subClusterConf = new Configuration(getConf());
+FederationProxyProviderUtil
+.updateConfForFederation(subClusterConf,
subClusterInfo.getSubClusterId().toString());
Review Comment:
subClusterInfo.getSubClusterId() should be enough
> Make proxy server support YARN federation.
> --
>
> Key: YARN-11153
> URL: https://issues.apache.org/jira/browse/YARN-11153
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Affects Versions: 3.2.1
>
[jira] [Commented] (YARN-11272) [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With Zk
[
https://issues.apache.org/jira/browse/YARN-11272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582563#comment-17582563
]
ASF GitHub Bot commented on YARN-11272:
---
hadoop-yetus commented on PR #4781:
URL: https://github.com/apache/hadoop/pull/4781#issuecomment-1221556670
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 19s | | trunk passed |
| +1 :green_heart: | compile | 1m 1s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 55s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 50s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 3s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 51s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 43s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 26s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 43s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 37s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 27s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 43s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 4s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 99m 30s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4781/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4781 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c585d0dd1097 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0c779ae2a87269723f77a1c0a21e778470c4297a |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4781/1/testReport/ |
| Max. process+thread count | 729 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4781/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache
[jira] [Commented] (YARN-11177) Support getNewReservation, submitReservation, updateReservation, deleteReservation API's for Federation
[
https://issues.apache.org/jira/browse/YARN-11177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582601#comment-17582601
]
ASF GitHub Bot commented on YARN-11177:
---
hadoop-yetus commented on PR #4764:
URL: https://github.com/apache/hadoop/pull/4764#issuecomment-1221593412
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 20s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 6 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 32s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 29m 32s | | trunk passed |
| +1 :green_heart: | compile | 4m 59s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 4m 21s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 48s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 57s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 35s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 15s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 5m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 4s | | the patch passed |
| +1 :green_heart: | compile | 4m 7s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 4m 7s | | the patch passed |
| +1 :green_heart: | compile | 3m 21s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 3m 21s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 14s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 12s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 52s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 44s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 40s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 51s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 102m 32s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 3m 38s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not
generate ASF License warnings. |
| | | 251m 25s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4764/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4764 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 93ac58efb6fb 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a83a4b4e42e88eca63e0c4b8bd5696f0ddd39095 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch
[jira] [Commented] (YARN-10751) Add document for yarn log aggregation policies.
[ https://issues.apache.org/jira/browse/YARN-10751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582609#comment-17582609 ] Samrat Deb commented on YARN-10751: --- hi [~zhuqi] , Greetings! can i pick this up and created document ? > Add document for yarn log aggregation policies. > --- > > Key: YARN-10751 > URL: https://issues.apache.org/jira/browse/YARN-10751 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > > As discussed in > https://issues.apache.org/jira/browse/YARN-10743?focusedCommentId=17330998&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17330998 > The log aggregation policies are all lack of document. > This Jira will fill it. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9425) Make initialDelay configurable for FederationStateStoreService#scheduledExecutorService
[ https://issues.apache.org/jira/browse/YARN-9425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582640#comment-17582640 ] ASF GitHub Bot commented on YARN-9425: -- ayushtkn merged PR #4731: URL: https://github.com/apache/hadoop/pull/4731 > Make initialDelay configurable for > FederationStateStoreService#scheduledExecutorService > --- > > Key: YARN-9425 > URL: https://issues.apache.org/jira/browse/YARN-9425 > Project: Hadoop YARN > Issue Type: Bug > Components: federation >Affects Versions: 3.1.0 >Reporter: Shen Yinjie >Assignee: groot >Priority: Major > Labels: pull-request-available > Attachments: YARN-9425_1.patch, YARN-9425_2.patch, YARN-9425_3.patch, > YARN-9425_4.patch > > > When enable YARN federation, subclusters info in Router Web UI cannot be > loaded immediately, and client cannot find any active subclusters after 5mins > by default ,which is configured by > "yarn.federation.state-store.heartbeat-interval-secs". > IMA,we should seperate 'initialDely' and 'delay' for > FederationStateStoreService#scheduledExecutorService. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9425) Make initialDelay configurable for FederationStateStoreService#scheduledExecutorService
[ https://issues.apache.org/jira/browse/YARN-9425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582641#comment-17582641 ] Ayush Saxena commented on YARN-9425: Committed to trunk. Thanx [~groot] and [~shenyinjie] for the contribution!!! > Make initialDelay configurable for > FederationStateStoreService#scheduledExecutorService > --- > > Key: YARN-9425 > URL: https://issues.apache.org/jira/browse/YARN-9425 > Project: Hadoop YARN > Issue Type: Bug > Components: federation >Affects Versions: 3.1.0 >Reporter: Shen Yinjie >Assignee: groot >Priority: Major > Labels: pull-request-available > Attachments: YARN-9425_1.patch, YARN-9425_2.patch, YARN-9425_3.patch, > YARN-9425_4.patch > > > When enable YARN federation, subclusters info in Router Web UI cannot be > loaded immediately, and client cannot find any active subclusters after 5mins > by default ,which is configured by > "yarn.federation.state-store.heartbeat-interval-secs". > IMA,we should seperate 'initialDely' and 'delay' for > FederationStateStoreService#scheduledExecutorService. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11240) Fix incorrect placeholder in yarn-module
[ https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582669#comment-17582669 ] ASF GitHub Bot commented on YARN-11240: --- slfan1989 commented on PR #4678: URL: https://github.com/apache/hadoop/pull/4678#issuecomment-1221640742 @ayushtkn Can you help review this pr? Thank you very much! > Fix incorrect placeholder in yarn-module > > > Key: YARN-11240 > URL: https://issues.apache.org/jira/browse/YARN-11240 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Try to deal with the moudle problem at a time. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10846) Add dispatcher metrics to NM
[ https://issues.apache.org/jira/browse/YARN-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582670#comment-17582670 ] ASF GitHub Bot commented on YARN-10846: --- slfan1989 commented on PR #4687: URL: https://github.com/apache/hadoop/pull/4687#issuecomment-1221647149 @jojochuang Can you help review this part of the code? Thank you very much! > Add dispatcher metrics to NM > > > Key: YARN-10846 > URL: https://issues.apache.org/jira/browse/YARN-10846 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: chaosju >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Attachments: YARN-10846.001.patch, screenshot-1.png > > Time Spent: 1h 20m > Remaining Estimate: 0h > > base [YARN-9615|https://issues.apache.org/jira/browse/YARN-9615], add > dispatcher metrics to NM -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11240) Fix incorrect placeholder in yarn-module
[
https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582671#comment-17582671
]
ASF GitHub Bot commented on YARN-11240:
---
ayushtkn commented on code in PR #4678:
URL: https://github.com/apache/hadoop/pull/4678#discussion_r950919833
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/ServiceScheduler.java:
##
@@ -1139,7 +1139,7 @@ public void syncSysFs(Service yarnApp) {
LOG.info("YARN sysfs synchronized.");
}
} catch (IOException | URISyntaxException | InterruptedException e) {
- LOG.error("Fail to sync service spec: {}", e);
+ LOG.error("Fail to sync service spec: {}", e.getMessage(), e);
Review Comment:
No need to keep the message, just e is enough
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/test/java/org/apache/hadoop/yarn/service/TestServiceAM.java:
##
@@ -553,7 +553,7 @@ public void testSyncSysFS() {
am.stop();
am.close();
} catch (Exception e) {
- LOG.error("Fail to sync sysfs: {}", e);
+ LOG.error("Fail to sync sysfs: {}", e.getMessage(), e);
Review Comment:
Keep only e, message will come as part of that
> Fix incorrect placeholder in yarn-module
>
>
> Key: YARN-11240
> URL: https://issues.apache.org/jira/browse/YARN-11240
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Try to deal with the moudle problem at a time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11247) Remove unused classes introduced by YARN-9615
[
https://issues.apache.org/jira/browse/YARN-11247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582675#comment-17582675
]
ASF GitHub Bot commented on YARN-11247:
---
hadoop-yetus commented on PR #4720:
URL: https://github.com/apache/hadoop/pull/4720#issuecomment-1221668942
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 42s | | trunk passed |
| +1 :green_heart: | compile | 1m 5s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 1s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 6s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 46s | | the patch passed |
| +1 :green_heart: | compile | 0m 49s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 49s | | the patch passed |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 43s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 34s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 46s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 46s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 46s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 45s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 4m 58s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 103m 38s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4720/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4720 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 1e92d506fab3 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 78cd349f4ee65d1b883e72e71a5da0602744908a |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4720/3/testReport/ |
| Max. process+thread count | 559 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4720/3/console |
| versions |
[jira] [Created] (YARN-11274) Impove Nodemanager#NodeStatusUpdaterImpl Log
fanshilun created YARN-11274: Summary: Impove Nodemanager#NodeStatusUpdaterImpl Log Key: YARN-11274 URL: https://issues.apache.org/jira/browse/YARN-11274 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 3.4.0 Reporter: fanshilun Assignee: fanshilun 1.When RM performs master-slave switching, NM will re-register, report the list of running apps, and print the number of running apps 2.registerWithRM#containerReports is enough to print the number of prints, because the above calling process has already printed the container list, so there is no need to repeat printing 3.Use placeholders to display logs to avoid string concatenation -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11240) Fix incorrect placeholder in yarn-module
[
https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582685#comment-17582685
]
ASF GitHub Bot commented on YARN-11240:
---
slfan1989 commented on code in PR #4678:
URL: https://github.com/apache/hadoop/pull/4678#discussion_r950949402
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/ServiceScheduler.java:
##
@@ -1139,7 +1139,7 @@ public void syncSysFs(Service yarnApp) {
LOG.info("YARN sysfs synchronized.");
}
} catch (IOException | URISyntaxException | InterruptedException e) {
- LOG.error("Fail to sync service spec: {}", e);
+ LOG.error("Fail to sync service spec: {}", e.getMessage(), e);
Review Comment:
Thanks a lot for your help reviewing the code, but there is a problem with
the use of placeholders for logging.
When an exception occurs, the placeholder is not replaced, We can view the
following logs.
```
2022-08-22 09:59:13,755 ERROR [Listener at 0.0.0.0/51213]
service.TestServiceAM (TestServiceAM.java:testSyncSysFS(556)) - Fail to sync
sysfs: {}
java.lang.RuntimeException: tensorflow syncSysFs Error
at
org.apache.hadoop.yarn.service.ServiceScheduler.syncSysFs(ServiceScheduler.java:1137)
at
org.apache.hadoop.yarn.service.TestServiceAM.testSyncSysFS(TestServiceAM.java:551)
...
```
Because the log uses the void error(String var1, Throwable var2); method, it
cannot be replaced.
So I added e.getMessage() to make the placeholder take effect and the log
format would be better.
```
2022-08-22 10:03:55,450 ERROR [Listener at 0.0.0.0/51354]
service.TestServiceAM (TestServiceAM.java:testSyncSysFS(556)) - Fail to sync
sysfs: tensorflow syncSysFs Error
java.lang.RuntimeException: tensorflow syncSysFs Error
at
org.apache.hadoop.yarn.service.ServiceScheduler.syncSysFs(ServiceScheduler.java:1137)
at
org.apache.hadoop.yarn.service.TestServiceAM.testSyncSysFS(TestServiceAM.java:551)
...
```
> Fix incorrect placeholder in yarn-module
>
>
> Key: YARN-11240
> URL: https://issues.apache.org/jira/browse/YARN-11240
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Try to deal with the moudle problem at a time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11240) Fix incorrect placeholder in yarn-module
[
https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582686#comment-17582686
]
ASF GitHub Bot commented on YARN-11240:
---
slfan1989 commented on code in PR #4678:
URL: https://github.com/apache/hadoop/pull/4678#discussion_r950949402
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/ServiceScheduler.java:
##
@@ -1139,7 +1139,7 @@ public void syncSysFs(Service yarnApp) {
LOG.info("YARN sysfs synchronized.");
}
} catch (IOException | URISyntaxException | InterruptedException e) {
- LOG.error("Fail to sync service spec: {}", e);
+ LOG.error("Fail to sync service spec: {}", e.getMessage(), e);
Review Comment:
Thanks a lot for your help reviewing the code, but there is a problem with
the use of placeholders for logging.
When an exception occurs, the placeholder is not replaced, We can view the
following logs.
```
2022-08-22 09:59:13,755 ERROR [Listener at 0.0.0.0/51213]
service.TestServiceAM (TestServiceAM.java:testSyncSysFS(556)) -
Fail to sync sysfs: {}
java.lang.RuntimeException: tensorflow syncSysFs Error
at
org.apache.hadoop.yarn.service.ServiceScheduler.syncSysFs(ServiceScheduler.java:1137)
at
org.apache.hadoop.yarn.service.TestServiceAM.testSyncSysFS(TestServiceAM.java:551)
...
```
Because the log uses the void error(String var1, Throwable var2); method, it
cannot be replaced.
So I added e.getMessage() to make the placeholder take effect and the log
format would be better.
```
2022-08-22 10:03:55,450 ERROR [Listener at 0.0.0.0/51354]
service.TestServiceAM (TestServiceAM.java:testSyncSysFS(556)) -
Fail to sync sysfs: tensorflow syncSysFs Error
java.lang.RuntimeException: tensorflow syncSysFs Error
at
org.apache.hadoop.yarn.service.ServiceScheduler.syncSysFs(ServiceScheduler.java:1137)
at
org.apache.hadoop.yarn.service.TestServiceAM.testSyncSysFS(TestServiceAM.java:551)
...
```
> Fix incorrect placeholder in yarn-module
>
>
> Key: YARN-11240
> URL: https://issues.apache.org/jira/browse/YARN-11240
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Try to deal with the moudle problem at a time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11240) Fix incorrect placeholder in yarn-module
[
https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582688#comment-17582688
]
ASF GitHub Bot commented on YARN-11240:
---
slfan1989 commented on code in PR #4678:
URL: https://github.com/apache/hadoop/pull/4678#discussion_r950949839
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/ServiceScheduler.java:
##
@@ -1139,7 +1139,7 @@ public void syncSysFs(Service yarnApp) {
LOG.info("YARN sysfs synchronized.");
}
} catch (IOException | URISyntaxException | InterruptedException e) {
- LOG.error("Fail to sync service spec: {}", e);
+ LOG.error("Fail to sync service spec: {}", e.getMessage(), e);
Review Comment:
Can I remove the placeholder?
> Fix incorrect placeholder in yarn-module
>
>
> Key: YARN-11240
> URL: https://issues.apache.org/jira/browse/YARN-11240
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Try to deal with the moudle problem at a time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11240) Fix incorrect placeholder in yarn-module
[
https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582690#comment-17582690
]
ASF GitHub Bot commented on YARN-11240:
---
slfan1989 commented on code in PR #4678:
URL: https://github.com/apache/hadoop/pull/4678#discussion_r950949402
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/ServiceScheduler.java:
##
@@ -1139,7 +1139,7 @@ public void syncSysFs(Service yarnApp) {
LOG.info("YARN sysfs synchronized.");
}
} catch (IOException | URISyntaxException | InterruptedException e) {
- LOG.error("Fail to sync service spec: {}", e);
+ LOG.error("Fail to sync service spec: {}", e.getMessage(), e);
Review Comment:
Thank you very much for helping to review the code!
There is a problem with the use of placeholders in this part of the code.
When an exception occurs, the placeholder is not replaced, We can view the
following logs.
```
2022-08-22 09:59:13,755 ERROR [Listener at 0.0.0.0/51213]
service.TestServiceAM (TestServiceAM.java:testSyncSysFS(556)) -
Fail to sync sysfs: {}
java.lang.RuntimeException: tensorflow syncSysFs Error
at
org.apache.hadoop.yarn.service.ServiceScheduler.syncSysFs(ServiceScheduler.java:1137)
at
org.apache.hadoop.yarn.service.TestServiceAM.testSyncSysFS(TestServiceAM.java:551)
...
```
Because the log uses the void error(String var1, Throwable var2); method, it
cannot be replaced.
So I added e.getMessage() to make the placeholder take effect and the log
format would be better.
```
2022-08-22 10:03:55,450 ERROR [Listener at 0.0.0.0/51354]
service.TestServiceAM (TestServiceAM.java:testSyncSysFS(556)) -
Fail to sync sysfs: tensorflow syncSysFs Error
java.lang.RuntimeException: tensorflow syncSysFs Error
at
org.apache.hadoop.yarn.service.ServiceScheduler.syncSysFs(ServiceScheduler.java:1137)
at
org.apache.hadoop.yarn.service.TestServiceAM.testSyncSysFS(TestServiceAM.java:551)
...
```
> Fix incorrect placeholder in yarn-module
>
>
> Key: YARN-11240
> URL: https://issues.apache.org/jira/browse/YARN-11240
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Try to deal with the moudle problem at a time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11240) Fix incorrect placeholder in yarn-module
[
https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582695#comment-17582695
]
ASF GitHub Bot commented on YARN-11240:
---
ayushtkn commented on code in PR #4678:
URL: https://github.com/apache/hadoop/pull/4678#discussion_r950956126
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/ServiceScheduler.java:
##
@@ -1139,7 +1139,7 @@ public void syncSysFs(Service yarnApp) {
LOG.info("YARN sysfs synchronized.");
}
} catch (IOException | URISyntaxException | InterruptedException e) {
- LOG.error("Fail to sync service spec: {}", e);
+ LOG.error("Fail to sync service spec: {}", e.getMessage(), e);
Review Comment:
Yes, tensorflow syncSysFs Error is getting printed twice, we don’t need that
> Fix incorrect placeholder in yarn-module
>
>
> Key: YARN-11240
> URL: https://issues.apache.org/jira/browse/YARN-11240
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Try to deal with the moudle problem at a time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11240) Fix incorrect placeholder in yarn-module
[
https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582697#comment-17582697
]
ASF GitHub Bot commented on YARN-11240:
---
slfan1989 commented on code in PR #4678:
URL: https://github.com/apache/hadoop/pull/4678#discussion_r950956935
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/ServiceScheduler.java:
##
@@ -1139,7 +1139,7 @@ public void syncSysFs(Service yarnApp) {
LOG.info("YARN sysfs synchronized.");
}
} catch (IOException | URISyntaxException | InterruptedException e) {
- LOG.error("Fail to sync service spec: {}", e);
+ LOG.error("Fail to sync service spec: {}", e.getMessage(), e);
Review Comment:
Thanks for your suggestion, I will fix it.
> Fix incorrect placeholder in yarn-module
>
>
> Key: YARN-11240
> URL: https://issues.apache.org/jira/browse/YARN-11240
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Try to deal with the moudle problem at a time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11247) Remove unused classes introduced by YARN-9615
[ https://issues.apache.org/jira/browse/YARN-11247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582698#comment-17582698 ] ASF GitHub Bot commented on YARN-11247: --- slfan1989 commented on PR #4720: URL: https://github.com/apache/hadoop/pull/4720#issuecomment-1221723661 @ayushtkn Can you help review this pr? I want to delete DisableEventTypeMetrics.java because this class is not used. Thank you very much! > Remove unused classes introduced by YARN-9615 > - > > Key: YARN-11247 > URL: https://issues.apache.org/jira/browse/YARN-11247 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: DisableEventTypeMetrics-Not used.png > > > YARN-9615 adds Metric to RM's dispatcher, but the patch introduces a class > without any usage > org.apache.hadoop.yarn.metrics#DisableEventTypeMetrics > 1. Without any code references > 2. Without any test code references > 3. Delete this class, the local can be compiled successfully > I think this class can be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[ https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-11274: - Summary: Improve Nodemanager#NodeStatusUpdaterImpl Log (was: Impove Nodemanager#NodeStatusUpdaterImpl Log) > Improve Nodemanager#NodeStatusUpdaterImpl Log > - > > Key: YARN-11274 > URL: https://issues.apache.org/jira/browse/YARN-11274 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > > 1.When RM performs master-slave switching, NM will re-register, report the > list of running apps, and print the number of running apps > 2.registerWithRM#containerReports is enough to print the number of prints, > because the above calling process has already printed the container list, so > there is no need to repeat printing > 3.Use placeholders to display logs to avoid string concatenation -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[ https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-11274: -- Labels: pull-request-available (was: ) > Improve Nodemanager#NodeStatusUpdaterImpl Log > - > > Key: YARN-11274 > URL: https://issues.apache.org/jira/browse/YARN-11274 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > > 1.When RM performs master-slave switching, NM will re-register, report the > list of running apps, and print the number of running apps > 2.registerWithRM#containerReports is enough to print the number of prints, > because the above calling process has already printed the container list, so > there is no need to repeat printing > 3.Use placeholders to display logs to avoid string concatenation -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[ https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582711#comment-17582711 ] ASF GitHub Bot commented on YARN-11274: --- slfan1989 opened a new pull request, #4783: URL: https://github.com/apache/hadoop/pull/4783 JIRA:YARN-11274. Impove Nodemanager#NodeStatusUpdaterImpl Log. The improvements are as follows: 1.When RM failover, NM will re-register, report the list of running apps, But there is no log, add this part of the log, print and report the number of apps. 2.registerWithRM#containerReports is enough to print the number of prints, because the above calling process has already printed the container list, so there is no need to repeat printing. 3.Use placeholders to display logs to avoid string concatenation. > Improve Nodemanager#NodeStatusUpdaterImpl Log > - > > Key: YARN-11274 > URL: https://issues.apache.org/jira/browse/YARN-11274 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > > 1.When RM performs master-slave switching, NM will re-register, report the > list of running apps, and print the number of running apps > 2.registerWithRM#containerReports is enough to print the number of prints, > because the above calling process has already printed the container list, so > there is no need to repeat printing > 3.Use placeholders to display logs to avoid string concatenation -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[
https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582713#comment-17582713
]
ASF GitHub Bot commented on YARN-11274:
---
slfan1989 commented on code in PR #4783:
URL: https://github.com/apache/hadoop/pull/4783#discussion_r950984562
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java:
##
@@ -641,6 +639,7 @@ private List getRunningApplications() {
runningApplications.add(appEntry.getKey());
}
}
+LOG.info("Running Applications Size : {}.", runningApplications.size());
Review Comment:
add log, print the number of running apps.
> Improve Nodemanager#NodeStatusUpdaterImpl Log
> -
>
> Key: YARN-11274
> URL: https://issues.apache.org/jira/browse/YARN-11274
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1.When RM performs master-slave switching, NM will re-register, report the
> list of running apps, and print the number of running apps
> 2.registerWithRM#containerReports is enough to print the number of prints,
> because the above calling process has already printed the container list, so
> there is no need to repeat printing
> 3.Use placeholders to display logs to avoid string concatenation
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[
https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582717#comment-17582717
]
ASF GitHub Bot commented on YARN-11274:
---
slfan1989 commented on code in PR #4783:
URL: https://github.com/apache/hadoop/pull/4783#discussion_r950986447
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java:
##
@@ -399,7 +397,7 @@ nodeManagerVersionId, containerReports,
getRunningApplications(),
nodeLabels, physicalResource, nodeAttributes, nodeStatus);
if (containerReports != null && !containerReports.isEmpty()) {
-LOG.info("Registering with RM using containers :" + containerReports);
Review Comment:
This is enough to print directly.
Line#394: getNMContainerStatuses()
The list of Containers has been printed, no need to reprint.
Line#402: I modified it to print size directly.
> Improve Nodemanager#NodeStatusUpdaterImpl Log
> -
>
> Key: YARN-11274
> URL: https://issues.apache.org/jira/browse/YARN-11274
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1.When RM performs master-slave switching, NM will re-register, report the
> list of running apps, and print the number of running apps
> 2.registerWithRM#containerReports is enough to print the number of prints,
> because the above calling process has already printed the container list, so
> there is no need to repeat printing
> 3.Use placeholders to display logs to avoid string concatenation
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11153) Make proxy server support YARN federation.
[
https://issues.apache.org/jira/browse/YARN-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582718#comment-17582718
]
ASF GitHub Bot commented on YARN-11153:
---
zhengchenyu commented on code in PR #4314:
URL: https://github.com/apache/hadoop/pull/4314#discussion_r950986663
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/FedAppReportFetcher.java:
##
@@ -0,0 +1,110 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.yarn.server.webproxy;
+
+import java.io.IOException;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.apache.commons.lang3.tuple.Pair;
+import org.apache.hadoop.classification.VisibleForTesting;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.ipc.RPC;
+import org.apache.hadoop.yarn.api.ApplicationClientProtocol;
+import org.apache.hadoop.yarn.api.records.ApplicationId;
+import org.apache.hadoop.yarn.client.ClientRMProxy;
+import org.apache.hadoop.yarn.exceptions.YarnException;
+import
org.apache.hadoop.yarn.server.federation.failover.FederationProxyProviderUtil;
+import org.apache.hadoop.yarn.server.federation.store.records.SubClusterId;
+import org.apache.hadoop.yarn.server.federation.store.records.SubClusterInfo;
+import
org.apache.hadoop.yarn.server.federation.utils.FederationStateStoreFacade;
+import org.apache.hadoop.yarn.util.StringHelper;
+import org.apache.hadoop.yarn.webapp.util.WebAppUtils;
+
+public class FedAppReportFetcher extends AppReportFetcher {
+
+ private final Map> subClusters;
+ private FederationStateStoreFacade federationFacade;
+
+ /**
+ * Create a new Connection to the RM/Application History Server to fetch
+ * Application reports.
+ *
+ * @param conf the conf to use to know where the RM is.
+ */
+ public FedAppReportFetcher(Configuration conf) {
+super(conf);
+subClusters = new ConcurrentHashMap<>();
+federationFacade = FederationStateStoreFacade.getInstance();
+ }
+
+ /**
+ * Get an application report for the specified application id from the RM and
+ * fall back to the Application History Server if not found in RM.
+ *
+ * @param appId id of the application to get.
+ * @return the ApplicationReport for the appId.
+ * @throws YarnException on any error.
+ * @throws IOException connection exception.
+ */
+ @Override
+ public FetchedAppReport getApplicationReport(ApplicationId appId)
+ throws YarnException, IOException {
+SubClusterId scid = federationFacade.getApplicationHomeSubCluster(appId);
+createSubclusterIfAbsent(scid);
+ApplicationClientProtocol applicationsManager =
subClusters.get(scid).getRight();
+
+return super.getApplicationReport(applicationsManager, appId);
+ }
+
+ @Override
+ public String getRmAppPageUrlBase(ApplicationId appId)
+ throws IOException, YarnException {
+SubClusterId scid = federationFacade.getApplicationHomeSubCluster(appId);
+createSubclusterIfAbsent(scid);
+
+SubClusterInfo subClusterInfo = subClusters.get(scid).getLeft();
+String scheme = WebAppUtils.getHttpSchemePrefix(getConf());
+return StringHelper.pjoin(scheme +
subClusterInfo.getRMWebServiceAddress(), "cluster", "app");
+ }
+
+ private void createSubclusterIfAbsent(SubClusterId scId) throws
YarnException, IOException {
+if (subClusters.containsKey(scId)) {
+ return;
+}
+SubClusterInfo subClusterInfo = federationFacade.getSubCluster(scId);
+Configuration subClusterConf = new Configuration(getConf());
+FederationProxyProviderUtil
+.updateConfForFederation(subClusterConf,
subClusterInfo.getSubClusterId().toString());
Review Comment:
Of course not. but explicit call is not wrong.
> Make proxy server support YARN federation.
> --
>
> Key: YARN-11153
> URL: https://issues.apache.org/jira/browse/YARN-11153
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Affects Versions: 3.2.1
>
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[
https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582719#comment-17582719
]
ASF GitHub Bot commented on YARN-11274:
---
slfan1989 commented on code in PR #4783:
URL: https://github.com/apache/hadoop/pull/4783#discussion_r950987242
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java:
##
@@ -399,7 +397,7 @@ nodeManagerVersionId, containerReports,
getRunningApplications(),
nodeLabels, physicalResource, nodeAttributes, nodeStatus);
if (containerReports != null && !containerReports.isEmpty()) {
-LOG.info("Registering with RM using containers :" + containerReports);
+LOG.info("Registering with RM using containers.size : {}." +
containerReports.size());
Review Comment:
This is enough to print directly.
Line#394: getNMContainerStatuses()
The list of Containers has been printed, no need to reprint.
Line#402: I modified it to print size directly.
> Improve Nodemanager#NodeStatusUpdaterImpl Log
> -
>
> Key: YARN-11274
> URL: https://issues.apache.org/jira/browse/YARN-11274
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1.When RM performs master-slave switching, NM will re-register, report the
> list of running apps, and print the number of running apps
> 2.registerWithRM#containerReports is enough to print the number of prints,
> because the above calling process has already printed the container list, so
> there is no need to repeat printing
> 3.Use placeholders to display logs to avoid string concatenation
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[
https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582720#comment-17582720
]
ASF GitHub Bot commented on YARN-11274:
---
slfan1989 commented on code in PR #4783:
URL: https://github.com/apache/hadoop/pull/4783#discussion_r950987242
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java:
##
@@ -399,7 +397,7 @@ nodeManagerVersionId, containerReports,
getRunningApplications(),
nodeLabels, physicalResource, nodeAttributes, nodeStatus);
if (containerReports != null && !containerReports.isEmpty()) {
-LOG.info("Registering with RM using containers :" + containerReports);
+LOG.info("Registering with RM using containers.size : {}." +
containerReports.size());
Review Comment:
It is enough to print the number directly.
This function will call getNMContainerStatuses()
Line#394: getNMContainerStatuses()
The list of Containers has been printed, no need to reprint.
Line#402: I modified it to print size directly.
> Improve Nodemanager#NodeStatusUpdaterImpl Log
> -
>
> Key: YARN-11274
> URL: https://issues.apache.org/jira/browse/YARN-11274
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1.When RM performs master-slave switching, NM will re-register, report the
> list of running apps, and print the number of running apps
> 2.registerWithRM#containerReports is enough to print the number of prints,
> because the above calling process has already printed the container list, so
> there is no need to repeat printing
> 3.Use placeholders to display logs to avoid string concatenation
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11196) NUMA Awareness support in DefaultContainerExecutor
[ https://issues.apache.org/jira/browse/YARN-11196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582728#comment-17582728 ] ASF GitHub Bot commented on YARN-11196: --- Samrat002 commented on PR #4742: URL: https://github.com/apache/hadoop/pull/4742#issuecomment-1221790812 check style is failing due to accesive params passed in the func args ! created a follow up jira for fix :- [YARN-11256](https://issues.apache.org/jira/browse/YARN-11256) > NUMA Awareness support in DefaultContainerExecutor > -- > > Key: YARN-11196 > URL: https://issues.apache.org/jira/browse/YARN-11196 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.3.3 >Reporter: Prabhu Joseph >Assignee: Samrat Deb >Priority: Major > Labels: pull-request-available > > [YARN-5764|https://issues.apache.org/jira/browse/YARN-5764] has added support > of NUMA Awareness for Containers launched through LinuxContainerExecutor. > This feature is useful to have in DefaultContainerExecutor as well. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11240) Fix incorrect placeholder in yarn-module
[
https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582729#comment-17582729
]
ASF GitHub Bot commented on YARN-11240:
---
hadoop-yetus commented on PR #4678:
URL: https://github.com/apache/hadoop/pull/4678#issuecomment-1221806964
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 10s | | trunk passed |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 40s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 48s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 3s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 22s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 21s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 20m 59s | | hadoop-yarn-services-core in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not
generate ASF License warnings. |
| | | 122m 12s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4678/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4678 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux fba86fa7218f 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 5820a0ad1dd5709ec243ffa471da6177d76d44eb |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4678/5/testReport/ |
| Max. process+thread count | 814 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core
|
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4678/5/console |
| versions | git=2.25.
[jira] [Commented] (YARN-11196) NUMA Awareness support in DefaultContainerExecutor
[
https://issues.apache.org/jira/browse/YARN-11196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582730#comment-17582730
]
ASF GitHub Bot commented on YARN-11196:
---
PrabhuJoseph commented on code in PR #4742:
URL: https://github.com/apache/hadoop/pull/4742#discussion_r951007590
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java:
##
@@ -350,6 +384,14 @@ public int launchContainer(ContainerStartContext ctx)
return exitCode;
} finally {
if (shExec != null) shExec.close();
+ try {
Review Comment:
Shall we use the same postComplete method here as well.
> NUMA Awareness support in DefaultContainerExecutor
> --
>
> Key: YARN-11196
> URL: https://issues.apache.org/jira/browse/YARN-11196
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 3.3.3
>Reporter: Prabhu Joseph
>Assignee: Samrat Deb
>Priority: Major
> Labels: pull-request-available
>
> [YARN-5764|https://issues.apache.org/jira/browse/YARN-5764] has added support
> of NUMA Awareness for Containers launched through LinuxContainerExecutor.
> This feature is useful to have in DefaultContainerExecutor as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10846) Add dispatcher metrics to NM
[
https://issues.apache.org/jira/browse/YARN-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582736#comment-17582736
]
ASF GitHub Bot commented on YARN-10846:
---
hadoop-yetus commented on PR #4687:
URL: https://github.com/apache/hadoop/pull/4687#issuecomment-1221828012
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 0s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 24s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 32s | | trunk passed |
| +1 :green_heart: | compile | 9m 54s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 8m 57s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 49s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 32s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 4m 18s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 43s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 33s | | the patch passed |
| +1 :green_heart: | compile | 9m 11s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 9m 11s | | the patch passed |
| +1 :green_heart: | compile | 8m 44s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 44s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 2m 2s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4687/7/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 1 new + 53 unchanged -
1 fixed = 54 total (was 54) |
| +1 :green_heart: | mvnsite | 4m 14s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 52s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 47s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 45s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 5m 32s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 102m 42s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 24m 55s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 1m 25s | | The patch does not
generate ASF License warnings. |
| | | 309m 36s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4687/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4687 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c9cf9710fa50 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 9b09131e7d28a5cdc26a744c6289798c18da1985 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[
https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582744#comment-17582744
]
ASF GitHub Bot commented on YARN-11274:
---
hadoop-yetus commented on PR #4783:
URL: https://github.com/apache/hadoop/pull/4783#issuecomment-1221854424
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 48s | | trunk passed |
| +1 :green_heart: | compile | 1m 48s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 58s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 7s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 55s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 22s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 45s | | the patch passed |
| +1 :green_heart: | compile | 1m 30s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 30s | | the patch passed |
| +1 :green_heart: | compile | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 34s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4783/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 2 new + 78 unchanged - 0 fixed = 80 total (was 78) |
| +1 :green_heart: | mvnsite | 0m 46s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 39s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 37s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 32s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 56s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 24m 6s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 124m 48s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4783/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4783 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d66fce48dcae 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e7ecf871eba4a5e6e3475c1141dbc96c790e1602 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4783/1
[jira] [Commented] (YARN-11187) Remove WhiteBox in yarn module.
[
https://issues.apache.org/jira/browse/YARN-11187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582772#comment-17582772
]
ASF GitHub Bot commented on YARN-11187:
---
hadoop-yetus commented on PR #4463:
URL: https://github.com/apache/hadoop/pull/4463#issuecomment-1221925728
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 16s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 1s | | trunk passed |
| +1 :green_heart: | compile | 1m 46s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 49s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 58s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 49s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 19s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 43s | | the patch passed |
| +1 :green_heart: | compile | 1m 32s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 32s | |
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new +
105 unchanged - 9 fixed = 105 total (was 114) |
| +1 :green_heart: | compile | 1m 24s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 24s | |
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new
+ 78 unchanged - 9 fixed = 78 total (was 87) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 30s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4463/14/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 45s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 38s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 34s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 9s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 23m 28s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not
generate ASF License warnings. |
| | | 131m 49s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4463/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4463 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux b42c7d3b8b5f 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | mave
