[jira] [Commented] (YARN-11394) Fix hadoop-yarn-server-resourcemanager module Java Doc Errors.
[
https://issues.apache.org/jira/browse/YARN-11394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679400#comment-17679400
]
ASF GitHub Bot commented on YARN-11394:
---
hadoop-yetus commented on PR #5288:
URL: https://github.com/apache/hadoop/pull/5288#issuecomment-1399188342
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 30s | | trunk passed |
| +1 :green_heart: | compile | 1m 4s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 0m 56s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 55s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 1s | | trunk passed |
| -1 :x: | javadoc | 0m 54s |
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5288/6/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 27m 39s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 0m 58s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 0m 58s | | the patch passed |
| +1 :green_heart: | compile | 0m 50s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 50s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 42s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 0 new + 702 unchanged - 51 fixed = 702 total (was 753) |
| +1 :green_heart: | mvnsite | 0m 55s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 39s | |
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 generated 0 new + 1
unchanged - 100 fixed = 1 total (was 101) |
| +1 :green_heart: | javadoc | 0m 36s | |
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_352-8u352-ga-1~20.04-b08
with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 generated 0 new + 1
unchanged - 342 fixed = 1 total (was 343) |
| +1 :green_heart: | spotbugs | 2m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 27m 7s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 102m 20s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 219m 52s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5288/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5288 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 82cf9bdc4c3d 4.15.0-200-generic #211-Ubuntu SMP Thu
[jira] [Commented] (YARN-11217) [Federation] Add dumpSchedulerLogs REST APIs for Router
[
https://issues.apache.org/jira/browse/YARN-11217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679376#comment-17679376
]
ASF GitHub Bot commented on YARN-11217:
---
slfan1989 commented on code in PR #5272:
URL: https://github.com/apache/hadoop/pull/5272#discussion_r1083220366
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationInterceptorREST.java:
##
@@ -1145,10 +1146,66 @@ public SchedulerTypeInfo getSchedulerInfo() {
throw new NotImplementedException("Code is not implemented");
}
+ /**
+ * This method dumps the scheduler logs for the time got in input, and it is
+ * reachable by using {@link RMWSConsts#SCHEDULER_LOGS}.
+ *
+ * @param time the period of time. It is a FormParam.
+ * @param hsr the servlet request
+ * @return the result of the operation
+ * @throws IOException when it cannot create dump log file
+ */
@Override
public String dumpSchedulerLogs(String time, HttpServletRequest hsr)
throws IOException {
-throw new NotImplementedException("Code is not implemented");
+
+try {
+ if (StringUtils.isBlank(time)) {
+routerMetrics.incrDumpSchedulerLogsFailedRetrieved();
+throw new IllegalArgumentException("Parameter error, the time is empty
or null.");
+ }
+
Review Comment:
Thanks for your suggestion, I will modify the code.
> [Federation] Add dumpSchedulerLogs REST APIs for Router
> ---
>
> Key: YARN-11217
> URL: https://issues.apache.org/jira/browse/YARN-11217
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.4.0, 3.3.4
>Reporter: Shilun Fan
>Assignee: Shilun Fan
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11217) [Federation] Add dumpSchedulerLogs REST APIs for Router
[
https://issues.apache.org/jira/browse/YARN-11217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679356#comment-17679356
]
ASF GitHub Bot commented on YARN-11217:
---
goiri commented on code in PR #5272:
URL: https://github.com/apache/hadoop/pull/5272#discussion_r1083135742
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationInterceptorREST.java:
##
@@ -1145,10 +1146,66 @@ public SchedulerTypeInfo getSchedulerInfo() {
throw new NotImplementedException("Code is not implemented");
}
+ /**
+ * This method dumps the scheduler logs for the time got in input, and it is
+ * reachable by using {@link RMWSConsts#SCHEDULER_LOGS}.
+ *
+ * @param time the period of time. It is a FormParam.
+ * @param hsr the servlet request
+ * @return the result of the operation
+ * @throws IOException when it cannot create dump log file
+ */
@Override
public String dumpSchedulerLogs(String time, HttpServletRequest hsr)
throws IOException {
-throw new NotImplementedException("Code is not implemented");
+
+try {
+ if (StringUtils.isBlank(time)) {
+routerMetrics.incrDumpSchedulerLogsFailedRetrieved();
+throw new IllegalArgumentException("Parameter error, the time is empty
or null.");
+ }
+
Review Comment:
You can move the isBlank outside of the try.
> [Federation] Add dumpSchedulerLogs REST APIs for Router
> ---
>
> Key: YARN-11217
> URL: https://issues.apache.org/jira/browse/YARN-11217
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.4.0, 3.3.4
>Reporter: Shilun Fan
>Assignee: Shilun Fan
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9088) Non-exclusive labels break QueueMetrics
[ https://issues.apache.org/jira/browse/YARN-9088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679248#comment-17679248 ] C.J. Collier commented on YARN-9088: I'll review the changes and see if I can pick up where karthikpal left off. Here is a list of the files changed in that other patch ordered by number of changes to the file. hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/\ scheduler/QueueMetrics.java scheduler/AppSchedulingInfo.java scheduler/TestQueueMetrics.java scheduler/capacity/CSQueueMetrics.java scheduler/common/fica/FiCaSchedulerApp.java scheduler/fair/FSAppAttempt.java scheduler/capacity/LeafQueue.java scheduler/SchedulerApplicationAttempt.java scheduler/capacity/CSQueueUtils.java scheduler/capacity/TestNodeLabelContainerAllocation.java scheduler/TestSchedulerApplicationAttempt.java scheduler/capacity/TestCapacityScheduler.java monitor/invariants/TestMetricsInvariantChecker.java scheduler/fair/FairScheduler.java > Non-exclusive labels break QueueMetrics > --- > > Key: YARN-9088 > URL: https://issues.apache.org/jira/browse/YARN-9088 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler, resourcemanager >Affects Versions: 2.8.5 >Reporter: Brandon Scheller >Priority: Major > Labels: metrics, nodelabel > > QueueMetrics are broken (random/negative values) when non-exclusive labels > are being used and unlabeled containers run on labeled nodes. > This is caused by the change in the patch here: > https://issues.apache.org/jira/browse/YARN-6467 > It assumes that a container's label will be the same as the node's label that > it is running on. > If you look within the patch, sometimes metrics are updated using the > request.getNodeLabelExpression(). And sometimes they are updated using > node.getPartition(). > This means that in the case where the node is labeled while the container > request isn't, these metrics only get updated when referring to the default > queue. This stops metrics from balancing out and results in incorrect and > negative values in QueueMetrics. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11412) Create a Concurrent Users Manager to enable thread safe concurrent resource usage tracking of user
[ https://issues.apache.org/jira/browse/YARN-11412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679223#comment-17679223 ] ASF GitHub Bot commented on YARN-11412: --- krishan1390 commented on PR #5318: URL: https://github.com/apache/hadoop/pull/5318#issuecomment-1398641086 @wangdatan @sunilgovind @PrabhuJoseph can you please take a look at this PR - I am working on adding the test cases, but would be great if you can share feedback. > Create a Concurrent Users Manager to enable thread safe concurrent resource > usage tracking of user > -- > > Key: YARN-11412 > URL: https://issues.apache.org/jira/browse/YARN-11412 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Krishan Goyal >Assignee: Krishan Goyal >Priority: Major > Labels: pull-request-available > > We will build ConcurrentUsersManager whose objective is similar to > UsersManager but whose APIs provide better concurrent performance with > different class invariants > > Both ConcurrentUsersManager & UsersManager will extend a new class > AbstractCSUsersManager which implements AbstractUsersManager > > The primary behavioural change of ConcurrentUsersManager is that it will > provide user resource limits which are eventually consistent wrt containers > (allocation or release) / users (active or inactive) - These limits affect > scheduling and preemption but at massive scale small variances in individual > decisions for a container are fine as long as the scheduler progresses in an > acceptable way (i.e - users are unconcerned about these variances) > > More details of low level details - > [https://docs.google.com/document/d/1czUh2XU3_X_eRIJAsSM40hRuHYSHw4ymyfky39dIj4s/edit] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11422) fs2cs failes to generate valid capacity-scheduler.xml
[
https://issues.apache.org/jira/browse/YARN-11422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tamas Domok updated YARN-11422:
---
Description:
Attachments:
- [^fair-scheduler-issue.xml]
- [^yarn-site.xml]
Save them to /tmp/fair-scheduler.xml and /tmp/yarn-site.xml.
Run the following command:
{code}
rm -rf /tmp/output/*; yarn fs2cs --cluster-resource
memory-mb=36201037824,vcores=4914 --no-terminal-rule-check -y
/tmp/yarn-site.xml -f /tmp/fair-scheduler.xml -o /tmp/output
{code}
The conversion fails with the following error:
{code}
2023-01-20 13:37:42,250 INFO service.AbstractService: Service
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
failed in state STOPPED
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceStop(CapacityScheduler.java:428)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.service.AbstractService.close(AbstractService.java:248)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.ConvertedConfigValidator.validateConvertedConfig(ConvertedConfigValidator.java:74)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:179)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:47)
2023-01-20 13:37:42,251 ERROR converter.ConvertedConfigValidator: Could not
start Capacity Scheduler
org.apache.hadoop.yarn.exceptions.YarnException: Failed to initialize queues
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:807)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:318)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:412)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.ConvertedConfigValidator.validateConvertedConfig(ConvertedConfigValidator.java:70)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:179)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:47)
Suppressed: java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceStop(CapacityScheduler.java:428)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.service.AbstractService.close(AbstractService.java:248)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.ConvertedConfigValidator.validateConvertedConfig(ConvertedConfigValidator.java:74)
... 2 more
Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Unable to
construct queue ordering policy=fair queue=root.default
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getQueueOrderingPolicy(CapacitySchedulerConfiguration.java:2053)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.setupQueueConfigs(ParentQueue.java:172)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.(ParentQueue.java:140)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.(ParentQueue.java:115)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.parseQueue(CapacitySchedulerQueueManager.java:276)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.parseQueue(CapacitySchedulerQueueManager.java:283)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.initializeQueues(CapacitySchedulerQueueManager.java:176)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:798)
... 5 more
Caused by: java.lang.ClassNotFoundException: fair
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Clas
[jira] [Created] (YARN-11422) fs2cs failes to generate valid capacity-scheduler.xml
Tamas Domok created YARN-11422:
--
Summary: fs2cs failes to generate valid capacity-scheduler.xml
Key: YARN-11422
URL: https://issues.apache.org/jira/browse/YARN-11422
Project: Hadoop YARN
Issue Type: Bug
Components: capacityscheduler
Affects Versions: 3.4.0
Reporter: Tamas Domok
Assignee: Tamas Domok
Attachments: fair-scheduler-issue.xml, yarn-site.xml
Attachments:
- [^fair-scheduler-issue.xml]
- [^yarn-site.xml]
Save them to /tmp/fair-scheduler.xml and /tmp/yarn-site.xml.
Run the following command:
{code}
rm -rf /tmp/output/*; yarn fs2cs --cluster-resource
memory-mb=36201037824,vcores=4914 --no-terminal-rule-check -y
/tmp/yarn-site.xml -f /tmp/fair-scheduler.xml -o /tmp/output
{code}
The conversion fails with the following error:
{code}
2023-01-20 13:37:42,250 INFO service.AbstractService: Service
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
failed in state STOPPED
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceStop(CapacityScheduler.java:428)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.service.AbstractService.close(AbstractService.java:248)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.ConvertedConfigValidator.validateConvertedConfig(ConvertedConfigValidator.java:74)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:179)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:47)
2023-01-20 13:37:42,251 ERROR converter.ConvertedConfigValidator: Could not
start Capacity Scheduler
org.apache.hadoop.yarn.exceptions.YarnException: Failed to initialize queues
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:807)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:318)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:412)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.ConvertedConfigValidator.validateConvertedConfig(ConvertedConfigValidator.java:70)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:179)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:47)
Suppressed: java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceStop(CapacityScheduler.java:428)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.service.AbstractService.close(AbstractService.java:248)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.ConvertedConfigValidator.validateConvertedConfig(ConvertedConfigValidator.java:74)
... 2 more
Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Unable to
construct queue ordering policy=fair queue=root.default
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getQueueOrderingPolicy(CapacitySchedulerConfiguration.java:2053)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.setupQueueConfigs(ParentQueue.java:172)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.(ParentQueue.java:140)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.(ParentQueue.java:115)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.parseQueue(CapacitySchedulerQueueManager.java:276)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.parseQueue(CapacitySchedulerQueueManager.java:283)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.initializeQueues(CapacitySchedulerQueueManager.java:176)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:798)
... 5 more
Caused by: java.lang.ClassNotFoundException: fair
at java.net.URLClassLoader.findClass(URLClass
[jira] [Commented] (YARN-11421) Graceful Decommission ignores launched containers and gets deactivated before timeout
[
https://issues.apache.org/jira/browse/YARN-11421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679114#comment-17679114
]
Abhishek Dixit commented on YARN-11421:
---
[~prabhujoseph]
The above fix you mentioned doesn't mitigate the issue. Will create a new PR
with the fix.
> Graceful Decommission ignores launched containers and gets deactivated before
> timeout
> -
>
> Key: YARN-11421
> URL: https://issues.apache.org/jira/browse/YARN-11421
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 3.2.1, 3.3.1, 3.3.4
>Reporter: Abhishek Dixit
>Priority: Major
>
> During Graceful Decommission, a Node gets deactivated before timeout even
> though there are launched containers on that node.
> We have observed cases when graceful decommission signal is sent to node and
> Containers are launched at NodeManager and at the same time, in such cases
> ResourceManager moves the node from Decommissioning to Decommissioned state
> because launced containers are not checked in DeactivateNodeTransition.
> We will suggest using a MultiArc transition instead of
> DeactivateNodeTransition which checks for AM containers from the scheduler
> and then decides whether to keep the node in Decommissioning state or move it
> to Decommissioned State.
>
> {code:java}
> .addTransition(NodeState.DECOMMISSIONING, NodeState.DECOMMISSIONED,
> RMNodeEventType.DECOMMISSION, new
> DeactivateNodeTransition(NodeState.DECOMMISSIONED)){code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11375) [Federation] Support refreshAdminAcls、refreshServiceAcls API's for Federation
[
https://issues.apache.org/jira/browse/YARN-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679054#comment-17679054
]
ASF GitHub Bot commented on YARN-11375:
---
hadoop-yetus commented on PR #5312:
URL: https://github.com/apache/hadoop/pull/5312#issuecomment-1398057383
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 31s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 55s | | trunk passed |
| +1 :green_heart: | compile | 9m 36s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 8m 28s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 48s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 11s | | trunk passed |
| -1 :x: | javadoc | 1m 0s |
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5312/3/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 3m 27s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 7m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 43s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 59s | | the patch passed |
| +1 :green_heart: | compile | 9m 2s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | cc | 9m 2s | | the patch passed |
| +1 :green_heart: | javac | 9m 2s | | the patch passed |
| +1 :green_heart: | compile | 8m 24s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | cc | 8m 24s | | the patch passed |
| +1 :green_heart: | javac | 8m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 37s | |
hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 6 unchanged - 9
fixed = 6 total (was 15) |
| +1 :green_heart: | mvnsite | 3m 45s | | the patch passed |
| -1 :x: | javadoc | 0m 55s |
[/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5312/3/artifact/out/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 3m 10s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 7m 32s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 12s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 11s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 41s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 98m 56s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 0m 45s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 56s | | The patc
