[jira] [Comment Edited] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions
[
https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940655#comment-16940655
]
Bibin Chundatt edited comment on YARN-9858 at 9/30/19 5:03 AM:
---
Thank you [~jhung]
+1 LGTM for latest patches. I will commit it by EOD if no objections.
was (Author: bibinchundatt):
Thank yoy [~jhung]
+1 LGTM for latest patches. I will commit it by EOD if no objections.
> Optimize RMContext getExclusiveEnforcedPartitions
> --
>
> Key: YARN-9858
> URL: https://issues.apache.org/jira/browse/YARN-9858
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jonathan Hung
>Assignee: Jonathan Hung
>Priority: Major
> Labels: release-blocker
> Attachments: YARN-9858-branch-2.001.patch,
> YARN-9858-branch-3.1.001.patch, YARN-9858-branch-3.2.001.patch,
> YARN-9858.001.patch, YARN-9858.002.patch, YARN-9858.003.patch
>
>
> Follow-up from YARN-9730. RMContextImpl#getExclusiveEnforcedPartitions is a
> hot code path, need to optimize it .
> Since AMS allocate invoked by multiple handlers locking on conf will occur
> {code}
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2841)
> - waiting to lock <0x7f1f8107c748> (a
> org.apache.hadoop.yarn.conf.YarnConfiguration)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1214)
> at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1268)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions
[
https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939181#comment-16939181
]
Jonathan Hung edited comment on YARN-9858 at 9/27/19 7:29 AM:
--
Thanks [~bibinchundatt]. This won't trigger during normal operation but without
null check , it caused unit tests to fail (see
https://issues.apache.org/jira/browse/YARN-9730?focusedCommentId=16938009=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16938009).
Not sure why jenkins is not running, I triggered a build manually.
[https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-YARN-Build/24846]
was (Author: jhung):
Thanks [~bibinchundatt]. This won't trigger during normal operation but without
null check , it caused unit tests to fail (see
https://issues.apache.org/jira/browse/YARN-9730?focusedCommentId=16938009=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16938009).
> Optimize RMContext getExclusiveEnforcedPartitions
> --
>
> Key: YARN-9858
> URL: https://issues.apache.org/jira/browse/YARN-9858
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jonathan Hung
>Assignee: Jonathan Hung
>Priority: Major
> Labels: release-blocker
> Attachments: YARN-9858.001.patch, YARN-9858.002.patch
>
>
> Follow-up from YARN-9730. RMContextImpl#getExclusiveEnforcedPartitions is a
> hot code path, need to optimize it .
> Since AMS allocate invoked by multiple handlers locking on conf will occur
> {code}
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2841)
> - waiting to lock <0x7f1f8107c748> (a
> org.apache.hadoop.yarn.conf.YarnConfiguration)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1214)
> at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1268)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions
[
https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939136#comment-16939136
]
Bibin Chundatt edited comment on YARN-9858 at 9/27/19 5:46 AM:
---
[~jhung]
Patch could cause *exclusiveEnforcedPartitions* getting set multiple times
incase of concurrent execution.
Its a possibility since its invoked by multiple handler.
Alternative could be to set the *exclusiveEnforcedPartitions* after the
creation of RMContext at
# Resourcemanager#serviceInit
# Resourcemanager#resetRMContext
All activeservices would be in NEW STATE when set it too. Thoughts?
was (Author: bibinchundatt):
[~jhung]
Patch could cause *exclusiveEnforcedPartitions* getting set multiple times
incase of concurrent execution.
Its a possibility since its invoked by multiple handler.
Alternative could be to set the *exclusiveEnforcedPartitions* after the
creation of RMContext at
# Resourcemanager#serviceInit
# Resourcemanager#resetRMContext
All the activeservices would be in stopped when we set it too.. Thoughts?
> Optimize RMContext getExclusiveEnforcedPartitions
> --
>
> Key: YARN-9858
> URL: https://issues.apache.org/jira/browse/YARN-9858
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jonathan Hung
>Assignee: Jonathan Hung
>Priority: Major
> Labels: release-blocker
> Attachments: YARN-9858.001.patch
>
>
> Follow-up from YARN-9730. RMContextImpl#getExclusiveEnforcedPartitions is a
> hot code path, need to optimize it .
> Since AMS allocate invoked by multiple handlers locking on conf will occur
> {code}
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2841)
> - waiting to lock <0x7f1f8107c748> (a
> org.apache.hadoop.yarn.conf.YarnConfiguration)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1214)
> at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1268)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
