[jira] [Commented] (YARN-1028) Add FailoverProxyProvider like capability to RMProxy
[
https://issues.apache.org/jira/browse/YARN-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848581#comment-13848581
]
Bikas Saha commented on YARN-1028:
--
Why do we need this singleton like abstraction now?
{code}
+ private static ClientRMProxy INSTANCE = new ClientRMProxy();
{code}
I dont think we should make YarnClientImpl HA aware. The main point of RMProxy
was to hide this from upper layers like the client code. For the purposes of
the log, we could add an API to the RMProxy interface that gets the rm address
relevant to the protocol being proxied.
{code}
- private static InetSocketAddress getRmAddress(Configuration conf) {
{code}
This code seems to be duplicating stuff RMProxy/ClientRMProxy. Can we call the
existing methods in RMProxy instead?
{code}
+ public T getProxy() {
.
+return (T) YarnRPC.create(conf).getProxy(protocol, rmAddress,
conf);
{code}
Is this logic not valid in an HA scenario?
{code}
+ MapClass? extends Exception, RetryPolicy exceptionToPolicyMap =
+ new HashMapClass? extends Exception, RetryPolicy();
+ exceptionToPolicyMap.put(ConnectException.class, retryPolicy);
+ //TO DO: after HADOOP-9576, IOException can be changed to EOFException
+ exceptionToPolicyMap.put(IOException.class, retryPolicy);
+ return RetryPolicies.retryByException(
+ RetryPolicies.TRY_ONCE_THEN_FAIL, exceptionToPolicyMap);
{code}
Nice test! It would help that after failing over we can verify that the new
active is really the one we expect to be active.
Will mini yarn HA cluster work without fixed ports? Can the test cover that
case also if simple to extend.
Dont see the initial value of this being set to -1 anywhere.
{code}
+assertFalse(RM never turned active, -1 == cluster.getActiveRMIndex());
{code}
New class does not seem to add anything?
{code}
+ private class ShortCircuitedNodeManager extends CustomNodeManager {
{code}
Can we please leave a short note on how the mini yarn cluster behaves with
multiple RM's?
Add FailoverProxyProvider like capability to RMProxy
Key: YARN-1028
URL: https://issues.apache.org/jira/browse/YARN-1028
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Bikas Saha
Assignee: Karthik Kambatla
Attachments: yarn-1028-1.patch, yarn-1028-2.patch, yarn-1028-3.patch,
yarn-1028-4.patch, yarn-1028-5.patch, yarn-1028-6.patch, yarn-1028-7.patch,
yarn-1028-8.patch, yarn-1028-draft-cumulative.patch
RMProxy layer currently abstracts RM discovery and implements it by looking
up service information from configuration. Motivated by HDFS and using
existing classes from Common, we can add failover proxy providers that may
provide RM discovery in extensible ways.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (YARN-1495) Allow moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848587#comment-13848587 ] Bikas Saha commented on YARN-1495: -- The app submission context saved in the store would need to be updated with the new queue information, after the scheduler has accepted the move but before the user gets notified about move completion. Allow moving apps between queues Key: YARN-1495 URL: https://issues.apache.org/jira/browse/YARN-1495 Project: Hadoop YARN Issue Type: New Feature Components: scheduler Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza This is an umbrella JIRA for work needed to allow moving YARN applications from one queue to another. The work will consist of additions in the command line options, additions in the client RM protocol, and changes in the schedulers to support this. I have a picture of how this should function in the Fair Scheduler, but I'm not familiar enough with the Capacity Scheduler for the same there. Ultimately, the decision to whether an application can be moved should go down to the scheduler - some schedulers may wish not to support this at all. However, schedulers that do support it should share some common semantics around ACLs and what happens to running containers. Here is how I see the general semantics working out: * A move request is issued by the client. After it gets past ACLs, the scheduler checks whether executing the move will violate any constraints. For the Fair Scheduler, these would be queue maxRunningApps and queue maxResources constraints * All running containers are transferred from the old queue to the new queue * All outstanding requests are transferred from the old queue to the new queue Here is I see the ACLs of this working out: * To move an app from a queue a user must have modify access on the app or administer access on the queue * To move an app to a queue a user must have submit access on the queue or administer access on the queue -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (YARN-311) Dynamic node resource configuration: core scheduler changes
[
https://issues.apache.org/jira/browse/YARN-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848615#comment-13848615
]
Bikas Saha commented on YARN-311:
-
Sorry for coming in so late on this.
Some javadoc on ResourceOption would be helpful. Explanation of the context of
dynamic resource changes on a node and resulting over-commitment of resources
would be good.
ResourceOption as a name did not make it clear to me the nature/use of the
object.
Would it be less error prone if we compared the total size of schedulernode and
rmnode instead of the difference in their current available capacity?
{code}
+Resource oldAvailableResource = node.getAvailableResource();
+Resource newAvailableResource = Resources.subtract(
+rmNode.getTotalCapability(), node.getUsedResource());
{code}
Also, in the update to the node. Why are we updating only the availableResource
and skipping totalresource? Total resource is used during scheduling decisions.
{code}
+ @Override
+ public synchronized void applyDeltaOnAvailableResource(Resource
deltaResource) {
+// we can only adjust available resource if total resource is changed.
+Resources.addTo(this.availableResource, deltaResource);
+ }
{code}
The current impl of addTo will work for both +ve and -ve deltas but given that
there are addTo and subtractFrom methods, its not clear to me if that is a
coincidence or not. Ideally there should have been one update method that by
definition should handle +ve and -ve updates.
Since changing the resource on a node would be an admin/service operation, why
are we adding resourceOption to the rmnode and setting it in
registernodemanager?
{code}
RMNode rmNode = new RMNodeImpl(nodeId, rmContext, host, cmPort, httpPort,
-resolve(host), capability, nodeManagerVersion);
+resolve(host), ResourceOption.newInstance(capability,
RMNode.OVER_COMMIT_TIMEOUT_MILLIS_DEFAULT),
+nodeManagerVersion);
{code}
Similarly, why are we trying to update the node on every heartbeat. I was
expecting that whenever the node resource would be updated then an event would
be sent to the scheduler. Upon receiving the event, the scheduler would make a
one time update of the internal book-keeping objects.
{code}
+// Update resource if any change
+SchedulerUtils.updateResourceIfChanged(node, nm, clusterResource, LOG);
{code}
Again, I apologize for coming in so late on this jira.
Dynamic node resource configuration: core scheduler changes
---
Key: YARN-311
URL: https://issues.apache.org/jira/browse/YARN-311
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager, scheduler
Reporter: Junping Du
Assignee: Junping Du
Fix For: 2.4.0
Attachments: YARN-311-v1.patch, YARN-311-v10.patch,
YARN-311-v11.patch, YARN-311-v12.patch, YARN-311-v12b.patch,
YARN-311-v13.patch, YARN-311-v2.patch, YARN-311-v3.patch, YARN-311-v4.patch,
YARN-311-v4.patch, YARN-311-v5.patch, YARN-311-v6.1.patch,
YARN-311-v6.2.patch, YARN-311-v6.patch, YARN-311-v7.patch, YARN-311-v8.patch,
YARN-311-v9.patch
As the first step, we go for resource change on RM side and expose admin APIs
(admin protocol, CLI, REST and JMX API) later. In this jira, we will only
contain changes in scheduler.
The flow to update node's resource and awareness in resource scheduling is:
1. Resource update is through admin API to RM and take effect on RMNodeImpl.
2. When next NM heartbeat for updating status comes, the RMNode's resource
change will be aware and the delta resource is added to schedulerNode's
availableResource before actual scheduling happens.
3. Scheduler do resource allocation according to new availableResource in
SchedulerNode.
For more design details, please refer proposal and discussions in parent
JIRA: YARN-291.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (YARN-312) Add updateNodeResource in ResourceManagerAdministrationProtocol
[ https://issues.apache.org/jira/browse/YARN-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848617#comment-13848617 ] Bikas Saha commented on YARN-312: - Related to YARN-311, IMO we should be sending an event to the scheduler to process the node resource update after the RM is notified about the update. Add updateNodeResource in ResourceManagerAdministrationProtocol --- Key: YARN-312 URL: https://issues.apache.org/jira/browse/YARN-312 Project: Hadoop YARN Issue Type: Sub-task Components: api Affects Versions: 2.2.0 Reporter: Junping Du Assignee: Junping Du Attachments: YARN-312-v1.patch, YARN-312-v10.patch, YARN-312-v2.patch, YARN-312-v3.patch, YARN-312-v4.1.patch, YARN-312-v4.patch, YARN-312-v5.1.patch, YARN-312-v5.patch, YARN-312-v6.patch, YARN-312-v7.1.patch, YARN-312-v7.1.patch, YARN-312-v7.patch, YARN-312-v8.patch, YARN-312-v9.patch Add fundamental RPC (ResourceManagerAdministrationProtocol) to support node's resource change. For design detail, please refer parent JIRA: YARN-291. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (YARN-1307) Rethink znode structure for RM HA
[
https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848641#comment-13848641
]
Bikas Saha commented on YARN-1307:
--
its going to help scale zk store now that we are storing apps for a longer
period. with 10K apps there will be 20K-30K znodes created under the root
znode. I dont know if anyone has tried creating so many znodes under a single
parent. IIRC, the number of znodes needed to store (and thus retrieve) the data
should be the same as earlier (1 per app and app-attempt) Thus we should not be
doing many more zk operations compared to the current structure. [~ozawa]?
Rethink znode structure for RM HA
-
Key: YARN-1307
URL: https://issues.apache.org/jira/browse/YARN-1307
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Tsuyoshi OZAWA
Assignee: Tsuyoshi OZAWA
Attachments: YARN-1307.1.patch, YARN-1307.10.patch,
YARN-1307.11.patch, YARN-1307.2.patch, YARN-1307.3.patch,
YARN-1307.4-2.patch, YARN-1307.4-3.patch, YARN-1307.4.patch,
YARN-1307.5.patch, YARN-1307.6.patch, YARN-1307.7.patch, YARN-1307.8.patch,
YARN-1307.9.patch, YARN-1307.9.patch
Rethink for znode structure for RM HA is proposed in some JIRAs(YARN-659,
YARN-1222). The motivation of this JIRA is quoted from Bikas' comment in
YARN-1222:
{quote}
We should move to creating a node hierarchy for apps such that all znodes for
an app are stored under an app znode instead of the app root znode. This will
help in removeApplication and also in scaling better on ZK. The earlier code
was written this way to ensure create/delete happens under a root znode for
fencing. But given that we have moved to multi-operations globally, this isnt
required anymore.
{quote}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (YARN-1125) Add shutdown support to non-service RM components
[ https://issues.apache.org/jira/browse/YARN-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848647#comment-13848647 ] Bikas Saha commented on YARN-1125: -- We probably need to do this on a case by case basis determined by the amount of effort needed vs the gains from a clean stop. [~kkambatl] reports that his experiments have not shown any memory leaks in the current approach of deleting all components. Add shutdown support to non-service RM components - Key: YARN-1125 URL: https://issues.apache.org/jira/browse/YARN-1125 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.1.0-beta Reporter: Karthik Kambatla Assignee: Xuan Gong The ResourceManager has certain non-service components like the Scheduler. While transitioning to standby, these components should be completely turned off. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (YARN-1451) TestResourceManager relies on the capacity scheduler
[ https://issues.apache.org/jira/browse/YARN-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1451: - Summary: TestResourceManager relies on the capacity scheduler (was: A couple tests rely on the capacity scheduler) TestResourceManager relies on the capacity scheduler Key: YARN-1451 URL: https://issues.apache.org/jira/browse/YARN-1451 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Priority: Minor TestNetworkedJob and TestResourceManager rely on the capacity scheduler. For TestNetworkedJob, the queue name chosen for the job is different for the Fair Scheduler. Configuring the Capacity Scheduler explicitly would make this dependency more clear. TestResourceManager relies on a scheduler that assigns multiple containers in a single heartbeat, which not all schedulers do by default. It also relies on schedulers that don't consider CPU capacities. It would be simple to change the test to use multiple heartbeats increase the vcore capacities of the nodes in the test. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (YARN-1451) TestResourceManager relies on the capacity scheduler
[ https://issues.apache.org/jira/browse/YARN-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1451: - Description: TestResourceManager rely on the capacity scheduler. For TestNetworkedJob, the queue name chosen for the job is different for the Fair Scheduler. Configuring the Capacity Scheduler explicitly would make this dependency more clear. TestResourceManager relies on a scheduler that assigns multiple containers in a single heartbeat, which not all schedulers do by default. It also relies on schedulers that don't consider CPU capacities. It would be simple to change the test to use multiple heartbeats increase the vcore capacities of the nodes in the test. was: TestNetworkedJob and TestResourceManager rely on the capacity scheduler. For TestNetworkedJob, the queue name chosen for the job is different for the Fair Scheduler. Configuring the Capacity Scheduler explicitly would make this dependency more clear. TestResourceManager relies on a scheduler that assigns multiple containers in a single heartbeat, which not all schedulers do by default. It also relies on schedulers that don't consider CPU capacities. It would be simple to change the test to use multiple heartbeats increase the vcore capacities of the nodes in the test. TestResourceManager relies on the capacity scheduler Key: YARN-1451 URL: https://issues.apache.org/jira/browse/YARN-1451 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Priority: Minor TestResourceManager rely on the capacity scheduler. For TestNetworkedJob, the queue name chosen for the job is different for the Fair Scheduler. Configuring the Capacity Scheduler explicitly would make this dependency more clear. TestResourceManager relies on a scheduler that assigns multiple containers in a single heartbeat, which not all schedulers do by default. It also relies on schedulers that don't consider CPU capacities. It would be simple to change the test to use multiple heartbeats increase the vcore capacities of the nodes in the test. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (YARN-1451) TestResourceManager relies on the capacity scheduler
[ https://issues.apache.org/jira/browse/YARN-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1451: - Description: TestResourceManager rely on the capacity scheduler. It relies on a scheduler that assigns multiple containers in a single heartbeat, which not all schedulers do by default. It also relies on schedulers that don't consider CPU capacities. It would be simple to change the test to use multiple heartbeats and increase the vcore capacities of the nodes in the test. was: TestResourceManager rely on the capacity scheduler. For TestNetworkedJob, the queue name chosen for the job is different for the Fair Scheduler. Configuring the Capacity Scheduler explicitly would make this dependency more clear. TestResourceManager relies on a scheduler that assigns multiple containers in a single heartbeat, which not all schedulers do by default. It also relies on schedulers that don't consider CPU capacities. It would be simple to change the test to use multiple heartbeats increase the vcore capacities of the nodes in the test. TestResourceManager relies on the capacity scheduler Key: YARN-1451 URL: https://issues.apache.org/jira/browse/YARN-1451 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Priority: Minor TestResourceManager rely on the capacity scheduler. It relies on a scheduler that assigns multiple containers in a single heartbeat, which not all schedulers do by default. It also relies on schedulers that don't consider CPU capacities. It would be simple to change the test to use multiple heartbeats and increase the vcore capacities of the nodes in the test. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (YARN-1451) TestResourceManager relies on the capacity scheduler
[ https://issues.apache.org/jira/browse/YARN-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1451: - Attachment: YARN-1451.patch TestResourceManager relies on the capacity scheduler Key: YARN-1451 URL: https://issues.apache.org/jira/browse/YARN-1451 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Priority: Minor Attachments: YARN-1451.patch TestResourceManager rely on the capacity scheduler. It relies on a scheduler that assigns multiple containers in a single heartbeat, which not all schedulers do by default. It also relies on schedulers that don't consider CPU capacities. It would be simple to change the test to use multiple heartbeats and increase the vcore capacities of the nodes in the test. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (YARN-1451) TestResourceManager relies on the capacity scheduler
[
https://issues.apache.org/jira/browse/YARN-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13848776#comment-13848776
]
Hadoop QA commented on YARN-1451:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12618845/YARN-1451.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2665//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2665//console
This message is automatically generated.
TestResourceManager relies on the capacity scheduler
Key: YARN-1451
URL: https://issues.apache.org/jira/browse/YARN-1451
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
Priority: Minor
Attachments: YARN-1451.patch
TestResourceManager rely on the capacity scheduler.
It relies on a scheduler that assigns multiple containers in a single
heartbeat, which not all schedulers do by default. It also relies on
schedulers that don't consider CPU capacities. It would be simple to change
the test to use multiple heartbeats and increase the vcore capacities of the
nodes in the test.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
