[
https://issues.apache.org/jira/browse/MAPREDUCE-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated MAPREDUCE-7100:
--------------------------------
Description:
We are using hadoop 2.7.3 and the computer layer is running out of the storage
cluster (that is, node managers are running on a different set of nodes from
data node). The problem we meet is that the container allocation is quite slow.
After some debugging, we found that in
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor#addContainerReq()
{code}
protected void addContainerReq(ContainerRequest req) {
// Create resource requests
for (String host : req.hosts) {
// Data-local
if (!isNodeBlacklisted(host)) {
addResourceRequest(req.priority, host, req.capability,
null);
}
}
// Nothing Rack-local for now
for (String rack : req.racks) {
addResourceRequest(req.priority, rack, req.capability,
null);
}
// Off-switch
addResourceRequest(req.priority, ResourceRequest.ANY, req.capability,
req.nodeLabelExpression);
}
{code}
was:
We are using hadoop 2.7.3 and the computer layer is running out of the storage
cluster (that is, node managers are running on a different set of nodes from
data node). The problem we meet is that the container allocation is quite slow.
After some debugging, we found that in
{{org.apache.hadoop.mapreduce.v2.app.rm}}
> Provide options to skip adding container request for data-local and
> rack-local respectively
> -------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7100
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7100
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
> Reporter: Xiang Li
> Priority: Minor
>
> We are using hadoop 2.7.3 and the computer layer is running out of the
> storage cluster (that is, node managers are running on a different set of
> nodes from data node). The problem we meet is that the container allocation
> is quite slow.
> After some debugging, we found that in
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor#addContainerReq()
> {code}
> protected void addContainerReq(ContainerRequest req) {
> // Create resource requests
> for (String host : req.hosts) {
> // Data-local
> if (!isNodeBlacklisted(host)) {
> addResourceRequest(req.priority, host, req.capability,
> null);
> }
> }
> // Nothing Rack-local for now
> for (String rack : req.racks) {
> addResourceRequest(req.priority, rack, req.capability,
> null);
> }
> // Off-switch
> addResourceRequest(req.priority, ResourceRequest.ANY, req.capability,
> req.nodeLabelExpression);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
