[jira] [Commented] (MAPREDUCE-4288) ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one when no job is running
[ https://issues.apache.org/jira/browse/MAPREDUCE-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539585#comment-14539585 ] Rohith commented on MAPREDUCE-4288: --- Linking to similar issue in MR > ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one > when no job is running > --- > > Key: MAPREDUCE-4288 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4288 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.0.0-alpha >Reporter: Nishan Shetty > > When no job is running in the cluster invoke the ClusterStatus.getMapTasks() > and ClusterStatus.getReduceTasks() API's > Observed that these API's are returning one instead of zero(as no job is > running) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4288) ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one when no job is running
[ https://issues.apache.org/jira/browse/MAPREDUCE-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539583#comment-14539583 ] Rohith commented on MAPREDUCE-4288: --- The issue is still exist in the latest code base. This issue require more discussion on whether to change the hard coded values which may break comatibility for the MR client OR bring out a new design to handle this dielema. If not planning to fix , I think this can be closed as wont fix. > ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one > when no job is running > --- > > Key: MAPREDUCE-4288 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4288 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.0.0-alpha >Reporter: Nishan Shetty > > When no job is running in the cluster invoke the ClusterStatus.getMapTasks() > and ClusterStatus.getReduceTasks() API's > Observed that these API's are returning one instead of zero(as no job is > running) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4288) ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one when no job is running
[ https://issues.apache.org/jira/browse/MAPREDUCE-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396130#comment-13396130 ] Karthik Kambatla commented on MAPREDUCE-4288: - In YARN, the ClusterMetrics should only correspond to numNodeManagers, numActiveJobs(), numActiveContainers(), availableResources(). Other job/app-specific metrics should move to the corresponding AMs. JobStatus would be a good place to have these metrics. Subsequently, JobClient.getClusterStatus() can correspond to the job-specific metrics (would be a misnomer). Comments? > ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one > when no job is running > --- > > Key: MAPREDUCE-4288 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4288 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.0.0-alpha >Reporter: Nishan Shetty >Assignee: Karthik Kambatla > > When no job is running in the cluster invoke the ClusterStatus.getMapTasks() > and ClusterStatus.getReduceTasks() API's > Observed that these API's are returning one instead of zero(as no job is > running) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4288) ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one when no job is running
[
https://issues.apache.org/jira/browse/MAPREDUCE-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291555#comment-13291555
]
Devaraj K commented on MAPREDUCE-4288:
--
It is coming due to the hard-coded values in the below code,
{code:title=ResourceMgrDelegate.java|borderStyle=solid}
public ClusterMetrics getClusterMetrics() throws IOException,
InterruptedException {
GetClusterMetricsRequest request =
recordFactory.newRecordInstance(GetClusterMetricsRequest.class);
GetClusterMetricsResponse response =
applicationsManager.getClusterMetrics(request);
YarnClusterMetrics metrics = response.getClusterMetrics();
ClusterMetrics oldMetrics = new ClusterMetrics(1, 1, 1, 1, 1, 1,
metrics.getNumNodeManagers() * 10, metrics.getNumNodeManagers() * 2, 1,
metrics.getNumNodeManagers(), 0, 0);
return oldMetrics;
}
{code}
Here we cannot get runningMaps, runningReduces, occupiedMapSlots...etc from RM
because the yarn cluster is completely based on the resources and resource
usages.
It doesn't look good to show these hard-coded values always to the user when
they try to get cluster status using the JobClient.getClusterStatus() API.
Any thoughts on this?
> ClusterStatus.getMapTasks() and ClusterStatus.getReduceTasks() is giving one
> when no job is running
> ---
>
> Key: MAPREDUCE-4288
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4288
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 2.0.0-alpha
>Reporter: Nishan Shetty
>
> When no job is running in the cluster invoke the ClusterStatus.getMapTasks()
> and ClusterStatus.getReduceTasks() API's
> Observed that these API's are returning one instead of zero(as no job is
> running)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
