[
https://issues.apache.org/jira/browse/HADOOP-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647544#action_12647544
]
Karam Singh commented on HADOOP-4658:
-------------------------------------
Submitted 4 sleep jobs with different user -:
Job J1, User1, maps=208, reduces=0, sleeptime=120000 milliseconds.
Job J2, User2, maps=208, reduces=0, sleeptime=120000 milliseconds.
Job J3, User3, maps=104, reduces=0, sleeptime=124000 milliseconds.
Job J4, User4, maps=104, reduces=0, sleeptime=124000 milliseconds.
When all job starts running, the status from Jobtacker UI is -:
Cluster Summary: Nodes=104, Maps=208, Reduces=0, Map_Capacity=208,
Reduce_Capacity=208, Avg. Task/Node=4.00
Queue: Name=default, running_maps=207, waiting_maps=423
J1 having pending_maps=159, running_maps=52
J2 having pending_maps=159, running-maps=52
J3 having pending_maps=52, running_maps=52
J4 having pending_maps=52, running_maps=52
When J3 and J4 finish, status from Jobtacker UI is -:
Cluster Summary: Nodes=104, Maps=208, Reduces=0, Map_Capacity=208,
Reduce_Capacity=208, Avg. Task/Node=4.00
Queues: Name=default, running_maps=207, waiting_maps=34
J1 having pending_maps=0, running_maps=104
J2 having pending_maps=33, running_maps=71
Observation -:
1. J2 should also be using 104 as its user-limit should expand back to 50% like
J1 whereas it is using 71 slots.
2. Actually running_maps=J1+j2=175, pending_maps=J1+j2=33, where as cluster
status and queue information shows maps=208 (running), where slots 33 went.
Note -: speculative execution was true.
> User limit is not expanding back properly.
> ------------------------------------------
>
> Key: HADOOP-4658
> URL: https://issues.apache.org/jira/browse/HADOOP-4658
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Environment: GC=100% nodes=104, map_capacity=208,
> reduce_capacity=208, user-limit=25%;
> Reporter: Karam Singh
>
> User limit is not expanding back properly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
