[
https://issues.apache.org/jira/browse/HADOOP-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647558#action_12647558
]
Amar Kamat commented on HADOOP-4658:
------------------------------------
Here is the scenario
_Setup :_
j1 and j2 are 2 jobs submitted by user user1 and user2 resp and run for longer
time.
j3 and j4 are 2 jobs submitted by user user3 and user4 resp and run for less
time .
Only "default" queue is used.
_Analysis :_
||jobs||num-running-tasks||limit||who will schedule||comment||
|j1, j2, j3, j4|25, 25, 25, 25| 25| j1 |everyone is over limit|
|j1, j2, j3*, j4|40, 25, 10, 25| 25| j1 |everyone is over limit and job3 is
finishing off|
|j1, j2, j4|50, 25, 25| 33| j2 |only user1 is over limit as limit is re-defined|
|j1, j2, j4*|50, 33, 17| 33| j1 |everyone is over limit and job4 is finishing
off|
|j1, j2|67, 33| 50| j2 |user1 is over limit as limits is redefined|
Note that j1 might actually have 50 maps to run but then is can run 67 with
this setup. So it can go ahead an run 17 speculative tasks instead of job2
running its 17 genuine tasks.
With the proposed change
||jobs||num-running-tasks||limit||who will schedule||comment||
|j1, j2, j3, j4|25, 25, 25, 25| 25| j1 |everyone is over limit|
|j1, j2, j3*, j4|33, 25, 17, 25| 33| j2|user1 is over limit as limit is
redefined and job3 is finishing off, slots will be equally divided|
|j1, j2, j3*, j4|33, 33, 9, 25| 33| j1 |everyone is overlimit and job3 is
finishing off|
|j1, j2, j4|42, 33, 25| 33| j1 |everyone is overlimit|
|j1, j2, j4*|50, 33, 17| 50| j2 |as user1 is over limit as limit are redefined
and job4 is finishing off|
|j1, j2|50, 50| 50| j1 |as everyone is overlimit|
> User limit is not expanding back properly.
> ------------------------------------------
>
> Key: HADOOP-4658
> URL: https://issues.apache.org/jira/browse/HADOOP-4658
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Environment: GC=100% nodes=104, map_capacity=208,
> reduce_capacity=208, user-limit=25%;
> Reporter: Karam Singh
> Assignee: Amar Kamat
>
> User limit is not expanding back properly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
