[
https://issues.apache.org/jira/browse/TEZ-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058237#comment-14058237
]
Bikas Saha edited comment on TEZ-1269 at 7/11/14 2:18 AM:
----------------------------------------------------------
1) Removes the session specific check to release an idle container.
2) Creates a min held containers that maintains a minimum held pool only in
session mode.
3) Randomizes the idle container expire time between a min and max value to
prevent a cliff of de-allocations.
Existing tests broke for 1 and 2. So fixing them gives some coverage. This also
fixes TestMRRJobsDAGApi.testBroadcastAndOneToOneExample() so it now can run in
non-session mode and get expected container reuse for the 1-1 edge.
Will think of some way to test that min-max randomness if folks think thats
necessary. [~gopalv] [~hitesh] Please review.
was (Author: bikassaha):
1) Removes the session specific check to release an idle container.
2) Creates a min held containers that maintains a minimum held pool only in
session mode.
3) Randomizes the idle container expire time between a min and max value to
prevent a cliff of de-allocations.
Existing tests broke for 1 and 2. So fixing them gives some coverage. Will
think of some way to test that min-max randomness if folks think thats
necessary. [~gopalv] [~hitesh] Please review.
> TaskScheduler prematurely releases containers
> ---------------------------------------------
>
> Key: TEZ-1269
> URL: https://issues.apache.org/jira/browse/TEZ-1269
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-1269.1.patch
>
>
> It checks for session mode and if not true, and if there are no outstanding
> requests, then it releases the containers before the container timeout has
> expired. If the state machine is on its way to scheduling new tasks during
> this time then they will not be able to reuse these containers.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
