[
https://issues.apache.org/jira/browse/MAPREDUCE-6749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15550874#comment-15550874
]
Devaraj K commented on MAPREDUCE-6749:
--------------------------------------
Thanks [~Naganarasimha] for looking into this and looking forward to work on
this with you.
{quote}
Is it required to have number of total Maps(/reducers) to be run using
container reuse? i felt we can just keep whether to reuse the containers are
not and one assumption that map containers will be reused only for map and
reduce only for reduce
{quote}
I think the limit configuration for no of map/reduce reuse containers would
allow other applications to start running without waiting for the Job to be
finished when reuse is enabled. If there is a big Job running which could
occupy the entire cluster, and then any high priority application gets
submitted this limit for maps/reduce container would probably give a room for
high priority application to start running without preempting the containers of
the previous Job. By default there is no limit for number of containers to be
reused and if any user/Job wanted to have this constraint they can configure
it.
{quote}
did not understand the need of RMContainerReuseRequestor.java, may be i will go
through MAPREDUCE-6773 but was expecting more like just TaskAttemptIimpl gets
TA_DONE event was expecting it do have a new ReuseContainerTransition where in
if required it will reassign a new task to it else will continue to
CLEANUP_CONTAINER_TRANSITION. may be we can discuss more on this in
MAPREDUCE-6773 ?
{quote}
RMContainerReuseRequestor has the functionality for checking the container
reuse constraints and adding to reuse containers if it is eligible. This class
abstracts the reuse functionality from regular RMContainerRequestor and
RMContainerReuseRequestor will be used only when the RMContainer reuse is
enabled. If you want to have a try this feature, you can apply MAPREDUCE-6773,
MAPREDUCE-6781, MAPREDUCE-6784, MAPREDUCE-6785, MAPREDUCE-6786 and then try
this feature. I would be happy to get your feedback or suggestions if any.
{quote}
If we have some conclusion on issue 2, then we can think of whether only
TaskAttemptImpl state is sufficient or we need to modify others too.
{quote}
Sure, We can discuss it further if required.
{quote} Logs generated by the tasks in a container will be appended in the same
container log file we need to ensure we are able to display trhough JHS and
other places where ever required to display
{quote}
TaskAttempt is linked to the container and the corresponding container log
would be displayed for TaskAttempt through the JHS. Here we should note that
the whole container log which is displaying for TaskAttempt is not applicable
to the TaskAttempt and the log can be identified easily which part applicable
to it.
> MR AM should reuse containers for Map/Reduce Tasks
> --------------------------------------------------
>
> Key: MAPREDUCE-6749
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6749
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster, mrv2
> Reporter: Devaraj K
> Assignee: Devaraj K
> Attachments: MAPREDUCE-6749-Container Reuse-v0.pdf
>
>
> It is with the continuation of MAPREDUCE-3902, MR AM should reuse containers
> for Map/Reduce Tasks similar to the JVM Reuse feature we had in MRv1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
