[
https://issues.apache.org/jira/browse/MAPREDUCE-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088090#comment-13088090
]
Todd Lipcon commented on MAPREDUCE-2649:
----------------------------------------
Coming late to see this patch. One question from an MR2 n00b: in MR1, these
configs were very error prone since different jobs took up wildly variant
amounts of RAM. So a value of 100 retained jobs would work fine for small jobs
but fail miserably with OOME if bigger jobs were submitted. Then we added all
of the limits to try to curtail this, but it's a bit of a stopgap rather than a
real solution.
Is this better in MR2 because the state on the RM is much smaller? Or is there
still per-container state being retained?
One thing we should consider is to adopt an interface like HBase's "HeapSize"
-- objects that are going to be retained/cached need to implement it, and then
the limit scan be set based on memory usage rather than strict counts.
> MR279: Fate of finished Applications on RM
> ------------------------------------------
>
> Key: MAPREDUCE-2649
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2649
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Reporter: Thomas Graves
> Assignee: Thomas Graves
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2649-patch-mr279.txt, MAPREDUCE-2649-v2.patch,
> MAPREDUCE-2649-v3.patch, MAPREDUCE-2649-v4.patch, MAPREDUCE-2649-v5.patch,
> MAPREDUCE-2649-v6.patch, MAPREDUCE-2649-v7.patch
>
>
> Today RM keeps the references of finished application for ever. Though this
> is not sustainable long term, it keeps
> the user experience saner. Users can revisit RM UI and check the status of
> their apps.
> We need to think of purging old references yet keeping the UX sane.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
