[
https://issues.apache.org/jira/browse/MAPREDUCE-345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer resolved MAPREDUCE-345.
----------------------------------------
Resolution: Incomplete
Closing this as stale.
> Investigate whether the array in the JobInProgress that holds TIP references
> can be removed
> -------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-345
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-345
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Devaraj Das
>
> The array, in JobInProgress, that holds the references to TIPs is required
> now only to serve the clients (web UI via JSPs and JobClient APIs, that
> traverse the array and create the output). The array can now be removed since
> HADOOP-2119 introduces some datastructures for running/non-running TIPs and
> we could probably add some more if required (e.g., for completed TIPs). That
> way, we will gain when we have large jobs (many tasks) in the sense that we
> don't have to go through the big array every time a client request is made.
> We could instead iterate over the datastructures. Also, we should do these
> traversals without locking the JobTracker to avoid cases where long traversal
> leads to loss of heartbeats, etc. Some staleness at the client side (in favor
> of improving the JobTracker's performance) is probably okay.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
