[ http://issues.apache.org/jira/browse/HADOOP-815?page=comments#action_12458069 ] Arun C Murthy commented on HADOOP-815: --------------------------------------
Change 1: I plan to remove: org.apache.hadoop.mapred.TaskInProgress.totalTaskIds (String[])& org.apache.hadoop.mapred.TaskInProgress.usableTaskIds (TreeSet) and replace them with: ArrayList<String> usableTaskIds totalTaskIds isn't used anywhere except in org.apache.hadoop.mapred.TaskInProgress.init() and we don't need usableTaskIds to be a TreeSet, and ArrayList should suffice... (I'll keep updating this issue with proposed changes as I glean more info from memory profiles of the JobTracker.) > Investigate and fix the extremely large memory-footprint of JobTracker > ---------------------------------------------------------------------- > > Key: HADOOP-815 > URL: http://issues.apache.org/jira/browse/HADOOP-815 > Project: Hadoop > Issue Type: Bug > Components: mapred > Affects Versions: 0.9.1 > Reporter: Arun C Murthy > Assigned To: Arun C Murthy > Fix For: 0.10.0 > > > The JobTracker's memory footprint seems excessively large, especially when > many jobs are submitted. > Here is the 'top' output of a JobTracker which has scheduled ~1k jobs thus > far: > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > > 31877 arunc 19 0 2362m 261m 13m S 14.0 12.9 24:48.08 java > Clearly VIRTual memory of 2364Mb v/s 261Mb of RESident memory is symptomatic > of this issue... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
