[ 
https://issues.apache.org/jira/browse/MAPREDUCE-7277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089839#comment-17089839
 ] 

Jim Brennan commented on MAPREDUCE-7277:
----------------------------------------

Thanks for the patch [~jeagles]!

Question about readIndexFileToCache() in the case where we throw IOException if 
we fail to construct the SpillRecord.  In the old code, we just executed the 
finally clause and then threw the IOException without doing the queue.add() and 
total memory update.  In the new code, we are still doing the queue.add() in 
this case.  Was this intended?

Also, I don't understand why you call checkTotalMemoryUsed() here - is this 
left over debug code?


> IndexCache totalMemoryUsed differs from cache contents.
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-7277
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7277
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Jonathan Turner Eagles
>            Assignee: Jonathan Turner Eagles
>            Priority: Major
>         Attachments: IndexCacheActualSize.png, MAPREDUCE-7277.001.patch
>
>
> It was observed recently in a nodemanager OOM that the memory was filled with 
> SpillRecords. However, the IndexCache was only 15% full (1.5MB used on a 10MB 
> configured cache size). In particular was noted that the booking variable 
> totalMemoryUsed, was out of sync with the contents of the cache showing 96% 
> full, thereby drastically reducing the effectiveness of the cache.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to