[ 
https://issues.apache.org/jira/browse/MAPREDUCE-7277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093982#comment-17093982
 ] 

Jonathan Turner Eagles commented on MAPREDUCE-7277:
---------------------------------------------------

I've decided the easiest and most effective way to do this is to switch logging 
from apache commons to slf4j logging. The dependency is present in 2.10 (and 
even 2.8). That gives us the full benefit of cpu reduction from the LOG.debug 
statements that are unnecessarily computing a debug log string that is thrown 
away when debug logging is disabled.

> IndexCache totalMemoryUsed differs from cache contents.
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-7277
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7277
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Jonathan Turner Eagles
>            Assignee: Jonathan Turner Eagles
>            Priority: Major
>         Attachments: IndexCacheActualSize.png, MAPREDUCE-7277.001.patch, 
> MAPREDUCE-7277.002.patch, MAPREDUCE-7277.003.patch, 
> MAPREDUCE-7277.004-branch-2.10.patch, MAPREDUCE-7277.004.patch
>
>
> It was observed recently in a nodemanager OOM that the memory was filled with 
> SpillRecords. However, the IndexCache was only 15% full (1.5MB used on a 10MB 
> configured cache size). In particular was noted that the booking variable 
> totalMemoryUsed, was out of sync with the contents of the cache showing 96% 
> full, thereby drastically reducing the effectiveness of the cache.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to