[
https://issues.apache.org/jira/browse/MAPREDUCE-7277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089097#comment-17089097
]
Jonathan Turner Eagles commented on MAPREDUCE-7277:
---------------------------------------------------
It was also noted that during testing, testCreateRace, which is trying to test
this the race of adding and removing items from the cache in a multithreaded
environment, is actually broken in multiple ways. 1) getIndexInformation is
passing index 1000 which throws an exception as only 0-999 is valid 2) the
threads are designed to run for 15 seconds but the code only ran each thread a
single time and then exited. 3) the threads were never interrupted before
joining.(as they were already finished after running a single time, throwing
and exception, an then exiting).
> IndexCache totalMemoryUsed differs from cache contents.
> -------------------------------------------------------
>
> Key: MAPREDUCE-7277
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7277
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Jonathan Turner Eagles
> Assignee: Jonathan Turner Eagles
> Priority: Major
> Attachments: IndexCacheActualSize.png, MAPREDUCE-7277.001.patch
>
>
> It was observed recently in a nodemanager OOM that the memory was filled with
> SpillRecords. However, the IndexCache was only 15% full (1.5MB used on a 10MB
> configured cache size). In particular was noted that the booking variable
> totalMemoryUsed, was out of sync with the contents of the cache showing 96%
> full, thereby drastically reducing the effectiveness of the cache.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
