[
https://issues.apache.org/jira/browse/HADOOP-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505800#comment-14505800
]
Lari Hotari commented on HADOOP-7154:
-------------------------------------
A note about MALLOC_ARENA_MAX:
MALLOC_ARENA_MAX is broken on glibc < 2.15 (like Ubuntu 10.04) . The fix was
made for 2.16 and backported to 2.15 . MALLOC_ARENA_MAX doesn't work on Ubuntu
10.04 because of [this
bug|https://sourceware.org/bugzilla/show_bug.cgi?id=13071].
The same bug seems to be reported to Redhat as
https://bugzilla.redhat.com/show_bug.cgi?id=799327 . Other reports:
https://sourceware.org/bugzilla/show_bug.cgi?id=13137 ,
https://sourceware.org/bugzilla/show_bug.cgi?id=13754 ,
https://sourceware.org/bugzilla/show_bug.cgi?id=11261 .
This is the commit to glibc fixing the bug:
https://github.com/bminor/glibc/commit/41b81892f11fe1353123e892158b53de73863d62
(backport for 2.15 is
https://github.com/bminor/glibc/commit/7cf8e20d03a43b1375e90d381a16caa2686e4fdf
).
> Should set MALLOC_ARENA_MAX in hadoop-config.sh
> -----------------------------------------------
>
> Key: HADOOP-7154
> URL: https://issues.apache.org/jira/browse/HADOOP-7154
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 1.0.4, 0.22.0
>
> Attachments: hadoop-7154.txt
>
>
> New versions of glibc present in RHEL6 include a new arena allocator design.
> In several clusters we've seen this new allocator cause huge amounts of
> virtual memory to be used, since when multiple threads perform allocations,
> they each get their own memory arena. On a 64-bit system, these arenas are
> 64M mappings, and the maximum number of arenas is 8 times the number of
> cores. We've observed a DN process using 14GB of vmem for only 300M of
> resident set. This causes all kinds of nasty issues for obvious reasons.
> Setting MALLOC_ARENA_MAX to a low number will restrict the number of memory
> arenas and bound the virtual memory, with no noticeable downside in
> performance - we've been recommending MALLOC_ARENA_MAX=4. We should set this
> in hadoop-env.sh to avoid this issue as RHEL6 becomes more and more common.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
