[ https://issues.apache.org/jira/browse/HBASE-21162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16610491#comment-16610491 ]
Hudson commented on HBASE-21162: -------------------------------- Results for branch branch-1.4 [build #457 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/457/]: (x) *{color:red}-1 overall{color}* ---- details (if available): (x) {color:red}-1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/457//General_Nightly_Build_Report/] (x) {color:red}-1 jdk7 checks{color} -- For more information [see jdk7 report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/457//JDK7_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/457//JDK8_Nightly_Build_Report_(Hadoop2)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > Revert suspicious change to BoundedByteBufferPool and disable use of direct > buffers for IPC reservoir by default > ---------------------------------------------------------------------------------------------------------------- > > Key: HBASE-21162 > URL: https://issues.apache.org/jira/browse/HBASE-21162 > Project: HBase > Issue Type: Bug > Affects Versions: 1.4.7 > Reporter: Andrew Purtell > Assignee: Andrew Purtell > Priority: Critical > Fix For: 1.5.0, 1.4.8 > > Attachments: HBASE-21162-branch-1.patch, HBASE-21162-branch-1.patch, > HBASE-21162-branch-1.patch > > > We had a production incident where we traced the issue to a direct buffer > leak. On a hunch we tried setting hbase.ipc.server.reservoir.enabled = false > and after that no native memory leak could be observed in any regionserver > process under the triggering load. > On HBASE-19239 (Fix findbugs and error-prone issues) I made a change to > BoundedByteBufferPool that is suspicious given this finding. It was committed > to branch-1.4 and branch-1. I'm going to revert this change. > In addition the allocation of direct memory for the server RPC reservoir is a > bit problematic in that tracing native memory or direct buffer leaks to a > particular class or compilation unit is difficult, so I also propose > allocating the reservoir on the heap by default instead. Should there be a > leak it is much easier to do an analysis of a heap dump with familiar tools > to find it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)