[ https://issues.apache.org/jira/browse/HBASE-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262388#comment-13262388 ]
ramkrishna.s.vasudevan commented on HBASE-5881: ----------------------------------------------- We need to document this? > BuiltIn Gzip compressor & decompressor not getting pooled, leading to native > memory leak > ---------------------------------------------------------------------------------------- > > Key: HBASE-5881 > URL: https://issues.apache.org/jira/browse/HBASE-5881 > Project: HBase > Issue Type: Bug > Components: io > Affects Versions: 0.92.1 > Reporter: Gopinathan A > Priority: Critical > Fix For: 0.92.2, 0.94.0, 0.96.0 > > > This issue will occur only in hadoop 23.x & above/ > In hadoop 0.20.x > {code} > public static void returnDecompressor(Decompressor decompressor) { > if (decompressor == null) { > return; > } > decompressor.reset(); > payback(decompressorPool, decompressor); > } > {code} > In hadoop 0.23.x > {code} > public static void returnDecompressor(Decompressor decompressor) { > if (decompressor == null) { > return; > } > // if the decompressor can't be reused, don't pool it. > if (decompressor.getClass().isAnnotationPresent(DoNotPool.class)) { > return; > } > decompressor.reset(); > payback(decompressorPool, decompressor); > } > {code} > Here annotation has been added. By default this library will be loaded if > there are no native library. > {code} > @DoNotPool > public class BuiltInGzipDecompressor > {code} > Due to this each time new compressor/decompressor will be loaded, this leads > to native memory leak. > {noformat} > 2012-04-25 22:11:48,093 INFO org.apache.hadoop.io.compress.CodecPool: Got > brand-new decompressor [.gz] > 2012-04-25 22:11:48,093 INFO org.apache.hadoop.io.compress.CodecPool: Got > brand-new decompressor [.gz] > 2012-04-25 22:11:48,093 INFO org.apache.hadoop.io.compress.CodecPool: Got > brand-new decompressor [.gz] > {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira