[ https://issues.apache.org/jira/browse/HBASE-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206335#comment-13206335 ]
Phabricator commented on HBASE-5387: ------------------------------------ lhofhansl has accepted the revision "[jira] [HBASE-5387] Reuse compression streams in HFileBlock.Writer". Nice patch Mikhail. Two more nits inline, can be done on commit. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/io/hfile/Compression.java:216 Make an HConstant for "io.file.buffer.size" and/or add it to hbase-defaults.xml? src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:613 Since it confused many of us, maybe add a brief comment that is OK to keep the BAOS per Writer, because of the specific, limited lifetime of a writer object. REVISION DETAIL https://reviews.facebook.net/D1719 > Reuse compression streams in HFileBlock.Writer > ---------------------------------------------- > > Key: HBASE-5387 > URL: https://issues.apache.org/jira/browse/HBASE-5387 > Project: HBase > Issue Type: Bug > Affects Versions: 0.94.0 > Reporter: Mikhail Bautin > Assignee: Mikhail Bautin > Priority: Critical > Fix For: 0.94.0 > > Attachments: D1719.1.patch, D1719.2.patch, D1719.3.patch, > Fix-deflater-leak-2012-02-10_18_48_45.patch, > Fix-deflater-leak-2012-02-11_17_13_10.patch > > > We need to to reuse compression streams in HFileBlock.Writer instead of > allocating them every time. The motivation is that when using Java's built-in > implementation of Gzip, we allocate a new GZIPOutputStream object and an > associated native data structure every time we create a compression stream. > The native data structure is only deallocated in the finalizer. This is one > suspected cause of recent TestHFileBlock failures on Hadoop QA: > https://builds.apache.org/job/HBase-TRUNK/2658/testReport/org.apache.hadoop.hbase.io.hfile/TestHFileBlock/testPreviousOffset_1_/. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira