[
https://issues.apache.org/jira/browse/HBASE-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206287#comment-13206287
]
Zhihong Yu edited comment on HBASE-5387 at 2/11/12 10:32 PM:
-------------------------------------------------------------
We could also do something heuristically. Get a random float and if it is <
0.001 (for example, every 1000's time on average) throw away the old BAOS and
get a new one.
Can the BAOS really get > the blocksize?
Also what happens when somebody (foolishly, but it does happen) sets the block
size to 512mb, in that case we might want to free this BOAS every single time.
was (Author: lhofhansl):
We could also do something heuristically. Get a random float and if it is <
0.001 (for example, every 1000's time on average) throw away the old BOAS and
get a new one.
Can the BOAS really get > the the blocksize?
Also what happens when somebody (foolishly, but it does happen) sets the block
size to 512mb, in that case we might want to free this BOAS every single time.
> Reuse compression streams in HFileBlock.Writer
> ----------------------------------------------
>
> Key: HBASE-5387
> URL: https://issues.apache.org/jira/browse/HBASE-5387
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.0
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Critical
> Fix For: 0.94.0
>
> Attachments: D1719.1.patch,
> Fix-deflater-leak-2012-02-10_18_48_45.patch
>
>
> We need to to reuse compression streams in HFileBlock.Writer instead of
> allocating them every time. The motivation is that when using Java's built-in
> implementation of Gzip, we allocate a new GZIPOutputStream object and an
> associated native data structure every time we create a compression stream.
> The native data structure is only deallocated in the finalizer. This is one
> suspected cause of recent TestHFileBlock failures on Hadoop QA:
> https://builds.apache.org/job/HBase-TRUNK/2658/testReport/org.apache.hadoop.hbase.io.hfile/TestHFileBlock/testPreviousOffset_1_/.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
