[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353979#comment-16353979 ]
Edward Bortnikov commented on HBASE-18294: ------------------------------------------ [~eshcar], [~anoop.hbase], [~ram_krish], maybe we can take the following approach to your dispute on RB. The code makes the following config variable definitions: {code:java} public static final String HREGION_MEMSTORE_FLUSH_SIZE = "hbase.hregion.memstore.flush.size"; public static final String HREGION_MEMSTORE_OFFHEAP_FLUSH_SIZE = "hbase.hregion.memstore.offheap.flush.size";{code} The former is the legacy flush size threshold, whereas the latter is new. However, the further treatment is different - HREGION_MEMORY_FLUSH_SIZE is actually treated as *on-heap* threshold. This is confusing I guess - especially for admins. Having said that, we do need separate accounting for on-heap and off-heap memory, as [~eshcar] explained above. Let me suggest a change that is more digestible for users imo. Let HREGION_MEMORY_FLUSH_SIZE retain its legacy meaning (and the 128M default) - namely, the overall max memory the system is willing to allocate for a store. Furthermore, let's define a new variable, HREGION_MEMSTORE_OFFHEAP_SIZE_RATIO, to define a fraction of the former that can be allocated offheap (0, by default). How about that? > Reduce global heap pressure: flush based on heap occupancy > ---------------------------------------------------------- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement > Affects Versions: 3.0.0 > Reporter: Eshcar Hillel > Assignee: Eshcar Hillel > Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch, > HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)