[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16304679#comment-16304679
]
stack commented on HBASE-18294:
-------------------------------
bq. But change the meaning of heapSize itself....
Yeah, this is an issue. We are saying that in the offheap case, heapsize means
onheap overhead + offheap datasize. Yeah, this is a mangling of the original
understanding. Needs qualification at least.
bq. IMHO region flush size means when the data size is this much do a flush.
That'd be best but then there is the overhead. Perhaps we could make it so
overhead is tracked in background and then flush threshold serves as
memory-usage barrier and as desired file-size indicator but we've not done the
work. Ignoring overhead at the region scope will make us to run into the global
barrier especially if key is non-negligible in size (the global barrier does
track overhead); going to the global barrier is a perf killer.
bq. Any way I dont want to be a blocker here. So I would move aside now
You ain't a blocker. We need your expertise in here sir. What you think of the
[~eshcar] change? Is that the right way to do accounting of the offheap size?
Thanks.
> Reduce global heap pressure: flush based on heap occupancy
> ----------------------------------------------------------
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 3.0.0
> Reporter: Eshcar Hillel
> Assignee: Eshcar Hillel
> Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch,
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch,
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch,
> HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch,
> HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch,
> HBASE-18294.13.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the
> store to another threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size
> (key-value only) to the threshold where it should compare the heap size
> (which includes index size, and metadata).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
