[
https://issues.apache.org/jira/browse/HBASE-14920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15267488#comment-15267488
]
Eshcar Hillel commented on HBASE-14920:
---------------------------------------
New patch is available. Also available in RB
https://reviews.apache.org/r/45080/.
I changed the setting of the inmemory flush threshold according to what
[~anoop.hbase] have suggested.
Anoop also raised a concern that flushing the tail of the compaction pipeline
is not enough.
As I see it, a call to flush data to disk aims for reducing the memory held by
the region, and this goal is achieved. Furthermore, in most cases the largest
portion of the data resides in the tail segment in the pipeline, therefore
almost all the data will be flushed to disk. Finally this wouldn’t be the first
case where you need to flush more than once in order to completely empty the
memory - see HRegion#doClose()
bq. After an in memory flush, we will reduce some heap overhead and will reduce
that delta from memstore size. I can see a call to reduce the variable what we
keep at RS level. Another we have at every HRegion level, do we update that
also?
After an in-memory compaction is completed the memstore invokes
RegionServicesForStores#addAndGetGlobalMemstoreSize(size) which then invokes
HRegion#addAndGetGlobalMemstoreSize(size) which updates the region counter and
takes care to update the RegionServer counter.
{code}
public long addAndGetGlobalMemstoreSize(long memStoreSize) {
if (this.rsAccounting != null) {
rsAccounting.addAndGetGlobalMemstoreSize(memStoreSize);
}
return this.memstoreSize.addAndGet(memStoreSize);
}
{code}
None of the counters (RS, region, segment) are new, all of them existed before
this patch, so I fail to see the problem.
bq. No need of a region lock in this case
The region lock is *only* held while flushing the active segment into the
pipeline, and *not* during compaction
{code}
void flushInMemory() throws IOException {
// Phase I: Update the pipeline
getRegionServices().blockUpdates();
try {
MutableSegment active = getActive();
LOG.info("IN-MEMORY FLUSH: Pushing active segment into compaction pipeline,
" +
"and initiating compaction.");
pushActiveToPipeline(active);
} finally {
getRegionServices().unblockUpdates();
}
...
{code}
> Compacting Memstore
> -------------------
>
> Key: HBASE-14920
> URL: https://issues.apache.org/jira/browse/HBASE-14920
> Project: HBase
> Issue Type: Sub-task
> Reporter: Eshcar Hillel
> Assignee: Eshcar Hillel
> Attachments: HBASE-14920-V01.patch, HBASE-14920-V02.patch,
> HBASE-14920-V03.patch, HBASE-14920-V04.patch, HBASE-14920-V05.patch,
> move.to.junit4.patch
>
>
> Implementation of a new compacting memstore with non-optimized immutable
> segment representation
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
