[
https://issues.apache.org/jira/browse/HADOOP-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HADOOP-1644.
---------------------------
Resolution: Fixed
Committed after testing on three different machines -- macosx and two linux
machines one of which was an old K6 single-processor -- and checking no javadoc
errors. Resolving.
> [hbase] Compactions should not block updates
> --------------------------------------------
>
> Key: HADOOP-1644
> URL: https://issues.apache.org/jira/browse/HADOOP-1644
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.15.0
> Reporter: stack
> Assignee: stack
> Fix For: 0.15.0
>
> Attachments: interlacing.patch, non-blocking-compaction-v2.patch,
> non-blocking-compaction.patch
>
>
> Currently, compactions take a long time. During compaction, updates are
> carried by the HRegions' memcache (+ backing HLog). memcache is unable to
> flush to disk until compaction completes.
> Under sustained, substantial -- rows that contain multiple columns one of
> which is a web page -- updates by multiple concurrent clients (10 in this
> case), a common hbase usage scenario, the memcache grows fast and often to
> orders of magnitude in excess of the configured 'flush-to-disk' threshold.
> This throws the whole system out of kilter. When memcache does get to run
> after compaction completes -- assuming you have sufficent RAM and the region
> server doesn't OOME -- then the resulting on-disk file will be way larger
> than any other on-disk HStoreFile bringing on a region split ..... but the
> resulting split will produce regions that themselves need to be immediately
> split because each half is beyond the configured limit, and so on...
> In another issue yet to be posted, tuning and some pointed memcache flushes
> makes the above condition less extreme but until compaction durations come
> close to the memcache flush threshold compactions will remain disruptive.
> Its allowed that compactions may never be fast enough as per bigtable paper
> (This is a 'wish' issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
