Merge flush/merging compaction; save on i/o by merging an existing file with
content memstore
---------------------------------------------------------------------------------------------
Key: HBASE-5476
URL: https://issues.apache.org/jira/browse/HBASE-5476
Project: HBase
Issue Type: Improvement
Reporter: stack
Compactions are slow. To improve, we can work on making compactions faster
(TODO) or make it so compactions have less work to do. This issue is about the
later; doing something the bigtable paper talks off where the content of memory
is merged with existing content in the filesystem (I'm sure we've discussed
this a bunch in the past but can't find an explicit issue for it).
We save on the reading of the size of a memstore from the filesystem if we
merge flush/merge compact.
BT seems to include memstore content in minor compactions. We should look at
doing that (would snapshot at start of compaction and then would integrate the
snapshot into resultant compacted file). Alternatively we could flush into a
small file that is already in the filesystem (this could be tougher given that
we have this flush-compaction separation at the moment... how would we make it
so the file we're merging into is not picked up for a normal compaction).
Doing the merge compaction/flush merge would slow down flushes. It could back
up memory into the global barrier such that we stop taking on writes altogether.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
