Stripe compaction is a feature which has been implemented long ago in HBase but I've never seen extensive usage in the community. And recently I found that a big company in China, Meituan, has made use of stripe compaction in their production cluster. One of the team member shared some information on a github PR and she agreed that I could share it in the mailing list.
Hi, @Apache9 <https://github.com/Apache9> , it's my pleasure to share these > information. > We use StripeCompactionPolicy in almost all of our production clusters. > And we let recently data in memstore flush to L0, limit stripe size to > about 10G. Most of our regions are large than 50G, there even exist regions > larger than 2T... StripeCompactionPolicy has no major compactions, and it > can limit compactions in only L0 and one stripe files, and it can perform > cells deletion in one stripe just like the major-compaction. The pressure > of compactions is broken down. Though the total files count in a region > maybe a little larger than normal compactions, because the files are > organized as in mini-regions, it works well for most read requests. > And we also implemented a fast split and compact method based on > StripeCompactionPolicy, practiced in all our production clusters, results > show that split is very light weight and no need to perform read+write > files compactions right after split. Details are in HBASE-25302 > <https://issues.apache.org/jira/browse/HBASE-25302>, hope you have > interest... > Thanks. The original link: https://github.com/apache/hbase/pull/3152#issuecomment-824166990
