It is great to hear this is still in use in a large production and getting improvement!
On Tue, Apr 27, 2021 at 3:01 AM 张铎(Duo Zhang) <[email protected]> wrote: > Stripe compaction is a feature which has been implemented long ago in HBase > but I've never seen extensive usage in the community. And recently I found > that a big company in China, Meituan, has made use of stripe compaction in > their production cluster. One of the team member shared some information on > a github PR and she agreed that I could share it in the mailing list. > > Hi, @Apache9 <https://github.com/Apache9> , it's my pleasure to share > these > > information. > > We use StripeCompactionPolicy in almost all of our production clusters. > > And we let recently data in memstore flush to L0, limit stripe size to > > about 10G. Most of our regions are large than 50G, there even exist > regions > > larger than 2T... StripeCompactionPolicy has no major compactions, and it > > can limit compactions in only L0 and one stripe files, and it can perform > > cells deletion in one stripe just like the major-compaction. The pressure > > of compactions is broken down. Though the total files count in a region > > maybe a little larger than normal compactions, because the files are > > organized as in mini-regions, it works well for most read requests. > > And we also implemented a fast split and compact method based on > > StripeCompactionPolicy, practiced in all our production clusters, results > > show that split is very light weight and no need to perform read+write > > files compactions right after split. Details are in HBASE-25302 > > <https://issues.apache.org/jira/browse/HBASE-25302>, hope you have > > interest... > > Thanks. > > > The original link: > https://github.com/apache/hbase/pull/3152#issuecomment-824166990 >
