Have you considered below options to reduce number/time of compactions? - Reduce memstore flushes to create fewer hfiles there by reducing number of compactions. By setting hbase.hregion.memstore.flush.size and other parameters. - hbase.offpeak.start & end , so that compactions triggered during off peak time - disabling major compaction and running manually on days with low load OR delay major compactions using *hbase.hregion.majorcompaction* - Checkout date tiered compaction, which doesn't compact older hfiles https://issues.apache.org/jira/browse/HBASE-15337
On Tue, Sep 18, 2018 at 6:53 AM Ted Yu <yuzhih...@gmail.com> wrote: > Hi, > To my knowledge, stripe compaction has not seen patches for a few years. > > Have you looked at : > http://hbase.apache.org/book.html#ops.date.tiered > > If the above doesn't suit your needs, can you tell us more about your use > case ? > > Thanks > > On Mon, Sep 17, 2018 at 11:39 AM Austin Heyne <ahe...@ccri.com> wrote: > > > The HBase cluster we're running has well over 100TB of data spread > > across two tables and as you'd guess we're suffering from compaction > > times taking way to long (and we need to double or triple that volume). > > I've found information on Stripe Compactions [1] which it seems like we > > could benefit a lot from, however, 'experimental' is a scary word for > > productions systems so I just wanted to get a general sentiment around > > the stability of the feature. Any experience or input would be very > > helpful. > > > > Thanks again, > > Austin > > > > [1] https://hbase.apache.org/book.html#ops.stripe > > > > >
