Frank: Can you take a look at the following to see if it may help with your use case(s) ?
HBASE-15181 A simple implementation of date based tiered compaction Cheers On Fri, Mar 18, 2016 at 9:58 AM, Frank Luo <j...@merkleinc.com> wrote: > There are two reasons I am hesitating going that route. > > One is that most of tables are fairly small. Going to 10GB will force > tables to shrink to some nodes but not evenly distributed around the > cluster, hence discouraging parallelism. But I think I can manage this > issue if the second is resolved. > > The second issue, which I have battled with for two years now, is that I > am doing online puts, which occasionally triggers compacts when a region is > heavily inserted, and whenever it happens, all subsequent read/write are > all on hold and I can see time out error on the client side. A typical > compact runs for 4 minutes now and I have to increase timeout on a number > of places to accommodate that. So if I increase the size to 10 GB, will > compact time double? > > -----Original Message----- > From: Jean-Marc Spaggiari [mailto:jean-m...@spaggiari.org] > Sent: Friday, March 18, 2016 11:34 AM > To: user > Subject: Re: is it a good idea to disable tables not currently hot? > > So you can safely increase your maximum region size to 10GB, which will > divide the number of regions by 2. When you will be on 1.1.2 you can also > do online merge to reduce this number of regions. The might help too. > > JMS > > 2016-03-18 12:32 GMT-04:00 Frank Luo <j...@merkleinc.com>: > > > 0.98 on hdp 2.2 currently. > > > > Soon will be on hdp2.3.4, which has HBase 1.1.2. > > > > -----Original Message----- > > From: Jean-Marc Spaggiari [mailto:jean-m...@spaggiari.org] > > Sent: Friday, March 18, 2016 11:29 AM > > To: user > > Subject: Re: is it a good idea to disable tables not currently hot? > > > > Hi Frank, > > > > It might be doable. > > > > What HBase version are you running? > > > > JMS > > > > 2016-03-18 12:25 GMT-04:00 Frank Luo <j...@merkleinc.com>: > > > > > No one has experience disabling tables? > > > > > > -----Original Message----- > > > From: Frank Luo [mailto:j...@merkleinc.com] > > > Sent: Thursday, March 17, 2016 4:51 PM > > > To: user@hbase.apache.org > > > Subject: is it a good idea to disable tables not currently hot? > > > > > > We have a multi tenants environment and each client occupies x > > > number of hbase regions. We currently have about 500 regions per > > > region server and I understand the guideline is less than 200. So we > > > need to reduce the region counts. Increasing region file size is no > > > more an option because we are already at 5G and I don’t want to go > higher. > > > > > > Due to our unique use cases, all clients are running for a few hours > > > in a day, then being quiet for the rest of time. So I am thinking > > > whether it is a good idea to disable all quiet tables and only > > > enable them when they are ready to run. Does anyone have experience on > that? > > > > > > One thing I worry about is the Balancer. I am pretty sure the > > > balancer will be confused when regions come and go. And I cannot > > > afford not to have it running in case of region server crashes and > > > come back. So doesn’t anyone have good ideas how to handle it? > > > > > > I already doing compact myself so that is not an issue. > > > > > > Another related question, if a region is enabled but not active > > > read/write, how much resources it takes in terms of region server? > > > > > > Thanks! > > > > > > Frank Luo > > > > > > > Merkle was named a leader in Customer Insights Services Providers by > > Forrester Research < > > http://www.merkleinc.com/who-we-are-customer-relationship-marketing-ag > > ency/awards-recognition/merkle-named-leader-forrester?utm_source=email > > footer&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter > > > > > > > Forrester Research report names 500friends, a Merkle Company, a leader > > in customer Loyalty Solutions for Midsize Organizations< > > http://www.merkleinc.com/who-we-are-customer-relationship-marketing-ag > > ency/awards-recognition/500friends-merkle-company-named?utm_source=ema > > ilfooter&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter > > > > > This email and any attachments transmitted with it are intended for > > use by the intended recipient(s) only. If you have received this email > > in error, please notify the sender immediately and then delete it. If > > you are not the intended recipient, you must not keep, use, disclose, > > copy or distribute this email without the author’s prior permission. > > We take precautions to minimize the risk of transmitting software > > viruses, but we advise you to perform your own virus checks on any > > attachment to this message. We cannot accept liability for any loss or > > damage caused by software viruses. The information contained in this > > communication may be confidential and may be subject to the > attorney-client privilege. > > > Merkle was named a leader in Customer Insights Services Providers by > Forrester Research > < > http://www.merkleinc.com/who-we-are-customer-relationship-marketing-agency/awards-recognition/merkle-named-leader-forrester?utm_source=emailfooter&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter > > > > Forrester Research report names 500friends, a Merkle Company, a leader in > customer Loyalty Solutions for Midsize Organizations< > http://www.merkleinc.com/who-we-are-customer-relationship-marketing-agency/awards-recognition/500friends-merkle-company-named?utm_source=emailfooter&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter > > > This email and any attachments transmitted with it are intended for use by > the intended recipient(s) only. If you have received this email in error, > please notify the sender immediately and then delete it. If you are not the > intended recipient, you must not keep, use, disclose, copy or distribute > this email without the author’s prior permission. We take precautions to > minimize the risk of transmitting software viruses, but we advise you to > perform your own virus checks on any attachment to this message. We cannot > accept liability for any loss or damage caused by software viruses. The > information contained in this communication may be confidential and may be > subject to the attorney-client privilege. >
