There are two reasons I am hesitating going that route. One is that most of tables are fairly small. Going to 10GB will force tables to shrink to some nodes but not evenly distributed around the cluster, hence discouraging parallelism. But I think I can manage this issue if the second is resolved.
The second issue, which I have battled with for two years now, is that I am doing online puts, which occasionally triggers compacts when a region is heavily inserted, and whenever it happens, all subsequent read/write are all on hold and I can see time out error on the client side. A typical compact runs for 4 minutes now and I have to increase timeout on a number of places to accommodate that. So if I increase the size to 10 GB, will compact time double? -----Original Message----- From: Jean-Marc Spaggiari [mailto:jean-m...@spaggiari.org] Sent: Friday, March 18, 2016 11:34 AM To: user Subject: Re: is it a good idea to disable tables not currently hot? So you can safely increase your maximum region size to 10GB, which will divide the number of regions by 2. When you will be on 1.1.2 you can also do online merge to reduce this number of regions. The might help too. JMS 2016-03-18 12:32 GMT-04:00 Frank Luo <j...@merkleinc.com>: > 0.98 on hdp 2.2 currently. > > Soon will be on hdp2.3.4, which has HBase 1.1.2. > > -----Original Message----- > From: Jean-Marc Spaggiari [mailto:jean-m...@spaggiari.org] > Sent: Friday, March 18, 2016 11:29 AM > To: user > Subject: Re: is it a good idea to disable tables not currently hot? > > Hi Frank, > > It might be doable. > > What HBase version are you running? > > JMS > > 2016-03-18 12:25 GMT-04:00 Frank Luo <j...@merkleinc.com>: > > > No one has experience disabling tables? > > > > -----Original Message----- > > From: Frank Luo [mailto:j...@merkleinc.com] > > Sent: Thursday, March 17, 2016 4:51 PM > > To: user@hbase.apache.org > > Subject: is it a good idea to disable tables not currently hot? > > > > We have a multi tenants environment and each client occupies x > > number of hbase regions. We currently have about 500 regions per > > region server and I understand the guideline is less than 200. So we > > need to reduce the region counts. Increasing region file size is no > > more an option because we are already at 5G and I don’t want to go higher. > > > > Due to our unique use cases, all clients are running for a few hours > > in a day, then being quiet for the rest of time. So I am thinking > > whether it is a good idea to disable all quiet tables and only > > enable them when they are ready to run. Does anyone have experience on that? > > > > One thing I worry about is the Balancer. I am pretty sure the > > balancer will be confused when regions come and go. And I cannot > > afford not to have it running in case of region server crashes and > > come back. So doesn’t anyone have good ideas how to handle it? > > > > I already doing compact myself so that is not an issue. > > > > Another related question, if a region is enabled but not active > > read/write, how much resources it takes in terms of region server? > > > > Thanks! > > > > Frank Luo > > > > Merkle was named a leader in Customer Insights Services Providers by > Forrester Research < > http://www.merkleinc.com/who-we-are-customer-relationship-marketing-ag > ency/awards-recognition/merkle-named-leader-forrester?utm_source=email > footer&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter > > > > Forrester Research report names 500friends, a Merkle Company, a leader > in customer Loyalty Solutions for Midsize Organizations< > http://www.merkleinc.com/who-we-are-customer-relationship-marketing-ag > ency/awards-recognition/500friends-merkle-company-named?utm_source=ema > ilfooter&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter > > > This email and any attachments transmitted with it are intended for > use by the intended recipient(s) only. If you have received this email > in error, please notify the sender immediately and then delete it. If > you are not the intended recipient, you must not keep, use, disclose, > copy or distribute this email without the author’s prior permission. > We take precautions to minimize the risk of transmitting software > viruses, but we advise you to perform your own virus checks on any > attachment to this message. We cannot accept liability for any loss or > damage caused by software viruses. The information contained in this > communication may be confidential and may be subject to the attorney-client > privilege. > Merkle was named a leader in Customer Insights Services Providers by Forrester Research <http://www.merkleinc.com/who-we-are-customer-relationship-marketing-agency/awards-recognition/merkle-named-leader-forrester?utm_source=emailfooter&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter> Forrester Research report names 500friends, a Merkle Company, a leader in customer Loyalty Solutions for Midsize Organizations<http://www.merkleinc.com/who-we-are-customer-relationship-marketing-agency/awards-recognition/500friends-merkle-company-named?utm_source=emailfooter&utm_medium=email&utm_campaign=2016MonthlyEmployeeFooter> This email and any attachments transmitted with it are intended for use by the intended recipient(s) only. If you have received this email in error, please notify the sender immediately and then delete it. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this email without the author’s prior permission. We take precautions to minimize the risk of transmitting software viruses, but we advise you to perform your own virus checks on any attachment to this message. We cannot accept liability for any loss or damage caused by software viruses. The information contained in this communication may be confidential and may be subject to the attorney-client privilege.
