Re: Optimizations for a Read-only database

2017-03-17 Thread jeff saremi
Thanks Anoop. We'll be sure to keep it below 80% From: Anoop John Sent: Friday, March 17, 2017 12:49:30 PM To: user@hbase.apache.org Subject: Re: Optimizations for a Read-only database >From HBase server perspective we need restrict memstore size + blo

Re: Optimizations for a Read-only database

2017-03-17 Thread Anoop John
. Thanks a lot > > > From: Kevin O'Dell > Sent: Friday, March 17, 2017 10:55:49 AM > To: user@hbase.apache.org > Subject: Re: Optimizations for a Read-only database > > Hi Jeff, > > You can definitely lower the memstore, the las

Re: Optimizations for a Read-only database

2017-03-17 Thread jeff saremi
I'll go through these recommendations, Kevin. Thanks a lot From: Kevin O'Dell Sent: Friday, March 17, 2017 10:55:49 AM To: user@hbase.apache.org Subject: Re: Optimizations for a Read-only database Hi Jeff, You can definitely lower the memstore, the

Re: Optimizations for a Read-only database

2017-03-17 Thread Kevin O'Dell
Sorry for the Spam, As a follow up, if you are using large heaps with G1GC(~100GB) you can set memstore to .1 and block cache to .8, though there might be a barrier for how much you are allowed to set the systems to use. Leaving 10GB for regular HBase operations is MORE than enough though. On Fr

Re: Optimizations for a Read-only database

2017-03-17 Thread Kevin O'Dell
Hi Jeff, You can definitely lower the memstore, the last time I looked there it had to be set to .1 at lowest it could go. I would not recommend disabling compactions ever, bad things will occur and it can end up impacting your read performance greatly. I would recommend looking at the Intel G1

Re: Optimizations for a Read-only database

2017-03-17 Thread ramkrishna vasudevan
Hi Default memstore is 0.42 of the heap size. You can just make it 0.1 and rest give to block cache. It will ensure that you have maximum data in the block cache. It is better for reads. In case of latest trunk you can even use bucket cache in offheap mode but as it is not available as a release

Optimizations for a Read-only database

2017-03-17 Thread jeff saremi
We're creating a readonly database and would like to know the recommended optimizations we could do. We'd be loading data via direct write to HFiles. One thing i could immediately think of is to eliminate the memory for Memstore. What is the minimum that we could get away with? How about disabl