Re:Re: maybe waste on blockCache

2016-06-16 Thread WangYQ
I set all user tables with blockCache on, but set the IN_MEMORY conf to false At 2016-06-16 18:18:44, "Heng Chen" wrote: >bq. if we do not set any user tables IN_MEMORY to true, then the whole >hbase just need to cache hbase:meta data to in_memory

Re: HBase number of columns

2016-06-16 Thread Saad Mufti
There is no real column schema in HBase other than defining the column family, each write to a column writes a cell with the column name plus value, so in theory number of columns doesn't really matter. What matters is how much data you read and write. That said there are settings in the column

RE: Writing visibility labels with HFileOutputFormat2

2016-06-16 Thread Ellis, Tom (Financial Markets IT)
Hi Again Ram, "hbase.security.visibility.mutations.checkauths" - for now the method of set_auths 'client','system' along with only giving 'client' read on 'hbase:labels' is working for me. "Coming to reading the HFile and creating a bulk load, I think we should be more cautious here " - I

Re: maybe waste on blockCache

2016-06-16 Thread Heng Chen
bq. if we do not set any user tables IN_MEMORY to true, then the whole hbase just need to cache hbase:meta data to in_memory LruBlockCache. You set blockcache to be false for other tables? 2016-06-16 16:21 GMT+08:00 WangYQ : > in hbase 0.98.10, if we use

maybe waste on blockCache

2016-06-16 Thread WangYQ
in hbase 0.98.10, if we use LruBlockCache, and set regionServer's max heap to 10G in default: the size of in_memory priority of LruBlockCache is : 10G * 0.4 * 0.25 = 1G 0.4: hfile.block.cache.size 0.25: hbase.lru.blockcache.memory.percentage if we do not set any user tables IN_MEMORY to