Juhani: You can also consider https://issues.apache.org/jira/browse/HBASE-1537 which is not in hbase-0.89.20100924+28<http://archive.cloudera.com/cdh/3/hbase-0.89.20100924+28/> You can apply Andrew's patch yourself.
On Thu, Dec 9, 2010 at 10:29 AM, Jean-Daniel Cryans <[email protected]>wrote: > > Regarding using a lot of families... They are currently partitioned in a > > manner that reflects the various data groups that are likely to be read > > together... We're doing a lot of big scans on the regions of only one of > > those families, with scans of the full table being much shorter/rarer. By > > having separate store files I was hoping this separation would result in > > less overhead from not reading data that we simply don't need(stuff from > the > > other families). Is the overhead from splitting the store files up large > > enough to make any savings on file access times not worth it? Or am I > > missing something else? > > Well I'm missing a lot of information about your particular use case, > so there's no possible way for me to tell whether the overhead will be > bigger than if you used only one family in your specific case. > > So what I can tell you is that in general more families is less > efficient in HBase. A region with 50 families works like 50 regions, > except that HBase doesn't handle that very well (for example, flush > size is calculated as the sum of all the families). What I mean is > that having 50 actual regions would be a lot better. > > Also, when talking about "overhead", it's usually better to have some > data points to be able to compare solutions. Have you tried your few > use cases on a few different designs? Any benchmarking? > > Hope that helps, > > J-D >
