On Fri, Oct 7, 2011 at 10:47 AM, David McNelis <[email protected]>wrote:
> So at the end of the day its going to be limited by available memory then? > Going by this line: > > Do note that a minimum of 1MB per memtable is used by the per-memtable > arena allocator <https://issues.apache.org/jira/browse/CASSANDRA-2252> also > introduced in 1.0, which is worth keeping in mind if you are looking at > going from thousands to tens of thousands of ColumnFamilies. > > Then you'd be looking at a requirement of 1gig memory for each 1024 > column families had on your cluster, regardless of number of keyspaces those > lived in. So in order to have 10s of thousands of CFs, one would need 10s > of gigs of ram on each node just to handle that overhead...at least as of > v1. > > On Fri, Oct 7, 2011 at 9:40 AM, Jonathan Ellis <[email protected]> wrote: > >> On Fri, Oct 7, 2011 at 9:36 AM, David McNelis >> <[email protected]> wrote: >> > In some documentation I've read it says that >> > keyspace's take up the majority of the resources >> >> This has never been the case. >> >> > in a couple of older >> > threads they talked about getting the number of column families down. >> >> This was good advice pre-0.8. >> >> I covered the state of 0.8 and 1.0 here: >> >> http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-improved-memory-and-disk-space-management >> >> -- >> Jonathan Ellis >> Project Chair, Apache Cassandra >> co-founder of DataStax, the source for professional Cassandra support >> http://www.datastax.com >> > > > > -- > *David McNelis* > Lead Software Engineer > Agentis Energy > www.agentisenergy.com > o: 630.359.6395 > c: 219.384.5143 > > *A Smart Grid technology company focused on helping consumers of energy > control an often under-managed resource.* > > > You do not want to do multiple keyspaces because a Cassandra client needs to do an RPC call to change keyspace. There will be no effective way to keep a connect pool warm for each keyspace. Imagine you if 1000 keyspaces and 1000 servers. That is a lot of connections. This shortcoming of having to make an RPC to change keyspace was my motivation for suggesting: https://issues.apache.org/jira/browse/CASSANDRA-3130
