Re: Multiple keyspace question
A problem of many keyspaces is clients are bound to a keyspace so connection pooling multiple keyspaces is an issue. Cql has support for some limited cross keyspace operations. On Sunday, July 8, 2012, aaron morton aa...@thelastpickle.com wrote: I would do a test to see the latency difference under load between having 1 KS with 5 CF's and 50 KS with 5 CF's. Your test will need to read and write to all the CF's. Having many CF's may result in more frequent memtables flushes. (Personally it's not an approach I would take.) Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 7/07/2012, at 8:15 AM, Shahryar Sedghi wrote: Aaron I am going to have many (over 50 eventually) keyspaces with limited number of CFs (5-6) do you think this one can cause a problem too. Thanks On Fri, Jul 6, 2012 at 2:28 PM, aaron morton aa...@thelastpickle.com wrote: Also, all CF's in the same KS share one commit log. So all writes for the row row key, across all CF's, are committed at the same time. Some other settings, such as caches in 1.1, are machine wide. If you have a small KS for something like app config, I'd say go with whatever feels right. If you are talking about two full application KS's I would think about their prospective workloads and growth patterns. Will you always want to manage the two together ? Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 6/07/2012, at 9:47 PM, Robin Verlangen wrote: Hi Ben, The amount of keyspaces is not the problem: the amount of column families is. Each column family adds a certain amount of memory usage to the system. You can cope with this by adding memory or using generic column families that store different types of data. With kind regards, Robin Verlangen Software engineer W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies. 2012/7/6 Ben Kaehne ben.kae...@sirca.org.au Good evening, I have read multiple keyspaces are bad before in a few discussions, but to what extent? We have some reasonably powerful machines and looking to host an additional (currently we have 1) 2 keyspaces within our cassandra cluster (of 3 nodes, using RF3). At what point does adding extra keyspaces start becoming an issue? Is there anything special we should be considering or watching out for as we implement this? I could not imagine that all cassandra users out there are running one massive keyspace, and at the same time can not imaging that all cassandra users have multiple clusters just to host different keyspaces. Regards. -- -Ben
Re: Multiple keyspace question
I would do a test to see the latency difference under load between having 1 KS with 5 CF's and 50 KS with 5 CF's. Your test will need to read and write to all the CF's. Having many CF's may result in more frequent memtables flushes. (Personally it's not an approach I would take.) Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 7/07/2012, at 8:15 AM, Shahryar Sedghi wrote: Aaron I am going to have many (over 50 eventually) keyspaces with limited number of CFs (5-6) do you think this one can cause a problem too. Thanks On Fri, Jul 6, 2012 at 2:28 PM, aaron morton aa...@thelastpickle.com wrote: Also, all CF's in the same KS share one commit log. So all writes for the row row key, across all CF's, are committed at the same time. Some other settings, such as caches in 1.1, are machine wide. If you have a small KS for something like app config, I'd say go with whatever feels right. If you are talking about two full application KS's I would think about their prospective workloads and growth patterns. Will you always want to manage the two together ? Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 6/07/2012, at 9:47 PM, Robin Verlangen wrote: Hi Ben, The amount of keyspaces is not the problem: the amount of column families is. Each column family adds a certain amount of memory usage to the system. You can cope with this by adding memory or using generic column families that store different types of data. With kind regards, Robin Verlangen Software engineer W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies. 2012/7/6 Ben Kaehne ben.kae...@sirca.org.au Good evening, I have read multiple keyspaces are bad before in a few discussions, but to what extent? We have some reasonably powerful machines and looking to host an additional (currently we have 1) 2 keyspaces within our cassandra cluster (of 3 nodes, using RF3). At what point does adding extra keyspaces start becoming an issue? Is there anything special we should be considering or watching out for as we implement this? I could not imagine that all cassandra users out there are running one massive keyspace, and at the same time can not imaging that all cassandra users have multiple clusters just to host different keyspaces. Regards. -- -Ben -- Life is what happens while you are making other plans. ~ John Lennon
Re: Multiple keyspace question
Hi Ben, The amount of keyspaces is not the problem: the amount of column families is. Each column family adds a certain amount of memory usage to the system. You can cope with this by adding memory or using generic column families that store different types of data. With kind regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies. 2012/7/6 Ben Kaehne ben.kae...@sirca.org.au Good evening, I have read multiple keyspaces are bad before in a few discussions, but to what extent? We have some reasonably powerful machines and looking to host an additional (currently we have 1) 2 keyspaces within our cassandra cluster (of 3 nodes, using RF3). At what point does adding extra keyspaces start becoming an issue? Is there anything special we should be considering or watching out for as we implement this? I could not imagine that all cassandra users out there are running one massive keyspace, and at the same time can not imaging that all cassandra users have multiple clusters just to host different keyspaces. Regards. -- -Ben
Re: Multiple keyspace question
Also, all CF's in the same KS share one commit log. So all writes for the row row key, across all CF's, are committed at the same time. Some other settings, such as caches in 1.1, are machine wide. If you have a small KS for something like app config, I'd say go with whatever feels right. If you are talking about two full application KS's I would think about their prospective workloads and growth patterns. Will you always want to manage the two together ? Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 6/07/2012, at 9:47 PM, Robin Verlangen wrote: Hi Ben, The amount of keyspaces is not the problem: the amount of column families is. Each column family adds a certain amount of memory usage to the system. You can cope with this by adding memory or using generic column families that store different types of data. With kind regards, Robin Verlangen Software engineer W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies. 2012/7/6 Ben Kaehne ben.kae...@sirca.org.au Good evening, I have read multiple keyspaces are bad before in a few discussions, but to what extent? We have some reasonably powerful machines and looking to host an additional (currently we have 1) 2 keyspaces within our cassandra cluster (of 3 nodes, using RF3). At what point does adding extra keyspaces start becoming an issue? Is there anything special we should be considering or watching out for as we implement this? I could not imagine that all cassandra users out there are running one massive keyspace, and at the same time can not imaging that all cassandra users have multiple clusters just to host different keyspaces. Regards. -- -Ben
Re: Multiple keyspace question
Aaron I am going to have many (over 50 eventually) keyspaces with limited number of CFs (5-6) do you think this one can cause a problem too. Thanks On Fri, Jul 6, 2012 at 2:28 PM, aaron morton aa...@thelastpickle.comwrote: Also, all CF's in the same KS share one commit log. So all writes for the row row key, across all CF's, are committed at the same time. Some other settings, such as caches in 1.1, are machine wide. If you have a small KS for something like app config, I'd say go with whatever feels right. If you are talking about two full application KS's I would think about their prospective workloads and growth patterns. Will you always want to manage the two together ? Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 6/07/2012, at 9:47 PM, Robin Verlangen wrote: Hi Ben, The amount of keyspaces is not the problem: the amount of column families is. Each column family adds a certain amount of memory usage to the system. You can cope with this by adding memory or using generic column families that store different types of data. With kind regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies. 2012/7/6 Ben Kaehne ben.kae...@sirca.org.au Good evening, I have read multiple keyspaces are bad before in a few discussions, but to what extent? We have some reasonably powerful machines and looking to host an additional (currently we have 1) 2 keyspaces within our cassandra cluster (of 3 nodes, using RF3). At what point does adding extra keyspaces start becoming an issue? Is there anything special we should be considering or watching out for as we implement this? I could not imagine that all cassandra users out there are running one massive keyspace, and at the same time can not imaging that all cassandra users have multiple clusters just to host different keyspaces. Regards. -- -Ben -- Life is what happens while you are making other plans. ~ John Lennon
