Re: Loading data on-demand in Cassandra
Pierre Chalamet chalamet.net> writes: > > Hi,You do not need to have 700 Go of data in RAM. Cassandra is able to store on disks and query from there if data is not cached in memory. Caches are maintained by C* by itself but you still have to some configuration.Supposing you want to store around 800 Go and with a RF=3, you will need at least 6 servers if you want to store all data of your db (keeping max 400 Go per server) : 800x3/400=6.There is no native implementation of trigger in C*. Anyway, there is an extension bringing this feature: https://github.com/hmsonline/cassandra-triggers. This should allow you to be notified of mutations (ie: not query). Some peoples on this ML are involved in this, maybe they could help on this.Cheers,- Pierre > From: Oliver Plohmann objectscape.org> > > Date: Sun, 12 Aug 2012 21:24:43 +0200 > To: cassandra.apache.org> > ReplyTo: user cassandra.apache.org > > Subject: Loading data on-demand in Cassandra > > Hello, > I'm looking a bit into Cassandra to > see whether it would be something to go with for my company. I > searched through the Internet, looked through the FAQs, etc. but > there are still some few open questions. Hope I don't bother > anybody with the usual beginner questions ... > Is there a way to do load-on-demand > of data in Cassandra? For the time being, we cannot afford to > built up a cluster that holds our 700 GB SQL-Database in RAM. So > we need to be able to load data on-demand from our relational > database. Can this be done in Cassandra? Then there also needs to > be a way to unload data in order to reclaim RAM space. Would be > nice if it were possible to register for an asynchronous > notification in case some value was changed. Can this be done? > Thanks for any answers. > Regards, Oliver > I would consider looking into distributed caching technology (ehcache, gemfire)
Re: Loading data on-demand in Cassandra
Hi, You do not need to have 700 Go of data in RAM. Cassandra is able to store on disks and query from there if data is not cached in memory. Caches are maintained by C* by itself but you still have to some configuration. Supposing you want to store around 800 Go and with a RF=3, you will need at least 6 servers if you want to store all data of your db (keeping max 400 Go per server) : 800x3/400=6. There is no native implementation of trigger in C*. Anyway, there is an extension bringing this feature: https://github.com/hmsonline/cassandra-triggers. This should allow you to be notified of mutations (ie: not query). Some peoples on this ML are involved in this, maybe they could help on this. Cheers, - Pierre -Original Message- From: Oliver Plohmann Date: Sun, 12 Aug 2012 21:24:43 To: Reply-To: user@cassandra.apache.org Subject: Loading data on-demand in Cassandra Hello, I'm looking a bit into Cassandra to see whether it would be something to go with for my company. I searched through the Internet, looked through the FAQs, etc. but there are still some few open questions. Hope I don't bother anybody with the usual beginner questions ... Is there a way to do load-on-demand of data in Cassandra? For the time being, we cannot afford to built up a cluster that holds our 700 GB SQL-Database in RAM. So we need to be able to load data on-demand from our relational database. Can this be done in Cassandra? Then there also needs to be a way to unload data in order to reclaim RAM space. Would be nice if it were possible to register for an asynchronous notification in case some value was changed. Can this be done? Thanks for any answers. Regards, Oliver
Re: Loading data on-demand in Cassandra
Thanks Dave. Does anybody know of a distributed in-memory system that can do this and that supports structured data (e.g. tables)? /Oliver Am 12.08.2012 um 21:39 schrieb Dave Brosius : > When data is first written it does remain in memory until that memory is > flushed. After the data is only on disk, it remains there until a read for > that row-key/column is requested so in essense it's always load on demand. > > Currently there is no support for async notifications of changes. > > > > On 08/12/2012 03:24 PM, Oliver Plohmann wrote: >> >> Hello, >> >> I'm looking a bit into Cassandra to see whether it would be something to go >> with for my company. I searched through the Internet, looked through the >> FAQs, etc. but there are still some few open questions. Hope I don't bother >> anybody with the usual beginner questions ... >> >> Is there a way to do load-on-demand of data in Cassandra? For the time >> being, we cannot afford to built up a cluster that holds our 700 GB >> SQL-Database in RAM. So we need to be able to load data on-demand from our >> relational database. Can this be done in Cassandra? Then there also needs to >> be a way to unload data in order to reclaim RAM space. Would be nice if it >> were possible to register for an asynchronous notification in case some >> value was changed. Can this be done? >> >> Thanks for any answers. >> Regards, Oliver >> >
Re: Loading data on-demand in Cassandra
When data is first written it does remain in memory until that memory is flushed. After the data is only on disk, it remains there until a read for that row-key/column is requested so in essense it's always load on demand. Currently there is no support for async notifications of changes. On 08/12/2012 03:24 PM, Oliver Plohmann wrote: Hello, I'm looking a bit into Cassandra to see whether it would be something to go with for my company. I searched through the Internet, looked through the FAQs, etc. but there are still some few open questions. Hope I don't bother anybody with the usual beginner questions ... Is there a way to do load-on-demand of data in Cassandra? For the time being, we cannot afford to built up a cluster that holds our 700 GB SQL-Database in RAM. So we need to be able to load data on-demand from our relational database. Can this be done in Cassandra? Then there also needs to be a way to unload data in order to reclaim RAM space. Would be nice if it were possible to register for an asynchronous notification in case some value was changed. Can this be done? Thanks for any answers. Regards, Oliver
Loading data on-demand in Cassandra
Hello, I'm looking a bit into Cassandra to see whether it would be something to go with for my company. I searched through the Internet, looked through the FAQs, etc. but there are still some few open questions. Hope I don't bother anybody with the usual beginner questions ... Is there a way to do load-on-demand of data in Cassandra? For the time being, we cannot afford to built up a cluster that holds our 700 GB SQL-Database in RAM. So we need to be able to load data on-demand from our relational database. Can this be done in Cassandra? Then there also needs to be a way to unload data in order to reclaim RAM space. Would be nice if it were possible to register for an asynchronous notification in case some value was changed. Can this be done? Thanks for any answers. Regards, Oliver
