Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions
On Mon, Oct 28, 2013 at 5:46 PM, Qing He qing...@radisys.com wrote: In my hard drive-less use case, I need an in-core-db/cache that can be in the same db cluster with real db (with hard drive) with the same sql api so that the current openstack code do not need to be changed, instead, just a pluggin with some configurations. This is pretty much the original use case that dogpile.cache grew out of, see: http://docs.sqlalchemy.org/en/rel_0_9/orm/examples.html#dogpile-caching -Original Message- From: Morgan Fainberg [mailto:m...@metacloud.com] Sent: Monday, October 28, 2013 10:22 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions In light of what Dolph said with regards to Keystone, we are using dogpile.cache to implement memoization in front of our driver calls. It it has the ability to cache directly as well, but it has been effective (so far) for our use-case. That being said, I am unsure if caching in front of MySQL is really what we want. I believe that we should be caching after processing work (hence memoization mechanism) instead of at the SQL layer. This also means we can be measured in what we cache (oh hey, it makes no sense to cache X because it needs to be real time or there isn't a performance issue with that query / call, but Y does a ton of processing and is an expensive join/temptable query). In my experience, unless the whole application is designed with caching in mind, caching something as broad as MySQL calls (or any SQL store) is likely going to net exactly what Shawn Hartsock stated, adding a second performance issue. --Morgan On Mon, Oct 28, 2013 at 10:05 AM, Shawn Hartsock hartso...@vmware.com wrote: I once heard a quote.. I had a performance problem, so I added caching. now I have two performance problems. this. 1,000 times this. Just to float this thought ... make sure it's considered... I've seen a *lot* of people misuse caching when what the really want is memoization. * http://stackoverflow.com/questions/1988804/what-is-memoization-and-how -can-i-use-it-in-python * http://stackoverflow.com/questions/10879137/how-can-i-memoize-a-class- instantiation-in-python ... I'm not sure what you're trying to do. So YMMV, TTFN, BBQ. # Shawn Hartsock - Original Message - From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org Sent: Monday, October 28, 2013 12:12:49 PM Subject: Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions Excerpts from Dolph Mathews's message of 2013-10-28 08:40:19 -0700: It's not specific to mysql (or sql at all), but keystone is using dogpile.cache around driver calls to a similar effect. http://dogpilecache.readthedocs.org/en/latest/ It can persist to memcache, redis, etc. I once heard a quote.. I had a performance problem, so I added caching. now I have two performance problems. Caching is unbelievably awesome in the jobs it can do well. When the problem is straight forward and the requirements are few, it is just the right thing to relieve engineering pressure to make an application more scalable. However, IMO, more than narrow, well defined cache usage is a sign that the application needs some reworking to scale. I like the principle of let's use dogpile so we don't have to reinvent multi-level caching. However, let's make sure we look at each performance issue individually, rather than just throwing them all in a cache box and waving the memcache wand. https://github.com/openstack/keystone/blob/master/keystone/common/c ache/core.py On Fri, Oct 25, 2013 at 6:53 PM, Qing He qing...@radisys.com wrote: All, Has anyone looked at the options of putting a distributed caching system in front of mysql server to improve performance? This should be similar to Oracle Coherence, or VMware VFabric SQLFire. ** ** Thanks, ** ** Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions
Dolph, Thanks! It is good that python example is provided! Qing From: Dolph Mathews [mailto:dolph.math...@gmail.com] Sent: Thursday, October 31, 2013 9:25 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions On Mon, Oct 28, 2013 at 5:46 PM, Qing He qing...@radisys.commailto:qing...@radisys.com wrote: In my hard drive-less use case, I need an in-core-db/cache that can be in the same db cluster with real db (with hard drive) with the same sql api so that the current openstack code do not need to be changed, instead, just a pluggin with some configurations. This is pretty much the original use case that dogpile.cache grew out of, see: http://docs.sqlalchemy.org/en/rel_0_9/orm/examples.html#dogpile-caching -Original Message- From: Morgan Fainberg [mailto:m...@metacloud.commailto:m...@metacloud.com] Sent: Monday, October 28, 2013 10:22 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions In light of what Dolph said with regards to Keystone, we are using dogpile.cache to implement memoization in front of our driver calls. It it has the ability to cache directly as well, but it has been effective (so far) for our use-case. That being said, I am unsure if caching in front of MySQL is really what we want. I believe that we should be caching after processing work (hence memoization mechanism) instead of at the SQL layer. This also means we can be measured in what we cache (oh hey, it makes no sense to cache X because it needs to be real time or there isn't a performance issue with that query / call, but Y does a ton of processing and is an expensive join/temptable query). In my experience, unless the whole application is designed with caching in mind, caching something as broad as MySQL calls (or any SQL store) is likely going to net exactly what Shawn Hartsock stated, adding a second performance issue. --Morgan On Mon, Oct 28, 2013 at 10:05 AM, Shawn Hartsock hartso...@vmware.commailto:hartso...@vmware.com wrote: I once heard a quote.. I had a performance problem, so I added caching. now I have two performance problems. this. 1,000 times this. Just to float this thought ... make sure it's considered... I've seen a *lot* of people misuse caching when what the really want is memoization. * http://stackoverflow.com/questions/1988804/what-is-memoization-and-how -can-i-use-it-in-python * http://stackoverflow.com/questions/10879137/how-can-i-memoize-a-class- instantiation-in-python ... I'm not sure what you're trying to do. So YMMV, TTFN, BBQ. # Shawn Hartsock - Original Message - From: Clint Byrum cl...@fewbar.commailto:cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Sent: Monday, October 28, 2013 12:12:49 PM Subject: Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions Excerpts from Dolph Mathews's message of 2013-10-28 08:40:19 -0700: It's not specific to mysql (or sql at all), but keystone is using dogpile.cache around driver calls to a similar effect. http://dogpilecache.readthedocs.org/en/latest/ It can persist to memcache, redis, etc. I once heard a quote.. I had a performance problem, so I added caching. now I have two performance problems. Caching is unbelievably awesome in the jobs it can do well. When the problem is straight forward and the requirements are few, it is just the right thing to relieve engineering pressure to make an application more scalable. However, IMO, more than narrow, well defined cache usage is a sign that the application needs some reworking to scale. I like the principle of let's use dogpile so we don't have to reinvent multi-level caching. However, let's make sure we look at each performance issue individually, rather than just throwing them all in a cache box and waving the memcache wand. https://github.com/openstack/keystone/blob/master/keystone/common/c ache/core.py On Fri, Oct 25, 2013 at 6:53 PM, Qing He qing...@radisys.commailto:qing...@radisys.com wrote: All, Has anyone looked at the options of putting a distributed caching system in front of mysql server to improve performance? This should be similar to Oracle Coherence, or VMware VFabric SQLFire. ** ** Thanks, ** ** Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev
Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions
It's not specific to mysql (or sql at all), but keystone is using dogpile.cache around driver calls to a similar effect. http://dogpilecache.readthedocs.org/en/latest/ It can persist to memcache, redis, etc. https://github.com/openstack/keystone/blob/master/keystone/common/cache/core.py On Fri, Oct 25, 2013 at 6:53 PM, Qing He qing...@radisys.com wrote: All, Has anyone looked at the options of putting a distributed caching system in front of mysql server to improve performance? This should be similar to Oracle Coherence, or VMware VFabric SQLFire. ** ** Thanks, ** ** Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions
Excerpts from Dolph Mathews's message of 2013-10-28 08:40:19 -0700: It's not specific to mysql (or sql at all), but keystone is using dogpile.cache around driver calls to a similar effect. http://dogpilecache.readthedocs.org/en/latest/ It can persist to memcache, redis, etc. I once heard a quote.. I had a performance problem, so I added caching. now I have two performance problems. Caching is unbelievably awesome in the jobs it can do well. When the problem is straight forward and the requirements are few, it is just the right thing to relieve engineering pressure to make an application more scalable. However, IMO, more than narrow, well defined cache usage is a sign that the application needs some reworking to scale. I like the principle of let's use dogpile so we don't have to reinvent multi-level caching. However, let's make sure we look at each performance issue individually, rather than just throwing them all in a cache box and waving the memcache wand. https://github.com/openstack/keystone/blob/master/keystone/common/cache/core.py On Fri, Oct 25, 2013 at 6:53 PM, Qing He qing...@radisys.com wrote: All, Has anyone looked at the options of putting a distributed caching system in front of mysql server to improve performance? This should be similar to Oracle Coherence, or VMware VFabric SQLFire. ** ** Thanks, ** ** Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions
I once heard a quote.. I had a performance problem, so I added caching. now I have two performance problems. this. 1,000 times this. Just to float this thought ... make sure it's considered... I've seen a *lot* of people misuse caching when what the really want is memoization. * http://stackoverflow.com/questions/1988804/what-is-memoization-and-how-can-i-use-it-in-python * http://stackoverflow.com/questions/10879137/how-can-i-memoize-a-class-instantiation-in-python ... I'm not sure what you're trying to do. So YMMV, TTFN, BBQ. # Shawn Hartsock - Original Message - From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org Sent: Monday, October 28, 2013 12:12:49 PM Subject: Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions Excerpts from Dolph Mathews's message of 2013-10-28 08:40:19 -0700: It's not specific to mysql (or sql at all), but keystone is using dogpile.cache around driver calls to a similar effect. http://dogpilecache.readthedocs.org/en/latest/ It can persist to memcache, redis, etc. I once heard a quote.. I had a performance problem, so I added caching. now I have two performance problems. Caching is unbelievably awesome in the jobs it can do well. When the problem is straight forward and the requirements are few, it is just the right thing to relieve engineering pressure to make an application more scalable. However, IMO, more than narrow, well defined cache usage is a sign that the application needs some reworking to scale. I like the principle of let's use dogpile so we don't have to reinvent multi-level caching. However, let's make sure we look at each performance issue individually, rather than just throwing them all in a cache box and waving the memcache wand. https://github.com/openstack/keystone/blob/master/keystone/common/cache/core.py On Fri, Oct 25, 2013 at 6:53 PM, Qing He qing...@radisys.com wrote: All, Has anyone looked at the options of putting a distributed caching system in front of mysql server to improve performance? This should be similar to Oracle Coherence, or VMware VFabric SQLFire. ** ** Thanks, ** ** Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions
In light of what Dolph said with regards to Keystone, we are using dogpile.cache to implement memoization in front of our driver calls. It it has the ability to cache directly as well, but it has been effective (so far) for our use-case. That being said, I am unsure if caching in front of MySQL is really what we want. I believe that we should be caching after processing work (hence memoization mechanism) instead of at the SQL layer. This also means we can be measured in what we cache (oh hey, it makes no sense to cache X because it needs to be real time or there isn't a performance issue with that query / call, but Y does a ton of processing and is an expensive join/temptable query). In my experience, unless the whole application is designed with caching in mind, caching something as broad as MySQL calls (or any SQL store) is likely going to net exactly what Shawn Hartsock stated, adding a second performance issue. --Morgan On Mon, Oct 28, 2013 at 10:05 AM, Shawn Hartsock hartso...@vmware.com wrote: I once heard a quote.. I had a performance problem, so I added caching. now I have two performance problems. this. 1,000 times this. Just to float this thought ... make sure it's considered... I've seen a *lot* of people misuse caching when what the really want is memoization. * http://stackoverflow.com/questions/1988804/what-is-memoization-and-how-can-i-use-it-in-python * http://stackoverflow.com/questions/10879137/how-can-i-memoize-a-class-instantiation-in-python ... I'm not sure what you're trying to do. So YMMV, TTFN, BBQ. # Shawn Hartsock - Original Message - From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org Sent: Monday, October 28, 2013 12:12:49 PM Subject: Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions Excerpts from Dolph Mathews's message of 2013-10-28 08:40:19 -0700: It's not specific to mysql (or sql at all), but keystone is using dogpile.cache around driver calls to a similar effect. http://dogpilecache.readthedocs.org/en/latest/ It can persist to memcache, redis, etc. I once heard a quote.. I had a performance problem, so I added caching. now I have two performance problems. Caching is unbelievably awesome in the jobs it can do well. When the problem is straight forward and the requirements are few, it is just the right thing to relieve engineering pressure to make an application more scalable. However, IMO, more than narrow, well defined cache usage is a sign that the application needs some reworking to scale. I like the principle of let's use dogpile so we don't have to reinvent multi-level caching. However, let's make sure we look at each performance issue individually, rather than just throwing them all in a cache box and waving the memcache wand. https://github.com/openstack/keystone/blob/master/keystone/common/cache/core.py On Fri, Oct 25, 2013 at 6:53 PM, Qing He qing...@radisys.com wrote: All, Has anyone looked at the options of putting a distributed caching system in front of mysql server to improve performance? This should be similar to Oracle Coherence, or VMware VFabric SQLFire. ** ** Thanks, ** ** Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions
On 10/28/2013 01:21 PM, Morgan Fainberg wrote: In light of what Dolph said with regards to Keystone, we are using dogpile.cache to implement memoization in front of our driver calls. It it has the ability to cache directly as well, but it has been effective (so far) for our use-case. That being said, I am unsure if caching in front of MySQL is really what we want. I believe that we should be caching after processing work (hence memoization mechanism) instead of at the SQL layer. This also means we can be measured in what we cache (oh hey, it makes no sense to cache X because it needs to be real time or there isn't a performance issue with that query / call, but Y does a ton of processing and is an expensive join/temptable query). In my experience, unless the whole application is designed with caching in mind, caching something as broad as MySQL calls (or any SQL store) is likely going to net exactly what Shawn Hartsock stated, adding a second performance issue. Quite right. In terms of Keystone specifics, I would argue that adding proper pagination (not done in the controller or worse, the client, instead done in the drivers that can handle it) would be of far greater performance benefit than cross-request caching. -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions
Thanks Morgan. In my embedded situation, There is no hard drive where I run the Openstack controller, keystone, and other control functions, thus, my only choice has to be to use an in-core-db (cache) for the realtime transactions into the database, and then use another process to sync the data into the real database cluster through network located off the controller node. From: Dolph Mathews [mailto:dolph.math...@gmail.com] Sent: Monday, October 28, 2013 8:40 AM To: OpenStack Development Mailing List Cc: Morgan Fainberg Subject: Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions It's not specific to mysql (or sql at all), but keystone is using dogpile.cache around driver calls to a similar effect. http://dogpilecache.readthedocs.org/en/latest/ It can persist to memcache, redis, etc. https://github.com/openstack/keystone/blob/master/keystone/common/cache/core.py On Fri, Oct 25, 2013 at 6:53 PM, Qing He qing...@radisys.commailto:qing...@radisys.com wrote: All, Has anyone looked at the options of putting a distributed caching system in front of mysql server to improve performance? This should be similar to Oracle Coherence, or VMware VFabric SQLFire. Thanks, Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] distibuted caching system in front of mysql server for openstack transactions
All, Has anyone looked at the options of putting a distributed caching system in front of mysql server to improve performance? This should be similar to Oracle Coherence, or VMware VFabric SQLFire. Thanks, Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
