On Mon, Oct 28, 2013 at 5:46 PM, Qing He <qing...@radisys.com> wrote:
> In my hard drive-less use case, I need an in-core-db/cache that can be in > the same db cluster with real db (with hard drive) with the same sql api so > that the current openstack code do not need to be changed, instead, just a > pluggin with some configurations. > This is pretty much the original use case that dogpile.cache grew out of, see: http://docs.sqlalchemy.org/en/rel_0_9/orm/examples.html#dogpile-caching > > -----Original Message----- > From: Morgan Fainberg [mailto:m...@metacloud.com] > Sent: Monday, October 28, 2013 10:22 AM > To: OpenStack Development Mailing List (not for usage questions) > Subject: Re: [openstack-dev] distibuted caching system in front of mysql > server for openstack transactions > > In light of what Dolph said with regards to Keystone, we are using > dogpile.cache to implement memoization in front of our driver calls. > It it has the ability to cache directly as well, but it has been effective > (so far) for our use-case. > > That being said, I am unsure if caching in front of MySQL is really what > we want. I believe that we should be caching after processing work (hence > memoization mechanism) instead of at the SQL layer. This also means we can > be measured in what we cache (oh hey, it makes no sense to cache X because > it needs to be "real time" or there isn't a performance issue with that > query / call, but Y does a ton of processing and is an expensive > join/temptable query). In my experience, unless the whole application is > designed with caching in mind, caching something as broad as MySQL calls > (or any SQL store) is likely going to net exactly what Shawn Hartsock > stated, adding a second performance issue. > > --Morgan > > On Mon, Oct 28, 2013 at 10:05 AM, Shawn Hartsock <hartso...@vmware.com> > wrote: > >> > >> I once heard a quote.. "I had a performance problem, so I added caching. > >> now I have two performance problems." > >> > > this. 1,000 times this. > > > > Just to float this thought ... make sure it's considered... > > > > I've seen a *lot* of people misuse caching when what the really want is > memoization. > > > > * > > http://stackoverflow.com/questions/1988804/what-is-memoization-and-how > > -can-i-use-it-in-python > > * > > http://stackoverflow.com/questions/10879137/how-can-i-memoize-a-class- > > instantiation-in-python > > > > ... I'm not sure what you're trying to do. So YMMV, TTFN, BBQ. > > > > # Shawn Hartsock > > > > ----- Original Message ----- > >> From: "Clint Byrum" <cl...@fewbar.com> > >> To: "openstack-dev" <openstack-dev@lists.openstack.org> > >> Sent: Monday, October 28, 2013 12:12:49 PM > >> Subject: Re: [openstack-dev] distibuted caching system in front of > mysql server for openstack transactions > >> > >> Excerpts from Dolph Mathews's message of 2013-10-28 08:40:19 -0700: > >> > It's not specific to mysql (or sql at all), but keystone is using > >> > dogpile.cache around driver calls to a similar effect. > >> > > >> > http://dogpilecache.readthedocs.org/en/latest/ > >> > > >> > It can persist to memcache, redis, etc. > >> > > >> > >> I once heard a quote.. "I had a performance problem, so I added caching. > >> now I have two performance problems." > >> > >> Caching is unbelievably awesome in the jobs it can do well. When the > >> problem is straight forward and the requirements are few, it is just > >> the right thing to relieve engineering pressure to make an > >> application more scalable. > >> > >> However, IMO, more than narrow, well defined cache usage is a sign > >> that the application needs some reworking to scale. > >> > >> I like the principle of "let's use dogpile so we don't have to > >> reinvent multi-level caching". However, let's make sure we look at > >> each performance issue individually, rather than just throwing them > >> all in a cache box and waving the memcache wand. > >> > >> > > >> > https://github.com/openstack/keystone/blob/master/keystone/common/c > >> > ache/core.py > >> > > >> > On Fri, Oct 25, 2013 at 6:53 PM, Qing He <qing...@radisys.com> wrote: > >> > > >> > > All,**** > >> > > > >> > > Has anyone looked at the options of putting a distributed caching > >> > > system in front of mysql server to improve performance? This > >> > > should be similar to Oracle Coherence, or VMware VFabric > >> > > SQLFire.**** > >> > > > >> > > ** ** > >> > > > >> > > Thanks,**** > >> > > > >> > > ** ** > >> > > > >> > > Qing**** > >> > > > >> > > _______________________________________________ > >> > > OpenStack-dev mailing list > >> > > OpenStack-dev@lists.openstack.org > >> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > > >> > > > >> > > >> > >> _______________________________________________ > >> OpenStack-dev mailing list > >> OpenStack-dev@lists.openstack.org > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > > > _______________________________________________ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- -Dolph
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev