[sqlalchemy] Re: sqlalchemy + beaker cache
Thanks for the explanation.
I think a ".property" accessor on AssociationProxy is the right
solution.
Right now I am doing something of this sort:
User.groups._get_property().comparator but not sure if that's what
should be done or not.
On Jul 29, 7:20 am, Michael Bayer wrote:
> An association proxy is a Python descriptor attached to a class and by itself
> is not anything like the InstrumentedAttribute/relationship() object used for
> a relationship. So it cannot be passed to mapper options that expect
> InstrumentedAttribute, or those options need to be enhanced to detect when an
> association proxy is passed in, which would be to check for its class as an
> AssociationProxy object, then call _get_property() on it to get the ultimate
> relationship().
>
> It would be a good idea for us to stick a ".property" accessor on
> AssociationProxy so this kind of thing would work automatically but I'd want
> to have tests that it works under all kinds of join()/options() scenarios.
> added #2236 for that.
>
> On Jul 29, 2011, at 12:32 AM, espresso maker wrote:
>
>
>
>
>
>
>
> > Anyone know of another sqlalchemy + beaker example I can look it?
>
> > On Jul 27, 2:05 pm, espresso maker wrote:
> >> Hi there,
>
> >> I am trying to follow the setup in this
> >> examplehttp://www.sqlalchemy.org/trac/browser/examples/beaker_cachingto
> >> enable beaker caching in sqlalchemy. However, I ran into an issue.
>
> >> #1. When I try to cache a relation that happens to be an association
> >> proxy I get the following error:
>
> >> AttributeError: 'AssociationProxy' object has no attribute 'property'
>
> >> This is how my query looks like:
>
> >> def get_user(user_id):
> >> return Session.query(User).\
> >> options(FromCache('default', 'user')).\
> >> options(RelationshipCache('default', 'user_groups',
> >> User.groups)).\
> >> get(user_id)
>
> >> Anyone ran into this problem?
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "sqlalchemy" group.
> > To post to this group, send email to sqlalchemy@googlegroups.com.
> > To unsubscribe from this group, send email to
> > sqlalchemy+unsubscr...@googlegroups.com.
> > For more options, visit this group
> > athttp://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: sqlalchemy + beaker cache
An association proxy is a Python descriptor attached to a class and by itself
is not anything like the InstrumentedAttribute/relationship() object used for a
relationship. So it cannot be passed to mapper options that expect
InstrumentedAttribute, or those options need to be enhanced to detect when an
association proxy is passed in, which would be to check for its class as an
AssociationProxy object, then call _get_property() on it to get the ultimate
relationship().
It would be a good idea for us to stick a ".property" accessor on
AssociationProxy so this kind of thing would work automatically but I'd want to
have tests that it works under all kinds of join()/options() scenarios. added
#2236 for that.
On Jul 29, 2011, at 12:32 AM, espresso maker wrote:
> Anyone know of another sqlalchemy + beaker example I can look it?
>
> On Jul 27, 2:05 pm, espresso maker wrote:
>> Hi there,
>>
>> I am trying to follow the setup in this
>> examplehttp://www.sqlalchemy.org/trac/browser/examples/beaker_cachingto
>> enable beaker caching in sqlalchemy. However, I ran into an issue.
>>
>> #1. When I try to cache a relation that happens to be an association
>> proxy I get the following error:
>>
>> AttributeError: 'AssociationProxy' object has no attribute 'property'
>>
>> This is how my query looks like:
>>
>> def get_user(user_id):
>> return Session.query(User).\
>>options(FromCache('default', 'user')).\
>>options(RelationshipCache('default', 'user_groups',
>> User.groups)).\
>>get(user_id)
>>
>> Anyone ran into this problem?
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> To unsubscribe from this group, send email to
> sqlalchemy+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: sqlalchemy + beaker cache
Anyone know of another sqlalchemy + beaker example I can look it?
On Jul 27, 2:05 pm, espresso maker wrote:
> Hi there,
>
> I am trying to follow the setup in this
> examplehttp://www.sqlalchemy.org/trac/browser/examples/beaker_cachingto
> enable beaker caching in sqlalchemy. However, I ran into an issue.
>
> #1. When I try to cache a relation that happens to be an association
> proxy I get the following error:
>
> AttributeError: 'AssociationProxy' object has no attribute 'property'
>
> This is how my query looks like:
>
> def get_user(user_id):
> return Session.query(User).\
> options(FromCache('default', 'user')).\
> options(RelationshipCache('default', 'user_groups',
> User.groups)).\
> get(user_id)
>
> Anyone ran into this problem?
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: sqlalchemy + beaker cache
Another issue I run into intermittently is the following:
TypeError: can't pickle function objects
Module myproject.lib.account_api:98 in get_user view
>> get(user_id)
Module sqlalchemy.orm.query:637 in get view
>> return self._get(key, ident)
Module sqlalchemy.orm.query:1968 in _get view
>> return q.one()
Module sqlalchemy.orm.query:1656 in one view
>> ret = list(self)
Module myproject.model.caching:75 in __iter__ view
>> return self.get_value(createfunc=lambda: list(Query.__iter__(self)))
Module myproject.model.caching:93 in get_value view
>> ret = cache.get_value(cache_key, createfunc=createfunc)
Module beaker.cache:214 in get view
>> return self._get_value(key, **kw).get_value()
Module beaker.container:299 in get_value view
>> self.set_value(v)
Module beaker.container:332 in set_value view
>> self.namespace.set_value(self.key, (storedtime, self.expire_argument,
>> value))
Module beaker.ext.memcached:67 in set_value view
>> self.mc.set(self._format_key(key), value)
Module memcache:515 in set view
>> return self._set("set", key, val, time, min_compress_len)
Module memcache:725 in _set view
>> store_info = self._val_to_store_info(val, min_compress_len)
Module memcache:697 in _val_to_store_info view
>> pickler.dump(val)
Module copy_reg:70 in _reduce_ex view
>> raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle function objects
On Jul 27, 2:05 pm, espresso maker wrote:
> Hi there,
>
> I am trying to follow the setup in this
> examplehttp://www.sqlalchemy.org/trac/browser/examples/beaker_cachingto
> enable beaker caching in sqlalchemy. However, I ran into an issue.
>
> #1. When I try to cache a relation that happens to be an association
> proxy I get the following error:
>
> AttributeError: 'AssociationProxy' object has no attribute 'property'
>
> This is how my query looks like:
>
> def get_user(user_id):
> return Session.query(User).\
> options(FromCache('default', 'user')).\
> options(RelationshipCache('default', 'user_groups',
> User.groups)).\
> get(user_id)
>
> Anyone ran into this problem?
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Sqlalchemy beaker cache and memcached
Thank you for a quick response. It's much appreciated. Let me be more specific: 1. Do you use the code provided in the above link? If not how do cache and retrieve results via query.options() or cache.get() and then session.merge() ? 2. Do you use it in any framework like pylons or diango? 3. My problem is that I want to cache results of rpc function calls that return sqlalchemy table objects so that method output is heavily based on the parameters received. While one method is responsible for fetching records others do data manipulation so they must invalidate all parametrized cached results of the 'get method' (and use some kind of wildcart?). I saw that it is possible to cache diffrent results based on diffrent filter() values (example in the link) but that effectively means building the same query to invalidate data in the add/delete/edit methods (and how do I do this while cached resullts are diffrent among diffrent values submitted to filter(). I dont know if I'm being clear enough, can you or anyone provide any input on that particular matter? On 9 Cze, 02:14, BenH wrote: > Hi, > > I use Beaker in production to help speed up the delivery of game > content. > We've seen enormous (seconds to milliseconds) speed ups for caching > large queries that don't change. > > We don't use it at the query level but as a way to cache whole results > from sqlalchemy. > As long as you remember to merge all the objects back into the Session > you're fine. > > We currently don't use the 'memcache' but the 'memory' setting, it's > the easiest to setup and gives very good results. > The only caveat I have about the memory caching is that there is no > way of setting how big it gets, if that's a problem use memcache > instead. > For me, I just set the lifetime of objects to an hour and that stops > the memory growing without bounds. > > File caching doesn't give as good results because you have to hit the > disk and that will lead to IO problems especially if you are using a > database which will be using the disk as well. > > I hope this helps, > > Ben Hesketh > > On Jun 8, 2:43 pm, £ukasz Czuja wrote: > > > > > > > > > Hi, > > > I reviewed a couple of messages on the list concerning caching. While: > > >http://www.sqlalchemy.org/trac/browser/examples/beaker_caching > > > is quite comprehensive, it does not include any information about > > performance gains or wether this example works for more complicated > > queries, joining couple of tables, returning agregates or diffrent > > objects from the same query. Are there any limitations on data types > > that cannot be cached? Also there is no information as wether this > > example would work if beaker is setup with memcache as storage and > > wether it would be faster than file storage and if so by how much? > > > Can any one provide some numbers on this concept? Does anyone have a > > success story to share? > > > I'm dying to see how this performs and if it is usable on production > > environment. > > > Any input is appreciated. Thanks in advance. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Sqlalchemy beaker cache and memcached
Hi, I use Beaker in production to help speed up the delivery of game content. We've seen enormous (seconds to milliseconds) speed ups for caching large queries that don't change. We don't use it at the query level but as a way to cache whole results from sqlalchemy. As long as you remember to merge all the objects back into the Session you're fine. We currently don't use the 'memcache' but the 'memory' setting, it's the easiest to setup and gives very good results. The only caveat I have about the memory caching is that there is no way of setting how big it gets, if that's a problem use memcache instead. For me, I just set the lifetime of objects to an hour and that stops the memory growing without bounds. File caching doesn't give as good results because you have to hit the disk and that will lead to IO problems especially if you are using a database which will be using the disk as well. I hope this helps, Ben Hesketh On Jun 8, 2:43 pm, Łukasz Czuja wrote: > Hi, > > I reviewed a couple of messages on the list concerning caching. While: > > http://www.sqlalchemy.org/trac/browser/examples/beaker_caching > > is quite comprehensive, it does not include any information about > performance gains or wether this example works for more complicated > queries, joining couple of tables, returning agregates or diffrent > objects from the same query. Are there any limitations on data types > that cannot be cached? Also there is no information as wether this > example would work if beaker is setup with memcache as storage and > wether it would be faster than file storage and if so by how much? > > Can any one provide some numbers on this concept? Does anyone have a > success story to share? > > I'm dying to see how this performs and if it is usable on production > environment. > > Any input is appreciated. Thanks in advance. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
