I wasn't able to achieve what I wanted without mucking around with private methods but the code below does work very nicely with reference tables. The code generator pokes in the log10 rowcount as of the time the schema was sampled (I use log10 so that we don't get a bunch of spurious diffs if a reference table had grown by a couple of rows since the last time the generator was run)
We have a hacked version of a LRU dict as per http://code.activestate.com/recipes/498245/ and have mailed the Shove maintainer with a request to have LRU replacement policy available for Shove caches at some point in the future. There is probably a much, much better way of doing this but I am happy that SA made this hack so easy. pjjH # There is probably a much more elegant way to hook Session # 'gets'. The idea is to call lookup to prefetch small tables and # merge them into the session. def _get(self, key, ident): self.session._merge_into_session(self._only_mapper_zero(), key [1]) return Query._get(self, key, ident) # prefetch any small reference tables def _merge_into_session(self, klass, *args): if type(klass) == Mapper: m = klass else: m = class_mapper(klass) cache = None try: cache = _cache_of_caches[m] except KeyError: _cache_of_caches[m] = {} i = m.local_table.info log10_cardinality = i.get('log10_cardinality', -1) if log10_cardinality > 0 and log10_cardinality <= 4: cache = _cache_of_caches[m] #print "prefetching for %s" % (m) for o in self.query(klass): pk_as_tuple = tuple(m.primary_key_from_instance(o)) #print "prefetch populating cache with key %s and object %s " %(pk_as_tuple, o) cache[pk_as_tuple] = o On Apr 15, 6:26 pm, "phrrn...@googlemail.com" <phrrn...@googlemail.com> wrote: > Beautiful. I am impressed at the elegance of this sample and the > others also. > > pjjH > > On Apr 15, 3:26 pm, "Michael Bayer" <mike...@zzzcomputing.com> wrote: > > > for SQLA integration you'd probably build this into the Session/Query, > > which is most easily accomplished by subclassing Query. some examples > > are in the distribution in examples/query_caching. I use a variant of the > > "with_cache_key" version for my own purposes. > > > phrrn...@googlemail.com wrote: > > > > Hi, > > > I would like to export a 'lookup' interface to programmers that does > > > object caching (a bounded size cache with a LRU replacement policy) > > > and prefetching of small tables for read-only reference data. This is > > > a rewrite in Python over SQL Alchemy of similar functionality in Perl > > > (originally implemented over Class::DBI) > > > > I would like to use some general purpose and configurable mechanism > > > such as Shove : > > >http://pypi.python.org/pypi/shove > > > > for c in lookup(Country, [23, 45, 47, 48, 'CA', 'IE', 'FR']): > > > print "%s %s" % (c.name, c.iso_code) > > > > Ideally, the above code should cause an initialization of a cache for > > > 'Country' objects followed by a prefetch of all countries (as the > > > table is small). The lookup() function would return a list of the same > > > cardinality as the input list where the elements of the return list > > > are the corrosponding object ref or -- in the case that the element > > > could not be found -- None. > > > > I intend to use the session to retrieve the data, inflate the objects > > > and so on. > > > I am wondering where is a good place to put this kind of behavior in? > > > a session or mapper extension? > > > > In the meantime, I will implement the behavior I want 'outside' of SA > > > and hope that I can eventually use a native SA mechanism. > > > > thanks, > > > pjjH --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
