I wrote an 'fkdict' dict-like class for mappings with a fixed set of
keys but I'm wondering if there's a simpler way to go about it.
First off, the main motivation for it is to save memory in case of many
dicts with the same keys, for example when reading from a
csv.DictReader or constructing dicts out of rows fetched from a
database. For example, test_mem(dict) takes up around 246 MB according
to the Windows task manager while test_mem(fkdict) takes around 49 MB:
def test_mem(maptype):
d = [(i,str(i)) for i in range(1000)]
ds = [maptype(d) for i in xrange(10000)]
raw_input('finished')
An additional benefit is predictable ordering (e.g.
fkdict.fromkeys('abcd').keys() == list('abcd')), like several
ordered-dict recipes.
The implementation I came up with goes like this: each fkdict instance
stores only the values as a list in self._values. The keys and the
mapping of keys to indices are stored in a dynamically generated
subclass of fkdict, so that self._keys and self._key2index are also
accessible from the instance. The dynamically generated subclasses are
cached so that the second time an fkdict with the same keys is created,
the cached class is called.
Since the keys are determined in fkdict.__init__(), this scheme
requires changing self.__class__ to the dynamically generated subclass.
As much as I appreciate Python's dynamic nature, I am not particularly
comfortable with objects that change their class and the implications
this may have in the future (e.g. how well does this play with
inheritance). Is this a valid use case for type-changing behavior or is
there a better, more "mainstream" OO design pattern for this ? I can
post the relevant code if necessary.
George
--
http://mail.python.org/mailman/listinfo/python-list
