You could also try using a key-value store. I am using pytc, a Python API for Tokyo Cabinet. It seems to figure out quite nicely when to go to disk, and when to use memory. But I have not done extensive tests.
Here is some example code for using pytc: http://github.com/turian/pytc-example/tree/master Joseph On Jun 28, 7:13 pm, mclovin <hanoo...@gmail.com> wrote: > Hello all, > > I need to have a dictionary of about 8 gigs (well the data it is > processing is around 4gb). so naturally i am running into memory > errors. > > So i looked around and found bsddb which acts like a dictionary object > only offloads the data from the RAM to the HDD, however that only > supports strings. > > my dictionaries hold my own class objects > > Is there something like it that is more flexible? -- http://mail.python.org/mailman/listinfo/python-list
