Mudcat: >My initial thought was to put the data in large dictionaries and shelve >them (and possibly zipping them to save storage space until the data is >needed). However, these are huge files.
ZODB solves that problem for you. http://www.zope.org/Wikis/ZODB/FrontPage More in particular "5.3 BTrees Package": http://www.zope.org/Wikis/ZODB/guide/node6.html#SECTION000630000000000000000 But I've only used ZODB for small databases compared to yours. It's supposed to scale very well, but I can't speak from experience. -- René Pijlman -- http://mail.python.org/mailman/listinfo/python-list