> > I'm working on a CGI script that pulls XML data from a public database > > Ah, I missed that bit on first read. Consider using something different than > CGI here if you want to do caching. FCGI would allow you to do in-memory > caching, for example, as would mod_python and a lot of other solutions.
What I'm aiming for is sort of a "permanent" disk cache/mirror that adds records as needed. The main issue is that the database (PubMed) requests that we limit requests to once every three seconds. I often need to access data for hundreds of records, so I've set up a cron job to cache the needed records to disk overnight. I was using shelve before, but the size of the file grew to about 500 mB and I started having issues with shelve performance. So, I just stopped using shelve and am storing each record separately on disk now. Thanks! Orest -- http://mail.python.org/mailman/listinfo/python-list
