Roy Smith <r...@panix.com> writes: > I agree that application-level name cacheing is "wrong", but sometimes > doing it the wrong way just makes sense. I could whip up a simple > cacheing wrapper around getaddrinfo() in 5 minutes. Depending on the > environment (both technology and bureaucracy), getting a cacheing > nameserver installed might take anywhere from 5 minutes to a few days to ...
IMHO this really isn't one of those times. The in-app wrapper would only be usable to just that process, and we already know that the OP has multiple processes running the same app on the same machine. They would benefit from being able to share the cache, so now your wrapper gets more complicated. If it's not a nameserver then it's something that fills in for one. And then, since the application appears to be a large scale web spider, it probably wants to run on a cluster, and the cache should be shared across all the machines. So you really probably want an industrial strength nameserver with a big persistent cache, and maybe a smaller local cache because of high locality when crawling specific sites, etc. Also, since this is a production application, doing something in 5 minutes is less important than making it solid and configurable. -- http://mail.python.org/mailman/listinfo/python-list