On 4/27/2012 9:55 PM, Paul Rubin wrote:
John Nagle<na...@animats.com> writes:
I may do that to prevent the stall. But the real problem was all
those DNS requests. Parallizing them wouldn't help much when it took
hours to grind through them all.
True dat. But building a DNS cache into the application seems like a
kludge. Unless the number of requests is insane, running a caching
nameserver on the local box seems cleaner.
I know. When I have a bit more time, I'll figure out why
CentOS 5 and Webmin didn't set up a caching DNS resolver by
default.
Sometimes the number of requests IS insane. When the
system hits a page with a thousand links, it has to resolve
all of them. (Beyond a thousand links, we classify it as
link spam and stop. The record so far is a page with over
10,000 links.)
John Nagle
--
http://mail.python.org/mailman/listinfo/python-list
