Why do you say that the bottleneck of the crawler will always be downloading the page? Is it becasue there isn't already a modual to do this and I will have to start from scratch? Or a bandwidth issue?
-- http://mail.python.org/mailman/listinfo/python-list