Hello everybody, really new to python, so bear with me. I am trying to
do some very basic scraping tool. Bascally it just grabs a page xy
times and tells me how long it took. When I do this once, it is
blazingly fast, but when I increase the number of repetitions, it is
slowing down considerably (1 is like 3 ms, 100 takes 6 seconds). I
have done implementations in couple more languages (php, ruby) and
none of them seems to suffer from a similar problem and it seems, that
it behaves linearly. Maybe it is a known issue in urllib2, or I am
simply using it badly. I am using python 2.4.3, machine has CentOS,
below is the sc. Thanks in advance
import urllib2
from datetime import datetime
def application():
start = datetime.now()
req = urllib2.Request("http://127.0.0.1/gdc/about";, None,
{'Accept': 'application/json'})
for number in range(100):
response = urllib2.urlopen(req)
end = datetime.now()
output = end - start
print output
application()
--
http://mail.python.org/mailman/listinfo/python-list
