> > > Let me make an additional remark however: you should > > not expect to get complete details in a list like this - but only > > hints towards a solution for your problem (i.e. > > there remains some work for you). > > Thus, I expect you to read the "cookielib/cookiejar" documentation > > (part of Python's standard documentation) in order to understand > > my example code - before I would be ready to provide further details.
Ok so after reading the documentation for urllib2 and cookielib I came up with the following code: #START from urllib2 import urlopen , Request from cookielib import CookieJar import re regex = re.compile(r'<span class=\'x\'>\{(.*)\}<span class=\'x\'>') base_url = "http://quiz.gambitresearch.com" job_url = base_url + "/job/" cookies = CookieJar() r = Request(base_url) #prepare the request object cookies.add_cookie_header(r) #allow to have cookies R = urlopen(r) #read the url cookies.extract_cookies(R,r) #take the cookies from the response R and adds #them to the request object #build the new url t = R.read() v = str(eval(regex.findall(t)[0])) job_url = job_url + v # Here I create a new request to the url containing the email address r2 = Request(job_url) cookies.add_cookie_header(r2) #I prepare the request for cookies adding the cookies that I extracted before. #perform the request and print the page R2 = urlopen(r2) t2 = R2.read() print job_url print t2 #END This still doesn't work, but I really can't understand why. As far as I have understood first I have to instantiate a Request object and allow it to receive and set cookies (I do this with r = Request() and cookies.add_cookie_header(r)) Next I perform the request (urlopen), save the cookies in the CookieJar (cookies.extract_cookies(R,r)). I evaluate the new address and I create a new Request for it (r2 = Request) I add the cookies stored in the cookiejar in my new request (cookies.add_cookie_header(r2)) Then I perform the request (R2 = urlopen(r2)) and read the page (t2 = R2.read()) What am I doing wrong? Do I misunderstand something in the process? Thanks again in advance for the help, Cheers, Luca -- http://mail.python.org/mailman/listinfo/python-list