hi,when i use scrapy redis ,i found this problem:
i write two spider,the first spider parse list urls and then obtain detail url
then push detail url to redis schedule,
when i run the second spider ,
i want the second spider call some function,like def start_requets(self): (this
method is login method,and get cookies,after get cookies can visit detail
url),this spider didn't call def start_request(self): function,this spider get
detail url from redis schedule,and then this spider request detail url,and
response code 302(because the second spider not cookies).
i want to call some function before my spider get url from redis schedule ,
after spider run these function ,this spider get url from redis schedule,then
request these url.
def start_requests(self):
url = 'http://XXXXX'
yield FormRequest(url=url,formdata=data,
callback=self.get_checkcode,dont_filter=True)
hope ,some one give me advice.
thank you .
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
