Hi all,
I'm using scrapy for a site with bulletin board (phpbb) and I would like
to start "scraping" the pages ONLY after the autentication went good.
In my code, the authentication is done inside star_request method:
def start_requests(self):
self.log("start_requests called")
return [
Request(
"http://<mysite>/phpBB3",
callback=self.parse_welcome,
priority=100
)
]
def parse_welcome(self, response):
self.log("parse_welcome called")
request = FormRequest.from_response(
response,
formnumber=1,
formdata={"username": "rightusername", "password":
"rightpassword"}
)
return request
rules = (
Rule(LinkExtractor(),callback = 'parse_standard',follow=True),
)
[cut]
From the output that I got, it seems that some pages are scraped
without the autentication first.
Am I wrong/missing something? Am I using "priority" in the right way?
Thanks,
Massimo
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
