Hi, i would like to know how to get the calling/reference url once we are parsing data from child url ?
e.g.: we are on url "http://myweb.com/list?page=2"; what list 20 contact urls once we are on the contact url (e.g.: http://myweb.com/contact?id=123) how can we get the calling URL (e.g.: http://myweb.com/list?page=2) ? here is a sample of my code: def parse_start_url(self, response): urls = Selector(response).xpath('//div[contains(@class, "lien-ville")]/ul/li/a/@href').extract() for u in urls: yield scrapy.Request(response.urljoin(u), callback=self.parse_contact) def parse_contact(self, response): yield { "count" : self.counter, "page number" : self.page_num, "contact page url" : response.url, "reference url" : self.url } in my code i would like in parse_contact be able to get the url passed to parse_start_url. How can i do that ? thx -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
