With recursion you need a base case where your function will not call the
function and recurse again. So at some condition you will want to yield an
Item instead of another FormRequest.
On Wednesday, May 20, 2015 at 11:52:12 AM UTC-7, Mehdi Nazari wrote:
>
> Hi All,
>
> Been a while I'm working on a project based off scrapy. I happen to come
> at a point where I need to make a recursive call originating from the same
> parse function.
> here is the code I'm trying to get to work.
>
> def parse_search_result(self, response):
>
> try:
> # import pdb; pdb.set_trace()
> next_page =
> response.xpath("//div[@class='align_center']//node()[following::span and
> not(@class='noborder')]/@href").extract()[0]
> next_page = str(next_page).translate(None, delete)
> next_page_url = '{0}{1}'.format(self.base_url[0], next_page)
> yield FormRequest(next_page_url, method="GET", callback =
> self.parse_search_result)
> except:
> pass
>
> yield FormRequest(response.url, method="GET", callback =
> self.parse_applications)
>
> I don know if I am looking at this piece of logic in a correct way?
>
>
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
