Hi all, the page I am scraping has the following structure:
*Listing 1* *Link to more details for listing 1* *Listing 2* *Link to more details for listing 2* *Listing 2* *Link to more details for listing 2* I want to follow the "*Link to more details for listing 1" * from my spider code that processes listing 1, fill a field for item 1, and then move onto listing 2, follow the listing 2 link, etc and iterate through all the items in order. However, scrapy's twistedreactor/callback philosophy seems to prevent this, and forces me to *first *do all the items without following links, and *then *follow all the links using a linkextractor and play games to match up each followed link to the listing/item it came from. Can I just confirm that this is correct? -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
