So, I'm crawling a site with ajax and the code is like this..
harvest_base(self, response):
#...code...
for opt_num, size in enumerate(size_list):
#...code...
keeper = {}
keeper['option'] = size
keeper['opt_num'] = opt_num
#...code...
yield Request(url=ajax_url, meta=keeper, callback=self.
harvest_matrix)
harvest_matrix(self, response):
#...code...
item['name'] += ' ' + response.meta['option']
#...code...
return item
So, I'm getting this as a name:
Thing SMALL MEDIUM LARGE EXTRA LARGE
So, clearly the meta dict isn't being overwritten by keeper on each call.
Lil help?
Thanks
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
