How can I use contracts where I have multiple parsers which pass fields to
each other using the meta attribute of requests?
For example take the following parse methods:
def parse_level_one(self, response):
"""
@url <some-url>
@metascrapes field_a field_b
"""
# populate loader
return Request(url=url, callback=self.parse_level_two,
meta={'loader': loader.load_item()})
def parse_level_two(self, response):
"""Parse detail page
"""
loader = MyItemLoader(response.meta['loader'], response=response)
and the contract:
class MetaScrapesContract(Contract):
"""Same as ScrapesContract for when the item is within
the meta attribute of a Request object.
@metascrapes field_1 field_2
"""
name = 'metascrapes'
def post_process(self, output):
for x in output:
if isinstance(x, Request):
for arg in self.args:
if arg not in x.meta['loader']:
raise ContractFail("'{}' field is
missing".format(arg))
I have noticed that although the contract is behaving as it should, there
is an error on parse_level_two when I try to use loader.load_item() I get
AttributeError: use Item['field'].
Besides this, the normal ScrapesContract contract doesnt work because the
loader cannot be instantiated
def parse_level_two(self, response):
"""Parse detail page
@url <url>
@scrapes field1 field2
"""
loader = MyItemLoader(response.meta['loader'], response=response)
KeyErrror: 'loader'
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
