This script finds all of the links on a website that match a certain keyword. If a URL is a match, I would like to yield a request and save the responses for each url. Once all of them have been crawled, I would then like to iterate through a dict, or list, of the responses and find the email and phone number that is used the most time throughout a website. I think I have it setup as I like, but the output is yielding an empty response each time.
Here is my attempt (Four methods; the remaining ones are just helpers): http://pastebin.com/wLb9FguG This code works, but the response item is always from the previous website for some reason. I can't figure it out. -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
