Hi, HTTPCACHE_ENABLED and REDIRECT_ENABLED are enabled in my scraper. So if I hit a cached url, I will see ['cached'] at the end of the line in the log file. But when a page redirects, nothing is at the end of the line. I see ['partial'] for the downloaded pages and ['cached'] for cached pages. As redirected pages don't have anything, does it mean they are cached or they are being hit everytime the crawler tries to crawl them?
Thanks -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/groups/opt_out.
