You need to set the domain you want to crawl. Example.com isn't a real domain...
On Fri, Oct 30, 2015 at 9:45 AM, Prabhakar <[email protected]> wrote: > Yes, the scrapy spider prints traceback. > > 2015-10-30 17:41:35+0530 [myspider] DEBUG: Crawled (200) <GET > https://example.com/searchid?ss=1&id=1000> (referer: > https://example.com/search?ss=1) ['partial'] > 2015-10-30 17:41:35+0530 [myspider] DEBUG: Crawled (200) <GET > https://example.com/searchid?ss=2&id=2000> (referer: > https://example.com/search?ss=2) ['partial'] > 2015-10-30 17:41:35+0530 [myspider] DEBUG: Crawled (200) <GET > https://example.com/searchid?ss=3&id=3000> (referer: > https://example.com/search?ss=3) ['partial'] > > > And How can I fix this. > > > On Friday, October 30, 2015 at 2:15:02 PM UTC+5:30, Nikolaos-Digenis > Karagiannis wrote: >> >> You may be experiencing data loss in your proxy. >> How do you know that your crawl fails? Does it print a traceback? >> >> >> On Saturday, 4 January 2014 15:16:45 UTC+2, Shivkumar Agrawal wrote: >>> >>> Hi >>> >>> I have created a crawler in scrapy. It uses a list of proxies to crawl. >>> I have 3-4 sites to crawl daily. During crawling, scrapy logs show >>> ['partial'] message and my crawl is failed on those request. I have spent >>> lot of time on googling with no luck. >>> Can anybody help in this matter >>> >>> >>> >>> >> -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
