Setting a UA is what this site is looking for: $ curl -I 'https://www.yell.com/' -H 'User-Agent: Mozilla/5.0' HTTP/1.1 200 OK Server: nginx Date: Sat, 05 Mar 2016 04:30:50 GMT Content-Type: text/html
On Sunday, February 14, 2016 at 11:54:10 AM UTC-7, Mario wrote: > > I’m trying to open yell.com but I keep getting this issue: > > [image: ss] > > Here’s how I try to open page: > > scrapy shell 'https://www.yell.com' > > I’ve tried adding user agent but nothing changes :/ > > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
