Just tried using scrapy shell with UA and it worked. Previously I've tried it and it didn't work at all. Really weird... Thanks for the help!
субота, 05. март 2016. 05.32.39 UTC+1, Steven Almeroth је написао/ла: > > Setting a UA is what this site is looking for: > > $ curl -I 'https://www.yell.com/' -H 'User-Agent: Mozilla/5.0' > HTTP/1.1 200 OK > Server: nginx > Date: Sat, 05 Mar 2016 04:30:50 GMT > Content-Type: text/html > > > On Sunday, February 14, 2016 at 11:54:10 AM UTC-7, Mario wrote: >> >> I’m trying to open yell.com but I keep getting this issue: >> >> [image: ss] >> >> Here’s how I try to open page: >> >> scrapy shell 'https://www.yell.com' >> >> I’ve tried adding user agent but nothing changes :/ >> >> > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
