Perfect! Thanks.
On Mon, Mar 31, 2014 at 4:21 PM, Rolando Espinoza La Fuente < [email protected]> wrote: > The target web server replies with an empty body when the user agent is > the scrapy's default. I'm able to see the content by changing the user > agent: > > $ scrapy shell http://www.stvalentinshop.com/all/faguo-linden-camel.html--set > USER_AGENT=googlebot > > Regards > Rolando > > > > On Mon, Mar 31, 2014 at 8:20 AM, Andreas Bloch <[email protected]> wrote: > >> I'm receiving empty response bodies on some sites: >> >> In [1]: fetch('http://www.stvalentinshop.com/all/faguo-linden-camel.html >> ') >> >> In [2]: type(response) >> Out[2]: scrapy.http.response.html.HtmlResponse >> >> In [3]: response.body >> Out[3]: '' >> >> Doing a simple urlopen with urllib works... >> >> In [9]: response = urllib2.urlopen(' >> http://www.stvalentinshop.com/all/faguo-linden-camel.html').read() >> In [10]: response (works) >> >> What it wrong here? >> >> -- >> You received this message because you are subscribed to the Google Groups >> "scrapy-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/scrapy-users. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to a topic in the > Google Groups "scrapy-users" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/scrapy-users/3hhBkwWxcM8/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
