It's not a problem with scrapy or google. You need to understand that the '#', and what comes after, should not be sent via HTTP. See:
http://stackoverflow.com/questions/11078916/scrapy-ignoring-content-after-tag-in-the-url On 7 November 2014 at 18:13, Jo Chasinga <[email protected]> wrote: > In scrapy shell when I tried using fetch on a Google' search result page: > > $ scrapy shell " > http://www.google.com/?gws_rd=ssl#q=%22german+beer+near%22&start=0"; > > I got a response without the query string after '#' [s] request <GET > http://www.google.com/?gws_rd=ssl#q=jeffrey+m+liebmann> [s] response <200 > http://www.google.com/?gws_rd=ssl> > > Is this issue belongs to scrapy or Google? Tried pasting the whole URL + > query string and Google led me to the results just fine. > > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
