Please read my question on stack overflow is more easy to show my code and taht http://stackoverflow.com/questions/30396154/scrapy-json-response-body-different-from-json-response-in-website
Em sexta-feira, 22 de maio de 2015 14:13:12 UTC+1, Inês Martins escreveu: > > I am quite new to scrapy. But I have made a few spiders. If you want can > see my doubt in form of question in SO here > http://stackoverflow.com/questions/30396154/scrapy-json-response-body-different-from-json-response-in-website > I am trying to scrapy reviews from this website > http://www.reviewed.com/search/products?sort=rating,desc . Through > firebug I can see that there is a POST request that sends out a json > response with the products. > > What is really confusing me is that my response.body is not the same > response from website, that I see in firebug separator "response". In > scrapy spider, there are only 10 hits and in response from website there > are 20 hits, and really there are 20 products in the firt page (start_url). > What am I doing wrong? Am I passing a wrong Request? > > I have also tested in RESTclient with headers and that but there is also > giving me a json response with only 10 hits... > > Other question, sorry if it is stupid, how can I tell if I need a Request > or a FormRequest? > > Thanks so much for your time and help. > > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
