Request that second url and parse with json. El viernes, 10 de octubre de 2014 02:29:26 UTC-2, Chetan Motamarri escribió: > > Hi All, > > Does anyone has any idea of how to extract javascript data using scrapy..? > > My case is, I need to scrape all discussions in > http://steamcommunity.com/workshop/discussions/?appid=570. (to be more > precise *I want number of threads created on each date*). For example in > this url, there are 19 pages for discussions. When page 2 is clicked, > entire page is not refreshing, only discussions content is reloading. > > How can I use scrapy here. Please help me. > > Using Firebug I figured out discussions data is coming from this url > http://steamcommunity.com/forum/3703047/Workshop/render/-1/?start=0&count=15 > > I know that I can pull all that data using selenium. But I want the > process to be faster, so I decided to use scrapy. >
-- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
