Hi All, Does anyone has any idea of how to extract javascript data using scrapy..?
My case is, I need to scrape all discussions in http://steamcommunity.com/workshop/discussions/?appid=570. (to be more precise *I want number of threads created on each date*). For example in this url, there are 19 pages for discussions. When page 2 is clicked, entire page is not refreshing, only discussions content is reloading. How can I use scrapy here. Please help me. Using Firebug I figured out discussions data is coming from this url http://steamcommunity.com/forum/3703047/Workshop/render/-1/?start=0&count=15 I know that I can pull all that data using selenium. But I want the process to be faster, so I decided to use scrapy. -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
