Hey, Scott! You can use Splash (https://splash.readthedocs.org/en/stable/), which is a JS rendering service that integrates with Scrapy through Scrapy-splash ( https://github.com/scrapinghub/scrapy-splash).
There is this tutorial explaining it in details: http://blog.scrapinghub.com/2015/03/02/handling-javascript-in-scrapy-with-splash/ Regards, Valdir. On Fri, Dec 18, 2015 at 5:14 AM, Scott Chen <[email protected]> wrote: > Hi all, > > I'm trying to scrape realtime data that is generated by a javascript > <noscript> on a webpage. The html file does not contain the info I would > like to scrape. All I can see from the source is a <noscript> tag at the > place where my data is at when I use inspect element. > > Therefore, I was wondering if scrapy is capable of scraping data generate > by javascripts. > > Thanks in advance! > > Scott > > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- [image: Scrapinghub] <https://scrapinghub.com> Valdir Stumm Junior Developer Evangelist, Scrapinghub [image: Skype]stummjr[image: Twitter] <https://twitter.com/stummjr>[image: Github] <https://github.com/stummjr>[image: Twitter] <https://twitter.com/scrapinghub>[image: LinkedIn] <https://www.linkedin.com/company/scrapinghub>[image: Github] <https://github.com/scrapinghub> *We turn web content into structured data. Lead maintainers of Scrapy <http://scrapy.org>.* -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
