Hey all
I'm experimenting with writing a scraper at the moment and have hit a
major hump.
Part of the DOM is added after the page has loaded via javascript.
This means when I make an a request the HTML response I receive back
doesn't accurately represent the page.
Here's a simplified example:
@http_obj = Net::HTTP.new("targetdomain.com")
response, page_data = @http_obj.request_get( "/" )
# page data doesn't contain all of the HTML that is actually shown
Is there anyway library or gem that could simulate the browser
updating the DOM with the Javascript or any other way I could approach
this short of decoding the obfuscated Javascript file?
Thanks in advance
Gav
--
You received this message because you are subscribed to the Google Groups "Ruby
on Rails: Talk" group.
To post to this group, send email to rubyonrails-t...@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.
