On Thursday, December 27, 2018 at 9:38:18 AM UTC-5, Rafael Belo wrote:
>
> You won't have some friendly parsed javascript response. The javascript
> it's not information itself, it's a lot of command the will handle
> browser's DOM. That's why we're using a driver to get this informations.
>
> If you get request and parse it, you'll get the raw html with javascript
> code, but if you use a driver, then it'll get the response and execute the
> loaded javascript. This is the key.
>
> Em qui, 27 de dez de 2018 às 11:30, fugee ohu <fuge...@gmail.com
> <javascript:>> escreveu:
>
>>
>>
>> On Wednesday, December 26, 2018 at 6:14:19 PM UTC-5, Walter Lee Davis
>> wrote:
>>>
>>>
>>> > On Dec 26, 2018, at 5:32 PM, fugee ohu <fuge...@gmail.com> wrote:
>>> >
>>> >
>>> >
>>> > On Wednesday, December 26, 2018 at 2:15:37 PM UTC-5, Walter Lee Davis
>>> wrote:
>>> >
>>> > > On Dec 26, 2018, at 2:11 PM, fugee ohu <fuge...@gmail.com> wrote:
>>> > >
>>> > >
>>> > >
>>> > > On Wednesday, December 26, 2018 at 11:58:35 AM UTC-5, Rafael Belo
>>> wrote:
>>> > > You've to include Capybara::DSL.
>>> > >
>>> > > ```
>>> > > include Capybara::DSL
>>> > > ```
>>> > >
>>> > > Em qua, 26 de dez de 2018 às 13:21, fugee ohu <fuge...@gmail.com>
>>> escreveu:
>>> > >
>>> > >
>>> > > On Wednesday, December 26, 2018 at 10:25:33 AM UTC-5, Rafael Belo
>>> wrote:
>>> > > Capybara has a friendly interface for your web drivers, you can
>>> integrate it with selenium, webkit, poltergeist and other.
>>> > > Try to use it, I think you will like it.
>>> > >
>>> > > https://github.com/teamcapybara/capybara
>>> > >
>>> > > Em qua, 26 de dez de 2018 às 12:08, fugee ohu <fuge...@gmail.com>
>>> escreveu:
>>> > >
>>> > >
>>> > > On Wednesday, December 26, 2018 at 9:49:28 AM UTC-5, Rafael Belo
>>> wrote:
>>> > > Yes, if you are using capybara, you may use `visit '
>>> http://myurl.com/goes-here'` <http://myurl.com/goes-here'>
>>> > >
>>> > > Em qua, 26 de dez de 2018 às 11:21, fugee ohu <fuge...@gmail.com>
>>> escreveu:
>>> > >
>>> > >
>>> > > On Wednesday, December 26, 2018 at 8:19:47 AM UTC-5, Rafael Belo
>>> wrote:
>>> > > You are passing an URL instead of a script.
>>> > > This function "execute_script" it's to execute sobre javascript
>>> "script".
>>> > > You've to visit the page with the browser support.
>>> > > If you're using Capybara, you shoul use the "visit" function passing
>>> the URL that you want to go.
>>> > > But if not, you've to look which command your driver has to visit
>>> URL's.
>>> > >
>>> > > Em quarta-feira, 26 de dezembro de 2018 03:51:39 UTC-3, fugee ohu
>>> escreveu:
>>> > >
>>> > >
>>> > > On Tuesday, December 25, 2018 at 6:40:23 PM UTC-5, Hassan Schroeder
>>> wrote:
>>> > > On Tue, Dec 25, 2018 at 3:16 PM fugee ohu <fuge...@gmail.com>
>>> wrote:
>>> > > >
>>> > > > How do i scrape dynamic content from Struts framework with Ruby
>>> > >
>>> > > Same as any web source: send a request, parse the response. Is
>>> > > there some particular issue you're encountering?
>>> > >
>>> > > --
>>> > > Hassan Schroeder ------------------------ hassan.s...@gmail.com
>>> > > twitter: @hassan
>>> > > Consulting Availability : Silicon Valley or remote
>>> > >
>>> > > browser.execute_script('
>>> https://gpsfront.sitename.com/getI2iRecommendingResults.do?callback=jQuery18307882644047005491_1545806199753¤tItemList=32819755026&categoryId=200001521&shopId=2339135&companyId=238468932&recommendType=&scenario=pcDetailLeftTopSell&limit=6&offset=0&_=1545806304149')
>>>
>>>
>>> > >
>>> > > Selenium::WebDriver::Error::UnknownError: unknown error:
>>> Runtime.evaluate threw exception: SyntaxError: Unexpected end of input
>>> > > (Session info: chrome=71.0.3578.80)
>>> > > (Driver info: chromedriver=2.42.591071
>>> (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 4.15.0-43-generic
>>> x86_64)
>>> > >
>>> > >
>>> > >
>>> > > Must I use the url?
>>> > >
>>> > >
>>> > > --
>>> > > You received this message because you are subscribed to a topic in
>>> the Google Groups "Ruby on Rails: Talk" group.
>>> > > To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/rubyonrails-talk/CpOPHz-zFsc/unsubscribe.
>>>
>>> > > To unsubscribe from this group and all its topics, send an email to
>>> rubyonrails-ta...@googlegroups.com.
>>> > > To post to this group, send email to rubyonra...@googlegroups.com.
>>> > > To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/rubyonrails-talk/acaef991-80d3-4c6c-bccc-fdc017ef4734%40googlegroups.com.
>>>
>>>
>>> > > For more options, visit https://groups.google.com/d/optout.
>>> > >
>>> > >
>>> > > --
>>> > > Rafael Belo
>>> > > Web Developer
>>> > > Skype: rafaelrpbelo
>>> > > Twitter: @rafaelrpbelo
>>> > > Linkedin: rafaelrpbelo
>>> > >
>>> > > This is the first time I'm hearing Capybara recommended for web
>>> scraping Is this the preferred method for what I'm trying to do?
>>> > >
>>> > > --
>>> > > You received this message because you are subscribed to a topic in
>>> the Google Groups "Ruby on Rails: Talk" group.
>>> > > To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/rubyonrails-talk/CpOPHz-zFsc/unsubscribe.
>>>
>>> > > To unsubscribe from this group and all its topics, send an email to
>>> rubyonrails-ta...@googlegroups.com.
>>> > > To post to this group, send email to rubyonra...@googlegroups.com.
>>> > > To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/rubyonrails-talk/6d8212f3-dae8-444c-9004-28a4c5b0b103%40googlegroups.com.
>>>
>>>
>>> > > For more options, visit https://groups.google.com/d/optout.
>>> > >
>>> > >
>>> > > --
>>> > > Rafael Belo
>>> > > Web Developer
>>> > > Skype: rafaelrpbelo
>>> > > Twitter: @rafaelrpbelo
>>> > > Linkedin: rafaelrpbelo
>>> > >
>>> > > I need to work in rails console and when I run `visit ...` rails
>>> complains of no matching route
>>> > >
>>> > > --
>>> > > You received this message because you are subscribed to a topic in
>>> the Google Groups "Ruby on Rails: Talk" group.
>>> > > To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/rubyonrails-talk/CpOPHz-zFsc/unsubscribe.
>>>
>>> > > To unsubscribe from this group and all its topics, send an email to
>>> rubyonrails-ta...@googlegroups.com.
>>> > > To post to this group, send email to rubyonra...@googlegroups.com.
>>> > > To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/rubyonrails-talk/2ae5f27c-eb96-4de2-bc2d-0974387b21c7%40googlegroups.com.
>>>
>>>
>>> > > For more options, visit https://groups.google.com/d/optout.
>>> > >
>>> > >
>>> > > --
>>> > > Rafael Belo
>>> > > Web Developer
>>> > > Skype: rafaelrpbelo
>>> > > Twitter: @rafaelrpbelo
>>> > > Linkedin: rafaelrpbelo
>>> > >
>>> > > require 'capybara/rails'
>>> > > include Capybara::DSL
>>> > >
>>> > > No change I still get the same routing error
>>> ActionController::RoutingError (No route matches [GET]
>>> "/getI2iRecommendingResults.do"):
>>> > >
>>> > >
>>> >
>>> > You'll probably get better answers if you show your work. Try writing
>>> a single script that demonstrates what you want to do, and post it as a
>>> Gist. Link it here, show what the output looks like, and see where that
>>> leads you. Often times, working in the constraints of making the example
>>> work in a single script forces you to reconsider the problem, or shows you
>>> a simple error you made while configuring something more complex.
>>> >
>>> > Walter
>>> >
>>> >
>>> > I'd like to write a script after I get my commands down, for now I'm
>>> working in rails console It may not be the intended use of capybara to
>>> visit url's not defined in routes.rb Should I be using something else to
>>> make the request
>>>
>>> Try using an API tool, like Faraday.
>>>
>>> gem 'faraday'
>>> require 'faraday'
>>> response = Faraday.get('https://entire.url.of/your/api/data.json')
>>>
>>> whatever_parsing_tool_you_want.parse(response.body)
>>>
>>> Walter
>>>
>>> >
>>> > --
>>> > You received this message because you are subscribed to the Google
>>> Groups "Ruby on Rails: Talk" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> an email to rubyonrails-ta...@googlegroups.com.
>>> > To post to this group, send email to rubyonra...@googlegroups.com.
>>> > To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/rubyonrails-talk/ba70d205-f364-4c1e-ba25-b8edd4650c0f%40googlegroups.com.
>>>
>>>
>>> > For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>> I used Nokogiri::HTML.parse(response.body) It isn't converting the
>> javascript response to something more friendly
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Ruby on Rails: Talk" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/rubyonrails-talk/CpOPHz-zFsc/unsubscribe
>> .
>> To unsubscribe from this group and all its topics, send an email to
>> rubyonrails-ta...@googlegroups.com <javascript:>.
>> To post to this group, send email to rubyonra...@googlegroups.com
>> <javascript:>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/rubyonrails-talk/96003e5a-48e6-41f8-922b-d67223602b27%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/rubyonrails-talk/96003e5a-48e6-41f8-922b-d67223602b27%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> --
> *Rafael Belo*
> Web Developer
> Skype: rafaelrpbelo
> Twitter: @rafaelrpbelo
> Linkedin: rafaelrpbelo <https://www.linkedin.com/in/rafaelrpbelo/>
>
require 'selenium-webdriver
driver=Selenium::Webdriver.for:chrome
driver.get ("https://gpsfront.website.com/getI2iRecommendingResults.do?
callback=jQuery18307882644045605491_1545806199753¤tItemList=32819755026&categoryId=200121521&shopId=2339135&companyId=238423932&recommendType=&scenario=pcDetailLeftTopSell&limit=6&offset=0&_=1545800704149")
The result:
=> nil
The path I posted here is ficticious but the real path returned => nil
--
You received this message because you are subscribed to the Google Groups "Ruby
on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to rubyonrails-talk+unsubscr...@googlegroups.com.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/rubyonrails-talk/b694619f-90a0-49e2-8a90-d6bc052a8262%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
