Re: reading data from a web site
hw wrote: > On Wednesday, November 20, 2019 3:29:00 AM CET Olivier wrote: > > hw writes: > > > Hi, > > > > > > how can I read data from a web site which is using multiple frames and > > > some > > > javascript? > > > > Provided that the web site does not change too often and that they don't > > implement stupid "security" features, this should not be too complicate. > > > > Each frame is a web page, with it own URL. So you can examine the source > > code of the web page to find the URL of the first frame and second frame > > > > Them you can use any Perl library you like to load that URLand pars it > > for what you are looking for. > > > > Then use that data to load the second frame with a URL modified to > > include the type of data you have selected. > > > > Being frames makes it much easier, you hould not have to care about the > > javascript too much. > > The web site seems to be created by a program running on the server, i. e. > there is not really a web site. When I access it with lynx or with > WWW::Mechanize, the answer from the server says that neither frames, nor > javascript is supported, and it is not possible to log in. Of course lynx cannot process frames. But that is not what I meant to tell you. Open the web page with your browser, FireFox, Chromium, whatever, the CTRL-U to display the source. In that source, you should see some tages > Can WWW::Mechanize somehow trick the server into assuming that frames and > javascript are supported by the client? -- -- To unsubscribe, e-mail: beginners-unsubscr...@perl.org For additional commands, e-mail: beginners-h...@perl.org http://learn.perl.org/
Re: reading data from a web site
On Wed, 20 Nov 2019 19:52:46 +0100 hw wrote: > On Wednesday, November 20, 2019 3:29:00 AM CET Olivier wrote: > > hw writes: > > > Hi, > > > > > > how can I read data from a web site which is using multiple frames and > > > some > > > javascript? > > > > Provided that the web site does not change too often and that they don't > > implement stupid "security" features, this should not be too complicate. > > > > Each frame is a web page, with it own URL. So you can examine the source > > code of the web page to find the URL of the first frame and second frame > > > > Them you can use any Perl library you like to load that URLand pars it > > for what you are looking for. > > > > Then use that data to load the second frame with a URL modified to > > include the type of data you have selected. > > > > Being frames makes it much easier, you hould not have to care about the > > javascript too much. > > The web site seems to be created by a program running on the server, i. e. > there is not really a web site. When I access it with lynx or with > WWW::Mechanize, the answer from the server says that neither frames, nor > javascript is supported, and it is not possible to log in. > > Can WWW::Mechanize somehow trick the server into assuming that frames and > javascript are supported by the client? > Hi! See: https://github.com/shlomif/Freenode-programming-channel-FAQ/blob/master/FAQ_with_ToC__generated.md#how-can-i-write-code-to-perform-operations-on-web-sites-for-me-that-otherwise-should-be-done-manually (short URL: https://is.gd/ExAHTa ) Regards, Shlomi -- Shlomi Fish https://www.shlomifish.org/ https://github.com/shlomif/Freenode-programming-channel-FAQ Chuck Norris knows exactly which thing is rotten in the kingdom of Denmark. — http://www.shlomifish.org/humour/bits/facts/Chuck-Norris/ Please reply to list if it's a mailing list post - http://shlom.in/reply . -- To unsubscribe, e-mail: beginners-unsubscr...@perl.org For additional commands, e-mail: beginners-h...@perl.org http://learn.perl.org/
Re: reading data from a web site
I think so: https://stackoverflow.com/questions/11501883/wwwmechanize-work-with-javascript Well, maybe not: https://itworkarounds.blogspot.com/2011/07/automate-web-clicks-in-perl-with.html Maybe this is better - maybe it isn't: https://metacpan.org/pod/WWW::Scripter::Plugin::JavaScript Mike On 11/20/19 12:52 PM, hw wrote: The web site seems to be created by a program running on the server, i. e. there is not really a web site. When I access it with lynx or with WWW::Mechanize, the answer from the server says that neither frames, nor javascript is supported, and it is not possible to log in. Can WWW::Mechanize somehow trick the server into assuming that frames and javascript are supported by the client? -- To unsubscribe, e-mail: beginners-unsubscr...@perl.org For additional commands, e-mail: beginners-h...@perl.org http://learn.perl.org/
Re: reading data from a web site
On Wednesday, November 20, 2019 3:29:00 AM CET Olivier wrote: > hw writes: > > Hi, > > > > how can I read data from a web site which is using multiple frames and > > some > > javascript? > > Provided that the web site does not change too often and that they don't > implement stupid "security" features, this should not be too complicate. > > Each frame is a web page, with it own URL. So you can examine the source > code of the web page to find the URL of the first frame and second frame > > Them you can use any Perl library you like to load that URLand pars it > for what you are looking for. > > Then use that data to load the second frame with a URL modified to > include the type of data you have selected. > > Being frames makes it much easier, you hould not have to care about the > javascript too much. The web site seems to be created by a program running on the server, i. e. there is not really a web site. When I access it with lynx or with WWW::Mechanize, the answer from the server says that neither frames, nor javascript is supported, and it is not possible to log in. Can WWW::Mechanize somehow trick the server into assuming that frames and javascript are supported by the client? -- To unsubscribe, e-mail: beginners-unsubscr...@perl.org For additional commands, e-mail: beginners-h...@perl.org http://learn.perl.org/
