Re: scripting lynx
As an alternative: cURL can do nearly everything you can do with a browser (and a lot more) curl -d datatopost http://curl.haxx.se/ It packaged for Debian as well. Cheers, Marcel Craig Sanders [EMAIL PROTECTED] 1 Aug 2001, at 8:33: On Wed, Aug 01, 2001 at 12:17:20AM +0200, Russell Coker wrote: I want to script lynx to post data to a web site and save the results. I am using the --post_data option but have been unable to find documentation on the format of data expected on standard input. I have been trying the following: Hour=0_6 Day=2001-08-01 Getlist=List --- Where I want to simulate a click on the button named Getlist and put the quoted values in the Hour and Day variables. try without the quotes: cat __EOF__ | lynx -dump -post_data ... Hour=0_6 Day=2001-08-01 Getlist=List __EOF__ alternatively, use POST from libwww-perl (LWP). cat __EOF__ | POST -s -d $URL Hour=0_6 Day=2001-08-01 Getlist=List __EOF__ or use the LWP modules to make yourself a web-bot. craig -- craig sanders [EMAIL PROTECTED] Fabricati Diem, PVNC. -- motto of the Ankh-Morpork City Watch -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- __ .´ `. : :' ! Enjoy `. `´ Debian/GNU Linux `- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: scripting lynx
On Thu, Aug 02, 2001 at 05:48:38PM -0700, Kris Blackwood wrote: I tried to install LWP and had no success. Is there a way to install it w/o using the tar ball? apt-get install libwww-perl please learn to quote properly. your reply goes UNDERNEATH the quoted material, not above it. this allows the quoted message to be read in sequential order rather than reverse chronological order. top-posting screws up the chronological order of the replies making it a jarring chore to make sense of them - you have to scroll backwards and forwards trying to match who said what to whom and when. the longer a thread goes on, the worse it gets. you should also delete excess quoted material including headers and signature lines - quote just enough to provide context for your reply, and no more. some useful URLs on the topic of correct quoting netiquette: http://www.math.fu-berlin.de/~guckes/mail/edit.html http://www.iwillfollow.com/email.htm http://home.online.no/~vidaandr/news/FAQquoting.html craig -- craig sanders -- craig sanders [EMAIL PROTECTED] Fabricati Diem, PVNC. -- motto of the Ankh-Morpork City Watch -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: scripting lynx
On Wed, Aug 01, 2001 at 06:08:29PM +0200, Russell Coker wrote:
I tried both of them with no difference.
interesing. either should have worked.
or use the LWP modules to make yourself a web-bot.
I may have to do that. Thanks for the suggestions.
you may need to set the Referer: header in the HTTP request. some cgi
scripts check the referer...(yes, that's pointless and stupid, but it's
quite common).
and set the user agent to something like:
$ua-agent('Mozilla/4.51 (Macintosh; I; PPC)');
i generally use netscape on mac as my user-agent in web robots because:
a) moronic sites generally don't block netscape on macintosh
(i have seen some sites that block netscape on linux with a stupid
message like sorry, we don't support your browser/operating-system.
unfortunately, brain-dead web design is not yet a capital crime)
b) said moronic sites generally wont output moronic IE-specific junk
if they detect netscape. sometimes. if you're lucky.
btw, the perl HTML::TokeParser module is excellent for extracting stuff
from web pages. i used this (plus LWP::UserAgent, HTTP::Cookies, and
HTTP::Request) to write a wrapper script for searching the Melbourne
Trading Post site, which is one of the most brain-dead cretinous sites
i've ever had the misfortune of having to use.
there's also HTML::TableExtract for getting data out of html tables.
these modules are all packaged for debian.
craig
--
craig sanders [EMAIL PROTECTED]
Fabricati Diem, PVNC.
-- motto of the Ankh-Morpork City Watch
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: scripting lynx
At 18:08 01/08/01 +0200, Russell Coker wrote: On Wed, 1 Aug 2001 00:33, Craig Sanders wrote: On Wed, Aug 01, 2001 at 12:17:20AM +0200, Russell Coker wrote: I want to script lynx to post data to a web site and save the results. I am using the --post_data option but have been unable to find documentation on the format of data expected on standard input. [...] or use the LWP modules to make yourself a web-bot. I may have to do that. Thanks for the suggestions. wget and a few shell commands may do the trick too -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: scripting lynx
Here's a script I use to make Orange (a phone company) send me a
password reminder SMS message via their website when there is a network
problem here. The script is a rip off of some other script i found,
it's written in python. it will probably be easy enough to adapt for
what you want.
My reason for using it in the first place was because I didn't want to
have to pay Orange more money so i can send SMS messages via their
website, but password reminders for the SMS service are free :)
Regards,
Robert Davidson.
On Wed, Aug 01, 2001 at 06:11:34PM +0200, Olivier MACCHIONI wrote:
At 18:08 01/08/01 +0200, Russell Coker wrote:
On Wed, 1 Aug 2001 00:33, Craig Sanders wrote:
On Wed, Aug 01, 2001 at 12:17:20AM +0200, Russell Coker wrote:
I want to script lynx to post data to a web site and save the results.
I am using the --post_data option but have been unable to find
documentation on the format of data expected on standard input.
[...]
or use the LWP modules to make yourself a web-bot.
I may have to do that. Thanks for the suggestions.
wget and a few shell commands may do the trick too
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
#!/usr/bin/python
import httplib, sys, time
# Trick Orange into sending me my password for my phone - Makes my phone beep!
### build the query string
qs = devNum=xx
### connect and send the server a path
httpobj = httplib.HTTP('www.orangemail.com.au', 8080)
httpobj.putrequest('POST', '/servlet/orangemail.CreatePassword')
### now generate the rest of the HTTP headers...
httpobj.putheader('Accept', '*/*')
httpobj.putheader('Connection', 'Keep-Alive')
httpobj.putheader('Content-type', 'application/x-www-form-urlencoded')
httpobj.putheader('Content-length', '%d' % len(qs))
httpobj.endheaders()
httpobj.send(qs)
### find out what the server said in response...
reply, msg, hdrs = httpobj.getreply()
if reply != 200:
sys.stdout.write(httpobj.getfile().read())
Re: scripting lynx
On Wed, Aug 01, 2001 at 12:17:20AM +0200, Russell Coker wrote: I want to script lynx to post data to a web site and save the results. I am using the --post_data option but have been unable to find documentation on the format of data expected on standard input. I have been trying the following: Hour=0_6 Day=2001-08-01 Getlist=List --- Where I want to simulate a click on the button named Getlist and put the quoted values in the Hour and Day variables. try without the quotes: cat __EOF__ | lynx -dump -post_data ... Hour=0_6 Day=2001-08-01 Getlist=List __EOF__ alternatively, use POST from libwww-perl (LWP). cat __EOF__ | POST -s -d $URL Hour=0_6 Day=2001-08-01 Getlist=List __EOF__ or use the LWP modules to make yourself a web-bot. craig -- craig sanders [EMAIL PROTECTED] Fabricati Diem, PVNC. -- motto of the Ankh-Morpork City Watch -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
