Hi all,
I wrote some code a few months ago to fetch an atom feed from a
members-only google group, which meant having a cookie jar and sending
a post to the login form. That wasn't too much of a pain: get cookie,
then fetch xml.
Today, the login process has changed to:
1. post to login
2. get($the_xml)
3. that's not xml! ... m/"([^"]+)">Click here to sign in/
4. get($1)
5. not there yet! ... get($ans->header('refresh'))
6. get($the_xml)
So, this is simple but ugly, so of course I want an abstraction layer.
And, I think I should put this on CPAN, but what to name it?
LWP::EmulateRFC2616
Internet::DoinItWrong::Google::2008_08_01
Acme::UsingTehInternet::OnFriday
The emulation part is where I call the get_basic_credentials() method
and basically pretend that the server didn't reinvent the http spec
using javascript.
And of course, it should probably support plugins, which is why I'm
leaning toward the LWP::EmulateRFC2616 namespace. The acme namespace
seems wrong. Mechanize and Scraper don't really fit either -- because
fetching xml *shouldn't* have anything to do with scraping.
Finally, if you have notes on other sites which have failed to read the
http spec, I would love to add your plugin. Also, any thoughts on the
API? Particularly, plugless plugin definition?
Thanks,
Eric
--
A counterintuitive sansevieria trifasciata was once literalized
guiltily.
--Product of Artificial Intelligence
---------------------------------------------------
http://scratchcomputing.com
---------------------------------------------------