Anyway anytime the thing gets updated you should see a different ETag value and a different Last-Modified, so you really just need to look at the headers, which is what HEAD is for! :o).
So given that you should be able to check for a newer version without actually asking for the whole file then all you need is to cache a copy. Doesn't the libwww stuff have some functionality for dealing with a client-side cache? I could be wrong but I seem to remember SOMETHING someplace that does. I'm pretty sure there is some code in CPAN that will check with the origin server to see if its cached copy is still up to date, refetch the contents of the URL if necessary and cache it, and return the locally cached copy. Maybe AxKit already does all this? It would seem like that would be sensible, but then again perhaps not!
Kjetil Kjernsmo wrote:
Hi all!
I'm working on a main page for my TABOO now, and I have figured it would do well as an aggregation of stuff from my own Providers and some RSS feeds I get from other sites.
But about the feeds from other sites, lets take Slashdot.org as an example. They kindly request that you do not retrieve the RSS file more than once every 30 minutes. So, since I just aggregate it, and present it to users as a part of the page, it means that if I get users more often than every 30 mins, I should present them with a locally cached copy.
How do people do this? I'm surely not the first to think about this...?
I thought about writing a Provider that could cache it locally based on the headers that the foreign site sends. Is that a viable path, and has anyone tried it?
The next thing is related (I guess): When I read /. page about not retrieving that feed more than once every 30 minutes, I thought: Well, that should be possible to express in the HTTP response... So, I just (re)read section 13 of RFC 2616, and it does; with Cache-control or Expires headers... But they are not to be found in the headers:
http://slashdot.org/slashdot.rdf HTTP/1.1 200 OK Connection: close Date: Sun, 01 Aug 2004 12:33:16 GMT Accept-Ranges: bytes ETag: "1ae259-7b5-410cde4f" Server: Apache/1.3.29 (Unix) mod_gzip/1.3.26.1a mod_perl/1.29 Content-Length: 1973 Content-Type: text/xml Last-Modified: Sun, 01 Aug 2004 12:13:03 GMT Client-Date: Sun, 01 Aug 2004 12:34:15 GMT Client-Peer: 66.35.250.150:80 Client-Response-Num: 1 X-Bender: An upgrade? I thought we all agreed I was perfect. X-Powered-By: Slash 2.003000
(uhm, what's Bender doing there, and what is he trying to tell us...? :-) )
Is /. brain-damaged, or is the brain-damage in my end....?
Cheers,
Kjetil
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
