That sounds like a great idea. Its still a useful module, afterall.
Sorry I didn't have time to respond to your last email. In general, the points were valid.
-ofer
Mattias Holmlund wrote:
Based on the feedback i have gotten from Ofer Nave and Nigel Horne, I have realized that HTTP::Cache might be a too generic name for my module, since it only handles simple get-requests. I will therefore change the name of my module. If noone objects, I will upload my module to CPAN as HTTP::SimpleCache on Thursday.
I think there is room for a more generic module for caching HTTP-requests and responses. One could probably write such a module by inheriting from LWP::UserAgent and making a new implementation of simple_request. This would mean that the rest of LWP could be used as normal with special headers, cookies, robots.txt processing etc. However, I personally have very little interest in such a module right now, and I don't think it's a good idea to write a module that I have no intention of using. I will therefore leave it up to someone else and publish my module as is (with the UserAgent suggestion from Ofer).
/Mattias
On m�n, 2004-09-27 at 18:25, Mattias Holmlund wrote:
Hi,
I have written a perl module that I want to publish on CPAN. The module implements a cache for http requests. It provides a single method get that fetches a url via http. The result of each get is stored in a cache on disk, and if the same url has been requested before, the proper ETag and If-Modified-Since headers are sent to the http server. The server can then respond that the object in the cache is up-to-date and HTTP::Cache will return a cached version of the data instead of fetching it from the server. This speeds up the HTTP get and saves bandwidth for both the server and the client.
The module is very simple to use. Simply create a HTTP::Cache object and call the get-method on the returned object to fetch a url. You do not have to care if the data is fetched from the server or from the cache (but you can find out if you want to know).
Sample usage:
my $c = HTTP::Cache->new( { BasePath => "/tmp/cache", # Directory to store the cache in. MaxAge => 8*24, # How many hours should items be # kept in the cache after they # were last accessed? # Default is 8*24. Verbose => 1, # Print messages to STDERR. # Default is 0. UserAgent => "my-spider", # The user-agent string to use. # Default is "perl-http-cache". } );
my( $content, $error ) = $c->get( $url );
if( defined( $content ) ) { # Data retrieved and stored in $content. # $error indicates if the data was found in the cache (0) # if it was fetched from the server but equal to the cache (1) # or if it was fetched from the server and different from the # cache (2). } else { print STDERR "Failed to fetch $url. " . "Error returned by server: $error"; }
Does anyone object to putting this module on CPAN or is it redundant? Is HTTP::Cache a good name for it?
Regards,
Mattias Holmlund
