RE: [RFC] Apache::Expires
On Thu, 16 Nov 2000, Geoffrey Young wrote: > it's the lack of a 304 that's bothering me (today :) > > If I just put your lines into a handler I get this from netscape: > If-Modified-Since => Thu, 16 Nov 2000 12:48:04 GMT; length=1150 > > but, since there is no modification time for the 'document' to compare to > locally, > my headers out are: > > Expires => Thu, 16 Nov 2000 12:56:45 GMT > Last-Modified => Thu, 16 Nov 2000 12:50:45 GMT > > with status 200 since the document was (again) generated (with new headers). Okay, so what you want is not to implement the headers but rather the If-Modified handler. My best advice is stay true to the laziness that led us to Perl in the first place and just set up mod_proxy. It will handle this for you quite nicely. If you really don't want to use a proxy server, the first step is to handle the If-Modified-Since part, and your proposal for intercepting those sounds fine. You'd need to keep a database mapping URLs to last-mod times and expiration times. You'd look up the incoming URL, send back the appropriate response it it's not modified and hasn't expired, and let it pass through in all other cases. Of course this is sort of lame because if it's really good until the time in your Expires header you should be caching it to hand out to other clients without re-generating. To do a real caching scheme with good performance, I would propose this: - Use URLs that map to real file names. - Create your content generation handler as a 404-handler which gets invoked when one of those files is not on disk. - When you generate a page, send it to the client and then write it to disk. - Keep a little database of expiration times, which a cron job can use to periodically delete expired pages. This is the best possible performance you can get from a caching scheme, since when things are cached they are served as static files and Apache worries about the If-Modified stuff for you. If you're paranoid about serving a page that has been expired for even a second, you'll have to do something more complex with a TransHandler that decides whether or not you can use the static file. This will be a bit slower. I think AxKit does something along these lines, and you could probably steal some code for it from there. - Perrin
Re: [RFC] Apache::Expires
> > > I guess what I am really after is intercepting the If-Modified-Since tag and > return a 304 prior to content generation - maybe in a fixup handler... This makes sense to me. The fixup handler has to decide if new content has to be delivered e.g. the db has changed or to return the 304 > I > think it is the entity tag stuff that is starting to throw me, too... > As long as your browsers doesn't request with a If-Match or If-Non-Match header, you can simply forget about the Entity-Tag and as far as I know the entity tag isn't supported very widely by the bowsers. Gerald - Gerald Richterecos electronic communication services gmbh Internetconnect * Webserver/-design/-datenbanken * Consulting Post: Tulpenstrasse 5 D-55276 Dienheim b. Mainz E-Mail: [EMAIL PROTECTED] Voice:+49 6133 925151 WWW:http://www.ecos.de Fax: +49 6133 925152 -
RE: [RFC] Apache::Expires
> -Original Message-
> From: Perrin Harkins [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, November 15, 2000 6:19 PM
> To: Geoffrey Young
> Cc: '[EMAIL PROTECTED]'
> Subject: Re: [RFC] Apache::Expires
>
>
> On Wed, 15 Nov 2000, Geoffrey Young wrote:
> > I was wondering if anyone has some experience with expire
> headers for
> > dynamic documents - kinda like mod_expires but for dynamic stuff.
>
> We do this, and let mod_proxy use our headers to control its cache and
> handle If-Modified requests.
I guess I was thinking about sans proxy...
>
> Um, it's pretty easy:
>
> my $last_mod = time;
> my $expires = time + 360; # expires in one hour
>
> $r->header_out('Expires' => Apache::Util::ht_time($expires));
> $r->header_out('Last-Modified' =>
> Apache::Util::ht_time($last_modified));
>
> Or did you have something different in mind?
maybe... I might not have my head in the right place, though...
it's the lack of a 304 that's bothering me (today :)
If I just put your lines into a handler I get this from netscape:
If-Modified-Since => Thu, 16 Nov 2000 12:48:04 GMT; length=1150
but, since there is no modification time for the 'document' to compare to
locally,
my headers out are:
Expires => Thu, 16 Nov 2000 12:56:45 GMT
Last-Modified => Thu, 16 Nov 2000 12:50:45 GMT
with status 200 since the document was (again) generated (with new headers).
whereas for static documents, I see:
If-Modified-Since => Wed, 15 Nov 2000 14:45:13 GMT; length=2057
and a 304 out that contains:
Last-Modified => Wed, 15 Nov 2000 14:45:13 GMT
ETag => "20f73-809-3a12a179"
I guess what I am really after is intercepting the If-Modified-Since tag and
return a 304 prior to content generation - maybe in a fixup handler... I
think it is the entity tag stuff that is starting to throw me, too...
make sense? It's a new day (but before my coffee) so who knows...
thanks for the input
--Geoff
>
> - Perrin
>
RE: [RFC] Apache::Expires
On Wed, 15 Nov 2000, Geoffrey Young wrote: > > > > -Original Message- > > From: Robin Berjon [mailto:[EMAIL PROTECTED]] > > Sent: Wednesday, November 15, 2000 10:43 AM > > To: Geoffrey Young > > Cc: '[EMAIL PROTECTED]' > > Subject: Re: [RFC] Apache::Expires > > > > > > At 10:26 15/11/2000 -0500, Geoffrey Young wrote: > > >anyone else see an interest in an Apache::Expires (or > > whatever)? Is there > > >something out there already? I was thinking of working on > > it if there is > > >nothing out there, but it would require lots of RFC reading > > and wrestling > > >with stuff like entity tags... > > > > It would definitely be interesting to provide an easy to use API to > > expiration and caching (at least one that would cover common > > needs), it's > > easy to get mixed up in the details. I think Andreas wrote > > something about > > all those issues last year, but I can't track it down right now. > > I think he wrote the section in the guide on headers - that will definitely > help sort things out... It's kinda a chapter, not a section: http://perl.apache.org/guide/correct_headers.html > > the more I think about it, thoug, I think it will be quite difficult. > > say you want to cache a dynamic document for a week from creation - how do > you define when the week started? > > maybe some compromises will be needed, like generically saying "months start > at 1" and "weeks start at sunday", etc... > > just thinking out loud... > > --Geoff > > > > > > > -- robin b. > > Always remember you're unique just like everyone else. > > > _ Stas Bekman JAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide http://perl.apache.org/guide mailto:[EMAIL PROTECTED] http://apachetoday.com http://jazzvalley.com http://singlesheaven.com http://perlmonth.com perl.org apache.org
Re: [RFC] Apache::Expires
On Wed, 15 Nov 2000, Geoffrey Young wrote:
> I was wondering if anyone has some experience with expire headers for
> dynamic documents - kinda like mod_expires but for dynamic stuff.
We do this, and let mod_proxy use our headers to control its cache and
handle If-Modified requests.
> anyone else see an interest in an Apache::Expires (or whatever)? Is there
> something out there already? I was thinking of working on it if there is
> nothing out there, but it would require lots of RFC reading and wrestling
> with stuff like entity tags...
Um, it's pretty easy:
my $last_mod = time;
my $expires = time + 360; # expires in one hour
$r->header_out('Expires' => Apache::Util::ht_time($expires));
$r->header_out('Last-Modified' => Apache::Util::ht_time($last_modified));
Or did you have something different in mind?
- Perrin
Re: [RFC] Apache::Expires
Geoffrey Young <[EMAIL PROTECTED]> writes: > hi all... > > I was wondering if anyone has some experience with expire headers for > dynamic documents - kinda like mod_expires but for dynamic stuff. Andy Wardley has already clued me in, in two entirely different ways, on doing something very similar with the Template Toolkit - I have modtimes in files I'm templating or records pulled from a database... > why? well, say I have a handler that creates, say, a css document on the > fly based upon fields in a database. the css contents won't change > regularly (if ever) but every request results in dynamic processing. > Wouldn't it be nice to be able to cache dynamic results if I say it's ok? > > (yes, one could just spit out a static document weekly or whatever, but > that's not the point of this discussion, ok :) Again, that's the TT way :-) -- Dave Hodgkinson, http://www.hodgkinson.org Editor-in-chief, The Highway Star http://www.deep-purple.com Apache, mod_perl, MySQL, Sybase hired gun for, well, hire -
RE: [RFC] Apache::Expires
> -Original Message- > From: Bill Moseley [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, November 15, 2000 1:12 PM > To: Geoffrey Young; '[EMAIL PROTECTED]' > Subject: Re: [RFC] Apache::Expires > > > At 10:26 AM 11/15/00 -0500, Geoffrey Young wrote: > >hi all... > > > >I was wondering if anyone has some experience with expire headers for > >dynamic documents - kinda like mod_expires but for dynamic stuff. > > Geoff, > > Are you thinking about client/browsers or proxy caching with regard to > this? Or does it matter? I was thinking about client/browser stuff specifically, but I guess properly formed headers apply to both :) > > I currently use Last-modified and Content-length headers in my dynamic > content that doesn't change much, but I've never considered > using Expires, > but maybe it's because I'm not fully up on what help Expires does. I keep forgetting that Expires is _supposed_ to be a proxy thing - the official MS documentation says that MSIE caches based on Expires, though. (http://support.microsoft.com/support/kb/articles/Q234/0/67.ASP) I dunno about the rest... --Geoff > > I have assumed that most browsers cache my documents and > don't re-request > them in their current session > so am I correct that Expires > would only help > for cases of browsers/clients that return to the page sometime in the > future yet before the document Expires, and after closing > their browser? I > wonder how to determine how many requests that would save. > > Also, if a cached document is past its Expired time, does > that force the > client to get a new document, or can it still use If-Modified-Since? > mod_expires indicates that a new document must be loaded, but RFC 2616 > indicates that it can use If-Modified-Since (who know what > the clients will > do). > > I should know this too, but what effect does the presence of > a query string > in the URL have on this? > > > Bill Moseley > mailto:[EMAIL PROTECTED] >
Re: [RFC] Apache::Expires
At 10:26 AM 11/15/00 -0500, Geoffrey Young wrote: >hi all... > >I was wondering if anyone has some experience with expire headers for >dynamic documents - kinda like mod_expires but for dynamic stuff. Geoff, Are you thinking about client/browsers or proxy caching with regard to this? Or does it matter? I currently use Last-modified and Content-length headers in my dynamic content that doesn't change much, but I've never considered using Expires, but maybe it's because I'm not fully up on what help Expires does. I have assumed that most browsers cache my documents and don't re-request them in their current session, so am I correct that Expires would only help for cases of browsers/clients that return to the page sometime in the future yet before the document Expires, and after closing their browser? I wonder how to determine how many requests that would save. Also, if a cached document is past its Expired time, does that force the client to get a new document, or can it still use If-Modified-Since? mod_expires indicates that a new document must be loaded, but RFC 2616 indicates that it can use If-Modified-Since (who know what the clients will do). I should know this too, but what effect does the presence of a query string in the URL have on this? Bill Moseley mailto:[EMAIL PROTECTED]
RE: [RFC] Apache::Expires
> -Original Message- > From: Robin Berjon [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, November 15, 2000 12:19 PM > To: Geoffrey Young > Cc: '[EMAIL PROTECTED]' > Subject: RE: [RFC] Apache::Expires > > > At 11:17 15/11/2000 -0500, Geoffrey Young wrote: > >> easy to get mixed up in the details. I think Andreas wrote > >> something about > >> all those issues last year, but I can't track it down right now. > > > >I think he wrote the section in the guide on headers - that > will definitely > >help sort things out... > > Ah, yes, in a place that obvious I had no chance of finding it ;) > > >say you want to cache a dynamic document for a week from > creation - how do > >you define when the week started? > > You mean if say, on tuesday a document is generated and says > it will expire > in a week, if it is rerequested on friday it will still need to say it > expires the following tuesday right ? something like that... for instance, using mod_expires you can specify a one week from modification time for expiration. that's more difficult to do with dynamic stuff since you don't have a start point (the 'document' is modified on every access). I was thinking that to specify a one week expiration date would set the Expires header (or Last-Modified or whatever) to "last monday + one week". documents created on friday only really get 3 days instead of a week, but... similarly for months, years, days, etc... you loose true granularity of control (?) but gain some sort of caching this way. I dunno... > That would require persistence, > probably a db or dbm. yeah, except that I suspect the overhead of that would be just as bad as the processing to create the document in the first place... --Geoff > > -- robin b. > I don't suffer from insanity. I enjoy every minute of it. >
RE: [RFC] Apache::Expires
At 11:17 15/11/2000 -0500, Geoffrey Young wrote: >> easy to get mixed up in the details. I think Andreas wrote >> something about >> all those issues last year, but I can't track it down right now. > >I think he wrote the section in the guide on headers - that will definitely >help sort things out... Ah, yes, in a place that obvious I had no chance of finding it ;) >say you want to cache a dynamic document for a week from creation - how do >you define when the week started? You mean if say, on tuesday a document is generated and says it will expire in a week, if it is rerequested on friday it will still need to say it expires the following tuesday right ? That would require persistence, probably a db or dbm. -- robin b. I don't suffer from insanity. I enjoy every minute of it.
Re: [RFC] Apache::Expires
Hi, the next alpha release of Embperl 2.0 will have the ability to generate Expired/Last-Modified headers and handle if-modified-since requests. The programmer has to provide the time (in seconds) when a document should expires and/or can provide a function, that tells Embperl if a document has expired. Upon this data Embperl generates the headers (as far as possible) and caches the output of the document (or subcomponent). Cacheing is already working in the current alpha realase (2.0a16) and the http header stuff is the next I plan to do. I don't know if this is very usefull for a more general Module, because it's all written in C and uses Embperl's internal DOM storage, but on the other side Embperl 2.0 will not only be able to process his own syntax, but also plain Perl, XML (e.g. XSP), and other custom syntaxes (e.g. SSI and others). Gerald - Original Message - From: "Geoffrey Young" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Wednesday, November 15, 2000 4:26 PM Subject: [RFC] Apache::Expires > hi all... > > I was wondering if anyone has some experience with expire headers for > dynamic documents - kinda like mod_expires but for dynamic stuff. > > why? well, say I have a handler that creates, say, a css document on the > fly based upon fields in a database. the css contents won't change > regularly (if ever) but every request results in dynamic processing. > Wouldn't it be nice to be able to cache dynamic results if I say it's ok? > > (yes, one could just spit out a static document weekly or whatever, but > that's not the point of this discussion, ok :) > > anyone else see an interest in an Apache::Expires (or whatever)? Is there > something out there already? I was thinking of working on it if there is > nothing out there, but it would require lots of RFC reading and wrestling > with stuff like entity tags... > > comments/experiences welcome... > > --Geoff > > - Gerald Richterecos electronic communication services gmbh Internetconnect * Webserver/-design/-datenbanken * Consulting Post: Tulpenstrasse 5 D-55276 Dienheim b. Mainz E-Mail: [EMAIL PROTECTED] Voice:+49 6133 925151 WWW:http://www.ecos.de Fax: +49 6133 925152 -
RE: [RFC] Apache::Expires
> -Original Message- > From: Robin Berjon [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, November 15, 2000 10:43 AM > To: Geoffrey Young > Cc: '[EMAIL PROTECTED]' > Subject: Re: [RFC] Apache::Expires > > > At 10:26 15/11/2000 -0500, Geoffrey Young wrote: > >anyone else see an interest in an Apache::Expires (or > whatever)? Is there > >something out there already? I was thinking of working on > it if there is > >nothing out there, but it would require lots of RFC reading > and wrestling > >with stuff like entity tags... > > It would definitely be interesting to provide an easy to use API to > expiration and caching (at least one that would cover common > needs), it's > easy to get mixed up in the details. I think Andreas wrote > something about > all those issues last year, but I can't track it down right now. I think he wrote the section in the guide on headers - that will definitely help sort things out... the more I think about it, thoug, I think it will be quite difficult. say you want to cache a dynamic document for a week from creation - how do you define when the week started? maybe some compromises will be needed, like generically saying "months start at 1" and "weeks start at sunday", etc... just thinking out loud... --Geoff > > -- robin b. > Always remember you're unique just like everyone else. >
Re: [RFC] Apache::Expires
At 10:26 15/11/2000 -0500, Geoffrey Young wrote: >anyone else see an interest in an Apache::Expires (or whatever)? Is there >something out there already? I was thinking of working on it if there is >nothing out there, but it would require lots of RFC reading and wrestling >with stuff like entity tags... It would definitely be interesting to provide an easy to use API to expiration and caching (at least one that would cover common needs), it's easy to get mixed up in the details. I think Andreas wrote something about all those issues last year, but I can't track it down right now. -- robin b. Always remember you're unique just like everyone else.
[RFC] Apache::Expires
hi all... I was wondering if anyone has some experience with expire headers for dynamic documents - kinda like mod_expires but for dynamic stuff. why? well, say I have a handler that creates, say, a css document on the fly based upon fields in a database. the css contents won't change regularly (if ever) but every request results in dynamic processing. Wouldn't it be nice to be able to cache dynamic results if I say it's ok? (yes, one could just spit out a static document weekly or whatever, but that's not the point of this discussion, ok :) anyone else see an interest in an Apache::Expires (or whatever)? Is there something out there already? I was thinking of working on it if there is nothing out there, but it would require lots of RFC reading and wrestling with stuff like entity tags... comments/experiences welcome... --Geoff
