If any of you have time, your comments to this outline for degraded mode and prefetching would be appreciated.
Prefetch -------- Prefetching is easy to dispatch with: at some VCL determined time before the TTL expires, we try to refresh the object from the backend so that it never grows stale. Deciding that prefetching should be an option for an object, must be decided at fetch time, in order to cache the requesting headers, at the same time, a time is set for when to prefetch. The timer can either be absolute, or if negative relative to obj.ttl. The interesting question is which headers to send to the backend, and which client to pretend to be. The simplest solution is probably to replay the headers used to fetch the object in the first place, but this may wrongly account the fetch to a particular client/cookie/user/account. The alternative is to filter headers to the bare minimum, respecting Vary:, and hope that gives the expected result. Either way, VCL control is desirable, and an filtering/additive approach is preferable, since it gives a known set of headers to work from. sub vcl_fetch { set obj.prefetch = -2m; unset obj.req.http.cookie; set obj.req.myhdr = req.myhdr; } When the pretech timer expires, vcl_prefetch{} is invoked to call the shot: sub vcl_prefetch { if (obj.uses > 10) { prefetch; } pass; } When prefetching an object, the new incomplete object should not be marked "busy" in the "hold up" sense of the concept, since we can reasonably expect a serviceable copy of the object to exists (the one which triggered the prefetch). If however, this copy fails to satisfy (eg. Vary:) the prefetching object should hold up the client request. XXX: I wonder if we hold up more clients than we need with Vary: today ? Prefetching should probably be separately quota'ed on the backends, along with general rate-limiting on these. Prefetching is not guaranteed to happen in the same VCL as it was armed, and therefore requests to prefetch must enter in vcl_recv{} like every other request. For that reason, the backend used is not a hard reference, but a soft reference only for affinity. For the lack of any better idea, I think all prefetching will look like it happend from a client with IP# 127.0.0.2 Degraded mode ------------- Degraded mode is the intentional serving of technically stale objects instead of returning errors. The condition for returning stale content is: * Client must be marked as accepting degraded objects (VCL: "client.degraded = true", default true) before lookup. * Object must be within it's timelimit for degraded mode (VCL: "obj.stale_time = 1h", default 30 seconds). * An attempt to fetch the object from the backend must be in progress or recently (VCL: "backend.backoff = 1m", default 15 seconds) have failed. In vcl_fetch, it would be desirable to be able to check good from bad responses: sub vcl_fetch { if (obj.status > 200 && oldobj) { oldobj.stale_time += 5m; set obj = oldobj; } } XXX: This requires holding refs/locks on both objects, that may be too tricky, and possibly a restart is a better way ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. _______________________________________________ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc