Brian Pane wrote: > Graham Leggett wrote: > >>> For the expiration case, there's a much easier solution than >>> shadowing the >>> incomplete response. Add a new state for cache entries: >>> "being_updated." >>> When you get a request for a cached object that's past its expiration >>> date, >>> set the cache entry's state to "being_updated" and start retrieving >>> the new >>> content. Meanwhile, as other threads handle requests for the same >>> object, >>> they check the state of the cache entry and, because it's currently >>> being >>> updated, they deliver the old copy from the cache rather than >>> dispatching >>> the request to the backend system that's already working on a different >>> instance of the same request. As long as the thread that's getting >>> the new >>> content can replace the old content with the new content atomically, >>> there's >>> no reason to make any other threads wait for the new content. >> >> >> >> Hmmm... ok, not a bad idea, however I see a few hassles though. >> >> What if a user force-reloads the page. In theory the cached copy >> should be expired immediately - but here we don't, because shadow >> threads need to access the cached content in the meantime. > > > > I actually think we *shouldn't* expire the cached copy in this > case. I want the server, rather than the client, to be able to > decide whether to expire a cache entry. If the client gets to > dictate the expiration semantics, then we're vulnerable to a > DoS attack (all a malicious client would need to do is start > submitting lots of requests that invalidate cache entries). > this is not RFC compliant, so some people will prefer RFC compliance. to give the more sane (imho ;-) use case, we have a directive which will ignore the client cache request.
>> Also - there will be a load spike until the first page is cached (in >> the case where no previous cache existed). > > > > Yep. I think the ideal solution would be to combine this > technique with shadowing support. Then mod_cache could do > one of four things depending on the cache state: > * cache hit, normal case: deliver the object from cache > * cache hit, while another thread is updating: deliver the old object > * cache miss, normal case: go retrieve the content > * cache miss, while another thread is updating: shadow the response in > progress > >> What happens if an attempt to update an expired cached page hangs? The >> proxy could find itself serving stale content for a long time while a >> timeout occurs. > > > > One solution is to add a timestamp for the "being updated" flag. > If another thread sees that the flag is set but the timestamp is > more than 'n' seconds old, that thread sets the timestamp to the > current time and tries to retrieve the object itself. (This > logic just needs to be atomic so that we don't end up with multiple > threads all deciding to retrieve the content at once.) I was actually thinking that we could use another priority queue and a seperate thread(s) to call the back end, limiting the amount of traffic which can ever get to the backend at all. > >>> * It's going to take a while to make the shadowing work >>> (due to all the race conditions that have to be addressed). >> >> >> >> I don't see any significant race conditions though. >> >> All that needs to happen is that all cached responses are >> readable-from-the-cache the moment they are created, rather than the >> moment the download is complete. A flag against the cache entry marks >> it as "still busy". If this flag is set, shadowed threads know to keep >> waiting for more data appearing in the cache. If the flag is not set, >> the shadow thread is a normal CACHE_OUT case, and finishes up the >> request knowing it to be complete. > > > > The race conditions arise in cases where, for example, we find out > halfway through a streamed request that it isn't cacheable after all > because it's just exceeded the max cache size. Therefore we won't > be caching this response permanently, but we do need to keep the > buffered buckets around until all threads that are shadowing the > response have finished using those buckets. And then, after the > last of those threads finishes using the data, we need to delete > the saved brigade. But if the response is *really* large, we can't > keep the whole thing in memory until the last thread finishes > delivering it. Instead, we'd have to incrementally delete buckets > from the start of the brigade as all the threads finish dealing > with each one. (And keep new threads from shadowing the response > from the moment when we first delete a bucket.) This can all be > done with reference counts and careful synchronization, but it's > going to be challenging to debug. > > Brian > >
