Re: ETag support
On Mon, 19 Apr 2004, Mati wrote: See http://devel.squid-cache.org/howto.html and http://devel.squid-cache.org/CVS.html. We are using the SourceForge services for the public developer CVS trees and web space, together with a set of scripts to make source version tracking considerably easier. ok... if I understand correctly I have to pass you my SF account name: ahouhpuc and when you add me somewhere, I will have access to the CVS... am I right? Correct. Now spen a few minutes to read The rules http://devel.squid-cache.org/rules.html CVS tools and branches http://devel.squid-cache.org/CVS.html The tools is only required if you are to create and/or maintain a development branch of Squid. REgards Henrik
Re: ETag support
On Thu, 15 Apr 2004, Mati wrote: if (!httpReplyValidatorsMatch(.. sorry... I wasn't looking at the source whlie writing this mail and i hoped the question wouldn't be ambiguous... apparently i was wrong... its in clientReplyContext::clientGetsOldEntry() in client_side_reply.cc If I understand correctly this part is primarily in case the server returns a ETag incompatible with the clients request. But looking at the httpReplyValidatorsMatch function in Squid-3 I doubt this at all works. Each test needs the following in front: if (one.buf() || two.buf()) to only test for ETag or Content-MD5 it it exists in either reply.. And further, using Content-MD5 here is not a good idea as it is not included in 304 replies.. It is an entity-header and as such MUST/SHOULD NOT be included in 304 replies.. (RFC 2616 10.3.5 304 Not Modified, paragraph 4 If the conditional...) Regards Henrik
Re: ETag support
Hi, As we go on with our project some new questions regarding the code arose. i. the function cacheHit. We need to know when this function is called. I mean, it seems it is called in two cases: firstly when the entry client requested is found in the store snd squid has to decide what to do with it, and secondly, when client request was passed to the origin server and new entry is also in store and squid again has to decide what to return to requesting client. Is that all? There is some IMS mathcing so we figured it is good place to add ETag matching, and we want to know what can we assume on its flow context... ii. function httpReplyValidatorsMatch(). It is called when server responded with 304 Not Modified and squid decides whether to pass this 304 to requesting client, or to return old cached entry... to be more exact there is an if: if (!httpReplyValidatorsMatch(.. we analised the code and we think there is no possibility that in thath place this function would return false... is it true? iii. let's say a client's request contains an If-Modified-Since header and squid has no matching entry so it has to pass the request to origin server... currently it passes the request with the IMS. is it correct? I mean when server responds 304 squid won't be able to cache the response... ok... these are our questions. And regarding Vary... the two go intimately together. There is not very much value in implementing ETag if not having Vary using it.. If all you want it to implement the missing ETag functionality with no respect to Vary then the patch is basically not needed. The bulk of the patch implements the Vary+ETag combo. ok... we think that full ETag support is also very powerful for the users even wihout vary. but we try to support vary anyway. Currently we have about a half of ETag support implemented. In a week or two we'll have it finished and then we'll go for the vary :D and regarding documentation. we eventually found it in doc/programming-guide, but a quick glance on it suggests that places of our interests, ie client_side*, are not documented better than in squid 2.5... Anyway we collect our experiences with the code and after the vary implementetion we will be able to contribute to the documentation also. Regards, Mati.
Re: ETag support
On Thu, 15 Apr 2004, Mateusz Srebrny wrote: Hi, As we go on with our project some new questions regarding the code arose. i. the function cacheHit. We need to know when this function is called. I mean, it seems it is called in two cases: firstly when the entry client requested is found in the store snd squid has to decide what to do with it, and secondly, when client request was passed to the origin server and new entry is also in store and squid again has to decide what to return to requesting client. Is that all? Sounds correct. cacheHit is called to parse header data as it is being read in, and it is this function which knows when the reply heaers have been fully seen. In future it is planned to have this redone to split headers from the payload, an pass around a already parsed reply header. There is some IMS mathcing so we figured it is good place to add ETag matching, and we want to know what can we assume on its flow context... In principle where there is IMS matching ETag matching of this object also belongs. There may be one or two special cases, but not much. ii. function httpReplyValidatorsMatch(). It is called when server responded with 304 Not Modified and squid decides whether to pass this 304 to requesting client, or to return old cached entry... to be more exact there is an if: if (!httpReplyValidatorsMatch(.. we analised the code and we think there is no possibility that in thath place this function would return false... is it true? Hmm.. in what function? There is many such ifs with somewhat different purpose. iii. let's say a client's request contains an If-Modified-Since header and squid has no matching entry so it has to pass the request to origin server... currently it passes the request with the IMS. Yes. is it correct? Doubtful, but it is a thin line. Not forwarding the IMS on objects not in the cache gives more predictable and stable results, but at the same time may significantly reduce the local browser cache hit ratio.. mean when server responds 304 squid won't be able to cache the response... Correct. In the simple world before Vary, If-None-Match etc these actually could have been cached but we never got to it. Now with the more expressive HTTP/1.1 validators 304 can no longer be cached, as it is impossible to tell from the 304 reply alone which entity it belongs to or if there is variance or not.. So for future it is probably best to not forward validators when nothing is known about the object if we thing the reply may be cacheable. But then we also risk breaking some applications which assumes cache validators is end-to-end.. ok... we think that full ETag support is also very powerful for the users even wihout vary. Indeed. The ETag based cache validators is very useful in guaranteeing consistent results. but we try to support vary anyway. Currently we have about a half of ETag support implemented. Sounds great! In a week or two we'll have it finished and then we'll go for the vary :D That will be an interesting experience involving many parts of Squid.. and regarding documentation. we eventually found it in doc/programming-guide, but a quick glance on it suggests that places of our interests, ie client_side*, are not documented better than in squid 2.5... Sorry to hear that. Should be a sign that it is still working more or less in the same manner except for the C++ translation. Anyway we collect our experiences with the code and after the vary implementetion we will be able to contribute to the documentation also. Sounds even better!
Re: ETag support
ii. function httpReplyValidatorsMatch(). It is called when server responded with 304 Not Modified and squid decides whether to pass this 304 to requesting client, or to return old cached entry... to be more exact there is an if: if (!httpReplyValidatorsMatch(.. we analised the code and we think there is no possibility that in thath place this function would return false... is it true? Hmm.. in what function? There is many such ifs with somewhat different purpose. sorry... I wasn't looking at the source whlie writing this mail and i hoped the question wouldn't be ambiguous... apparently i was wrong... its in clientReplyContext::clientGetsOldEntry() in client_side_reply.cc thx for other answers they will help us :), Mati.
Re: ETag support
On Sun, 11 Apr 2004, Mati wrote: 1. because we are working on ETags, it would be very useful for us to have our own access to CVS. what should we do to gain it? See http://devel.squid-cache.org/howto.html and http://devel.squid-cache.org/CVS.html. We are using the SourceForge services for the public developer CVS trees and web space, together with a set of scripts to make source version tracking considerably easier. 2. our whole project is continously being recorded on our site http://rainbow.mimuw.edu.pl/~ms189442/mmsquid we invite everyone interested to visit the site - any suggestions are welcome. Regarding documentation.. Squid-3.0 is considerably more documented than Squid-2.5 was as Robert kindly has spent a great deal of time on documenting the new designs while refactoring and redesigning to C++. This documentation is found in the (incomplete) programmers guide found in the doc/Programming-Guide directory. Yes, there is a lot of difference compared to Squid-2.5, and it is not practical to document the difference. Instead we try to focus on documenting the current code starting with the new or rewritten parts. If there is any questions on the code, old or new or patched, please ask. Will try to help explaining things. Regards Henrik
