Re: Improving ServletConnection to make it cache-aware
Alexander Klimetschek napisał(a): Grzegorz Kossakowski schrieb: While implementing this some questions arisen: 1. What about error handling? Should we provide some mechanism for passing exception from one servlet to the another? Or maybe just a error code and string message is sufficient? Since it is one stacktrace if you call an internal servlet backed up by a SitemapServlet, you can simply pass on the ProcessingException and not serializing an error page into the stream between both servlets. In the old blocks-fw-impl there is a patch that does exactly that, it re-throws the exception: https://issues.apache.org/jira/browse/COCOON-1954 It works fine for me. Thanks for the pointer. BTW: The following problems have been issues with the blocks-fw I have encountered during real-world usage, especially when using cforms in the "backend"-servlet. We should check those in the new implementation. https://issues.apache.org/jira/browse/COCOON-1964 https://issues.apache.org/jira/browse/COCOON-1939 My current priority is implementing postable sources and polishing patches that I've already provided. This could be added on the third position of my TODO list but I do not behave impolitely sweeping up all interesting work related to servlet services. Do you have any plans associated with these issues in foreseeable future? -- Grzegorz Kossakowski
Re: Improving ServletConnection to make it cache-aware
Grzegorz Kossakowski schrieb: While implementing this some questions arisen: 1. What about error handling? Should we provide some mechanism for passing exception from one servlet to the another? Or maybe just a error code and string message is sufficient? Since it is one stacktrace if you call an internal servlet backed up by a SitemapServlet, you can simply pass on the ProcessingException and not serializing an error page into the stream between both servlets. In the old blocks-fw-impl there is a patch that does exactly that, it re-throws the exception: https://issues.apache.org/jira/browse/COCOON-1954 BTW: The following problems have been issues with the blocks-fw I have encountered during real-world usage, especially when using cforms in the "backend"-servlet. We should check those in the new implementation. https://issues.apache.org/jira/browse/COCOON-1964 https://issues.apache.org/jira/browse/COCOON-1939 Alex -- Alexander Klimetschek http://www.mindquarry.com
Re: Improving ServletConnection to make it cache-aware
Daniel Fagerstrom napisał(a): I agree with everything this far, it would also be nice to add ETag handling to it. The idea is that the servlet-service-fw should work with all kinds of servlets. And using Last-Modified and ETag headers are the two main ways to handle caching for HTTP, so by supporting both we make caching work for a larger share of the servlets. But the first priority is of course to make it work with the SitemapServlet. Using ETags together with If-None-Match is analogous to use Last-Modified together with If-Modified-Since as you described above. Some extra care is needed if the servlet called from the ServletConnection returns both an ETag and a Last-Modified header. Good point. I'll implement this as soon as we are sure that basic functionality works well. We should start from making pipelines more HTTP-compliant. This demands taking If-Modified-Since headers into account and returning appropriate status code when caching pipeline is processed. Behavior of non-caching pipelines should not change. Agree. There is some getLastModified info on the cachedResponse object in the AbstractCachingProcessingPipeline. It doesn't seem like it is used for setting the Last-Modified header or used together with the If-Modified-Since header however. It turned out that some functionality was already implemented (there were proper support for If-Modified-Since) but some was lacking. I've provided patch: https://issues.apache.org/jira/browse/COCOON-2009 Also it might be that one could use the SourceValidity object (or maybe a hash key based on it) as an ETag. You mean, Sitemap should create ETag header? It would be hard to implement without confusing the pipeline code even more. I'm not anyway authoritative but I think pipeline code badly needs redesign. Then we should implement setIfModifiedSince and getIfModifiedSince from java.net.URLConnection and construct requests according to value of that property. Also getResponseCode method should be implemented. All changes proposed above will enable us to implement source validation of ServletSource very easily. Comments? Thoughts? Seem like the right direction to me. Great. I've implemented this functionality and provided patch here: https://issues.apache.org/jira/browse/COCOON-2010 Although I've tested it a little bit I would be grateful if you did the same. Also some comments on the actual code would be helpful before I polish and clean up everything. Please be patient, it's my first attempt to mess with some near-core Cocoon stuff. While implementing this some questions arisen: 1. What about error handling? Should we provide some mechanism for passing exception from one servlet to the another? Or maybe just a error code and string message is sufficient? 2. Should block request and block response classes be synchronized? I mean, do we have take care of synchronization of this classes? 3. See comments in issues for other doubts I will work now on using these functionality for serving server-side resources (xsls, flowscript files etc.) of Ajax and Forms blocks. This way we'll get some heavy testing of this new functionality. -- Grzegorz Kossakowski
Re: Improving ServletConnection to make it cache-aware
Grzegorz Kossakowski wrote: Hello, I would like to discuss making ServletConnection cache-aware. Daniel suggested earlier to utilize standard HTTP protocol concepts. I totally agree with his opinion and would like to propose solution, but first let's discuss requirements. Requirements Requirements that ServletConnection must meet are really simple: 1. ServletConnection should provide data that can be used for constructing small validation object. 2. ServletConnection should expose functionality for checking if previous response is still valid taking as input validation object only. 3. We would like ServletConnection to make as few as possible round trips in every situation it encounters. To satisfy these requirements I propose to use concept of HTTP conditional gets[1], more precisely If-Modified-Since request-header[2] field. This way we have following cases: * ServletConnection does not have information needed to create If-Modified-Since header, but response includes Last-Modified header and full content. Validity object can be created. * ServletConnection does not have information needed to create If-Modified-Since header and response does not include Last-Modified header but includes full content. Validity object cannot be created. * ServletConnection does have information needed to create If-Modified-Since. Resource has not been modified so 302 status code is returned as response and response does not include full content. Thus ServletConnection can just tell that content is still valid and can be fetched from cache. * ServletConnection does have information needed to create If-Modified-Since. Resource has been modified so 200 status code is returned as response and response includes full content. ServletConnection tells that cached content is invalid and returns fresh content. Requirements are satisfied: 1. Last-Modified header can be used to construct validation object. 2. Taking date from validation object enables ServletConnection to formulate conditional GET and then response HTTP code settles if resource is still valid. 3. In every case we have only one round trip. I agree with everything this far, it would also be nice to add ETag handling to it. The idea is that the servlet-service-fw should work with all kinds of servlets. And using Last-Modified and ETag headers are the two main ways to handle caching for HTTP, so by supporting both we make caching work for a larger share of the servlets. But the first priority is of course to make it work with the SitemapServlet. Using ETags together with If-None-Match is analogous to use Last-Modified together with If-Modified-Since as you described above. Some extra care is needed if the servlet called from the ServletConnection returns both an ETag and a Last-Modified header. Implementation proposal === We should start from making pipelines more HTTP-compliant. This demands taking If-Modified-Since headers into account and returning appropriate status code when caching pipeline is processed. Behavior of non-caching pipelines should not change. Agree. There is some getLastModified info on the cachedResponse object in the AbstractCachingProcessingPipeline. It doesn't seem like it is used for setting the Last-Modified header or used together with the If-Modified-Since header however. Also it might be that one could use the SourceValidity object (or maybe a hash key based on it) as an ETag. Then we should implement setIfModifiedSince and getIfModifiedSince from java.net.URLConnection and construct requests according to value of that property. Also getResponseCode method should be implemented. All changes proposed above will enable us to implement source validation of ServletSource very easily. Comments? Thoughts? Seem like the right direction to me. I can start implementing this as soon as we came with agreement on this. However, I would like to point out that I'll need some support to make changes in pipeline stuff. I've taken a look on code and not everything seems to be clear. Any volunteer on the board? ;-) I can't say that the pipeline code is entirely clear to me either. Maybe some of the original authors are still around? Last remark. I know that my English is quite poor and it could be that I do not express my thoughts clearly enough. I'm really working on it and you should not hesitate to ask when something is hard to understand. Don't worry about that. I don't have any problem to understand what you write. As soon as I had learned a little bit more about the HTTP protocol it was perfectly clear ;) /Daniel
Improving ServletConnection to make it cache-aware
Hello, I would like to discuss making ServletConnection cache-aware. Daniel suggested earlier to utilize standard HTTP protocol concepts. I totally agree with his opinion and would like to propose solution, but first let's discuss requirements. Requirements Requirements that ServletConnection must meet are really simple: 1. ServletConnection should provide data that can be used for constructing small validation object. 2. ServletConnection should expose functionality for checking if previous response is still valid taking as input validation object only. 3. We would like ServletConnection to make as few as possible round trips in every situation it encounters. To satisfy these requirements I propose to use concept of HTTP conditional gets[1], more precisely If-Modified-Since request-header[2] field. This way we have following cases: * ServletConnection does not have information needed to create If-Modified-Since header, but response includes Last-Modified header and full content. Validity object can be created. * ServletConnection does not have information needed to create If-Modified-Since header and response does not include Last-Modified header but includes full content. Validity object cannot be created. * ServletConnection does have information needed to create If-Modified-Since. Resource has not been modified so 302 status code is returned as response and response does not include full content. Thus ServletConnection can just tell that content is still valid and can be fetched from cache. * ServletConnection does have information needed to create If-Modified-Since. Resource has been modified so 200 status code is returned as response and response includes full content. ServletConnection tells that cached content is invalid and returns fresh content. Requirements are satisfied: 1. Last-Modified header can be used to construct validation object. 2. Taking date from validation object enables ServletConnection to formulate conditional GET and then response HTTP code settles if resource is still valid. 3. In every case we have only one round trip. Implementation proposal === We should start from making pipelines more HTTP-compliant. This demands taking If-Modified-Since headers into account and returning appropriate status code when caching pipeline is processed. Behavior of non-caching pipelines should not change. Then we should implement setIfModifiedSince and getIfModifiedSince from java.net.URLConnection and construct requests according to value of that property. Also getResponseCode method should be implemented. All changes proposed above will enable us to implement source validation of ServletSource very easily. Comments? Thoughts? I can start implementing this as soon as we came with agreement on this. However, I would like to point out that I'll need some support to make changes in pipeline stuff. I've taken a look on code and not everything seems to be clear. Any volunteer on the board? ;-) Last remark. I know that my English is quite poor and it could be that I do not express my thoughts clearly enough. I'm really working on it and you should not hesitate to ask when something is hard to understand. [1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.3 [2] http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.25 -- Grzegorz Kossakowski
