Re: mod_cache caching the 301 Moved Permanently
On Thu, Jul 07, 2005 at 09:22:25PM +0200, [EMAIL PROTECTED] wrote: Have you checked http://mail-archives.apache.org/mod_mbox/httpd-dev/200504.mbox/[EMAIL PROTECTED] ? It contains a small patch which was not discussed any further here. Hi, i tried your patch today. Unfortunately cache-handle-cache_obj-info.status isn't set during cache_out_filter(), but cache-handle-status is. (It gets set in mod_disk_cache.c:499 during recall_headers().) I tried to use that and it seems to work. However, not knowing much of how the whole thing fits together, i do not claim that this is the solution. An updated version of your patch below. Thanks Regards Hansjoerg --- mod_cache.c.orig2005-04-11 17:47:03.0 +0200 +++ mod_cache.c 2005-07-11 16:52:35.913264943 +0200 @@ -219,7 +219,10 @@ ap_log_error(APLOG_MARK, APLOG_DEBUG, APR_SUCCESS, r-server, cache: running CACHE_OUT filter); - + +/* restore status of cached response */ +r-status = cache-handle-status; + /* recall_headers() was called in cache_select_url() */ cache-provider-recall_body(cache-handle, r-pool, bb); -- IT ServicesUniversity of Innsbruck CFB4 D6E7 33F4 34C0 18B9 6661 E355 4337 3F8B D9C2 http://purl.org/net/hansjoerg.pehofer/public_key signature.asc Description: Digital signature
Re: mod_cache caching the 301 Moved Permanently
Hansjoerg Pehofer wrote: On Thu, Jul 07, 2005 at 09:22:25PM +0200, [EMAIL PROTECTED] wrote: Have you checked http://mail-archives.apache.org/mod_mbox/httpd-dev/200504.mbox/[EMAIL PROTECTED] ? It contains a small patch which was not discussed any further here. Hi, i tried your patch today. Unfortunately cache-handle-cache_obj-info.status isn't set during cache_out_filter(), but cache-handle-status is. (It gets set in mod_disk_cache.c:499 during recall_headers().) Many thanks for the feedback and the corrected patch. I checked my patch again and must admit that it does only make sense on Apache 2.1. For Apache 2.0.x your patch is the correct one. Question to the mod_cache gurus: While mod_mem_cache restores the status during recall_headers in Apache 2.0.54 it seems that mod_mem_cache does not do this in Apache 2.1.x (Maybe fixed meanwhile my latest revision of mod_mem_cache.c is 178625 and viewcvs is currently down), whereas mod_disk_cache still does this in Apache 2.1.x in file_cache_recall_mydata which is called by open_entity. Is this behaviour intended? Regards Rüdiger [..cut..]
Re: mod_cache caching the 301 Moved Permanently
[EMAIL PROTECTED] wrote: Hansjoerg Pehofer wrote: On Thu, Jul 07, 2005 at 09:22:25PM +0200, [EMAIL PROTECTED] wrote: Have you checked http://mail-archives.apache.org/mod_mbox/httpd-dev/200504.mbox/[EMAIL PROTECTED] ? It contains a small patch which was not discussed any further here. Hi, i tried your patch today. Unfortunately cache-handle-cache_obj-info.status isn't set during cache_out_filter(), but cache-handle-status is. (It gets set in mod_disk_cache.c:499 during recall_headers().) Many thanks for the feedback and the corrected patch. I checked my patch again and must admit that it does only make sense on Apache 2.1. For Apache 2.0.x your patch is the correct one. Question to the mod_cache gurus: While mod_mem_cache restores the status during recall_headers in Apache 2.0.54 it seems that mod_mem_cache does not do this in Apache 2.1.x (Maybe fixed meanwhile my latest revision of mod_mem_cache.c is 178625 and viewcvs is currently down), whereas mod_disk_cache still does this in Apache 2.1.x in file_cache_recall_mydata which is called by open_entity. Is this behaviour intended? 2.0 is quite a bit behind when it comes to mod_cache. mod_mem_cache has gotten a lot less attention in 2.1 as mod_disk_cache has. That said, in 2.1, I think we should restore status in mod_cache, not in the cache providers. Ofcourse, if that isn't possible for whatever reason, the restoration has to be done in _all_ cache providers. The above might explain the discrepancies... Sander
Re: mod_cache caching the 301 Moved Permanently
Hi, it has been some time since the original thread. This is in reply to [1]. Sander Striker wrote: [EMAIL PROTECTED] wrote: The problem seems to be, that the proxied backend server that is cached via mod_disk_cache originally delivers HTTP status 301 and the Location http://www.beach-clothing.com/where-to-buy/, but once cached mod_disk_cache delivers HTTP status 200 instead of 301 (but correctly redelivering the Location header). I have not proved this for myself so far, but this seems the problem to me. This wouldn't surprise me one bit. The 2.1 branch has seen quite a bit of churn in this area. Any chance you could give 2.1 a go and see if that works correctly? It is the same problem as described in issue 32226 [2]. Unfortunately, the described behavior is still present in both httpd 2.0.54 and 2.1.6-alpha with mod_disk_cache active. I can not reproduce it with 2.1.6-alpha and mod_mem_cache. Tested with the following setup: Frontend: httpd 2.1.6-alpha on 127.0.0.1, cache, disk_cache and proxy* modules loaded, out of the box httpd.conf plus: CacheRoot /cache/alpha CacheEnable disk / CacheDirLevels 1 CacheDirLength 2 ProxyPass/ http://backend.tld/ ProxyPassReverse / http://backend.tld/ Backend: Apache 1.3, a file cachetest/gonk/index.html relative to DocumentRoot. A first request to the Proxy (GET /cachetest/gonk) correctly responds in 301ing to /cachetest/gonk/ first -- and then getting the index document. There are 2 .data/.header couples in the cachedir at this point (the redirect-message and the index document ). A second request with a second client, or if the browser's cache has been cleared, returns the cached Moved Permanently message with status 200. This message contains a link in the backend's view of the URLs (as discussed in the original thread). The /cache/alpha/Kx/[EMAIL PROTECTED] Cache File contains: [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@@-a89Nû^C^@@Žô8C[EMAIL PROTECTED]89[EMAIL PROTECTED]89[EMAIL PROTECTED]/cachetest/gonk?Date: Thu, 07 Jul 2005 12:34 :21 GMT Server: Apache Location: http://127.0.0.1/cachetest/gonk/ Content-Type: text/html; charset=iso-8859-1 Expires: Thu, 07 Jul 2005 12:35:21 GMT Host: 127.0.0.1 User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050513 Debian/1.7.8-1 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: de-at,de;q=0.8,en;q=0.5,it;q=0.3 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Max-Forwards: 10 X-Forwarded-For: 127.0.0.1 X-Forwarded-Host: 127.0.0.1 X-Forwarded-Server: 127.0.0.1 [3], which is an attachment to [2] contains a fragment that keeps mod_cache.c from handling redirects at all -- if i understood it right, however it works for us. This is probably only the second best i approach, since 301 is cacheable. Regards, Hansjörg References: [1] http://mail-archives.apache.org/mod_mbox/httpd-dev/200504.mbox/[EMAIL PROTECTED] [2] http://issues.apache.org/bugzilla/show_bug.cgi?id=32226 [3] http://issues.apache.org/bugzilla/attachment.cgi?id=13433 -- IT ServicesUniversity of Innsbruck CFB4 D6E7 33F4 34C0 18B9 6661 E355 4337 3F8B D9C2 http://purl.org/net/hansjoerg.pehofer/public_key signature.asc Description: Digital signature
Re: mod_cache caching the 301 Moved Permanently
Have you checked http://mail-archives.apache.org/mod_mbox/httpd-dev/200504.mbox/[EMAIL PROTECTED] ? It contains a small patch which was not discussed any further here. Regards Rüdiger Hansjoerg Pehofer wrote: Hi, it has been some time since the original thread. This is in reply to [1].
Re: mod_cache caching the 301 Moved Permanently
At 22/04/2005 11:21 (), Olaf van der Spek wrote: On 4/22/05, Justin Erenkrantz [EMAIL PROTECTED] wrote: On Thu, Apr 21, 2005 at 10:04:54AM +0530, Devendra Singh wrote: Hi, I am writing to the Developer List because I did not get any response on the Users List and thought that the topic might be relevant to the dev list. If a request comes for a directory w/o trailing slash, it gets cached and the subsequent requests see: Moved Permanently The document has moved here for try to access the URL: http://www.beach-clothing.com/where-to-buy I don't get it. What's your problem? -- justin The 'here' link is to http://www.beach-clothing.com:8080/where-to-buy/ while he wants it do be to http://www.beach-clothing.com/where-to-buy/ When the Front-End Apache (running on port 80) gets a request for http://www.beach-clothing.com/where-to-buy (notice the lack of trailing slash), it issues a 301 Moved Permanently to http://www.beach-clothing.com/where-to-buy/ (notice the added trailing slash). The Front-End Apache proxies this request to Back-End Apache running on 8080 which becomes http://www.beach-clothing.com:8080/where-to-buy/ and this is Cached by the Front-End Apache. The subsequent request for this URL gives http://www.beach-clothing.com:8080/where-to-buy/ from the Cache. This is not as expected. Thus I feel that either 301 responses should not be proxied or they should not be Cached at all. Thanks. Devendra Singh
Re: mod_cache caching the 301 Moved Permanently
Sander Striker wrote: [EMAIL PROTECTED] wrote: The problem seems to be, that the proxied backend server that is cached via mod_disk_cache originally delivers HTTP status 301 and the Location http://www.beach-clothing.com/where-to-buy/, but once cached mod_disk_cache delivers HTTP status 200 instead of 301 (but correctly redelivering the Location header). I have not proved this for myself so far, but this seems the problem to me. This wouldn't surprise me one bit. The 2.1 branch has seen quite a bit of churn in this area. Any chance you could give 2.1 a go and see if that works correctly? Sorry not enough time right now to check this on the living object. But I think what is missing is something like the following (untested) patch (done against 2.0.54, but code in trunk is the same at this part): --- mod_cache.c.orig2005-04-11 17:47:03.0 +0200 +++ mod_cache.c 2005-04-22 23:43:19.0 +0200 @@ -220,6 +220,8 @@ static int cache_out_filter(ap_filter_t ap_log_error(APLOG_MARK, APLOG_DEBUG, APR_SUCCESS, r-server, cache: running CACHE_OUT filter); +/* restore status of cached response */ +r-status = cache-handle-cache_obj-info.status; /* recall_headers() was called in cache_select_url() */ cache-provider-recall_body(cache-handle, r-pool, bb); I found no place in the code where the saved status code is restored to the response, so I think this could be the right place to do so. Could somebody crosscheck who is more familar with the code? Sander? Justin? Sander Regards Rüdiger
Re: mod_cache caching the 301 Moved Permanently
Olaf van der Spek wrote: On 4/22/05, Justin Erenkrantz [EMAIL PROTECTED] wrote: [..cut..] I don't get it. What's your problem? -- justin The 'here' link is to http://www.beach-clothing.com:8080/where-to-buy/ while he wants it do be to http://www.beach-clothing.com/where-to-buy/ His problem is a different one. If you type http://www.beach-clothing.com/where-to-buy (without the slash) you have the following headers (first my request / second server reply): GET /where-to-buy HTTP/1.1 Host: www.beach-clothing.com User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/20050319 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plai n;q=0.8,image/png,*/*;q=0.5 Accept-Language: de,en;q=0.8,de-de;q=0.5,en-gb;q=0.3 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive HTTP/1.x 200 OK Date: Fri, 22 Apr 2005 20:39:58 GMT Server: Apache Content-Type: text/html; charset=iso-8859-1 Location: http://www.beach-clothing.com/where-to-buy/ Cache-Control: max-age=432000 Expires: Tue, 26 Apr 2005 04:58:38 GMT Content-Length: 256 Age: 142880 Keep-Alive: timeout=3, max=20 Connection: Keep-Alive The problem seems to be, that the proxied backend server that is cached via mod_disk_cache originally delivers HTTP status 301 and the Location http://www.beach-clothing.com/where-to-buy/, but once cached mod_disk_cache delivers HTTP status 200 instead of 301 (but correctly redelivering the Location header). I have not proved this for myself so far, but this seems the problem to me. I think the difference between the here link and the Location header can be explained by the ProxyPassReverse directive which rewrites the Location header, but not the HTML code in the error page. Regards Rüdiger
Re: mod_cache caching the 301 Moved Permanently
[EMAIL PROTECTED] wrote: The problem seems to be, that the proxied backend server that is cached via mod_disk_cache originally delivers HTTP status 301 and the Location http://www.beach-clothing.com/where-to-buy/, but once cached mod_disk_cache delivers HTTP status 200 instead of 301 (but correctly redelivering the Location header). I have not proved this for myself so far, but this seems the problem to me. This wouldn't surprise me one bit. The 2.1 branch has seen quite a bit of churn in this area. Any chance you could give 2.1 a go and see if that works correctly? Sander
Re: mod_cache caching the 301 Moved Permanently
At 21/04/05 10:04 (), Devendra Singh wrote:
Hi,
I am writing to the Developer List because I did not get any response on
the Users List and thought that the topic might be relevant to the dev list.
If a request comes for a directory w/o trailing slash, it gets cached and
the subsequent requests see:
Moved Permanently
The document has moved here
for try to access the URL:
http://www.beach-clothing.com/where-to-buy
The configuration has two apache 2.0.53. The one running on port 80 has
mod_disk_cache which sends requests to 8080 via ProxyPassReverse.
Here is my Virtual Host Config:
Apache on Port 80:
VirtualHost 66.235.181.69
ServerAlias beach-clothing.com www.beach-clothing.com
ServerName www.beach-clothing.com
ServerAdmin [EMAIL PROTECTED]
DocumentRoot /var/www/public_html/beach-clothing
ServerPath /beach-clothing
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.beach-clothing\.com [NC]
RewriteCond %{HTTP_HOST} !^$
RewriteRule ^/(.*) http://www.beach-clothing.com/$1 [L,R=301]
#Enable Caching
CacheDisable /index.html
CacheEnable disk /
RewriteCond %{SERVER_PORT} ^25$
RewriteRule /* - [F]
RewriteRule ^(.*)\/$ $1/index.html [P]
RewriteRule \.(gif|jpg|png|txt|css|js|ico|swf)$ - [last]
RewriteRule ^/(.*)$ http://www.beach-clothing.com:8080/$1 [proxy]
ProxyPassReverse / http://www.beach-clothing.com:8080/
/VirtualHost
Apache on port 8080:
VirtualHost 66.235.181.69
ServerAlias beach-clothing.com www.beach-clothing.com
ServerName www.beach-clothing.com
ServerAdmin [EMAIL PROTECTED]
DocumentRoot /var/www/public_html/beach-clothing
#Enable Caching for 48 Hours
Location /cgi
ExpiresActive off
/Location
FilesMatch \.(mp|pl|cgi)$
ExpiresActive off
/FilesMatch
Location /
ExpiresActive on
ExpiresByType text/html A432000
/Location
/VirtualHost
UseCanonicalName is off into the Server Config Section of both the Apache.
How do I solve this problem? If anyone has any clue / pointer?
Hi,
Sorry, replying my own post.
I modified the modules/experimental/mod_cache.c (around line 374) to
eliminate the caching of HTTP_MOVED_PERMANENTLY and the problem was solved.
/*
* what responses should we not cache?
*
* At this point we decide based on the response headers whether it
* is appropriate _NOT_ to cache the data from the server. There are
* a whole lot of conditions that prevent us from caching this data.
* They are tested here one by one to be clear and unambiguous.
*/
if (r-status != HTTP_OK r-status != HTTP_NON_AUTHORITATIVE
r-status != HTTP_MULTIPLE_CHOICES
r-status != HTTP_MOVED_PERMANENTLY -- Remove this line.
r-status != HTTP_NOT_MODIFIED) {
reason = apr_psprintf(p, Response status %d, r-status);
}
Please comment, is this a right approach?
Thanks.
Devendra Singh
Re: mod_cache caching the 301 Moved Permanently
On Thu, Apr 21, 2005 at 10:04:54AM +0530, Devendra Singh wrote: Hi, I am writing to the Developer List because I did not get any response on the Users List and thought that the topic might be relevant to the dev list. If a request comes for a directory w/o trailing slash, it gets cached and the subsequent requests see: Moved Permanently The document has moved here for try to access the URL: http://www.beach-clothing.com/where-to-buy I don't get it. What's your problem? -- justin
Re: mod_cache caching the 301 Moved Permanently
On 4/22/05, Justin Erenkrantz [EMAIL PROTECTED] wrote: On Thu, Apr 21, 2005 at 10:04:54AM +0530, Devendra Singh wrote: Hi, I am writing to the Developer List because I did not get any response on the Users List and thought that the topic might be relevant to the dev list. If a request comes for a directory w/o trailing slash, it gets cached and the subsequent requests see: Moved Permanently The document has moved here for try to access the URL: http://www.beach-clothing.com/where-to-buy I don't get it. What's your problem? -- justin The 'here' link is to http://www.beach-clothing.com:8080/where-to-buy/ while he wants it do be to http://www.beach-clothing.com/where-to-buy/
