Re: [squid-users] Why does squid 2.6 stable 16 running in accelerator mode NEVER caches html requests ?
This response is just about your configuration, not the cache problem. Nothing to add on the cache problem. On ons, 2007-10-10 at 09:26 -0700, Raemaekers Mark wrote: What is wrong in my configuration that the accelerator does not cache the file, but always asks for it ? #[root] #[root]cat squid.conf #ACCELERATOR ### http_port 80 defaultsite=10.10.10.3 cache_peer omp.wc.be parent 80 0 no-query originserver defaultsite should be the official domain name. cache_peer should probably be using the IP, so you can change the DNS to point to the reverse proxy when things works.. ##ACLS### acl all src 0.0.0.0/0.0.0.0 acl port80 port 80 http_access allow port80 always_direct allow all Don't do always_direct in an accelerator. Forwarding is best done via cache_peer. Using always_direct like this leaves you vulnerable to being an open proxy. Regards Henrik signature.asc Description: This is a digitally signed message part
Re: [squid-users] Why does squid 2.6 stable 16 running in accelerator mode NEVER caches html requests ?
On tor, 2007-10-11 at 14:42 +1300, Amos Jeffries wrote: defaultsite should be the public FQDN of your website you also need either accel or vhost option to perform accelerator actions in 2.6+ defaultsite is quite sufficient.. but yes, should be the official site name. Regards Henrik signature.asc Description: This is a digitally signed message part
Re: [squid-users] Why does squid 2.6 stable 16 running in accelerator mode NEVER caches html requests ?
Thanks for your input, but unfortunately it is still not working. each time I launch : [curl-machine]#curl http://omp.wc.be/home.html I get in store.log : 1192090981.782 RELEASE 00 0018 CE042BF5E20CD75407DE72B1C8D0CA2C 200 1192093905 1192089432-1 text/html 18/18 GET http://omp.wc.be/home.html 1192090981.782 SWAPOUT 00 0019 EFE6712C44DC67B734C8EC1E9436327C 200 1192093910 1192089432-1 text/html 18/18 GET http://omp.wc.be/home.html ... and I see on the http server ther reqest coming in, so squid is still not caching. I do not understand why squid RELEASES the object first and then does SWAPOUT to disk. Here you see the result of curl -I http://omp.wc.be/home.html (also same behaviour on squid machine, meaning squid machine still contacts http server. [curl-machine]#curl -I http://omp.wc.be/home.html HTTP/1.0 200 OK Date: Thu, 11 Oct 2007 09:27:48 GMT Server: Apache/2.0.52 (Red Hat) Last-Modified: Thu, 11 Oct 2007 07:57:12 GMT ETag: 99c3-12-f3228600 Accept-Ranges: bytes Content-Length: 18 Content-Type: text/html; charset=UTF-8 X-Cache: MISS from omp.wc.be Via: 1.0 omp.wc.be:80 (squid/2.6.STABLE16) Connection: close tail -n 42 cache.log says : [EMAIL PROTECTED] logs]# tail -n 42 cache.log 2007/10/11 10:34:17| Starting Squid Cache version 2.6.STABLE16 for i686-pc-linux-gnu... 2007/10/11 10:34:17| Process ID 19543 2007/10/11 10:34:17| With 1024 file descriptors available 2007/10/11 10:34:17| Using epoll for the IO loop 2007/10/11 10:34:17| Performing DNS Tests... 2007/10/11 10:34:17| Successful DNS name lookup tests... 2007/10/11 10:34:17| DNS Socket created at 0.0.0.0, port 32776, FD 5 2007/10/11 10:34:17| Adding nameserver 127.0.0.1 from squid.conf 2007/10/11 10:34:17| Unlinkd pipe opened on FD 9 2007/10/11 10:34:17| Swap maxSize 102400 KB, estimated 7876 objects 2007/10/11 10:34:17| Target number of buckets: 393 2007/10/11 10:34:17| Using 8192 Store buckets 2007/10/11 10:34:17| Max Mem size: 51200 KB 2007/10/11 10:34:17| Max Swap size: 102400 KB 2007/10/11 10:34:17| Rebuilding storage in /usr/local/squid/var/cache (CLEAN) 2007/10/11 10:34:17| Using Least Load store dir selection 2007/10/11 10:34:17| Current Directory is /usr/local/squid/var/logs 2007/10/11 10:34:17| Loaded Icons. 2007/10/11 10:34:17| Accepting accelerated HTTP connections at 0.0.0.0, port 80, FD 11. 2007/10/11 10:34:17| Accepting ICP messages at 0.0.0.0, port 3130, FD 12. 2007/10/11 10:34:17| Accepting SNMP messages on port 3401, FD 13. 2007/10/11 10:34:17| WCCP Disabled. 2007/10/11 10:34:17| Configuring Parent 10.10.10.3/80/0 2007/10/11 10:34:17| Ready to serve requests. 2007/10/11 10:34:18| Done reading /usr/local/squid/var/cache swaplog (1 entries) 2007/10/11 10:34:18| Finished rebuilding storage from disk. 2007/10/11 10:34:18| 1 Entries scanned 2007/10/11 10:34:18| 0 Invalid entries. 2007/10/11 10:34:18| 0 With invalid flags. 2007/10/11 10:34:18| 1 Objects loaded. 2007/10/11 10:34:18| 0 Objects expired. 2007/10/11 10:34:18| 0 Objects cancelled. 2007/10/11 10:34:18| 0 Duplicate URLs purged. 2007/10/11 10:34:18| 0 Swapfile clashes avoided. 2007/10/11 10:34:18| Took 0.3 seconds ( 3.8 objects/sec). 2007/10/11 10:34:18| Beginning Validation Procedure 2007/10/11 10:34:18| Completed Validation Procedure 2007/10/11 10:34:18| Validated 1 Entries 2007/10/11 10:34:18| store_swap_size = 4k 2007/10/11 10:34:18| storeLateRelease: released 0 objects Very strange is that : 1. apparantly 1 Object is loaded (and I only ask for 1 object) home.html and I am the only user on this test system. this indicates to me that the page is cached, but the squid allways fetches the object from the http sever ... Using following conf (using comments by Amos) ... #ACCELERATOR ### http_port 80 accel defaultsite=omp.wc.be cache_peer 10.10.10.3 parent 80 0 no-query originserver name=webserver ##ACLS### acl all src all cache_peer_domain webserver omp.wc.be acl mydomain dstdomain omp.wc.be http_access allow mydomain never_direct allow mydomain http_access deny all #CACHING RULES ### acl QUERY urlpath_regex cgi-bin \? cache deny QUERY cache allow all refresh_pattern . 1440 20% 4320 #MISCELLANIOUS ##3 visible_hostname omp.wc.be cache_effective_user squid #cache_effective_group squid dns_testnames omp.wc.be dns_nameservers 127.0.0.1 cache_mem 50 MB [EMAIL PROTECTED] etc]# ... I allready tried the following : 1. removeing the cache_dir (squid -k shutdown ; rm -fr /usr/local/squid/var/cache/* ; squid -z) and recreating it; start squid 2. added the dns_nameservers 127.0.0.1 statement, bcc otherwise I get warning at startup that there are no DNS servers specified in /etc/resolv.conf /etc/resolv.conf only contains files an I am using /etc/hosts for dns lookup on the squid machine. cat /etc/hosts looks like this : 127.0.0.1 rack4top localhost.localdomain localhost 13.1.1.1
Re: [squid-users] Why does squid 2.6 stable 16 running in accelerator mode NEVER caches html requests ?
I found the cause of my problem !! Apparantly curl uses BY DEFAULT the pragma : no-cache option ... This means indeed that the WC never caches the pages. By using curl-loader (who does not use the no-cache option) I noticed that indeed object was cached ! so I compared capture of both request and then I noticed the pragma option ... I spend 4 days on this default value and I want to thank you for your effort ! --- Raemaekers Mark [EMAIL PROTECTED] wrote: Thanks for your input, but unfortunately it is still not working. each time I launch : [curl-machine]#curl http://omp.wc.be/home.html I get in store.log : 1192090981.782 RELEASE 00 0018 CE042BF5E20CD75407DE72B1C8D0CA2C 200 1192093905 1192089432-1 text/html 18/18 GET http://omp.wc.be/home.html 1192090981.782 SWAPOUT 00 0019 EFE6712C44DC67B734C8EC1E9436327C 200 1192093910 1192089432-1 text/html 18/18 GET http://omp.wc.be/home.html ... and I see on the http server ther reqest coming in, so squid is still not caching. I do not understand why squid RELEASES the object first and then does SWAPOUT to disk. Here you see the result of curl -I http://omp.wc.be/home.html (also same behaviour on squid machine, meaning squid machine still contacts http server. [curl-machine]#curl -I http://omp.wc.be/home.html HTTP/1.0 200 OK Date: Thu, 11 Oct 2007 09:27:48 GMT Server: Apache/2.0.52 (Red Hat) Last-Modified: Thu, 11 Oct 2007 07:57:12 GMT ETag: 99c3-12-f3228600 Accept-Ranges: bytes Content-Length: 18 Content-Type: text/html; charset=UTF-8 X-Cache: MISS from omp.wc.be Via: 1.0 omp.wc.be:80 (squid/2.6.STABLE16) Connection: close tail -n 42 cache.log says : [EMAIL PROTECTED] logs]# tail -n 42 cache.log 2007/10/11 10:34:17| Starting Squid Cache version 2.6.STABLE16 for i686-pc-linux-gnu... 2007/10/11 10:34:17| Process ID 19543 2007/10/11 10:34:17| With 1024 file descriptors available 2007/10/11 10:34:17| Using epoll for the IO loop 2007/10/11 10:34:17| Performing DNS Tests... 2007/10/11 10:34:17| Successful DNS name lookup tests... 2007/10/11 10:34:17| DNS Socket created at 0.0.0.0, port 32776, FD 5 2007/10/11 10:34:17| Adding nameserver 127.0.0.1 from squid.conf 2007/10/11 10:34:17| Unlinkd pipe opened on FD 9 2007/10/11 10:34:17| Swap maxSize 102400 KB, estimated 7876 objects 2007/10/11 10:34:17| Target number of buckets: 393 2007/10/11 10:34:17| Using 8192 Store buckets 2007/10/11 10:34:17| Max Mem size: 51200 KB 2007/10/11 10:34:17| Max Swap size: 102400 KB 2007/10/11 10:34:17| Rebuilding storage in /usr/local/squid/var/cache (CLEAN) 2007/10/11 10:34:17| Using Least Load store dir selection 2007/10/11 10:34:17| Current Directory is /usr/local/squid/var/logs 2007/10/11 10:34:17| Loaded Icons. 2007/10/11 10:34:17| Accepting accelerated HTTP connections at 0.0.0.0, port 80, FD 11. 2007/10/11 10:34:17| Accepting ICP messages at 0.0.0.0, port 3130, FD 12. 2007/10/11 10:34:17| Accepting SNMP messages on port 3401, FD 13. 2007/10/11 10:34:17| WCCP Disabled. 2007/10/11 10:34:17| Configuring Parent 10.10.10.3/80/0 2007/10/11 10:34:17| Ready to serve requests. 2007/10/11 10:34:18| Done reading /usr/local/squid/var/cache swaplog (1 entries) 2007/10/11 10:34:18| Finished rebuilding storage from disk. 2007/10/11 10:34:18| 1 Entries scanned 2007/10/11 10:34:18| 0 Invalid entries. 2007/10/11 10:34:18| 0 With invalid flags. 2007/10/11 10:34:18| 1 Objects loaded. 2007/10/11 10:34:18| 0 Objects expired. 2007/10/11 10:34:18| 0 Objects cancelled. 2007/10/11 10:34:18| 0 Duplicate URLs purged. 2007/10/11 10:34:18| 0 Swapfile clashes avoided. 2007/10/11 10:34:18| Took 0.3 seconds ( 3.8 objects/sec). 2007/10/11 10:34:18| Beginning Validation Procedure 2007/10/11 10:34:18| Completed Validation Procedure 2007/10/11 10:34:18| Validated 1 Entries 2007/10/11 10:34:18| store_swap_size = 4k 2007/10/11 10:34:18| storeLateRelease: released 0 objects Very strange is that : 1. apparantly 1 Object is loaded (and I only ask for 1 object) home.html and I am the only user on this test system. this indicates to me that the page is cached, but the squid allways fetches the object from the http sever ... Using following conf (using comments by Amos) ... #ACCELERATOR ### http_port 80 accel defaultsite=omp.wc.be cache_peer 10.10.10.3 parent 80 0 no-query originserver name=webserver ##ACLS### acl all src all cache_peer_domain webserver omp.wc.be acl mydomain dstdomain omp.wc.be http_access allow mydomain never_direct allow mydomain http_access deny all #CACHING RULES ### acl QUERY urlpath_regex cgi-bin \? cache deny QUERY cache allow all refresh_pattern . 1440 20% 4320 #MISCELLANIOUS ##3 visible_hostname omp.wc.be cache_effective_user squid #cache_effective_group squid
[squid-users] Why does squid 2.6 stable 16 running in accelerator mode NEVER caches html requests ?
Dear squid-cache.org, I have three RHEL4 servers : 1. configured with squid 2.6 Version 2.6.STABLE16 in accelerator mode 2. configured with (apache) httpd daemon 3. this machine has curl installed (to generate http requests) Each time I do curl http://omp.wc.be/home.html, I notice that the WC ALLWAYS sends the reqeust to the http server. It does NOT cache my home.html page. If I remove the always_direct allow all statement then I get the error back from WC that it is unable to forward request. I have : cache allow all statement. refresh_pattern but even if I comment it, I have the same issue. What is wrong in my configuration that the accelerator does not cache the file, but always asks for it ? #[root] #[root]cat squid.conf #ACCELERATOR ### http_port 80 defaultsite=10.10.10.3 cache_peer omp.wc.be parent 80 0 no-query originserver ##ACLS### acl all src 0.0.0.0/0.0.0.0 acl port80 port 80 http_access allow port80 always_direct allow all #CACHING RULES ### acl QUERY urlpath_regex cgi-bin \? cache deny QUERY cache allow all refresh_pattern . 1440 20% 4320 #MISCELLANIOUS ##3 visible_hostname omp.wc.be cache_effective_user squid cache_effective_group squid dns_testnames omp.wc.be cache_mem 50 MB #[root] #[root] #[root] note : 10.10.10.3 is the IP of the http server omp.wc.be is referenced as 10.10.10.3 by DNS. on curl-machine omp.wc.be is resolved as 12.1.1.1 (=listening interface on squid) note : home.html is only a one-line html page with no cgi, ... in it ! Many thanks in advance ! Boardwalk for $500? In 2007? Ha! Play Monopoly Here and Now (it's updated for today's economy) at Yahoo! Games. http://get.games.yahoo.com/proddesc?gamekey=monopolyherenow
Re: [squid-users] Why does squid 2.6 stable 16 running in accelerator mode NEVER caches html requests ?
Dear squid-cache.org, I have three RHEL4 servers : 1. configured with squid 2.6 Version 2.6.STABLE16 in accelerator mode 2. configured with (apache) httpd daemon 3. this machine has curl installed (to generate http requests) Each time I do curl http://omp.wc.be/home.html, I notice that the WC ALLWAYS sends the reqeust to the http server. It does NOT cache my home.html page. If I remove the always_direct allow all statement then I get the error back from WC that it is unable to forward request. I have : cache allow all statement. refresh_pattern but even if I comment it, I have the same issue. What is wrong in my configuration that the accelerator does not cache the file, but always asks for it ? #[root] #[root]cat squid.conf #ACCELERATOR ### http_port 80 defaultsite=10.10.10.3 defaultsite should be the public FQDN of your website you also need either accel or vhost option to perform accelerator actions in 2.6+ So; http_port 80 accel defaultsite=omp.wc.be cache_peer omp.wc.be parent 80 0 no-query originserver Probably better off NOT being the public name. No need for DNS trickery then: cache_peer 10.10.10.3 parent 80 0 no-query originserver name=webserver (you can now define omp.wc.be as the squid interface, no exceptions needed.) ##ACLS### acl all src 0.0.0.0/0.0.0.0 acl all src all # nicer way of saying it. acl port80 port 80 http_access allow port80 always_direct allow all All the above do are make your proxy an open proxy on port 80. I wouldn't be surprised to find general web requests for random spam domains logged in access.log Better to use some rules specifically limiting to your domain, and to pass to named peer instead of general upstreams. cache_peer_domain webserver omp.wc.be acl mydomain dstdomain omp.wc.be http_access allow mydomain never_direct allow mydomain (With just that, anybody, internal or external can get to your reverse-hosted domain through squid.) # ... other rules http_access deny all #CACHING RULES ### acl QUERY urlpath_regex cgi-bin \? cache deny QUERY cache allow all refresh_pattern . 1440 20% 4320 #MISCELLANIOUS ##3 visible_hostname omp.wc.be cache_effective_user squid cache_effective_group squid Better to leave group unset and allow OS security policy to handle the user/group permissions. dns_testnames omp.wc.be cache_mem 50 MB #[root] #[root] #[root] note : 10.10.10.3 is the IP of the http server omp.wc.be is referenced as 10.10.10.3 by DNS. on curl-machine omp.wc.be is resolved as 12.1.1.1 (=listening interface on squid) note : home.html is only a one-line html page with no cgi, ... in it ! Many thanks in advance ! Amos
