Re: [squid-users] Trying to improve the Byte Hit Ratio, any tips ?
Hello squid users, I am green when the word squid comes. I have a box running suse 10.3 and i need someone to help me configure squid well. am green at this. Please someone with a few minutes to help me, i will be glad On 1/6/09, Amos Jeffries squ...@treenet.co.nz wrote: Vianney Lejeune wrote: Hello, I'm trying to improve the Byte Hit Ratio of SquidCache on my network. There is 220 computers in the LAN, using internet on a general usage basis. The maximum bandwidth is 4Mbps in/out, the total amount of data is estimated to be 30 to 60 Gbytes daily. This is the report from cachemgr: = Average HTTP requests per minute since start:1023.9 Average ICP messages per minute since start:0.0 Select loop called: 1208577 times, 5.619 ms avg Cache information for squid: Request Hit Ratios:5min: 37.9%, 60min: 41.1% Byte Hit Ratios:5min: 13.2%, 60min: 13.8% (It's quite low, these values are usual) Request Memory Hit Ratios:5min: 2.0%, 60min: 2.6% (I rebooted the server 3 hours ago, this can explain these low values) Request Disk Hit Ratios:5min: 41.3%, 60min: 36.3% Storage Swap size:27654312 KB Storage Mem size:190364 KB Mean Object Size:29.65 KB Requests given to unlinkd:33035 Median Service Times (seconds) 5 min60 min: HTTP Requests (All): 0.23230 0.46965 Cache Misses: 0.35832 0.72387 Cache Hits:0.19742 0.35832 Near Hits: 0.20843 0.55240 Not-Modified Replies: 0.03829 0.05331 DNS Lookups: 0.00094 0.00779 ICP Queries: 0.0 0.0 = This is my squid.conf file: = http_port 3128 transparent hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex cgi-bin \? cache deny QUERY Without cache peers you can drop the above QEURY acl. That will raise both hit ratios on semi-dynamic objects. BUT, see addition to refresh_pattern below... acl apache rep_header Server ^Apache broken_vary_encoding allow apache maximum_object_size 128 MB Re: the above maximum. There may be huge objects going through that can be cached. cache_mem 250 MB maximum_object_size_in_memory 50 KB memory, memory, memory. The more you can throw at the problem the more objects can be kept and served while hot. Squid with 64-bit can easily handle many GBs of memory cache. (at cost of slow shutdown when it saves the hottest to disk for the next round.) cache_replacement_policy heap LFUDA Been a while since I looked at these, to maximize bytes you want the policy that looks at object size as well as 'coldness'. To remove the smaller cool objects before the larger equally cool ones. cache_dir ufs /data/spool/squid 3 16 256 Your cache dir is only 30GB. Thats one days traffic or less by your above statements. For good hit ratios you may need at least 7 days, preferrably as close to 30 as possible. Depending on your OS, AUFS(Linux) or diskd(*BSD) may prove much faster access than UFS. access_log none cache_log none The above is generating log file named none. Would be more useful to set debug_options ALL,0. If you really don't want to know about the critical problems that do happen then set filename to /dev/null as well. cache_store_log none log_ip_on_direct off hosts_file /etc/hosts refresh_pattern ^ftp:144020%10080 refresh_pattern ^gopher:14400%1440 without QUERY acl above, you wil need this right here in the pattern order: refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern .020%4320 quick_abort_min 0 KB quick_abort_max 0 KB range_offset_limit 0 KB Be careful, but you may want to play at setting these to continue downloads. (quick_abort -1 KB) That will cause all partial and restarted downloads to become HIT later. At risk of some wastage. half_closed_clients off shutdown_lifetime 0 seconds acl all src 0.0.0.0/0.0.0.0 acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443# https acl SSL_ports port 563# snews acl SSL_ports port 873# rsync acl Safe_ports port 80# http acl Safe_ports port 21# ftp acl Safe_ports port 443# https acl Safe_ports port 70# gopher acl Safe_ports port 210# wais acl Safe_ports port 1025-65535# unregistered ports acl Safe_ports port 280# http-mgmt acl Safe_ports port 488# gss-http acl Safe_ports port 591# filemaker acl Safe_ports port 777# multiling http acl Safe_ports port 631# cups acl Safe_ports port 873# rsync acl Safe_ports port 901# SWAT acl purge method PURGE acl CONNECT method CONNECT acl ReseauLocal src 10.0.0.0/16 http_access allow manager localhost http_access deny manager http_access allow purge localhost http_access deny purge http_access allow localhost http_access
[squid-users] Squid store open errors
Hi, I got several errors in cache log like the following: storeAufsOpenDone: (1) Operation not permitted /var/spool/squid/12/28/0012287A Sometimes I get the same but with File not found error. I suppose it's related to non clean sytem restart which omitted some object from swap.state and from Squid. Is there any way to fix this problems after non clean restart ? Thanks Itzcak
RE: [squid-users] NTLM and transparent/interception confusion
That's exactly what I opted for... I configured WPAD which should work with the majority of browsers out there. And we also authenticate against the hardware (another LDAP connection) to even connect to the open wireless. -Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Monday, January 05, 2009 10:18 PM To: Johnson, S Cc: Kinkie; Guido Serassio; squid-users@squid-cache.org Subject: Re: [squid-users] NTLM and transparent/interception confusion Johnson, S wrote: Keep in mind, group policies cannot always be used as in our environment. We are a K-12 education and are mandated by federal law to monitor and protect student access to the internet. We are now allowing students to bring their own notebooks in on a trial basis (to be permanent after this summer when we work out the bugs) to do research on their own computers. We have to monitor their access to the internet and deny bad sites, again mandated by federal law. So their authentication mechanism is AD/LDAP to their user ID set up for them to access network resources on the network. Since their computers are not on our domain (nor do we want them to be), we cannot push group policies down to their computer. In that case your best bet would be to lock down general port-80 access to them entirely. Using WPAD 'auto-detect' or with students setting browsers set manually. That will go a long way toward blocking risky behavior by malware on mobile devices. Second best after that would be to setup some helper where they can authenticate against some other system and the helper permits their requests past Squid for a time. This provides almost no protection from malware once the student is browsing a legit session. Amos The solution Bluecoat had was very secure, but again their devices are about $50,000usd / device. As an education provider, that money is hard to come by especially when we would need 3 devices for the load. Their authentication mechanism is SOX (sarbane oxley) tested and compliant. It also works with any computer outbound to the internet. There's no proxy configuration to worry about; it's all done at the proxy. Granted, I used WCCP to configure this on Bluecoat which allowed me a lot of flexibility to add in multiple proxies with ease (and the users would never know the difference). sj -Original Message- From: Kinkie [mailto:gkin...@gmail.com] Sent: Saturday, January 03, 2009 12:51 PM To: Guido Serassio Cc: Johnson, S; squid-users@squid-cache.org Subject: Re: [squid-users] NTLM and transparent/interception confusion On Sat, Jan 3, 2009 at 11:14 AM, Guido Serassio guido.seras...@acmeconsulting.it wrote: Hi Kinkie, At 18.45 02/01/2009, Kinkie wrote: Could you try to get a network trace of a successfully authenticated http transaction? I would love to see how they do it... Websense too is using something similar for filtering: They maintain an IP Address/Username table on the policy server. The table can be populated using different ways: - A logon agent, a little executable running on every client at logon time - Direct query to the user workstation - A DC agent that query DCs for user sessions There isn't any kind of web browser authentication, and this solution cannot work with non Windows clients or machine non domain member. Multiuser terminal server environments cannot be supported and the WS policy server should be Windows based and domain member for full functionality. Yuck... IIRC Squid's session helper can do that too then. This is NOT authentication and it's absolutely insecure: even windows nowadays supports remote desktops (3 users can share one IP) and SNAT (connection sharing), and it's pretty easy to hijack an user's credentials (simply log on to his workstation as soon as possible after he's logged out). an nmblookup-based external authentication helper could be set up to do one of these, but after all what's the point? If the user has a proper Windows infrasctructure, it's much easier to use group policies to configure the browsers.. Thanks for the clarification Guido! -- Please be using Current Stable Squid 2.7.STABLE5 or 3.0.STABLE11 Current Beta Squid 3.1.0.3 -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
[squid-users] How do I see who's connected to my SQUID server?
I'm using NTLM_AUTH for my authentication mechanism, but if I run smbstatus I do not see anyone connected (and I know I am). Is there another tool to see who is connected to my server or at least their IP address? (I know I could dig through logs... but I just want a quick snapshot of who is using it at this particular moment). Thanks Scott
[squid-users] Squid 2.6 and Wordpress
I'm running squid 2.6 from CentOS 5 repository, as an http accelerator
for wordpress.
I've got it working to a point using a configuration I found elsewhere
(I'm not very familiar with squid yet, and most docs I've found are not
relevant to this version).
It is successfully caching the images as stuff and other static content,
but I would like it cache some of dynamic pages as well. I've played
around with the refresh_pattern's a little bit, but didn't have any
luck. Here is an example from the access log.
ping.php is specifically not cached, but the other one ?p=1 I would
like to cache.
1231260321.328682 192.168.1.20 TCP_MEM_HIT/200 1541 GET
http://wp.dom.tld/wp-content/themes/default/images/bg_body.gif - NONE/-
image/gif
1231260321.331 3 192.168.1.20 TCP_MEM_HIT/200 8116 GET
http://wp.dom.tld/wp-content/themes/default/images/wp_blog_head_01.png -
NONE/- image/g
1231260321.332 0 192.168.1.20 TCP_MEM_HIT/200 2033 GET
http://wp.dom.tld/wp-content/themes/default/images/wp_blog_head_02.png -
NONE/- image/g
1231260321.482150 192.168.1.20 TCP_MEM_HIT/200 739 GET
http://wp.dom.tld/wp-content/themes/default/images/icon_rss.gif - NONE/-
image/gif
1231260324.383 2 192.168.1.20 TCP_MISS/200 12 GET
http://wp.dom.tld/ping.php - FIRST_UP_PARENT/192.168.1.101 text/html
1231260326.500854 192.168.1.20 TCP_MISS/200 3209 GET
http://wp.dom.tld/?p=1 - FIRST_UP_PARENT/192.168.1.101 text/html
1231260326.674 27 192.168.1.20 TCP_MEM_HIT/200 11252 GET
http://wp.dom.tld/wp-content/themes/default/style.css - NONE/- text/css
1231260327.406733 192.168.1.20 TCP_MEM_HIT/200 1541 GET
http://wp.dom.tld/wp-content/themes/default/images/bg_body.gif - NONE/-
image/gif
1231260327.410 3 192.168.1.20 TCP_MEM_HIT/200 8116 GET
http://wp.dom.tld/wp-content/themes/default/images/wp_blog_head_01.png -
NONE/- image/g
1231260327.411 0 192.168.1.20 TCP_MEM_HIT/200 2033 GET
http://wp.dom.tld/wp-content/themes/default/images/wp_blog_head_02.png -
NONE/- image/g
1231260327.417 6 192.168.1.20 TCP_MEM_HIT/200 739 GET
http://wp.dom.tld/wp-content/themes/default/images/icon_rss.gif - NONE/-
image/gif
# /etc/squid/squid.conf ##
# Basic parameters
visible_hostname localhost
# This line indicates the server we will be proxying for
http_port 80 defaultsite=wordpress.dom.tld
# And the IP Address for it
cache_peer 10.10.10.10 parent 80 0 no-query originserver
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
# Where the cache files will be, memory and such
cache_dir ufs /var/spool/squid 1 16 256
cache_mem 256 MB
maximum_object_size_in_memory 32 KB
# Log locations and format
logformat common %a %ui %un [%tl] %rm %ru HTTP/%rv %Hs %st %Ss:%Sh
logformat combined %a %ui %un [%tl] %rm %ru HTTP/%rv %Hs %st
%{Referer}h %{User-Agent}h %Ss:%Sh
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
hosts_file /etc/hosts
# Basic ACLs
acl all src 0.0.0.0/0.0.0.0
acl mydomain dstdomain .example.com
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl Safe_ports port 80
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access allow localhost
http_access allow all
http_access allow mydomain
http_access deny all
http_reply_access allow all
icp_access allow all
cache_effective_group squid
coredump_dir /var/spool/squid
forwarded_for on
emulate_httpd_log on
redirect_rewrites_host_header off
buffered_logs on
refresh_pattern -i ping.php 0 0% 0 override-expire
refresh_pattern -i .php 0 50% 30
refresh_pattern -i .js0 50% 300
refresh_pattern -i .css 0 50% 300
refresh_pattern -i .jpg 0 50% 300
refresh_pattern -i .gif 0 50% 300
refresh_pattern -i .png 0 50% 300
refresh_pattern -i .swf 0 50% 300
refresh_pattern -i .html 0 50% 300
refresh_pattern -i .shtml 0 50% 300
client_persistent_connections on
server_persistent_connections on
pipeline_prefetch on
strip_query_terms off
Re: [squid-users] How do I see who's connected to my SQUID server?
something basic like netstat -an will give you some info, if you want squid specific info, you can setup cachemgr ... Johnson, S wrote: I'm using NTLM_AUTH for my authentication mechanism, but if I run smbstatus I do not see anyone connected (and I know I am). Is there another tool to see who is connected to my server or at least their IP address? (I know I could dig through logs... but I just want a quick snapshot of who is using it at this particular moment). Thanks Scott
[squid-users] SquidGuard Replacement
When logging in to MS Technet, I get this: ERROR The requested URL could not be retrieved The following error was encountered while trying to retrieve the URL: http:443 Unable to determine IP address from host name The DNS server returned: Name Error: The domain name does not exist.This means that the cache was not able to resolve the hostname presented in the URL. Check if the address is correct. Your cache administrator is root. Generated Tue, 06 Jan 2009 19:12:01 GMT by dev.activenetwerx.int (squid/3.0.STABLE9) What does http:443 mean? This is only a problem when squidGuard is enabled? The url that it tanked on is: https://login.live.com/login.srf?wa=wsignin1.0rpsnv=10ct=1231267843rver=5.5.4177.0wp=MCMBIwlcxt=technet%24technet%24technetwreply=https%3a%2f%2ftechnet.microsoft.com%2fen-ca%2fsubscriptions%2fmanage%2fbb980931.aspxlc=1033id=254354cru=http%3a%2f%2ftechnet.microsoft.com%2fen-ca%2fsubscriptions%2fdefault.aspx Why would it work without squidGuard? I am seeming to have a lot of problems with squidGuard, anyone got a reco on a replacement? Thanks! jlc
Re: [squid-users] transparent proxy not working!! any advice?
Hello, after adding the ACL below. I've got the following result. if im not mistaken, it has something to due with the dynamic issue? should I set it as standard 0 or ?! *Jan 6 20:21:39.294: WCCP-PKT:D90: Sending I_See_You packet to 192.168.0.183 w/ rcv_id 0019 *Jan 6 20:21:39.298: WCCP-PKT:D80: Sending I_See_You packet to 192.168.0.183 w/ rcv_id 0019 *Jan 6 20:21:57.290: WCCP-EVNT:D90: Here_I_Am packet from 192.168.0.183 w/bad rcv_id *Jan 6 20:21:57.290: WCCP-PKT:D90: Sending I_See_You packet to 192.168.0.183 w/ rcv_id 001A *Jan 6 20:21:57.290: WCCP-EVNT:D80: Here_I_Am packet from 192.168.0.183 w/bad rcv_id *Jan 6 20:21:57.290: WCCP-PKT:D80: Sending I_See_You packet to 192.168.0.183 w/ rcv_id 001A *Jan 6 20:22:04.294: WCCP-PKT:D90: Sending Removal_Query packet to 192.168.0.183w/ rcv_id 001B *Jan 6 20:22:04.298: WCCP-PKT:D80: Sending Removal_Query packet to 192.168.0.183w/ rcv_id 001B *Jan 6 20:22:09.294: %WCCP-1-SERVICELOST: Service 90 lost on WCCP client 192.168.0.183 *Jan 6 20:22:09.298: %WCCP-1-SERVICELOST: Service 80 lost on WCCP client 192.168.0.183 *Jan 6 20:22:15.298: WCCP-PKT:D90: Sending I_See_You packet to 192.168.0.183 w/ rcv_id 001C *Jan 6 20:22:15.298: WCCP-PKT:D80: Sending I_See_You packet to 192.168.0.183 w/ rcv_id 001C -- From: Roland Roland r_o_l_a_...@hotmail.com Sent: Monday, January 05, 2009 9:50 PM To: Ritter, Nicholas nicholas.rit...@americantv.com; sq...@vdvyver.net Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Hello, thanks for the advice ill proceed and add the new ACL. in the meantime, to answer your question yes Squid is on the same interface as all the other clients. what sort of entries should I add to tht access list? PS: my IOS is Version 12.4(17b), RELEASE SOFTWARE (fc2) Cisco 2811 (revision 53.51) -- From: Ritter, Nicholas nicholas.rit...@americantv.com Sent: Monday, January 05, 2009 9:23 PM To: r_o_l_a_...@hotmail.com; sq...@vdvyver.net Cc: squid-users@squid-cache.org Subject: RE: [squid-users] transparent proxy not working!! any advice? The error on the Cisco router is stating that the squid box is trying to tell the router that it is able to service the wccp group 80 and 90, but for some reason the router does not see those groups as ones it is servicing. This is odd. Try doing the following in the router: ip access-list 180 permit any any ip wccp web-cache redirect-list 180 ip wccp 80 redirect-list 180 ip wccp 90 redirect-list 180 Is the squid box on the same router interface as the rest of the clients? If it is, you may need to add lines to the access-list 180, or put the squid box on the secondary interface of the router and do a ip wccp redirect exclude in statement on that interface. Which IOS feature set and version is this? WCCP is buggy in some IOS releases. From: r_o_l_a_...@hotmail.com [mailto:r_o_l_a_...@hotmail.com] Sent: Mon 1/5/2009 8:43 AM To: sq...@vdvyver.net Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Hello, actually I have both of set on the lan interface ( am I mistaken to set the redirect out on the lan interface? should I be setting it on the interface facing the internet?) ip wccp 80 redirect in ip wccp 90 redirect out as for the wiki provided, I fail to see what's missing! obviously there is something, but I'm not detecting it! -- From: Regardt van de Vyver sq...@vdvyver.net Sent: Monday, January 05, 2009 12:46 AM Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Roland Roland wrote: Hello, the output of the debugging is as such: *Jan 4 23:16:43.205: WCCP-EVNT:D90: Here_I_Am packet from 192.168.0.183: service not active *Jan 4 23:16:43.205: WCCP-EVNT:D80: Here_I_Am packet from 192.168.0.183: service not active what service is that?! -- From: Regardt van de Vyver sq...@vdvyver.net Sent: Sunday, January 04, 2009 9:33 PM Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Roland Roland wrote: i've just created a new box with the following options: but wccp with router is still not working! any advice? using centos 5.2 and squid 2.6 firewall enabled SElinux permissive --- done the following: yum update yum yum install squid squid -z --- gedit /etc/rc.d/init.d/rc.local #added: modprobe ip_gre ifconfig gre0 192.168.0.183 netmask 255.255.255.0 up #this is the same ip as my eth0 gedit /etc/sysconfig/iptables #added: -A
Re: [squid-users] SquidGuard Replacement
Joseph L. Casale schrieb: When logging in to MS Technet, I get this: ERROR The requested URL could not be retrieved The following error was encountered while trying to retrieve the URL: http:443 Unable to determine IP address from host name The DNS server returned: Name Error: The domain name does not exist.This means that the cache was not able to resolve the hostname presented in the URL. Check if the address is correct. Your cache administrator is root. Generated Tue, 06 Jan 2009 19:12:01 GMT by dev.activenetwerx.int (squid/3.0.STABLE9) What does http:443 mean? This is only a problem when squidGuard is enabled? The url that it tanked on is: https://login.live.com/login.srf?wa=wsignin1.0rpsnv=10ct=1231267843rver=5.5.4177.0wp=MCMBIwlcxt=technet%24technet%24technetwreply=https%3a%2f%2ftechnet.microsoft.com%2fen-ca%2fsubscriptions%2fmanage%2fbb980931.aspxlc=1033id=254354cru=http%3a%2f%2ftechnet.microsoft.com%2fen-ca%2fsubscriptions%2fdefault.aspx Why would it work without squidGuard? I am seeming to have a lot of problems with squidGuard, anyone got a reco on a replacement? Thanks! jlc Hello Joseph, I'm using Squid3STABLE9 and SquidGuard 1.3 on three openSUSE10.3 boxes and tested the URL you gave us above without hanving any problems to access the TechNet site. So this must be something with your specific setup. What's the version of SG are you using ? Maybe you can post your problem to http://www.squidguard.org/mailinglist.html Regards, - Philipp
RE: [squid-users] SquidGuard Replacement
I'm using Squid3STABLE9 and SquidGuard 1.3 on three openSUSE10.3 boxes and tested the URL you gave us above without hanving any problems to access the TechNet site. So this must be something with your specific setup. What's the version of SG are you using ? Maybe you can post your problem to http://www.squidguard.org/mailinglist.html Philipp, I am using Squid3STABLE9 and SquidGuard 1.3-1.el5.rf on a couple of CentOS 5 boxes? My SquidGuard has only a local net defined with an acl blocking many shalla lists. My squid.conf is as follows: acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localnet http_access allow localhost http_access deny all icp_access allow localnet icp_access deny all htcp_access allow localnet htcp_access deny all http_port 3128 hierarchy_stoplist cgi-bin ? access_log /var/log/squid/access.log squid url_rewrite_program /usr/bin/squidGuard refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern (cgi-bin|\?)0 0% 0 refresh_pattern . 0 20% 4320 icp_port 3130 coredump_dir /var/spool/squid Both of my servers are independent with identical configs and exhibit the same behavior, how does your config compare? Thanks! jlc
Re: [squid-users] Squid 2.6 and Wordpress
Dan Casey wrote:
I'm running squid 2.6 from CentOS 5 repository, as an http accelerator
for wordpress.
I've got it working to a point using a configuration I found elsewhere
(I'm not very familiar with squid yet, and most docs I've found are not
relevant to this version).
It is successfully caching the images as stuff and other static content,
but I would like it cache some of dynamic pages as well. I've played
around with the refresh_pattern's a little bit, but didn't have any
luck. Here is an example from the access log.
ping.php is specifically not cached, but the other one ?p=1 I would
like to cache.
1231260321.328682 192.168.1.20 TCP_MEM_HIT/200 1541 GET
http://wp.dom.tld/wp-content/themes/default/images/bg_body.gif - NONE/-
image/gif
1231260321.331 3 192.168.1.20 TCP_MEM_HIT/200 8116 GET
http://wp.dom.tld/wp-content/themes/default/images/wp_blog_head_01.png -
NONE/- image/g
1231260321.332 0 192.168.1.20 TCP_MEM_HIT/200 2033 GET
http://wp.dom.tld/wp-content/themes/default/images/wp_blog_head_02.png -
NONE/- image/g
1231260321.482150 192.168.1.20 TCP_MEM_HIT/200 739 GET
http://wp.dom.tld/wp-content/themes/default/images/icon_rss.gif - NONE/-
image/gif
1231260324.383 2 192.168.1.20 TCP_MISS/200 12 GET
http://wp.dom.tld/ping.php - FIRST_UP_PARENT/192.168.1.101 text/html
1231260326.500854 192.168.1.20 TCP_MISS/200 3209 GET
http://wp.dom.tld/?p=1 - FIRST_UP_PARENT/192.168.1.101 text/html
What is the output of wget -O /dev/null -S http://wp.dom.tld/?p=1;?
Refresh patterns only help if expiry information is absent.
1231260326.674 27 192.168.1.20 TCP_MEM_HIT/200 11252 GET
http://wp.dom.tld/wp-content/themes/default/style.css - NONE/- text/css
1231260327.406733 192.168.1.20 TCP_MEM_HIT/200 1541 GET
http://wp.dom.tld/wp-content/themes/default/images/bg_body.gif - NONE/-
image/gif
1231260327.410 3 192.168.1.20 TCP_MEM_HIT/200 8116 GET
http://wp.dom.tld/wp-content/themes/default/images/wp_blog_head_01.png -
NONE/- image/g
1231260327.411 0 192.168.1.20 TCP_MEM_HIT/200 2033 GET
http://wp.dom.tld/wp-content/themes/default/images/wp_blog_head_02.png -
NONE/- image/g
1231260327.417 6 192.168.1.20 TCP_MEM_HIT/200 739 GET
http://wp.dom.tld/wp-content/themes/default/images/icon_rss.gif - NONE/-
image/gif
# /etc/squid/squid.conf ##
# Basic parameters
visible_hostname localhost
# This line indicates the server we will be proxying for
http_port 80 defaultsite=wordpress.dom.tld
This should probably be...
http_port 80 accel defaultsite=wordpress.dom.tld
...to explicitly enable acceleration mode.
# And the IP Address for it
cache_peer 10.10.10.10 parent 80 0 no-query originserver
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
# Where the cache files will be, memory and such
cache_dir ufs /var/spool/squid 1 16 256
cache_mem 256 MB
maximum_object_size_in_memory 32 KB
# Log locations and format
logformat common %a %ui %un [%tl] %rm %ru HTTP/%rv %Hs %st %Ss:%Sh
logformat combined %a %ui %un [%tl] %rm %ru HTTP/%rv %Hs %st
%{Referer}h %{User-Agent}h %Ss:%Sh
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
hosts_file /etc/hosts
# Basic ACLs
acl all src 0.0.0.0/0.0.0.0
acl mydomain dstdomain .example.com
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl Safe_ports port 80
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access allow localhost
http_access allow all
Enabling http_access allow all (even in accelerator mode) is...
inadvisable. Removing it, and setting the mydomain ACL properly is a
much better choice.
http_access allow mydomain
http_access deny all
http_reply_access allow all
icp_access allow all
cache_effective_group squid
coredump_dir /var/spool/squid
forwarded_for on
emulate_httpd_log on
redirect_rewrites_host_header off
buffered_logs on
refresh_pattern -i ping.php 0 0% 0 override-expire
refresh_pattern -i .php 0 50% 30
refresh_pattern -i .js0 50% 300
refresh_pattern -i .css 0 50% 300
refresh_pattern -i .jpg 0 50% 300
refresh_pattern -i .gif 0 50% 300
refresh_pattern -i .png 0 50% 300
refresh_pattern -i .swf 0 50% 300
refresh_pattern -i .html 0 50% 300
refresh_pattern -i .shtml 0 50% 300
client_persistent_connections on
server_persistent_connections on
pipeline_prefetch on
strip_query_terms off
Chris
RE: [squid-users] transparent proxy not working!! any advice?
Ok...so the squid server and the router are seeing eachother initiallythen it fails. On the squid box you need to make sure the firewall is allowing UDP port 2048 from the the router and that the GRE tunnel is functioning properly, and is setup in iptables properly. The other issue is that may be needed is that access-list (access-list 180, from my last email) should have the ip of the squid box in it as a deny entry. The reason for this is that you want to avoid traffic being 'looped' from the router to the squid box. You can setup WCCP where you are using no service groups and just the web-cache and web-cache redirect, etc. The two things that can break doing that are: multiple squid servers in a WCCP setup, and support for apps/ports other than port 80. Nick From: Roland Roland [mailto:r_o_l_a_...@hotmail.com] Sent: Tue 1/6/2009 1:48 PM To: Ritter, Nicholas; sq...@vdvyver.net Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Hello, after adding the ACL below. I've got the following result. if im not mistaken, it has something to due with the dynamic issue? should I set it as standard 0 or ?! *Jan 6 20:21:39.294: WCCP-PKT:D90: Sending I_See_You packet to 192.168.0.183 w/ rcv_id 0019 *Jan 6 20:21:39.298: WCCP-PKT:D80: Sending I_See_You packet to 192.168.0.183 w/ rcv_id 0019 *Jan 6 20:21:57.290: WCCP-EVNT:D90: Here_I_Am packet from 192.168.0.183 w/bad rcv_id *Jan 6 20:21:57.290: WCCP-PKT:D90: Sending I_See_You packet to 192.168.0.183 w/ rcv_id 001A *Jan 6 20:21:57.290: WCCP-EVNT:D80: Here_I_Am packet from 192.168.0.183 w/bad rcv_id *Jan 6 20:21:57.290: WCCP-PKT:D80: Sending I_See_You packet to 192.168.0.183 w/ rcv_id 001A *Jan 6 20:22:04.294: WCCP-PKT:D90: Sending Removal_Query packet to 192.168.0.183w/ rcv_id 001B *Jan 6 20:22:04.298: WCCP-PKT:D80: Sending Removal_Query packet to 192.168.0.183w/ rcv_id 001B *Jan 6 20:22:09.294: %WCCP-1-SERVICELOST: Service 90 lost on WCCP client 192.168.0.183 *Jan 6 20:22:09.298: %WCCP-1-SERVICELOST: Service 80 lost on WCCP client 192.168.0.183 *Jan 6 20:22:15.298: WCCP-PKT:D90: Sending I_See_You packet to 192.168.0.183 w/ rcv_id 001C *Jan 6 20:22:15.298: WCCP-PKT:D80: Sending I_See_You packet to 192.168.0.183 w/ rcv_id 001C -- From: Roland Roland r_o_l_a_...@hotmail.com Sent: Monday, January 05, 2009 9:50 PM To: Ritter, Nicholas nicholas.rit...@americantv.com; sq...@vdvyver.net Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Hello, thanks for the advice ill proceed and add the new ACL. in the meantime, to answer your question yes Squid is on the same interface as all the other clients. what sort of entries should I add to tht access list? PS: my IOS is Version 12.4(17b), RELEASE SOFTWARE (fc2) Cisco 2811 (revision 53.51) -- From: Ritter, Nicholas nicholas.rit...@americantv.com Sent: Monday, January 05, 2009 9:23 PM To: r_o_l_a_...@hotmail.com; sq...@vdvyver.net Cc: squid-users@squid-cache.org Subject: RE: [squid-users] transparent proxy not working!! any advice? The error on the Cisco router is stating that the squid box is trying to tell the router that it is able to service the wccp group 80 and 90, but for some reason the router does not see those groups as ones it is servicing. This is odd. Try doing the following in the router: ip access-list 180 permit any any ip wccp web-cache redirect-list 180 ip wccp 80 redirect-list 180 ip wccp 90 redirect-list 180 Is the squid box on the same router interface as the rest of the clients? If it is, you may need to add lines to the access-list 180, or put the squid box on the secondary interface of the router and do a ip wccp redirect exclude in statement on that interface. Which IOS feature set and version is this? WCCP is buggy in some IOS releases. From: r_o_l_a_...@hotmail.com [mailto:r_o_l_a_...@hotmail.com] Sent: Mon 1/5/2009 8:43 AM To: sq...@vdvyver.net Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Hello, actually I have both of set on the lan interface ( am I mistaken to set the redirect out on the lan interface? should I be setting it on the interface facing the internet?) ip wccp 80 redirect in ip wccp 90 redirect out as for the wiki provided, I fail to see what's missing! obviously there is something, but I'm not detecting it! -- From: Regardt van de Vyver sq...@vdvyver.net Sent: Monday, January 05, 2009 12:46 AM Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Roland Roland wrote: Hello, the output of the debugging is as such:
RE: [squid-users] NTLM and transparent/interception confusion
... We are a K-12 education and are mandated by federal law to monitor and protect student access to the internet. ... We are now allowing students to bring their own notebooks ... Yep, yet another instance of the classic problem filter 'the net' rather than individual computers. (Same thing comes up with iPhones and with wireless.) By requiring NTLM authentication against your own domain, you're requiring users to use your names and passwords even if they use their own computer; that gets you part way. The remaining issue is how to avoid the dreaded dead end browser that won't do anything until its proxy options are changed but can't even display the directions for changing those options. One possible solution is to put a webserver on port 80 that gives detailed instructions on how to set the proxy options in all the various browsers. Reconfigured browsers will access the filter on a special port (3128? 8080?) which works withOUT the problematic transparent/intercepting and withOUT any IPtables rules for the filter. No matter what newly arrived browsers try to view, they'll just see your page of instructions. The biggest problem here may be that K-12 includes some very young kids. Hopefully any young kid that brings their own computer from home can deal with this stuff. (Definitely no problem beyond about third grade - kids are geeks.) Use lots of screenshots and train the teachers. Making this network billboard idea work requires a couple things: 1) use IPtables to redirect all the port 80 traffic to port 80 (!?)- the net effect is the IP address of the desired website will be replaced with the IP address of your own server 2) use an .htaccess file with an ErrorDocument 404 ... in it so all browsers get to your page of instructions no matter what page they originally desired See http://contentfilter.futuragts.com/wiki/doku.php?id=network_billboard for details. thanks! -Chuck Kollars
[squid-users] Defining BL's via acls
What kind of performance issues should I expect if I remove squidGuard and simply make a series of acl's pointing to shalla bl files directly then denying them with http_access deny statements? Given the size of the shalla lists, what would any seasoned squid admins expect as a scalability threshold on this approach? Thanks! jlc
[squid-users] Shalla Lists down?
I've noticed that the Shalla block lists for SquidGuard seem to be down. Does anyone know what's up? -- This e-mail may contain technical information which is controlled by the United States Government, Department of State, International Traffic Arms Regulations (ITAR) (22 CFR 120-130) which requires an export license prior to sharing with foreign persons. Lacking such license, ITAR technical data is limited to US Legal Residents only. It is the responsibility of the organization and individual in control of this data to abide by US export laws. If you are not a US Legal Resident, immediately forward this e-mail to not...@hartwellcorp.com or reply to sender without reading any further. Take no other action with this e-mail until contacted. Notice: The information in this document and document itself, in whole or in part, in any form (Information) is proprietary and/or confidential property of Hartwell Corporation, Placentia, California. Hartwell Corporation and its successors and assignees retain and reserve all right, title and interest in this information in whole or in part and in all forms. This Information is provided to the original recipient only for confidential use, with the understanding that it will not be used in any manner detrimental to the interests of Hartwell Corporation, and subject to return on request. Reproduction, transmission, distribution or publication of this Information in any form, in whole or in part, for any purpose without prior written permission of Hartwell Corporation is strictly prohibited. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Re: [squid-users] Problem configure squid 3.1
build-essential did it. It configured. Thanks for your help. Gregori Parker wrote: I'm sorry, I meant apt-get install libc-dev (I'm obviously not a Debian user) I've also read that you may need the 'build-essential' package as well, so you might want to try that -Original Message- From: Gregori Parker [mailto:gregori.par...@theplatform.com] Sent: Monday, January 05, 2009 4:33 PM To: w...@msdrd.com Cc: squid-users@squid-cache.org Subject: RE: [squid-users] Problem configure squid 3.1 Try 'apt-get libc-dev' and report back -Original Message- From: Wilson Hernandez - MSD, S. A. [mailto:w...@msdrd.com] Sent: Monday, January 05, 2009 6:01 PM To: Gregori Parker Cc: squid-users@squid-cache.org Subject: Re: [squid-users] Problem configure squid 3.1 I've already have it installed and still not working. Gregori Parker wrote: Sounds like you need a c++ compiler, do a 'apt-get gcc' (you're running debian IIRC) -Original Message- From: Wilson Hernandez [mailto:w...@msdrd.com] Sent: Monday, January 05, 2009 1:50 PM To: squid-users@squid-cache.org Subject: [squid-users] Problem configure squid 3.1 Hello. Me again. It seems that everyhting I try to do can't go smoothly. Now, I'm trying to get squid-3.1.0.3 installed in my system trying to upgrade from an older version but now come accross a problem when I run ./configure I get the following error (I searched the internet but, can't get a solutions) : checking for C++ compiler default output file name... configure: error: C++ compiler cannot create executables See `config.log' for more details. configure: error: ./configure failed for lib/libTrie I removed the previous squid version which was installed as a package. Please help. Thanks. -- *Wilson Hernandez* Presidente 829.848.9595 809.766.0441 www.msdrd.com http://www.msdrd.com Conservando el medio ambiente
Re: [squid-users] How do I see who's connected to my SQUID server?
Rick Chisholm wrote: something basic like netstat -an will give you some info, if you want squid specific info, you can setup cachemgr ... Or for a quick random dump squidclient. squidclient mgr:client_list reports all the client IPs that connected in the last N hours and some stats about their usage. This is identical to the cachemgr page. Amos Johnson, S wrote: I'm using NTLM_AUTH for my authentication mechanism, but if I run smbstatus I do not see anyone connected (and I know I am). Is there another tool to see who is connected to my server or at least their IP address? (I know I could dig through logs... but I just want a quick snapshot of who is using it at this particular moment). Thanks Scott -- Please be using Current Stable Squid 2.7.STABLE5 or 3.0.STABLE11 Current Beta Squid 3.1.0.3
Re: [squid-users] Squid store open errors
Itzcak Pechtalt wrote: Hi, I got several errors in cache log like the following: storeAufsOpenDone: (1) Operation not permitted /var/spool/squid/12/28/0012287A Sometimes I get the same but with File not found error. I suppose it's related to non clean sytem restart which omitted some object from swap.state and from Squid. Is there any way to fix this problems after non clean restart ? No. Those messages are a result of Squid cleaning up bad state in the cache. You need to locate why Squid did not shutdown properly and fix that. Amos -- Please be using Current Stable Squid 2.7.STABLE5 or 3.0.STABLE11 Current Beta Squid 3.1.0.3
Re: [squid-users] Squid 2.6 and Wordpress
Dan Casey wrote:
I'm running squid 2.6 from CentOS 5 repository, as an http accelerator
for wordpress.
I've got it working to a point using a configuration I found elsewhere
(I'm not very familiar with squid yet, and most docs I've found are not
relevant to this version).
The FAQ has a current configuration setup tutorial
http://wiki.squid-cache.org/SquidFaq/ReverseProxy
there are large access control settings missing from your config that
are needed to make things work efficiently and securely. Additional to
the notes I've made below.
It is successfully caching the images as stuff and other static content,
but I would like it cache some of dynamic pages as well. I've played
around with the refresh_pattern's a little bit, but didn't have any
luck. Here is an example from the access log.
ping.php is specifically not cached, but the other one ?p=1 I would
like to cache.
# /etc/squid/squid.conf ##
# Basic parameters
visible_hostname localhost
Calling the gateway machine localhost may cause all sorts of trouble
and confusion. Please try using a proper machine or domain FQDN.
# This line indicates the server we will be proxying for
http_port 80 defaultsite=wordpress.dom.tld
# And the IP Address for it
cache_peer 10.10.10.10 parent 80 0 no-query originserver
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
# Where the cache files will be, memory and such
cache_dir ufs /var/spool/squid 1 16 256
cache_mem 256 MB
maximum_object_size_in_memory 32 KB
# Log locations and format
logformat common %a %ui %un [%tl] %rm %ru HTTP/%rv %Hs %st %Ss:%Sh
logformat combined %a %ui %un [%tl] %rm %ru HTTP/%rv %Hs %st
%{Referer}h %{User-Agent}h %Ss:%Sh
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
store log is not usually needed. you can save yourself a lot of storage
space setting this to none.
hosts_file /etc/hosts
# Basic ACLs
acl all src 0.0.0.0/0.0.0.0
acl mydomain dstdomain .example.com
That should be set properly to the wordpress domain.
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl Safe_ports port 80
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access allow localhost
http_access allow all
Drop that allow all!
http_access allow mydomain
http_access deny all
http_reply_access allow all
icp_access allow all
cache_effective_group squid
should only need cache_effective_user here. Group plays wickedness with
permissions.
coredump_dir /var/spool/squid
forwarded_for on
emulate_httpd_log on
redirect_rewrites_host_header off
buffered_logs on
refresh_pattern -i ping.php 0 0% 0 override-expire
refresh_pattern -i .php 0 50% 30
refresh_pattern -i .js0 50% 300
refresh_pattern -i .css 0 50% 300
refresh_pattern -i .jpg 0 50% 300
refresh_pattern -i .gif 0 50% 300
refresh_pattern -i .png 0 50% 300
refresh_pattern -i .swf 0 50% 300
refresh_pattern -i .html 0 50% 300
refresh_pattern -i .shtml 0 50% 300
client_persistent_connections on
server_persistent_connections on
pipeline_prefetch on
strip_query_terms off
Amos
--
Please be using
Current Stable Squid 2.7.STABLE5 or 3.0.STABLE11
Current Beta Squid 3.1.0.3
Re: [squid-users] Defining BL's via acls
Joseph L. Casale wrote: What kind of performance issues should I expect if I remove squidGuard and simply make a series of acl's pointing to shalla bl files directly then denying them with http_access deny statements? Depends on your chosen ACL type and the number of patterns. Many regex may be slower than DG, many dstdomain or dst may improve response time. Given the size of the shalla lists, what would any seasoned squid admins expect as a scalability threshold on this approach? Thanks! jlc Amos -- Please be using Current Stable Squid 2.7.STABLE5 or 3.0.STABLE11 Current Beta Squid 3.1.0.3
RE: [squid-users] Defining BL's via acls
Depends on your chosen ACL type and the number of patterns. Many regex may be slower than DG, many dstdomain or dst may improve response time. It looks like the lists are far too large for any regex type acls but the acl name dstdomain file is causing me issues with the way the shalla lists are formatted, some urls are complete and some aren't and the incomplete sites do not have begin with a . so they aren't matched, any way around this? I don't know why squidgaurd is broken in all three of my setups, maybe an issue with the rpmforge package? No one has a 1.4 rpm and I won't compile on these production systems :( Later this week, I will try to fire up a vm with CentOS and test it out. Thanks! jlc
Re: [squid-users] How do I see who's connected to my SQUID server?
2009/1/7 Amos Jeffries squ...@treenet.co.nz: Rick Chisholm wrote: something basic like netstat -an will give you some info, if you want squid specific info, you can setup cachemgr ... Or for a quick random dump squidclient. squidclient mgr:client_list reports all the client IPs that connected in the last N hours and some stats about their usage. This is identical to the cachemgr page. Amos, How to know the connection number from Squid to each realserver? Thanks. Ralf.
[squid-users] authenticate_ip_shortcircuit
hello all, just little question to know if somebody plans including ip_shortcircuit in 'squidclient mgr:' info pages. It should be interesting to get the list of mappings ip/identification. I should also be interesting to get another shortcircuit instruction like 'ip_shortcircuit_size' to limit the size of this list. many thks Vincent - ATTENTION: The information in this electronic mail message is private and confidential, and only intended for the addressee. Should you receive this message by mistake, you are hereby notified that any disclosure, reproduction, distribution or use of this message is strictly prohibited. Please inform the sender by reply transmission and delete the message without copying or opening it. Messages and attachments are scanned for all viruses known. If this message contains password-protected attachments, the files have NOT been scanned for viruses by the ING mail domain. Always scan attachments before opening them. -
[squid-users] Possible regressions in dstdom_regex based matching
Hi, I have created a filter file named block.lst with the following contents /youtube. /orkut. /songs.pk /senduit. I have created an acl based on the domain regex for the above filter by specifying the following rule in the squid.conf acl list_blocked dstdom_regex /squid/lists/block.lst to block all users from accessing the domains specified in the above list, i have specified the following rule in squid.conf http_access denyall list_blocked As a result of the above changes, access to http://www.youtube.com, http://youtube.com, http://www.orkut.com and http://orkut.com is restricted, But however, http://www.songs.pk and http://songs.pk remains accessible and at the same time http://www.senduit.com remains accessible while http://senduit.com becomes restricted. I see similar issues with a couple of domains. I really have no clue why this is happening. I am using squid3-STABLE11 and have seen these issues in earlier versions as well. Please let me know if theres something wrong with what i am doing. Regards ~Sameer
