[squid-users] "Complex" acl process - Many Ips, many different places, many logins, and many websites ...
Hi everyone,
I'm having some troubles to understand how the acl process is working.
I'm trying to link a mySQL database to my squid in order to allow me
to setup some specific access rights according to some specific users
from different places to different websites.
What I did is an acl that will check the domain and the source_ip
external_acl_type ExternalisBad ttl=20 %SRC %DST /etc/squid3/external_bad
acl isBad external ExternalisBad
And I also created my own auth_param block
auth_param basic program /etc/squid3/sql_auth
auth_param basic children 20
auth_param basic realm Username and password
auth_param basic credentialsttl 1 minute
Now, when someone's trying to to access a website, this is what I do
http_access allow sql_auth isBad
It is working, but the thing is: it doesn't care about if the username
is linked to the %SRC Ip or not... So basically, if you have are
registered with full access rights in another place, you will be able
to access to all the content even if you're access is supposed to be
denied. Does that make sense ?
I added the %IDENT to the externcal_acl_type rule. Since the sql_auth
process is called before I was thinking that maybe the %IDENT would be
stored somewhere somehow and be accessible in the isBad acl right
away...
external_acl_type ExternalisBad ttl=20 %SRC %IDENT %DST /etc/squid3/external_bad
Apparently this is not working.
Does any one have any idea on how to do what I want to do ?
If you want me to be more specific, let me know!
Thank you so much Guys,
Julien
PS:
debian:/squid3 -v
Squid Cache: Version 3.0.STABLE8
configure options: '--build=i486-linux-gnu' '--prefix=/usr'
'--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
'--infodir=${prefix}/share/info' '--sysconfdir=/etc'
'--localstatedir=/var' '--libexecdir=${prefix}/lib/squid3'
'--disable-maintainer-mode' '--disable-dependency-tracking'
'--srcdir=.' '--datadir=/usr/share/squid3' '--sysconfdir=/etc/squid3'
'--mandir=/usr/share/man' '--with-cppunit-basedir=/usr'
'--enable-inline' '--enable-async-io=8'
'--enable-storeio=ufs,aufs,coss,diskd,null'
'--enable-removal-policies=lru,heap' '--enable-delay-pools'
'--enable-cache-digests' '--enable-underscores' '--enable-icap-client'
'--enable-follow-x-forwarded-for' '--enable-auth=basic,digest,ntlm'
'--enable-basic-auth-helpers=LDAP,MSNT,NCSA,PAM,SASL,SMB,YP,getpwnam,multi-domain-NTLM'
'--enable-ntlm-auth-helpers=SMB'
'--enable-digest-auth-helpers=ldap,password'
'--enable-external-acl-helpers=ip_user,ldap_group,session,unix_group,wbinfo_group'
'--with-filedescriptors=65536' '--with-default-user=proxy'
'--enable-epoll' '--enable-linux-netfilter'
'build_alias=i486-linux-gnu' 'CC=cc' 'CFLAGS=-g -O2 -g -Wall -O2'
'LDFLAGS=' 'CPPFLAGS=' 'CXX=g++' 'CXXFLAGS=-g -O2 -g -Wall -O2'
'FFLAGS=-g -O2'
Re: [squid-users] client_side_request.cc
Wong wrote: Dear All, I experienced messages below and squid exiting abnormally. Squid version 3S15 Need your advise & help. Thx & Rgds, Wong ---snip--- 2009/06/01 08:29:27| client_side_request.cc(825) redirecting body_pipe 0x85fd94c*1 from request 0x8525c90 to 0x886bcd0 These are normal. Visible only because of the level of debug_options. 2009/06/01 10:05:51| Preparing for shutdown after 67188 requests 2009/06/01 10:05:51| Waiting 5 seconds for active connections to finish 2009/06/01 10:05:51| FD 25 Closing HTTP connection 2009/06/01 10:05:51| WARNING: redirector #1 (FD 10) exited 2009/06/01 10:05:51| WARNING: redirector #9 (FD 18) exited 2009/06/01 10:05:51| Too few redirector processes are running 2009/06/01 10:05:51| Starting new helpers 2009/06/01 10:05:51| helperOpenServers: Starting 9/15 'squidGuard' processes 2009/06/01 10:05:52| WARNING: redirector #10 (FD 19) exited I assume the problem you are reporting is the redirectors starting up again during a shutdown. Is this correct? Amos -- Yes Amos, you're absolutely correct. How can I solve this problem? Now I increase the redirector and monitoring progress. Thx & Rgds, Wong
Re: [squid-users] Evaluating traffic for caching benefits.
On Mon, 1 Jun 2009 17:29:18 -0700, Ray Van Dolson wrote: > Any suggestions on how to go about evaluating web traffic for > "cacheability"? I have access to a port that can see all the web > traffic in our company. > > I'd like to be able to gauge how many hits there are to common sites to > get a feel for how much bandwidth savings we could potentially gain by > implementing a company-wide web cache. Depending on how much you tune the config, you should expect 10% of web traffic to be a lower bound (no tuning) and 50% an upper bound. That is for HTTP traffic only, so the overall % is less depending on the non-HTTP going through your network. > > I suppose creative use of tcpdump could be used here (obviously not > catching https traffic), but maybe there's a more polished tool or some > slicker way to do this. The most reliable way to know is to setup a test proxy and start pushing a small amount of the traffic through it. The summary overview of Squid contains measure of % bandwidth that has been local HIT (saved from going external). The tools out there (www.ircache.net/cgi-bin/cacheability.py and redbot.org) are more spot-check tools for finding out why something particular isn't caching once the resource is known. If anyone knows of a stand-alone tool please speak up, I'm interested as well. Amos
[squid-users] Evaluating traffic for caching benefits.
Any suggestions on how to go about evaluating web traffic for "cacheability"? I have access to a port that can see all the web traffic in our company. I'd like to be able to gauge how many hits there are to common sites to get a feel for how much bandwidth savings we could potentially gain by implementing a company-wide web cache. I suppose creative use of tcpdump could be used here (obviously not catching https traffic), but maybe there's a more polished tool or some slicker way to do this. Thanks, Ray
[squid-users] Web servers behind squid
Hi I'm running a squid cache proxy in bridge mode. (INTERNET)-(LINUX ROUTER/FIREWALL)-(SQUID BRIDGE)-(SWITCH)-(PCS AND SERVERS) With this setup, websites hosted on the LAN doesn't work anymore. The following iptables rules redirect HTTP request to squid: ebtables -t broute -A BROUTING -p IPv4 --ip-protocol 6 --ip-destination-port 80 -j redirect --redirect-target ACCEPT iptables -t nat -A PREROUTING -i br0 -p tcp --dport 80 -j REDIRECT --to-port 3128 How can I continu to serve my users but to keep my websites working ?
Re: [squid-users] How to set up squid?
Amos Jeffries wrote: Yan Seiner wrote: Amos: Since ordinary mortals like me ;-) apparently cannot edit the wiki, here's my final form of the iptables and policy route rules. First on the firewall: iptables -t mangle -A PREROUTING -j ACCEPT -p tcp --dport 80 -s squidboxIP iptables -t mangle -A PREROUTING -j MARK --set-mark 3 -p tcp --dport 80 iptables -A FORWARD -i internalif -o internalif -p tcp --dport 80 -j ACCEPT Is this FORWARDING is not exactly part of the config. I would think routers should allow internal routes by default anyway. But I'll add it as a gotcha to be wary of anyway. Normally yes. This is my firewall doing this; I DENY by default and then enable only what I need. Since my forward chain has a default DENY policy, then forwarding between internal if is forbidden. Also, this is with a 2.4 kernel; IIRC the forwarding rules changed quite a bit with 2.6 kernels so on a box with 2.6 kernel you may well be right. ip rule add fwmark 3 table proxy ip route add default via squidboxIP dev internalif table proxy Then on the squid box itself: iptables -t nat -A PREROUTING -s localnet/24 -i lan0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128 iptables -t nat -A OUTPUT -s squidboxIP/32 -p tcp -m owner ! --uid-owner squid-user -m tcp --dport 80 -j REDIRECT --to-ports 3128 The first rule catches all of the requests that have been redirected by the firewall for proxying. The second rule catches all of the outbound packets originating on the squidbox itself. This can have some weird side effects; my squid box also handles several ipcams and so all this traffic is now routed through squid. Note that you have to change the uid-owner to the uid of the squid process. This is avoidable if the ipcams are sending traffic to the squid box for handling anyway. The NAT rules on the squid box need to omit catching any traffic already destined to itself: iptables -t nat -A PREROUTING -s SQUIDIP -p tcp --dport 80 -j ACCEPT Good point. I'll add that. --Yan
Re: [squid-users] How to set up squid?
Yan Seiner wrote: Yan Seiner wrote: Amos Jeffries wrote: Yan Seiner wrote: Amos Jeffries wrote: Yan Seiner wrote: I have a question about setting up squid in my environment. My network is fairly generic: a firewall running openwrt, 4 mb flash and 8 mb ram, providing NAT a server providing DNS and DHCP services; this machine is also used for terminal services so users are logged in to this machine directly assorted clients I've had squid set up on a 'opt-in' basis. Now I have a request to make it transparent for all users with the intent of disabling web access during specified hours. The problem I have is that my firewall is not able to run squid, and all the examples assume that the squid box is either the firewall or provides NAT. Is it possible, without a huge amount of complications, to run squid on this sort of setup? If so, does anyone have a recipe for doing so? Squid box had best be the one doing NAT because all source info is lost during NAT interception and Squid needs to look it up. Note I wrote "NAT interception", thats a more correct name for "transparent". Squid does not have to be on the firewall or router to do NAT though: http://wiki.squid-cache.org/ConfigExamples/Intercept/IptablesPolicyRoute the tutorial ironically was written for people using OpenWRT :) Amos Hi Amos: Obvously I got something just half right: The requested URL could not be retrieved While trying to retrieve the URL: http://arstechnica.com/tech-policy/news/2009/05/landmark-study-drm-truly-does-make-pirates-out-of-us-all.ars The following error was encountered: Unable to determine IP address from host name for /arstechnica.com/ The dnsserver returned: Server Failure: The name server was unable to process this query. This means that: Is it actually using the '/' there? It looks a lot like the 'transparent' option to http_port is missing still. I've configured this as best as I can following http://wiki.squid-cache.org/ConfigExamples/Intercept/IptablesPolicyRoute on the firewall/router and http://wiki.squid-cache.org/ConfigExamples/Intercept/LinuxRedirect on the squid box. As soon as I enable the iptablesPolicyRoute on the fw my DNS fails I can't figure out why Those rules should only affect tcp packets to port 80. Does anyone have this setup working? Could they please send me some instructions for morons? That was them ;). Does the Squid box have normal DNS if its used as a regular proxy without the PolicyRouting? Amos DUH! OK, my turn to feel stupid Turns out my firewall rules were blocking forwarding from internal_if to internal_if - so the firewall "loopback" to the squid box was getting dropped. Now everything is OK; on to the next step - time based web access - which is why I started this whole thing! --Yan !DSPAM:4a214d26279661804284693! Amos: Since ordinary mortals like me ;-) apparently cannot edit the wiki, here's my final form of the iptables and policy route rules. First on the firewall: iptables -t mangle -A PREROUTING -j ACCEPT -p tcp --dport 80 -s squidboxIP iptables -t mangle -A PREROUTING -j MARK --set-mark 3 -p tcp --dport 80 iptables -A FORWARD -i internalif -o internalif -p tcp --dport 80 -j ACCEPT Is this FORWARDING is not exactly part of the config. I would think routers should allow internal routes by default anyway. But I'll add it as a gotcha to be wary of anyway. ip rule add fwmark 3 table proxy ip route add default via squidboxIP dev internalif table proxy Then on the squid box itself: iptables -t nat -A PREROUTING -s localnet/24 -i lan0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128 iptables -t nat -A OUTPUT -s squidboxIP/32 -p tcp -m owner ! --uid-owner squid-user -m tcp --dport 80 -j REDIRECT --to-ports 3128 The first rule catches all of the requests that have been redirected by the firewall for proxying. The second rule catches all of the outbound packets originating on the squidbox itself. This can have some weird side effects; my squid box also handles several ipcams and so all this traffic is now routed through squid. Note that you have to change the uid-owner to the uid of the squid process. This is avoidable if the ipcams are sending traffic to the squid box for handling anyway. The NAT rules on the squid box need to omit catching any traffic already destined to itself: iptables -t nat -A PREROUTING -s SQUIDIP -p tcp --dport 80 -j ACCEPT as the first rule of the squid box. Amos -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE15 Current Beta Squid 3.1.0.8 or 3.0.STABLE16-RC1
Re: [squid-users] How to set up squid?
Yan Seiner wrote: Amos Jeffries wrote: Yan Seiner wrote: Amos Jeffries wrote: Yan Seiner wrote: I have a question about setting up squid in my environment. My network is fairly generic: a firewall running openwrt, 4 mb flash and 8 mb ram, providing NAT a server providing DNS and DHCP services; this machine is also used for terminal services so users are logged in to this machine directly assorted clients I've had squid set up on a 'opt-in' basis. Now I have a request to make it transparent for all users with the intent of disabling web access during specified hours. The problem I have is that my firewall is not able to run squid, and all the examples assume that the squid box is either the firewall or provides NAT. Is it possible, without a huge amount of complications, to run squid on this sort of setup? If so, does anyone have a recipe for doing so? Squid box had best be the one doing NAT because all source info is lost during NAT interception and Squid needs to look it up. Note I wrote "NAT interception", thats a more correct name for "transparent". Squid does not have to be on the firewall or router to do NAT though: http://wiki.squid-cache.org/ConfigExamples/Intercept/IptablesPolicyRoute the tutorial ironically was written for people using OpenWRT :) Amos Hi Amos: Obvously I got something just half right: The requested URL could not be retrieved While trying to retrieve the URL: http://arstechnica.com/tech-policy/news/2009/05/landmark-study-drm-truly-does-make-pirates-out-of-us-all.ars The following error was encountered: Unable to determine IP address from host name for /arstechnica.com/ The dnsserver returned: Server Failure: The name server was unable to process this query. This means that: Is it actually using the '/' there? It looks a lot like the 'transparent' option to http_port is missing still. I've configured this as best as I can following http://wiki.squid-cache.org/ConfigExamples/Intercept/IptablesPolicyRoute on the firewall/router and http://wiki.squid-cache.org/ConfigExamples/Intercept/LinuxRedirect on the squid box. As soon as I enable the iptablesPolicyRoute on the fw my DNS fails I can't figure out why Those rules should only affect tcp packets to port 80. Does anyone have this setup working? Could they please send me some instructions for morons? That was them ;). Does the Squid box have normal DNS if its used as a regular proxy without the PolicyRouting? Amos DUH! OK, my turn to feel stupid Turns out my firewall rules were blocking forwarding from internal_if to internal_if - so the firewall "loopback" to the squid box was getting dropped. Now everything is OK; on to the next step - time based web access - which is why I started this whole thing! --Yan !DSPAM:4a214d26279661804284693! Amos: Since ordinary mortals like me ;-) apparently cannot edit the wiki, here's my final form of the iptables and policy route rules. First on the firewall: iptables -t mangle -A PREROUTING -j ACCEPT -p tcp --dport 80 -s squidboxIP iptables -t mangle -A PREROUTING -j MARK --set-mark 3 -p tcp --dport 80 iptables -A FORWARD -i internalif -o internalif -p tcp --dport 80 -j ACCEPT ip rule add fwmark 3 table proxy ip route add default via squidboxIP dev internalif table proxy Then on the squid box itself: iptables -t nat -A PREROUTING -s localnet/24 -i lan0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128 iptables -t nat -A OUTPUT -s squidboxIP/32 -p tcp -m owner ! --uid-owner squid-user -m tcp --dport 80 -j REDIRECT --to-ports 3128 The first rule catches all of the requests that have been redirected by the firewall for proxying. The second rule catches all of the outbound packets originating on the squidbox itself. This can have some weird side effects; my squid box also handles several ipcams and so all this traffic is now routed through squid. Note that you have to change the uid-owner to the uid of the squid process. With this setup, the firewall still does nat; squid just caches the traffic. --Yan
Re: [squid-users] client_side_request.cc
Wong wrote: Dear All, I experienced messages below and squid exiting abnormally. Squid version 3S15 Need your advise & help. Thx & Rgds, Wong ---snip--- 2009/06/01 08:29:27| client_side_request.cc(825) redirecting body_pipe 0x85fd94c*1 from request 0x8525c90 to 0x886bcd0 These are normal. Visible only because of the level of debug_options. 2009/06/01 10:05:51| Preparing for shutdown after 67188 requests 2009/06/01 10:05:51| Waiting 5 seconds for active connections to finish 2009/06/01 10:05:51| FD 25 Closing HTTP connection 2009/06/01 10:05:51| WARNING: redirector #1 (FD 10) exited 2009/06/01 10:05:51| WARNING: redirector #9 (FD 18) exited 2009/06/01 10:05:51| Too few redirector processes are running 2009/06/01 10:05:51| Starting new helpers 2009/06/01 10:05:51| helperOpenServers: Starting 9/15 'squidGuard' processes 2009/06/01 10:05:52| WARNING: redirector #10 (FD 19) exited I assume the problem you are reporting is the redirectors starting up again during a shutdown. Is this correct? Amos -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE15 Current Beta Squid 3.1.0.8 or 3.0.STABLE16-RC1
