Re: [squid-users] Even/Odd SRC ACL
On Friday 27 June 2014 11:58 AM, Nishant Sharma wrote: On Friday 27 June 2014 10:05 AM, Amos Jeffries wrote: acl even src 0.0.0.0/0.0.0.1 tcp_outgoing_address wan1 even tcp_outgoing_address wan2 !even wan1 wan2 in the config are the actual WAN IP Addresses (IPv4) and NAT rules are properly set-up for both the WANs. If I divide the LAN into two /25 subnets it works fine. But not with masked bits. Is there any debug option that I could enable to see how these ACLs are being matched or by-passed? debug_options ALL,9 can be an overkill for this? Here are the debug logs. I see that it is trying to compare SRC-IP:Port pair against the ACL and result is always 0. Any pointers? 2014/06/27 12:02:37.882| ACLList::matches: checking !EVEN 2014/06/27 12:02:37.883| ACL::checklistMatches: checking 'EVEN' 2014/06/27 12:02:37.883| aclIpAddrNetworkCompare: compare: 192.168.2.121:49287/[:::::::] (192.168.2.121:49287) vs 0.0.0.0-[::]/[:::::::] 2014/06/27 12:02:37.883| aclIpMatchIp: '192.168.2.121:49287' NOT found 2014/06/27 12:02:37.883| ACL::ChecklistMatches: result for 'EVEN' is 0 2014/06/27 12:02:37.883| ACLList::matches: result is true 2014/06/27 12:02:37.883| aclmatchAclList: 0xbfbfe290 returning true (AND list satisfied) 2014/06/27 12:02:37.883| ACLChecklist::markFinished: 0xbfbfe290 checklist processing finished 2014/06/27 12:02:37.883| FilledChecklist.cc(168) ~ACLFilledChecklist: ACLFilledChecklist destroyed 0xbfbfe290 2014/06/27 12:02:37.883| ACLChecklist::~ACLChecklist: destroyed 0xbfbfe290 2014/06/27 12:02:37.883| FilledChecklist.cc(168) ~ACLFilledChecklist: ACLFilledChecklist destroyed 0xbfbfe2b0 2014/06/27 12:02:37.883| ACLChecklist::~ACLChecklist: destroyed 0xbfbfe2b0 2014/06/27 12:02:37.883| fwdConnectStart: got outgoing addr 2xx.1xx.3x.xx, tos 0 Thanks regards, Nishant
Re: [squid-users] WARNING: swapfile header inconsistent with available data
On Thu, 2014-06-26 at 18:03 +0300, Eliezer Croitoru wrote: On 06/25/2014 04:06 PM, Jasper Van Der Westhuizen wrote: As a matter of interest, in my cache logs I see many lines like these 2014/06/25 14:52:58 kid1| WARNING: swapfile header inconsistent with available data 2014/06/25 14:53:28 kid1| WARNING: swapfile header inconsistent with available data What Storage are you using? What is the fstab content? Eliezer Hi Eliezer These proxies are VM's with the cache and logs sitting on ext3 LV's. /dev/VG01/SQUIDCACHE /var/cache/squid ext3 defaults1 1 Regards Jasper
Re: [squid-users] Even/Odd SRC ACL
On 27/06/2014 6:43 p.m., Nishant Sharma wrote: On Friday 27 June 2014 11:58 AM, Nishant Sharma wrote: On Friday 27 June 2014 10:05 AM, Amos Jeffries wrote: acl even src 0.0.0.0/0.0.0.1 tcp_outgoing_address wan1 even tcp_outgoing_address wan2 !even wan1 wan2 in the config are the actual WAN IP Addresses (IPv4) and NAT rules are properly set-up for both the WANs. If I divide the LAN into two /25 subnets it works fine. But not with masked bits. Is there any debug option that I could enable to see how these ACLs are being matched or by-passed? debug_options ALL,9 can be an overkill for this? Here are the debug logs. I see that it is trying to compare SRC-IP:Port pair against the ACL and result is always 0. Any pointers? Ah, Squid-3 is using CIDR masking. Sorry should have remembered earlier how strict this is. The two /25 subnets (or groups of /26 etc) is the way to go. Amos
Re: [squid-users] WARNING: swapfile header inconsistent with available data
On 27/06/2014 6:53 p.m., Jasper Van Der Westhuizen wrote: On Thu, 2014-06-26 at 18:03 +0300, Eliezer Croitoru wrote: On 06/25/2014 04:06 PM, Jasper Van Der Westhuizen wrote: As a matter of interest, in my cache logs I see many lines like these 2014/06/25 14:52:58 kid1| WARNING: swapfile header inconsistent with available data 2014/06/25 14:53:28 kid1| WARNING: swapfile header inconsistent with available data What Storage are you using? What is the fstab content? Eliezer Hi Eliezer These proxies are VM's with the cache and logs sitting on ext3 LV's. /dev/VG01/SQUIDCACHE /var/cache/squid ext3 defaults1 1 Regards Jasper Do any of those cache_dir share an LV? Amos
Re: [squid-users] Even/Odd SRC ACL
On Friday 27 June 2014 12:34 PM, Amos Jeffries wrote: Ah, Squid-3 is using CIDR masking. Sorry should have remembered earlier how strict this is. The two /25 subnets (or groups of /26 etc) is the way to go. Thanks for the clarification. So, would it be possible in future? I don't know how complicated it would be to implement. Thanks again. Regards, Nishant
Re: [squid-users] Even/Odd SRC ACL
Hi Sharma, would using a random ACL for outgoing IP selection be good enough? Francesco On Fri, Jun 27, 2014 at 9:18 AM, Nishant Sharma codemarau...@gmail.com wrote: On Friday 27 June 2014 12:34 PM, Amos Jeffries wrote: Ah, Squid-3 is using CIDR masking. Sorry should have remembered earlier how strict this is. The two /25 subnets (or groups of /26 etc) is the way to go. Thanks for the clarification. So, would it be possible in future? I don't know how complicated it would be to implement. Thanks again. Regards, Nishant -- Francesco
Re: [squid-users] Even/Odd SRC ACL
Hi Francesco, On Friday 27 June 2014 01:35 PM, Kinkie wrote: Hi Sharma, would using a random ACL for outgoing IP selection be good enough? Francesco random ACL with sticky outgoing IP per client to take care of HTTPS and badly designed HTTP portals which log a user off if his/her source IP changes. Thanks regards, Nishant
[squid-users] Reverse proxy and TCP_MISS_ABORTED:FIRSTUP_PARENT
Hi, we are running a squid 3.4 in accel mode for one https site. We frequently see somthing like PUT https://domain.com/file.txt HTTP/1.1 0 0 TCP_MISS_ABORTED:FIRSTUP_PARENT I tried to find out what this should mean; does it tell me that the parent did not respond in time or does this refer to the client? Regards, Jakob
Re: [squid-users] WARNING: swapfile header inconsistent with available data
On Fri, 2014-06-27 at 19:06 +1200, Amos Jeffries wrote: On 27/06/2014 6:53 p.m., Jasper Van Der Westhuizen wrote: On Thu, 2014-06-26 at 18:03 +0300, Eliezer Croitoru wrote: On 06/25/2014 04:06 PM, Jasper Van Der Westhuizen wrote: As a matter of interest, in my cache logs I see many lines like these 2014/06/25 14:52:58 kid1| WARNING: swapfile header inconsistent with available data 2014/06/25 14:53:28 kid1| WARNING: swapfile header inconsistent with available data What Storage are you using? What is the fstab content? Eliezer Hi Eliezer These proxies are VM's with the cache and logs sitting on ext3 LV's. /dev/VG01/SQUIDCACHE /var/cache/squid ext3 defaults1 1 Regards Jasper Do any of those cache_dir share an LV? Amos Hi Amos No, each proxy server has it's own cache_dir on it's own LV. Regards Jasper
Re: Fwd: Re: [squid-users] google picking up squid as
How about contacting google for advise? They are the one that forces you to the issue. They don't like it that you have a 1k clients behind your IP address. They should tell you what to do. You can tell them that you are using squid as a forward proxy to enforce usage acls on users inside the network. It's not a share to use squid... It's a shame that you cannot get a reasonable explanation to the reason you are blocked... There is only 1 client behind the IP address as it is a test server so something is going wrong with either routing or requests to google. Google will not answer any emails. I suppose one alternative is to use unbound in conjunction with squid and not redirect any requests to google?
[squid-users] read_timeout
Hello, Is it possible to set read_timeout value to a negative value in order to have infinite timeout on this event ? I use Squid Cache: Version 2.7.STABLE9, I try to set read_timeout to -1 but I have some assert in commSetTimeout() which crash squid daemon. Thanks
[squid-users] ssl-bump not working in non transparent mode
I have setup a squid 3.3.9 with ssl-bump enabled. When i access through transparent mode its working fine but when i use proxy address in my borrower (non transparent mode) its not working. Following is my squid configuration: visible_hostname 10.10.16.56 http_port 10.10.16.56:3127 intercept http_port 10.10.16.56:3128 https_port 10.10.16.56:3129 generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/mycert.pem intercept ssl-bump always_direct allow all ssl_bump server-first all sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s /usr/local/squid/var/lib/ssl_db -M 4MB sslcrtd_children 10 hierarchy_stoplist cgi-bin ? negative_ttl 0 icap_enable on icap_send_client_ip on icap_service service_req reqmod_precache bypass=1 icap://127.0.0.1:1344/srv_clamav adaptation_access service_req allow all icap_service service_resp respmod_precache bypass=1 icap://127.0.0.1:1344/srv_clamav adaptation_access service_resp allow all
[squid-users] missing SNI support in squid makes trouble with sslbump
Hello, I use squid 3.4.5 and sslbump works great for the most big sites like google and facebook There are some destinations, which share there ip with other virual webserver, so the client gets a default certificate from the server with a wrong CN. With SNI the client get the right certificate with the correct CN. I configured ssl_bump server-first all, but to me it looks like squid doesn't do SNI and so gets the wrong certificate. Does anybody know a workaround for this problem ? -- Regards Dieter -- I do not get viruses because I do not use MS software. If you use Outlook then please do not put my email address in your address-book so that WHEN you get a virus it won't use my address in the From field.
Re: [squid-users] read_timeout
OK, if i understand a negative read_timeout value reset global structure of timeout. So, is a 0 value for read_timeout token in squid conf file means no timeout ? Thanks for your answer On 06/27/14 14:43, Jeremy Hustache wrote: Hello, Is it possible to set read_timeout value to a negative value in order to have infinite timeout on this event ? I use Squid Cache: Version 2.7.STABLE9, I try to set read_timeout to -1 but I have some assert in commSetTimeout() which crash squid daemon. Thanks
RE: Fwd: Re: [squid-users] google picking up squid as
That's very odd. I'd try calling them... There are quite a few folks blocking proxies these days. What I do is remove the via and forwarded for headers with the following command: check_hostnames off forwarded_for delete via off I realize this breaks the RFC, but lest be blocked if detected as a squid proxy. sux Best regards, The Geek Guy Lawrence Pingree http://www.lawrencepingree.com/resume/ Author of The Manager's Guide to Becoming Great http://www.Management-Book.com -Original Message- From: sq...@proxyplayer.co.uk [mailto:sq...@proxyplayer.co.uk] Sent: Friday, June 27, 2014 4:43 AM To: squid-users@squid-cache.org Subject: Re: Fwd: Re: [squid-users] google picking up squid as How about contacting google for advise? They are the one that forces you to the issue. They don't like it that you have a 1k clients behind your IP address. They should tell you what to do. You can tell them that you are using squid as a forward proxy to enforce usage acls on users inside the network. It's not a share to use squid... It's a shame that you cannot get a reasonable explanation to the reason you are blocked... There is only 1 client behind the IP address as it is a test server so something is going wrong with either routing or requests to google. Google will not answer any emails. I suppose one alternative is to use unbound in conjunction with squid and not redirect any requests to google?
Re: [squid-users] missing SNI support in squid makes trouble with sslbump
On 06/27/2014 07:44 AM, Dieter Bloms wrote: I use squid 3.4.5 and sslbump works great for the most big sites like google and facebook There are some destinations, which share there ip with other virual webserver, so the client gets a default certificate from the server with a wrong CN. With SNI the client get the right certificate with the correct CN. I configured ssl_bump server-first all, but to me it looks like squid doesn't do SNI and so gets the wrong certificate. You are correct. With server-first bumping, Squid does not know client SNI when establishing the connection with the server. With client-first bumping, Squid knows client SNI (IIRC), but does not know the server certificate details when establishing the connection with the client and so it cannot mimic some important ones. Pick your poison. Does anybody know a workaround for this problem ? Unfortunately, there is no way to detect SNI presence using ACLs right now. Furthermore, client-first is not compatible with many transactions, especially if you intercept. You can use TCP/IP-level information to instruct Squid to use client-first bumping for sites that require SNI _and_ appear to work in client-first mode (if any). The Peek and Splice project lays the groundwork for proper SNI support, but there is currently no volunteer or sponsor for the SNI-specific changes IIRC: http://wiki.squid-cache.org/Features/SslPeekAndSplice Cheers, Alex.
Re: [squid-users] read_timeout
On 06/27/2014 07:56 AM, Jeremy Hustache wrote: OK, if i understand a negative read_timeout value reset global structure of timeout. So, is a 0 value for read_timeout token in squid conf file means no timeout ? I did not check Squid2 sources, but AFAICT, Squid3 does not treat a zero read_timeout value specially, and I doubt it should. Squid should check for overflows instead, but does not (yet?). If you want a large read_timeout, use a large value. For example, two years should be large enough for virtually all practical purposes and small enough to prevent (current time + timeout) overflows in the foreseeable future. Please note that large timeouts create stuck connections in most deployment environments, and those stuck connections not only consume file descriptors but may eat 10s of MBs of RAM in environments where Squid opens SSL connections to servers. HTH, Alex. On 06/27/14 14:43, Jeremy Hustache wrote: Hello, Is it possible to set read_timeout value to a negative value in order to have infinite timeout on this event ? I use Squid Cache: Version 2.7.STABLE9, I try to set read_timeout to -1 but I have some assert in commSetTimeout() which crash squid daemon. Thanks
Re: Fwd: Re: [squid-users] google picking up squid as
On 06/27/2014 09:00 AM, Lawrence Pingree wrote: forwarded_for delete via off I realize this breaks the RFC, More importantly, it breaks Squid's loop detection mechanism. In many environments, breaking that mechanism creates an easy-to-abuse Squid DoS attack vector. Modern Squids have a workaround that can partially restore the loop cutting code AFAICT: Consider adding request_header_add X-UseSomeUniqueNameHere useAnyValueHere all to your squid.conf so that looping HTTP request headers get larger and larger with every iteration until Squid refuses to process the looping request. To cut loops faster, you can also deny incoming requests that carry that unique-to-your-setup header. HTH, Alex.
[squid-users] What is a reasonable size for squid.conf?
I am running a non-caching reverse proxy using version 3.3.10. My squid.conf is currently clocking in 60k lines (not including comments or blank lines). Combined with the conf files in my conf.d directory, I have a total of 89k lines in configuration. I have definitely noticed -k reconfigure calls taking on the order of 20 seconds to run when it used to be less than a couple seconds. (Same results with -k test). I've tried searching for anything related to max lines and similar, but it usually talks about squid.conf configuration options and not the file itself. If this is not documented per se, are there any anecdotal examples that have this many lines or more? I only see this growing over time. Thanks, Owen
[squid-users] SNMP cacheClients
I am monitoring my squid server via SNMP and graphing in Cacti. Of particular importance to me is the number of clients which is a graph of the cacheClients statistic (1.3.2.1.15.0). The graph shows we reach a maximum of 1300 clients. This seems a bit odd to me as we only have around 200 users. Even if you double that (each user has their desktop and wireless device) you don't get anywhere close to 1300. Therefore what is this SNMP value truly reporting? What constitutes a client? Is it per IP? Thanks
Re: [squid-users] What is a reasonable size for squid.conf?
My squid.conf is 3380 bytes, and 99 total lines, with around 35 lines blank or commented out. If you had been upgrading from any 3.1 or older squid, they had a LOT of unnecessary lines in there for TAG related entries and excess documentation of every little line. Mike On 6/27/2014 2:51 PM, Owen Crow wrote: I am running a non-caching reverse proxy using version 3.3.10. My squid.conf is currently clocking in 60k lines (not including comments or blank lines). Combined with the conf files in my conf.d directory, I have a total of 89k lines in configuration. I have definitely noticed -k reconfigure calls taking on the order of 20 seconds to run when it used to be less than a couple seconds. (Same results with -k test). I've tried searching for anything related to max lines and similar, but it usually talks about squid.conf configuration options and not the file itself. If this is not documented per se, are there any anecdotal examples that have this many lines or more? I only see this growing over time. Thanks, Owen
Re: [squid-users] SNMP cacheClients
On 06/27/2014 02:11 PM, Matthew Ceroni wrote: I am monitoring my squid server via SNMP and graphing in Cacti. Of particular importance to me is the number of clients which is a graph of the cacheClients statistic (1.3.2.1.15.0). The graph shows we reach a maximum of 1300 clients. This seems a bit odd to me as we only have around 200 users. Even if you double that (each user has their desktop and wireless device) you don't get anywhere close to 1300. Therefore what is this SNMP value truly reporting? What constitutes a client? Is it per IP? I am not an SNMP expert, but I believe SNMP cacheClients is Number of clients accessing cache on the Cache Manager mgr:info page (you can check by correlating SNMP and mgr:info output). If you are using SMP Squid (e.g., multiple Squid workers), please see the Number of clients accessing cache table row at http://wiki.squid-cache.org/Features/CacheManager#SMP_considerations HTH, Alex.
Re: [squid-users] Force ASCII encoding for access.log fields?
[serverIP],[clientIP], 4012,692,498,GET,200,º^_x°*,username,20/Jun/2014:00:06:36 The log format you used does not match this log line. The format produces: [squid-listening-IP],[clientIP], 4012,692,498,GET,200,º^_x°*,username,20/Jun/2014:00:06:36 Thanks for the correction. To expand on that point, on some of our proxies, we have more than one IP being serviced by a single daemon. Recording which IP received the traffic is essential to proper accounting (e.g. FreeRADIUS). URL-encoding is the %xx character encoding, it can be (and is) applied to anything which can legitimately contain non-ASCII characters or ASCII special characters. Content-Type header is not one of those places. You can use the '#' format modifier to URL-encode that %mt field explicitly. Like so: %#mt Amos, thank you so much for sharing this. I plan to try it as soon as ... If you will share the exact Squid version you are using I would also like to check the code to see if the mt code is being correctly setup, that log entry looks a bit like random memory being displayed as if it were text. ... as soon as I finish upgrading from squid-3.1.10-16.el6 to 3.1.10-20.el6, both of which are packaged and delivered via the CentOS repo :). Totally ashamed I didn't even notice there was an update available before posting. I plan to schedule an outage to patch and I'll report back with my findings. If you suspect random memory chunks are being written to the file as a consequence of this outdated version of Squid, and even the more recent version I plan to move to does not address this condition, feel free to share. This particular proxy is pretty active. We're averaging between 800,000 - 1.2M lines in the access log per day. The proxy is non-caching, running with 512MB RAM and 1GB swap (don't ask). More soon, MD
