RE: [squid-users] Squid 3.4.6 is not caching anything
Hi, Sadly I have deleted my access.log files - and have rolled back to Squid 3.1, and caching is working perfectly. I still need to upgrade to Squid 3.4.6 sometime soon for the sslbump feature. I tried with the djmaza.info site with the Squid 3.4.6, and nothing is cached. I am going to try and set up a testing Squid 3.4.6 server this weekend and will reply to this message if I am still having problems with caching. -Original Message- From: Eliezer Croitoru [mailto:elie...@ngtech.co.il] Sent: Monday, 30 June 2014 2:16 p.m. To: squid-users@squid-cache.org Subject: Re: [squid-users] Squid 3.4.6 is not caching anything Hey Liam, If you can run a test on the access.log it would supply a bit more information without intruding to the url level: cat access.log |awk '{print $4}'|sort|uniq -c The result will be a tiny statistics about the "character" of your usage. Since browsers tends to cache content them-self sometimes that cache is giving you something you cannot just see by looking for a "HIT" in this form or another. I remember that most squid analytical tools are testing for "HIT" objects ignoring all other sides of the cache. If you have tried djmaza as I suggested and you have not seen a single HIT when surfing it there is indeed something strange. I myself use squid 3.4.5 and I do see that there is not HIGH rate of HITs but I do understand why it can happens and sometimes even understand why it happens. Once you will have the results of the access.log parsing we will be smarter. Eliezer On 06/30/2014 05:05 AM, l...@kzz.se wrote: > I have tried deleting the cache and setting its size to 10GB. I ran > squid -z again and it created the directories before squid -z froze. > The maximum object size is set to 5GB, and I have checked some sites > using redbot.org to see if the can be cached or not. It says that they > can, and I have had the squid proxy running for about 48 hrs now with > about 50 clients connected. I have scanned the access.log and there is > not a single hit. Even if the same page is requested many times. > > Are there some settings that I am missing in squid.conf that is > stopping the cache from working? Do you know where I can obtain a > already compiled x86 package for Debian 7 with --enable-ssl and --enable-ssl-crtd? > > Thanks for your help so far.
Re: [squid-users] SSL bump working on most site...cert pinning issue?
On Mon, 2014-06-30 at 22:56 +1000, Dan Charlesworth wrote: > Yeah, pinned SSL ‘aint gonna be bumped. The Twitter apps are another popular > one that use pinning. > > As far as your broken_sites ACL goes, you can’t use `dstdomain` because the > only thing Squid can see of the destination before bumping an intercepted > connection is the IP address. So for `ssl_bump none` you’ll need to be use > `dst` ACLs instead. > > ProTip: Here are the Apple and Akamai public IP blocks (to use in a dst > equivalent of your broken_sites), respectively: 17.0.0.0/8, 23.0.0.0/12. > > Good luck > > On 30 Jun 2014, at 10:38 pm, James Lay wrote: > > > Topic pretty much says it...most sites work fine using my below set up, > > but some (Apple's app store) do not. I'm wondering if cert pinning is > > the issue? Since this set up is basically two separate sessions, I > > packet captured both. The side the I have control over gives me a TLS > > Record Layer Alert Close Notify. I am unable to decrypt the other side > > as the device in question is an iDevice and I can't capture the master > > secret. > > > > I've even tried to ACL certain sites to not bump, but they don't go > > through. Below is my complete setup. This is running the below: > > > > Squid Cache: Version 3.4.6 > > configure options: '--prefix=/opt' '--enable-icap-client' > > '--enable-ssl' '--enable-linux-netfilter' > > '--enable-follow-x-forwarded-for' '--with-large-files' > > '--sysconfdir=/opt/etc/squid' > > > > > > Any assistance with troubleshooting would be wonderful...thank you. > > > > James > > > > > > > > $IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport > > 80 -j REDIRECT --to-port 3128 > > $IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport > > 443 -j REDIRECT --to-port 3129 > > > > > > acl localnet src 192.168.1.0/24 > > > > acl SSL_ports port 443 > > acl Safe_ports port 80 # http > > acl Safe_ports port 21 # ftp > > acl Safe_ports port 443 # https > > acl Safe_ports port 70 # gopher > > acl Safe_ports port 210 # wais > > acl Safe_ports port 1025-65535 # unregistered ports > > acl Safe_ports port 280 # http-mgmt > > acl Safe_ports port 488 # gss-http > > acl Safe_ports port 591 # filemaker > > acl Safe_ports port 777 # multiling http > > > > acl CONNECT method CONNECT > > acl broken_sites dstdomain textnow.me > > acl broken_sites dstdomain akamaiedge.net > > acl broken_sites dstdomain akamaihd.net > > acl broken_sites dstdomain apple.com > > acl allowed_sites url_regex "/opt/etc/squid/url.txt" > > acl all_others dst all > > acl SSL method CONNECT > > > > > > http_access deny !Safe_ports > > http_access deny CONNECT !SSL_ports > > > > http_access allow manager localhost > > http_access deny manager > > > > http_access allow allowed_sites > > http_access deny all_others > > http_access allow localnet > > http_access allow localhost > > > > http_access deny all > > icp_access deny all > > > > sslproxy_cert_error allow broken_sites > > sslproxy_cert_error deny all > > > > sslproxy_options ALL > > ssl_bump none broken_sites > > ssl_bump server-first all > > > > http_port 192.168.1.253:3128 intercept > > https_port 192.168.1.253:3129 intercept ssl-bump > > generate-host-certificates=on cert=/opt/sslsplit/sslsplit.crt > > key=/opt/sslsplit/sslsplitca.key options=ALL sslflags=NO_SESSION_REUSE > > > > always_direct allow all > > > > > > hierarchy_stoplist cgi-bin ? > > > > access_log syslog:daemon.info common > > > > refresh_pattern ^ftp: 144020% 10080 > > refresh_pattern ^gopher:14400% 1440 > > refresh_pattern -i (cgi-bin|\?) 0 0% 0 > > refresh_pattern . 0 20% 4320 > > > > icp_port 3130 > > > > coredump_dir /opt/var > > > > So adding: acl broken_sites dst 23.0.0.0/12 now gives me the below: Jun 30 20:16:51 gateway (squid-1): 192.168.1.100 - - [30/Jun/2014:20:16:51 -0600] "CONNECT 23.204.162.217:443 HTTP/1.1" 403 3385 TCP_DENIED:HIER_NONE Jun 30 20:16:51 gateway (squid-1): 192.168.1.100 - - [30/Jun/2014:20:16:51 -0600] "NONE error:invalid-request HTTP/0.0" 400 3981 TAG_NONE:HIER_NONE So something is off. Any help on these beastie? Thank you. James
Re: [squid-users] Fwd: Squidblacklist.org - A better blacklist for Squid-ACL. Blacklisting Evolved.
On Monday 30 June 2014 at 16:12:58, James Lay wrote: > Please don't peddle your (subscription fee based no less...yugh) > garbage Just out of interest, I took a look at what was being offered by this guy (http://www.squidblacklist.org) and I noticed two things: 1. It's a subscription-based service 2. It's licensed under "Creative Commons Attribution 3.0 Unported License" with a direct link to http://creativecommons.org/licenses/by/3.0/deed.en_US That link states "You are free to: Share — copy and redistribute the material in any medium or format Adapt — remix, transform, and build upon the material for any purpose, even commercially." So, I contacted the original poster of the promotional email (not to this list, as far as I can tell, although the reply was copied here), asking "Does this mean that if I subscribe to your list, I can sell the content on to my customers?" and got the following interesting reply: > You read and interpret correctly. > > What our subscribers do with the lists we provide is none of our concern. > > > -- > Signed, > > Benjamin E. Nichols > http://www.squidblacklist.org So, if anyone thinks there's even the slightest value in using these lists, we only need a single subscription between us, and then the lists can be distributed for free (or 1¢ per copy, or whatever someone thinks is reasonable). So, it may be subscription-only, but we could easily make it one subscription per world, if we want to. Antony. -- This sentence contains exactly threee erors. Please reply to the list; please don't CC me.
[squid-users] Re: Connection pinning in Squid 3.1
Any reason not to build squid from newest sources ? Will probably increase your chances of getting better support, as 2.1 is not much newer than 2.7 :-) (Still using latest 2.7, with private mods, myself. Solid as a rock.) -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Connection-pinning-in-Squid-3-1-tp4666560p4666562.html Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] Re: Probs with squid 3.4.4 and cache_peer parent
Did you try without Antivirus ? Not so into the squid code, but I would suspect a problem in the interface to Trend, first. As squid is crashing already during/immediately after startup. BTW: What should happen here ? maximum_object_size 1 KB maximum_object_size 50 MB Probably, you can delete the first of them, in both squid.conf's MfG -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Probs-with-squid-3-4-4-and-cache-peer-parent-tp4666557p4666561.html Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] Connection pinning in Squid 3.1
Hi, I'm having trouble with connection pinning. I'm on SUSE Linux Enterprise (SLES) 11 SP3, so I'm stuck with squid3-3.1.12-8.16.18.1 at the moment. My scenario: Firefox, Squid and a parent proxy (McAfee Web Gateway). The parent proxy offers "Proxy-Authenticate: Negotiate" and "Proxy-Authenticate: NTLM" to provide for single sign on. Firefox jumps on "Negotiate" the first time but the parent proxy knows about Firefox's problem and offers only "NTLM" the next time. This scenario has been working with Squid 2.7 for quite some time (years actually). Now I'm in the process of migrating to Squid 3.1. The configuration condenses to: http_port 8080 acl me src 1.2.3.4/32 http_access allow me http_access deny all cache_peer myparent.dmz.prv parent 8080 0 no-query \ no-digest login=PASS name=myparent.dmz.prv cache_peer_access myparent.dmz.prv allow always_direct deny all never_direct allow all I tried with "connection-auth=on" at "http_port" and "cache_peer" but that did not help. The name= clause seems redundant, it is an artifact of a local load balancer configuration. I removed it to eliminate possible interferences. Originally it was: cache_peer 127.0.0.1 parent 8090 0 no-query \ no-digest login=PASS name=myparent.dmz.prv I can see with tcpdump that Squid not even remotely maintains a 1:1 relationship between inbound and outbound TCP connections. Instead, it seems to jump on the first free outbound connection for nearly every incoming request. This reliably breaks the NTLM authentication scheme and as a result password requests keep popping up in the browser. I could probably resort to 2.7.STABLE5, which is delivered with SLES 11 SP3 too. But that seems to be the cowards way :-) and I still have some time to do some tests before moving towards production. So if anyone would take the time and guide me through some debugging I would be happy to help sorting this out. Kind regards, Robert
Re: [squid-users] SSL bump working on most site...cert pinning issue?
On 2014-06-30 07:13, Dan Charlesworth wrote: No worries. Sounds like this is the feature you should be waiting with baited breath for: http://wiki.squid-cache.org/Features/SslPeekAndSplice I’m not a developer so I have no idea how far along that is right now. On 30 Jun 2014, at 11:05 pm, James Lay wrote: On Mon, 2014-06-30 at 22:56 +1000, Dan Charlesworth wrote: Yeah, pinned SSL ‘aint gonna be bumped. The Twitter apps are another popular one that use pinning. As far as your broken_sites ACL goes, you can’t use `dstdomain` because the only thing Squid can see of the destination before bumping an intercepted connection is the IP address. So for `ssl_bump none` you’ll need to be use `dst` ACLs instead. ProTip: Here are the Apple and Akamai public IP blocks (to use in a dst equivalent of your broken_sites), respectively: 17.0.0.0/8, 23.0.0.0/12. Good luck On 30 Jun 2014, at 10:38 pm, James Lay wrote: Topic pretty much says it...most sites work fine using my below set up, but some (Apple's app store) do not. I'm wondering if cert pinning is the issue? Since this set up is basically two separate sessions, I packet captured both. The side the I have control over gives me a TLS Record Layer Alert Close Notify. I am unable to decrypt the other side as the device in question is an iDevice and I can't capture the master secret. I've even tried to ACL certain sites to not bump, but they don't go through. Below is my complete setup. This is running the below: Ah good catch thank you. I've seen expensive proxy appliances just tunnel the traffic through, but they get the host and domain name to all control...which is really all I'm wanting to do is control what sites are allowed. I'll give your suggestions a go...thank you. James Thanks Dan..looks like that's what I'll be watching for. James
Re: [squid-users] Intercept HTTPS without using certificates - Just apply a QoS on the connexion
If your company allows you, you could look into a relatively inexpensive Linux-based software router called Mikrotik. They have something called PCQ which does well as a QOS policy. Regards On Fri, May 16, 2014 at 7:03 PM, Antoine Klein wrote: > Ok i fear to waste many time to understand that, but it could be interesting > ^^ > > Thanks for your replies ! > > 2014-05-15 15:10 GMT-04:00 Alex Crow : >> Hi, >> >> Welcome to the practically incomprehensible world of QoS on Linux - look up >> "LARTC" and then feel the fear! >> >> It's really powerful but even after 14 years of managing Linux gateways I >> still prefer you just use shorewall to take away the complexity - and you >> are welcome to call me lazy ;-) >> >> Alex >> >> >> On 15/05/14 20:04, Antoine Klein wrote: >>> >>> Ok thanks, it could be a good idea ! >>> >>> Do you know if we can apply a QoS with the bucket concept of delay >>> pool using the Linux QoS Tools ? >>> >>> 2014-05-15 14:41 GMT-04:00 Leonardo Rodrigues : Em 15/05/14 14:59, Antoine Klein escreveu: > Hi there, > > I need to install squid to apply a QoS in a private network with the > delay > pool. > In fact, this network offer a public WIFI, so that's not possible to > configure a proxy on clients. > > Is it possible to intercept HTTPS connexion, apply a Delay Pool and > forward the request without decipher the SSL packet ? > I really dont think that's possible. Anyway, you can always use your Linux (or whatever OS you're using) QoS tools to acchieve something similar to delay pools but on NATted connections. You can have squid intercepting TCP/80 connections and apply delay pools, the TCP/443 (and all other indeed) connections can be throttled by QoS SO tools. -- Atenciosamente / Sincerily, Leonardo Rodrigues Solutti Tecnologia http://www.solutti.com.br Minha armadilha de SPAM, NÃO mandem email gertru...@solutti.com.br My SPAMTRAP, do not email it >>> >>> >> > > > > -- > Antoine KLEIN
[squid-users] Probs with squid 3.4.4 and cache_peer parent
Hello, I've setup a internal proxy with squid 3.4.4 on SLES 11 SP3. And with the same version of squid and OS a proxy in DMZ. The internal proxy crashed every 5 minutes. I can't find the reason. 2014/06/30 16:09:06 kid1| Set Current Directory to /var/cache/squid 2014/06/30 16:09:06 kid1| Starting Squid Cache version 3.4.4 for x86_64-suse-linux-gnu... 2014/06/30 16:09:06 kid1| Process ID 31884 2014/06/30 16:09:06 kid1| Process Roles: worker 2014/06/30 16:09:06 kid1| With 40096 file descriptors available 2014/06/30 16:09:06 kid1| Initializing IP Cache... 2014/06/30 16:09:06 kid1| DNS Socket created at 0.0.0.0, FD 8 2014/06/30 16:09:06 kid1| Adding nameserver 194.99.121.30 from squid.conf 2014/06/30 16:09:06 kid1| Adding nameserver 212.121.128.10 from squid.conf 2014/06/30 16:09:06 kid1| Adding nameserver 10.20.94.32 from squid.conf 2014/06/30 16:09:06 kid1| helperOpenServers: Starting 0/200 'squidGuard' processes 2014/06/30 16:09:06 kid1| helperOpenServers: No 'squidGuard' processes needed. 2014/06/30 16:09:06 kid1| helperOpenServers: Starting 0/128 'ntlm_auth' processes 2014/06/30 16:09:06 kid1| helperStatefulOpenServers: No 'ntlm_auth' processes needed. 2014/06/30 16:09:07 kid1| helperOpenServers: Starting 10/80 'ext_ldap_group_acl' processes 2014/06/30 16:09:07 kid1| Logfile: opening log udp://127.0.0.1: 2014/06/30 16:09:07 kid1| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec 2014/06/30 16:09:07 kid1| Store logging disabled 2014/06/30 16:09:07 kid1| Swap maxSize 0 + 4194304 KB, estimated 322638 objects 2014/06/30 16:09:07 kid1| Target number of buckets: 16131 2014/06/30 16:09:07 kid1| Using 16384 Store buckets 2014/06/30 16:09:07 kid1| Max Mem size: 4194304 KB 2014/06/30 16:09:07 kid1| Max Swap size: 0 KB 2014/06/30 16:09:07 kid1| Using Least Load store dir selection 2014/06/30 16:09:07 kid1| Set Current Directory to /var/cache/squid 2014/06/30 16:09:07 kid1| Finished loading MIME types and icons. 2014/06/30 16:09:07 kid1| HTCP Disabled. 2014/06/30 16:09:07 kid1| Pinger socket opened on FD 34 2014/06/30 16:09:07 kid1| Configuring Parent 194.99.121.200/3128/0 2014/06/30 16:09:07 kid1| Squid plugin modules loaded: 0 2014/06/30 16:09:07 kid1| Adaptation support is on 2014/06/30 16:09:07 kid1| Accepting HTTP Socket connections at local=0.0.0.0:3128 remote=[::] FD 30 flags=9 2014/06/30 16:09:07 kid1| Accepting SNMP messages on 10.143.153.27:3401 2014/06/30 16:09:07 kid1| Sending SNMP messages from 10.143.153.27:3401 2014/06/30 16:09:07| pinger: Initialising ICMP pinger ... 2014/06/30 16:09:07| icmp_sock: (1) Operation not permitted 2014/06/30 16:09:07| pinger: Unable to start ICMP pinger. 2014/06/30 16:09:07| icmp_sock: (97) Address family not supported by protocol 2014/06/30 16:09:07| pinger: Unable to start ICMPv6 pinger. 2014/06/30 16:09:07| FATAL: pinger: Unable to open any ICMP sockets. 2014/06/30 16:09:07 kid1| Starting new redirector helpers... 2014/06/30 16:09:07 kid1| helperOpenServers: Starting 1/200 'squidGuard' processes 2014/06/30 16:09:07 kid1| Starting new redirector helpers... 2014/06/30 16:09:07 kid1| helperOpenServers: Starting 1/200 'squidGuard' processes 2014/06/30 16:09:07 kid1| recv: (111) Connection refused 2014/06/30 16:09:07 kid1| Closing Pinger socket on FD 34 2014/06/30 16:09:07 kid1| temporary disabling (Forbidden) digest from 194.99.121.200 2014/06/30 16:09:08 kid1| storeLateRelease: released 0 objects (squid-1)(_Z5deathi+0x49)[0x7f8804808229] /lib64/libpthread.so.0(+0xf810)[0x7f880410e810] (squid-1)(_Z19cbdataInternalAlloci+0x27)[0x7f88046aac67] (squid-1)(_ZN15ServerStateData15startAdaptationERK8RefCountIN10Adaptation12ServiceGroupEEP11HttpRequest+0x250)[0x7f88047ffbf0] (squid-1)(_ZN15ServerStateData26noteAdaptationAclCheckDoneE8RefCountIN10Adaptation12ServiceGroupEE+0x62)[0x7f88048000a2] (squid-1)(_ZN12UnaryMemFunTIN10Adaptation9InitiatorE8RefCountINS0_12ServiceGroupEES4_E6doDialEv+0x6a)[0x7f88049240aa] (squid-1)(_ZN9JobDialerIN10Adaptation9InitiatorEE4dialER9AsyncCall+0x35)[0x7f88049237d5] (squid-1)(_ZN9AsyncCall4makeEv+0x313)[0x7f8804888e73] (squid-1)(_ZN14AsyncCallQueue8fireNextEv+0x200)[0x7f880488c4c0] (squid-1)(_ZN14AsyncCallQueue4fireEv+0x28)[0x7f880488c848] (squid-1)(_ZN9EventLoop7runOnceEv+0xe4)[0x7f8804716824] (squid-1)(_ZN9EventLoop3runEv+0x28)[0x7f8804716988] (squid-1)(_Z9SquidMainiPPc+0x464)[0x7f8804797544] (squid-1)(+0x25afa9)[0x7f8804797fa9] /lib64/libc.so.6(__libc_start_main+0xe6)[0x7f8800efcc16] (squid-1)(+0x13fb09)[0x7f880467cb09] FATAL: Received Segment Violation...dying. 2014/06/30 16:09:27 kid1| Closing HTTP port 0.0.0.0:3128 2014/06/30 16:09:27 kid1| storeDirWriteCleanLogs: Starting... 2014/06/30 16:09:27 kid1| Finished. Wrote 0 entries. 2014/06/30 16:09:27 kid1| Took 0.00 seconds ( 0.00 entries/sec). CPU Usage: 0.312 seconds = 0.212 user + 0.100 sys Maximum Resident Size: 82800 KB Page faults with physical i/o: 0 Memory usage for squid via mallinfo(): total space in arena:7244 KB Ordinary blo
[squid-users] Fwd: Squidblacklist.org - A better blacklist for Squid-ACL. Blacklisting Evolved.
Good morning List Troll! Please don't peddle your (subscription fee based no less...yugh) garbage off listor heck ON list for that matter. Squid-users admin, kindly nuke/destroy/delete/erase the below...thank you. James Original Message Subject: Squidblacklist.org - A better blacklist for Squid-ACL. Blacklisting Evolved. Date: 2014-06-30 07:35 From: "Benjamin E. Nichols" To: j...@slave-tothe-box.net Reply-To: webmas...@squidblacklist.org Do you leverage a web filter on your networks? If so, then you should know that there is room for a better blacklist, and we intend to fill that gap. It would be a pleasure to serve you. If you would like samples of our works, we will gladly email you some upon request. Signed, Benjamin E. Nichols http://www.squidblacklist.org
Re: [squid-users] SSL bump working on most site...cert pinning issue?
No worries. Sounds like this is the feature you should be waiting with baited breath for: http://wiki.squid-cache.org/Features/SslPeekAndSplice I’m not a developer so I have no idea how far along that is right now. On 30 Jun 2014, at 11:05 pm, James Lay wrote: > On Mon, 2014-06-30 at 22:56 +1000, Dan Charlesworth wrote: >> Yeah, pinned SSL ‘aint gonna be bumped. The Twitter apps are another popular >> one that use pinning. >> >> As far as your broken_sites ACL goes, you can’t use `dstdomain` because the >> only thing Squid can see of the destination before bumping an intercepted >> connection is the IP address. So for `ssl_bump none` you’ll need to be use >> `dst` ACLs instead. >> >> ProTip: Here are the Apple and Akamai public IP blocks (to use in a dst >> equivalent of your broken_sites), respectively: 17.0.0.0/8, 23.0.0.0/12. >> >> Good luck >> >> On 30 Jun 2014, at 10:38 pm, James Lay wrote: >> >>> Topic pretty much says it...most sites work fine using my below set up, >>> but some (Apple's app store) do not. I'm wondering if cert pinning is >>> the issue? Since this set up is basically two separate sessions, I >>> packet captured both. The side the I have control over gives me a TLS >>> Record Layer Alert Close Notify. I am unable to decrypt the other side >>> as the device in question is an iDevice and I can't capture the master >>> secret. >>> >>> I've even tried to ACL certain sites to not bump, but they don't go >>> through. Below is my complete setup. This is running the below: >>> >>> Squid Cache: Version 3.4.6 >>> configure options: '--prefix=/opt' '--enable-icap-client' >>> '--enable-ssl' '--enable-linux-netfilter' >>> '--enable-follow-x-forwarded-for' '--with-large-files' >>> '--sysconfdir=/opt/etc/squid' >>> >>> >>> Any assistance with troubleshooting would be wonderful...thank you. >>> >>> James >>> >>> >>> >>> $IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport >>> 80 -j REDIRECT --to-port 3128 >>> $IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport >>> 443 -j REDIRECT --to-port 3129 >>> >>> >>> acl localnet src 192.168.1.0/24 >>> >>> acl SSL_ports port 443 >>> acl Safe_ports port 80 # http >>> acl Safe_ports port 21 # ftp >>> acl Safe_ports port 443 # https >>> acl Safe_ports port 70 # gopher >>> acl Safe_ports port 210 # wais >>> acl Safe_ports port 1025-65535 # unregistered ports >>> acl Safe_ports port 280 # http-mgmt >>> acl Safe_ports port 488 # gss-http >>> acl Safe_ports port 591 # filemaker >>> acl Safe_ports port 777 # multiling http >>> >>> acl CONNECT method CONNECT >>> acl broken_sites dstdomain textnow.me >>> acl broken_sites dstdomain akamaiedge.net >>> acl broken_sites dstdomain akamaihd.net >>> acl broken_sites dstdomain apple.com >>> acl allowed_sites url_regex "/opt/etc/squid/url.txt" >>> acl all_others dst all >>> acl SSL method CONNECT >>> >>> >>> http_access deny !Safe_ports >>> http_access deny CONNECT !SSL_ports >>> >>> http_access allow manager localhost >>> http_access deny manager >>> >>> http_access allow allowed_sites >>> http_access deny all_others >>> http_access allow localnet >>> http_access allow localhost >>> >>> http_access deny all >>> icp_access deny all >>> >>> sslproxy_cert_error allow broken_sites >>> sslproxy_cert_error deny all >>> >>> sslproxy_options ALL >>> ssl_bump none broken_sites >>> ssl_bump server-first all >>> >>> http_port 192.168.1.253:3128 intercept >>> https_port 192.168.1.253:3129 intercept ssl-bump >>> generate-host-certificates=on cert=/opt/sslsplit/sslsplit.crt >>> key=/opt/sslsplit/sslsplitca.key options=ALL sslflags=NO_SESSION_REUSE >>> >>> always_direct allow all >>> >>> >>> hierarchy_stoplist cgi-bin ? >>> >>> access_log syslog:daemon.info common >>> >>> refresh_pattern ^ftp: 144020% 10080 >>> refresh_pattern ^gopher:14400% 1440 >>> refresh_pattern -i (cgi-bin|\?) 0 0% 0 >>> refresh_pattern . 0 20% 4320 >>> >>> icp_port 3130 >>> >>> coredump_dir /opt/var >>> >>> > > Ah good catch thank you. I've seen expensive proxy appliances just > tunnel the traffic through, but they get the host and domain name to all > control...which is really all I'm wanting to do is control what sites > are allowed. I'll give your suggestions a go...thank you. > > James >
Re: [squid-users] SSL bump working on most site...cert pinning issue?
On Mon, 2014-06-30 at 22:56 +1000, Dan Charlesworth wrote: > Yeah, pinned SSL ‘aint gonna be bumped. The Twitter apps are another popular > one that use pinning. > > As far as your broken_sites ACL goes, you can’t use `dstdomain` because the > only thing Squid can see of the destination before bumping an intercepted > connection is the IP address. So for `ssl_bump none` you’ll need to be use > `dst` ACLs instead. > > ProTip: Here are the Apple and Akamai public IP blocks (to use in a dst > equivalent of your broken_sites), respectively: 17.0.0.0/8, 23.0.0.0/12. > > Good luck > > On 30 Jun 2014, at 10:38 pm, James Lay wrote: > > > Topic pretty much says it...most sites work fine using my below set up, > > but some (Apple's app store) do not. I'm wondering if cert pinning is > > the issue? Since this set up is basically two separate sessions, I > > packet captured both. The side the I have control over gives me a TLS > > Record Layer Alert Close Notify. I am unable to decrypt the other side > > as the device in question is an iDevice and I can't capture the master > > secret. > > > > I've even tried to ACL certain sites to not bump, but they don't go > > through. Below is my complete setup. This is running the below: > > > > Squid Cache: Version 3.4.6 > > configure options: '--prefix=/opt' '--enable-icap-client' > > '--enable-ssl' '--enable-linux-netfilter' > > '--enable-follow-x-forwarded-for' '--with-large-files' > > '--sysconfdir=/opt/etc/squid' > > > > > > Any assistance with troubleshooting would be wonderful...thank you. > > > > James > > > > > > > > $IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport > > 80 -j REDIRECT --to-port 3128 > > $IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport > > 443 -j REDIRECT --to-port 3129 > > > > > > acl localnet src 192.168.1.0/24 > > > > acl SSL_ports port 443 > > acl Safe_ports port 80 # http > > acl Safe_ports port 21 # ftp > > acl Safe_ports port 443 # https > > acl Safe_ports port 70 # gopher > > acl Safe_ports port 210 # wais > > acl Safe_ports port 1025-65535 # unregistered ports > > acl Safe_ports port 280 # http-mgmt > > acl Safe_ports port 488 # gss-http > > acl Safe_ports port 591 # filemaker > > acl Safe_ports port 777 # multiling http > > > > acl CONNECT method CONNECT > > acl broken_sites dstdomain textnow.me > > acl broken_sites dstdomain akamaiedge.net > > acl broken_sites dstdomain akamaihd.net > > acl broken_sites dstdomain apple.com > > acl allowed_sites url_regex "/opt/etc/squid/url.txt" > > acl all_others dst all > > acl SSL method CONNECT > > > > > > http_access deny !Safe_ports > > http_access deny CONNECT !SSL_ports > > > > http_access allow manager localhost > > http_access deny manager > > > > http_access allow allowed_sites > > http_access deny all_others > > http_access allow localnet > > http_access allow localhost > > > > http_access deny all > > icp_access deny all > > > > sslproxy_cert_error allow broken_sites > > sslproxy_cert_error deny all > > > > sslproxy_options ALL > > ssl_bump none broken_sites > > ssl_bump server-first all > > > > http_port 192.168.1.253:3128 intercept > > https_port 192.168.1.253:3129 intercept ssl-bump > > generate-host-certificates=on cert=/opt/sslsplit/sslsplit.crt > > key=/opt/sslsplit/sslsplitca.key options=ALL sslflags=NO_SESSION_REUSE > > > > always_direct allow all > > > > > > hierarchy_stoplist cgi-bin ? > > > > access_log syslog:daemon.info common > > > > refresh_pattern ^ftp: 144020% 10080 > > refresh_pattern ^gopher:14400% 1440 > > refresh_pattern -i (cgi-bin|\?) 0 0% 0 > > refresh_pattern . 0 20% 4320 > > > > icp_port 3130 > > > > coredump_dir /opt/var > > > > Ah good catch thank you. I've seen expensive proxy appliances just tunnel the traffic through, but they get the host and domain name to all control...which is really all I'm wanting to do is control what sites are allowed. I'll give your suggestions a go...thank you. James
Re: [squid-users] SSL bump working on most site...cert pinning issue?
Yeah, pinned SSL ‘aint gonna be bumped. The Twitter apps are another popular one that use pinning. As far as your broken_sites ACL goes, you can’t use `dstdomain` because the only thing Squid can see of the destination before bumping an intercepted connection is the IP address. So for `ssl_bump none` you’ll need to be use `dst` ACLs instead. ProTip: Here are the Apple and Akamai public IP blocks (to use in a dst equivalent of your broken_sites), respectively: 17.0.0.0/8, 23.0.0.0/12. Good luck On 30 Jun 2014, at 10:38 pm, James Lay wrote: > Topic pretty much says it...most sites work fine using my below set up, > but some (Apple's app store) do not. I'm wondering if cert pinning is > the issue? Since this set up is basically two separate sessions, I > packet captured both. The side the I have control over gives me a TLS > Record Layer Alert Close Notify. I am unable to decrypt the other side > as the device in question is an iDevice and I can't capture the master > secret. > > I've even tried to ACL certain sites to not bump, but they don't go > through. Below is my complete setup. This is running the below: > > Squid Cache: Version 3.4.6 > configure options: '--prefix=/opt' '--enable-icap-client' > '--enable-ssl' '--enable-linux-netfilter' > '--enable-follow-x-forwarded-for' '--with-large-files' > '--sysconfdir=/opt/etc/squid' > > > Any assistance with troubleshooting would be wonderful...thank you. > > James > > > > $IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport > 80 -j REDIRECT --to-port 3128 > $IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport > 443 -j REDIRECT --to-port 3129 > > > acl localnet src 192.168.1.0/24 > > acl SSL_ports port 443 > acl Safe_ports port 80# http > acl Safe_ports port 21# ftp > acl Safe_ports port 443 # https > acl Safe_ports port 70# gopher > acl Safe_ports port 210 # wais > acl Safe_ports port 1025-65535# unregistered ports > acl Safe_ports port 280 # http-mgmt > acl Safe_ports port 488 # gss-http > acl Safe_ports port 591 # filemaker > acl Safe_ports port 777 # multiling http > > acl CONNECT method CONNECT > acl broken_sites dstdomain textnow.me > acl broken_sites dstdomain akamaiedge.net > acl broken_sites dstdomain akamaihd.net > acl broken_sites dstdomain apple.com > acl allowed_sites url_regex "/opt/etc/squid/url.txt" > acl all_others dst all > acl SSL method CONNECT > > > http_access deny !Safe_ports > http_access deny CONNECT !SSL_ports > > http_access allow manager localhost > http_access deny manager > > http_access allow allowed_sites > http_access deny all_others > http_access allow localnet > http_access allow localhost > > http_access deny all > icp_access deny all > > sslproxy_cert_error allow broken_sites > sslproxy_cert_error deny all > > sslproxy_options ALL > ssl_bump none broken_sites > ssl_bump server-first all > > http_port 192.168.1.253:3128 intercept > https_port 192.168.1.253:3129 intercept ssl-bump > generate-host-certificates=on cert=/opt/sslsplit/sslsplit.crt > key=/opt/sslsplit/sslsplitca.key options=ALL sslflags=NO_SESSION_REUSE > > always_direct allow all > > > hierarchy_stoplist cgi-bin ? > > access_log syslog:daemon.info common > > refresh_pattern ^ftp: 144020% 10080 > refresh_pattern ^gopher: 14400% 1440 > refresh_pattern -i (cgi-bin|\?) 0 0% 0 > refresh_pattern . 0 20% 4320 > > icp_port 3130 > > coredump_dir /opt/var > >
[squid-users] SSL bump working on most site...cert pinning issue?
Topic pretty much says it...most sites work fine using my below set up, but some (Apple's app store) do not. I'm wondering if cert pinning is the issue? Since this set up is basically two separate sessions, I packet captured both. The side the I have control over gives me a TLS Record Layer Alert Close Notify. I am unable to decrypt the other side as the device in question is an iDevice and I can't capture the master secret. I've even tried to ACL certain sites to not bump, but they don't go through. Below is my complete setup. This is running the below: Squid Cache: Version 3.4.6 configure options: '--prefix=/opt' '--enable-icap-client' '--enable-ssl' '--enable-linux-netfilter' '--enable-follow-x-forwarded-for' '--with-large-files' '--sysconfdir=/opt/etc/squid' Any assistance with troubleshooting would be wonderful...thank you. James $IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport 80 -j REDIRECT --to-port 3128 $IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport 443 -j REDIRECT --to-port 3129 acl localnet src 192.168.1.0/24 acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT acl broken_sites dstdomain textnow.me acl broken_sites dstdomain akamaiedge.net acl broken_sites dstdomain akamaihd.net acl broken_sites dstdomain apple.com acl allowed_sites url_regex "/opt/etc/squid/url.txt" acl all_others dst all acl SSL method CONNECT http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow manager localhost http_access deny manager http_access allow allowed_sites http_access deny all_others http_access allow localnet http_access allow localhost http_access deny all icp_access deny all sslproxy_cert_error allow broken_sites sslproxy_cert_error deny all sslproxy_options ALL ssl_bump none broken_sites ssl_bump server-first all http_port 192.168.1.253:3128 intercept https_port 192.168.1.253:3129 intercept ssl-bump generate-host-certificates=on cert=/opt/sslsplit/sslsplit.crt key=/opt/sslsplit/sslsplitca.key options=ALL sslflags=NO_SESSION_REUSE always_direct allow all hierarchy_stoplist cgi-bin ? access_log syslog:daemon.info common refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern -i (cgi-bin|\?) 0 0% 0 refresh_pattern . 0 20% 4320 icp_port 3130 coredump_dir /opt/var
[squid-users] Two way SSL
Hello, we need to configure two way ssl for reverse http proxy (squid). client -> (https two-way ssl) -> squid -> (https one-way ssl) -> server Is there any examples of configuration file? Regards, Vlado -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Two-way-SSL-tp4666548.html Sent from the Squid - Users mailing list archive at Nabble.com.
RE: [squid-users] ssl-bump not working in non transparent mode
Thanks for your reply. I used following line & its working fine: http_port 10.10.16.56:3128 ssl-bump intercept generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/mycert.pem But now its showing certificate error for every https website. How we can resolve this error? > Date: Sat, 28 Jun 2014 21:47:48 +0300 > From: elie...@ngtech.co.il > To: squid-users@squid-cache.org > Subject: Re: [squid-users] ssl-bump not working in non transparent mode > > Hey Nil, > > Are you aware that you need to use the "ssl-bump" flags and > dynamic_cert_mem etc on the forward regular proxy mode? > such as: > http_port 10.10.16.56:3128 ssl-bump ...(all other settings) > > For it to work? > > Eliezer > > On 06/27/2014 03:45 PM, Nil Nik wrote: >> http_port 10.10.16.56:3127 intercept >> http_port 10.10.16.56:3128 >> https_port 10.10.16.56:3129 generate-host-certificates=on >> dynamic_cert_mem_cache_size=4MB cert=/etc/squid/mycert.pem intercept ssl-bump >
Re: [squid-users] FATAL: No valid signing SSL certificate configured for https_port
I would say +1 for binary search.. Remove all specials and make it: https_port 10.x.x.95:443 accel cert=/usr/newrprgate/CertAuth/cert/cert.crt key=/usr/newrprgate/CertAuth/cert/key.pem defaultsite=server_1.uk Which will minimize it to a working settings which works on every linux version with any openssl library I know of. If it won't work I will verify that the certificates are in the right format and if not convert them to the right format.. Else then that is to compile it from src on this or similar machine and find out if you have the same issue with a self signed certificate. I have not tested it yet on my build node but unless something is really odd it should work with no issues. Eliezer On 06/30/2014 02:07 PM, John Gardner wrote: Eliezer The line that was working but is now causing problems is; https_port 10.x.x.95:443 accel cert=/usr/newrprgate/CertAuth/cert/cert.crt key=/usr/newrprgate/CertAuth/cert/key.pem cipher=ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM options=NO_SSLv2 defaultsite=server_1.uk John
Re: [squid-users] FATAL: No valid signing SSL certificate configured for https_port
Eliezer The line that was working but is now causing problems is; https_port 10.x.x.95:443 accel cert=/usr/newrprgate/CertAuth/cert/cert.crt key=/usr/newrprgate/CertAuth/cert/key.pem cipher=ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM options=NO_SSLv2 defaultsite=server_1.uk John On 30 June 2014 12:06, John Gardner wrote: > Eliezer > > The line that was working but is now causing problems is; > > https_port 10.x.x.95:443 accel > cert=/usr/newrprgate/CertAuth/cert/cert.crt > key=/usr/newrprgate/CertAuth/cert/key.pem > cipher=ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM > options=NO_SSLv2 defaultsite=server_1.uk > > On 30 June 2014 01:49, Eliezer Croitoru wrote: >> On 06/29/2014 09:30 PM, John Gardner wrote: >>> >>> FATAL: No valid signing SSL certificate configured for https_port >>> 10.x.x.95:443 and Squid terminates. >> >> >> Can you share the relevant line from squid.conf?(replacing confidential >> data) >> >> (I am planning for the next release 3.4.6 to release a Oracle version of the >> RPM but it will be only 6.5 compatible) >> >> Eliezer
Re: [squid-users] ACL Problem
On 06/30/2014 12:25 PM, Der Dutz wrote: Hi Eliezer, Thanks for your kind respond. actually im reposting because i see onhttp://marc.info/ that my email is unreadable because the format from the email client i used (yahoo internal send mail editor), because its unreadable then im afraid no one will reply to it. Ok for the squid problem, i think it is cause by the squid server, because when im skipping squid server, the web access for this url not having these problem. In the access log i only see the user can access the main web This is not 100% true since it can be the combination of the two in some cases. From what I see at the logs the error is not from your squid server. You can try to remove the forward_for headers if they are being present which can cause similar issues. Please try again in private mode of firefox or something similar in other browsers to ensure local cache will not be used for the requests. Make sure what access.log you are getting and what you do have in it to verify that the denial is not comming from your server. Eliezer [root@localhost html]# tail -f /var/log/squid/access.log | grep 192.25.80.58 2014-06-30 16:26:42 64 192.25.80.58 TCP_MISS/200 30289 GEThttp://989321dut38h.sbobet.com/euro/ - DIRECT/103.11.41.9 text/html 2014-06-30 16:26:42 -131 192.25.80.58 TCP_MISS/200 48308 GEThttp://989321dut38h.sbobet.com/en/resource/e/euro-static.js? - DIRECT/103.11.41.9 application/x-javascript 2014-06-30 16:26:42 -137 192.25.80.58 TCP_MISS/200 15143 GEThttp://989321dut38h.sbobet.com/en/resource/e/euro-dynamic.js? - DIRECT/103.11.41.9 application/x-javascript but for the other css / js file needed for these main web is not found in access.log.
Re: [squid-users] ACL Problem
Hi Eliezer, Thanks for your kind respond. actually im reposting because i see on http://marc.info/ that my email is unreadable because the format from the email client i used (yahoo internal send mail editor), because its unreadable then im afraid no one will reply to it. Ok for the squid problem, i think it is cause by the squid server, because when im skipping squid server, the web access for this url not having these problem. In the access log i only see the user can access the main web [root@localhost html]# tail -f /var/log/squid/access.log | grep 192.25.80.58 2014-06-30 16:26:42 64 192.25.80.58 TCP_MISS/200 30289 GET http://989321dut38h.sbobet.com/euro/ - DIRECT/103.11.41.9 text/html 2014-06-30 16:26:42 -131 192.25.80.58 TCP_MISS/200 48308 GET http://989321dut38h.sbobet.com/en/resource/e/euro-static.js? - DIRECT/103.11.41.9 application/x-javascript 2014-06-30 16:26:42 -137 192.25.80.58 TCP_MISS/200 15143 GET http://989321dut38h.sbobet.com/en/resource/e/euro-dynamic.js? - DIRECT/103.11.41.9 application/x-javascript but for the other css / js file needed for these main web is not found in access.log. Here is my squid.conf : http_port 888 transparent cache_mem 128 MB cache_mgr x cachemgr_passwd x all cache_dir aufs /var/spool/squid 8000 256 256 cache_dir aufs /var/spool/squid1 8000 256 256 cache_dir aufs /var/spool/squid2 8000 256 256 cache_dir aufs /var/spool/squid3 8000 256 256 cache_dir aufs /var/spool/squid4 8000 256 256 cache_dir aufs /var/spool/squid5 8000 256 256 cache_dir aufs /var/spool/squid6 8000 256 256 cache_dir aufs /var/spool/squid7 8000 256 256 cache_dir aufs /var/spool/squid8 8000 256 256 logformat squid %{%Y-%m-%d %H:%M:%S}tl %6tr %>a %Ss/%03Hs %http://*.googlesyndication.*/.* 720 90% 4320 # various windows versions refresh_pattern http://.*\.windowsupdate\.microsoft\.com/ 0 80% 20160 reload-into-ims refresh_pattern http://.*\.update\.microsoft\.com/ 0 80% 20160 reload-into-ims refresh_pattern http://download\.microsoft\.com/ 0 80% 20160 reload-into-ims refresh_pattern http://windowsupdate\.microsoft\.com/ 0 80% 20160 reload-into-ims refresh_pattern http://office\.microsoft\.com/ 0 80% 20160 reload-into-ims refresh_pattern http://w?xpsp[0-9]\.microsoft\.com/ 0 80% 20160 reload-into-ims refresh_pattern http://w2ksp[0-9]\.microsoft\.com/ 0 80% 20160 reload-into-ims refresh_pattern windowsupdate.com/.*\.(cab|exe) 4320 100% 43200 reload-into-ims refresh_pattern download.microsoft.com/.*\.(cab|exe) 4320 100% 43200 reload-into-ims refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe) 4320 100% 43200 reload-into-ims # and some other windows updaters refresh_pattern http://download\.macromedia\.com/ 0 80% 20160 reload-into-ims refresh_pattern ftp://ftp\.nai\.com/ 0 80% 20160 reload-into-ims refresh_pattern http://ftp\.software\.ibm\.com/ 0 80% 20160 reload-into-ims refresh_pattern http://.*\.grisoft\.com/ 0 80% 20160 reload-into-ims refresh_pattern http://download\.lavasoft\.de*/ 0 80% 20160 reload-into-ims refresh_pattern ftp://ftp\.nai\.com/ 0 80% 20160 reload-into-ims # repositories refresh_pattern http://.*\.archive\.ubuntu\.com/ 0 80% 20160 reload-into-ims refresh_pattern http://www\.getautomatix\.com/ 0 80% 20160 reload-into-ims refresh_pattern http://wine\.budgetdedicated\.com/ 0 80% 20160 reload-into-ims refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 10800 20% 10800 ignore-no-cache ignore-private override-expire ignore-reload ignore-auth negative-ttl=40320 max-stale=10 #acl googlesyn dstdomain *.googlesyndication.com #http_access deny googlesyn #acl blockeddomain dstdomain "/etc/blocked.domains.acl" #acl adsites dstdomain url_regex "/etc/adlist.acl" #acl adsip dst "/etc/adsip.acl" #acl adsites1 url_regex "/etc/adlist.txt" acl sbobet dstdomain *.sbobet.com/* acl sbobet dstdomain *.sbostatic.com/* always_direct allow sbobet #cache deny sbobet acl all src 0.0.0.0/0.0.0.0 acl client1 src 10.16.8.0/24 acl ippublic src x.x.x.x/29 acl client2 src 192.168.88.0/24 acl client3 src x.x.x.0/24 acl client4 src x.x.x.0/24 acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 563 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 563 # https, snews acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Saf