Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
Hi Jason, If you think the external acl method is too expensive to run, how do you expect to feed this NIDS data back into squid? I think you'd find you'd need an external acl check to do that bit anyway :-) I should have been clearer - my use of the term feedback loop was meant to imply that this was an out-of-band method of populating a cache of valid HTTPS IP addresses (as well as the any SNI and even 'Application name') for HTTPS, and observed non-HTTPS traffic over 443 and any other ports. This may compliment any active external acl style checking. Was just thinking out loud - probably a crazy idea if every seriously considered :) Regards, Chris ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
On 08/01/15 18:41, Chris Bennett wrote: Interesting thread so far. Has anyone thought of using Bro-IDS as a feedback loop for some of this advanced logic for bypassing bumping? The external acl method mentioned earlier probably out-does using some NIDS feedback loop. In my testing it causes squid to block that new connection until it returns, and that means your external acl script can simply attempt a SSL transaction against the end-server and in realtime figure out that it's SSL or not. And then cache it, blah, blah blah. The advantage is that it will do a lookup on new HTTPS sessions and potentially have the answer immediately (ie it can bump on first attempt), whereas a NIDS would only find out the answer after squid has defaulted to passthrough/splice mode, so it would only work properly on future connections to that site. I like the active external acl solution since it meets a need, but there is overhead. I'm not quite sure what Bro logs for non-HTTPS 443 traffic, but I thought I'd chime in with the above idea if anyone wants to expand on it further :) If you think the external acl method is too expensive to run, how do you expect to feed this NIDS data back into squid? I think you'd find you'd need an external acl check to do that bit anyway :-) -- Cheers Jason Haar Corporate Information Security Manager, Trimble Navigation Ltd. Phone: +1 408 481 8171 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
On 06/01/15 05:28, Eliezer Croitoru wrote: In 3.5 there will be present a new feature which called peek and splice that can give an interface to squid and the admin which will allow the admin to know couple things about the connection from squid and specifically first the client TLS request. Is there an example document showing just how to do this? Looking at the current docs, I can't quite figure out how to layer them all together to achieve what I'd imagine 99% of sysadmins wanting to do ssl-bump need to do. Even squid-3.4 works very well without peek/splice - if you are using it as a formal proxy. But it all falls apart with transparent tcp 443 - as squid only has the dst IP... What I'd like to do is to use peek to grab the SSL server name the client sends so that it is available to acls (and external acl calls - and logging?) as if the client had gone CONNECT server.name:443? A quick sniff with wireshark shows Firefox (as an example) sends the server name as a client SNI request in the first real packet (ie after the 3-way), so that smells to my naive understanding as good for a peek - and should allow squid to do an initial chat with the client, get the SNI, then dupe with the real server, then decide if to splice or bump the rest? Clients that don't support SNI will probably have to be spliced - I don't care - I'm only interested in running AV scanners and porn filters over HTTPS requests from web browsers - the 0.1% remaining SSL traffic can slip through the cracks for all I care ;-) -- Cheers Jason Haar Corporate Information Security Manager, Trimble Navigation Ltd. Phone: +1 408 481 8171 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sounds good, but server world is not end on Linux. ;) Now exists another *NIX systems. And will exists further. Also. I have an idea, gents. Do we can easy and quickly detect SSL Pinned destinations? And remember it, for example, in database? In another words - both problems is similar. Either non-HTTPS traffic over 443 port, or pinned certs. Can we detect both of them automatically and add to exclude list? WBR, Yuri 05.01.2015 8:44, Eliezer Croitoru пишет: Hey Thread(Jason,Yuri,Douglas...), There are couple aspects about the ssl and connections in general and as we talk about ssl port I first would like to put couple things on the table. * Squid is a http caching proxy and there for every feature which is out of the http related scope should not be handled by squid at all. * Any squid operation is an application level and there for is limited by the software(kernel + squid). * There is a difference between servers taking a load of 1k requests per second to a SMB which handles about 50 requests per second. In general it's better to not intercept a connection which is not bump-able. The decision about if to DROP\REJECT or ACCEPT the connection should better not involve squid in general if possible. Squid offers a very nice interface but if you compare the Linux kernel forwarding capabilities compared to squid you would see that squid is very limited in the userspace. So in a case the helper only needs to know if the connection is bump-able there are other alternatives in the Linux kernel!! And if you need logs.. you can use the *helper*(which one you ever choose to work with) to log... So now for the real thing: My opinion about external_acl vs other solutions is that if squid with an external_acl works for you and you understand what it means from performance and security aspects try it, test it and then use it. But if your squid load is high and in the case squid slows down the bumped connections too much(compared to linux forwarding) I would try to use something like NFQUEUE to just test if the connection is bump-able or not by IP and DST PORT. * some information about NFQUEUE https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/ http://suricata-ids.org/ * Some examples: https://www.wzdftpd.net/redmine/projects/nfqueue-bindings/repository/entry/examples/rewrite.py http://danmcinerney.org/reliable-dns-spoofing-with-python-scapy-nfqueue/ http://5d4a.wordpress.com/2011/08/25/having-fun-with-nfqueue-and-scapy/ A squid helper is nice but... a NFQUEUE helper that can verify if to FORWARD or BUMP the connection would be a better suited solution to my opinion. All The Bests, Eliezer Croitoru On 01/05/2015 03:07 AM, Douglas Davenport wrote: Seems to me it would be more useful as an external ACL so that a decision could be made based on other factors eg src or dstdomain whether to deny or allow the un-bumpable connection. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUqk4WAAoJENNXIZxhPexGWNoH/Ak2w0TZ+fU2Jy1bIOgWP82V P5UJmB3DDSRwrqi4Y/bfUDGT1V3Cbjn8/RRTqTl7lSbBwGpSd8wSLGSTua6mqMY6 OIedOB7oBrJ9p8d1F7//ZsBrvGHequnD7Zp1DvBXVcptcVvFi56oAeFNjhRk1tcN 8EX2mkvgrDye7o7RRXPw1ukxbAJ0883A+gO3XpSARMUQEhsXJFFoygHUo7OIjdr+ oBrv/aypN8VOFvkHj50vDwtt4Rq7PPDYJRtHms2cIGsjK4+P1Rt1lxhr0GC/qbtZ rfqIvP5LRmkID/lvHFhWC38APdjsgCcTICuJoPgKGDPX9YMAWtKdznu2XYpHNfw= =LZkA -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
On 01/01/15 00:11, James Harper wrote: The helper connects to the IP:port and tries to obtain the certificate, and then caches the result (in an sqlite database). If it can't do so within a fairly short time it returns failure (but keeps trying a bit longer and caches it for next time). Alternatively if the IP used to be SSL but is now timing out it returns the previously cached value. Negative results are cached for an increasing amount of time each time it fails, on the basis that it probably isn't SSL. That sounds great James! I'd certainly like to take a look at it too However, you say SSL - did you mean HTTPS? ie discovering a ip:port is a IMAPS server doesn't really help squid talk to it - surely you want to discover HTTPS servers - and everything else should be pass-through/splice? Yes. That would be required for completeness, although maybe the certificate info would still be useful even if the underlying protocol wasn't http... James ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
Much of the discussion so far has been about bumping traffic on port 443, bumping SSL-encapsulated HTTP traffic and not bumping (allowing) other traffic. Since port 443 is used for many protocols, it is in many cases dangerous to allow non-bumpable traffic: SSH tunnels using port 443 are common, so are VPNs. Do you know a security officer who does not want to block an SSH tunnel, or an app that can share corporate documents on public websites? If there is not more attention to these kinds of applications that use port 443 to circumvent corporate firewalls, Squid will be doomed to be used only in environments where the priority for security is low to non-existent. Just type punching holes in corporate firewalls or ssh tunnel proxy in Google to see how easy it is to use an SSH tunnel. I am the author of ufdbGuard, a filter for Squid and besides filtering based on URLs, ufdbGuard also probes port 443 to see what kind of traffic the server is expecting. By using probes, ufdbGuard can detect SSH tunnels, popular chat protocols, etc. but it is not a 100% guaranteed solution because ufdbGuard cannot not see the traffic that flows through the proxy, i.e. there is not yet an interface for this type of traffic inspection. Marcus On 01/05/2015 07:59 AM, Eliezer Croitoru wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hey Yuri, Indeed there are other *NIX systems and for each and every one of them there is a solution in need. SSL Pinned destinations cannot be identified automatically since the are pinned inside a software and the certificate will not show anything about that. The basic tests we can do are: - - The host is using ssl or tls or not at all(based on the selective answer of the service) - - - - If the connection is using tls\ssl then inspect the components of the certificate(such as rootCA validation against the local machine certificates DB) Depend on the goal of the certificate validation the decision will be made to either allow the connection uninspected or to bump it as is without any smart identification. If indeed there is a database sqlite3\mysql\postgres\redis\memcached\others it can be used in the iptables level. Also a point in this DB and this cache is that it will be persistent so what so ever the *NIX system is there is an option once the IP + port was tagged as non-bump-able it is better be in the FIREWALL level override better then squid external_acl. Reason: If the kernel does what it needs to do then squid should not touch the packets. It's not always right but it's a point in the issue. I still do not know how to work with NFQUEUE and I am sure that there is an option to make a fast decision and if not then let the connection be BUMPED. I have written a small golang script that can check couple things about the ssl session at: http://www1.ngtech.co.il/squid/ssl_helpers/ssl_validator.go Besides this helper there is another script which do couple things in another level. ## If any thing will be decided for squid internals it will be after a proof of concept that we can implement together. Can we take this thread to storm and put on the table a proof of concept logic for ssl inspection\bumping and bypassing? Eliezer On 01/05/2015 10:40 AM, Yuri Voinov wrote: Sounds good, but server world is not end on Linux. ;) Now exists another *NIX systems. And will exists further. Also. I have an idea, gents. Do we can easy and quickly detect SSL Pinned destinations? And remember it, for example, in database? In another words - both problems is similar. Either non-HTTPS traffic over 443 port, or pinned certs. Can we detect both of them automatically and add to exclude list? WBR, Yuri -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJUqmB0AAoJENxnfXtQ8ZQUPKkH/RQmpV8bVcQ1HKMWYLgzDIY1 qcW5AtSsiuuSQsDjew336/jE6WajfT3e5jKvvEKnLc993klJJcxlpETMRfqA+ekK nMnycoMGSX66bgnKb/TX2PBdUNqYj3WsC5ujDyQ/37VBYY7NNIc95VR5W460jj+F X9/sErTOsgy2/cYJ3Mz1+sHzH/IleehnosAtWaPBqXPCvi8Q4ud+SUFa8O3xgTPG 7BMM26prgMT0C8mpC/GRnMydEn4Pir3E5FEgkNZmMB7ZeQuHp0ZVKwUzvHqW/WP3 7pUydl3JKn1KP4bo1gXNO5m1Bf+X3xa9OiUTbzZ7VFrVPxXa7gOVKgZdCWpktcg= =K9bZ -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
On 01/05/2015 11:11 AM, Yuri Voinov wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 And also: don't forget about bogus homebrew internet-bankings. Which is uses bogus SSL-certs with bogus GOST realisations. And bogus Java-based clients. All of them also uses 443 port. And often HTTPS with homebrew bogus features. We don't know, how to bump it. What about it? Pass-through? Pass-through. This is clean exclusion. So don't worry about SSH/Tor. To block them we will be use another solutions. DPI. And not always technical. Revoke administrative rights from clients is basics of security, like physical access to infrastructure. If they can't install Tor/SSH - why we can worry about this traffic? We have (and can solve) two simple problems. HTTP over 443 port. And SSL Pinning. That's all, folks. It is clear what *you* want. You prefer Squid + HTTP filter + Cisco/DPI + tcputils + sniffer + manual maintenance of ACLs/exclude list 05.01.2015 17:51, Marcus Kool пишет: Much of the discussion so far has been about bumping traffic on port 443, bumping SSL-encapsulated HTTP traffic and not bumping (allowing) other traffic. Since port 443 is used for many protocols, it is in many cases dangerous to allow non-bumpable traffic: SSH tunnels using port 443 are common, so are VPNs. Do you know a security officer who does not want to block an SSH tunnel, or an app that can share corporate documents on public websites? If there is not more attention to these kinds of applications that use port 443 to circumvent corporate firewalls, Squid will be doomed to be used only in environments where the priority for security is low to non-existent. Just type punching holes in corporate firewalls or ssh tunnel proxy in Google to see how easy it is to use an SSH tunnel. I am the author of ufdbGuard, a filter for Squid and besides filtering based on URLs, ufdbGuard also probes port 443 to see what kind of traffic the server is expecting. By using probes, ufdbGuard can detect SSH tunnels, popular chat protocols, etc. but it is not a 100% guaranteed solution because ufdbGuard cannot not see the traffic that flows through the proxy, i.e. there is not yet an interface for this type of traffic inspection. Marcus On 01/05/2015 07:59 AM, Eliezer Croitoru wrote: Hey Yuri, Indeed there are other *NIX systems and for each and every one of them there is a solution in need. SSL Pinned destinations cannot be identified automatically since the are pinned inside a software and the certificate will not show anything about that. The basic tests we can do are: - The host is using ssl or tls or not at all(based on the selective answer of the service) - - If the connection is using tls\ssl then inspect the components of the certificate(such as rootCA validation against the local machine certificates DB) Depend on the goal of the certificate validation the decision will be made to either allow the connection uninspected or to bump it as is without any smart identification. If indeed there is a database sqlite3\mysql\postgres\redis\memcached\others it can be used in the iptables level. Also a point in this DB and this cache is that it will be persistent so what so ever the *NIX system is there is an option once the IP + port was tagged as non-bump-able it is better be in the FIREWALL level override better then squid external_acl. Reason: If the kernel does what it needs to do then squid should not touch the packets. It's not always right but it's a point in the issue. I still do not know how to work with NFQUEUE and I am sure that there is an option to make a fast decision and if not then let the connection be BUMPED. I have written a small golang script that can check couple things about the ssl session at: http://www1.ngtech.co.il/squid/ssl_helpers/ssl_validator.go Besides this helper there is another script which do couple things in another level. ## If any thing will be decided for squid internals it will be after a proof of concept that we can implement together. Can we take this thread to storm and put on the table a proof of concept logic for ssl inspection\bumping and bypassing? Eliezer On 01/05/2015 10:40 AM, Yuri Voinov wrote: Sounds good, but server world is not end on Linux. ;) Now exists another *NIX systems. And will exists further. Also. I have an idea, gents. Do we can easy and quickly detect SSL Pinned destinations? And remember it, for example, in database? In another words - both problems is similar. Either non-HTTPS traffic over 443 port, or pinned certs. Can we detect both of them automatically and add to exclude list? WBR, Yuri ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I think, non-HTTP/HTTPS security issues is never ever Squid function. Squid is not all-in-one-security-solution. It's only HTTP proxy. For others security breches (i.e SSH tunnels, various browser tunnel-related plugins, Tor etc., ) we have another tools. Cisco NBAR proticol discovery, other DPI solution. AFAIK, primitive Cisco extended ACL + tcpiputils.com and simple sniffer can make miracles. Following I offer to retirn to the Earth and limit our functionality as I said before. Just HTTP/HTTPS over 443 port. I understand that exists wonderful and overall solutions like ufdbGuard. But we talk not about it. 05.01.2015 17:51, Marcus Kool пишет: Much of the discussion so far has been about bumping traffic on port 443, bumping SSL-encapsulated HTTP traffic and not bumping (allowing) other traffic. Since port 443 is used for many protocols, it is in many cases dangerous to allow non-bumpable traffic: SSH tunnels using port 443 are common, so are VPNs. Do you know a security officer who does not want to block an SSH tunnel, or an app that can share corporate documents on public websites? If there is not more attention to these kinds of applications that use port 443 to circumvent corporate firewalls, Squid will be doomed to be used only in environments where the priority for security is low to non-existent. Just type punching holes in corporate firewalls or ssh tunnel proxy in Google to see how easy it is to use an SSH tunnel. I am the author of ufdbGuard, a filter for Squid and besides filtering based on URLs, ufdbGuard also probes port 443 to see what kind of traffic the server is expecting. By using probes, ufdbGuard can detect SSH tunnels, popular chat protocols, etc. but it is not a 100% guaranteed solution because ufdbGuard cannot not see the traffic that flows through the proxy, i.e. there is not yet an interface for this type of traffic inspection. Marcus On 01/05/2015 07:59 AM, Eliezer Croitoru wrote: Hey Yuri, Indeed there are other *NIX systems and for each and every one of them there is a solution in need. SSL Pinned destinations cannot be identified automatically since the are pinned inside a software and the certificate will not show anything about that. The basic tests we can do are: - The host is using ssl or tls or not at all(based on the selective answer of the service) - - If the connection is using tls\ssl then inspect the components of the certificate(such as rootCA validation against the local machine certificates DB) Depend on the goal of the certificate validation the decision will be made to either allow the connection uninspected or to bump it as is without any smart identification. If indeed there is a database sqlite3\mysql\postgres\redis\memcached\others it can be used in the iptables level. Also a point in this DB and this cache is that it will be persistent so what so ever the *NIX system is there is an option once the IP + port was tagged as non-bump-able it is better be in the FIREWALL level override better then squid external_acl. Reason: If the kernel does what it needs to do then squid should not touch the packets. It's not always right but it's a point in the issue. I still do not know how to work with NFQUEUE and I am sure that there is an option to make a fast decision and if not then let the connection be BUMPED. I have written a small golang script that can check couple things about the ssl session at: http://www1.ngtech.co.il/squid/ssl_helpers/ssl_validator.go Besides this helper there is another script which do couple things in another level. ## If any thing will be decided for squid internals it will be after a proof of concept that we can implement together. Can we take this thread to storm and put on the table a proof of concept logic for ssl inspection\bumping and bypassing? Eliezer On 01/05/2015 10:40 AM, Yuri Voinov wrote: Sounds good, but server world is not end on Linux. ;) Now exists another *NIX systems. And will exists further. Also. I have an idea, gents. Do we can easy and quickly detect SSL Pinned destinations? And remember it, for example, in database? In another words - both problems is similar. Either non-HTTPS traffic over 443 port, or pinned certs. Can we detect both of them automatically and add to exclude list? WBR, Yuri ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUqoe7AAoJENNXIZxhPexGJqoH/2byMvv1s1Qgk5Wh1EwB8ai/ B6wsw6IvHitGvkrS77OEkEtZYjMsbTEv7XEpZuBvsylIGTr8Zp9amyjcTLOG/A+2
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
Marcus, not to distract from the very important main points being discussed here but I have to question your last line: i.e. there is not yet an interface for this type of traffic inspection. Is that not the whole point of Squid's ICAP interface and HTTPS bumping? Or do you just mean that ufdbguard doesn't utilize that yet. If so you might want to consider adding that functionality... On Mon, Jan 5, 2015 at 9:10 AM, Marcus Kool marcus.k...@urlfilterdb.com wrote: On 01/05/2015 11:11 AM, Yuri Voinov wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 And also: don't forget about bogus homebrew internet-bankings. Which is uses bogus SSL-certs with bogus GOST realisations. And bogus Java-based clients. All of them also uses 443 port. And often HTTPS with homebrew bogus features. We don't know, how to bump it. What about it? Pass-through? Pass-through. This is clean exclusion. So don't worry about SSH/Tor. To block them we will be use another solutions. DPI. And not always technical. Revoke administrative rights from clients is basics of security, like physical access to infrastructure. If they can't install Tor/SSH - why we can worry about this traffic? We have (and can solve) two simple problems. HTTP over 443 port. And SSL Pinning. That's all, folks. It is clear what *you* want. You prefer Squid + HTTP filter + Cisco/DPI + tcputils + sniffer + manual maintenance of ACLs/exclude list 05.01.2015 17:51, Marcus Kool пишет: Much of the discussion so far has been about bumping traffic on port 443, bumping SSL-encapsulated HTTP traffic and not bumping (allowing) other traffic. Since port 443 is used for many protocols, it is in many cases dangerous to allow non-bumpable traffic: SSH tunnels using port 443 are common, so are VPNs. Do you know a security officer who does not want to block an SSH tunnel, or an app that can share corporate documents on public websites? If there is not more attention to these kinds of applications that use port 443 to circumvent corporate firewalls, Squid will be doomed to be used only in environments where the priority for security is low to non-existent. Just type punching holes in corporate firewalls or ssh tunnel proxy in Google to see how easy it is to use an SSH tunnel. I am the author of ufdbGuard, a filter for Squid and besides filtering based on URLs, ufdbGuard also probes port 443 to see what kind of traffic the server is expecting. By using probes, ufdbGuard can detect SSH tunnels, popular chat protocols, etc. but it is not a 100% guaranteed solution because ufdbGuard cannot not see the traffic that flows through the proxy, i.e. there is not yet an interface for this type of traffic inspection. Marcus On 01/05/2015 07:59 AM, Eliezer Croitoru wrote: Hey Yuri, Indeed there are other *NIX systems and for each and every one of them there is a solution in need. SSL Pinned destinations cannot be identified automatically since the are pinned inside a software and the certificate will not show anything about that. The basic tests we can do are: - The host is using ssl or tls or not at all(based on the selective answer of the service) - - If the connection is using tls\ssl then inspect the components of the certificate(such as rootCA validation against the local machine certificates DB) Depend on the goal of the certificate validation the decision will be made to either allow the connection uninspected or to bump it as is without any smart identification. If indeed there is a database sqlite3\mysql\postgres\redis\memcached\others it can be used in the iptables level. Also a point in this DB and this cache is that it will be persistent so what so ever the *NIX system is there is an option once the IP + port was tagged as non-bump-able it is better be in the FIREWALL level override better then squid external_acl. Reason: If the kernel does what it needs to do then squid should not touch the packets. It's not always right but it's a point in the issue. I still do not know how to work with NFQUEUE and I am sure that there is an option to make a fast decision and if not then let the connection be BUMPED. I have written a small golang script that can check couple things about the ssl session at: http://www1.ngtech.co.il/squid/ssl_helpers/ssl_validator.go Besides this helper there is another script which do couple things in another level. ## If any thing will be decided for squid internals it will be after a proof of concept that we can implement together. Can we take this thread to storm and put on the table a proof of concept logic for ssl inspection\bumping and bypassing? Eliezer On 01/05/2015 10:40 AM, Yuri Voinov wrote: Sounds good, but server world is not end on Linux. ;) Now
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Wait a minute, gents. What about ICAP? What I skipped? 05.01.2015 20:38, Douglas Davenport пишет: Marcus, not to distract from the very important main points being discussed here but I have to question your last line: i.e. there is not yet an interface for this type of traffic inspection. Is that not the whole point of Squid's ICAP interface and HTTPS bumping? Or do you just mean that ufdbguard doesn't utilize that yet. If so you might want to consider adding that functionality... On Mon, Jan 5, 2015 at 9:10 AM, Marcus Kool marcus.k...@urlfilterdb.com mailto:marcus.k...@urlfilterdb.com wrote: On 01/05/2015 11:11 AM, Yuri Voinov wrote: And also: don't forget about bogus homebrew internet-bankings. Which is uses bogus SSL-certs with bogus GOST realisations. And bogus Java-based clients. All of them also uses 443 port. And often HTTPS with homebrew bogus features. We don't know, how to bump it. What about it? Pass-through? Pass-through. This is clean exclusion. So don't worry about SSH/Tor. To block them we will be use another solutions. DPI. And not always technical. Revoke administrative rights from clients is basics of security, like physical access to infrastructure. If they can't install Tor/SSH - why we can worry about this traffic? We have (and can solve) two simple problems. HTTP over 443 port. And SSL Pinning. That's all, folks. It is clear what *you* want. You prefer Squid + HTTP filter + Cisco/DPI + tcputils + sniffer + manual maintenance of ACLs/exclude list 05.01.2015 17:51, Marcus Kool пишет: Much of the discussion so far has been about bumping traffic on port 443, bumping SSL-encapsulated HTTP traffic and not bumping (allowing) other traffic. Since port 443 is used for many protocols, it is in many cases dangerous to allow non-bumpable traffic: SSH tunnels using port 443 are common, so are VPNs. Do you know a security officer who does not want to block an SSH tunnel, or an app that can share corporate documents on public websites? If there is not more attention to these kinds of applications that use port 443 to circumvent corporate firewalls, Squid will be doomed to be used only in environments where the priority for security is low to non-existent. Just type punching holes in corporate firewalls or ssh tunnel proxy in Google to see how easy it is to use an SSH tunnel. I am the author of ufdbGuard, a filter for Squid and besides filtering based on URLs, ufdbGuard also probes port 443 to see what kind of traffic the server is expecting. By using probes, ufdbGuard can detect SSH tunnels, popular chat protocols, etc. but it is not a 100% guaranteed solution because ufdbGuard cannot not see the traffic that flows through the proxy, i.e. there is not yet an interface for this type of traffic inspection. Marcus On 01/05/2015 07:59 AM, Eliezer Croitoru wrote: Hey Yuri, Indeed there are other *NIX systems and for each and every one of them there is a solution in need. SSL Pinned destinations cannot be identified automatically since the are pinned inside a software and the certificate will not show anything about that. The basic tests we can do are: - The host is using ssl or tls or not at all(based on the selective answer of the service) - - If the connection is using tls\ssl then inspect the components of the certificate(such as rootCA validation against the local machine certificates DB) Depend on the goal of the certificate validation the decision will be made to either allow the connection uninspected or to bump it as is without any smart identification. If indeed there is a database sqlite3\mysql\postgres\redis\memcached\others it can be used in the iptables level. Also a point in this DB and this cache is that it will be persistent so what so ever the *NIX system is there is an option once the IP + port was tagged as non-bump-able it is better be in the FIREWALL level override better then squid external_acl. Reason: If the kernel does what it needs to do then squid should not touch the packets. It's not always right but it's a point in the issue. I still do not know how to work with NFQUEUE and I am sure that there is an option to make a fast decision and if not then let the connection be BUMPED. I have written a small golang script that can check couple things about the ssl session at: http://www1.ngtech.co.il/squid/ssl_helpers/ssl_validator.go http://www1.ngtech.co.il/squid/ssl_helpers/ssl_validator.go Besides this helper there is another script which do couple things in another level. ## If any thing will be decided for squid internals it will be after a proof of concept that we can implement together. Can we take this thread to storm and put on the table a proof of concept logic for
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/05/2015 05:18 PM, Yuri Voinov wrote: We haven't filtering non_HTTP over port-443. Just recognize and pass. So let's separate security which is one of the goals of squid and which some like and other don't. For now squid 3.4 is stable and 3.5 is in beta and trunk is not for the public use. In 3.5 there will be present a new feature which called peek and splice that can give an interface to squid and the admin which will allow the admin to know couple things about the connection from squid and specifically first the client TLS request. Once squid bumped a connection there are couple steps until the connection is fully established between the client and the server: - - receive the TCP connection from client - - BUMP server or client FIRST - - determine what certificate to send to the client based on the server initial ssl response - - fake it - - send to the client - - MITM between two tls connections on the proxy while inspecting the content in the software layer. Peek and splice will add another step between the first part to the second and which will allow SNI usage. All the above is to allow better BUMPING. There might be or will be probably an interface that will identify or will try to identify inside the current stages of the connection bumping if the connection is indeed a TLS or another one. The first step of peek and splice can identify if the connection from the client side has started using a valid TLS\SSL headers. Leaving all the BUMPING yes or no You(Yuri) need a very specific tool or want a very specific tool. The basic interface of the external_acl can provide enough data on the connection in order to enforce some rules. You can either use the client IP address or just the destination IP and PORT. I cannot speak for the squid project but I am almost sure that the squid project will not provide you with an official helper and will not support it. However squid external_acl is there especially to help others achieve what they want using a variety of parameters from squid internals. The external_acl interface provides internal caching which supports caching ttl with different values for the two options either allow(OK) or DENY(ERR). My suggestions stays, don't use sqlite if possible. There is a sketch for a helper like you seems to want. Take the glove and write a pesudo code for the helper idea based on the assumptions: - - There is a DB which can store timestamps, ip, port, result of test, etc - - There is a way to check if the certificate is valid and the server works with TLS\SSL - - There is no way for the helper to know that a certificate is pined - - There is a way to add static records to the DB(web interface, cli) - - All the requests will come from the proxy IP address and can by some be identified as an attack. - - ufdbguard does not provide your needs since it uses url_rewrite interface and doesn't have the needed functionalities for you. The best I have seen until now was the python helper. If in couple(4-5) month nobody will do something with this I will see then what can be done with this if at all. Elizer -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJUqrvGAAoJENxnfXtQ8ZQUIDsH/iw82UdjIvwk/bycmuvTIgi2 FjgdXHcOP1EFi1aC6utgZ3ab8GVaOhV8PJpLpFSD3ZLbrFXfLg9c3ubY6EMHVxWz HNgYBQ/MetSCTHwNKiKHiu8pqy0CR9aTth91KwArrWtXYBqeGyCVCRvyccHc07u4 QVE3rUTNX0ICAgvfrhyUsjgjZVUCRC1dvZT7c2aVeowR1qyvfpK2JRVJSuaUI2oU HS2516qxzxHvyxJjjz1Cypn06BPiCp2wItIPdX9biEXid2DCJsGGrd9hjhkoZtgH AiAf7mpFnWZybw4934S1ubUE5x59v8rzpZiuVTE/iSIpVzAci2moeijqosL2yEg= =7quC -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Agreed. I'm expert on shell, not Perl/Python. :) But will try to make some useful with it. 05.01.2015 22:28, Eliezer Croitoru пишет: On 01/05/2015 05:18 PM, Yuri Voinov wrote: We haven't filtering non_HTTP over port-443. Just recognize and pass. So let's separate security which is one of the goals of squid and which some like and other don't. For now squid 3.4 is stable and 3.5 is in beta and trunk is not for the public use. In 3.5 there will be present a new feature which called peek and splice that can give an interface to squid and the admin which will allow the admin to know couple things about the connection from squid and specifically first the client TLS request. Once squid bumped a connection there are couple steps until the connection is fully established between the client and the server: - receive the TCP connection from client - BUMP server or client FIRST - determine what certificate to send to the client based on the server initial ssl response - fake it - send to the client - MITM between two tls connections on the proxy while inspecting the content in the software layer. Peek and splice will add another step between the first part to the second and which will allow SNI usage. All the above is to allow better BUMPING. There might be or will be probably an interface that will identify or will try to identify inside the current stages of the connection bumping if the connection is indeed a TLS or another one. The first step of peek and splice can identify if the connection from the client side has started using a valid TLS\SSL headers. Leaving all the BUMPING yes or no You(Yuri) need a very specific tool or want a very specific tool. The basic interface of the external_acl can provide enough data on the connection in order to enforce some rules. You can either use the client IP address or just the destination IP and PORT. I cannot speak for the squid project but I am almost sure that the squid project will not provide you with an official helper and will not support it. However squid external_acl is there especially to help others achieve what they want using a variety of parameters from squid internals. The external_acl interface provides internal caching which supports caching ttl with different values for the two options either allow(OK) or DENY(ERR). My suggestions stays, don't use sqlite if possible. There is a sketch for a helper like you seems to want. Take the glove and write a pesudo code for the helper idea based on the assumptions: - There is a DB which can store timestamps, ip, port, result of test, etc - There is a way to check if the certificate is valid and the server works with TLS\SSL - There is no way for the helper to know that a certificate is pined - There is a way to add static records to the DB(web interface, cli) - All the requests will come from the proxy IP address and can by some be identified as an attack. - ufdbguard does not provide your needs since it uses url_rewrite interface and doesn't have the needed functionalities for you. The best I have seen until now was the python helper. If in couple(4-5) month nobody will do something with this I will see then what can be done with this if at all. Elizer ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUqr3NAAoJENNXIZxhPexG5UoIAKxANAcs77iuSQlNOmbO9D4B xip3QbFhug2/LayR+Wa1Vd3UBUfkSUUdvvqedXRc6KKfCqxa5BECTYSpM0qT/L+h WasstLpV/Mm2seYRK/CUJbmAJDps6mAgB8MdU1Kq9XWUVYGuGHXnWa220sU/TuhW wD55VRDScX7cELOQyv4kCr/3mqobLD0KLAMwpDwtxel88eE9NYFW1OcIyM2XHtJd ouY/hM6sAlYusXITrQrbOy7Sw5yT6DjY+sm6NYx7NCpDyKZTZemU0QVN9hEG6H0s AmPi0m3OedUAmh83rXMS47+GyzIq3yxIqe52qOsFSsA5PoK/l93zqRivvUUTxyQ= =sq3n -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
On 01/05/2015 12:38 PM, Douglas Davenport wrote: Marcus, not to distract from the very important main points being discussed here but I have to question your last line: i.e. there is not yet an interface for this type of traffic inspection. Is that not the whole point of Squid's ICAP interface and HTTPS bumping? Or do you just mean that ufdbguard doesn't utilize that yet. If so you might want to consider adding that functionality... ICAP does HTTP filtering based on URL and HTTP content port-443 filtering based on FQDN (ICAP server only receives CONNECT FQDN:443) port-443 filtering based on HTTP content _if_ Squid uses sslbump ICAP does _not_ port-443 filtering for non-HTTP protocols ICAP was designed for HTTP-based content modification. There is no industry standard for filtering non-HTTP data. I have discussed this subject twice with the Squid development team but there is currently no sponsor to implement a new protocol to filter non-HTTP data in Squid. Marcus On Mon, Jan 5, 2015 at 9:10 AM, Marcus Kool marcus.k...@urlfilterdb.com mailto:marcus.k...@urlfilterdb.com wrote: On 01/05/2015 11:11 AM, Yuri Voinov wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 And also: don't forget about bogus homebrew internet-bankings. Which is uses bogus SSL-certs with bogus GOST realisations. And bogus Java-based clients. All of them also uses 443 port. And often HTTPS with homebrew bogus features. We don't know, how to bump it. What about it? Pass-through? Pass-through. This is clean exclusion. So don't worry about SSH/Tor. To block them we will be use another solutions. DPI. And not always technical. Revoke administrative rights from clients is basics of security, like physical access to infrastructure. If they can't install Tor/SSH - why we can worry about this traffic? We have (and can solve) two simple problems. HTTP over 443 port. And SSL Pinning. That's all, folks. It is clear what *you* want. You prefer Squid + HTTP filter + Cisco/DPI + tcputils + sniffer + manual maintenance of ACLs/exclude list 05.01.2015 17:51, Marcus Kool пишет: Much of the discussion so far has been about bumping traffic on port 443, bumping SSL-encapsulated HTTP traffic and not bumping (allowing) other traffic. Since port 443 is used for many protocols, it is in many cases dangerous to allow non-bumpable traffic: SSH tunnels using port 443 are common, so are VPNs. Do you know a security officer who does not want to block an SSH tunnel, or an app that can share corporate documents on public websites? If there is not more attention to these kinds of applications that use port 443 to circumvent corporate firewalls, Squid will be doomed to be used only in environments where the priority for security is low to non-existent. Just type punching holes in corporate firewalls or ssh tunnel proxy in Google to see how easy it is to use an SSH tunnel. I am the author of ufdbGuard, a filter for Squid and besides filtering based on URLs, ufdbGuard also probes port 443 to see what kind of traffic the server is expecting. By using probes, ufdbGuard can detect SSH tunnels, popular chat protocols, etc. but it is not a 100% guaranteed solution because ufdbGuard cannot not see the traffic that flows through the proxy, i.e. there is not yet an interface for this type of traffic inspection. Marcus On 01/05/2015 07:59 AM, Eliezer Croitoru wrote: Hey Yuri, Indeed there are other *NIX systems and for each and every one of them there is a solution in need. SSL Pinned destinations cannot be identified automatically since the are pinned inside a software and the certificate will not show anything about that. The basic tests we can do are: - The host is using ssl or tls or not at all(based on the selective answer of the service) - - If the connection is using tls\ssl then inspect the components of the certificate(such as rootCA validation against the local machine certificates DB) Depend on the goal of the certificate validation the decision will be made to either allow the connection uninspected or to bump it as is without any smart identification. If indeed there is a database sqlite3\mysql\postgres\redis\__memcached\others it can be used in the iptables level. Also a point in this DB and this cache is that it will be persistent so what
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 We haven't filtering non_HTTP over port-443. Just recognize and pass. 05.01.2015 21:15, Marcus Kool пишет: On 01/05/2015 12:38 PM, Douglas Davenport wrote: Marcus, not to distract from the very important main points being discussed here but I have to question your last line: i.e. there is not yet an interface for this type of traffic inspection. Is that not the whole point of Squid's ICAP interface and HTTPS bumping? Or do you just mean that ufdbguard doesn't utilize that yet. If so you might want to consider adding that functionality... ICAP does HTTP filtering based on URL and HTTP content port-443 filtering based on FQDN (ICAP server only receives CONNECT FQDN:443) port-443 filtering based on HTTP content _if_ Squid uses sslbump ICAP does _not_ port-443 filtering for non-HTTP protocols ICAP was designed for HTTP-based content modification. There is no industry standard for filtering non-HTTP data. I have discussed this subject twice with the Squid development team but there is currently no sponsor to implement a new protocol to filter non-HTTP data in Squid. Marcus On Mon, Jan 5, 2015 at 9:10 AM, Marcus Kool marcus.k...@urlfilterdb.com mailto:marcus.k...@urlfilterdb.com wrote: On 01/05/2015 11:11 AM, Yuri Voinov wrote: And also: don't forget about bogus homebrew internet-bankings. Which is uses bogus SSL-certs with bogus GOST realisations. And bogus Java-based clients. All of them also uses 443 port. And often HTTPS with homebrew bogus features. We don't know, how to bump it. What about it? Pass-through? Pass-through. This is clean exclusion. So don't worry about SSH/Tor. To block them we will be use another solutions. DPI. And not always technical. Revoke administrative rights from clients is basics of security, like physical access to infrastructure. If they can't install Tor/SSH - why we can worry about this traffic? We have (and can solve) two simple problems. HTTP over 443 port. And SSL Pinning. That's all, folks. It is clear what *you* want. You prefer Squid + HTTP filter + Cisco/DPI + tcputils + sniffer + manual maintenance of ACLs/exclude list 05.01.2015 17:51, Marcus Kool пишет: Much of the discussion so far has been about bumping traffic on port 443, bumping SSL-encapsulated HTTP traffic and not bumping (allowing) other traffic. Since port 443 is used for many protocols, it is in many cases dangerous to allow non-bumpable traffic: SSH tunnels using port 443 are common, so are VPNs. Do you know a security officer who does not want to block an SSH tunnel, or an app that can share corporate documents on public websites? If there is not more attention to these kinds of applications that use port 443 to circumvent corporate firewalls, Squid will be doomed to be used only in environments where the priority for security is low to non-existent. Just type punching holes in corporate firewalls or ssh tunnel proxy in Google to see how easy it is to use an SSH tunnel. I am the author of ufdbGuard, a filter for Squid and besides filtering based on URLs, ufdbGuard also probes port 443 to see what kind of traffic the server is expecting. By using probes, ufdbGuard can detect SSH tunnels, popular chat protocols, etc. but it is not a 100% guaranteed solution because ufdbGuard cannot not see the traffic that flows through the proxy, i.e. there is not yet an interface for this type of traffic inspection. Marcus On 01/05/2015 07:59 AM, Eliezer Croitoru wrote: Hey Yuri, Indeed there are other *NIX systems and for each and every one of them there is a solution in need. SSL Pinned destinations cannot be identified automatically since the are pinned inside a software and the certificate will not show anything about that. The basic tests we can do are: - The host is using ssl or tls or not at all(based on the selective answer of the service) - - If the connection is using tls\ssl then inspect the components of the certificate(such as rootCA validation against the local machine certificates DB) Depend on the goal of the certificate validation the decision will be made to either allow the connection uninspected or to bump it as is without any smart identification. If indeed there is a database sqlite3\mysql\postgres\redis\__memcached\others it can be used in the iptables level. Also a point in this DB and this cache is that it will be persistent so what so ever the *NIX system is there is an option once the IP + port was tagged as non-bump-able it is better be in the FIREWALL level override better then squid external_acl. Reason: If the kernel does what it needs to do then squid should not touch the packets. It's not always right but it's a point in the issue. I still do not know how
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 To return to Earth: I think, a good idea is built-in (ma be, in ssl_crtd?) functionality to check 443 port connection for Is an HTTPS inside? and if no, do not bump by default. This is so simple and fast, is it? And we can have some config option to disable this function...Or not. ;) 05.01.2015 2:17, Douglas Davenport пишет: I saw a very similar feature in ufdbGuard which is a URL filter implemented as a Squid Redirector. They have a feature which probes the destination server for a valid HTTPS cert in parallel to the user's connection and terminates it if it turns out not to be a valid HTTPS cert. Their code is open source, maybe this could be helpful in creating such a helper? http://www.urlfilterdb.com/home.html On Sat, Jan 3, 2015 at 3:45 AM, Yuri Voinov yvoi...@gmail.com mailto:yvoi...@gmail.com wrote: Term HTTPS often uses as Any connect over 443 port 03.01.2015 13:59, Jason Haar пишет: On 01/01/15 00:11, James Harper wrote: The helper connects to the IP:port and tries to obtain the certificate, and then caches the result (in an sqlite database). If it can't do so within a fairly short time it returns failure (but keeps trying a bit longer and caches it for next time). Alternatively if the IP used to be SSL but is now timing out it returns the previously cached value. Negative results are cached for an increasing amount of time each time it fails, on the basis that it probably isn't SSL. That sounds great James! I'd certainly like to take a look at it too However, you say SSL - did you mean HTTPS? ie discovering a ip:port is a IMAPS server doesn't really help squid talk to it - surely you want to discover HTTPS servers - and everything else should be pass-through/splice? ___ squid-users mailing list squid-users@lists.squid-cache.org mailto:squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUqaH3AAoJENNXIZxhPexG+VAH/3+fQfAUA1IdCXTdvZXjR2Ih 2AAa2d/mPOQtk1RNTk7PUxp1tIuUVt054euuwnhxItGSIb6OB7U2fTHK1k3BF+ta BG6fyghpKYFBHJkloYX6m45g7K3vgpKEjVPDQZuaUz1CBZ67ie/ThngxmgNwFqaO HbDvcX6FnvYeplRDrsx8DATD7fqujw5wy6ZI+23bXAOf4j7PO6zwIeoh4hSkMhr/ 7ZRBYv2T6iYh+sL3XiYgVh9fWcGy2O2ovJLW/2AA4YXnlEAGLgbgVZCiF6jIdomn iSiel6enLOCDneLsMcW1h+n7HNTTpv6N2D5ATMEcf8kz3ACmH99mBhk2bh9jQ94= =Iw5c -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
Seems to me it would be more useful as an external ACL so that a decision could be made based on other factors eg src or dstdomain whether to deny or allow the un-bumpable connection. On Sun, Jan 4, 2015 at 4:29 PM, Yuri Voinov yvoi...@gmail.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 As I can see, we have two major problems with SSL Bump now. 1. Stupid apps and it's stupid developers - like ICQ and other stupid IM - which is hope 443 port is never be blocked due to using for logons/internet banking etc. This stupid way broke standards (?) and make us crazy. Now single solution is catch them manually and pass it without bumping. This is the simplest problem. And I hope it will be solved in core - i.e. in Squid directly. 2. SSL Pinned sites. We cannot do with them anything excluding sniff it and pass by IP without bump. First problems seems to solve easy. Either by helper, or by squid - no matter. It's really simple. Just check SSL cert on server side - and make decision - to bump, or not to bump. The second problem seems difficult and now I can't see any reasonable solution, excluding sniffer/manual add to acl. Any ideas? Will be write helper? WBR, Yuri 05.01.2015 2:17, Douglas Davenport пишет: I saw a very similar feature in ufdbGuard which is a URL filter implemented as a Squid Redirector. They have a feature which probes the destination server for a valid HTTPS cert in parallel to the user's connection and terminates it if it turns out not to be a valid HTTPS cert. Their code is open source, maybe this could be helpful in creating such a helper? http://www.urlfilterdb.com/home.html On Sat, Jan 3, 2015 at 3:45 AM, Yuri Voinov yvoi...@gmail.com mailto:yvoi...@gmail.com yvoi...@gmail.com wrote: Term HTTPS often uses as Any connect over 443 port 03.01.2015 13:59, Jason Haar пишет: On 01/01/15 00:11, James Harper wrote: The helper connects to the IP:port and tries to obtain the certificate, and then caches the result (in an sqlite database). If it can't do so within a fairly short time it returns failure (but keeps trying a bit longer and caches it for next time). Alternatively if the IP used to be SSL but is now timing out it returns the previously cached value. Negative results are cached for an increasing amount of time each time it fails, on the basis that it probably isn't SSL. That sounds great James! I'd certainly like to take a look at it too However, you say SSL - did you mean HTTPS? ie discovering a ip:port is a IMAPS server doesn't really help squid talk to it - surely you want to discover HTTPS servers - and everything else should be pass-through/splice? ___ squid-users mailing list squid-users@lists.squid-cache.org mailto:squid-users@lists.squid-cache.org squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUqbC7AAoJENNXIZxhPexGwwkH/j8XR2fQ4v/r3M2zFuDuhVsP JZMM93IvZrGYRzJjAmmwg7ZUoYdwWWEaXoY6GygO+RdZESWfPvh00cSsxwRKfmvm 2s7sLDKlPnfUsf9fyWnihCtJg9hETZTsvUqK9I+iopiM1DHq/qwX3Pjkb2e2T45u JuqU5ySBZPEt6G1gRn/+F2EyHdhWpa9OOtfeTAt4/oaJIuLoHP7855fif/1eg59U QlISZkLjDcL4DqEVM+9UJh9TSN+dawj/Ks+3b+MT8sA/xvVdOyqhLMqnm4MPadSv yvK5nQWW4rkfHOJ1zwWq3hAMLjCIXjY4q1NxNQAxdK5ESZvszecvXg3JMKo/THw= =Ygen -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
On 05/01/15 15:44, Eliezer Croitoru wrote: A squid helper is nice but... a NFQUEUE helper that can verify if to FORWARD or BUMP the connection would be a better suited solution to my opinion. Not sure if you're ignoring the ssl-peek work, but squid still needs to be able to peek in order for squid to know the actual HTTPS server name the client is connecting to before it's able to call any external helper/etc. As that involves understanding SSL (which is a huge chunk of code) - that means it's not appropriate for a kernel solution - it has to be done at Layer-7 (ie squid - but not some app called by squid as that's too late to see the data it needs) eg after hearing how James Harper wrote his own external https-tester script, I've written my own and have been merrily testing it under squid-3.4.10 (ie not 3.5 with peek). In proxy-mode it works great, the https-tester script is passed the DNS name and port, it manually uses curl to test that to ensure it's a real HTTPS server and returns OK, else it returns ERR - making squid fall-back on passthrough/splice mode. That means it can detect non-SSL apps, as well as client-cert protected HTTPS webservers (which you also have to drop back to splice with - you can never successfully MiTM a client-cert based SSL session). However, the moment you try to do transparent https proxy, things break. In that case, squid-3.4 only sees the destination IP, and https_tester can only try to curl -k https://ip.add.ress:port/; - which only works for *some* webservers. A lot have WAFs on them and righteously ditch the incoming connection when they recognise the client (my script) doesn't know what their hostname is. eg any HTTPS site using cloudfront.net is in that category. Of course it still works - but in passthrough mode - which isn't the outcome we're after. I'm going to have to look at squid-3.5 ;-) -- Cheers Jason Haar Corporate Information Security Manager, Trimble Navigation Ltd. Phone: +1 408 481 8171 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hey Thread(Jason,Yuri,Douglas...), There are couple aspects about the ssl and connections in general and as we talk about ssl port I first would like to put couple things on the table. * Squid is a http caching proxy and there for every feature which is out of the http related scope should not be handled by squid at all. * Any squid operation is an application level and there for is limited by the software(kernel + squid). * There is a difference between servers taking a load of 1k requests per second to a SMB which handles about 50 requests per second. In general it's better to not intercept a connection which is not bump-able. The decision about if to DROP\REJECT or ACCEPT the connection should better not involve squid in general if possible. Squid offers a very nice interface but if you compare the Linux kernel forwarding capabilities compared to squid you would see that squid is very limited in the userspace. So in a case the helper only needs to know if the connection is bump-able there are other alternatives in the Linux kernel!! And if you need logs.. you can use the *helper*(which one you ever choose to work with) to log... So now for the real thing: My opinion about external_acl vs other solutions is that if squid with an external_acl works for you and you understand what it means from performance and security aspects try it, test it and then use it. But if your squid load is high and in the case squid slows down the bumped connections too much(compared to linux forwarding) I would try to use something like NFQUEUE to just test if the connection is bump-able or not by IP and DST PORT. * some information about NFQUEUE https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/ http://suricata-ids.org/ * Some examples: https://www.wzdftpd.net/redmine/projects/nfqueue-bindings/repository/entry/examples/rewrite.py http://danmcinerney.org/reliable-dns-spoofing-with-python-scapy-nfqueue/ http://5d4a.wordpress.com/2011/08/25/having-fun-with-nfqueue-and-scapy/ A squid helper is nice but... a NFQUEUE helper that can verify if to FORWARD or BUMP the connection would be a better suited solution to my opinion. All The Bests, Eliezer Croitoru On 01/05/2015 03:07 AM, Douglas Davenport wrote: Seems to me it would be more useful as an external ACL so that a decision could be made based on other factors eg src or dstdomain whether to deny or allow the un-bumpable connection. -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJUqfp/AAoJENxnfXtQ8ZQUSwoH/icug7X4oexKW1hewZ+u6mUd /MTS+0pfrcl3ZiKi3oNcYBbnI+o1oqDVgYdW2XKpeWuUaZpvDF0NsIsO0Aj+0kjy BccCiofUQABKPuG2MtM4ODMUbouoob3eBWdVmbbRr3KWyAu0aPnjMHUWX5QeSAif 8FF38xQvnR4EOLi7UmT8UOV4iugloxg8feDjIxRcRPJalfAGrOyfGmFYuxoFELjg 7XMTTLkJW2DuteIl4M1cdKRSJKt/CUKah3z5D3EyDBQcHhV4xDUk3ncTLXXr1cZH kLnkFQWC2Jq43S2zEin7STJE3zumWu/YN/s2Wj3oAmyPAQBAlkNpHB4VPoF9w+c= =L6HT -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
On 01/01/15 00:11, James Harper wrote: The helper connects to the IP:port and tries to obtain the certificate, and then caches the result (in an sqlite database). If it can't do so within a fairly short time it returns failure (but keeps trying a bit longer and caches it for next time). Alternatively if the IP used to be SSL but is now timing out it returns the previously cached value. Negative results are cached for an increasing amount of time each time it fails, on the basis that it probably isn't SSL. That sounds great James! I'd certainly like to take a look at it too However, you say SSL - did you mean HTTPS? ie discovering a ip:port is a IMAPS server doesn't really help squid talk to it - surely you want to discover HTTPS servers - and everything else should be pass-through/splice? -- Cheers Jason Haar Corporate Information Security Manager, Trimble Navigation Ltd. Phone: +1 408 481 8171 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Term HTTPS often uses as Any connect over 443 port 03.01.2015 13:59, Jason Haar пишет: On 01/01/15 00:11, James Harper wrote: The helper connects to the IP:port and tries to obtain the certificate, and then caches the result (in an sqlite database). If it can't do so within a fairly short time it returns failure (but keeps trying a bit longer and caches it for next time). Alternatively if the IP used to be SSL but is now timing out it returns the previously cached value. Negative results are cached for an increasing amount of time each time it fails, on the basis that it probably isn't SSL. That sounds great James! I'd certainly like to take a look at it too However, you say SSL - did you mean HTTPS? ie discovering a ip:port is a IMAPS server doesn't really help squid talk to it - surely you want to discover HTTPS servers - and everything else should be pass-through/splice? -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUp6wSAAoJENNXIZxhPexGb8MH/irRYZBuoGjHZrpcI6kweMGv YqSjFYUasZ/hlDoN6bbJUKqAfeos0am0OuTy2FxOmA0YVxIEz6sJKj9FzeMJtOSW NTZk7IJ7mT6aRg+hKfW3JCEl68RcLb0J/eSNvG6QR6HcqHQODiEE489zcq+o+yn0 Z45P1WwgQLv6PIIeNXnM7nFtA0ce3D54agu/fr7zC3c1Z72A04BMU0W4dFC9M6Ob T2NQz2CsSp+nIDFZjHTaZuBmw5ZjMtrsoO79FT5GyX1lT+tCR9angtI+TYSSis15 o+/aw1U+yWScZXhdNUz/gjWLbW8WL/9ygVY43Y2laPII+WdlFhaJozEhr5h1e+A= =2f0+ -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2/01/2015 1:21 p.m., Eliezer Croitoru wrote: Hey Yuri, You would want to avoid sqlite as far as you can due to it's Whole DB file LOCK nature. Indeed. My experience with SQLite has been that it is vastly slower than other DB options even a remote SQL database. FWIW other Squid helpers written in C/C++ use BerkleyDB, and if written in Perl use the DBI module for generic SQL DB's. There is also the option of not using a DB at all in the initial version. Squid has a cache of previous helper results which it uses instead of calling the helper repeatedly for the same input. Yuri, regarding Squid packaged helpers... Distribution of any code within the Squid package requires: 1) copyright under GPLv2+ or a GPL compatible license (such as Public Domain, or BSD 3-clause license). 2) compliance with the Squid coding and documentation guidelines. - One guideline is that the code be as portable as possible. So take that into consideration with scripts. * I also request authors/submitters of new helpers to please provide at least 1 years support for it. Both in assistance of the community here in squid-users learning how to use the helper, and in support maintaining its code through its early period when most bugs are found. Amos -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUphGSAAoJELJo5wb/XPRj600H/i23sN/OgYocxWTAZ3/3lvM/ I/Z7KA/JTBCsVbWybO9WlNHZQkm9cE+F4H2a62Aw6MVojYERq7+p30/sB29GxGhS 05+JwpzvQ+bQfDgAi8fp4XPSnjuSBjA7MdQPUf6sAH8qFfOtuLYX3DBFKVGA77dY mLxLsZcozJyqzMbwA55A4nU5sOBZo0je9LMdYXt0ag05RVVLRz4baGKKmZIH6rd2 HbZ5O2glTHhOv1UJ/oHoeX0LD3U1bg88iUD455/SEYwOgR56D0E6XfnK8MWYgYy7 7tsj1FkkVBxQKso6YT8+rD1czR1+eyQa4MT92nWMUBxsOmNifS4ujkhO1zZNIGQ= =yJ2S -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2/01/2015 4:33 p.m., Amos Jeffries wrote: Yuri, regarding Squid packaged helpers... Opps sorry that should have been directed at James or anyone wishing to bundle his helper with Squid. Distribution of any code within the Squid package requires: 1) copyright under GPLv2+ or a GPL compatible license (such as Public Domain, or BSD 3-clause license). 2) compliance with the Squid coding and documentation guidelines. - One guideline is that the code be as portable as possible. So take that into consideration with scripts. * I also request authors/submitters of new helpers to please provide at least 1 years support for it. Both in assistance of the community here in squid-users learning how to use the helper, and in support maintaining its code through its early period when most bugs are found. Amos -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUphLKAAoJELJo5wb/XPRjcR4IAM+oU38WD75nhjqZL88adyfC 9GkmtXuKRUSErHMNdllBuedLdg0d1ws4PDS/u3MqNoGiNKb5EXxdX1Uu508ym9E2 dmFqatGSsjYyhOFkFx3saFCSm1bX2w+dcJDlp9c30YKRAuTQfABicYg1b+y6CU6S RU1yRZpSAkzE/8bjl1iufNHf6CJ2BiIVFLmyf6SpDPvYOzMTQ3S8pGbslvi3H2Dd 1MkU1caInOhSifNvKpXUx8CcODBkCI4F7GXO/kNxmNEOHp0OD0xempa3sgUjmsIJ rzLdnt84MeG1LRSJONajJshLp72FemQOl19KjfgmSO97EnhSKMFk2cTOlEuMKN4= =fMGW -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
Probably non-HTTPS protocol being used. As bumping gets more popular we are hearing about a number of services abusing port 443 for non-HTTPS protocols on the false assumption that the TLS layer goes all the way to the origin server without inspection. That has never been a true assumption, CDN frontends have always decrypted. I've mentioned it before, and it's been pointed out that it probably doesn't scale, but I wrote a callout helper for this reason and for better logging. Basically: external_acl_type cert_callout concurrency=20 %DST %PORT /usr/local/squid/libexec/ext_cert_callout_acl acl is_ssl external cert_callout IS_SSL ssl_bump bump is_ssl ssl_bump splice all The helper connects to the IP:port and tries to obtain the certificate, and then caches the result (in an sqlite database). If it can't do so within a fairly short time it returns failure (but keeps trying a bit longer and caches it for next time). Alternatively if the IP used to be SSL but is now timing out it returns the previously cached value. Negative results are cached for an increasing amount of time each time it fails, on the basis that it probably isn't SSL. So it's somewhat optimistic - on a slow link or for a slow server a given IP:port might be spliced the first time when it should be bumped, but it will learn fairly quickly. And for non-SSL connections that appear to just hang in response to the request there will be an additional delay the first time. But otherwise it learns about the target without requiring admin intervention. It also returns the name of the cert so it can be logged instead of the IP (for transparent connections), although that isn't ideal as the same IP might be used for many services (google/youtube/etc) I can post it if anyone is interested, although it would require a bit of work to be completely useful. James ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 31/12/2014 2:12 a.m., Yuri Voinov wrote: Hi gents, I found strange issue. Squid 3.4.10. Intercept. HTTPS bumping. All works fine. All configs correct. Whenever all web https sites works perfectly - especially in Chrome, most cloud clients works like charm (SpiderOak is!), Google Drive client application (PC) could not work. Note: Web Google Docs works. Web Google drive works. Note: Google support info - even I if pass dozen Google URL's without bump - cannot help. It doesn't work when server-first bumping is on and works othervise. So, the Serious Question is: Why? :) Any idea? Probably non-HTTPS protocol being used. As bumping gets more popular we are hearing about a number of services abusing port 443 for non-HTTPS protocols on the false assumption that the TLS layer goes all the way to the origin server without inspection. That has never been a true assumption, CDN frontends have always decrypted. Amos -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUoq9PAAoJELJo5wb/XPRjcSQIAJk349r6X95aJbaTTueFWfgP RkAOwmDF7M6kiir8XgJ0D4+LXXL4mTxwkuGlIDe1au1jrhMAY8S1aIoU1Dc0BN5x z4j9m1OeSeTrMAVvDCSFdf+pvAWOpLclY+f6b5HY773vus0kqT8PWhb4wVaIHP3m 9JtgpKOD9ElWdZosGVtdIl3uaExrmFGFbNLeVVo6VdeSE/D21YcH2HtEo3b+dopx 9T1SCMHItT83lr0LLFTlgTlKBrjREuKMNHN1L6FWkyJSe9OSu1efPvm23MG33G8a ShbFMrWh9Zo50fVXZRidAyXlvjBcX7aHM/ZSb8+3ID5O7Ao484aPvbSSck9Hq/g= =IeYG -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
SSL Pinning? (I know Dropbox does this) my two cents only :) Raf From: squid-users squid-users-boun...@lists.squid-cache.org on behalf of Yuri Voinov yvoi...@gmail.com Sent: Tuesday, December 30, 2014 2:12 PM To: squid-users@lists.squid-cache.org Subject: [squid-users] Squid 3 SSL bump: Google drive application could not connect -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi gents, I found strange issue. Squid 3.4.10. Intercept. HTTPS bumping. All works fine. All configs correct. Whenever all web https sites works perfectly - especially in Chrome, most cloud clients works like charm (SpiderOak is!), Google Drive client application (PC) could not work. Note: Web Google Docs works. Web Google drive works. Note: Google support info - even I if pass dozen Google URL's without bump - cannot help. It doesn't work when server-first bumping is on and works othervise. So, the Serious Question is: Why? :) Any idea? -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUoqSqAAoJENNXIZxhPexGMlsH/iEoVUCBOdF174Kj8I9mBbI3 RFdsouwnd6IMT0HTBt+MtjHK00NU/8yKVwgmUy4IyqVK1k6j+96ipGR6SeRsVzu1 RC7eym7ZLlRBWKUx+evQ5VhYXv7c1sX6UzD8irAUAp1019mZCVm5LfM4JJ/Vadw+ 3mlOj4ASQGjDnA3BU+YKUhITFFdOY8vGrZKrJvY8JtkpOZ6i+GKW4V8UB+uGXicL cYpx+LEbIV6UCL+lA4O3vOiQPBEVymu+TCeQ2gp6sxwNE+927TNrJEh8hp6IXha1 KAgareSZK3lp+/NXQwJTRsUw1YF768iPWP+RcB1Ao+2CGDjGk49b6WigvvHKlL0= =Vbof -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 May be. Does workaround exists? 30.12.2014 20:09, Rafael Akchurin ?: SSL Pinning? (I know Dropbox does this) my two cents only :) Raf From: squid-users squid-users-boun...@lists.squid-cache.org on behalf of Yuri Voinov yvoi...@gmail.com Sent: Tuesday, December 30, 2014 2:12 PM To: squid-users@lists.squid-cache.org Subject: [squid-users] Squid 3 SSL bump: Google drive application could not connect Hi gents, I found strange issue. Squid 3.4.10. Intercept. HTTPS bumping. All works fine. All configs correct. Whenever all web https sites works perfectly - especially in Chrome, most cloud clients works like charm (SpiderOak is!), Google Drive client application (PC) could not work. Note: Web Google Docs works. Web Google drive works. Note: Google support info - even I if pass dozen Google URL's without bump - cannot help. It doesn't work when server-first bumping is on and works othervise. So, the Serious Question is: Why? :) Any idea? ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUorRuAAoJENNXIZxhPexGRPEH/0Vlh/kZINRlo3IaDe/9UYSW M0oaXQemB2Wg/wvNMdZ8SA3F4dUJUuHgS/y7FXcCgP/KgWnudsoJ7oPiHEVNPzt3 L8K7rNPy3d/c/+baXilh4/xErp/mAOKU/mLBqd0GQYQ2N7bAsWpsWqt7/dTGxWkU kLVgFJr9JblxVdABAZ7JTooye3bLskdrAB/865vZOyQcveozW6d4TKZwaEGFrq/d b/3Mki4T6YLMG248jVN+43W2us6Z598geDLn8aJN+zb/s6TBEzxy1d5tUROM4a2A 1rE7B92o+9leZi+JdQAGX4l7Um1WVmrnih52w+Pxz/PR/k7Hz+fCcQBlUtsqvMk= =Lf+I -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
Hello Yuri, Luckily the same topic was just discussed on our forum – please see if this can help https://groups.google.com/d/msg/quintolabs-content-security-for-squid-proxy/GKIV3FpYSBE/9IET-4hg_tEJ It describes the iptables settings for successful SSL bump exclusions for Dropbox clients / Google Drive / iTunes (bypassing SSL Bump because of SSL Pinning). Best regards, Raf From: squid-users [mailto:squid-users-boun...@lists.squid-cache.org] On Behalf Of Rafael Akchurin Sent: Tuesday, December 30, 2014 4:23 PM To: Yuri Voinov; squid-users@lists.squid-cache.org Subject: Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect Only exclusion from SSL Bump as far as I know. raf From: Yuri Voinov yvoi...@gmail.commailto:yvoi...@gmail.com Sent: Tuesday, December 30, 2014 3:19 PM To: Rafael Akchurin; squid-users@lists.squid-cache.orgmailto:squid-users@lists.squid-cache.org Subject: Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 May be. Does workaround exists? 30.12.2014 20:09, Rafael Akchurin ?: SSL Pinning? (I know Dropbox does this) my two cents only :) Raf From: squid-users mailto:squid-users-boun...@lists.squid-cache.org squid-users-boun...@lists.squid-cache.orgmailto:squid-users-boun...@lists.squid-cache.org on behalf of Yuri Voinov mailto:yvoi...@gmail.com yvoi...@gmail.commailto:yvoi...@gmail.com Sent: Tuesday, December 30, 2014 2:12 PM To: mailto:squid-users@lists.squid-cache.org squid-users@lists.squid-cache.orgmailto:squid-users@lists.squid-cache.org Subject: [squid-users] Squid 3 SSL bump: Google drive application could not connect Hi gents, I found strange issue. Squid 3.4.10. Intercept. HTTPS bumping. All works fine. All configs correct. Whenever all web https sites works perfectly - especially in Chrome, most cloud clients works like charm (SpiderOak is!), Google Drive client application (PC) could not work. Note: Web Google Docs works. Web Google drive works. Note: Google support info - even I if pass dozen Google URL's without bump - cannot help. It doesn't work when server-first bumping is on and works othervise. So, the Serious Question is: Why? :) Any idea? ___ squid-users mailing list mailto:squid-users@lists.squid-cache.org squid-users@lists.squid-cache.orgmailto:squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUorRuAAoJENNXIZxhPexGRPEH/0Vlh/kZINRlo3IaDe/9UYSW M0oaXQemB2Wg/wvNMdZ8SA3F4dUJUuHgS/y7FXcCgP/KgWnudsoJ7oPiHEVNPzt3 L8K7rNPy3d/c/+baXilh4/xErp/mAOKU/mLBqd0GQYQ2N7bAsWpsWqt7/dTGxWkU kLVgFJr9JblxVdABAZ7JTooye3bLskdrAB/865vZOyQcveozW6d4TKZwaEGFrq/d b/3Mki4T6YLMG248jVN+43W2us6Z598geDLn8aJN+zb/s6TBEzxy1d5tUROM4a2A 1rE7B92o+9leZi+JdQAGX4l7Um1WVmrnih52w+Pxz/PR/k7Hz+fCcQBlUtsqvMk= =Lf+I -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Already found this lonely right post ;) I have Google-Fu too :) And it longer than you :) Anyway, all of these issues solved. I have snoop (not Windoze wireshark - all great things makes in console, ya!) and take a look on single client traffic during bumping. As I haven't iptables (no penguins, please!), but I have Cisco 2911, I pass some Windows Update, Symantec Update (which is not work too) bypassing Squid. Cisco is greatest. All others are probably suxx :) The complete solution looks like: access-list 121 remark ACL for HTTPS WCCP access-list 121 remark Squid proxies bypass access-list 121 deny ip host 192.168.200.3 any access-list 121 remark WU bypass access-list 121 deny tcp any 191.232.0.0 0.7.255.255 access-list 121 deny tcp any 65.52.0.0 0.3.255.255 access-list 121 remark Symantec bypass access-list 121 deny tcp any host 195.215.221.99 access-list 121 deny tcp any host 195.215.221.104 access-list 121 deny tcp any host 213.248.114.172 access-list 121 deny tcp any host 213.248.114.173 access-list 121 deny tcp any host 213.248.114.174 access-list 121 deny tcp any host 213.248.114.175 access-list 121 deny tcp any host 77.67.22.168 access-list 121 deny tcp any host 77.67.22.171 access-list 121 deny tcp any host 77.67.22.173 access-list 121 deny tcp any host 213.248.114.171 access-list 121 remark LAN clients proxy port 443 access-list 121 permit tcp 192.168.0.0 0.0.255.255 any eq 443 access-list 121 remark all others bypass WCCP access-list 121 deny ip any any So, all others issue solves similar. Want to do something good - do it yourself! That's the way. :) 30.12.2014 23:39, Rafael Akchurin пишет: Hello Yuri, Luckily the same topic was just discussed on our forum – please see if this can help https://groups.google.com/d/msg/quintolabs-content-security-for-squid-proxy/GKIV3FpYSBE/9IET-4hg_tEJ It describes the iptables settings for successful SSL bump exclusions for Dropbox clients / Google Drive / iTunes (bypassing SSL Bump because of SSL Pinning). Best regards, Raf *From:*squid-users [mailto:squid-users-boun...@lists.squid-cache.org] *On Behalf Of *Rafael Akchurin *Sent:* Tuesday, December 30, 2014 4:23 PM *To:* Yuri Voinov; squid-users@lists.squid-cache.org *Subject:* Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect Only exclusion from SSL Bump as far as I know. raf - *From:*Yuri Voinov yvoi...@gmail.com mailto:yvoi...@gmail.com *Sent:* Tuesday, December 30, 2014 3:19 PM *To:* Rafael Akchurin; squid-users@lists.squid-cache.org mailto:squid-users@lists.squid-cache.org *Subject:* Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect May be. Does workaround exists? 30.12.2014 20:09, Rafael Akchurin ?: SSL Pinning? (I know Dropbox does this) my two cents only :) Raf From: squid-users mailto:squid-users-boun...@lists.squid-cache.org squid-users-boun...@lists.squid-cache.org mailto:squid-users-boun...@lists.squid-cache.orgon behalf of Yuri Voinov mailto:yvoi...@gmail.com yvoi...@gmail.com mailto:yvoi...@gmail.com Sent: Tuesday, December 30, 2014 2:12 PM To: mailto:squid-users@lists.squid-cache.org squid-users@lists.squid-cache.org mailto:squid-users@lists.squid-cache.org Subject: [squid-users] Squid 3 SSL bump: Google drive application could not connect Hi gents, I found strange issue. Squid 3.4.10. Intercept. HTTPS bumping. All works fine. All configs correct. Whenever all web https sites works perfectly - especially in Chrome, most cloud clients works like charm (SpiderOak is!), Google Drive client application (PC) could not work. Note: Web Google Docs works. Web Google drive works. Note: Google support info - even I if pass dozen Google URL's without bump - cannot help. It doesn't work when server-first bumping is on and works othervise. So, the Serious Question is: Why? :) Any idea? ___ squid-users mailing list mailto:squid-users@lists.squid-cache.org squid-users@lists.squid-cache.org mailto:squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUowFgAAoJENNXIZxhPexGHxkIAM2mb+OjhevZWpgdwiKHP2E0 D+8UM6/c7OZcJ2uSjIWN7DG0h+b86/ATul+9S+mZHl1DLBYpGUKW9J5I3iIQb+sr 5xR2ReFkuFeSpZASXex2yq5lfmACPdiUzI9iVhe7DPJqKJNiIzvHLq4ZRnjJN4Ih 0u0NGuPKfkkWFJ/SmXAceEdS7sT/lT0cVm1JgpurVzipelBUNbLQUd0yKrpbIz2x ia7gwu3ZFi2aY2DvrfP7ntkoZpLl+SyDI/PkFIEaAr2+KaMcTbUXVQcVTZ7S6eLu pgCNil0x8AFApWSIg+P68DcFcIS/nUIvNqXjuvr0ikqGwLEAqvueM6LPKifsdSg= =J+Cs -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sure. Squid 3 WCCP key config part: # WCCPv2 parameters wccp2_router 192.168.200.2 wccp2_forwarding_method l2 wccp2_return_method l2 wccp2_service standard 0 wccp2_rebuild_wait off wccp2_service standard 0 wccp2_service dynamic 70 wccp2_service_info 70 protocol=tcp flags=dst_ip_hash,src_ip_alt_hash,src_port_alt_hash priority=240 ports=443 Cisco config key parts: ! ip wccp web-cache redirect-list 120 ip wccp 70 redirect-list 121 ! ! ! This interface look to Squid proxy (internal networks on another interface) interface GigabitEthernet0/1 ip address 192.168.200.2 255.255.255.0 ip wccp web-cache redirect out ip wccp 70 redirect out ip nbar protocol-discovery ip virtual-reassembly in duplex auto speed auto ! access-list 120 remark ACL for HTTP WCCP access-list 120 remark Squid proxies bypass WCCP access-list 120 deny ip host 192.168.200.3 any access-list 120 remark LAN clients proxy port 80 access-list 120 permit tcp 192.168.0.0 0.0.255.255 any eq www access-list 120 remark all others bypass WCCP access-list 120 deny ip any any ! access-list 121 remark ACL for HTTPS WCCP access-list 121 remark Squid proxies bypass access-list 121 deny ip host 192.168.200.3 any access-list 121 remark LAN clients proxy port 443 access-list 121 permit tcp 192.168.0.0 0.0.255.255 any eq 443 access-list 121 remark all others bypass WCCP access-list 121 deny ip any any ! That's all. :) 31.12.2014 2:10, Rafael Akchurin пишет: Glad that it worked. May be useful to dump here your squid.conf to better understand how to configure squid to transparently work with wccp traffic coming from your Cisco router? Raf *From:*Yuri Voinov [mailto:yvoi...@gmail.com] *Sent:* Tuesday, December 30, 2014 8:48 PM *To:* Rafael Akchurin; squid-users@lists.squid-cache.org *Subject:* Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect Already found this lonely right post ;) I have Google-Fu too :) And it longer than you :) Anyway, all of these issues solved. I have snoop (not Windoze wireshark - all great things makes in console, ya!) and take a look on single client traffic during bumping. As I haven't iptables (no penguins, please!), but I have Cisco 2911, I pass some Windows Update, Symantec Update (which is not work too) bypassing Squid. Cisco is greatest. All others are probably suxx :) The complete solution looks like: access-list 121 remark ACL for HTTPS WCCP access-list 121 remark Squid proxies bypass access-list 121 deny ip host 192.168.200.3 any access-list 121 remark WU bypass access-list 121 deny tcp any 191.232.0.0 0.7.255.255 access-list 121 deny tcp any 65.52.0.0 0.3.255.255 access-list 121 remark Symantec bypass access-list 121 deny tcp any host 195.215.221.99 access-list 121 deny tcp any host 195.215.221.104 access-list 121 deny tcp any host 213.248.114.172 access-list 121 deny tcp any host 213.248.114.173 access-list 121 deny tcp any host 213.248.114.174 access-list 121 deny tcp any host 213.248.114.175 access-list 121 deny tcp any host 77.67.22.168 access-list 121 deny tcp any host 77.67.22.171 access-list 121 deny tcp any host 77.67.22.173 access-list 121 deny tcp any host 213.248.114.171 access-list 121 remark LAN clients proxy port 443 access-list 121 permit tcp 192.168.0.0 0.0.255.255 any eq 443 access-list 121 remark all others bypass WCCP access-list 121 deny ip any any So, all others issue solves similar. Want to do something good - do it yourself! That's the way. :) 30.12.2014 23:39, Rafael Akchurin пишет: Hello Yuri, Luckily the same topic was just discussed on our forum – please see if this can help https://groups.google.com/d/msg/quintolabs-content-security-for-squid-proxy/GKIV3FpYSBE/9IET-4hg_tEJ It describes the iptables settings for successful SSL bump exclusions for Dropbox clients / Google Drive / iTunes (bypassing SSL Bump because of SSL Pinning). Best regards, Raf *From:*squid-users [mailto:squid-users-boun...@lists.squid-cache.org] *On Behalf Of *Rafael Akchurin *Sent:* Tuesday, December 30, 2014 4:23 PM *To:* Yuri Voinov; squid-users@lists.squid-cache.org mailto:squid-users@lists.squid-cache.org *Subject:* Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect Only exclusion from SSL Bump as far as I know. raf - *From:*Yuri Voinov yvoi...@gmail.com mailto:yvoi...@gmail.com mailto:yvoi...@gmail.com mailto:yvoi...@gmail.com *Sent:* Tuesday, December 30, 2014 3:19 PM *To:* Rafael Akchurin; squid-users@lists.squid-cache.org mailto:squid-users@lists.squid-cache.org mailto:squid-users@lists.squid-cache.org mailto:squid-users@lists.squid-cache.org *Subject:* Re: [squid
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
Perfect thanks a lot!!! Raf :) From: Yuri Voinov [mailto:yvoi...@gmail.com] Sent: Tuesday, December 30, 2014 9:23 PM To: Rafael Akchurin; squid-users@lists.squid-cache.org Subject: Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 WCCP only, of course. To reduce Cisco CPU usage. Also, iOS version 15.4 with SECURITYK9 techno pack activated. 31.12.2014 2:21, Rafael Akchurin пишет: Just for me to completely clarify: - how exactly your Squid gets the traffic from your clients? (explicit proxy or cisco WCCP?) raf *From:*Yuri Voinov [mailto:yvoi...@gmail.com] *Sent:* Tuesday, December 30, 2014 9:16 PM *To:* Rafael Akchurin; squid-users@lists.squid-cache.orgmailto:squid-users@lists.squid-cache.org *Subject:* Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect To finalize a solution, see the our favorite: http://www.squid-cache.org/mail-archive/squid-users/201406/0369.html Why use iptables, ipfilter,Cisco, etc?! Only Squid, only hardcore! Revert cisco config back: R2911(config)#no access-list 121 R2911(config)#access-list 121 remark ACL for HTTPS WCCP R2911(config)#access-list 121 remark Squid proxies bypass R2911(config)#access-list 121 deny ip host 192.168.200.3 any R2911(config)#access-list 121 deny ip host 192.168.100.251 any R2911(config)#access-list 121 remark Videoserver R2911(config)#access-list 121 deny ip host 192.168.200.5 any R2911(config)#access-list 121 remark LAN clients proxy port 443 R2911(config)#access-list 121 permit tcp 192.168.0.0 0.0.255.255 any eq 443 R2911(config)#access-list 121 remark all others bypass WCCP R2911(config)#access-list 121 deny ip any any R2911(config)#^Z R2911#wr Building configuration... [OK] Write acl file with IP/net with SSL Pinning: root @ ktulhu /usr/local/squid/etc # cat dst.nobump # BCC bypass 91.198.63.0/24 # Salyk bypass 212.154.165.148/32 # WU bypass 191.232.0.0/13 65.52.0.0/14 # Symantec bypass 195.215.221.99/32 195.215.221.104/32 213.248.114.172/32 213.248.114.173/32 213.248.114.174/32 213.248.114.175/32 77.67.22.168/32 77.67.22.171/32 77.67.22.173/32 213.248.114.171/32 Add needful nets/apps to acl by your taste. Add to squid config: # SSL bump acl acl net_bump src /usr/local/squid/etc/net.bump # HTTP-use 443 port apps acl url_nobump dstdom_regex \.icq\.* # SSL Pinning servers. Only ip-based dst acl! acl dst_nobump dst /usr/local/squid/etc/dst.nobump # SSL bump rules sslproxy_cert_error allow all ssl_bump none localhost ssl_bump none url_nobump ssl_bump none dst_nobump ssl_bump server-first net_bump Yahooo! The same result with Squid only! 30.12.2014 23:39, Rafael Akchurin пишет: SSL Pinning -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJUowmnAAoJENNXIZxhPexGEtwH/10nuDG9+Z7AG2W+nh64X7JV 5JmvvaC778yUYnMUaPJTLPK3hxVuQshVMaE2x4jhuxBEkhtKPWBJZg8JFLFinzf5 nDINk8zz0j4fLCXmDAJaXz2NMacUviCiKFY8k63SumxKeTIBU20DuLk9glggTpfY 3RgdNWfvmma9iv8QW/s2UJFbRdJS0cLjra4XFFQBZLyGEJPTOcft3slWX3QgHVCD SB3CZWy2gwbLVphiCiG91HxBtUUUzSLqPc60RdSwOCoSOaBMHZgy8yjZ8VRgQkyi uz41hhp1mCMfssNjoLdCvr/AxJG990yQ24MiCDuzN9fYVNzUPdXF+q4E5G/avtk= =FkuL -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect
On Dec 30, 2014 7:04 PM, Amos Jeffries squ...@treenet.co.nz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 31/12/2014 6:30 a.m., shawn wilson wrote: On Dec 30, 2014 8:57 AM, Amos Jeffries wrote: As bumping gets more popular we are hearing about a number of services abusing port 443 for non-HTTPS protocols on the false assumption that the TLS layer goes all the way to the origin server without inspection. That has never been a true assumption, CDN frontends have always decrypted. OT but you use 443 because people expect it to be encrypted web data and don't block it. And DPI doesn't tell you anything more. web is no longer just HTTP and that is part of the problem. People treating port 443 as if any of the web protocols can use it just by being wrapped in TLS. Worse than that - I'm mainly thinking ssh (which won't survive DPI). Port 443 is specifically registered for HTTP over TLS (aka HTTPS). Web includes HTTP, but also includes protocols like RSS, WebSockets, SPDY, QUIC, COAP, even IRC and Jabber at times. The other non-HTTP protocols have other non-443 ports registered or available for their use. Some like SMTP even switch their main port between encrypted and non-encrypted as needed. I know it can be hard to get unusual ports opened past firewalls, but that is not being helped by everything using only a handful of ports. [I have a long rant at this point about lazy corporates, but its 2015 in a few hrs so I'll drop it for now]. My point isn't even about lazy corporates but this: how many airlines will block ssh over port 22 and how many will block it over 443? (And if that doesn't work OpenVPN on 443 and ssh through that) I assume Google thought along similar lines when they talked about which port to put their binary Drive data on. You want people to stop using 443 for non-https traffic, get people to stop blocking the other ssl ports. This is OT but here's the topical point - if you're going to bump http+ssl traffic, you need to know that due to some people blocking alternative ports for secure services, you'll always see non-http traffic here. The IETF might give you a port but only smart long term business decisions will allow you to keep it - that's far past over for 443/tcp at this point I think :/ ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
