[squid-users] QUIC - squid tproxy with c-icap content filtering server

[Ben Goz]

By the help of God.

My squid machine is configured using tproxy and c-icap content filtering
server.
the http and https traffic redirected with iptables rules to squid ports.

What additional configurations (on squid or iptables rule) should be done
to support QUIC protocol so I can redirect the web content received to
c-icap server?

Thanks,
Ben
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] make URL bypass squid proxy

[ngtech1ltd]

Hey Rob,

it’s a great question.
I will assume you are using squid 5.x since it’s the stable one.
There is a configuration reference documentation at:
http://www.squid-cache.org/Versions/v5/cfgman/

And the relevant one is ‘acl’:
http://www.squid-cache.org/Versions/v5/cfgman/acl.html

In the config we the next options:
* dstdom_regex
* dstdomain
* ssl::server_name
* ssl::server_name_regex

Each one of them is different in two things.
The dstdomain is happening when the http request verification is done, usually 
in the http_access part.
So for a CONNECT request in your case it should work but will only work for 
http_access rules ie allow or deny the connection to being proxied by the proxy.
All this happens before the bump stage.
The dstdom_regex is doing the same but with another "lookup" mechanism.
Just to illustrate we will use the domain www.example.com.

With dstdomain .example.com which is a wildcard domain the www.example .com 
will match since what it does it..
Take apart the .example.com into .com and .example and example (logically, not 
the code actually does..)
Then when a request for www.example-2.com arrives the dstdomain acl checks the 
strings one by one ie:
com
example-2
www

So the "com" would match and since it has a dot "." as a prefix it continues to 
the next part of the domain name ie:
example-2

and it will try to match it to "example" from the ".example.com'.
then it will fail and there for it will declare the specific definition in the 
acl as "do not match.
But for www.example .com it would be different:

Squid will find that the .com is in the "com" part of www.example.com and will 
continue to the next part ie
"example" and it will match.
Then it will see the dot "." which means that all subdomains are a part of this 
rule ie both example.com and any other domain
under .example.com will match so... www.example.com and www2.example.com and 
w.example.com will match and this is compared to
example.com

with dstdomain example.com it will only match a full match of example.com but 
not www.example.com and it's pointless to define both:
- example.com
- .example.com

In the same acl file.
Squid will compare it to the basic proxy url request and the Host: header 
inside of it.
(Amos might remember which one of these wins or which one is the main one)

So that's how a dstdomain test works.
The dstdom_regex is applied on the same "part" of the proxy request ie request 
url and Host header but...
Tries to match it with a regex that is a much more "CPU" intensive but can do 
magic if you know how to use it right.
The pros are that you can use one regex to match about 100 domains however 
dstdomain is much faster in many cases.
To test regex in general I love:
https://rubular.com/

and it's very simple to put a line with the url or the domain itself and write 
the pattern and then get instant result.
There are many sites which does this exact same thing.
You need: activate.redshift3d.com  for the test.

For the server_name dstdom and regex the difference is on what part of the 
"known" information on the request done.
Ie it's not on the known plain text proxy request but a "peek" into the client 
or server TLS part of the connection bumping.
It's possible on TLS 1.2 with ease but 1.3 with encryption it's a whole other 
story.
(Alex and his team is more involved in this part then me)

It's a bit more complex in the server_name part since from what I remember the 
server_name can be the SNI itself or parts of the
certificate of the server.
In the certificate of the server in many cases there is no real domain but a 
catch all ie multidomain which is for example:
*.example.com

The ssl::server_name is like dstdomain so .example.com should match both SNI 
with www.example.com and the *.example.com
If it doesn't then it’s a bug..

About the ssl::server_name_regex you will need to test and match the regex 
against couple possibilities that are in the certificate ie:
*.example.com
activation.example.com

etc...

The current certificate I see is:
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
c0:71:02:fc:1f:e5:64:5b:0d:d6:ae:c8:1a:17:e6:80
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = GB, ST = Greater Manchester, L = Salford, O = Sectigo 
Limited, CN = Sectigo RSA Domain Validation Secure Server CA
Validity
Not Before: Dec 14 00:00:00 2022 GMT
Not After : Jan 12 23:59:59 2024 GMT
Subject: CN = activate.redshift3d.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (2048 bit)
Modulus:
00:cd:f9:bd:58:a4:7a:16:3f:23:12:9d:e0:1e:39:
a0:ef:66:ae:b3:ae:32:5c:69:9e:cc:89:83:fb:a2:
e0:c7:9d:86:aa:e6:9b:b3:d9:0c:bb:35:36:2f:cf:
be:ec:1e:62:ca:d8:ab:16:66:6e:00:8d:f7:42:3d:
5b:d6:bf:a0:30:a1:c2:6f:5e:69:76:ef:0c:38:73:

Re: [squid-users] make URL bypass squid proxy

[robert k Wild]

very clever, so you bunch all the acls up

acl NoSSLInterceptAnyOf any-of NoSSLInterceptDstDom
NoSSLInterceptDstDomFile NoSSLInterceptRegEx NoSSLInterceptRegExFile

the key word is "any-of" ie if the url hits any one do that first

what about instead of making it

ssl::server_name_regex

i make it

*dstdom_regex*

On Thu, 29 Jun 2023 at 01:38,  wrote:

> Hey Rob,
>
> The first thing is to allow the domain in the http_acces just to be sure
> and use a basic deny all bottom line.
> Let me try to simplify your squid.conf
> In a link:
> https://gist.github.com/elico/b49f4a28d4b5db5ba882b10d40872d5e
>
> In plain text:
> ## START OF FILE
> # SSL Interception  basic rules
> acl DiscoverSNIHost at_step SslBump1
>
> acl NoSSLInterceptRegEx ssl::server_name_regex (^|.*\.)redshift3d\.com$
> acl NoSSLInterceptRegExFile ssl::server_name_regex
> "/usr/local/squid/etc/no-intercept-ssl-regex.txt"
>
> acl NoSSLInterceptDstDom ssl::server_name .redshift3d.com
> acl NoSSLInterceptDstDomFile ssl::server_name
> "/usr/local/squid/etc/no-intercept-ssl-dstdom.txt"
>
> ## Any of will test what ever rule match first in a first match/hit fasion
> acl NoSSLInterceptAnyOf any-of NoSSLInterceptDstDom
> NoSSLInterceptDstDomFile NoSSLInterceptRegEx NoSSLInterceptRegExFile
>
> ssl_bump peek DiscoverSNIHost
> ssl_bump splice NoSSLInterceptAnyOf
> ssl_bump bump all
>
> #SSL Bump port
> http_port 3128 ssl-bump cert=/usr/local/squid/etc/ssl_cert/myCA.pem
> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s
> /var/lib/ssl_db -M 4MB
>
> ## http_access acls, will apply on incomming requests and not on responses
> acl special_url_regex url_regex https?://(^|.*\.)redshift3d\.com\/
> acl special_url_regex_file url_regex
> "/usr/local/squid/etc/special_url_regex.txt"
>
> acl special_url_dst_dom dstdomain .redshift3d.com
> acl special_url_dst_dom_file dstdomain
> "/usr/local/squid/etc/special_url_dstdom.txt"
>
> acl special_url_any_of any-of special_url_dst_dom special_url_dst_dom_file
> special_url_regex special_url_regex_file
>
> acl localnet src 192.168.0.0/16
> acl localnet src 10.0.0.0/8
>
> http_access allow localnet special_url_any_of
> http_access deny all
> ## END OF FILE
>
>  Once the above will work try to add other http_access rule like reply
> access rules
>
> Let me know what happens,
> Eliezer
>
> From: robert k Wild 
> Sent: Tuesday, June 27, 2023 09:36
> To: ngtech1...@gmail.com
> Cc: Squid Users 
> Subject: Re: [squid-users] make URL bypass squid proxy
>
> Hi Eliezer,
>
> this is a snippet of my whitelist and no intercept SSL config
>
> #SSL Interception
> acl DiscoverSNIHost at_step SslBump1
> acl NoSSLIntercept ssl::server_name_regex
> "/usr/local/squid/etc/interceptssl.txt"
> ssl_bump peek DiscoverSNIHost
> ssl_bump splice NoSSLIntercept
> ssl_bump bump all
> #
> #SSL Bump
> http_port 3128 ssl-bump cert=/usr/local/squid/etc/ssl_cert/myCA.pem
> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s
> /var/lib/ssl_db -M 4MB
> #
> #deny up MIME types
> acl upmime req_mime_type "/usr/local/squid/etc/mimedeny.txt"
> #
> #deny URL links
> acl url_links url_regex "/usr/local/squid/etc/linksurl.txt"
> #
> #allow special URL paths
> acl special_url url_regex "/usr/local/squid/etc/urlspecial.txt"
> #
> #deny down MIME types
> acl downmime rep_mime_type "/usr/local/squid/etc/mimedeny.txt"
> #
> http_reply_access allow special_url
> http_reply_access deny downmime
> #http_access deny upmime
> #http_access deny url_links
> #
> #HTTP_HTTPS whitelist websites
> acl whitelist ssl::server_name_regex "/usr/local/squid/etc/urlwhite.txt"
> #
> http_access allow activation whitelist
> http_access deny all
>
> so basically no SSL interception
>
> #SSL Interception
> acl DiscoverSNIHost at_step SslBump1
> acl NoSSLIntercept ssl::server_name_regex
> "/usr/local/squid/etc/interceptssl.txt"
> ssl_bump peek DiscoverSNIHost
> ssl_bump splice NoSSLIntercept
> ssl_bump bump all
>
> and whitelisting
>
> #HTTP_HTTPS whitelist websites
> acl whitelist ssl::server_name_regex "/usr/local/squid/etc/urlwhite.txt"
>
> in both txt files ie
>
> /usr/local/squid/etc/interceptssl.txt
> /usr/local/squid/etc/urlwhite.txt
>
> i have a URL that first i have to whitelist and then if i want squid not
> to inspect the url traffic i put it in the SSL interception (i do this as
> some websites dont like MITM )
>
> but even putting the URL in question in both files im still having issues
> with this website ie its still being detected that its passing through a
> proxy
>
> thanks,
> rob
>
> On Mon, 26 Jun 2023 at 23:35,  wrote:
> Hey Robert,
>
> I am not sure what forward proxy setup you have there.
> A simple forward proxy?
> What tool are you using for whitelisting?
> You can use an external acl helper to allow dynamic updates of the
> whitelists or
> to periodic update your lists and reload.
> It