Re: [squid-users] using clang to compile squid 4-5

[graf huy]

 Hi,
Using update-alternatives to set clang++ was a success, and export 
CC=/usr/bin/clang does the job (trying two differents methods). Debian project 
recommended to use export (CC,CCX) for versioning and only update-alternatives 
for alternative as it says.Anyway compilation with clang was as quick as gcc 
can do.
/usr/local/squid/sbin# ./squid -v
Squid Cache: Version 4.5
Service Name: squid

This binary uses OpenSSL 1.1.1a  20 Nov 2018. For legal restrictions on 
distribution see https://www.openssl.org/source/license.html

configure options:  '--enable-ssl' '--enable-ssl-crtd' '--localstatedir=/var' 
'--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' 
'--libexecdir=/usr/lib/squid' '--mandir=/usr/share/man' '--enable-inline' 
'--disable-arch-native' '--enable-async-io=8' 
'--enable-storeio=ufs,aufs,diskd,rock' '--enable-removal-policies=lru,heap' 
'--enable-delay-pools' '--enable-cache-digests' '--enable-icap-client' 
'--enable-follow-x-forwarded-for' 
'--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB' 
'--enable-auth-digest=file,LDAP' '--enable-auth-negotiate=kerberos,wrapper' 
'--enable-auth-ntlm=fake,SMB_LM' 
'--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,time_quota,unix_group,wbinfo_group'
 '--enable-security-cert-validators=fake' 
'--enable-storeid-rewrite-helpers=file' '--enable-url-rewrite-helpers=fake' 
'--enable-eui' '--enable-esi' '--enable-icmp' '--enable-zph-qos' 
'--enable-ecap' '--disable-translation' '--with-swapdir=/var/spool/squid' 
'--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid' 
'--with-filedescriptors=65536' '--with-large-files' '--with-default-user=proxy' 
'--with-gnutls' '--with-openssl' 'CC=/usr/bin/clang'



>Seconded. With both my Squid Project and Debian pkg-squid Team hat's on
>I'd like to know why you feel any need to force the compiler?
There were errors to patch with memcpy because of the gcc version  8.2.0-14 
Debian wich considers some warnings as errors, besides some targets are BSD's.
smblib-util.c: In function ‘SMB_Negotiate’:
smblib-util.c:404:9: error: ‘strncpy’ output may be truncated copying 79 bytes 
from a string of length 79 [-Werror=stringop-truncation]
 strncpy(p, Con_Handle -> Svr_PDom, sizeof(Con_Handle -> Svr_PDom) - 1);
 ^~
smblib-util.c:427:9: error: ‘strncpy’ output may be truncated copying 79 bytes 
from a string of length 79 [-Werror=stringop-truncation]
 strncpy(p, Con_Handle -> Svr_PDom, sizeof(Con_Handle -> Svr_PDom) - 1);
 ^~
smblib-util.c: In function ‘SMB_TreeConnect’:
smblib-util.c:541:5: error: ‘strncpy’ specified bound 129 equals destination 
size [-Werror=stringop-truncation]
 strncpy(tree -> path, path, sizeof(tree -> path));
 ^
smblib-util.c:542:5: error: ‘strncpy’ specified bound 20 equals destination 
size [-Werror=stringop-truncation]
 strncpy(tree -> device_type, device, sizeof(tree -> device_type));
 ^

Although my heart bleeds to use a compiler not compliant with GPL on Debian.
Great to known you are on this package, I thought there was only Luigi 
Gangitano (lu...@debian.org) for ages...

Best regards.


Le mercredi 23 janvier 2019 à 05:25:04 UTC+1, Amos Jeffries 
 a écrit :  
 
 On 23/01/19 5:17 am, Alex Rousskov wrote:
> On 1/22/19 6:21 AM, graf huy wrote:
> 
>> The Makefile is modified so each line with gcc is replaced with clang
>> and each line of g++ replaced with clang++. But gcc is still used.
> 
> I am not sure you are doing that,

Seconded. With both my Squid Project and Debian pkg-squid Team hat's on
I'd like to know why you feel any need to force the compiler?


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
  ___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid not coming up with dynamic host certificate on ssl bum

[Amos Jeffries]

On 30/01/19 8:11 pm, bandeep2000 wrote:
> Have squid in transparent, want to ssl bump all the connections which
> are not whitelisted, but when given *generate-host-certificates=on ,
> *squid keeps crashing when trying to bring it up after service restart.
> 
> 
> 
> */var/log/messages*
> 
> Jan 30 07:05:52 ban-squid-proxy22 squid[23323]: Squid Parent: (squid-1)
> process 23441 started
> 
> Jan 30 07:05:52 ban-squid-proxy22 (squid-1): The ssl_crtd helpers are
> crashing too rapidly, need help!
> 

There is the error. cache.log should contain more details and possibly
instructions about the error.

Probably you did not initialize the certificate database or it needs to
be purged and reinitialized.




> 
> *squid.conf details:*
> *
> *
> 
> visible_hostname squid
> 


This name will clash with any other proxy calling itself "squid" and the
traffic may/will be rejected as forwarding loop.

Using the full hostname or FQDN is best to ensure the URLs of objects
provided direct from Squid to clients can be fetched by those clients.


> 
> cache deny all
> 
> #Handling HTTP requests
> 
> http_port 3128 intercept
> 
> acl allowed_http_sites dstdomain ...
> acl blacklist url_regex -i /.(.*?)

The above is equivalent to:

  acl blacklist url_regex /.

Meaning "blacklist" matches any URI containing a '/' character followed
by one other character...

> 
> 
> http_access allow allowed_http_sites
> 
> http_access deny blacklist
> 


... all URLs start with "scheme://" therefore the first '/' always
exists and is always followed by the second '/'.

... So any traffic with a URL is blacklisted.

The only traffic allowed is that on the whitelist or with URI or URN -
the latter two do not require the '//' scheme delimiters. So they
usually will not match, but may do so.


> 
> #Handling HTTPS requests
> 
> #https_port 3130 cert=/etc/pki/tls/certs/squidCA.pem ssl-bump intercept
> 
> #/root/openssl/squid.crt  squid.csr  /root/openssl/squid.key
> 
> *https_port 3130 cert=/root/openssl/squid.crt
> key=/root/openssl/squid.key ssl-bump intercept
> generate-host-certificates=on version=1
> options=NO_SSLv2,NO_SSLv3,SINGLE_DH_USE*
> 

Please put the traffic mode ("intercept" for these) as the first option
after the port number. The Squid "-k parse" checks can only verify
correct operation and help instructions if the mode is known when the
other options are interpreted.



> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
> 
> 
> acl SSL_port port 443
> 
> http_access allow SSL_port
> 

So any attempt to open opaque tunnels (uses a URI not a URL) to port 443
to any domain is allowed by any client who can get TCP connections to
reach your proxy port 3128.
 Also to any server in the allowed_https_sites whitelist regardless of
whether the client is your LAN or an external attacker.

(NP: there are good reasons we recommend the default !Safe_ports and
"CONNECT !SSL_ports" ACL checks as to be used firs and your rules
second. Mostly it is about protecting your network from abusers.)


> acl allowed_https_sites ssl::server_name ...
> 
> 
> acl step1 at_step SslBump1
> 
> acl step2 at_step SslBump2
> 
> acl step3 at_step SslBump3
> 
> 
> ssl_bump peek step1 all
> 
> #ssl_bump peek all
> 
> ssl_bump splice step2 allowed_https_sites
> 
> ssl_bump splice step3 allowed_https_sites
> 

No traffic should ever reach the step3. Since step2 always finishes with
the above splice or the below bump actions. There are no other
possibilities at step2 which would ever lead to step3 (server
credentials) being checked.


> ssl_bump bump step2 all
> 

Note: the "all" ACL is always pointless on ssl_bump lines and seems to
often confuse people into thinking that a line matches all traffic (it
does nothing). I recommend removing those and re-checking the rules
against your understanding of what your policy needs to make happen.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] using clang to compile squid 4-5

[Amos Jeffries]

On 30/01/19 9:38 pm, graf huy wrote:
> Hi,
> 
> Using update-alternatives to set clang++ was a success, and export
> CC=/usr/bin/clang does the job (trying two differents methods). Debian
> project recommended to use export (CC,CCX) for versioning and only
> update-alternatives for alternative as it says.
> Anyway compilation with clang was as quick as gcc can do.
> 

Great to know. Nothing to fix on that front for us then.

> 
>>Seconded. With both my Squid Project and Debian pkg-squid Team hat's on
>>I'd like to know why you feel any need to force the compiler?
> 
> There were errors to patch with memcpy because of the gcc version 
> 8.2.0-14 Debian wich considers some warnings as errors, besides some
> targets are BSD's.
> 

I'm not sure I follow that. Are you building with patched GCC? or a
patched libc? or something else?

I build with the Debian Sid GCC version myself and have not seen these
in any test builds with that or similar GCC versions. Am trying your
build options now just to be sure.


On the matter of BSD's, since they provide clang as the default compiler
they (FreeBSD at least) build with clan/clang++ without any need for
CC/CXX setting. We have Jenkins tests for that.


> 
> Great to known you are on this package, I thought there was only Luigi
> Gangitano (lu...@debian.org) for ages...
> 

Welcome. There are several of us, with Luigi as team lead.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] daily releases

[Vieri]

Hi,

Does anyone know of a convenient one-liner to get the latest daily release 
tarball, eg. 
http://www.squid-cache.org/Versions/v4/squid-4.5-20190128-r568e66b7c.tar.gz, 
without having to search for it manually on the web?

Either that or a symlink that would always point to the "latest daily".

Thanks,

Vieri

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] daily releases

[L . P . H . van Belle]

Hai, 
 
I use this for the latest 4.xx release
mkdir squid && curl -q -L $(lynx -listonly -nonumbers -dump 
http://www.squid-cache.org/Versions/v4/  | grep squid-4.5 | grep ".tar.gz") | 
tar -xz -C squid  --strip-components 1 -f -  
 
and this one for the daily
mkdir squid-$(date +%F) && curl -q -L $(lynx -listonly -nonumbers -dump 
http://www.squid-cache.org/Versions/v4/  | grep "squid-4.5-" | grep ".tar.gz") 
| tar -xz -C squid-$(date +%F)  --strip-components 1 -f -
 
maybe is possible in another way but this works for me. 
 
greetz, 
 
Louis
 

Van: squid-users [mailto:squid-users-boun...@lists.squid-cache.org] Namens Vieri
Verzonden: woensdag 30 januari 2019 15:11
Aan: squid-users@lists.squid-cache.org
Onderwerp: [squid-users] daily releases



Hi,

Does anyone know of a convenient one-liner to get the latest daily release 
tarball, eg. 
http://www.squid-cache.org/Versions/v4/squid-4.5-20190128-r568e66b7c.tar.gz, 
without having to search for it manually on the web?

Either that or a symlink that would always point to the "latest daily".

Thanks,

Vieri










___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] using clang to compile squid 4-5

[L . P . H . van Belle]

Hai, 

Good to hear there are more then Luigi :-) 

I builded debian packages yesterday for squid 4.5 
Which was pretty simple and worked fine in the end. 

Get the source of 4.4  ( apt-get source -t unstable squid  )
Copy the debian folder from 4.4 into the 4.5 folder.

And changed in the changelog the squid version, builded fine. 
Test build failed, my change was. 

diff squid-4.4/debian/rules squid-4.5/debian/rules
22c22
< DEB_INSTALL_DOCS_squid-common := debian/copyright CONTRIBUTORS CREDITS 
QUICKSTART RELEASENOTES.html SPONSORS
---
> DEB_INSTALL_DOCS_squid-common := debian/copyright CONTRIBUTORS CREDITS 
> QUICKSTART SPONSORS

4.5 was missing : RELEASENOTES.html 
Uhm must say, i builded the "squid-4.5-20190128-r568e66b7c" version. 

Working now on a backport for debian stretch. 

Thank for all the good work Amos. ( and Luigi ) ! 

@Amos, any chance to upload 4.5 to unstable or experimental? 
I was hoping that 4.5 would get into buster, im guessing thats not going to 
happen? 


Greetz, 

Louis





> -Oorspronkelijk bericht-
> Van: squid-users 
> [mailto:squid-users-boun...@lists.squid-cache.org] Namens 
> Amos Jeffries
> Verzonden: woensdag 30 januari 2019 14:08
> Aan: graf huy; squid-users@lists.squid-cache.org
> Onderwerp: Re: [squid-users] using clang to compile squid 4-5
> 
> On 30/01/19 9:38 pm, graf huy wrote:
> > Hi,
> > 
> > Using update-alternatives to set clang++ was a success, and export
> > CC=/usr/bin/clang does the job (trying two differents 
> methods). Debian
> > project recommended to use export (CC,CCX) for versioning and only
> > update-alternatives for alternative as it says.
> > Anyway compilation with clang was as quick as gcc can do.
> > 
> 
> Great to know. Nothing to fix on that front for us then.
> 
> > 
> >>Seconded. With both my Squid Project and Debian pkg-squid 
> Team hat's on
> >>I'd like to know why you feel any need to force the compiler?
> > 
> > There were errors to patch with memcpy because of the gcc version 
> > 8.2.0-14 Debian wich considers some warnings as errors, besides some
> > targets are BSD's.
> > 
> 
> I'm not sure I follow that. Are you building with patched GCC? or a
> patched libc? or something else?
> 
> I build with the Debian Sid GCC version myself and have not seen these
> in any test builds with that or similar GCC versions. Am trying your
> build options now just to be sure.
> 
> 
> On the matter of BSD's, since they provide clang as the 
> default compiler
> they (FreeBSD at least) build with clan/clang++ without any need for
> CC/CXX setting. We have Jenkins tests for that.
> 
> 
> > 
> > Great to known you are on this package, I thought there was 
> only Luigi
> > Gangitano (lu...@debian.org) for ages...
> > 
> 
> Welcome. There are several of us, with Luigi as team lead.
> 
> Amos
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
> 

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Using a static wildcard certificate with ssl-bump in explicit forward proxy mode

[Bill Bernsen]

Amos, thank you for the quick response. My original question could use an
example to clarify.

client --> example.com (HTTPS squid proxy) --> instance.example.com
(HTTPS server)

The HTTPS squid proxy on example.com has a trusted wildcard certificate for
*.example.com
The HTTPS server on instance.example.com has an untrusted certificate for
instance.example.com

So without MITM, the client issues a CONNECT to squid running on example.com
which does its TLS, authenticates, connects to upstream then goes into
tunneling mode. The client does the TLS handshake with instance.example.com,
receives its untrusted certificate, and isn't happy.

I'm looking for a MITM mode that, instead of requiring a CA that can
dynamically create trusted certs on the fly, will return a wildcard
certificate for all requests (or even better, for any requests matching
hosts in its subdomain). Is that something that exists?

I hacked up my own version of ssl_crtd to serve a static cert and ran into
another wrinkle. Is there a version of squid that supports ssl-bump with
https_port?

On Fri, Jan 25, 2019 at 9:42 PM Amos Jeffries  wrote:

> On 26/01/19 5:51 am, Bill Bernsen wrote:
> > Hi,
> >
> > I have squid running as an explicit forward proxy on the
> > host example.com  controlling access to all hosts
> > in *.example.com . All the hosts in *.example.com
> >  have self-signed certificates that I want to
> > appear as trusted to user browsers. I don't have the option of obtaining
> > a trusted CA. I do, however, have a trusted wildcard certificate for
> > *.example.com  available. Is there a way that I can
> > tell squid to present this static wildcard certificate to clients in
> > lieu of all upstream server certificates?
>
>
> As a forward proxy clients are *not* connecting to any of the
> *.example.com domains. They are connecting to your proxy hostname - and
> telling it to take care of the origin connections. So all clients need
> is trust for the CA which signed the proxy's certificate.
>
> The proxy is the only agent in the path which needs to trust the
> wildcard *.example.com certificate.
>
>
> Amos
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] daily releases

[Amos Jeffries]

On 31/01/19 3:10 am, Vieri wrote:
> Hi,
> 
> Does anyone know of a convenient one-liner to get the latest daily
> release tarball, eg.
> http://www.squid-cache.org/Versions/v4/squid-4.5-20190128-r568e66b7c.tar.gz,
> without having to search for it manually on the web?
> 

The contents of the tarball are provided by rsync to optimize update
bandwidth:




Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] using clang to compile squid 4-5

[Amos Jeffries]

On 31/01/19 5:12 am, L.P.H. van Belle wrote:
> Hai, 
> 
> Good to hear there are more then Luigi :-) 
> 
> I builded debian packages yesterday for squid 4.5 
> Which was pretty simple and worked fine in the end. 
> 
> Get the source of 4.4  ( apt-get source -t unstable squid  )
> Copy the debian folder from 4.4 into the 4.5 folder.
> 
> And changed in the changelog the squid version, builded fine. 
> Test build failed, my change was. 
> 
> diff squid-4.4/debian/rules squid-4.5/debian/rules
> 22c22
> < DEB_INSTALL_DOCS_squid-common := debian/copyright CONTRIBUTORS CREDITS 
> QUICKSTART RELEASENOTES.html SPONSORS
> ---
>> DEB_INSTALL_DOCS_squid-common := debian/copyright CONTRIBUTORS CREDITS 
>> QUICKSTART SPONSORS
> 
> 4.5 was missing : RELEASENOTES.html 
> Uhm must say, i builded the "squid-4.5-20190128-r568e66b7c" version. 

Aye, looking into that is on my worklist for this weekend while waiting
for the release prep testing.


> 
> Working now on a backport for debian stretch. 
> 
> Thank for all the good work Amos. ( and Luigi ) ! 
> 
> @Amos, any chance to upload 4.5 to unstable or experimental? 
> I was hoping that 4.5 would get into buster, im guessing thats not going to 
> happen? 
> 

Um, that will depend on Luigi's time between now and then. We have until
the 10th to roll and test it.

I have updated the version in the team repository (pushed just now) but
the changes are relatively minor and 4.6 is due this weekend. If I can
replicate this strncpy issue in the next day a temporary fix for should
also be added before next upload.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Using a static wildcard certificate with ssl-bump in explicit forward proxy mode

[Amos Jeffries]

On 31/01/19 7:47 am, Bill Bernsen wrote:
> Amos, thank you for the quick response. My original question could use
> an example to clarify.
> 
> client --> example.com (HTTPS squid proxy)
> --> instance.example.com (HTTPS server)
> 

Sadly this does not by itself actually clarify the issue. Are those
domains the machine hostnames or the HTTP(S) message URL domains?
 (The description later does that, so this comment is more an FYI.)


> The HTTPS squid proxy on example.com has a trusted

... definition of that "on" and thus its implications for traffic syntax
and the resulting behaviour limitations is the key point(s) I am trying
to understand here.


> wildcard certificate for *.example.com
> The HTTPS server on instance.example.com 
> has an untrusted certificate for instance.example.com
> 
> So without MITM, the client issues a CONNECT to squid running on
> example.com  which does its TLS, authenticates,
> connects to upstream then goes into tunneling mode. The client does the
> TLS handshake with instance.example.com ,
> receives its untrusted certificate, and isn't happy.


In this case the client is fully aware that the proxy exists.

=> The proxy did *not* get a request to contact instance.example.com -
therefore it did not connect to instance.example.com.

==> The proxy was asked for a tunnel to "example.com" and all security
validation done by the proxy will be comparing the *exact* FQDN
"example.com" against values in that traffic.

The client *separately* (inside the tunneled opaque bytes) contacts the
server and negotiates use of the "instance.example.com" virtual host.

=> The proxy has zero involvement and zero knowledge of this.

==> all security validation done by the client itself and will be
comparing the negotiated FQDN "instance.example.com" against values in
that traffic.

PS. Note there are no wildcards other than the cert field(s). The things
tested against that wildcard has always an exact actual value.


> 
> I'm looking for a MITM mode that, instead of requiring a CA that can
> dynamically create trusted certs on the fly, will return a wildcard
> certificate for all requests (or even better, for any requests matching
> hosts in its subdomain). Is that something that exists?

MITM mode the client does not connect to the proxy. The client connects
to an origin.

 The network NAT or whatever doing the *intercept* part is delivering
the traffic to the proxy.

 Squid is _itself_ generating the CONNECT which shows up (to simplify
processing and failure handling). What data is available determines what
can be done:

SSL-Bump step-1 : only TCP SYN packet details are available.
   ==> raw-IP:port and nothing else.

SSL-Bump step-2 : TLS SNI (maybe)
   ==> exact FQDN value "instance.example.com"

Notice that at no point yet is the MITM ever aware that "example.com"
plays a part and no reason to even suspect a wildcard existence.

At step-3 things get a little bit easier since the server X.509
certificate is available. That cert is the only place the wildcard can
come from, and is in a certificate field where free-form text is allowed
and *often* used. One sub-domain may also sit on a separate server not
using this same cert with wildcard. So care needs to be taken to avoid
issues from that data source.


> 
> I hacked up my own version of ssl_crtd to serve a static cert and ran

Fine. Though you should be able to use generate-host-certificates=off
and place your static cert into the cert= parameter. If that is not
working it is a Squid bug to be fixed.

I have PRs already underway to make the generate option take the CA cert
and leave cert= for the static certs like your case needs. No ETA on
when that will be completed though.


> into another wrinkle. Is there a version of squid that supports ssl-bump
> with https_port?

All versions of Squid-3.2 and later support SSL-Bump on https_port.

Though since forward-proxy and MITM have mutually exclusive X.509
certificate requirements currently the only way to do SSL-Bump on
https_port is to use "intercept" or "tproxy" traffic modes - leaving
explicit TLS proxy with the forward-proxy mode.


Amos



signature.asc
Description: OpenPGP digital signature
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users