Re: [squid-dev] squid-5.0.5-20210223-r4af19cc24 difference in behaviors between openbsd and linux
I apologize for the ultra-long delay on this. I did just test this tonight and it worked properly under OpenBSD. What would be the process for submitting a bug report? -Robert > On Mar 29, 2021, at 4:33 AM, Amos Jeffries wrote: > > On 29/03/21 6:16 am, Eliezer Croitoru wrote: >> Hey Robert, >> I am not sure I understood what is the meaning of the description: >> openbsd: Requiring client certificates. >> linux: Not requiring any client certificates > > @Eliezer: > They are startup messages Squid prints in cache.log when a TLS server > context is initialized. > > > >> -Original Message- >> From: Robert Smith >> Sent: Sunday, March 28, 2021 7:27 PM >> Dear Squid-Dev list: >> I could use some help on this one: >> I have a build environment that is identical on linux, openbsd, and macosx >> In this scenario, I am developing under: >> Ubuntu 18.04 - All patches and updates applied as of 3/24 >> OpenBSD 6.8 - All patches and updates applied as of 3/24 >> I will note that I am really only using the libc from each system whereas >> every other component dependencies (which are not many! Good job squid >> team!) are a part of my build system. >> When building squid with the exact same tool chain and library stack, with >> the same configure options, I am seeing a difference in behavior on the two >> platforms: >> The difference is that after parsing the configuration file, the two systems >> differ in whether or not they will require client certificates: >> openbsd: Requiring client certificates. >> linux: Not requiring any client certificates >> > > What the message means depends on whether the http(s)_port, a cache_peer, or > the outgoing https:// context is being initialized. Options that directive > was supposed to be using (including the default security). > > Looking at your logs I see: > > > On OpenBSD Squid detects the presence of an IPv6 split-stack for networking. > Which means Squid has to clone the internal representation of all your > squid.conf *_port settings and setup separate contexts and state for IPv4 > versions of them. > There seems to be a bug in that cloning process which is turning on the TLS > client certificates feature. Please report this to our bugzilla so it does > not get forgotten until fixed. > > > On Linux Squid is detecting IPv6 disabled in the kernel networking setup. So > it is disabling its own IPv6 support. That said Linux has a hybrid-stack > networking so the cloning would not happen anyway. If IPv6 were enabled here > it would be somewhat more obvious that the IPv4 ports on OpenBSD are the odd > ones. > > > For a workaround you may be able to set sslflags=DELAYED_AUTH on the > http*_port lines and leave your ACLs as they are without anything requiring a > client certificate. > > > >> # openbsd >> root@openbsd:~# /root/squid.init conftest > >> 2021/03/28 10:47:31| Processing: http_port 3128 ssl-bump >> cert=/opt/osec/etc/ssl_cert/squid-ca-cert+key.pem >> generate-host-certificates=on dynamic_cert_mem_cache_size=16MB >> 2021/03/28 10:47:31| Processing: https_port 3129 intercept ssl-bump >> cert=/opt/osec/etc/ssl_cert/squid-ca-cert+key.pem >> generate-host-certificates=on dynamic_cert_mem_cache_size=16MB > >> 2021/03/28 10:47:31| Processing: tls_outgoing_options >> cafile=/opt/osec/etc/pki/tls/certs/ca-bundle.crt > > >> 2021/03/28 10:47:31| Initializing https:// proxy context >> 2021/03/28 10:47:31| Requiring client certificates. > > >> 2021/03/28 10:47:31| Initializing http_port [::]:3128 TLS contexts >> 2021/03/28 10:47:31| Using certificate in >> /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem >> 2021/03/28 10:47:31| Using certificate chain in >> /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem >> 2021/03/28 10:47:31| Adding issuer CA: /C=US/ST=Kansas/L=Overland >> Park/O=Company, Inc./OU=Area >> 77/CN=local.corp.dom/emailAddress=sslad...@company.com >> 2021/03/28 10:47:31| Using key in >> /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem >> 2021/03/28 10:47:31| Not requiring any client certificates > > >> 2021/03/28 10:47:31| Initializing http_port 0.0.0.0:3128 TLS contexts >> 2021/03/28 10:47:31| Using certificate in >> /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem >> 2021/03/28 10:47:31| Using certificate chain in >> /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem >> 2021/03/28 10:47:31| Adding issuer CA: /C=US/ST=Kansas/L=Overland >> Park/O=Company, Inc./OU=Area >> 77/CN=local.corp.dom/emailAddress=sslad...@company.com >> 2021/03/28 10:47:31| Using key in >> /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem >> 2021/03/28 10:47:31| Requiring client certificates. > > >> 2021/03/28 10:47:31| Initializing https_port [::]:3129 TLS contexts >> 2021/03/28 10:47:31| Using certificate in >> /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem >> 2021/03/28 10:47:31| Using certificate chain in >> /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem >> 2021/03/28 10:47:31| Adding issuer CA: /C=US/ST=Kansas/L=Overland >> Park/O=Company,
Re: [squid-dev] squid-5.0.5-20210223-r4af19cc24 difference in behaviors between openbsd and linux
On 29/03/21 6:16 am, Eliezer Croitoru wrote: Hey Robert, I am not sure I understood what is the meaning of the description: openbsd: Requiring client certificates. linux: Not requiring any client certificates @Eliezer: They are startup messages Squid prints in cache.log when a TLS server context is initialized. -Original Message- From: Robert Smith Sent: Sunday, March 28, 2021 7:27 PM Dear Squid-Dev list: I could use some help on this one: I have a build environment that is identical on linux, openbsd, and macosx In this scenario, I am developing under: Ubuntu 18.04 - All patches and updates applied as of 3/24 OpenBSD 6.8 - All patches and updates applied as of 3/24 I will note that I am really only using the libc from each system whereas every other component dependencies (which are not many! Good job squid team!) are a part of my build system. When building squid with the exact same tool chain and library stack, with the same configure options, I am seeing a difference in behavior on the two platforms: The difference is that after parsing the configuration file, the two systems differ in whether or not they will require client certificates: openbsd: Requiring client certificates. linux: Not requiring any client certificates What the message means depends on whether the http(s)_port, a cache_peer, or the outgoing https:// context is being initialized. Options that directive was supposed to be using (including the default security). Looking at your logs I see: On OpenBSD Squid detects the presence of an IPv6 split-stack for networking. Which means Squid has to clone the internal representation of all your squid.conf *_port settings and setup separate contexts and state for IPv4 versions of them. There seems to be a bug in that cloning process which is turning on the TLS client certificates feature. Please report this to our bugzilla so it does not get forgotten until fixed. On Linux Squid is detecting IPv6 disabled in the kernel networking setup. So it is disabling its own IPv6 support. That said Linux has a hybrid-stack networking so the cloning would not happen anyway. If IPv6 were enabled here it would be somewhat more obvious that the IPv4 ports on OpenBSD are the odd ones. For a workaround you may be able to set sslflags=DELAYED_AUTH on the http*_port lines and leave your ACLs as they are without anything requiring a client certificate. # openbsd root@openbsd:~# /root/squid.init conftest 2021/03/28 10:47:31| Processing: http_port 3128 ssl-bump cert=/opt/osec/etc/ssl_cert/squid-ca-cert+key.pem generate-host-certificates=on dynamic_cert_mem_cache_size=16MB 2021/03/28 10:47:31| Processing: https_port 3129 intercept ssl-bump cert=/opt/osec/etc/ssl_cert/squid-ca-cert+key.pem generate-host-certificates=on dynamic_cert_mem_cache_size=16MB 2021/03/28 10:47:31| Processing: tls_outgoing_options cafile=/opt/osec/etc/pki/tls/certs/ca-bundle.crt 2021/03/28 10:47:31| Initializing https:// proxy context 2021/03/28 10:47:31| Requiring client certificates. 2021/03/28 10:47:31| Initializing http_port [::]:3128 TLS contexts 2021/03/28 10:47:31| Using certificate in /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem 2021/03/28 10:47:31| Using certificate chain in /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem 2021/03/28 10:47:31| Adding issuer CA: /C=US/ST=Kansas/L=Overland Park/O=Company, Inc./OU=Area 77/CN=local.corp.dom/emailAddress=sslad...@company.com 2021/03/28 10:47:31| Using key in /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem 2021/03/28 10:47:31| Not requiring any client certificates 2021/03/28 10:47:31| Initializing http_port 0.0.0.0:3128 TLS contexts 2021/03/28 10:47:31| Using certificate in /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem 2021/03/28 10:47:31| Using certificate chain in /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem 2021/03/28 10:47:31| Adding issuer CA: /C=US/ST=Kansas/L=Overland Park/O=Company, Inc./OU=Area 77/CN=local.corp.dom/emailAddress=sslad...@company.com 2021/03/28 10:47:31| Using key in /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem 2021/03/28 10:47:31| Requiring client certificates. 2021/03/28 10:47:31| Initializing https_port [::]:3129 TLS contexts 2021/03/28 10:47:31| Using certificate in /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem 2021/03/28 10:47:31| Using certificate chain in /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem 2021/03/28 10:47:31| Adding issuer CA: /C=US/ST=Kansas/L=Overland Park/O=Company, Inc./OU=Area 77/CN=local.corp.dom/emailAddress=sslad...@company.com 2021/03/28 10:47:31| Using key in /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem 2021/03/28 10:47:31| Not requiring any client certificates 2021/03/28 10:47:31| Initializing https_port 0.0.0.0:3129 TLS contexts 2021/03/28 10:47:31| Using certificate in /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem 2021/03/28 10:47:31| Using certificate chain in /opt/osec/etc/ssl_cert/squid-ca-cert+key.pem 2021/03/28 10:47:31|
Re: [squid-dev] squid-5.0.5-20210223-r4af19cc24 difference in behaviors between openbsd and linux
Hey Robert, I am not sure I understood what is the meaning of the description: openbsd: Requiring client certificates. linux: Not requiring any client certificates In what sense? Let say you try You have then next config directives: http_port 3128 ssl-bump \ cert=/opt/osec/etc/ssl_cert/squid-ca-cert+key.pem \ generate-host-certificates=on dynamic_cert_mem_cache_size=16MB https_port 3129 intercept ssl-bump \ cert=/opt/osec/etc/ssl_cert/squid-ca-cert+key.pem \ generate-host-certificates=on dynamic_cert_mem_cache_size=16MB sslcrtd_program /opt/osec/libexec/security_file_certgen -s /opt/osec/etc/ssl_db -M 128MB acl step1 at_step SslBump1 ssl_bump peek step1 ssl_bump bump all ssl_bump splice all Which implies you do want ssl bump to work. To clear out: What is the desired results and where? How do you see that the expected result do not match the expectation? It would help if you would show the expectation using the relevant access.log output when you try to access let say https://www.google.com/404. Try to use the next to make it clear to me and probably others: https_proxy=http://127.0.0.1:3128/ curl https://www.google.com/404 -v https_proxy=http://127.0.0.1:3128/ curl https://www.google.com/404 -v -k I hope this would make more sense into the scenario you are having. Thanks, Eliezer Eliezer Croitoru Tech Support Mobile: +972-5-28704261 Email: ngtech1...@gmail.com Zoom: Coming soon -Original Message- From: squid-dev On Behalf Of Robert Smith Sent: Sunday, March 28, 2021 7:27 PM To: squid-dev@lists.squid-cache.org Subject: [squid-dev] squid-5.0.5-20210223-r4af19cc24 difference in behaviors between openbsd and linux Dear Squid-Dev list: I could use some help on this one: I have a build environment that is identical on linux, openbsd, and macosx In this scenario, I am developing under: Ubuntu 18.04 - All patches and updates applied as of 3/24 OpenBSD 6.8 - All patches and updates applied as of 3/24 I will note that I am really only using the libc from each system whereas every other component dependencies (which are not many! Good job squid team!) are a part of my build system. When building squid with the exact same tool chain and library stack, with the same configure options, I am seeing a difference in behavior on the two platforms: The difference is that after parsing the configuration file, the two systems differ in whether or not they will require client certificates: openbsd: Requiring client certificates. linux: Not requiring any client certificates One would think this was a run-time configuration difference, It is not. They are identical, Please see below: - all configuration, certificates, certificate databases under /opt/osec/etc on both systems are identical - the configuration file on both system is identical I have some suspicions about what the actual issue is. Using the configuration options below without any of the --enable-auth or --enable-auth* options (AUTH OPTIONS), both systems worked just fine and parse the configuration file identically. Of course, without auth. No good. After trying a number of different configure options and combinations, I discovered that on the linux platform, I could add the AUTH OPTIONS and remove the --enable-security-cert* (CERT OPTIONS): # --enable-security-cert-validators \ # --enable-security-cert-generators \ and then it would parse and run the way I was used to using peek & slice. Excited, thinking I'd found the issue, I ran the build on openbsd only to find the differences in functionality. BUILD & RUNTIME INFORMATION I will interleave these to make viewing easier. Please see below: # ## md5 sum of config file: # # openbsd root@openbsd:~# md5 /opt/osec/etc/squid.conf-bump MD5 (/opt/osec/etc/squid.conf-bump) = a0bf93867aaff1f35eb1af23dd5eb49b # linux root@linux:~# md5sum /opt/osec/etc/squid.conf-bump a0bf93867aaff1f35eb1af23dd5eb49b /opt/osec/etc/squid.conf-bump # ## Actual configuration (sanitized) # acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localhost manager http_access deny manager http_access allow localnet http_access allow localhost http_access deny all http_port 3128 ssl-bump \ cert=/opt/osec/etc/ssl_cert/
[squid-dev] squid-5.0.5-20210223-r4af19cc24 difference in behaviors between openbsd and linux
Dear Squid-Dev list: I could use some help on this one: I have a build environment that is identical on linux, openbsd, and macosx In this scenario, I am developing under: Ubuntu 18.04 - All patches and updates applied as of 3/24 OpenBSD 6.8 - All patches and updates applied as of 3/24 I will note that I am really only using the libc from each system whereas every other component dependencies (which are not many! Good job squid team!) are a part of my build system. When building squid with the exact same tool chain and library stack, with the same configure options, I am seeing a difference in behavior on the two platforms: The difference is that after parsing the configuration file, the two systems differ in whether or not they will require client certificates: openbsd: Requiring client certificates. linux: Not requiring any client certificates One would think this was a run-time configuration difference, It is not. They are identical, Please see below: - all configuration, certificates, certificate databases under /opt/osec/etc on both systems are identical - the configuration file on both system is identical I have some suspicions about what the actual issue is. Using the configuration options below without any of the --enable-auth or --enable-auth* options (AUTH OPTIONS), both systems worked just fine and parse the configuration file identically. Of course, without auth. No good. After trying a number of different configure options and combinations, I discovered that on the linux platform, I could add the AUTH OPTIONS and remove the --enable-security-cert* (CERT OPTIONS): # --enable-security-cert-validators \ # --enable-security-cert-generators \ and then it would parse and run the way I was used to using peek & slice. Excited, thinking I'd found the issue, I ran the build on openbsd only to find the differences in functionality. BUILD & RUNTIME INFORMATION I will interleave these to make viewing easier. Please see below: # ## md5 sum of config file: # # openbsd root@openbsd:~# md5 /opt/osec/etc/squid.conf-bump MD5 (/opt/osec/etc/squid.conf-bump) = a0bf93867aaff1f35eb1af23dd5eb49b # linux root@linux:~# md5sum /opt/osec/etc/squid.conf-bump a0bf93867aaff1f35eb1af23dd5eb49b /opt/osec/etc/squid.conf-bump # ## Actual configuration (sanitized) # acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localhost manager http_access deny manager http_access allow localnet http_access allow localhost http_access deny all http_port 3128 ssl-bump \ cert=/opt/osec/etc/ssl_cert/squid-ca-cert+key.pem \ generate-host-certificates=on dynamic_cert_mem_cache_size=16MB https_port 3129 intercept ssl-bump \ cert=/opt/osec/etc/ssl_cert/squid-ca-cert+key.pem \ generate-host-certificates=on dynamic_cert_mem_cache_size=16MB sslcrtd_program /opt/osec/libexec/security_file_certgen -s /opt/osec/etc/ssl_db -M 128MB acl step1 at_step SslBump1 ssl_bump peek step1 ssl_bump bump all ssl_bump splice all coredump_dir /var/spool/squid refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 cache_access_log /data/logs/access.log cache_log /data/logs/cache.log cache_store_log /data/logs/store.log shutdown_lifetime 5 seconds tls_outgoing_options cafile=/opt/osec/etc/pki/tls/certs/ca-bundle.crt on_unsupported_protocol tunnel all # ## -k parse # # openbsd root@openbsd:~# /root/squid.init conftest 2021/03/28 10:47:31| Startup: Initializing Authentication Schemes ... 2021/03/28 10:47:31| Startup: Initialized Authentication Scheme 'basic' 2021/03/28 10:47:31| Startup: Initialized Authentication Scheme 'digest' 2021/03/28 10:47:31| Startup: Initialized Authentication Scheme 'negotiate' 2021/03/28 10:47:31| Startup: Initialized Authentication Scheme 'ntlm' 2021/03/28 10:47:31| Startup: Initialized Authentication. 2021/03/28 10:47:31| Processing Configuration File: /opt/osec/etc/squid.conf-bump (depth 0) 2021/03/28 10:47:31| Processing: acl localnet src 10.0.0.0/8# RFC1918 possible internal network 2021/03/28 10:47:31| Processing: acl localnet src 172.16.0.0/12 # RFC1918 possible internal
