Re: [squid-users] Error negotiating SSL connection on FD 16
On 2/04/20 5:42 pm, saiyan_gc wrote: > Hi, thank you for reply me. Really appreciated! > > I modified the squid conf file to: > > http_port 2128 ssl-bump cert=/etc/squid/ssl_cert/example.com.cert \ > key=/etc/squid/ssl_cert/example.com.private \ > generate-host-certificates=on \ > dynamic_cert_mem_cache_size=4MB > https_port 3130 cert=/etc/squid/ssl_cert/example.com.cert \ > key=/etc/squid/ssl_cert/example.com.private > auth_param basic program /usr/lib/squid/basic_ncsa_auth /etc/squid/passwords > auth_param basic children 5 startup=0 idle=1 > auth_param basic credentialsttl 2 hours > auth_param basic casesensitive off > acl ncsa_users proxy_auth REQUIRED > acl step1 at_step SslBump1 > ssl_bump peek step1 > ssl_bump bump all > http_access deny !ncsa_users > http_access allow ncsa_users > > And it's working for http_port. I put the cert into > /etc/pki/trust-ca/source/anchor, and run a update-ca-trust command. And both > aws cli and curl command work now. I am still not sure why https_port desn't > work. What you have here is: * TLS explicit/forward proxy on port 3130. This requires a regular server certificate for the proxy to use it as a server encrypting traffic between the client and proxy. * Interception of HTTPS sent in CONNECT tunnels over a plain-text proxy on port 2128. This requires a CA certificate to sign auto-generated server certificates encrypting the traffic between client and origin server. That difference in cert type is why when one port works, the other will not. So first thing to do is make sure the cert types are correct. > The previous setting work with curl but not aws cli, not sure why it failure > during tls handshake. > The second thing you will need to do, is find out which port the tool is using and whether it is using it in the right mode of traffic. Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid transparent not caching apt requests from deb.debian.org
On 4/3/20 16:34, Alex Rousskov wrote: On 4/3/20 4:26 PM, zrm wrote: In the first case we get TCP_MISS every time because it isn't caching the data, in the second case it's only the first time and after that we get TCP_REFRESH_UNMODIFIED. But how and why is this happening? Those questions can often be answered by looking at HTTP headers received and sent by Squid when talking to clients and origin servers. You can see those headers in basic packet captures or if you set debug_options to ALL,2. Alex. apt to squid: - GET /debian/pool/main/v/vim/vim_8.1.0875-5_amd64.deb HTTP/1.1 Host: deb.debian.org User-Agent: Debian APT-HTTP/1.3 (1.8.2) -- squid to deb.debian.org: - GET /debian/pool/main/v/vim/vim_8.1.0875-5_amd64.deb HTTP/1.1 User-Agent: Debian APT-HTTP/1.3 (1.8.2) Host: deb.debian.org Via: 1.1 tproxy (squid/4.6) X-Forwarded-For: 192.168.111.55 Cache-Control: max-age=259200 Connection: keep-alive -- deb.debian.org to squid: - HTTP/1.1 200 OK Server: Apache X-Content-Type-Options: nosniff X-Frame-Options: sameorigin Referrer-Policy: no-referrer X-Xss-Protection: 1 Last-Modified: Sat, 15 Jun 2019 17:46:35 GMT ETag: "1389dc-58b605823fa6e" X-Clacks-Overhead: GNU Terry Pratchett Cache-Control: public, max-age=2592000 Content-Type: application/x-debian-package Via: 1.1 varnish Content-Length: 1280476 Accept-Ranges: bytes Date: Fri, 03 Apr 2020 05:28:46 GMT Via: 1.1 varnish Age: 4248100 Connection: keep-alive X-Served-By: cache-ams21028-AMS, cache-wdc5559-WDC X-Cache: HIT, HIT X-Cache-Hits: 1, 2 X-Timer: S1585891726.434375,VS0,VE0 -- squid to apt: - HTTP/1.1 200 OK Server: Apache X-Content-Type-Options: nosniff X-Frame-Options: sameorigin Referrer-Policy: no-referrer X-Xss-Protection: 1 Last-Modified: Sat, 15 Jun 2019 17:46:35 GMT ETag: "1389dc-58b605823fa6e" X-Clacks-Overhead: GNU Terry Pratchett Cache-Control: public, max-age=2592000 Content-Type: application/x-debian-package Content-Length: 1280476 Accept-Ranges: bytes Date: Fri, 03 Apr 2020 05:28:46 GMT Age: 4248100 X-Served-By: cache-ams21028-AMS, cache-wdc5559-WDC X-Cache: HIT, HIT X-Cache-Hits: 1, 2 X-Timer: S1585891726.434375,VS0,VE0 X-Cache: MISS from tproxy X-Cache-Lookup: MISS from tproxy:3130 Via: 1.1 varnish, 1.1 varnish, 1.1 tproxy (squid/4.6) Connection: keep-alive -- wget to squid: - GET /debian/pool/main/v/vim/vim_8.1.0875-5_amd64.deb HTTP/1.1 User-Agent: Wget/1.20.1 (linux-gnu) Accept: */* Accept-Encoding: identity Host: deb.debian.org Connection: Keep-Alive -- squid to deb.debian.org: -- GET /debian/pool/main/v/vim/vim_8.1.0875-5_amd64.deb HTTP/1.1 User-Agent: Wget/1.20.1 (linux-gnu) Accept: */* Accept-Encoding: identity Host: deb.debian.org Via: 1.1 tproxy (squid/4.6) X-Forwarded-For: 192.168.111.55 Cache-Control: max-age=259200 Connection: keep-alive -- deb.debian.org to squid: - HTTP/1.1 200 OK Server: Apache X-Content-Type-Options: nosniff X-Frame-Options: sameorigin Referrer-Policy: no-referrer X-Xss-Protection: 1 Last-Modified: Sat, 15 Jun 2019 17:46:35 GMT ETag: "1389dc-58b605823fa6e" X-Clacks-Overhead: GNU Terry Pratchett Cache-Control: public, max-age=2592000 Content-Type: application/x-debian-package Via: 1.1 varnish Content-Length: 1280476 Accept-Ranges: bytes Date: Fri, 03 Apr 2020 05:28:49 GMT Via: 1.1 varnish Age: 4248102 Connection: keep-alive X-Served-By: cache-ams21028-AMS, cache-wdc5583-WDC X-Cache: HIT, HIT X-Cache-Hits: 1, 1 X-Timer: S1585891729.316362,VS0,VE0 -- squid to wget: - HTTP/1.1 200 OK Server: Apache X-Content-Type-Options: nosniff X-Frame-Options: sameorigin Referrer-Policy: no-referrer X-Xss-Protection: 1 Last-Modified: Sat, 15 Jun 2019 17:46:35 GMT ETag: "1389dc-58b605823fa6e" X-Clacks-Overhead: GNU Terry Pratchett Cache-Control: public, max-age=2592000 Content-Type: application/x-debian-package Content-Length: 1280476 Accept-Ranges: bytes Date: Fri, 03 Apr 2020 05:28:49 GMT Age: 4248102 X-Served-By: cache-ams21028-AMS, cache-wdc5583-WDC X-Cache: HIT, HIT X-Cache-Hits: 1, 1 X-Timer: S1585891729.316362,VS0,VE0 X-Cache: MISS from tproxy X-Cache-Lookup: MISS from tproxy:3130 Via: 1.1 varnish, 1.1 varnish, 1.1 tproxy (squid/4.6) Connection: keep-alive -- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid transparent not caching apt requests from deb.debian.org
On 4/3/20 4:26 PM, zrm wrote: > In the first case we get TCP_MISS every time because it isn't caching > the data, in the second case it's only the first time and after that we > get TCP_REFRESH_UNMODIFIED. But how and why is this happening? Those questions can often be answered by looking at HTTP headers received and sent by Squid when talking to clients and origin servers. You can see those headers in basic packet captures or if you set debug_options to ALL,2. Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid transparent not caching apt requests from deb.debian.org
On Friday 03 April 2020 at 22:26:13, zrm wrote: > Greetings! Today I bring you a Squid cache mystery. > In the first case we get TCP_MISS every time because it isn't caching > the data, in the second case it's only the first time and after that we > get TCP_REFRESH_UNMODIFIED. But how and why is this happening? Given that this is an intercepting proxy and you're using HTTP (not HTTPS), can you do a packet capture with tshark or similar on the internal interface, to show the full details of the HTTP request which comes in to Squid from apt, and the same for wget, to see what difference there is? Antony. -- Normal people think "If it ain't broke, don't fix it". Engineers think "If it ain't broke, it doesn't have enough features yet". Please reply to the list; please *don't* CC me. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
[squid-users] Squid transparent not caching apt requests from deb.debian.org
Greetings! Today I bring you a Squid cache mystery. I configured a simple transparent proxy to cache package downloads for Debian, using Squid on Debian 10. When apt clients download packages from deb.debian.org, Squid says TCP_MISS, downloads the package, and then doesn't cache it. The mystery is, why not. It caches other requests by other applications. It caches other requests by apt -- packages from security.debian.org get cached. It caches requests for deb.debian.org when the client is not apt -- using wget or curl on the same URL causes it to be cached, as does pasting the HTTP request into netcat. The same HTTP request that apt sends. I checked the packets with WireShark and the TCP payload for the request is byte for byte identical, but when apt does it, it isn't cached. I'm not sure how it's even distinguishing the requests in order to behave differently. These are the changes from the default squid.conf packaged with Debian 10: http_access allow localnet http_port 3130 http_port 3128 intercept maximum_object_size_in_memory 4 MB Here are repeated requests using "apt remove -y vim;apt install vim", access.log: 1585891724.223560 192.168.111.55 TCP_MISS/200 1281195 GET http://deb.debian.org/debian/pool/main/v/vim/vim_8.1.0875-5_amd64.deb - ORIGINAL_DST/199.232.64.204 application/x-debian-package 1585891726.697277 192.168.111.55 TCP_MISS/200 1281195 GET http://deb.debian.org/debian/pool/main/v/vim/vim_8.1.0875-5_amd64.deb - ORIGINAL_DST/199.232.64.204 application/x-debian-package store.log: 1585891535.154 RELEASE -1 040017120100 200 1585891535 1560620795 1588483535 application/x-debian-package 1280476/1280476 GET http://deb.debian.org/debian/pool/main/v/vim/vim_8.1.0875-5_amd64.deb 1585891724.223 RELEASE -1 050017120100 200 1585891723 1560620795 1588483723 application/x-debian-package 1280476/1280476 GET http://deb.debian.org/debian/pool/main/v/vim/vim_8.1.0875-5_amd64.deb But here are repeated requests for the same url using wget, access.log: 1585891729.700433 192.168.111.55 TCP_MISS/200 1281195 GET http://deb.debian.org/debian/pool/main/v/vim/vim_8.1.0875-5_amd64.deb - ORIGINAL_DST/199.232.66.133 application/x-debian-package 1585891731.089 70 192.168.111.55 TCP_REFRESH_UNMODIFIED/200 1281232 GET http://deb.debian.org/debian/pool/main/v/vim/vim_8.1.0875-5_amd64.deb - ORIGINAL_DST/199.232.66.133 application/x-debian-package store.log: 1585891726.697 RELEASE -1 060017120100 200 1585891726 1560620795 1588483726 application/x-debian-package 1280476/1280476 GET http://deb.debian.org/debian/pool/main/v/vim/vim_8.1.0875-5_amd64.deb 1585891731.047 RELEASE -1 080017120100 304 1585891731-1 1588483731 unknown -1/-293 GET http://deb.debian.org/debian/pool/main/v/vim/vim_8.1.0875-5_amd64.deb In the first case we get TCP_MISS every time because it isn't caching the data, in the second case it's only the first time and after that we get TCP_REFRESH_UNMODIFIED. But how and why is this happening? ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
