Re: [squid-users] Squid and systemd

[James Lay]

Thanks...still a newb at systemd and that was totally the fix.

James

On Wed, 2018-06-13 at 10:03 -0300, Marcus Kool wrote:
> I have seen systemd killing daemons when it times out waiting for the
> pid file to appear.I suggest to doublecheck that the pid filename in
> the service file and in squid.conf are the same.
> Marcus
> On 13/06/18 09:27, James Lay wrote:
> WellI'll just say up front that systemd is not my friend. When
> running squid via cli: sudo /opt/squid/sbin/squid it runs like a
> champ. But using the service file at:
> https://raw.githubusercontent.com/squid-cache/squid/master/tools/syst
> emd/squid.service
> it times out after a few:
> 06:20:11 gateway squid[3669]: Created PID file
> (/opt/squid/var/run/squid.pid)06:20:11 gateway squid[3669]: Squid
> Parent: will start 1 kids06:20:11 gateway squid[3669]: Squid Parent:
> (squid-1) process 3678 started06:20:11 gateway squid[3678]: Set
> Current Directory to /opt/squid/var06:20:11 gateway squid[3678]:
> Starting Squid Cache version 4.0.24 for x86_64-pc-linux-
> gnu...06:20:11 gateway squid[3678]: Service Name: squid06:20:11
> gateway squid[3678]: Process ID 367806:20:11 gateway squid[3678]:
> Process Roles: worker06:20:11 gateway squid[3678]: With 1024 file
> descriptors available06:20:11 gateway squid[3678]: Initializing IP
> Cache...06:20:11 gateway squid[3678]: DNS Socket created at [::], FD
> 506:20:11 gateway squid[3678]: DNS Socket created at 0.0.0.0, FD
> 1006:20:11 gateway squid[3678]: Adding nameserver 192.168.1.253 from
> /etc/resolv.conf06:20:11 gateway squid[3678]: Adding nameserver
> 205.171.3.65 from /etc/resolv.conf06:20:11 gateway squid[3678]:
> Adding nameserver 205.171.2.65 from /etc/resolv.conf06:20:11 gateway
> squid[3678]: Adding domain slave-tothe-box.net from
> /etc/resolv.conf06:20:11 gateway squid[3678]: Adding domain slave-
> tothe-box.net from /etc/resolv.conf06:20:11 gateway squid[3678]:
> helperOpenServers: Starting 5/5 'security_file_certgen'
> processes06:20:11 gateway squid[3678]: Logfile: opening log
> syslog:daemon.info06:20:11 gateway squid[3678]: Store logging
> disabled06:20:11 gateway squid[3678]: Swap maxSize 0 + 262144 KB,
> estimated 20164 objects06:20:11 gateway squid[3678]: Target number of
> buckets: 100806:20:11 gateway squid[3678]: Using 8192 Store
> buckets06:20:11 gateway squid[3678]: Max Mem  size: 262144 KB06:20:11
> gateway squid[3678]: Max Swap size: 0 KB06:20:11 gateway squid[3678]:
> Using Least Load store dir selection06:20:11 gateway squid[3678]: Set
> Current Directory to /opt/squid/var06:20:11 gateway squid[3678]:
> Finished loading MIME types and icons.06:20:11 gateway squid[3678]:
> HTCP Disabled.06:20:11 gateway squid[3678]: Squid plugin modules
> loaded: 006:20:11 gateway squid[3678]: Adaptation support is
> off.06:20:11 gateway squid[3678]: Accepting HTTP Socket connections
> at local=x.x.x.x:3127 remote=[::] FD 21 flags=906:20:11 gateway
> squid[3678]: Accepting NAT intercepted HTTP Socket connections at
> local=x.x.x.x:3128 remote=[::] FD 22 flags=4106:20:11 gateway
> squid[3678]: Accepting NAT intercepted SSL bumped HTTPS Socket
> connections at local=x.x.x.x:3129 remote=[::] FD 23 flags=4106:20:12
> gateway squid[3678]: storeLateRelease: released 0 objects06:21:41
> gateway systemd[1]: squid.service: Start operation timed out.
> Terminating.06:21:41 gateway systemd[1]: squid.service: Killing
> process 3669 (squid) with signal SIGKILL.06:21:41 gateway sudo:
> pam_unix(sudo:session): session closed for user root06:21:41 gateway
> systemd[1]: squid.service: Killing process 3678 (squid) with signal
> SIGKILL.06:21:41 gateway jlay[2415] 192.168.1.2 46692 192.168.1.252
> 22: sudo systemctl start squid06:21:41 gateway systemd[1]:
> squid.service: Killing process 3680 (security_file_c) with signal
> SIGKILL.06:21:41 gateway systemd[1]: squid.service: Killing process
> 3682 (security_file_c) with signal SIGKILL.06:21:41 gateway
> systemd[1]: squid.service: Killing process 3683 (security_file_c)
> with signal SIGKILL.06:21:41 gateway systemd[1]: squid.service:
> Killing process 3684 (security_file_c) with signal SIGKILL.06:21:41
> gateway systemd[1]: squid.service: Killing process 3685
> (security_file_c) with signal SIGKILL.06:21:41 gateway systemd[1]:
> squid.service: Failed with result 'timeout'.06:21:41 gateway
> systemd[1]: Failed to start Squid Web Proxy Server.
> I've modded the service file to reflect different binary location,
> but that's about it. Thank you.
> James
> 
> ___squid-users mailing
> listsquid-users@lists.squid-cache.orghttp://lists.squid-cache.org/lis
> tinfo/squid-users
> ___squid-users mailing
> listsquid-users@lists.squid-cache.orghttp://lists.squid-cache.org/lis
> tinfo/squid-users
> 
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Squid and systemd

[James Lay]

WellI'll just say up front that systemd is not my friend.  When
running squid via cli:  sudo /opt/squid/sbin/squid it runs like a
champ.  But using the service file at:
https://raw.githubusercontent.com/squid-cache/squid/master/tools/system
d/squid.service
it times out after a few:
06:20:11 gateway squid[3669]: Created PID file
(/opt/squid/var/run/squid.pid)06:20:11 gateway squid[3669]: Squid
Parent: will start 1 kids06:20:11 gateway squid[3669]: Squid Parent:
(squid-1) process 3678 started06:20:11 gateway squid[3678]: Set Current
Directory to /opt/squid/var06:20:11 gateway squid[3678]: Starting Squid
Cache version 4.0.24 for x86_64-pc-linux-gnu...06:20:11 gateway
squid[3678]: Service Name: squid06:20:11 gateway squid[3678]: Process
ID 367806:20:11 gateway squid[3678]: Process Roles: worker06:20:11
gateway squid[3678]: With 1024 file descriptors available06:20:11
gateway squid[3678]: Initializing IP Cache...06:20:11 gateway
squid[3678]: DNS Socket created at [::], FD 506:20:11 gateway
squid[3678]: DNS Socket created at 0.0.0.0, FD 1006:20:11 gateway
squid[3678]: Adding nameserver 192.168.1.253 from
/etc/resolv.conf06:20:11 gateway squid[3678]: Adding nameserver
205.171.3.65 from /etc/resolv.conf06:20:11 gateway squid[3678]: Adding
nameserver 205.171.2.65 from /etc/resolv.conf06:20:11 gateway
squid[3678]: Adding domain slave-tothe-box.net from
/etc/resolv.conf06:20:11 gateway squid[3678]: Adding domain slave-
tothe-box.net from /etc/resolv.conf06:20:11 gateway squid[3678]:
helperOpenServers: Starting 5/5 'security_file_certgen'
processes06:20:11 gateway squid[3678]: Logfile: opening log
syslog:daemon.info06:20:11 gateway squid[3678]: Store logging
disabled06:20:11 gateway squid[3678]: Swap maxSize 0 + 262144 KB,
estimated 20164 objects06:20:11 gateway squid[3678]: Target number of
buckets: 100806:20:11 gateway squid[3678]: Using 8192 Store
buckets06:20:11 gateway squid[3678]: Max Mem  size: 262144 KB06:20:11
gateway squid[3678]: Max Swap size: 0 KB06:20:11 gateway squid[3678]:
Using Least Load store dir selection06:20:11 gateway squid[3678]: Set
Current Directory to /opt/squid/var06:20:11 gateway squid[3678]:
Finished loading MIME types and icons.06:20:11 gateway squid[3678]:
HTCP Disabled.06:20:11 gateway squid[3678]: Squid plugin modules
loaded: 006:20:11 gateway squid[3678]: Adaptation support is
off.06:20:11 gateway squid[3678]: Accepting HTTP Socket connections at
local=x.x.x.x:3127 remote=[::] FD 21 flags=906:20:11 gateway
squid[3678]: Accepting NAT intercepted HTTP Socket connections at
local=x.x.x.x:3128 remote=[::] FD 22 flags=4106:20:11 gateway
squid[3678]: Accepting NAT intercepted SSL bumped HTTPS Socket
connections at local=x.x.x.x:3129 remote=[::] FD 23 flags=4106:20:12
gateway squid[3678]: storeLateRelease: released 0 objects06:21:41
gateway systemd[1]: squid.service: Start operation timed out.
Terminating.06:21:41 gateway systemd[1]: squid.service: Killing process
3669 (squid) with signal SIGKILL.06:21:41 gateway sudo:
pam_unix(sudo:session): session closed for user root06:21:41 gateway
systemd[1]: squid.service: Killing process 3678 (squid) with signal
SIGKILL.06:21:41 gateway jlay[2415] 192.168.1.2 46692 192.168.1.252 22:
sudo systemctl start squid06:21:41 gateway systemd[1]: squid.service:
Killing process 3680 (security_file_c) with signal SIGKILL.06:21:41
gateway systemd[1]: squid.service: Killing process 3682
(security_file_c) with signal SIGKILL.06:21:41 gateway systemd[1]:
squid.service: Killing process 3683 (security_file_c) with signal
SIGKILL.06:21:41 gateway systemd[1]: squid.service: Killing process
3684 (security_file_c) with signal SIGKILL.06:21:41 gateway systemd[1]:
squid.service: Killing process 3685 (security_file_c) with signal
SIGKILL.06:21:41 gateway systemd[1]: squid.service: Failed with result
'timeout'.06:21:41 gateway systemd[1]: Failed to start Squid Web Proxy
Server.
I've modded the service file to reflect different binary location, but
that's about it.  Thank you.
James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] About to upgrade from 3 to 4

[James Lay]

On Sun, 2018-06-10 at 19:55 +1200, Amos Jeffries wrote:
> On 10/06/18 02:23, James Lay wrote:
> On Sat, 2018-06-09 at 07:17 -0600, James Lay wrote:
> On Sun, 2018-06-10 at 01:13 +1200, Amos Jeffries wrote:
> On 10/06/18 01:02, James Lay wrote:
> So in my config file I have:
> sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
> However I do not see this after compiling and installing. Has this
> goneaway in 4? Thank you.
> James
> 
> It's now called security_file_certgen.
> <http://www.squid-cache.org/Versions/v4/squid-4.0.24-RELEASENOTES.htm
> l#ss2.4>
> Amos
> 
> Thanks Amos...I'll read this before asking anymore questions ☺
> 
> 
> So ok...after making the changes to the config to account for
> newsecurity_file_certgen and tls_outgoing_options (thanks Amos!) I
> amgreeted with (hostname changed from real):
> FATAL: mimeLoadIcon: cannot parse internal URL:http://:0/sq
> uid-internal-static/icons/silk/image.png
> 
> There should be an error about no forward-proxy port as well.
> Squidrequires at least one port able to receive requests for those
> URLs fromclients. Port 3128 is normally that port, but you have
> repurposed it forinterception, which disqualifies it.
> The hostname in these URLs is taken from that port's IP
> addressreverse-DNS name, or the proxies public/visible hostname.
> Whichevermeets the requirement of being resolvable in DNS.
> 
> Here's my config line:
> ./configure --prefix=/opt/squid --with-openssl=/opt/libressl
> --sysconfdir=/opt/squid/etc --enable-ssl --enable-ssl-crtd--enable-
> linux-netfilter --enable-follow-x-forwarded-for--with-large-files --
> enable-xternal-acl-helpers=none
> Missing 'e' on --enable-external-acl-helpers.
> ...
> 
> sslproxy_cert_error allow alltls_outgoing_options
> capath=/etc/ssl/certs flags=DONT_VERIFY_PEER
> Please avoid DONT_VERIFY_PEER and "allow all" for cert errors. They
> areuseless for both production AND debugging since all they do is
> hidesecurity issues from *you*.
> It is best to watch for security issues and fix them. Not just
> ignoreeverything.
> Amos___squid-users
> mailing listsquid-users@lists.squid-cache.orghttp://lists.squid-cache
> .org/listinfo/squid-users

Thanks Amos...your insight always helps.  You were right on point...I
did have the no forward proxy error.  After adding an additional
http_port squid came right up...thanks again.

James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] About to upgrade from 3 to 4

[James Lay]

On Sat, 2018-06-09 at 07:17 -0600, James Lay wrote:
> On Sun, 2018-06-10 at 01:13 +1200, Amos Jeffries wrote:
> > On 10/06/18 01:02, James Lay wrote:
> > 
> > So in my config file I have:
> > sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
> > However I do not see this after compiling and installing. Has this
> > goneaway in 4? Thank you.
> > James
> > 
> > It's now called security_file_certgen.
> > <http://www.squid-cache.org/Versions/v4/squid-4.0.24-RELEASENOTES.h
> > tml#ss2.4>
> > Amos
> 
> Thanks Amos...I'll read this before asking anymore questions ☺
> 
> James
> ___squid-users mailing
> listsquid-users@lists.squid-cache.orghttp://lists.squid-cache.org/lis
> tinfo/squid-users

So ok...after making the changes to the config to account for
new  security_file_certgen and tls_outgoing_options (thanks Amos!) I am
greeted with (hostname changed from real):
FATAL: mimeLoadIcon: cannot parse internal URL: http://:0/squ
id-internal-static/icons/silk/image.png
Here's my config line:
./configure --prefix=/opt/squid --with-openssl=/opt/libressl --
sysconfdir=/opt/squid/etc --enable-ssl --enable-ssl-crtd --enable-
linux-netfilter --enable-follow-x-forwarded-for --with-large-files --
enable-xternal-acl-helpers=none
full config (I realize this might not be the most secure on the planet,
for now this is a dev box and I'm just testing functionality):
acl localnet src 192.168.1.0/24acl SSL_ports port 443acl Safe_ports
port 80acl Safe_ports port 443acl CONNECT method CONNECTacl
allowed_http_sites url_regex "/opt/squid/etc/http_url.txt"
http_access deny !Safe_portshttp_access deny CONNECT
!SSL_Portshttp_access allow SSL_portshttp_access allow
allowed_http_siteshttp_access deny all
acl broken_ips dst "/opt/squid/etc/broken_ips.txt"ssl_bump splice
broken_ipsacl broken_https_sites ssl::server_name_regex
"/opt/squid/etc/broken_url.txt"ssl_bump splice
broken_https_sitesssl_bump peek allacl allowed_https_sites
ssl::server_name_regex "/opt/squid/etc/http_url.txt"ssl_bump splice
allowed_https_sitesssl_bump terminate all
sslproxy_cert_error allow alltls_outgoing_options capath=/etc/ssl/certs
flags=DONT_VERIFY_PEER
sslcrtd_program /opt/squid/libexec/security_file_certgen -s
/opt/squid/var/ -M 4MBsslcrtd_children 5
http_port gateway:3128 intercepthttps_port gateway:3129 intercept ssl-
bump cert=/opt/squid/etc/certs/sslsplit_ca_cert.pem
cafile=/opt/squid/etc/certs/sslsplit_ca_cert.pem
key=/opt/squid/etc/certs/sslsplit_ca_key.pem generate-host-
certificates=on dynamic_cert_mem_cache_size=4MB
sslflags=NO_SESSION_REUSE
logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni
%ssl::>cert_subject %>Hs %___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] About to upgrade from 3 to 4

[James Lay]

On Sun, 2018-06-10 at 01:13 +1200, Amos Jeffries wrote:
> On 10/06/18 01:02, James Lay wrote:
> 
> So in my config file I have:
> sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
> However I do not see this after compiling and installing. Has this
> goneaway in 4? Thank you.
> James
> 
> It's now called security_file_certgen.
> <http://www.squid-cache.org/Versions/v4/squid-4.0.24-RELEASENOTES.htm
> l#ss2.4>
> Amos

Thanks Amos...I'll read this before asking anymore questions ☺

James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] About to upgrade from 3 to 4

[James Lay]

On Fri, 2018-06-08 at 09:36 -0600, James Lay wrote:
> On Sat, 2018-06-09 at 03:04 +1200, Amos Jeffries wrote:
> > On 09/06/18 02:33, James Lay wrote:
> > Hey all!
> > Topic says itI'm starting to look at doing an upgrade from 3 to
> > 4.Any glaring surprises? Doing a transparent forward proxy with
> > somepeek/splice for content filtering only (no decryption). Has
> > anyone gonethrough an upgrade, and how painful was it, if at all?
> > Thank you.
> > 
> > Which 3.x you are starting from is the issue.
> > From 3.5 to 4 should be the same as any of the 3.x single version
> > bumps.There is nothing special about v4 from a user perspective.
> > Amos___squid-users
> > mailing listsquid-users@lists.squid-cache.orghttp://lists.squid-cac
> > he.org/listinfo/squid-users
> 
> Thanks Amos...I'm going from 3.5.2 ☺
> 
> James
> ___squid-users mailing
> listsquid-users@lists.squid-cache.orghttp://lists.squid-cache.org/lis
> tinfo/squid-users

So in my config file I have:
sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
However I do not see this after compiling and installing.  Has this
gone away in 4?  Thank you.
James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] About to upgrade from 3 to 4

[James Lay]

On Sat, 2018-06-09 at 03:04 +1200, Amos Jeffries wrote:
> On 09/06/18 02:33, James Lay wrote:
> Hey all!
> Topic says itI'm starting to look at doing an upgrade from 3 to
> 4.Any glaring surprises? Doing a transparent forward proxy with
> somepeek/splice for content filtering only (no decryption). Has
> anyone gonethrough an upgrade, and how painful was it, if at all?
> Thank you.
> 
> Which 3.x you are starting from is the issue.
> From 3.5 to 4 should be the same as any of the 3.x single version
> bumps.There is nothing special about v4 from a user perspective.
> Amos___squid-users
> mailing listsquid-users@lists.squid-cache.orghttp://lists.squid-cache
> .org/listinfo/squid-users
> 

Thanks Amos...I'm going from 3.5.2 ☺

James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] About to upgrade from 3 to 4

[James Lay]

Hey all!

Topic says itI'm starting to look at doing an upgrade from 3 to 4. 
Any glaring surprises?  Doing a transparent forward proxy with some
peek/splice for content filtering only (no decryption).  Has anyone
gone through an upgrade, and how painful was it, if at all?  Thank you.

James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Working peek/splice no longer functioning on some sites

[James Lay]

On 2017-11-29 07:29, Amos Jeffries wrote:

On 28/11/17 03:50, James Lay wrote:

On Sun, 2017-11-26 at 09:50 +0200, Alex K wrote:

Perhaps an alternative is to peek only on step1:

acl step1 at_step SslBump1

ssl_bump peek step1
acl allowed_https_sites ssl::server_name_regex 
"/opt/etc/squid/http_url.txt"

ssl_bump splice allowed_https_sites
ssl_bump terminate all


Hrmm...wouldn't that negate the ability to read the cert on step2?



Yes it would.


In layman's terms I'm thinking:
"peek at step1"
"splice acl allow matched sni's"
"peek at step2"
"splice acl allow'd matched certs"
"terminate the rest"

Would that work Amos?



This is essentially what I suggested at the beginning.

Placing splice action and your ACLs on the first ssl_bump line ensures
that at each step if enough details are known to splice it will
happen.

The second line being "peek all" make peek happen at every step for
which it is possible (step 1 and step 2 - not step 3).

"terminate all" being last makes it happen for "all the rest", aka
step 3 if Squid gets that far without splicing.


The only difference is that my suggested way would also allow splicing
the CONNECT if it happens to be presented with a host name in the
authority-URI. Which cannot happen on your proxy unless your port 3128
happens to be intercepting traffic between clients and another proxy.


Ah...ok so this is my lack of understanding then of peek/splice.  Sounds 
like this is what I can try:


ssl_bump splice all
acl allowed_https_sites ssl::server_name_regex 
"/opt/etc/squid/http_url.txt"

ssl_bump splice allowed_https_sites
ssl_bump terminate all

Is that what you're meaning Amos?  Thanks again.

James




BTW please do not use port 3128 for intercept. It is officially
registered for HTTP proxy traffic and so qualifies as "well known".

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Working peek/splice no longer functioning on some sites

[James Lay]

On Sun, 2017-11-26 at 09:50 +0200, Alex K wrote:
> Perhaps an alternative is to peek only on step1:
> 
> acl step1 at_step SslBump1
> 
> ssl_bump peek step1
> acl allowed_https_sites ssl::server_name_regex
> "/opt/etc/squid/http_url.txt"
> ssl_bump splice allowed_https_sites
> ssl_bump terminate all
Hrmm...wouldn't that negate the ability to read the cert on step2?
In layman's terms I'm thinking:
"peek at step1"
"splice acl allow matched sni's"
"peek at step2"
"splice acl allow'd matched certs"
"terminate the rest"
Would that work Amos?

> On Nov 25, 2017 14:46,
>  "James Lay"  wrote:
> > On Sun, 2017-11-26 at 01:33 +1300, Amos Jeffries wrote:
> > > On 26/11/17 00:52, James Lay wrote:
> > > 
> > > > 
> > > > On Sat, 2017-11-25 at 23:48 +1300, Amos Jeffries wrote:
> > > > 
> > > > > 
> > > > > On 25/11/17 08:30, James Lay wrote:
> > > > > 
> > > > > > 
> > > > > > Topic says it...this setup has been working well for a long time, 
> > > > > > but 
> > > > > > now there are some sites that are failing the TLS handshake.  
> > > > > > Here's 
> > > > > > my setup: acl localnet src 192.168.1.0/24 acl SSL_ports port 443 
> > > > > > acl 
> > > > > >  acl SSL_ports port 443 acl 
> > > > > > Safe_ports port 80 acl Safe_ports port 443 acl CONNECT method 
> > > > > > CONNECT 
> > > > > > acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt" 
> > > > > > http_access deny !Safe_ports http_access deny CONNECT !SSL_Ports 
> > > > > > http_access allow SSL_ports http_access allow allowed_http_sites 
> > > > > > http_access deny all ssl_bump peek all acl allowed_https_sites 
> > > > > > ssl::server_name_regex "/opt/etc/squid/http_url.txt" ssl_bump 
> > > > > > splice 
> > > > > > allowed_https_sites ssl_bump terminate all 
> > > > > > 

> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > Because you have "peek all" being performed the transaction MUST pass
> > > > > your regex patterns with both TLS SNI from the client *and* the server
> > > > > certificate SubjectName values. Either one not matching will perform
> > > > > that "terminate all" on the TLS handshake.
> > > > > 
> > > > > 

> > > > 
> > > > 
> > > > Thanks Amos...do you have a suggestion for changing this to match one 
> > > > or 
> > > > the other instead of both?
> > > > 

> > > 
> > > 
> > > Doing the splice check before the peek should do that. First one of the 
> > > server_names data sources to match will then splice and non-matches fall 
> > > through to either peek or terminate if no more peeking possible.
> > > 
> > > Amos
> > > 

> > > > Perfect..I've modded my lines with:
> > > > acl broken_https_sites ssl::server_name_regex 
> > > > "/opt/etc/squid/broken_url.> > txt"
> > ssl_bump splice broken_https_sites
> > ssl_bump peek all
> > acl allowed_https_sites ssl::server_name_regex "/opt/etc/squid/http_url.txt"
> > ssl_bump splice allowed_https_sites
> > ssl_bump terminate all

> > Hopefully that fixes these up.  Another site besides the the one this 
> > thread is fbcdn.net.  Again, these DID work, but something within the last 
> > month has changed...guessing Facebook and Elder Scrolls Online have added 
> > additional TLS security.  Thanks as always Amos.
> > > > James

> > __> > _
> > 
> > squid-users mailing list
> > 
squid-users@lists.squid-cache.org
> > 
http://lists.squid-cache.org/listinfo/squid-users
> > 

> 
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Working peek/splice no longer functioning on some sites

[James Lay]

On Sun, 2017-11-26 at 01:33 +1300, Amos Jeffries wrote:
> On 26/11/17 00:52, James Lay wrote:
> > 
> > On Sat, 2017-11-25 at 23:48 +1300, Amos Jeffries wrote:
> > > 
> > > On 25/11/17 08:30, James Lay wrote:
> > > > 
> > > > Topic says it...this setup has been working well for a long
> > > > time, but 
> > > > now there are some sites that are failing the TLS handshake.
> > > >  Here's 
> > > > my setup: acl localnet src 192.168.1.0/24 acl SSL_ports port
> > > > 443 acl 
> > > > Safe_ports port 80 acl Safe_ports port 443 acl CONNECT method
> > > > CONNECT 
> > > > acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt" 
> > > > http_access deny !Safe_ports http_access deny CONNECT
> > > > !SSL_Ports 
> > > > http_access allow SSL_ports http_access allow
> > > > allowed_http_sites 
> > > > http_access deny all ssl_bump peek all acl allowed_https_sites 
> > > > ssl::server_name_regex "/opt/etc/squid/http_url.txt" ssl_bump
> > > > splice 
> > > > allowed_https_sites ssl_bump terminate all 
> > > 
> > > 
> > > Because you have "peek all" being performed the transaction MUST
> > > pass
> > > your regex patterns with both TLS SNI from the client *and* the
> > > server
> > > certificate SubjectName values. Either one not matching will
> > > perform
> > > that "terminate all" on the TLS handshake.
> > > 
> > Thanks Amos...do you have a suggestion for changing this to match
> > one or 
> > the other instead of both?
> Doing the splice check before the peek should do that. First one of
> the 
> server_names data sources to match will then splice and non-matches
> fall 
> through to either peek or terminate if no more peeking possible.
> 
> Amos
Perfect..I've modded my lines with:
acl broken_https_sites ssl::server_name_regex
"/opt/etc/squid/broken_url.txt"
ssl_bump splice broken_https_sites
ssl_bump peek all
acl allowed_https_sites ssl::server_name_regex
"/opt/etc/squid/http_url.txt"
ssl_bump splice allowed_https_sites
ssl_bump terminate all

Hopefully that fixes these up.  Another site besides the the one this
thread is fbcdn.net.  Again, these DID work, but something within the
last month has changed...guessing Facebook and Elder Scrolls Online
have added additional TLS security.  Thanks as always Amos.
James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Working peek/splice no longer functioning on some sites

[James Lay]

On Sat, 2017-11-25 at 23:48 +1300, Amos Jeffries wrote:
> On 25/11/17 08:30, James Lay wrote:
> > 
> > Topic says it...this setup has been working well for a long time,
> > but 
> > now there are some sites that are failing the TLS handshake.
> >  Here's my 
> > setup:
> > 
> > acl localnet src 192.168.1.0/24
> > acl SSL_ports port 443
> > acl Safe_ports port 80
> > acl Safe_ports port 443
> > acl CONNECT method CONNECT
> > acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt"
> > 
> > http_access deny !Safe_ports
> > http_access deny CONNECT !SSL_Ports
> > http_access allow SSL_ports
> > http_access allow allowed_http_sites
> > http_access deny all
> > 
> > 
> > ssl_bump peek all
> > acl allowed_https_sites ssl::server_name_regex
> > "/opt/etc/squid/http_url.txt"
> > ssl_bump splice allowed_https_sites
> > ssl_bump terminate all
> 
> Because you have "peek all" being performed the transaction MUST
> pass 
> your regex patterns with both TLS SNI from the client *and* the
> server 
> certificate SubjectName values. Either one not matching will perform 
> that "terminate all" on the TLS handshake.
> 
Thanks Amos...do you have a suggestion for changing this to match one
or the other instead of both?
James
> > 
> > 
> > sslproxy_cert_error allow all
> > sslproxy_capath /etc/ssl/certs
> > sslproxy_flags DONT_VERIFY_PEER
> > #sslproxy_options ALL
> > 

> 
> 
> 
> Also, please remove these "*_error allow all" and DONT_VERIFY_PEER lines 
> from your config. They are actively harmful.
> 
> 
> > 
> > 
> > sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
> > sslcrtd_children 5
> > 
> > http_port 3128 intercept
> > https_port 3129 intercept ssl-bump 
> > cert=/opt/etc/squid/certs/sslsplit_ca_cert.pem 
> > cafile=/opt/etc/squid/certs/sslsplit_ca_cert.pem 
> > key=/opt/etc/squid/certs/sslsplit_ca_key.pem 
> > 

> 
> 
> NP: when cert= and key= are in the same file you do not need to specify 
> key=.
> 
> 
> > 
> > generate-host-certificates=on 
> > dynamic_cert_mem_cache_size=4MB sslflags=NO_SESSION_REUSE
> > 
> > 

> 
> 
> It is also best to add "sslflags=NO_DEFAULT_CA" to these ports for 
> Squid-3. That will save a lot of useless memory overheads.
> 
> 
> 
> > 
> > 
> > logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni 
> > %ssl::>cert_subject %>Hs % > 
> > 

> 
> ...
> 
> > 
> > For example, the file http_url.txt contains:
> > 
> > account\.elderscrollsonline\.com
> > \.elderscrollsonline\.com
> > elderscrollsonline\.com
> > 
> > 
> > After doing some reading it looks like this is http2 traffic: 
> > 
https://wiki.squid-cache.org/Features/HTTP2.
> > .
> > 
> > 

> 
> 
> There is no sign of HTTP/2 in that PCAP trace. There is SPDY/3 and 
> HTTP/1.1 being offered by the client.
> 
> 
> If that is from the client to Squid, then please check the matching 
> Squid->server for what is going on there.
> 
> 
> 
> If the problem remains please try Squid-4. It has more advanced TLS 
> capabilities than Squid-3.
> 
> Amos
> ___
> squid-users mailing list
> 
squid-users@lists.squid-cache.org> 
http://lists.squid-cache.org/listinfo/squid-users> 
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Working peek/splice no longer functioning on some sites

[James Lay]

I should add this is squid-3.5.27.  Thank you.
On Fri, 2017-11-24 at 12:30 -0700, James wrote:
> Topic says it...this setup has been working well for a long time, but
> now there are some sites that are failing the TLS handshake.  Here's
> my setup:
> 
> acl localnet src 192.168.1.0/24
> acl SSL_ports port 443
> acl Safe_ports port 80
> acl Safe_ports port 443
> acl CONNECT method CONNECT
> acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt"
> 
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_Ports
> http_access allow SSL_ports
> http_access allow allowed_http_sites
> http_access deny all
> 
> 
> ssl_bump peek all
> acl allowed_https_sites ssl::server_name_regex
> "/opt/etc/squid/http_url.txt"
> ssl_bump splice allowed_https_sites
> ssl_bump terminate all
> 
> sslproxy_cert_error allow all
> sslproxy_capath /etc/ssl/certs
> sslproxy_flags DONT_VERIFY_PEER 
> #sslproxy_options ALL
> 
> sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
> sslcrtd_children 5
> 
> http_port 3128 intercept
> https_port 3129 intercept ssl-bump
> cert=/opt/etc/squid/certs/sslsplit_ca_cert.pem
> cafile=/opt/etc/squid/certs/sslsplit_ca_cert.pem
> key=/opt/etc/squid/certs/sslsplit_ca_key.pem  generate-host-
> certificates=on dynamic_cert_mem_cache_size=4MB
> sslflags=NO_SESSION_REUSE
> 
> 
> logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni
> %ssl::>cert_subject %>Hs %
> 
> access_log syslog:daemon.info mine 
> 
> refresh_pattern -i (cgi-bin|\?)   0   0%  0
> refresh_pattern . 0   20% 4320
> 
> coredump_dir /opt/var 
> 
> For example, the file http_url.txt contains:
> 
> account\.elderscrollsonline\.com
> \.elderscrollsonline\.com
> elderscrollsonline\.com
> 
> 
> After doing some reading it looks like this is http2 traffic:  https:
> //wiki.squid-cache.org/Features/HTTP2.
> 
> Is there anything I can do to continue using squid with more and more
> sites using http2?  Pcap enclosed..thank you.
> 
> James
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Working peek/splice no longer functioning on some sites

[James Lay]

Topic says it...this setup has been working well for a long time, but
now there are some sites that are failing the TLS handshake.  Here's my
setup:

acl localnet src 192.168.1.0/24
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443
acl CONNECT method CONNECT
acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt"

http_access deny !Safe_ports
http_access deny CONNECT !SSL_Ports
http_access allow SSL_ports
http_access allow allowed_http_sites
http_access deny all


ssl_bump peek all
acl allowed_https_sites ssl::server_name_regex
"/opt/etc/squid/http_url.txt"
ssl_bump splice allowed_https_sites
ssl_bump terminate all

sslproxy_cert_error allow all
sslproxy_capath /etc/ssl/certs
sslproxy_flags DONT_VERIFY_PEER 
#sslproxy_options ALL

sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
sslcrtd_children 5

http_port 3128 intercept
https_port 3129 intercept ssl-bump
cert=/opt/etc/squid/certs/sslsplit_ca_cert.pem
cafile=/opt/etc/squid/certs/sslsplit_ca_cert.pem
key=/opt/etc/squid/certs/sslsplit_ca_key.pem  generate-host-
certificates=on dynamic_cert_mem_cache_size=4MB
sslflags=NO_SESSION_REUSE


logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni
%ssl::>cert_subject %>Hs %

error.pcap
Description: application/vnd.tcpdump.pcap
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] [3.5.23]: mozilla.org failed using SSL transparent SSL23_GET_SERVER_HELLO:unknown protocol

[James Lay]

On Mon, 2017-01-23 at 19:54 -0700, Alex Rousskov wrote:
> On 01/23/2017 04:28 PM, David Touzeau wrote:
> > 
> > ssl_bump peek ssl_step1
> > ssl_bump splice all
> > 
> > sslproxy_flags DONT_VERIFY_PEER
> > sslproxy_cert_error allow all
> 
> > 
> > When connecting to mozilla.org using transparent, we receive this
> > error:
> > 
> > * About to connect() to www.mozilla.org port 443 (#0)
> > *   Trying 104.16.41.2...
> > * connected
> > * Connected to www.mozilla.org (104.16.41.2) port 443 (#0)
> > * successfully set certificate verify locations:
> > *   CAfile: none
> >   CApath: /etc/ssl/certs
> > * SSLv3, TLS handshake, Client hello (1):
> > * error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown
> > protocol
> > * Closing connection #0
> > curl: (35) error:140770FC:SSL
> > routines:SSL23_GET_SERVER_HELLO:unknown
> > protocol
> > 
> > 
> > And squid access.log
> > 
> > 1485110919.564  3 192.168.1.236 TAG_NONE/403 6263 CONNECT
> > 104.16.41.2:443 - HIER_NONE/- text/html
> Amos, please note that the above failing test is done using curl, not
> some fancy/non-HTTP/websocket traffic from a "browser".
> 
> David, you need to figure out why Squid is denying the intercepted
> connection attempt (the /403 part in your access.log). Check your
> http_access rules to start with. They were applied to the denied fake
> CONNECT request shown above.
> 
> AFAICT, Squid denies the [fake] CONNECT without bumping the client
> connection to serve a secure error message. That is _not_ what I
> would
> expect because usually Squid bumps to serve errors, even when dealing
> with non-bumping ssl_bump rules. However, I may be misinterpreting
> the
> "unknown protocol" part; perhaps OpenSSL can use that phrase for an
> unsupported TLS version as well? Or perhaps Squid failed to bump the
> client for some reason?
> 
> Capture packets to see what Squid is sending to curl.
> 
> 
> HTH,
> 
> Alex.
> 
> 
Seems like pretty standard stuff:
Jan 23 20:09:04 (squid): 192.168.1.109 - - [23/Jan/2017:20:09:04 -0700]
"CONNECT 104.16.40.2:443 HTTP/1.1" www.mozilla.org - 200 916167
TCP_TUNNEL:ORIGINAL_DST
TLSv12  TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256   secp256r1
James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Peeking on TLS traffic: unknown cipher returned

[James Lay]

Excellent...glad it worked.
James
On Sat, 2016-10-22 at 10:35 -0300, Leandro Barragan wrote:
> Thanks a lot James, compiling Squid 3.5.22 using that specific commit
> of LibreSSL worked as a charm! I no longer have that "unknown cipher
> returned" errors. I do have some errors with a tiny amount of sites,
> but I suppose its because of server-side misconfigurations that
> LibreSSL simply don't like.
> 
> 
> On 21 October 2016 at 13:01, James Lay 
> wrote:
> > 
> > On 2016-10-21 09:58, Leandro Barragan wrote:
> > > 
> > > 
> > > James, thanks for your advice! I've read your email on this list
> > > about
> > > LibreSSL. I tried to compile Squid with LibreSSL in the first
> > > place
> > > because of what you wrote about ChaCha20. But unfortunately, I
> > > couldn't, compilation stopped because of some obscure error.
> > > 
> > > Do you remember what version of squid and libressl you used? BTW
> > > I
> > > tried with OpenSSL 1.0.2g applying the CloudFare ChaCha20 patch,
> > > but
> > > it doesn't work either, same error (unknown cipher)
> > > 
> > > Thanks!
> > > 
> > > On 21 October 2016 at 10:55, James Lay 
> > > wrote:
> > > > 
> > > > 
> > > > On 2016-10-20 20:15, Leandro Barragan wrote:
> > > > > 
> > > > > 
> > > > > 
> > > > > Thanks for your time Alex! I modified my original config
> > > > > based on Amos
> > > > > recommendations, so I think now I have a more consistent peek
> > > > > & splice
> > > > > config:
> > > > > 
> > > > >  acl TF ssl::server_name_regex -i facebook fbcdn twitter
> > > > > reddit
> > > > >  ssl_bump peek all
> > > > >  ssl_bump terminate TF
> > > > >  ssl_bump splice all
> > > > > 
> > > > > As you mentioned, terminate closes the connection, it doesn't
> > > > > serve an
> > > > > error page (when it works, i.e. with reddit and twitter).
> > > > > 
> > > > > I've compiled Squid 3.5.22 using OpenSSL 1.0.2j and I'm
> > > > > having the
> > > > > same exact issue, even with this new config. Based on what
> > > > > you
> > > > > explained, I think it's a OpenSSL problem and Squid can't do
> > > > > anything
> > > > > about it. I have two reasons to believe that:
> > > > > 
> > > > > 1) The "unknown cipher returned" error get's triggered on
> > > > > terminated
> > > > > and non terminated (e.g. microsoft.com) sites, which makes me
> > > > > think it
> > > > > has nothing to do with Squid ACLs.
> > > > > 2) All problematic sites use a new cipher called "ChaCha20"
> > > > > (E.g.
> > > > > TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256according to
> > > > > Qualys
> > > > > online analyzer and TestSSLServer tool)
> > > > > 
> > > > > A lot of sites are using this new cipher. I'm back at the
> > > > > beginning, I
> > > > > will continue trying to compile Squid with patched versions
> > > > > of OpenSSL
> > > > > or LibreSSL.
> > > > > 
> > > > > Thanks!
> > > > > 
> > > > > On 20 October 2016 at 01:01, Alex Rousskov
> > > > >  wrote:
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > On 10/19/2016 12:44 AM, Leandro Barragan wrote:
> > > > > > 
> > > > > > > 
> > > > > > > > 
> > > > > > > > error:140920F8:SSL
> > > > > > > > routines:SSL3_GET_SERVER_HELLO:unknown cipher
> > > > > > > > returned (1/-1/0)
> > > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > I fail to see why is this happening. I only need to peek
> > > > > > > on the
> > > > > > > connection and make a decision based on SNI,
> > > > > > 
> > > > > > 
> > > > > > Please note that "peek and make a decision based on SNI" is
> > > > > > not what
> > > > &g

Re: [squid-users] Peeking on TLS traffic: unknown cipher returned

[James Lay]

On 2016-10-21 09:58, Leandro Barragan wrote:

James, thanks for your advice! I've read your email on this list about
LibreSSL. I tried to compile Squid with LibreSSL in the first place
because of what you wrote about ChaCha20. But unfortunately, I
couldn't, compilation stopped because of some obscure error.

Do you remember what version of squid and libressl you used? BTW I
tried with OpenSSL 1.0.2g applying the CloudFare ChaCha20 patch, but
it doesn't work either, same error (unknown cipher)

Thanks!

On 21 October 2016 at 10:55, James Lay  
wrote:

On 2016-10-20 20:15, Leandro Barragan wrote:


Thanks for your time Alex! I modified my original config based on 
Amos
recommendations, so I think now I have a more consistent peek & 
splice

config:

 acl TF ssl::server_name_regex -i facebook fbcdn twitter reddit
 ssl_bump peek all
 ssl_bump terminate TF
 ssl_bump splice all

As you mentioned, terminate closes the connection, it doesn't serve 
an

error page (when it works, i.e. with reddit and twitter).

I've compiled Squid 3.5.22 using OpenSSL 1.0.2j and I'm having the
same exact issue, even with this new config. Based on what you
explained, I think it's a OpenSSL problem and Squid can't do anything
about it. I have two reasons to believe that:

1) The "unknown cipher returned" error get's triggered on terminated
and non terminated (e.g. microsoft.com) sites, which makes me think 
it

has nothing to do with Squid ACLs.
2) All problematic sites use a new cipher called "ChaCha20" (E.g.
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256according to Qualys
online analyzer and TestSSLServer tool)

A lot of sites are using this new cipher. I'm back at the beginning, 
I
will continue trying to compile Squid with patched versions of 
OpenSSL

or LibreSSL.

Thanks!

On 20 October 2016 at 01:01, Alex Rousskov
 wrote:


On 10/19/2016 12:44 AM, Leandro Barragan wrote:


error:140920F8:SSL routines:SSL3_GET_SERVER_HELLO:unknown cipher
returned (1/-1/0)




I fail to see why is this happening. I only need to peek on the
connection and make a decision based on SNI,



Please note that "peek and make a decision based on SNI" is not what
your configuration tells Squid to do. Your configuration tells Squid 
to

peek during step2, which means making a decision based on server
certificates (and SNI).



I'm not Bumping, so I
don't understand why ciphers matter in my situation.



The ciphers matter because Squid v3 uses OpenSSL parsers during 
step1,
step2, and step3. FWIW, Squid v4 uses OpenSSL parsers during step2 
(a

little) and step3. It is possible to completely remove OpenSSL from
step2 but there is currently no project to do that AFAIK.



ssl_bump peek all step1
ssl_bump peek all step2
ssl_bump terminate face step3
ssl_bump terminate twitter step3
ssl_bump splice all step3



BTW, "step1", "step2", and "step3" ACLs do nothing useful in the 
above
config. You can safely remove them to arrive at the equivalent 
ssl_bump

configuration.


On 10/19/2016 07:42 AM, Amos Jeffries wrote:


Terminate means impersonating the server and responding to the 
client

with an HTTPS error page.



Terminate means "close client and server connections immediately". 
The
problem is not with the terminate action but with peeking (which 
relies

on OpenSSL, especially during step2, especially in Squid v3).


HTH,

Alex.



FWIW I've had great success with the git version of libressl and using 
the

below:

./configure --prefix=/opt/libressl

and for squid:

./configure --prefix=/opt --with-openssl=/opt/libressl --enable-ssl
--enable-ssl-crtd

James


I'm currently using squid-3.5.22 and using the below git for libressl:

commit b7ba692f72f232602efb3e720ab0510406bae69c
Author: Brent Cook 
Date:   Wed Sep 14 23:40:10 2016 -0500

What's the error you're getting when you try and compile?

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Peeking on TLS traffic: unknown cipher returned

[James Lay]

On 2016-10-20 20:15, Leandro Barragan wrote:

Thanks for your time Alex! I modified my original config based on Amos
recommendations, so I think now I have a more consistent peek & splice
config:

 acl TF ssl::server_name_regex -i facebook fbcdn twitter reddit
 ssl_bump peek all
 ssl_bump terminate TF
 ssl_bump splice all

As you mentioned, terminate closes the connection, it doesn't serve an
error page (when it works, i.e. with reddit and twitter).

I've compiled Squid 3.5.22 using OpenSSL 1.0.2j and I'm having the
same exact issue, even with this new config. Based on what you
explained, I think it's a OpenSSL problem and Squid can't do anything
about it. I have two reasons to believe that:

1) The "unknown cipher returned" error get's triggered on terminated
and non terminated (e.g. microsoft.com) sites, which makes me think it
has nothing to do with Squid ACLs.
2) All problematic sites use a new cipher called "ChaCha20" (E.g.
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256according to Qualys
online analyzer and TestSSLServer tool)

A lot of sites are using this new cipher. I'm back at the beginning, I
will continue trying to compile Squid with patched versions of OpenSSL
or LibreSSL.

Thanks!

On 20 October 2016 at 01:01, Alex Rousskov
 wrote:

On 10/19/2016 12:44 AM, Leandro Barragan wrote:

error:140920F8:SSL routines:SSL3_GET_SERVER_HELLO:unknown cipher 
returned (1/-1/0)



I fail to see why is this happening. I only need to peek on the
connection and make a decision based on SNI,


Please note that "peek and make a decision based on SNI" is not what
your configuration tells Squid to do. Your configuration tells Squid 
to

peek during step2, which means making a decision based on server
certificates (and SNI).



I'm not Bumping, so I
don't understand why ciphers matter in my situation.


The ciphers matter because Squid v3 uses OpenSSL parsers during step1,
step2, and step3. FWIW, Squid v4 uses OpenSSL parsers during step2 (a
little) and step3. It is possible to completely remove OpenSSL from
step2 but there is currently no project to do that AFAIK.



ssl_bump peek all step1
ssl_bump peek all step2
ssl_bump terminate face step3
ssl_bump terminate twitter step3
ssl_bump splice all step3


BTW, "step1", "step2", and "step3" ACLs do nothing useful in the above
config. You can safely remove them to arrive at the equivalent 
ssl_bump

configuration.


On 10/19/2016 07:42 AM, Amos Jeffries wrote:

Terminate means impersonating the server and responding to the client
with an HTTPS error page.


Terminate means "close client and server connections immediately". The
problem is not with the terminate action but with peeking (which 
relies

on OpenSSL, especially during step2, especially in Squid v3).


HTH,

Alex.


FWIW I've had great success with the git version of libressl and using 
the below:


./configure --prefix=/opt/libressl

and for squid:

./configure --prefix=/opt --with-openssl=/opt/libressl --enable-ssl 
--enable-ssl-crtd


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Additional ecap/icap questions

[James Lay]

On 2016-10-17 15:01, Alex Rousskov wrote:

On 10/17/2016 11:51 AM, James Lay wrote:

Here's what I'm wanting to accomplish and it's been proving a 
challenge:

 Detect keywords (think DLP maybe) in http/https flows.  I've got ecap
and icap compiled in and working.  My challenges:

a)with icap, it appears that the filter content adapters only work 
with

responses, not requestsI need both.


It depends on the ICAP service. Some work with requests, some with
responses, some with both kinds of messages.



I'm specifically looking at 
http://c-icap.sourceforge.net/c-icap-modules.conf-0.4.x.html#tag_srv_content_filtering_MaxBodyData. 
 This looks like it will do what I need, but as from my previous posts, 
it appears it only works with RESPMOD, not requests.


b)with icap, if I use the "echo" adapter I can see everything on the 
lo

interface, but decoding it has proven fruitless for me


If you are trying to manually decode ICAP traffic on a loopback
interface, please clarify what you are trying to accomplish with that.


I'm trying to match text in a stream, somehow.  Either with the above 
icap method, which would appear to be designed for this purpose, but 
only responses not request, or by decoding the stream and sending the 
decoded traffic to an interface where an IDS can match content.  In 
short, if someone drops an f-bomb in a chat let's say, I want it known.





c)with ecap, I configured per
http://wiki.squid-cache.org/ConfigExamples/ContentAdaptation/eCAP,
but I'm confused on the ecap_service line..examples show
"ecap://www.vigos.com/ecap_gzip", but what do I put in?


Just like with ICAP, you configure an eCAP adapter/service that you 
want

to use. I do not know whether it exists or needs to be written. For
example, if you want to find viruses, you can use an eCAP ClamAV 
adapter.




I thought I
didn't need a service for ecap..do I point this to localhost or 
something?


With eCAP, you do not need a server. With both ICAP and eCAP you need a
service or "adapter" that does whatever you want to do. ICAP and eCAP
are just protocols/API -- they cannot do anything useful on their own.

The eCAP service URI is just an identifier. It does not "point" to any
specific location. It is only used to distinguish one loaded eCAP
service from another loaded eCAP service.


Overall, you need some software that will "detect keywords". That
detection is not going to happen magically on its own. ICAP and eCAP 
are

just two ways to get the HTTP messages to that software. Some call that
_kind_ of software "ICAP service", "ICAP server plugin", "eCAP 
service",

"eCAP adapter", etc. You need to find or write a specific
service/plugin/adapter/etc. that does keyword detection.

Alex.


Thanks AlexI can't imagine that I'm the only one wanting to do this 
purely with open source software, but it appears that way.


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Additional ecap/icap questions

[James Lay]

Well this has been a pretty amazing bit of learning that's for sure.  
Here's what I'm wanting to accomplish and it's been proving a challenge: 
 Detect keywords (think DLP maybe) in http/https flows.  I've got ecap 
and icap compiled in and working.  My challenges:


a)with icap, it appears that the filter content adapters only work with 
responses, not requestsI need both.
b)with icap, if I use the "echo" adapter I can see everything on the lo 
interface, but decoding it has proven fruitless for me
c)with ecap, I configured per 
http://wiki.squid-cache.org/ConfigExamples/ContentAdaptation/eCAP, but 
I'm confused on the ecap_service line..examples show 
"ecap://www.vigos.com/ecap_gzip", but what do I put in?  I thought I 
didn't need a service for ecap..do I point this to localhost or 
something?


Anyway thank you.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 2.5.20 fails to compile with ecap

[James Lay]

On 2016-10-11 10:52, Alex Rousskov wrote:

On 10/11/2016 08:45 AM, James Lay wrote:

Can you point me in the right direction on where to tell squid that
libecap lives in /opt/ecap?


This is not my area of expertise, but if ./configure --enable-ecap does
not work "as is", then you may need to set PKG_CONFIG_PATH
appropriately. For example:

  export PKG_CONFIG_PATH=...
  ./configure --enable-ecap ...

Alex.


Last word on this...config line:

./configure --prefix=/opt --with-openssl=/opt/libressl --enable-ssl 
--enable-ssl-crtd --enable-linux-netfilter 
--enable-follow-x-forwarded-for --with-large-files 
--sysconfdir=/opt/etc/squid --enable-xternal-acl-helpers=none 
--enable-ecap


Symlinking the libecap.pc, /opt/ecap/lib/pkgconfig/libecap.pc to 
/usr/lib/x86_64-linux-gnu/pkgconfig/ did the trick...thanks so much!


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 2.5.20 fails to compile with ecap

[James Lay]

On 2016-10-11 08:42, Alex Rousskov wrote:

On 10/11/2016 06:54 AM, James Lay wrote:


EXT_LIBECAP_CFLAGS="-I/opt/ecap/include"
EXT_LIBECAP_LIBS="-L/opt/ecap/lib" ./configure --prefix=/opt
--with-openssl=/opt/libressl --enable-ssl --enable-ssl-crtd
--enable-linux-netfilter --enable-follow-x-forwarded-for
--with-large-files --sysconfdir=/opt/etc/squid
--enable-xternal-acl-helpers=none --enable-ecap


Your Squid executable is not linked with libecap, resulting in 
undefined

references to libecap symbols:

/bin/bash ../libtool  --tag=CXX   --mode=link g++ -Wall 
-Wpointer-arith

-Wwrite-strings -Wcomments -Wshadow -Woverloaded-virtual -Werror -pipe
-D_REENTRANT -m64   -g -O2 -march=native -std=c++11 -export-dynamic
-dlopen force -m64 -g -o squid





adaptation/.libs/libadaptation.a(libsquid_ecap_la-Host.o):
undefined reference to `libecap::headerTransferEncoding'


I am not sure why Your Squid executable is not linked with libecap, but
I suspect that manually setting EXT_LIBECAP_CFLAGS and EXT_LIBECAP_LIBS
confuses ./configure. You should not set those variables manually and
let Squid compute them automatically instead. You may need to set
PKG_CONFIG_PATH if Squid cannot find libecap.pc without it.


HTH,

Alex.
P.S. I assume you meant to type "3.5.20" in the Subject


Thanks Alex...and yes indeed...meant 3.5.20.  Can you point me in the 
right direction on where to tell squid that libecap lives in /opt/ecap?  
I'll give the pkg-config a go as well.


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Squid 2.5.20 fails to compile with ecap

[James Lay]

Pretty much topic..sorry for the wall of text here.  Config'd with:

EXT_LIBECAP_CFLAGS="-I/opt/ecap/include" EXT_LIBECAP_LIBS="-
L/opt/ecap/lib" ./configure --prefix=/opt --with-openssl=/opt/libressl
--enable-ssl --enable-ssl-crtd --enable-linux-netfilter --enable-
follow-x-forwarded-for --with-large-files --sysconfdir=/opt/etc/squid
--enable-xternal-acl-helpers=none --enable-ecap

Thank you.

James

/bin/bash ../libtool  --tag=CXX   --mode=link g++ -Wall -Wpointer-arith 
-Wwrite-strings -Wcomments -Wshadow -Woverloaded-virtual -Werror -pipe
-D_REENTRANT -m64   -g -O2 -march=native -std=c++11 -export-dynamic
-dlopen force -m64 -g -o squid AclRegs.o AuthReg.o AccessLogEntry.o
AsyncEngine.o YesNoNone.o cache_cf.o CacheDigest.o cache_manager.o
carp.o cbdata.o ChunkedCodingParser.o client_db.o client_side.o
client_side_reply.o client_side_request.o BodyPipe.o clientStream.o
CollapsedForwarding.o CompletionDispatcher.o ConfigOption.o
ConfigParser.o CpuAffinity.o CpuAffinityMap.o CpuAffinitySet.o
debug.o  disk.o DiskIO/DiskIOModule.o DiskIO/ReadRequest.o
DiskIO/WriteRequest.o dlink.o dns_internal.o DnsLookupDetails.o
errorpage.o ETag.o event.o EventLoop.o external_acl.o
ExternalACLEntry.o FadingCounter.o fatal.o fd.o fde.o filemap.o
fqdncache.o FwdState.o gopher.o helper.o htcp.o http.o HttpHdrCc.o
HttpHdrRange.o HttpHdrSc.o HttpHdrScTarget.o HttpHdrContRange.o
HttpHeader.o HttpHeaderTools.o HttpBody.o HttpMsg.o HttpParser.o
HttpReply.o RequestFlags.o HttpRequest.o HttpRequestMethod.o icp_v2.o
icp_v3.o int.o internal.o ipc.o ipcache.o  SquidList.o main.o
MasterXaction.o mem.o mem_node.o MemBuf.o MemObject.o mime.o
mime_header.o multicast.o neighbors.o Notes.o Packer.o
Parsing.o  pconn.o peer_digest.o peer_proxy_negotiate_auth.o
peer_select.o peer_sourcehash.o peer_userhash.o PeerPoolMgr.o
redirect.o refresh.o RemovalPolicy.o send-announce.o MemBlob.o SBuf.o
SBufExceptions.o SBufDetailedStats.o SBufStatsAction.o snmp_core.o
snmp_agent.o SquidMath.o SquidNew.o stat.o StatCounters.o StatHist.o
String.o StrList.o stmem.o store.o StoreFileSystem.o store_io.o
StoreIOState.o store_client.o store_digest.o store_dir.o
store_key_md5.o store_log.o store_rebuild.o store_swapin.o
store_swapmeta.o store_swapout.o StoreMetaUnpacker.o StoreMeta.o
StoreMetaMD5.o StoreMetaSTD.o StoreMetaSTDLFS.o StoreMetaURL.o
StoreMetaVary.o StoreStats.o StoreSwapLogData.o SwapDir.o Transients.o
MemStore.o time.o tools.o tunnel.o unlinkd.o url.o urn.o wccp.o wccp2.o
whois.o wordlist.o   LoadableModule.o LoadableModules.o
DiskIO/DiskIOModules_gen.o err_type.o err_detail_type.o globals.o
hier_code.o icp_opcode.o LogTags.o lookup_t.o repl_modules.o
swap_log_op.o auth/libacls.la ident/libident.la acl/libacls.la
acl/libstate.la auth/libauth.la libAIO.a libBlocking.a libDiskDaemon.a
libDiskThreads.a libIpcIo.a libMmapped.a acl/libapi.la base/libbase.la
libsquid.la ip/libip.la fs/libfs.la ssl/libsslsquid.la
ssl/libsslutil.la ipc/libipc.la mgr/libmgr.la anyp/libanyp.la
comm/libcomm.la eui/libeui.la helper/libhelper.la http/libsquid-http.la 
icmp/libicmp.la icmp/libicmp-core.la log/liblog.la format/libformat.la
clients/libclients.la servers/libservers.la ftp/libftp.la  DiskIO/AIO/A
IODiskIOModule.o DiskIO/Blocking/BlockingDiskIOModule.o
DiskIO/DiskDaemon/DiskDaemonDiskIOModule.o
DiskIO/DiskThreads/DiskThreadsDiskIOModule.o
DiskIO/IpcIo/IpcIoDiskIOModule.o DiskIO/Mmapped/MmappedDiskIOModule.o
repl/liblru.a -lrt -lpthread  -
lcrypt  adaptation/libadaptation.la  snmp/libsnmp.la
../lib/snmplib/libsnmplib.la parser/libsquid-parser.la
../lib/libmisccontainers.la ../lib/libmiscencoding.la
../lib/libmiscutil.la -L/opt/libressl/lib -lssl -lcrypto  -
lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err../compat/libcompat-
squid.la  -lm -lnsl -lresolv -lcap -lrt -L.. -lltdl 
libtool: link: rm -f .libs/squid.nm .libs/squid.nmS .libs/squid.nmT
libtool: link: rm -f ".libs/squid.nmI"
libtool: link: (cd .libs && gcc -m64 -Wall -g -O2 -c -fno-builtin
"squidS.c")
libtool: link: rm -f ".libs/squidS.c" ".libs/squid.nm"
".libs/squid.nmS" ".libs/squid.nmT" ".libs/squid.nmI"
libtool: link: g++ -Wall -Wpointer-arith -Wwrite-strings -Wcomments
-Wshadow -Woverloaded-virtual -Werror -pipe -D_REENTRANT -m64 -g -O2
-march=native -std=c++11 .libs/squidS.o -m64 -g -o squid AclRegs.o
AuthReg.o AccessLogEntry.o AsyncEngine.o YesNoNone.o cache_cf.o
CacheDigest.o cache_manager.o carp.o cbdata.o ChunkedCodingParser.o
client_db.o client_side.o client_side_reply.o client_side_request.o
BodyPipe.o clientStream.o CollapsedForwarding.o CompletionDispatcher.o
ConfigOption.o ConfigParser.o CpuAffinity.o CpuAffinityMap.o
CpuAffinitySet.o debug.o disk.o DiskIO/DiskIOModule.o
DiskIO/ReadRequest.o DiskIO/WriteRequest.o dlink.o dns_internal.o
DnsLookupDetails.o errorpage.o ETag.o event.o EventLoop.o
external_acl.o ExternalACLEntry.o FadingCounter.o fatal.o fd.o fde.o
filemap.o fqdncache.o FwdState.o gopher.o helper.o htcp.o http.o
HttpHdrCc.o HttpHdrRange.o HttpHdrSc.o HttpHdrScTarget.o
HttpHdrContRange.

Re: [squid-users] ICAP question

[James Lay]

On Mon, 2016-10-10 at 12:28 +0300, Eliezer Croitoru wrote:
> I am not sure but it seems to me like I might not understood squid
> ACLS
> right but yet to be 100% about it.
> acl PERMIT_REQUESTS type REQMOD RESPMOD
> icap_access allow localhost PERMIT_REQUESTS
> icap_access deny all
> The acl as far as I know doesn’t have any type such as ICAP request mode.
> Am I right?
> 
> Eliezer
> 
> 
> Eliezer Croitoru <http://ngtech.co.il/lmgtfy/> 
> > 
> Linux System Administrator
> Mobile+WhatsApp: +972-5-28704261
> Email: elie...@ngtech.co.il

I am not sure...I am going by the below:
http://wiki.squid-cache.org/ConfigExamples/ContentAdaptation/C-ICAP
James
>  
> 
> From: squid-users [mailto:squid-users-boun...@lists.squid-cache.org] On
> ] On
> Behalf Of James Lay
> Sent: Sunday, October 9, 2016 8:03 PM
> To: squid-users
> Subject: [squid-users] ICAP question
> 
> Trying to just get some content filtering working and I'm running into the
> below:
> 
> WARNING: Squid is configured to use ICAP method REQMOD for service
> icap://localhost:1344/srv_cfg_filter but OPTIONS response declares the
> methods are RESPMOD 
> 
> Here's the icap snippet from squid.conf:
> 
> icap_enable on
> icap_send_client_ip on
> icap_persistent_connections on
> icap_service srv_cfg_filter_req reqmod_precache
> icap://localhost:1344/srv_cfg_filter bypass=on
> adaptation_access srv_cfg_filter_req allow all
> icap_service srv_cfg_filter_resp respmod_precache
> icap://localhost:1344/srv_cfg_filter bypass=off
> adaptation_access srv_cfg_filter_resp allow all
> 
> interesting c-icap.conf bits:
> 
> ModulesDir /opt/icap/lib/c_icap
> ServicesDir /opt/icap/lib/c_icap
> acl localhost src 127.0.0.1/255.255.255.255
> acl PERMIT_REQUESTS type REQMOD RESPMOD
> icap_access allow localhost PERMIT_REQUESTS
> icap_access deny all
> Include srv_content_filtering.conf
> 
> lastly, srv_content_filtering.conf:
> 
> Service srv_cfg_filter srv_content_filtering.so
> srv_content_filtering.Match default body /(test)/ig score=5
> LogFormat mySrvContentFiltering "%tl, %>a %im %is %huo  [Scores:
> %{srv_content_filtering:scores}Sa] [ActionFilter:
> %{srv_content_filtering:action_filter}Sa] [Action:
> %{srv_content_filtering:action}Sa]"
> 
> not sure why I can't seem to get this to fly...any assistance would be
> appreciated...thank you.
> 
> James
> 
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] ICAP question

[James Lay]

On Sun, 2016-10-09 at 12:43 -0600, Alex Rousskov wrote:
> On 10/09/2016 11:02 AM, James Lay wrote:
> 
> > 
> > WARNING: Squid is configured to use ICAP method REQMOD for service
> > icap://localhost:1344/srv_cfg_filter but OPTIONS response declares
> > the
> > methods are RESPMOD
> If your srv_content_filtering.so service does not need to see HTTP
> requests, then you can remove srv_cfg_filter_req from your Squid
> configuration.
> 
> If your srv_content_filtering.so service needs to see both HTTP
> requests
> and responses, then you have two options, in no particular order:
> 
> A) Tell c-icap and/or srv_content_filtering.so to send a "Methods:
> REQMOD,RESPMOD" ICAP response header field in OPTIONS response.
> Sorry, I
> do not know how to do that in c-icap and even whether that is
> actually
> possible with that software. Please note that using one service URI
> for
> two modes is not uncommon in the ICAP world, but violates the
> following
> ICAP RFC 3507 MUST:
> 
>   Each service should have a distinct URI
>   and support only one method in addition to OPTIONS
> 
> B) Use different ICAP service URIs for different services (REQMOD and
> RESPMOD) and configure each service appropriately on both Squid and
> c-icap side. This is what RFC 3507 wants you to do. For example, some
> ICAP servers and services would allow you to use these URIs:
> 
>   * for REQMOD: icap://localhost:1344/srv_cfg_filter?mode=REQMOD
>   * for RESPMOD: icap://localhost:1344/srv_cfg_filter?mode=RESPMOD
> 
> 
> IIRC, Squid will try to use your service in both modes despite that
> WARNING. However, I do not know whether c-icap and that service
> itself
> will be happy about receiving REQMOD requests.
> 
> 
> HTH,
> 
> Alex.
> 
> 
> 
> > 
> > Here's the icap snippet from squid.conf:
> > 
> > icap_enable on
> > icap_send_client_ip on
> > icap_persistent_connections on
> > icap_service srv_cfg_filter_req reqmod_precache
> > icap://localhost:1344/srv_cfg_filter bypass=on
> > adaptation_access srv_cfg_filter_req allow all
> > icap_service srv_cfg_filter_resp respmod_precache
> > icap://localhost:1344/srv_cfg_filter bypass=off
> > adaptation_access srv_cfg_filter_resp allow all
> > 
> > interesting c-icap.conf bits:
> > 
> > ModulesDir /opt/icap/lib/c_icap
> > ServicesDir /opt/icap/lib/c_icap
> > acl localhost src 127.0.0.1/255.255.255.255
> > acl PERMIT_REQUESTS type REQMOD RESPMOD
> > icap_access allow localhost PERMIT_REQUESTS
> > icap_access deny all
> > Include srv_content_filtering.conf
> > 
> > lastly, srv_content_filtering.conf:
> > 
> > Service srv_cfg_filter srv_content_filtering.so
> > srv_content_filtering.Match default body /(test)/ig score=5
> > LogFormat mySrvContentFiltering "%tl, %>a %im %is %huo  [Scores:
> > %{srv_content_filtering:scores}Sa] [ActionFilter:
> > %{srv_content_filtering:action_filter}Sa] [Action:
> > %{srv_content_filtering:action}Sa]"
> > 
> > not sure why I can't seem to get this to fly...any assistance would
> > be
> > appreciated...thank you.
Thank you AlexI followed:
http://wiki.squid-cache.org/ConfigExamples/ContentAdaptation/C-ICAP
As best I could, but it looks like that adapter needs something
different.  I'll report my results here once I get it fixed.
James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] ICAP question

[James Lay]

Trying to just get some content filtering working and I'm running into
the below:

WARNING: Squid is configured to use ICAP method REQMOD for service
icap://localhost:1344/srv_cfg_filter but OPTIONS response declares the
methods are RESPMOD 

Here's the icap snippet from squid.conf:

icap_enable on
icap_send_client_ip on
icap_persistent_connections on
icap_service srv_cfg_filter_req reqmod_precache
icap://localhost:1344/srv_cfg_filter bypass=on
adaptation_access srv_cfg_filter_req allow all
icap_service srv_cfg_filter_resp respmod_precache
icap://localhost:1344/srv_cfg_filter bypass=off
adaptation_access srv_cfg_filter_resp allow all

interesting c-icap.conf bits:

ModulesDir /opt/icap/lib/c_icap
ServicesDir /opt/icap/lib/c_icap
acl localhost src 127.0.0.1/255.255.255.255
acl PERMIT_REQUESTS type REQMOD RESPMOD
icap_access allow localhost PERMIT_REQUESTS
icap_access deny all
Include srv_content_filtering.conf

lastly, srv_content_filtering.conf:

Service srv_cfg_filter srv_content_filtering.so
srv_content_filtering.Match default body /(test)/ig score=5
LogFormat mySrvContentFiltering "%tl, %>a %im %is %huo  [Scores:
%{srv_content_filtering:scores}Sa] [ActionFilter:
%{srv_content_filtering:action_filter}Sa] [Action:
%{srv_content_filtering:action}Sa]"

not sure why I can't seem to get this to fly...any assistance would be
appreciated...thank you.

James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Clarification on icap

[James Lay]

On 2016-09-26 10:40, Alex Rousskov wrote:

On 09/26/2016 08:55 AM, James Lay wrote:
any recommended open source ICAP/eCAP services that squid works well 
with?


You do not need an ICAP/eCAP service that Squid works well with. You
need an ICAP/eCAP service that integrates with your IDS. All production
ICAP/eCAP services are doing some specific adaptation (e.g., downgrade
image quality) or integrate with some specific adaptation library 
(e.g.,

ClamAV). They are useless to you unless they integrate with your IDS.

If there is no existing service that integrates with your IDS, you 
would
have to write and support one. If you go the ICAP route, many plug 
their

custom ICAP services into the free c-icap ICAP server. If you decide
going the eCAP route, then you do not need a server (eCAP is a
library/API, not a communication protocol, so your custom code plugs
directly into the host application such as Squid).

  http://wiki.squid-cache.org/SquidFaq/ContentAdaptation

Alex.


Excellent...thanks so much Alex!

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Clarification on icap

[James Lay]

On 2016-09-26 08:52, Alex Rousskov wrote:

On 09/26/2016 08:43 AM, James Lay wrote:

So, from what I've read, it appears that
squid sends the data to a listening ICAP/eCAP service, which in turn 
the

IDS can access, depending on the IDS...is that about right?


Not exactly.

Yes, Squid sends the message to the adaptation service ("listening" is
not a good verb for eCAP because, unlike ICAP, eCAP services are not
network services but "plugins" or libraries).

No, the IDS does not normally come to the adaptation service for
messages. Normally, the adaptation service itself needs to give IDS the
data. How that is done depends on the IDS interfaces, of course.

On a logical level, the message is transmitted using the following 
chain:


  Squid -> adaptation service -> IDS

And the allow/block decision (if any) is transmitted in the opposite
direction:

  Squid <- adaptation service <- IDS

Alex.


Beautiful...just what I needed.  Last question...any recommended open 
source ICAP/eCAP services that squid works well with?  Thanks again.


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Clarification on icap

[James Lay]

On 2016-09-26 08:30, Alex Rousskov wrote:

On 09/26/2016 05:41 AM, James Lay wrote:

So I'm going to try and get some visibility into tls traffic.  Not
concerned with the sslbumping of the traffic, but what I DON'T know 
what

to do is what to do with the traffic once it's decrypted.  This squid
machine runs IDS software as well, so my hope was to have the IDS
software listen to traffic that'd decrypted, but for the life of me 
I'm
not sure where to start.  Does squid pipe out a stream?  Or does the 
IDS

listen to a different "interface"?  Is this where ICAP comes in?


Squid-IDS integration is mostly independent from SslBump issues -- you
integrate traffic analysis of plain and secure traffic similarly. Your
options depend on IDS interfaces:

1. If IDS is content with passively looking at something Squid can log
(after the transaction is completed), then give IDS the logs (see
access_log and logformat directives). This is what Amos recommended in
his response. It is the best option if your IDS can use it.

2. If IDS is content with reacting to something Squid can log while
processing a message, then write or purchase a custom external ACL
script. External ACL input can be customized just like the access log.

3. If IDS needs access to message bodies, then use an ICAP or eCAP
service to give IDS whole messages. You may have to write or purchase
that service. How that service is going to give messages to IDS depends
on IDS interfaces. Some IDSes have APIs while others listen to raw
traffic (that a service can emulate and emit).


HTH,

Alex.


Ah..there's the rub Alex thanks.  I already have rock solid access 
controls with squids acl's and great logging.  Now I find that I need to 
inspect the actual content, i.e. message bodies.  So cool..I'm on the 
right track for ICAP or eCAP.  So, from what I've read, it appears that 
squid sends the data to a listening ICAP/eCAP service, which in turn the 
IDS can access, depending on the IDS...is that about right?


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Clarification on icap

[James Lay]

On 2016-09-26 06:50, Amos Jeffries wrote:

On 27/09/2016 12:41 a.m., James Lay wrote:

Hey all,

So I'm going to try and get some visibility into tls traffic.  Not
concerned with the sslbumping of the traffic, but what I DON'T know
what to do is what to do with the traffic once it's decrypted.  This
squid machine runs IDS software as well, so my hope was to have the 
IDS
software listen to traffic that'd decrypted, but for the life of me 
I'm

not sure where to start.  Does squid pipe out a stream?  Or does the
IDS listen to a different "interface"?  Is this where ICAP comes in?


Keeping it secure is of high importance. So ensuring that any
connections it goes over are securely encrypted somehow is important.

The best way to ensure data security is not to transmit it. What data
does the IDS actually need? and can you 'log' only those details to a
private pipe/socket the IDS is reading?

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users



Ah Amos...always vigilant...thank you.  Yea those are the questions I'm 
asking really...how can squid "present" the unencrypted data?  Pipe to a 
socket?  Log to a file?  Dump to a pcap?  As soon as I know the options 
of how squid can manipulate a session during bumping/decrypting, I'll be 
able to see if snort/suricata can "listen" to the data.  Does that make 
sense?  Thanks as always Amos.


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Clarification on icap

[James Lay]

Hey all,

So I'm going to try and get some visibility into tls traffic.  Not
concerned with the sslbumping of the traffic, but what I DON'T know
what to do is what to do with the traffic once it's decrypted.  This
squid machine runs IDS software as well, so my hope was to have the IDS
software listen to traffic that'd decrypted, but for the life of me I'm
not sure where to start.  Does squid pipe out a stream?  Or does the
IDS listen to a different "interface"?  Is this where ICAP comes in? 

Thanks for any assistance...just starting out so thought this would be
the best place to start.

James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 3.5.20 compile issue

[James Lay]

On Tue, 2016-09-20 at 11:05 +0930, LYMN wrote:
> On Mon, Sep 19, 2016 at 07:20:14PM -0600, James Lay wrote:
> > 
> > 
> > Well last word on this...squid starts but dies with:
> > /squid: symbol lookup error: ./squid: undefined symbol:
> > SSL_set_alpn_protos
> > So at this point I'll just go back to linking to libressl.  Thanks
> > all.
> > 
> 
> What does a "ldd squid" output?  You have built your openssl
> libraries
> to a non-standard place so perhaps squid cannot find them at run
> time?
> If this was the case then you either need to use LD_LIBRARY_PATH at
> run
> time or set LDFLAGS="-L/opt/openssl/lib -Wl,-R/opt/openssl/lib" at
> squid
> configure time.
> 
> 
Woo hoo!  Success!  Looks like this works with openssl-1.0.1.  Thanks
much all!
James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 3.5.20 compile issue

[James Lay]

On Mon, 2016-09-19 at 18:44 -0600, James Lay wrote:
> On Tue, 2016-09-20 at 10:12 +0930, LYMN wrote:
> > On Mon, Sep 19, 2016 at 06:37:44PM -0600, Alex Rousskov wrote:
> > > 
> > > On 09/19/2016 06:22 PM, James Lay wrote:
> > > > 
> > > > Ok so this is with the 1.0.2 branch of openssl:
> > > > 
> > > > dso_dlfcn.c:(.text+0x11): undefined reference to `dlopen'
> > > > dso_dlfcn.c:(.text+0x24): undefined reference to `dlsym'
> > > > dso_dlfcn.c:(.text+0x2f): undefined reference to `dlclose'
> > > You can probably force you way through this by linking with more
> > > system
> > > libraries, but I do not know exactly which ones you need. This
> > > smells
> > > like an environment or build configuration problem to me -- the
> > > linker
> > > does not know that your OpenSSL library depends on another system
> > > library that provides those [dynamic linking] functions.
> > > 
> > At a guess add this to the libraries list after openssl: -ldl
> > 
> Thank you...where would I add that to?  My config line?  Here it is:
> 
> ./configure --prefix=/opt --with-openssl=/opt/openssl --enable-ssl --
> enable-ssl-crtd --enable-linux-netfilter --enable-follow-x-forwarded-
> for --with-large-files --sysconfdir=/opt/etc/squid --enable-external-
> acl-helpers=none
> 
> James
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
Well last word on this...squid starts but dies with:
/squid: symbol lookup error: ./squid: undefined symbol:
SSL_set_alpn_protos
So at this point I'll just go back to linking to libressl.  Thanks all.
James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 3.5.20 compile issue

[James Lay]

On Tue, 2016-09-20 at 10:26 +0930, LYMN wrote:
> On Mon, Sep 19, 2016 at 06:44:38PM -0600, James Lay wrote:
> > 
> > > 
> > > > 
> > > > 
> > > At a guess add this to the libraries list after openssl: -ldl
> > > 
> > Thank you...where would I add that to?  My config line?  Here it
> > is:
> > ./configure --prefix=/opt --with-openssl=/opt/openssl --enable-ssl
> > --
> > enable-ssl-crtd --enable-linux-netfilter --enable-follow-x-
> > forwarded-
> > for --with-large-files --sysconfdir=/opt/etc/squid --enable-
> > external-
> > acl-helpers=none
> > 
> try setting LIBS="-ldl" in the environment before you do the
> configure.
> 
> 
Thanks again Brett.  Turns out you have to add the "shared" option:
./Configure enable-chacha enable-poly1305 --prefix=/opt/openssl shared
linux-x86_64
which you knowkind of makes sense because it's like...a SHARED
library 8-|  Yugh..I hate being dumb.
James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 3.5.20 compile issue

[James Lay]

On Tue, 2016-09-20 at 10:12 +0930, LYMN wrote:
> On Mon, Sep 19, 2016 at 06:37:44PM -0600, Alex Rousskov wrote:
> > 
> > On 09/19/2016 06:22 PM, James Lay wrote:
> > > 
> > > Ok so this is with the 1.0.2 branch of openssl:
> > > 
> > > dso_dlfcn.c:(.text+0x11): undefined reference to `dlopen'
> > > dso_dlfcn.c:(.text+0x24): undefined reference to `dlsym'
> > > dso_dlfcn.c:(.text+0x2f): undefined reference to `dlclose'
> > You can probably force you way through this by linking with more
> > system
> > libraries, but I do not know exactly which ones you need. This
> > smells
> > like an environment or build configuration problem to me -- the
> > linker
> > does not know that your OpenSSL library depends on another system
> > library that provides those [dynamic linking] functions.
> > 
> At a guess add this to the libraries list after openssl: -ldl
> 
Thank you...where would I add that to?  My config line?  Here it is:
./configure --prefix=/opt --with-openssl=/opt/openssl --enable-ssl --
enable-ssl-crtd --enable-linux-netfilter --enable-follow-x-forwarded-
for --with-large-files --sysconfdir=/opt/etc/squid --enable-external-
acl-helpers=none
James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 3.5.20 compile issue

[James Lay]

Thanks...off to git cloning the 1.0.1 branch...all this work for chacha
and poly...yugh 8-|
James
On Mon, 2016-09-19 at 18:37 -0600, Alex Rousskov wrote:
> On 09/19/2016 06:22 PM, James Lay wrote:
> > 
> > Ok so this is with the 1.0.2 branch of openssl:
> > 
> > dso_dlfcn.c:(.text+0x11): undefined reference to `dlopen'
> > dso_dlfcn.c:(.text+0x24): undefined reference to `dlsym'
> > dso_dlfcn.c:(.text+0x2f): undefined reference to `dlclose'
> You can probably force you way through this by linking with more
> system
> libraries, but I do not know exactly which ones you need. This smells
> like an environment or build configuration problem to me -- the
> linker
> does not know that your OpenSSL library depends on another system
> library that provides those [dynamic linking] functions.
> 
> 
> > 
> > Should I just try something different?  1.0.1?  1.0?
> I know that both v1.0 and v1.0.1 work in many environments. YMMV.
> 
> Alex.___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Squid 3.5.20 compile issue

[James Lay]

Ok so this is with the 1.0.2 branch of openssl:

make[3]: Entering directory `/home//nobackup/build/squid-
3.5.20/src/ssl'
/bin/bash ../../libtool  --tag=CXX   --mode=link g++ -Wall -Wpointer-
arith -Wwrite-strings -Wcomments -Wshadow -Woverloaded-virtual -Werror
-pipe -D_REENTRANT -m64   -g -O2 -march=native -std=c++11  -m64 -g -o
ssl_crtd ssl_crtd.o certificate_db.o libsslutil.la -L/opt/openssl/lib
-lssl -lcrypto../../compat/libcompat-squid.la  
libtool: link: g++ -Wall -Wpointer-arith -Wwrite-strings -Wcomments
-Wshadow -Woverloaded-virtual -Werror -pipe -D_REENTRANT -m64 -g -O2
-march=native -std=c++11 -m64 -g -o ssl_crtd ssl_crtd.o
certificate_db.o  ./.libs/libsslutil.a -L/opt/openssl/lib -lssl
-lcrypto ../../compat/.libs/libcompat-squid.a
/opt/openssl/lib/libcrypto.a(dso_dlfcn.o): In function
`dlfcn_globallookup':
dso_dlfcn.c:(.text+0x11): undefined reference to `dlopen'
dso_dlfcn.c:(.text+0x24): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x2f): undefined reference to `dlclose'
/opt/openssl/lib/libcrypto.a(dso_dlfcn.o): In function
`dlfcn_bind_func':
dso_dlfcn.c:(.text+0x334): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x3db): undefined reference to `dlerror'
/opt/openssl/lib/libcrypto.a(dso_dlfcn.o): In function
`dlfcn_bind_var':
dso_dlfcn.c:(.text+0x454): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x4fb): undefined reference to `dlerror'
/opt/openssl/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_load':
dso_dlfcn.c:(.text+0x569): undefined reference to `dlopen'
dso_dlfcn.c:(.text+0x5cb): undefined reference to `dlclose'
dso_dlfcn.c:(.text+0x603): undefined reference to `dlerror'
/opt/openssl/lib/libcrypto.a(dso_dlfcn.o): In function
`dlfcn_pathbyaddr':
dso_dlfcn.c:(.text+0x68f): undefined reference to `dladdr'
dso_dlfcn.c:(.text+0x6f1): undefined reference to `dlerror'
/opt/openssl/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_unload':
dso_dlfcn.c:(.text+0x742): undefined reference to `dlclose'
collect2: error: ld returned 1 exit status
make[3]: *** [ssl_crtd] Error 1
make[3]: Leaving directory `/home/nobackup/build/squid-3.5.20/src/ssl'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/nobackup/build/squid-3.5.20/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/nobackup/build/squid-3.5.20/src'
make: *** [all-recursive] Error 1

Should I just try something different?  1.0.1?  1.0?  Thank you.

James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 3.5.20 fails to compile with openssl

[James Lay]

On 2016-09-19 16:05, Alex Rousskov wrote:

On 09/19/2016 04:01 PM, James Lay wrote:


Openssl git latest commit version commit
e2562bbbe1e1c68ec5a3e02c1f151fd6149ee2ae.


Please see http://bugs.squid-cache.org/show_bug.cgi?id=4599


Thank you,

Alex.


And there you go...thanks Alex.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Squid 3.5.20 fails to compile with openssl

[James Lay]

So I know I posted this a while ago...thought I'd give it a shot today, 
but still no luck:


make[3]: Entering directory `/home/nobackup/build/squid-3.5.20/src/anyp'
depbase=`echo PortCfg.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ../../libtool  --tag=CXX   --mode=compile g++ 
-DHAVE_CONFIG_H   -I../.. -I../../include -I../../lib -I../../src 
-I../../include-I/opt/openssl/include  -Wall -Wpointer-arith 
-Wwrite-strings -Wcomments -Wshadow -Woverloaded-virtual -Werror -pipe 
-D_REENTRANT -m64   -g -O2 -march=native -std=c++11 -MT PortCfg.lo -MD 
-MP -MF $depbase.Tpo -c -o PortCfg.lo PortCfg.cc &&\

mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  g++ -DHAVE_CONFIG_H -I../.. -I../../include 
-I../../lib -I../../src -I../../include -I/opt/openssl/include -Wall 
-Wpointer-arith -Wwrite-strings -Wcomments -Wshadow -Woverloaded-virtual 
-Werror -pipe -D_REENTRANT -m64 -g -O2 -march=native -std=c++11 -MT 
PortCfg.lo -MD -MP -MF .deps/PortCfg.Tpo -c PortCfg.cc  -fPIC -DPIC -o 
.libs/PortCfg.o

In file included from ../../src/anyp/PortCfg.h:18:0,
 from PortCfg.cc:10:
../../src/ssl/gadgets.h:83:45: error: ‘CRYPTO_LOCK_X509’ was not 
declared in this scope
 typedef LockingPointer 
X509_Pointer;

 ^
../../src/ssl/gadgets.h:83:61: error: template argument 3 is invalid
 typedef LockingPointer 
X509_Pointer;

 ^
../../src/ssl/gadgets.h:83:75: error: invalid type in declaration before 
‘;’ token
 typedef LockingPointer 
X509_Pointer;
 
  ^
../../src/ssl/gadgets.h:89:53: error: ‘CRYPTO_LOCK_EVP_PKEY’ was not 
declared in this scope
 typedef LockingPointerCRYPTO_LOCK_EVP_PKEY> EVP_PKEY_Pointer;

 ^
../../src/ssl/gadgets.h:89:73: error: template argument 3 is invalid
 typedef LockingPointerCRYPTO_LOCK_EVP_PKEY> EVP_PKEY_Pointer;
 
^
../../src/ssl/gadgets.h:89:91: error: invalid type in declaration before 
‘;’ token
 typedef LockingPointerCRYPTO_LOCK_EVP_PKEY> EVP_PKEY_Pointer;
 
  ^
../../src/ssl/gadgets.h:116:43: error: ‘CRYPTO_LOCK_SSL’ was not 
declared in this scope

 typedef LockingPointer SSL_Pointer;
   ^
../../src/ssl/gadgets.h:116:58: error: template argument 3 is invalid
 typedef LockingPointer SSL_Pointer;
  ^
../../src/ssl/gadgets.h:116:71: error: invalid type in declaration 
before ‘;’ token

 typedef LockingPointer SSL_Pointer;
   ^
make[3]: *** [PortCfg.lo] Error 1
make[3]: Leaving directory `/home/nobackup/build/squid-3.5.20/src/anyp'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/nobackup/build/squid-3.5.20/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/nobackup/build/squid-3.5.20/src'
make: *** [all-recursive] Error 1

Openssl git latest commit version commit 
e2562bbbe1e1c68ec5a3e02c1f151fd6149ee2ae.  thank you.


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Yet another new cipher?

[James Lay]

On 2016-06-30 07:18, James Lay wrote:

On Fri, 2016-07-01 at 01:04 +1200, Amos Jeffries wrote:


On 1/07/2016 12:43 a.m., James Lay wrote: On Wed, 2016-06-29 at
19:33 -0600, James Lay wrote: Yugh...starting around 10:00 facebook
no longer works via peek/splice. pcap contents show:
1QTV01...CHLOSNI.VERSscontent.xx.fbcdn.netQTV1 after the
threeway handshake and an instant reset. Anyone know what this is?
Cause I haven't a cluescreenshot of success after bypassing
included. Thank you. I guess I should also say that this is from the
official Facebook app on Android...just updated on Tuesday.

 FWIW: I identified the last one from your posted wireshark details.
Looking at the "Unknown Ciphers:" list and looking up the hex codes
listed there in the IANA registry. The details posted so far about
this issue tells me nothing except that FB suddenly stopped working.
Amos ___ squid-users
mailing list squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users [1]

That's fair...I'm including a successful handshake...wireshark just
sees this as data.  Thanks Amos!

James

Links:
--
[1] http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users



Meh...this is QUIC Crypto:

https://docs.google.com/document/d/1g5nIXAIkN_Y-7XJW5K45IblHd_L2f5LTaDUDwvZ5L6g/edit

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Yet another new cipher?

[James Lay]

On Fri, 2016-07-01 at 01:04 +1200, Amos Jeffries wrote:
> On 1/07/2016 12:43 a.m., James Lay wrote:
> > 
> > On Wed, 2016-06-29 at 19:33 -0600, James Lay wrote:
> > > 
> > > Yugh...starting around 10:00 facebook no longer works via
> > > peek/splice.  pcap contents show:
> > > 
> > > 1QTV01...CHLOSNI.VERSscontent.xx.fbcdn.netQTV1
> > > 
> > > after the threeway handshake and an instant reset.  Anyone know
> > > what
> > > this is?  Cause I haven't a cluescreenshot of success after
> > > bypassing included.  Thank you.
> > > 
> > I guess I should also say that this is from the official Facebook
> > app
> > on Android...just updated on Tuesday.
> FWIW: I identified the last one from your posted wireshark details.
> Looking at the "Unknown Ciphers:" list and looking up the hex codes
> listed there in the IANA registry.
> 
> The details posted so far about this issue tells me nothing except
> that
> FB suddenly stopped working.
> 
> Amos
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
That's fair...I'm including a successful handshake...wireshark just
sees this as data.  Thanks Amos!
James

192.168.1.101-stream5.pcapng
Description: application/pcapng
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Yet another new cipher?

[James Lay]

On Wed, 2016-06-29 at 19:33 -0600, James Lay wrote:
> Yugh...starting around 10:00 facebook no longer works via
> peek/splice.  pcap contents show:
> 
> 1QTV01...CHLOSNI.VERSscontent.xx.fbcdn.netQTV1
> 
> after the threeway handshake and an instant reset.  Anyone know what
> this is?  Cause I haven't a cluescreenshot of success after
> bypassing included.  Thank you.
> 
> James
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
I guess I should also say that this is from the official Facebook app
on Android...just updated on Tuesday.
James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Yet another new cipher?

[James Lay]

Yugh...starting around 10:00 facebook no longer works via peek/splice.
 pcap contents show:

1QTV01...CHLOSNI.VERSscontent.xx.fbcdn.netQTV1

after the threeway handshake and an instant reset.  Anyone know what
this is?  Cause I haven't a cluescreenshot of success after
bypassing included.  Thank you.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Latest ssl and Squid stable compile issue

[James Lay]

On Thu, 2016-06-23 at 17:47 +1200, Amos Jeffries wrote:
> Yay that you got it going with LibreSSL.
> 
> But I'm still interested in why you got the errors in the first place
> with OpenSSL. It is supposed to be the better supported one :-P
> 
> So if you have the time to assist my edufication;
> 
>  what version OpenSSL was this exactly that you built against?
> ("git pulled latest" doesnt tell me much about what branch/version
> etc
> you ended up with.)
> 
> And was it only the libssl you built with, or also the matching
> libcrypto ? (libcrypto is what defines the OpenSSL CRYPTO_LOCK_*
> stuff).
> 
> Amos
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
Ah...well I went with this repo:
git clone https://github.com/openssl/openssl.git
And the errors I posted were where I stoppedunfortunately I've
already nuked the repo off the drive otherwise I'd give you the exact
info.  I can tell you I got the same errors with both 3.5.19 and
4.0.11.  Also compile line:
./configure --prefix=/opt --with-openssl=/opt/libressl --enable-ssl --
enable-ssl-crtd --enable-linux-netfilter --enable-follow-x-forwarded-
for --with-large-files --sysconfdir=/opt/etc/squid --enable-external-
acl-helpers=none
Sorry Amos...the one time you ask me for information and I don't have
it for you..: - (
James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Latest ssl and Squid stable compile issue

[James Lay]

It already has :)

Jun 22 09:41:09 gateway (squid-1): 192.168.1.109 - - 
[22/Jun/2016:09:41:09 -0600] "CONNECT 31.13.76.84:443 HTTP/1.1" 
i.instagram.com - 200 0 TAG_NONE:ORIGINAL_DST
Jun 22 15:09:26 gateway (squid-1): 192.168.1.109 - - 
[22/Jun/2016:15:09:26 -0600] "CONNECT 31.13.76.84:443 HTTP/1.1" 
i.instagram.com - 200 43538 TCP_TUNNEL:ORIGINAL_DST


The ole before and after trick :)  And:

strings /opt/libressl/bin/openssl | grep chacha
EVP_aead_chacha20_poly1305
chacha
 chacha20-poly1305
chacha20 poly1305
EVP_aead_chacha20_poly1305
EVP_aead_chacha20_poly1305

Woo hoo!

James

On 2016-06-22 15:17, Yuri Voinov wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

I suggest this will not solve your unknown cipher issue. :)


23.06.2016 3:12, James Lay пишет:
Had zero issues when compiling against libressl-2.4.1.  I now have 
ChaCha Poly cipher support...happy

days!


James

On 2016-06-22 13:29, James Lay wrote:

So yea...git pulled latest ssl, here's my results:

make[3]: Entering directory
`/home/nobackup/build/squid-3.5.19/src/anyp'
depbase=`echo PortCfg.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
 /bin/bash ../../libtool  --tag=CXX   --mode=compile g++
-DHAVE_CONFIG_H   -I../.. -I../../include -I../../lib -I../../src
-I../../include-I/opt/openssl/include  -Wall -Wpointer-arith
-Wwrite-strings -Wcomments -Wshadow -Woverloaded-virtual -Werror 
-pipe
-D_REENTRANT -m64   -g -O2 -march=native -std=c++11 -MT PortCfg.lo 
-MD

-MP -MF $depbase.Tpo -c -o PortCfg.lo PortCfg.cc &&\
 mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  g++ -DHAVE_CONFIG_H -I../.. -I../../include
-I../../lib -I../../src -I../../include -I/opt/openssl/include -Wall
-Wpointer-arith -Wwrite-strings -Wcomments -Wshadow
-Woverloaded-virtual -Werror -pipe -D_REENTRANT -m64 -g -O2
-march=native -std=c++11 -MT PortCfg.lo -MD -MP -MF .deps/PortCfg.Tpo
-c PortCfg.cc  -fPIC -DPIC -o .libs/PortCfg.o
In file included from ../../src/anyp/PortCfg.h:18:0,
 from PortCfg.cc:10:
../../src/ssl/gadgets.h:83:45: error: ‘CRYPTO_LOCK_X509’ was not
declared in this scope
 typedef LockingPointer
X509_Pointer;
 ^
../../src/ssl/gadgets.h:83:61: error: template argument 3 is invalid
 typedef LockingPointer
X509_Pointer;
 ^
../../src/ssl/gadgets.h:83:75: error: invalid type in declaration
before ‘;’ token
 typedef LockingPointer
X509_Pointer;

^
../../src/ssl/gadgets.h:89:53: error: ‘CRYPTO_LOCK_EVP_PKEY’ was
not declared in this scope
 typedef LockingPointer EVP_PKEY_Pointer;
 ^
../../src/ssl/gadgets.h:89:73: error: template argument 3 is invalid
 typedef LockingPointer EVP_PKEY_Pointer;

  ^
../../src/ssl/gadgets.h:89:91: error: invalid type in declaration
before ‘;’ token
 typedef LockingPointer EVP_PKEY_Pointer;

^
../../src/ssl/gadgets.h:116:43: error: ‘CRYPTO_LOCK_SSL’ was not
declared in this scope
 typedef LockingPointer
SSL_Pointer;
   ^
../../src/ssl/gadgets.h:116:58: error: template argument 3 is invalid
 typedef LockingPointer
SSL_Pointer;
  ^
../../src/ssl/gadgets.h:116:71: error: invalid type in declaration
before ‘;’ token
 typedef LockingPointer
SSL_Pointer;

^
make[3]: *** [PortCfg.lo] Error 1
make[3]: Leaving directory
`/home/jlay/nobackup/build/squid-3.5.19/src/anyp'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory
`/home/jlay/nobackup/build/squid-3.5.19/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory
`/home/jlay/nobackup/build/squid-3.5.19/src'
make: *** [all-recursive] Error 1

This is to hopefully compile in chacha supportshould I go with 
dev

4.0.11 squid instead?  Thank you.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJXawB+AAoJENNXIZxhPexGzXwH/3WAFGluGUDHx1BAIHjzCNvM
zGB0qFQaLTcSNACG7B7bNs5oDErCdxH7BUhwWC082L4Tu0FhZAivUYgD3GZZKuzr
QLjZ7wYcocXQsa1EVyVQaiQg4MdIpO6PRZniBq6pKephJrTFj9b3l3MYNcECNF6F
7Qla/Cocyf5hJpN7U6WxyzXOBB4CbecViGwMyQIBZ4s+B6B3BAHZaFg54UNGstqA
/qjwlMehh5Al/WntrJ1Ozfa59r0efiIO/VrvOylQf8HoR9gYBULq7tsl3EFGrsV2
08pcmXzQSJausCcllBWBnXKKTi4CWo/apkPE4puxYEcHLGSklqbOl7MK4oHkW44=
=TjQy
-END PGP SIGNATURE-


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Latest ssl and Squid stable compile issue

[James Lay]

Had zero issues when compiling against libressl-2.4.1.  I now have 
ChaCha Poly cipher support...happy days!


James

On 2016-06-22 13:29, James Lay wrote:

So yea...git pulled latest ssl, here's my results:

make[3]: Entering directory
`/home/nobackup/build/squid-3.5.19/src/anyp'
depbase=`echo PortCfg.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
 /bin/bash ../../libtool  --tag=CXX   --mode=compile g++
-DHAVE_CONFIG_H   -I../.. -I../../include -I../../lib -I../../src
-I../../include-I/opt/openssl/include  -Wall -Wpointer-arith
-Wwrite-strings -Wcomments -Wshadow -Woverloaded-virtual -Werror -pipe
-D_REENTRANT -m64   -g -O2 -march=native -std=c++11 -MT PortCfg.lo -MD
-MP -MF $depbase.Tpo -c -o PortCfg.lo PortCfg.cc &&\
 mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  g++ -DHAVE_CONFIG_H -I../.. -I../../include
-I../../lib -I../../src -I../../include -I/opt/openssl/include -Wall
-Wpointer-arith -Wwrite-strings -Wcomments -Wshadow
-Woverloaded-virtual -Werror -pipe -D_REENTRANT -m64 -g -O2
-march=native -std=c++11 -MT PortCfg.lo -MD -MP -MF .deps/PortCfg.Tpo
-c PortCfg.cc  -fPIC -DPIC -o .libs/PortCfg.o
In file included from ../../src/anyp/PortCfg.h:18:0,
 from PortCfg.cc:10:
../../src/ssl/gadgets.h:83:45: error: ‘CRYPTO_LOCK_X509’ was not
declared in this scope
 typedef LockingPointer
X509_Pointer;
 ^
../../src/ssl/gadgets.h:83:61: error: template argument 3 is invalid
 typedef LockingPointer
X509_Pointer;
 ^
../../src/ssl/gadgets.h:83:75: error: invalid type in declaration
before ‘;’ token
 typedef LockingPointer
X509_Pointer;

^
../../src/ssl/gadgets.h:89:53: error: ‘CRYPTO_LOCK_EVP_PKEY’ was
not declared in this scope
 typedef LockingPointer EVP_PKEY_Pointer;
 ^
../../src/ssl/gadgets.h:89:73: error: template argument 3 is invalid
 typedef LockingPointer EVP_PKEY_Pointer;

  ^
../../src/ssl/gadgets.h:89:91: error: invalid type in declaration
before ‘;’ token
 typedef LockingPointer EVP_PKEY_Pointer;

^
../../src/ssl/gadgets.h:116:43: error: ‘CRYPTO_LOCK_SSL’ was not
declared in this scope
 typedef LockingPointer
SSL_Pointer;
   ^
../../src/ssl/gadgets.h:116:58: error: template argument 3 is invalid
 typedef LockingPointer
SSL_Pointer;
  ^
../../src/ssl/gadgets.h:116:71: error: invalid type in declaration
before ‘;’ token
 typedef LockingPointer
SSL_Pointer;

^
make[3]: *** [PortCfg.lo] Error 1
make[3]: Leaving directory
`/home/jlay/nobackup/build/squid-3.5.19/src/anyp'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory
`/home/jlay/nobackup/build/squid-3.5.19/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory
`/home/jlay/nobackup/build/squid-3.5.19/src'
make: *** [all-recursive] Error 1

This is to hopefully compile in chacha supportshould I go with dev
4.0.11 squid instead?  Thank you.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Latest ssl and Squid stable compile issue

[James Lay]

So yea...git pulled latest ssl, here's my results:

make[3]: Entering directory `/home/nobackup/build/squid-
3.5.19/src/anyp'
depbase=`echo PortCfg.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ../../libtool  --tag=CXX   --mode=compile g++
-DHAVE_CONFIG_H   -I../.. -I../../include -I../../lib -I../../src
-I../../include-I/opt/openssl/include  -Wall -Wpointer-arith
-Wwrite-strings -Wcomments -Wshadow -Woverloaded-virtual -Werror -pipe
-D_REENTRANT -m64   -g -O2 -march=native -std=c++11 -MT PortCfg.lo -MD
-MP -MF $depbase.Tpo -c -o PortCfg.lo PortCfg.cc &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  g++ -DHAVE_CONFIG_H -I../.. -I../../include
-I../../lib -I../../src -I../../include -I/opt/openssl/include -Wall
-Wpointer-arith -Wwrite-strings -Wcomments -Wshadow -Woverloaded-
virtual -Werror -pipe -D_REENTRANT -m64 -g -O2 -march=native -std=c++11
-MT PortCfg.lo -MD -MP -MF .deps/PortCfg.Tpo -c PortCfg.cc  -fPIC -DPIC
-o .libs/PortCfg.o
In file included from ../../src/anyp/PortCfg.h:18:0,
 from PortCfg.cc:10:
../../src/ssl/gadgets.h:83:45: error: ‘CRYPTO_LOCK_X509’ was not
declared in this scope
 typedef LockingPointer
X509_Pointer;
 ^
../../src/ssl/gadgets.h:83:61: error: template argument 3 is invalid
 typedef LockingPointer
X509_Pointer;
 ^
../../src/ssl/gadgets.h:83:75: error: invalid type in declaration
before ‘;’ token
 typedef LockingPointer
X509_Pointer;
   
^
../../src/ssl/gadgets.h:89:53: error: ‘CRYPTO_LOCK_EVP_PKEY’ was not
declared in this scope
 typedef LockingPointer EVP_PKEY_Pointer;
 ^
../../src/ssl/gadgets.h:89:73: error: template argument 3 is invalid
 typedef LockingPointer EVP_PKEY_Pointer;
   
  ^
../../src/ssl/gadgets.h:89:91: error: invalid type in declaration
before ‘;’ token
 typedef LockingPointer EVP_PKEY_Pointer;
   
^
../../src/ssl/gadgets.h:116:43: error: ‘CRYPTO_LOCK_SSL’ was not
declared in this scope
 typedef LockingPointer
SSL_Pointer;
   ^
../../src/ssl/gadgets.h:116:58: error: template argument 3 is invalid
 typedef LockingPointer
SSL_Pointer;
  ^
../../src/ssl/gadgets.h:116:71: error: invalid type in declaration
before ‘;’ token
 typedef LockingPointer
SSL_Pointer;
   
^
make[3]: *** [PortCfg.lo] Error 1
make[3]: Leaving directory `/home/jlay/nobackup/build/squid-
3.5.19/src/anyp'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/jlay/nobackup/build/squid-3.5.19/src'

make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/jlay/nobackup/build/squid-3.5.19/src'

make: *** [all-recursive] Error 1

This is to hopefully compile in chacha supportshould I go with dev
4.0.11 squid instead?  Thank you.

James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Unknown Cipher Suite

[James Lay]

Ah crud...well shoot...thanks for the information...means I'll have to
bypass it for now..and using latest Squid...always keep it updated ; -
)  Thank you.
James
On Wed, 2016-06-22 at 22:58 +0600, Yuri Voinov wrote:
> OpenSSL still not support ChaCha-Poly this days. And unknown when be
> supported.
> 
> This time only exists unsupported patch from CloudFlare. And, as
> alternative, LibreSSL. Which is not available for all platforms.
> 
> 22.06.2016 22:48, Amos Jeffries пишет:
> > 
> > On 23/06/2016 4:12 a.m., James Lay wrote:
> > > 
> > > Well this is newstarted seeing this on Instagram.  Message I
> > > get
> > > when debugging:
> > > 
> > > 2016/06/22 09:43:26| Error negotiating SSL on FD 14:
> > > error:140920F8:SSL
> > > routines:SSL3_GET_SERVER_HELLO:unknown cipher returned (1/-1/0)
> > > 
> > > And sure enough...even Wireshark doesn't know what this is:
> > > 
> > > 
> > > Any hints on how what this is/how to fix?  Thanks all.
> > > 
> > Thats the new ChaCha and Poly1305 ciphers being used.
> > 
> > Time to update your OpenSSL library version. Maybe your Squid as
> > well if
> > you are using anything older than current latest.
> > 
> > Amos
> > 
> > ___
> > squid-users mailing list
> > squid-users@lists.squid-cache.org
> > http://lists.squid-cache.org/listinfo/squid-users
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Unknown Cipher Suite

[James Lay]

Well this is newstarted seeing this on Instagram.  Message I get
when debugging:

2016/06/22 09:43:26| Error negotiating SSL on FD 14: error:140920F8:SSL
routines:SSL3_GET_SERVER_HELLO:unknown cipher returned (1/-1/0)

And sure enough...even Wireshark doesn't know what this is:


Any hints on how what this is/how to fix?  Thanks all.

James___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Transparent Mode w/ Peek and Splice trouble

[James Lay]

On 2016-05-18 08:14, s...@kpa.gr wrote:

Hello!

I am currently setting up a squid server, which should serve as a
transparent proxy in our network.

We mainly need it to do the following:
Allow and Block Domains on HTTP and HTTPS protocol (withOUT bumping
the traffic). We only want to allow domain names on the SSL port, no
URLs.

It actually works fine for HTTP, but I can't configure the "peek and
splice" method for the HTTPS traffic.

I have come to a point, where HTTP access is being filtered exactly as
I wanted to, but following odd error occures when visiting HTTPS
sites:

When using "https_port 10.0.0.222:3130 cert=/root/cert.pem
key=/root/key.pem ssl-bump intercept"
I get an Access Denied Error for any Website I try to access, which
occured while "trying to retrieve the URL: 10.0.0.222:3130"!

If I configure the https_port option with "accel vhost allow-direct"
like the http_port, the allowed Pages work fine but with squid's
certificate.


Somewhere the Squid seems to redirect his actual https traffic back to
itself when using the "intercept" option and that is why I cannot use
the splice method.

You can find my configuration files on http://kpa.gr/squid-conf/

Thanks very much in advance,

Pantelis W
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users



Read:

http://thread.gmane.org/gmane.comp.web.squid.general/114384/focus=114389

I'm doing exactly what you're wanting.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Logging of https

[James Lay]

That's correctpeek/stare don't require a cert on the client end.  
Just keep in mind you won't get a full URL in the logs with https 
sites...just the host/ip:


Apr  7 09:30:31 gateway (squid-1): 192.168.1.106 - - 
[07/Apr/2016:09:30:31 -0600] "CONNECT 216.58.193.78:443 HTTP/1.1" 
safebrowsing.google.com - 200 871538 TCP_TUNNEL:ORIGINAL_DST


James

On 2016-04-07 07:11, Markey, Bruce wrote:

Ok thanks for that.  I think I have a slightly better understanding of
what is going on.That being said this is what I've come up with.

No caching.  All sites allowed, peeking at all.

I'm hoping this config will simply give me the logging that I'm
looking for and nothing else.  And from that link you sent I don't
have to install the client side cert?

Thanks

  1 #Access Lists
  2 acl internal src 192.168.200.0/21
  3 acl wireless src 192.168.100.0/23
  4
  5 #Ports allowed through Squid
  6 acl Safe_ports port 80
  7 acl Safe_ports port 443
  8 acl SSL_ports port 443
  9 acl CONNECT method CONNECT
 10
 11 #allow/deny
 12 http_access allow internal
 13 http_access allow wireless
 14 http_access deny !Safe_ports
 15 http_access deny CONNECT !SSL_ports
 16 http_access deny all
 17
 18 #Bumping
 19 acl step1 at_step SslBump1
 20 acl step2 at_step SslBump2
 21 acl step3 at_step SslBump3
 22
 23 ssl_bump peek all
 24 ssl_bump splice all
 25
 26 sslproxy_capath /etc/ssl/certs
 27
 28 sslcrtd_program /usr/lib/squid3/ssl_crtd -s /opt/var/ssl_db -M 6MB
 29 sslcrtd_children 5
 30
 31 #certs
 32 cert=/etc/squid3/certs/squid.pem
 33 cafile=/etc/squid3/certs/squid.pem
 34 key=/etc/squid3/certs/squid.pem generate-host-certificates=on
dynamic_cert_mem_cache_size=6MB sslflags=NO_SESSION_REUSE
 35
 36 logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni
%ssl::>cert_subject %>Hs % 54 wccp2_service_info 70 protocol=tcp flags=dst_ip_hash priority=240 
ports=443

 55

Bruce Markey | Network Security Analyst
STEINMAN COMMUNICATIONS
717.291.8758 (o) | bmar...@steinmancommunications.com
8 West King St | PO Box 1328, Lancaster, PA 17608-1328

-Original Message-
From: squid-users [mailto:squid-users-boun...@lists.squid-cache.org]
On Behalf Of James Lay
Sent: Thursday, March 24, 2016 4:14 PM
To: squid-users@lists.squid-cache.org
Subject: Re: [squid-users] Logging of https

On 2016-03-24 13:41, Markey, Bruce wrote:

I'm hoping this is a simple question, I've gotten/seen differing
answers and I'd just like a final answer.

With squid setup as a transparent proxy via wccp will there be any log
entries for https sites, even just the ip?  Just the initial get
request is what I'd expect.

( I have no interest in breaking https, I'd simply like to get any
data I can without having to go down that road)

If yes then what needs to be done to make that happen. Currently
everything is working on the http side perfectly.  Oh the https side
as soon as I enable wccp redirection of 443 to squid it breaks https.
 ( I'll add here that I've read all the peek and splice info and I
don't really understand it.)

Thanks

BRUCE MARKEY | Network Security Analyst

STEINMAN COMMUNICATIONS

717.291.8758 (o) | bmar...@steinmancommunications.com

8 West King St | PO Box 1328, Lancaster, PA 17608-1328


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users



Read this:

http://thread.gmane.org/gmane.comp.web.squid.general/114384/focus=114389

Sample messages:

allowed https:
Mar 24 14:02:11 gateway (squid-1): 192.168.1.101 - -
[24/Mar/2016:14:02:11 -0600] "CONNECT 209.59.180.48:443 HTTP/1.1" - -
200 5511 TCP_TUNNEL:ORIGINAL_DST

note the size, 5511, and the TCP_TUNNEL, this has no SNI

denied https:
Mar 24 13:36:01 gateway (squid-1): 192.168.1.101 - -
[24/Mar/2016:13:36:01 -0600] "CONNECT 54.171.35.38:443 HTTP/1.1" - - 
200

0 TAG_NONE:ORIGINAL_DST

note the size, 0, and the TAG_NONE, and this also has no SNI

Mar 24 13:36:01 gateway (squid-1): 192.168.1.101 - -
[24/Mar/2016:13:36:01 -0600] "CONNECT 54.171.177.121:443 HTTP/1.1"
track.appsflyer.com - 200 0 TAG_NONE:ORIGINAL_DST

again, size, and TAG_NONE, but we saw SNI for this one.

the above are the output when using the config info in the link.  Hope
that helps.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] filtering http(s) sites, transparently

[James Lay]

On Sun, 2016-04-03 at 21:18 -0700, Jok Thuau wrote:
> I'm attempting to build a transparent proxy (policy based routing on
> firewall to squid proxy) with the following behavior:
> 
> 
> 
> 1) proxies http traffic for a given set of domains, provide an message
> otherwise such "domain not allowed" or similar
> 2) proxies https traffic for a given set of domains (ideally, splicing
> those, so as not to break HSTS, if enabled), otherwise provide an
> error message (bumping and providing "domain not allowed")
> 
> 
> 
> I'm attempting this with a 3.5.15 compiled with icap (not yet used)
> and ssl-bumping.
> 
> 
> Part 1 seems easy enough (and is well documented)...
> 
> 
> acl whitelist dstdomain .domain1.tld
> 
> acl whitelist dstdomain .domain2.tld
> 
> 
> acl http_ok all-of whitelist !SSL_ports
> 
> 
> http_access allow http_ok
> http_access deny all
> 
> 
> Moving onto Part 2 (the peek and splice setup) appears to be the topic
> of a few discussions out there...
> 
> 
> acl sni_whitelist ssl::server_name .domain1.tld
> acl sni_whitelist ssl::server_name .domain2.tld
> 
> 
> ssl_bump peek step1
> ssl_bump splice sni_whitelist
> ssl_bump bump all
> 
> 
> It appears however that when combining the two, the generated
> certificate(s), instead of mimic'ing the original server's certificate
> comes out with the CN= where  is the ip used by the "connect"
> part of the connection. In addition, it appears that only the first
> entry ever matches (at this point, i've tried so many combinations,
> i'm no longer certain of anything). 
> 
> 
> If i remove *all* the http_access lines, then the behavior appears
> correct (from a "splicing/bumping" standpoint).
> 
> 
> Can anyone confirm that this is indeed possible to achieve?
> 
> 
> I believe, based on experimentation that any http_access i have,
> because of the "deny all" cause the bumping to "short circuit" and
> effectively send an early "access denied" based on the only
> information it has (the ip address from the "connect", rather than the
> SNI that would come later). 
> 
> 
> Would a setup where "deny http+!whitelist" so have the allow be the
> default allow for the bumping to work and get to step2 and match the
> sni* acls somehow? (with a "deny step2 !sni_whitelist").
> 
> 
> Is 3.5.15 capable of doing this? If this requires some feature/effort,
> what would be the procedure to sponsor that work?
> 
> 
> Thanks,
> Jok
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


This may assist:

http://article.gmane.org/gmane.comp.web.squid.general/114389

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Logging of https

[James Lay]

On 2016-03-24 13:41, Markey, Bruce wrote:

I'm hoping this is a simple question, I've gotten/seen differing
answers and I'd just like a final answer.

With squid setup as a transparent proxy via wccp will there be any log
entries for https sites, even just the ip?  Just the initial get
request is what I'd expect.

( I have no interest in breaking https, I'd simply like to get any
data I can without having to go down that road)

If yes then what needs to be done to make that happen. Currently
everything is working on the http side perfectly.  Oh the https side
as soon as I enable wccp redirection of 443 to squid it breaks https.
 ( I'll add here that I've read all the peek and splice info and I
don't really understand it.)

Thanks

BRUCE MARKEY | Network Security Analyst

STEINMAN COMMUNICATIONS

717.291.8758 (o) | bmar...@steinmancommunications.com

8 West King St | PO Box 1328, Lancaster, PA 17608-1328


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users



Read this:

http://thread.gmane.org/gmane.comp.web.squid.general/114384/focus=114389

Sample messages:

allowed https:
Mar 24 14:02:11 gateway (squid-1): 192.168.1.101 - - 
[24/Mar/2016:14:02:11 -0600] "CONNECT 209.59.180.48:443 HTTP/1.1" - - 
200 5511 TCP_TUNNEL:ORIGINAL_DST


note the size, 5511, and the TCP_TUNNEL, this has no SNI

denied https:
Mar 24 13:36:01 gateway (squid-1): 192.168.1.101 - - 
[24/Mar/2016:13:36:01 -0600] "CONNECT 54.171.35.38:443 HTTP/1.1" - - 200 
0 TAG_NONE:ORIGINAL_DST


note the size, 0, and the TAG_NONE, and this also has no SNI

Mar 24 13:36:01 gateway (squid-1): 192.168.1.101 - - 
[24/Mar/2016:13:36:01 -0600] "CONNECT 54.171.177.121:443 HTTP/1.1" 
track.appsflyer.com - 200 0 TAG_NONE:ORIGINAL_DST


again, size, and TAG_NONE, but we saw SNI for this one.

the above are the output when using the config info in the link.  Hope 
that helps.


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] HTTPS interception and filtering?

[James Lay]

That's the one.

James

On Mon, 2016-03-14 at 00:42 +0200, Eliezer Croitoru wrote:

> Are you referring to:
> http://thread.gmane.org/gmane.comp.web.squid.general/114384/focus=114389
> 
> Eliezer
> 
> On 12/03/2016 15:58, James Lay wrote:
> > On Sun, 2016-03-13 at 00:09 +1100, Tim Bates wrote:
> >> Is it possible to do this:
> >>
> >> * Intercept HTTPS and send it via Squid?
> >> * Apply ACLs to the intercepted HTTPS traffic based on host/domain name?
> >> * Not change any configuration on clients?
> >>
> >> Should I keep researching how this peeking and bumping and splicing and
> >> such works, or is it impossible?
> >>
> >> TB
> >> ___
> >> squid-users mailing list
> >> squid-users@lists.squid-cache.org
> >> <mailto:squid-users@lists.squid-cache.org>
> >> http://lists.squid-cache.org/listinfo/squid-users
> >
> > Search for my previous posts...I've posted full configs on how to do
> > exactly this.
> >
> > James
> >
> >
> > ___
> > squid-users mailing list
> > squid-users@lists.squid-cache.org
> > http://lists.squid-cache.org/listinfo/squid-users
> >
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users



___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] HTTPS interception and filtering?

[James Lay]

On Sun, 2016-03-13 at 00:09 +1100, Tim Bates wrote:

> Is it possible to do this:
> 
> * Intercept HTTPS and send it via Squid?
> * Apply ACLs to the intercepted HTTPS traffic based on host/domain name?
> * Not change any configuration on clients?
> 
> Should I keep researching how this peeking and bumping and splicing and 
> such works, or is it impossible?
> 
> TB
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


Search for my previous posts...I've posted full configs on how to do
exactly this.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] HTTPS Content Filtering without de-crypting traffic?

[James Lay]

 

On 2016-01-26 15:59, Panda Admin wrote: 

> Hello, 
> 
> I attempting to terminate https traffic based on ACLs using ssl_bumping 
> WITHOUT de-crypting the traffic in intercept/transparent mode.  Has anyone 
> got this to work before? I have copied my configuration and what my iptables 
> nat rules look like.  
> 
> I am using squid 3.5.13 with the following compile options: 
> 
> Squid Cache: Version 3.5.12 
> Service Name: squid 
> configure options:  '--prefix=/usr' '--localstatedir=/var' 
> '--libexecdir=/lib/squid3' '--datadir=/share/squid3' 
> '--sysconfdir=/etc/squid3' '--with-default-user=proxy' 
> '--with-logdir=/var/log/squid3' '--with-pidfile=/var/run/squid3.pid' 
> '--with-openssl' '-enable-ssl-crtd' '--enable-icap-client' 
> '--with-large-files' --enable-ltdl-convenience 
> 
> squid.conf: 
> 
> acl social dstdomain .google.com [1] .facebook.com [2] .reddit.com [3] 
> acl step1 at_step SslBump1 
> acl step2 at_step SslBump2 
> ssl_bump stare step2 all 
> ssl_bump terminate social 
> acl localnet src 192.168.50.0/24 [4] 
> acl SSL_ports port 443 
> acl Safe_ports port 80 # http 
> acl Safe_ports port 21 # ftp 
> acl Safe_ports port 443 # https 
> acl Safe_ports port 70 # gopher 
> acl Safe_ports port 210 # wais 
> acl Safe_ports port 1025-65535 # unregistered ports 
> acl Safe_ports port 280 # http-mgmt 
> acl Safe_ports port 488 # gss-http 
> acl Safe_ports port 591 # filemaker 
> acl Safe_ports port 777 # multiling http 
> acl CONNECT method CONNECT 
> http_access allow manager localhost 
> http_access deny manager 
> http_access deny !Safe_ports 
> http_access deny CONNECT !SSL_ports 
> http_access allow localnet 
> http_access allow localhost 
> http_access allow all 
> http_port 3128 transparent 
> https_port 3129 intercept ssl-bump cert=/etc/squid3/ssl_cert/squidSSL.pem 
> cache_dir ufs /cache/squid3/spool 100 16 256 
> access_log syslog:local5.info [5] squid 
> coredump_dir /var/spool/squid3 
> url_rewrite_program /usr/bin/squidGuard -c 
> /cache/config/daemons/squidguard/squidGuard.conf 
> url_rewrite_children 15 
> url_rewrite_access allow all 
> refresh_pattern ^ftp: 1440 20% 10080 
> refresh_pattern ^gopher: 1440 0% 1440 
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 
> refresh_pattern . 0 20% 4320 
> icap_enable on 
> icap_send_client_ip on 
> icap_send_client_username on 
> icap_client_username_encode off 
> icap_client_username_header X-Authenticated-User 
> icap_preview_enable on 
> icap_preview_size 1024 
> icap_service service_req reqmod_precache bypass=1 
> icap://127.0.0.1:1344/squidclamav [6] 
> adaptation_access service_req allow all 
> icap_service service_resp respmod_precache bypass=1 
> icap://127.0.0.1:1344/squidclamav [6] 
> adaptation_access service_resp allow all 
> 
> iptables -L -v -t nat(only relevant rules): 
> 
> Chain PREROUTING (policy ACCEPT 1083 packets, 233K bytes) 
> pkts bytes target prot opt in out source   
> destination  
> 157  9420 DNAT   tcp  --  eth1   any anywhere anywhere
>  tcp dpt:https to:192.168.11.1:3129 [7] 
> 
> Chain PREROUTING-daemon-tcp (1 references) 
> pkts bytes target prot opt in out source   
> destination  
> 443 26580 DNAT   tcp  --  eth1   any anywhere anywhere
>  tcp dpt:http /* 7:PFD::CF-3128 */ to:192.168.11.1:3128 [8] 
> 0 0 DNAT   tcp  --  eth2   any anywhere anywhere  
>tcp dpt:http /* 8:PFD::CF-3128 */ to:172.17.0.1:3128 [9] 
> 
> Right now I can't get it to terminate ANY https traffic. All it does is allow 
> it through.   
> Any and all help would be greatly appreciated! 
> 
> ~ Extremely Confused Squid User ~ 
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

Read: 

http://thread.gmane.org/gmane.comp.web.squid.general/114384/focus=114389


I'm doing exactly what you're wanting. 

James 
  

Links:
--
[1] http://google.com/
[2] http://facebook.com/
[3] http://reddit.com/
[4] http://192.168.50.0/24
[5] http://local5.info/
[6] http://127.0.0.1:1344/squidclamav
[7] http://192.168.11.1:3129/
[8] http://192.168.11.1:3128/
[9] http://172.17.0.1:3128/
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Fwd: Re: Squid Log messages Database

[James Lay]

On 2016-01-18 14:59, Antony Stone wrote:

Forwarding private reply back to the list...

--  Forwarded Message Starts  --

Thanks for your answer.
Sorry for my poor english, I'll try to reword because I'm not looking 
for a

log analyzer. In fact, I don't even need Squid itself installed.
What I'm looking for, is a list of all the logs a Squid can generate.
I'm not interested in collecting logs from a running squid. I'm 
interested

in the full list of logs a Squid can possibly generate, for every
specific.  cases. Like a dictionary containing every logs ordered (with 
a

little definition maybe :) ).

kind regards,

On Mon, Jan 18, 2016 at 6:16 PM, Antony Stone wrote:


On Monday 18 January 2016 at 17:55:51, romain noyer wrote:

> Is there a way to get all the messages a squid server can create and send
> to a syslog?

See the "syslog" method of:

http://www.squid-cache.org/Doc/config/access_log/
http://www.squid-cache.org/Doc/config/cache_store_log/

> The goal would be to be analyse and sort them, and finally extract the
ones
> which are relevant for security purpose.

You might also be interested in:

http://www.squid-cache.org/Misc/log-analysis.html


Regards,


Antony.


--  Forwarded Message Ends  --


This might help:

http://wiki.squid-cache.org/SquidFaq/SquidLogs

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] squid http & https intercept based on DNS server

[James Lay]

On Thu, 2015-11-12 at 09:37 +0300, Ahmad Alzaeem wrote:

> Sorry , didn’t understand , could you explain more ??
> 
> cheers
> 
> -Original Message-
> From: squid-users [mailto:squid-users-boun...@lists.squid-cache.org] On 
> Behalf Of James Lay
> Sent: Thursday, November 12, 2015 12:29 AM
> To: squid-users@lists.squid-cache.org
> Subject: Re: [squid-users] squid http & https intercept based on DNS server
> 
> On 2015-11-11 12:23, Ahmad Alzaeem wrote:
> > Hi guys
> > 
> > I want to ask a question
> > 
> > Assume I have a dns server that resolve all the names to the ip of 
> > squid
> > 
> > So we will have  all websites go to squid
> > 
> > The question is being asked here is :
> > 
> > If I used squid in intercept mode
> > 
> > Will I be able to handle http & https traffic without adding cert and 
> > CA in the clients browsers' ??
> > 
> > Again
> > 
> > Will I have issues with Https in  certs ?
> > 
> > cheers
> > ___
> > squid-users mailing list
> > squid-users@lists.squid-cache.org
> > http://lists.squid-cache.org/listinfo/squid-users
> 
> No.  Certain clients don't even use DNS, but a hardcoded IP (I'm looking at 
> you TextNow).
> 
> James
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


Some applications (I'm thinking mobile apps) may or may not use a
hostname...some may simply connect to an IP address, which makes control
over DNS irrelevant at that point.  Hope that helps.

James 
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] squid http & https intercept based on DNS server

[James Lay]

On 2015-11-11 12:23, Ahmad Alzaeem wrote:

Hi guys

I want to ask a question

Assume I have a dns server that resolve all the names to the ip of
squid

So we will have  all websites go to squid

The question is being asked here is :

If I used squid in intercept mode

Will I be able to handle http & https traffic without adding cert and
CA in the clients browsers' ??

Again

Will I have issues with Https in  certs ?

cheers
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


No.  Certain clients don't even use DNS, but a hardcoded IP (I'm looking 
at you TextNow).


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Fwd: Problems with the List

[James Lay]

On 2015-10-27 09:06 AM, Amos Jeffries wrote:

On 28/10/2015 2:29 a.m., Elvis Altherr wrote:


Hello Admins of the List

Seems there some problems with the list.. i receive strange Mails from
different users watch example below



Thanks. We had a spam run that looks like it was from one of the
subscribed user accounts. I believe it is now being garbaged, but still
in the process of confirming that and clearing the mail backlog.

Please feel free to drop anything sent to the list in the last few hrs
which has the subject line is "Fw: new message".

Sorry for the inconvenience.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


Thanks Amos...last hit I had was:

Oct 27 07:51:53 gateway postfix/cleanup[25879]: 7DAA02C0030: discard: 
header From: Kai-Chieh Ku  from 
lists.squid-cache.org[104.130.201.120]; 
from= 
to= proto=ESMTP helo=


which was about an hour and 20 minutes ago.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] SSL Peek and Splice

[James Lay]

On Thu, 2015-10-01 at 13:26 +0200, Job wrote:

> Hello,
> 
> by reading the 3.5 Squid verson "Peek and splice" features:
> http://wiki.squid-cache.org/Features/SslPeekAndSplice
> 
> i would like to ask you two questions, please:
> 
> 1. in this implementations, i have to install the selfmade Certification 
> Authority as for SSL Bump?
> 2. how can i block domain (dstdomain with squid) with Peek and Splice? It 
> seems not possible by reading the document
> 
> Thank you for your patience and many thanks!
> 
> Francesco
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


I've found that with peek/splice, instead of stare/bump, I did not need
to install the certificate on the end device (daughter got a new phone
and I forgot to install it...still worked anyway...cool).

Config below for exactly what you're wanting...change netblocks to what
you're using and change cert locations and what not.  Before just doing
a copy/paste and go, I would recommend reading the docs to get a better
understanding of what the below directives mean.  The file http_url.txt
is regex so it will have entries like \.apple\.com.

acl localnet src 192.168.1.0/24
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443
acl CONNECT method CONNECT
acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt"

http_access deny !Safe_ports
http_access deny CONNECT !SSL_Ports
http_access allow SSL_ports
http_access allow allowed_http_sites
http_access deny all

acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3

ssl_bump peek all
acl allowed_https_sites ssl::server_name_regex
"/opt/etc/squid/http_url.txt"
ssl_bump splice allowed_https_sites
ssl_bump terminate all

sslproxy_capath /etc/ssl/certs

sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
sslcrtd_children 5

http_port 3128 intercept
https_port 3129 intercept ssl-bump
cert=/opt/etc/squid/certs/sslsplit_ca_cert.pem
cafile=/opt/etc/squid/certs/sslsplit_ca_cert.pem
key=/opt/etc/squid/certs/sslsplit_ca_key.pem
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslflags=NO_SESSION_REUSE

logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni %
ssl::>cert_subject %>Hs %___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] 3.5.8 — SSL Bump questions

[James Lay]

On Fri, 2015-09-11 at 11:25 -0600, James Lay wrote:

> On 2015-09-11 09:39 AM, Alex Rousskov wrote:
> > On 09/11/2015 09:21 AM, James Lay wrote:
> >> On 2015-09-09 08:29 PM, Alex Rousskov wrote:
> >>> Please see http://bugs.squid-cache.org/show_bug.cgi?id=4303
> > 
> > 
> >> Confirming that this now works:
> >> 
> >> ssl_bump peek all
> >> ssl_bump splice step3 allowed_https_sites
> >> ssl_bump terminate all
> > 
> > FWIW, you do not need the "step3" ACL in there any more. The "all" in
> > "peek all" will match step1 and step2 because the peek action is only
> > applicable to step1 and step2.
> > 
> > Alex.
> 
> Thanks Alex...I'll test and report heremy config is shrinking by the 
> day...a good thing :)
> 
> James
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


Confirmed thank you Alex!

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] 3.5.8 — SSL Bump questions

[James Lay]

On 2015-09-11 09:39 AM, Alex Rousskov wrote:

On 09/11/2015 09:21 AM, James Lay wrote:

On 2015-09-09 08:29 PM, Alex Rousskov wrote:

Please see http://bugs.squid-cache.org/show_bug.cgi?id=4303




Confirming that this now works:

ssl_bump peek all
ssl_bump splice step3 allowed_https_sites
ssl_bump terminate all


FWIW, you do not need the "step3" ACL in there any more. The "all" in
"peek all" will match step1 and step2 because the peek action is only
applicable to step1 and step2.

Alex.


Thanks Alex...I'll test and report heremy config is shrinking by the 
day...a good thing :)


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] 3.5.8 — SSL Bump questions

[James Lay]

On 2015-09-09 08:29 PM, Alex Rousskov wrote:

On 09/09/2015 07:06 PM, Dan Charlesworth wrote:

if I change ssl_bump peek step1 to ssl_bump peek all, I get this 
assertion failure:


PeerConnector.cc:747: "!callback"


Please see http://bugs.squid-cache.org/show_bug.cgi?id=4303

Alex.





Confirming that this now works:

ssl_bump peek all
acl allowed_https_sites ssl::server_name_regex 
"/opt/etc/squid/http_url.txt"

ssl_bump splice step3 allowed_https_sites
ssl_bump terminate all

Sep 11 08:56:34 gateway (squid-1): 192.168.1.100 - - 
[11/Sep/2015:08:56:34 -0600] "CONNECT 69.192.193.29:443 HTTP/1.1" 
iadsdk.apple.com - 200 633 TCP_TUNNEL:ORIGINAL_DST peek


Thanks for this Alex.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] 3.5.8 — SSL Bump questions

[James Lay]

On 2015-09-08 02:32 PM, Alex Rousskov wrote:

On 09/08/2015 02:18 PM, James Lay wrote:


I'm currently having great success with 3.5.8 and this
peek/splice only method using transparent intercept:

###
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3

ssl_bump peek step1 all
ssl_bump peek step2 all
acl allowed_https_sites ssl::server_name_regex
"/opt/etc/squid/http_url.txt"
ssl_bump splice step3 allowed_https_sites
ssl_bump terminate all
###



Bugs notwithstanding, the above can be further simplified (in v3.5.8 
and

later):

 acl allowed_https_sites ...
 ssl_bump peek all
 ssl_bump splice allowed_https_sites
 ssl_bump terminate all


HTH,

Alex.


Hey thanks Alex...I will give that a test with 3.5.8.  I also recall in 
earlier builds that "ssl_bump peek all" only matched SNI, but did not 
match the cert subject, which is why I forced it with peeking at step1 
and step2.  Thanks again.


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] 3.5.8 — SSL Bump questions

[James Lay]

On 2015-09-08 01:54 PM, Alex Rousskov wrote:

On 09/07/2015 11:36 PM, Dan Charlesworth wrote:

First, here’s my config (shout out to James Lay):



acl client_hello_peeked at_step SslBump2
ssl_bump splice client_hello_peeked bump_bypass_domains
ssl_bump bump client_hello_peeked


Just in case somebody tries to copy this:

AFAICT, in Squid v3.5.8, the above config does not make sense. Since
client_hello_peeked does not match during step1, no ssl_bump rules will
patch during step1, and so the above is equivalent to:

  ssl_bump splice !all
  ssl_bump bump !all

which, in turn, should be equivalent to:

  ssl_bump splice all

because "splice" is the default ssl_bump action unless Squid has been
"staring". That, in turn, should be nearly equivalent to not using
SslBump at all. There are some side effects related to the
always-performed SslBump step1 actions that you may observe, but I 
doubt

you were after those side effects.

Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


I recall that in testing something similar was proposed, but it did not 
function as intended, but that wasgosh I'm not sure how many revs 
back.  I'm currently having great success with 3.5.8 and this 
peek/splice only method using transparent intercept:


###
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3

ssl_bump peek step1 all
ssl_bump peek step2 all
acl allowed_https_sites ssl::server_name_regex 
"/opt/etc/squid/http_url.txt"

ssl_bump splice step3 allowed_https_sites
ssl_bump terminate all
###

I didn't really have a reason to actually bump and decrypt, just to 
allow/disallow.  I still see peek only 
(http://bugs.squid-cache.org/show_bug.cgi?id=4256) in the logs for both 
successfully spliced and terminated sessions, but eh...I know it's 
working otherwise I'd have unhappy children :D


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] ssl_bump updates coming in 3.5.8

[James Lay]

On Fri, 2015-08-21 at 05:26 -0600, James Lay wrote:

> On Fri, 2015-08-21 at 19:28 +1200, Amos Jeffries wrote: 
> 
> > Hi all,
> > 
> >  Christos has managed (we think) to resolve a fairly major design issue
> > that has been plaguing the 3.5 series peek-and-splice feature so far.
> >  (<http://wiki.squid-cache.org/Features/SslPeekAndSplice>)
> > 
> > The problem was that Squid was not actually following the intended and
> > documented logic of skipping the impossible bumping actions. The patch
> > for that will be in 3.5 snaphots labelled r13895 or later (still waiting
> > on mirror updates as I write this 1-2hrs more maybe).
> > (<http://www.squid-cache.org/Versions/v3/3.5/>)
> > 
> > 
> > Since it is affecting the visible behaviour of squid.conf settings I
> > would like some volunteers to help test it out. Find what problems
> > remain, and let me know what to alert others to in the next formal release.
> > 
> > 
> > We need testing both from those having issues currently, and those who
> > managed to get a trial-and-error config going with older 3.5.
> > 
> > Hopefully, if you are using the at_step workarounds there should not be
> > any visible difference. But some of the at_step tests may be needless now.
> > 
> > Thank you in advance for any assistance.
> > 
> > Amos
> > ___
> > squid-users mailing list
> > squid-users@lists.squid-cache.org
> > http://lists.squid-cache.org/listinfo/squid-users
> 
> 
> Count me inI'll let you know my results...my config is in this
> list...it hasn't changed.
> 
> James 
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


Appears to work fine here:

Squid Cache: Version 3.5.7-20150821-r13895
Service Name: squid
configure options:  '--prefix=/opt' '--with-openssl' '--enable-ssl'
'--enable-ssl-crtd' '--enable-linux-netfilter'
'--enable-follow-x-forwarded-for' '--with-large-files'
'--sysconfdir=/opt/etc/squid' '--enable-external-acl-helpers=none'


Aug 21 06:21:11 gateway (squid-1): 192.168.1.100 - -
[21/Aug/2015:06:21:11 -0600] "CONNECT 69.192.193.247:443 HTTP/1.1"
configuration.apple.com - 200 9 TCP_TUNNEL:ORIGINAL_DST peek
Aug 21 06:21:29 gateway (squid-1): 192.168.1.100 - -
[21/Aug/2015:06:21:29 -0600] "CONNECT 17.173.66.95:443 HTTP/1.1"
pd-st.itunes.apple.com - 200 532 TCP_TUNNEL:ORIGINAL_DST peek
Aug 21 06:21:30 gateway (squid-1): 192.168.1.100 - -
[21/Aug/2015:06:21:30 -0600] "CONNECT 69.192.207.154:443 HTTP/1.1"
init.itunes.apple.com - 200 31123 TCP_TUNNEL:ORIGINAL_DST peek
Aug 21 06:21:30 gateway (squid-1): 192.168.1.100 - -
[21/Aug/2015:06:21:30 -0600] "CONNECT 17.173.66.135:443 HTTP/1.1"
xp.apple.com - 200 657 TCP_TUNNEL:ORIGINAL_DST peek
Aug 21 06:21:30 gateway (squid-1): 192.168.1.100 - -
[21/Aug/2015:06:21:30 -0600] "CONNECT 17.173.66.95:443 HTTP/1.1"
pd-st.itunes.apple.com - 200 2059 TCP_TUNNEL:ORIGINAL_DST peek
Aug 21 06:21:31 gateway (squid-1): 192.168.1.100 - -
[21/Aug/2015:06:21:31 -0600] "CONNECT 17.173.66.73:443 HTTP/1.1"
partiality.itunes.apple.com - 200 679 TCP_TUNNEL:ORIGINAL_DST peek
Aug 21 06:21:32 gateway (squid-1): 192.168.1.100 - -
[21/Aug/2015:06:21:32 -0600] "CONNECT 69.192.193.29:443 HTTP/1.1"
iadsdk.apple.com - 200 409 TCP_TUNNEL:ORIGINAL_DST peek
Aug 21 06:21:32 gateway (squid-1): 192.168.1.100 - -
[21/Aug/2015:06:21:32 -0600] "CONNECT 69.192.193.29:443 HTTP/1.1"
iadsdk.apple.com - 200 409 TCP_TUNNEL:ORIGINAL_DST peek

I still see only peek instead of the final splice/bump in the
logs...hoping that gets resolved soon.  Thanks Alex.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] ssl_bump updates coming in 3.5.8

[James Lay]

On Fri, 2015-08-21 at 19:28 +1200, Amos Jeffries wrote:

> Hi all,
> 
>  Christos has managed (we think) to resolve a fairly major design issue
> that has been plaguing the 3.5 series peek-and-splice feature so far.
>  ()
> 
> The problem was that Squid was not actually following the intended and
> documented logic of skipping the impossible bumping actions. The patch
> for that will be in 3.5 snaphots labelled r13895 or later (still waiting
> on mirror updates as I write this 1-2hrs more maybe).
> ()
> 
> 
> Since it is affecting the visible behaviour of squid.conf settings I
> would like some volunteers to help test it out. Find what problems
> remain, and let me know what to alert others to in the next formal release.
> 
> 
> We need testing both from those having issues currently, and those who
> managed to get a trial-and-error config going with older 3.5.
> 
> Hopefully, if you are using the at_step workarounds there should not be
> any visible difference. But some of the at_step tests may be needless now.
> 
> Thank you in advance for any assistance.
> 
> Amos
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


Count me inI'll let you know my results...my config is in this
list...it hasn't changed.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] ssl_crtd process doesn't start with Squid 3.5.6

[James Lay]

On Fri, 2015-07-24 at 19:15 -0500, Stanford Prescott wrote:
> Thanks for that. Any ideas why I am experiencing that?
> 
> 
> 
> Stan
> 
> 
> 
> 
> On Fri, Jul 24, 2015 at 7:07 PM, James Lay 
> wrote:
> 
> On Fri, 2015-07-24 at 17:25 -0500, Stanford Prescott wrote: 
> 
> > I have a working implementation of Squid 3.5.5 with
> > ssl-bump. When 3.5.5 is started with ssl-bump enabled all
> > the squid and ssl_crtd processes start and Squid functions
> > as intended when bumping ssl sites. However, when I bump
> > Squid to 3.5.6 squid seems to start but ssl_crtd does not
> > and Squid 3.5.6 cannot successfully bump ssl.
> > 
> > 
> > These are the config options I use for both 3.5.5 and 3.5.6.
> > 
> > --enable-storeio="diskd,ufs,aufs" --enable-linux-netfilter \
> > --enable-removal-policies="heap,lru" --enable-delay-pools
> > --libdir=/usr/lib/ \
> > --localstatedir=/var --with-dl --with-openssl
> > --enable-http-violations \
> > --with-large-files --with-libcap --disable-ipv6
> > --with-swapdir=/var/spool/squid \
> >  --enable-ssl-crtd --enable-follow-x-forwarded-for
> > 
> > 
> > 
> > This is the squid.conf file used for both versions.
> > 
> > visible_hostname smoothwallu3
> > 
> > # Uncomment the following to send debug info
> > to /var/log/squid/cache.log
> > debug_options ALL,1 33,2 28,9
> > 
> > # ACCESS CONTROLS
> > #
> > 
> > acl localhostgreen src 10.20.20.1
> > acl localnetgreen src 10.20.20.0/24
> > 
> > acl SSL_ports port 445 443 441 563
> > acl Safe_ports port 80# http
> > acl Safe_ports port 81# smoothwall http
> > acl Safe_ports port 21# ftp 
> > acl Safe_ports port 445 443 441 563# https, snews
> > acl Safe_ports port 70 # gopher
> > acl Safe_ports port 210   # wais  
> > acl Safe_ports port 1025-65535# unregistered ports
> > acl Safe_ports port 280   # http-mgmt
> > acl Safe_ports port 488   # gss-http 
> > acl Safe_ports port 591   # filemaker
> > acl Safe_ports port 777   # multiling http
> > 
> > acl CONNECT method CONNECT
> > 
> > # TAG: http_access
> > #
> > 
> > 
> > 
> > 
> > http_access allow localhost
> > http_access deny !Safe_ports
> > http_access deny CONNECT !SSL_ports
> > 
> > http_access allow localnetgreen
> > http_access allow CONNECT localnetgreen
> > 
> > http_access allow localhostgreen
> > http_access allow CONNECT localhostgreen
> > 
> > # http_port and https_port
> > 
> #
> > 
> > # For forward-proxy port. Squid uses this port to serve
> > error pages, ftp icons and communication with other proxies.
> > 
> #
> > http_port 3127
> > 
> > http_port 10.20.20.1:800 intercept
> > https_port 10.20.20.1:808 intercept ssl-bump
> > generate-host-certificates=on
> > dynamic_cert_mem_cache_size=4MB
> > cert=/var/smoothwall/mods/proxy/ssl_cert/squidCA.pem
> > 
> > 
> > http_port 127.0.0.1:800 intercept
> > 
> > sslproxy_cert_error allow all
> > sslproxy_flags DONT_VERIFY_PEER
> > sslproxy_session_cache_size 4 MB
> > 
> > ssl_bump none localhostgreen
> > 
> > acl step1 at_step SslBump1
> > acl step2 at_step SslBump2
> > ssl_bump peek step1
> > ssl_bump bump all
> > 
> > sslcrtd_program /var/smoothwall/mods/proxy/libexec/ssl_crtd
> > -s /var/smoothwall/mods/proxy/lib/ssl_

Re: [squid-users] ssl_crtd process doesn't start with Squid 3.5.6

[James Lay]

On Fri, 2015-07-24 at 17:25 -0500, Stanford Prescott wrote:
> I have a working implementation of Squid 3.5.5 with ssl-bump. When
> 3.5.5 is started with ssl-bump enabled all the squid and ssl_crtd
> processes start and Squid functions as intended when bumping ssl
> sites. However, when I bump Squid to 3.5.6 squid seems to start but
> ssl_crtd does not and Squid 3.5.6 cannot successfully bump ssl.
> 
> 
> 
> These are the config options I use for both 3.5.5 and 3.5.6.
> 
> 
> --enable-storeio="diskd,ufs,aufs" --enable-linux-netfilter \
> --enable-removal-policies="heap,lru" --enable-delay-pools
> --libdir=/usr/lib/ \
> --localstatedir=/var --with-dl --with-openssl --enable-http-violations
> \
> --with-large-files --with-libcap --disable-ipv6
> --with-swapdir=/var/spool/squid \
>  --enable-ssl-crtd --enable-follow-x-forwarded-for
> 
> 
> 
> 
> 
> This is the squid.conf file used for both versions.
> 
> 
> visible_hostname smoothwallu3
> 
> # Uncomment the following to send debug info
> to /var/log/squid/cache.log
> debug_options ALL,1 33,2 28,9
> 
> # ACCESS CONTROLS
> # 
> acl localhostgreen src 10.20.20.1
> acl localnetgreen src 10.20.20.0/24
> 
> acl SSL_ports port 445 443 441 563
> acl Safe_ports port 80# http
> acl Safe_ports port 81# smoothwall http
> acl Safe_ports port 21# ftp 
> acl Safe_ports port 445 443 441 563# https, snews
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210   # wais  
> acl Safe_ports port 1025-65535# unregistered ports
> acl Safe_ports port 280   # http-mgmt
> acl Safe_ports port 488   # gss-http 
> acl Safe_ports port 591   # filemaker
> acl Safe_ports port 777   # multiling http
> 
> acl CONNECT method CONNECT
> 
> # TAG: http_access
> # 
> 
> 
> 
> http_access allow localhost
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> 
> http_access allow localnetgreen
> http_access allow CONNECT localnetgreen
> 
> http_access allow localhostgreen
> http_access allow CONNECT localhostgreen
> 
> # http_port and https_port
> #
> 
> # For forward-proxy port. Squid uses this port to serve error pages,
> ftp icons and communication with other proxies.
> #
> http_port 3127
> 
> http_port 10.20.20.1:800 intercept
> https_port 10.20.20.1:808 intercept ssl-bump
> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> cert=/var/smoothwall/mods/proxy/ssl_cert/squidCA.pem
> 
> 
> http_port 127.0.0.1:800 intercept
> 
> sslproxy_cert_error allow all
> sslproxy_flags DONT_VERIFY_PEER
> sslproxy_session_cache_size 4 MB
> 
> ssl_bump none localhostgreen
> 
> acl step1 at_step SslBump1
> acl step2 at_step SslBump2
> ssl_bump peek step1
> ssl_bump bump all
> 
> sslcrtd_program /var/smoothwall/mods/proxy/libexec/ssl_crtd
> -s /var/smoothwall/mods/proxy/lib/ssl_db -M 4MB
> sslcrtd_children 5
> 
> http_access deny all
> 
> cache_replacement_policy heap GDSF
> memory_replacement_policy heap GDSF
> 
> # CACHE OPTIONS
> #
> 
> cache_effective_user squid
> cache_effective_group squid
> 
> cache_swap_high 100
> cache_swap_low 80
> 
> cache_access_log stdio:/var/log/squid/access.log
> cache_log /var/log/squid/cache.log
> cache_mem 64 MB
> 
> cache_dir diskd /var/spool/squid/cache 1024 16 256
> 
> maximum_object_size 33 MB
> 
> minimum_object_size 0 KB
> 
> 
> request_body_max_size 0 KB
> 
> # OTHER OPTIONS
> #
> 
> #via off
> forwarded_for off
> 
> pid_filename /var/run/squid.pid
> 
> shutdown_lifetime 30 seconds
> icp_port 3130
> 
> half_closed_clients off
> icap_enable on
> icap_send_client_ip on
> icap_send_client_username on
> icap_client_username_encode off
> icap_client_username_header X-Authenticated-User
> icap_preview_enable on
> icap_preview_size 1024
> icap_service service_avi_req reqmod_precache
> icap://localhost:1344/squidclamav bypass=off
> adaptation_access service_avi_req allow all
> icap_service service_avi_resp respmod_precache
> icap://localhost:1344/squidclamav bypass=on
> adaptation_access service_avi_resp allow all
> 
> umask 022
> 
> logfile_rotate 0
> 
> strip_query_terms off
> 
> redirect_program /usr/sbin/squidGuard
> url_rewrite_children 5
> 
> 
> 
> And the cache.log file when starting 3.5.6 with debug options on in
> squid.conf
> 
> 
> 2015/07/24 17:15:06.230| Acl.cc(380) ~ACL: freeing ACL
> adaptation_access
> 2015/07/24 17:15:06.230| Acl.cc(380) ~ACL: freeing ACL
> adaptation_access
> 2015/07/24 17:15:06.230| Acl.cc(380) ~ACL: freeing ACL 
> 2015/07/24 17:15:06.230| Acl.cc(380) ~ACL: freeing ACL 
> 20

Re: [squid-users] RE Peek and Splice error SSL_accept failed

[James Lay]

On Fri, 2015-07-24 at 12:09 +, Sebastian Kirschner wrote:

> Hi ,
> 
> I minimized the configuration a little bit(you could see it at the bottom of 
> these message).
> 
> Also I still try to understand why these error happen , I increased the Debug 
> level and saw that squid tried 48 times to peek but failed.
> At the end It says that it got an "Hello", does it mean that squid received 
> after 48 tries the "Hello" ?
> 
> If yes why it does need so many tries ?
> 
> -> Part of debug log <-
> 2015/07/24 11:05:42.866 kid1| client_side.cc(4242) clientPeekAndSpliceSSL: 
> Start peek and splice on FD 11
> 2015/07/24 11:05:42.866 kid1| bio.cc(120) read: FD 11 read 11 <= 11
> 2015/07/24 11:05:42.866 kid1| bio.cc(146) readAndBuffer: read 11 out of 11 
> bytes
> 2015/07/24 11:05:42.866 kid1| bio.cc(150) readAndBuffer: recorded 11 bytes of 
> TLS client Hello
> 2015/07/24 11:05:42.866 kid1| ModEpoll.cc(116) SetSelect: FD 11, type=1, 
> handler=1, client_data=0x7effbd078458, timeout=0
> 2015/07/24 11:05:42.866 kid1| client_side.cc(4245) clientPeekAndSpliceSSL: 
> SSL_accept failed.
> .
> .
> .
> 2015/07/24 11:05:42.874 kid1| client_side.cc(4242) clientPeekAndSpliceSSL: 
> Start peek and splice on FD 11
> 2015/07/24 11:05:42.874 kid1| bio.cc(120) read: FD 11 read 6 <= 11
> 2015/07/24 11:05:42.874 kid1| bio.cc(146) readAndBuffer: read 6 out of 11 
> bytes
> 2015/07/24 11:05:42.874 kid1| bio.cc(150) readAndBuffer: recorded 6 bytes of 
> TLS client Hello
> 2015/07/24 11:05:42.875 kid1| SBuf.cc(152) assign: SBuf2040 from c-string, 
> n=0)
> 2015/07/24 11:05:42.875 kid1| SBuf.cc(152) assign: SBuf2038 from c-string, 
> n=13)
> 2015/07/24 11:05:42.875 kid1| ModEpoll.cc(116) SetSelect: FD 11, type=1, 
> handler=1, client_data=0x7effbd078458, timeout=0
> 2015/07/24 11:05:42.875 kid1| client_side.cc(4245) clientPeekAndSpliceSSL: 
> SSL_accept failed.
> 2015/07/24 11:05:42.875 kid1| SBuf.cc(152) assign: SBuf2025 from c-string, 
> n=4294967295)
> 2015/07/24 11:05:42.875 kid1| client_side.cc(4259) clientPeekAndSpliceSSL: I 
> got hello. Start forwarding the request!!!
> 
> -> new configuration <-
> acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
> 
> acl SSL_ports port 443
> acl Safe_ports port 80  # http
> acl Safe_ports port 21  # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70  # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535  # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl CONNECT method CONNECT
> 
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access allow localhost manager
> http_access deny manager
> http_access allow localnet
> http_access allow localhost
> http_access deny all
> 
> # Listening Ports
> http_port 127.0.0.1:3120
> http_port 192.168.1.104:3128 intercept
> https_port 192.168.1.104:3129 intercept ssl-bump 
> generate-host-certificates=on dynamic_cert_mem_cache_size=10MB 
> cert=/etc/squid3/ssl_cert/myCA.pem
> 
> # some configuration options
> cache_effective_user proxy
> cache_effective_group proxy
> access_log /var/squid/logs/access.log
> cache_log /var/squid/logs/cache.log
> pinger_enable on
> pinger_program /lib/squid3/pinger
> sslproxy_capath /etc/ssl/certs
> sslcrtd_program /lib/squid3/ssl_crtd -s /var/squid/certs -M 4MB -b 2048
> 
> #ACLs
> acl step1 at_step SslBump1
> acl step2 at_step SslBump2
> acl step3 at_step SslBump3
> acl bypass ssl::server_name www.google.de
> 
> ssl_bump peek step1
> ssl_bump splice bypass step2
> ssl_bump bump all
> 
> # Debugging if needeed
> debug_options all,6 6,0 16,0 18,0 19,0 20,0 32,0 47,0 79,0 90,0 92,0
> 
> # Leave coredumps in the first cache dir
> coredump_dir /var/spool/squid3
> 
> #
> # Add any of your own refresh_pattern entries above these.
> #
> refresh_pattern ^ftp:   144020% 10080
> refresh_pattern ^gopher:14400%  1440
> refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
> refresh_pattern .   0   20% 4320
> 
> 
> Mit freundlichen Grüßen / Best Regards
> 
> Sebastian
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


Is that all sites or just a few special sites?

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Transparent Proxy Configuration

[James Lay]

On 2015-06-30 12:21 PM, Chris Greene wrote:

I’ve had Squid running on Ubuntu for a few weeks.  I’d configured the
proxy settings in the browsers.  Everything has been working well and
I've been pleased with the results.  But now I need to make this a
transparent proxy and I’m running into trouble & need some help.

I’ve got a Destination NAT rule set up on my router to forward TCP
port 80 traffic to my proxy.  And I removed proxy configuration
settings from the browsers.  After enabling this DNAT rule, I see
requests being logged to /var/log/squid3/access.log.

Results when navigating to http://www.google.com:
The following error was encountered while trying to retrieve the URL: /
  Invalid URL
Some aspect of the requested URL is incorrect.
Some possible problems are:
-Missing or incorrect access protocol (should be “http://” or similar)
-Missing hostname
-Illegal double-escape in the URL-Path
-Illegal character in hostname; underscores are not allowed.


Next, I added "intercept" to http_port like so:
  "http_port  192.166.2.55:3128  intercept"
Results: Access Denied.

My abbreviated /etc/squid3/squid.conf looks like this:
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localhost
http_access allow all

I'm new to Squid/Ubuntu, so I likely overlooked something.  What am I
missing?  What troubleshooting step(s) should I take next?
-DG


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


What's your DNAT line?  Assuming squid is on the box that you're running 
the DNAT line on...here's mine...redirect is all you need if the 
firewall/gateway is on the same box as squid:


$IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport 
80 -j REDIRECT --to-port 3128


And parts of my squid.conf:

acl localnet src 192.168.1.0/24

acl Safe_ports port 80
acl Safe_ports port 443

acl CONNECT method CONNECT
acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt"

http_access deny !Safe_ports
http_access deny CONNECT !SSL_Ports

http_access allow SSL_ports
http_access allow localnet
http_access deny all

http_port 3128 intercept


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Questions Regarding Transparent Proxy, HTTPS, and ssl_bump

[James Lay]

On Thu, 2015-06-25 at 08:06 -0400, Tom Mowbray wrote:
> James,
> 
> 
> 
> Thank for for your help.  Now that I have a better understanding of
> how the https traffic is handled, I've been able to get things working
> as intended.
> 
> 
> 
> 
> 
> -
> 
> Tom Mowbray
> 
> tmowb...@dalabs.com
> 703-829-6694
> 
> 
> 
> On Wed, Jun 24, 2015 at 2:05 PM, James Lay 
> wrote:
> 
> On 2015-06-24 11:46 AM, Tom Mowbray wrote:
> 
> James,
> 
> Yes, as a matter of fact I have read through those
> exact posts and
> modeled my config very similarly.  What I have found
> is that, however,
> when the line "http_access allow SSL_ports" is placed
> above the
> ssl_bump stuff and other acl's (as you have it), it
> seems to simply
> allow ALL https without doing any filtering
> whatsoever.
> 
> Thanks for the response.
> 
> -Tom Mowbray
> _tmowbray@dalabs.com_
> _703-829-6694_
> 
> 
> 
> On Wed, Jun 24, 2015 at 1:31 PM, James Lay
> 
> wrote:
> 
> 
> On 2015-06-24 09:41 AM, Tom Mowbray wrote:
> 
> 
> Squid 3.5.5
> 
> I seem to have some confusion about
> how acl lists are processed
> in
> squid.conf regarding the handling of
> SSL (HTTPS) traffic,
> attempting
> to use ssl_bump directives with
> transparent proxy.
> 
> Based on available documentation, I
> believe my squid.conf is
> correct,
> however it never seems to actually
> behave as expected.
> 
> I define the SSL port, as usual:
> 
> acl SSL_ports port 443
> 
> But here's where my confusion lies...
> Many state to place the
> following line above the ssl_bump
> configuration lines:
> 
> http_access allow SSL_ports
> 
> However when I do this, it appears to
> simply stop processing any
> other
> rules and allows ALL https traffic
> through the proxy (which is
> actually how I'd expect a standard ACL
> list to operate, but then
> how
> do I actually filter the traffic
> though our content-based ACL
> lists?).
> If I put the above line below the
> ssl_bump configuration options
> in
> my squid.conf, then it appears to BUMP
> all, even though I've told
> the
> config to SPLICE all https traffic,
> which doesn't work for our
> deployment.
> 
> So, does squid actually continue to
> process the https traffic
> using
> the ssl_bump rules if the "http_access
> allow SSL_ports" line is
> placed
> 

Re: [squid-users] Questions Regarding Transparent Proxy, HTTPS, and ssl_bump

[James Lay]

On Thu, 2015-06-25 at 13:57 +1200, Jason Haar wrote:

> On 25/06/15 06:05, James Lay wrote:
> > openssl s_client -connect x.x.x.x:443 
> Just a FYI but you can make openssl do SNI which helps debugging (ie
> doing it your way and then doing it with SNI)
> 
> openssl s_client -connect x.x.x.x:443 -servername www.site.name
> 
> (that will allow squid to see www.site.name as the SNI)
> 


Thanks Jasonappreciate that heads up.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Questions Regarding Transparent Proxy, HTTPS, and ssl_bump

[James Lay]

On 2015-06-24 11:46 AM, Tom Mowbray wrote:

James,

Yes, as a matter of fact I have read through those exact posts and
modeled my config very similarly.  What I have found is that, however,
when the line "http_access allow SSL_ports" is placed above the
ssl_bump stuff and other acl's (as you have it), it seems to simply
allow ALL https without doing any filtering whatsoever.

Thanks for the response.

-Tom Mowbray
_tmowbray@dalabs.com_
_703-829-6694_

On Wed, Jun 24, 2015 at 1:31 PM, James Lay 
wrote:


On 2015-06-24 09:41 AM, Tom Mowbray wrote:


Squid 3.5.5

I seem to have some confusion about how acl lists are processed
in
squid.conf regarding the handling of SSL (HTTPS) traffic,
attempting
to use ssl_bump directives with transparent proxy.

Based on available documentation, I believe my squid.conf is
correct,
however it never seems to actually behave as expected.

I define the SSL port, as usual:

acl SSL_ports port 443

But here's where my confusion lies... Many state to place the
following line above the ssl_bump configuration lines:

http_access allow SSL_ports

However when I do this, it appears to simply stop processing any
other
rules and allows ALL https traffic through the proxy (which is
actually how I'd expect a standard ACL list to operate, but then
how
do I actually filter the traffic though our content-based ACL
lists?).
If I put the above line below the ssl_bump configuration options
in
my squid.conf, then it appears to BUMP all, even though I've told
the
config to SPLICE all https traffic, which doesn't work for our
deployment.

So, does squid actually continue to process the https traffic
using
the ssl_bump rules if the "http_access allow SSL_ports" line is
placed
above it in the configuration?

I should note that we've been able to get filtering to work
correctly
when using our configuration in NON-transparent mode, however our
goal
is get this functionality working as a transparent proxy. We're
unable to load our self-signed cert onto client machines that
will be
accessing the proxy, so using the "bump" or man-in-the-middle
style
https filtering isn't a viable option for us.

Any help or advice is appreciated!

Thanks,

Tom


Tom,

You kinda have to change the way you think about filtering when it
comes to Squid 3.5.5 and SSL(TLS). Normal http traffic is
easyhere's where we're trying to go and here's a list of place
we're alloed to go...simple.

Not so with SSL(TLS). Squid can't filter, since Squid may or may
not know where we're going...and that's the issue..it's where those
ssl_bump atStep ACL's come in. Some sites when you connect to them
are easy-ish..when you connect your device sends a "Server Name
Information" (SNI) that says where you're going. Other sites don't
have any information until you complete the SSL handshake (how can
you filter a site name, until squid KNOWS the site or at least
domain name?).

If you're still wanting to go through with transparent (intercept)
proxy with SSL, search through the list for my SSL Deep dive
posts...that config is working for me so far (granted, not in an
enterprise environment). However, as Amos said,if you choose
not to install the cert on the client machines, you are either a)
going to be out of luck on LOT'S of websites because they will fail
the SSL handshake, or b) teaching your users to ignore the security
warnings of their browser'sneither of which is a good thing.

Hope that helps.

James



Tom,

You are right...that absolutely will allow all SSL initially...the 
filtering is down lower in the config here:


With single list of regex sites/domains like \.google\.com...peek, 
splice, no bump...I'm currently using this config section.


ssl_bump peek step1 all
ssl_bump peek step2 all
acl allowed_https_sites ssl::server_name_regex 
"/opt/etc/squid/http_url.txt"

ssl_bump splice step3 allowed_https_sites
ssl_bump terminate all


With broken acl list of networks list 208.85.40.0/21
###
ssl_bump peek step1 broken
ssl_bump peek step2 broken
ssl_bump splice broken
ssl_bump peek step1 all
ssl_bump peek step2 all
acl allowed_https_sites ssl::server_name_regex 
"/opt/etc/squid/http_url.txt"

ssl_bump bump allowed_https_sites
ssl_bump terminate all

In both configs above, the SNI and server names are checked, bounced off 
the http_url.txt list, and if the site/domain is NOT in the list the ssl 
session is terminated.  The big drag is, you won't be able to see that 
in the squid logs.  I have a bug open ( I don't remember the number :( ) 
to show this in the logs...so far in my setup I only see the first peek, 
nothing after that.  You can test the above setups with:

Re: [squid-users] Questions Regarding Transparent Proxy, HTTPS, and ssl_bump

[James Lay]

On 2015-06-24 09:41 AM, Tom Mowbray wrote:

Squid 3.5.5

I seem to have some confusion about how acl lists are processed in
squid.conf regarding the handling of SSL (HTTPS) traffic, attempting
to use ssl_bump directives with transparent proxy.

Based on available documentation, I believe my squid.conf is correct,
however it never seems to actually behave as expected.

I define the SSL port, as usual:

acl SSL_ports port 443

But here's where my confusion lies... Many state to place the
following line above the ssl_bump configuration lines:

http_access allow SSL_ports

However when I do this, it appears to simply stop processing any other
rules and allows ALL https traffic through the proxy (which is
actually how I'd expect a standard ACL list to operate, but then how
do I actually filter the traffic though our content-based ACL lists?).
 If I put the above line below the ssl_bump configuration options in
my squid.conf, then it appears to BUMP all, even though I've told the
config to SPLICE all https traffic, which doesn't work for our
deployment.

So, does squid actually continue to process the https traffic using
the ssl_bump rules if the "http_access allow SSL_ports" line is placed
above it in the configuration?

I should note that we've been able to get filtering to work correctly
when using our configuration in NON-transparent mode, however our goal
is get this functionality working as a transparent proxy.  We're
unable to load our self-signed cert onto client machines that will be
accessing the proxy, so using the "bump" or man-in-the-middle style
https filtering isn't a viable option for us.

Any help or advice is appreciated!

Thanks,

Tom


Tom,

You kinda have to change the way you think about filtering when it comes 
to Squid 3.5.5 and SSL(TLS).  Normal http traffic is easyhere's 
where we're trying to go and here's a list of place we're alloed to 
go...simple.


Not so with SSL(TLS).  Squid can't filter, since Squid may or may not 
know where we're going...and that's the issue..it's where those ssl_bump 
atStep ACL's come in.  Some sites when you connect to them are 
easy-ish..when you connect your device sends a "Server Name Information" 
(SNI) that says where you're going.  Other sites don't have any 
information until you complete the SSL handshake (how can you filter a 
site name, until squid KNOWS the site or at least domain name?).


If you're still wanting to go through with transparent (intercept) proxy 
with SSL, search through the list for my SSL Deep dive posts...that 
config is working for me so far (granted, not in an enterprise 
environment).  However, as Amos said,if you choose not to install 
the cert on the client machines, you are either a) going to be out of 
luck on LOT'S of websites because they will fail the SSL handshake, or 
b) teaching your users to ignore the security warnings of their 
browser'sneither of which is a good thing.


Hope that helps.

James

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Quick peek-splice clarification

[James Lay]

On Tue, 2015-06-23 at 09:11 +0200, Klavs Klavsen wrote:

> Hi James,
> 
> Did you ever find an answer for this?
> 
> James Lay wrote on 06/11/2015 02:16 AM:
> > All,
> >
> >  From the docs at:
> >
> > http://wiki.squid-cache.org/Features/SslPeekAndSplice
> >
> > *peek*
> >
> >
> > step1, step2
> >
> >
> > Receive SNI and client certificate (step1), or server certificate
> > (step2) while preserving the possibility of splicing the connection.
> > Peeking at the server certificate usually precludes future bumping of
> > the connection (see Limitations). This action is the focus of this project.
> >
> >
> > *stare*
> >
> >
> > step1, step2
> >
> >
> > Receive SNI and client certificate (step1), or server certificate
> > (step2) while preserving the possibility of bumping the connection.
> > Staring at the server certificate usually precludes future splicing of
> > the connection. Currently, we are not aware of any work being done to
> > support this action.
> >
> >
> >
> > I see a lot of:
> >
> > ssl_bump peek all
> >
> > Does this perform both step1 with SNI and client cert, AND server cert?
> > Thank you.
> >
> > James
> >
> >
> > ___
> > squid-users mailing list
> > squid-users@lists.squid-cache.org
> > http://lists.squid-cache.org/listinfo/squid-users
> >
> 
> 


Hi Klavs,

I did not.  I can tell you in my testing that:

ssl_bump peek step1 all
ssl_bump peek step2 all

versus

ssl_bump peek all

Did not give me the same results, so I'm going to assume a single
statement only performs SNI lookup, but maybe someone else on the list
has a better answer.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Properly filtering http and https traffic in a transparent proxy environment

[James Lay]

Resending this with photobucket links instead of including images:

http://i290.photobucket.com/albums/ll269/DigiDemon/allowed.png
http://i290.photobucket.com/albums/ll269/DigiDemon/terminate.png


Hey All,

Sohere's what I have for filtering http and https in the same
instance.  This is using iptables with -j REDIRECT lines.  Below is my
entire squid.conf, documented as well as I can:

#allow local network to connect to squid
acl localnet src 192.168.1.0/24

#safe ports are 80 and 443 in one acl, port 443 is another acl
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443

#allow the http CONNECT method
acl CONNECT method CONNECT

#our regex list of sites and domains that we allow ie, www\.apple\.com
and \.google\.com
acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt"

#we don't want to allow anything besides port 80 and port 443
http_access deny !Safe_ports

#we don't want CONNECT if we're not going to port 443
http_access deny CONNECT !SSL_Ports

#since we may not know the https site we're going to (ie connect direct
by IP), we must initially allow all https
http_access allow SSL_ports

#we allow http, but only sites and domains in our regex http_url.txt
list above
http_access allow allowed_http_sites

#drop any other http requests that are not in our regex http_url.txt
list above
http_access deny all

#break out the ssl_bump process by steps
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3

#look for site or domain name either by SNI in request (step1), or
server subject in certificate in response (step2)
ssl_bump peek step1 all
ssl_bump peek step2 all

#see if the server name we obtained from the previous peek's above are
in our http_url.txt list above
acl allowed_https_sites ssl::server_name_regex
"/opt/etc/squid/http_url.txt"

#if the server name is in our http_url.txt, allow it 
ssl_bump splice step3 allowed_https_sites

#if the server name is not in our http_url.txt terminate the handshake
it
ssl_bump terminate all

#cert path and allow all the ssl options
sslproxy_capath /etc/ssl/certs
sslproxy_options ALL

#standard crtdaemon options
sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
sslcrtd_children 5

#intercept 3128 for port 80, and 3129 for port 443.  Cert, cacert (these
are the same, read on the list this fixed an issue), and key, generate
ssl certs
http_port 3128 intercept
https_port 3129 intercept ssl-bump
cert=/opt/etc/squid/certs/sslsplit_ca_cert.pem
cafile=/opt/etc/squid/certs/sslsplit_ca_cert.pem
key=/opt/etc/squid/certs/sslsplit_ca_key.pem
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslflags=NO_SESSION_REUSE

#normal-ish log format, but we want to see the SNI, server cert subject,
and how we are bumping
logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni %
ssl::>cert_subject %>Hs %
;; ANSWER SECTION:
api.textnow.me. 600 IN A 209.59.180.48

>From the client:
[09:34:27 jlay@analysis:~/dev/squid$] openssl s_client -connect
209.59.180.48:443
CONNECTED(0003)
depth=1 C = US, ST = Arizona, L = Scottsdale, O = "GoDaddy.com, Inc.",
OU = http://certificates.godaddy.com/repository, CN = Go Daddy Secure
Certification Authority, serialNumber = 07969287
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
0 s:/OU=Domain Control Validated/CN=*.textnow.me


GET / HTTP/1.1

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 11 Jun 2015 15:36:54 GMT
Content-Type: text/plain
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.5.16
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: no-cache, no-store, must-revalidate, max-age=0

Server Squid log entry:
Jun 11 09:36:55 gateway (squid-1): 192.168.1.6 - - [11/Jun/2015:09:36:55
-0600] "CONNECT 209.59.180.48:443 HTTP/1.1" - - 200 364
TCP_TUNNEL:ORIGINAL_DST peek

I also notice that the CN does not show up in the logs...it WAS spliced
however because it matches our http_url.txt.  From the above, it appears
that only the step1 is getting logged.   The below log entry was used
with wget (sends SNI by default):
Jun 11 08:51:05 gateway squid: 192.168.1.6 - - [11/Jun/2015:08:51:05
-0600] "CONNECT 23.211.252.28:443 HTTP/1.1" www.apple.com - 200 14388
TCP_TUNNEL:ORIGINAL_DST peek

The above shows that I logged and retrieved my SNI at step1the
subsequent splice was not logged.  I still note that terminates do not
get logged ( I have a bug open for that but I think the core bug may be
that when using atStep you only see step1 regardless, but I could be
wrong).  I'm enclosing an image of what a https terminate looks like
from the server and the client (msn.com isn't in the http_url.txt), and
also what an https allow looks like (apple.com is in the list).

That's it.  I can verify that the above works for a single list of
allowed hosts(www.apple.com) and domains (google.com).  If there's
something I missed...something wrong...ways to improve...ANYTHING for
the betterment of Squid users please don't he

[squid-users] Quick peek-splice clarification

[James Lay]

All,

>From the docs at:

http://wiki.squid-cache.org/Features/SslPeekAndSplice

peek


step1, step2


Receive SNI and client
certificate (step1), or
server certificate
(step2) while preserving
the possibility of
splicing the connection.
Peeking at the server
certificate usually
precludes future bumping
of the connection (see
Limitations). This
action is the focus of
this project.


stare


step1, step2


Receive SNI and client
certificate (step1), or
server certificate
(step2) while preserving
the possibility of
bumping the connection.
Staring at the server
certificate usually
precludes future
splicing of the
connection. Currently,
we are not aware of any
work being done to
support this action.



I see a lot of:

ssl_bump peek all

Does this perform both step1 with SNI and client cert, AND server cert?
Thank you.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Installing certificate on Andriod to use with SSL-bump

[James Lay]

On 2015-06-10 10:22 AM, Amos Jeffries wrote:

On 10/06/2015 4:46 p.m., dkandle wrote:
I would like to be able to inspect traffic from my android device. I 
have a
transparent squid proxy working with SSL bump (using WiFi to get 
traffic
through my proxy server). Everything works fine as long as I go 
through a
browser. But I would like to see the other traffic which the OS and 
other
apps are sending. Squid uses a certificate I generated for the web 
sites and

I create an exception for those without issue.
If I install my certificate on the phone will it then accept the 
certificate

when squid returns it during the ssl setup?


Maybe.


To be clear, I see the phone use
port 443 to setup a secure session. However it rejects the certificate 
(as

it should) and terminates the session with no data being passed. I can
install my certificate on the phone, but will the android OS use that
certificate for all services or only for browser sessions?


Maybe.


If not, is there
some other way I can get my fake certificate accepted for all sessions 
for

which it is used?


Only by adding the CA cert your Squid signs with to the OS certificate
set. Whether it is actually used from there is application specific and
none of us have control over that.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


What kinda device?  I've put my ca cert on a couple Android 
devices...ranging from just email the cert and import all the way to 
cracking open a certificate .db file and inserting.


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] ssl_crtd breaks after short time

[James Lay]

On Tue, 2015-06-09 at 21:39 +0200, Klavs Klavsen wrote:

> Amos Jeffries wrote on 2015-06-09 17:10:
> [CUT]
> > You have to first configure ssl_bump in a way that lets Squid receive
> > the clientHello message (step1 -> peek) AND the serverHello message
> > (step2 -> peek). Then you can use those cert details to bump (step3 ->
> > bump).
> > The config is quite simple:
> >   ssl_bump peek all
> >   ssl_bump bump all
> > 
> I have this:
> ssl_bump peek step1 broken
> ssl_bump peek step2 broken
> ssl_bump splice broken
> ssl_bump peek step1 all
> ssl_bump peek step2 all
> ssl_bump bump all
> 
> > 
> > But there are cases like the client is resuming a previous TLS session
> > where there is no certificates involved. Squid cannot do anything, so it
> > automatically splices (3.5.4+ at least do). Or if you have configured
> > your Squid in a way that there are no mutually supported ciphers.
> > 
> 
> My client is curl.. I don't think that its caching any TLS sessions.
> 
> > 
> > It may just be your ssl_bump rules. But given that this is a google
> > domain there is a strong chance that you are encountering one of those
> > special case.
> >
> I'd like squid to disallow queries where it cannot see what domain name
> / url is going to be accessed.
> 
> I'd like all GET/POST etc. requests to go through squid - so they are
> controlled by the normal http_access rules as http (intercepted) is
> currently.
> 
> This worked with 3.4.12 :( (but only for 30 minutes or less)
> 
> You saw my full config.. how is it supposed to look with 3.5.5, for this
> to work as it did with 3.4.12 ?
> 
> sorry I'm a bit frustrated.. I can't seem to grasp what changed from
> 3.4.12 to 3.5.5, which means I suddenly can't filter https traffic
> anymore :(
> 


Gents,

I'm going to spin this off into a new thread..."Filtering http and https
traffic" sometime later today.  I have some questions, and maybe
solutions.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Utilities for testing question

[James Lay]

On Sat, 2015-06-06 at 13:49 +1200, Amos Jeffries wrote:

> On 6/06/2015 12:35 p.m., James Lay wrote:
> > All,
> > 
> > I'm looking for a command line app like wget or curl that I can use to
> > test TLS.  I'm trying to find out how to send a get request without
> > sending the SNI.  Any pointers would be appreciated.  Thank you.
> 
> The latest squidclient tool built with Squid has basic HTTPS
> capabilities and does not send SNI.
> 
> Amos
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


Thanks Amos...looks like in order to test intercept I'll need to run it
from a remote machine...luckily I can do that :)  Thanks again.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Utilities for testing question

[James Lay]

All,

I'm looking for a command line app like wget or curl that I can use to
test TLS.  I'm trying to find out how to send a get request without
sending the SNI.  Any pointers would be appreciated.  Thank you.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Looking for a recomendation for tutorial for transparent proxy under Ubuntu

[James Lay]

On 2015-06-01 10:40 AM, dkandle wrote:
I am using Ubuntu 14.04 on a server with multiple NICs. I would like to 
set

it up as a transparent proxy. I have the router working and I had squid
working as an explicit proxy (where I set the IP address of the server 
as

the proxy in my client's browser).
Is there a good tutorial which covers this set-up? I've tried setting 
the

iptables as some have advised but it has issues.
It is not at all clear to me how squid will know which interface faces 
the

Internet and which faces my client's subnet.

Thanks



--
View this message in context:
http://squid-web-proxy-cache.1019090.n4.nabble.com/Looking-for-a-recomendation-for-tutorial-for-transparent-proxy-under-Ubuntu-tp4671472.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


The official tutorials:

http://wiki.squid-cache.org/ConfigExamples#Interception

You'll most likely want:

http://wiki.squid-cache.org/ConfigExamples/Intercept/LinuxDnat

Of interest is the lack of interface specification, so here's what I'm 
using on a box that has an internal nic(192.168.1.0/24) and an external 
nic(real world external IP):


$IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport 
80 -j REDIRECT --to-port 3128
$IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport 
443 -j REDIRECT --to-port 3129


This redirects traffic from clients coming in on eth0 to Squid listening 
process on eth0.  If your squid listening process is not on the same 
nic, you'll need to use DNAT instead:


$IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport 
80 -j DNAT --to-destination ip.that.squid.listens.on:3128
$IPTABLES -t nat -A PREROUTING -i eth0 -s 192.168.1.96/28 -p tcp --dport 
443 -j DNAT --to-destination  ip.that.squid.listens.on:3129


Hope that helps.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] ssl_bump and SNI

[James Lay]

On Mon, 2015-06-01 at 12:12 +1000, Nathan Hoad wrote:

> Hello,
> 
> Here are some excerpts of what I've used, and an example Python helper:
> 
> https_port 60099 intercept ssl-bump tcpkeepalive
> cert=/path/to/cert.pem key=/path/to/key.pem options=NO_SSLv2,NO_SSLv3
> generate-host-certificates=on
> 
> external_acl_type sni ttl=30 concurrency=X children-max=Y
> children-startup=Z %ssl::>sni /path/to/your/helper
> 
> acl sni_exclusions external sni
> acl tcp_level at_step SslBump1
> acl client_hello_peeked at_step SslBump2
> 
> ssl_bump peek tcp_level all
> ssl_bump splice client_hello_peeked sni_exclusions
> ssl_bump bump all
> 
> Helper:
> 
> import sys
> 
> line = sys.stdin.read()
> 
> # run loop until an empty read, which indicates the process should shut down.
> while line:
> concurrency_id, sni = line.split()
> 
> if sni == 'wellsfargo.com':
> sys.stdout.write('%s OK\n' % concurrency_id)
> else:
> sys.stdout.write('%s ERR\n' % concurrency_id)
> 
> line = sys.stdin.read()
> 
> Hope that helps,
> 
> Nathan.
> 
> On 30 May 2015 at 01:14, James Lay  wrote:
> > On 2015-05-29 08:57 AM, Nathan Hoad wrote:
> >>
> >> Yes, I have it working on about a dozen deployments so far, using an
> >> external ACL to make bumping decisions based on the SNI server name
> >> and a few other things. No complaints from me, it Just Works.
> >> On 29/05/2015 5:50 pm, "sp_"  wrote:
> >>
> >>> Hello,
> >>>
> >>> does anyone have the working squid 3.5 with intercept + https?
> >>> I've googled a lot, but seems there is no any positive experience
> >>> with it.
> >>>
> >>> --
> >>> View this message in context:
> >>>
> >>
> >> http://squid-web-proxy-cache.1019090.n4.nabble.com/ssl-bump-and-SNI-tp4670207p4671432.html
> >>>
> >>> [1]
> >>> Sent from the Squid - Users mailing list archive at Nabble.com.
> >>> ___
> >>> squid-users mailing list
> >>> squid-users@lists.squid-cache.org
> >>> http://lists.squid-cache.org/listinfo/squid-users [2]
> >>
> >>
> >>
> >> Links:
> >> --
> >> [1]
> >>
> >> http://squid-web-proxy-cache.1019090.n4.nabble.com/ssl-bump-and-SNI-tp4670207p4671432.html
> >> [2] http://lists.squid-cache.org/listinfo/squid-users
> >>
> >> ___
> >> squid-users mailing list
> >> squid-users@lists.squid-cache.org
> >> http://lists.squid-cache.org/listinfo/squid-users
> >
> >
> > Nathan,
> >
> > Care to post your config and external helper?  I know I'd love to see
> > concrete examples.  Thank you.
> >
> > James
> >
> > ___
> > squid-users mailing list
> > squid-users@lists.squid-cache.org
> > http://lists.squid-cache.org/listinfo/squid-users


Thank you Nathan.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Ssl-bump deep dive (intercept last post and final thoughts)

[James Lay]

On Mon, 2015-06-01 at 13:00 +1200, Amos Jeffries wrote:

> On 1/06/2015 11:56 a.m., James Lay wrote:
> > So this has been REALLY good!  The tl;dr:  ssl-bumping is pretty easy
> > even with intercept, ssl-bumping with access control is a little more
> > difficult...jump to the config to skip the chit chat.
> > 
> > My goal has always been to a content filter based on url regex.  This
> > works just fine for http traffic, but is much more difficult for https
> > traffic just for the case of you may or may not know the host you're
> > going to, depending on the site/app.  I'll be real honest hereI'm
> > only doing this to protect/filter the traffic of two kids, on laptops,
> > iPhone, and Android phone, so it's a mixed bag of content and, since
> > it's just the two of them in a home environment, I get to play around
> > and see what works and what doesn't.
> > 
> > Below is a close as I can get transparent intercept ssl-bump with
> > content filtering with using a list of domains/urls with both http and
> > https.  I still have to use a list of broken sites, which are large
> > netblocks (17.0.0.0/8..Apple anyone?) because some of these I just can't
> > seem to get host/domain information during the ssl handshake.  As I
> > discovered after attempting to put this into "production", I have not
> > been able to emulate using wget or curl an https session that doesn't
> > have any SNI information, so that threw me for a loop.  TextNow is a
> > great example (I'm including a packet capture of this in this post).
> > There's no host information in the client hellothere's no host
> > information in the server hello.buried deep in the certificate ONLY
> > is the "commonName=.*textnow.me"...that's it.  This dashed my hopes of
> > using an url_regex for access control with all https sessions.  I have
> > "%ssl::>cert_subject" in my logging, and I never did see this log in any
> > of my tests...and I tested a BUNCH of different peek/stare/splice/bump
> > cominations..so I don't think squid is actually seeing this from the
> > certificate.
> > 
> > Another challenge is getting http url_regex filtering to work with https
> > filtering.  My method of filtering means not having an "http_access
> > allow localnet", which directly conflicted with also trying to filter
> > https.  The solution was to add an acl for port 443, then http_access to
> > just allow it, as our filtering was going to happen for https further
> > down.
> > 
> > I know there's a fair amount of people who just want to plop in some
> > config files, run a few commands, and be up and running.  The below
> > configuration has two additional files it references, http_url.txt,
> > which is an a list of domains/urls (\.apple\.com for example), and the
> > aptly named broken, which is a IP list (17.0.0.0/8).  The broken list
> > should be (semi) trusted and are sites that we just can't get SNI or
> > hostname information from.  If you've created a single cert/key pair
> > from the Squid documentation, you won't need the key= line in your
> > https_port directive.  If you've followed along in my posts, you already
> > have the configure line from my previous posts.  Change the
> > commands/config to fir where your squid config and ssl_db are.  So after
> > configuring, make sure you:
> > 
> > sudo /opt/libexec/ssl_crtd -c -s /opt/var/ssl_db
> > sudo chown -R nobody /opt/var/ssl_db/
> > 
> > As I believe in a lot of logging, and actually looking at said logging,
> > below is what you can expect to see in your logs (mine logs to syslog,
> > again, change this if you log to a different file):
> > 
> > Allowed http to .apple.com in http_url.txt:
> > May 31 17:03:48 gateway (squid-1): 192.168.1.100 - -
> > [31/May/2015:17:03:48 -0600] "GET
> > http://init.ess.apple.com/WebObjects/VCInit.woa/wa/getBag? HTTP/1.1" - -
> > 200 5243 TCP_MISS:ORIGINAL_DST -
> > Denied http to symcb.com not in http_url.txt
> > May 31 17:03:48 gateway (squid-1): 192.168.1.100 - -
> > [31/May/2015:17:03:48 -0600] "GET http://sd.symcb.com/sd.crt HTTP/1.1" -
> > - 403 3618 TCP_DENIED:HIER_NONE -
> > Spliced https IP in broken.txt (google block 216.58.192.0/19)
> > May 31 17:04:34 gateway (squid-1): 192.168.1.101 - -
> > [31/May/2015:17:04:34 -0600] "CONNECT 216.58.216.138:443 HTTP/1.1" - -
> > 200 568 TCP_TUNNEL:ORIGINAL_DST peek
> > Spliced https IP in broken.txt that we got SNI o

[squid-users] Ssl-bump deep dive (intercept last post and final thoughts)

[James Lay]

So this has been REALLY good!  The tl;dr:  ssl-bumping is pretty easy
even with intercept, ssl-bumping with access control is a little more
difficult...jump to the config to skip the chit chat.

My goal has always been to a content filter based on url regex.  This
works just fine for http traffic, but is much more difficult for https
traffic just for the case of you may or may not know the host you're
going to, depending on the site/app.  I'll be real honest hereI'm
only doing this to protect/filter the traffic of two kids, on laptops,
iPhone, and Android phone, so it's a mixed bag of content and, since
it's just the two of them in a home environment, I get to play around
and see what works and what doesn't.

Below is a close as I can get transparent intercept ssl-bump with
content filtering with using a list of domains/urls with both http and
https.  I still have to use a list of broken sites, which are large
netblocks (17.0.0.0/8..Apple anyone?) because some of these I just can't
seem to get host/domain information during the ssl handshake.  As I
discovered after attempting to put this into "production", I have not
been able to emulate using wget or curl an https session that doesn't
have any SNI information, so that threw me for a loop.  TextNow is a
great example (I'm including a packet capture of this in this post).
There's no host information in the client hellothere's no host
information in the server hello.buried deep in the certificate ONLY
is the "commonName=.*textnow.me"...that's it.  This dashed my hopes of
using an url_regex for access control with all https sessions.  I have
"%ssl::>cert_subject" in my logging, and I never did see this log in any
of my tests...and I tested a BUNCH of different peek/stare/splice/bump
cominations..so I don't think squid is actually seeing this from the
certificate.

Another challenge is getting http url_regex filtering to work with https
filtering.  My method of filtering means not having an "http_access
allow localnet", which directly conflicted with also trying to filter
https.  The solution was to add an acl for port 443, then http_access to
just allow it, as our filtering was going to happen for https further
down.

I know there's a fair amount of people who just want to plop in some
config files, run a few commands, and be up and running.  The below
configuration has two additional files it references, http_url.txt,
which is an a list of domains/urls (\.apple\.com for example), and the
aptly named broken, which is a IP list (17.0.0.0/8).  The broken list
should be (semi) trusted and are sites that we just can't get SNI or
hostname information from.  If you've created a single cert/key pair
from the Squid documentation, you won't need the key= line in your
https_port directive.  If you've followed along in my posts, you already
have the configure line from my previous posts.  Change the
commands/config to fir where your squid config and ssl_db are.  So after
configuring, make sure you:

sudo /opt/libexec/ssl_crtd -c -s /opt/var/ssl_db
sudo chown -R nobody /opt/var/ssl_db/

As I believe in a lot of logging, and actually looking at said logging,
below is what you can expect to see in your logs (mine logs to syslog,
again, change this if you log to a different file):

Allowed http to .apple.com in http_url.txt:
May 31 17:03:48 gateway (squid-1): 192.168.1.100 - -
[31/May/2015:17:03:48 -0600] "GET
http://init.ess.apple.com/WebObjects/VCInit.woa/wa/getBag? HTTP/1.1" - -
200 5243 TCP_MISS:ORIGINAL_DST -
Denied http to symcb.com not in http_url.txt
May 31 17:03:48 gateway (squid-1): 192.168.1.100 - -
[31/May/2015:17:03:48 -0600] "GET http://sd.symcb.com/sd.crt HTTP/1.1" -
- 403 3618 TCP_DENIED:HIER_NONE -
Spliced https IP in broken.txt (google block 216.58.192.0/19)
May 31 17:04:34 gateway (squid-1): 192.168.1.101 - -
[31/May/2015:17:04:34 -0600] "CONNECT 216.58.216.138:443 HTTP/1.1" - -
200 568 TCP_TUNNEL:ORIGINAL_DST peek
Spliced https IP in broken.txt that we got SNI or bumped site in
http_url.txt look exactly the same
May 31 17:09:45 gateway (squid-1): 192.168.1.100 - -
[31/May/2015:17:09:45 -0600] "CONNECT 23.222.157.21:443 HTTP/1.1"
init.itunes.apple.com - 200 30314 TCP_TUNNEL:ORIGINAL_DST peek

The only drag with the configuration is you won't see when an https
session is terminated when the IP/url is not in the broken.txt, or the
http_url.txt:

[17:20:53 jlay@analysis:~$] wget -d
--ca-certificate=/etc/ssl/certs/sslsplit.crt https://www.yahoo.com
Setting --ca-certificate (cacertificate) to /etc/ssl/certs/sslsplit.crt
DEBUG output created by Wget 1.16.1 on linux-gnu.

URI encoding = ‘UTF-8’
--2015-05-31 17:20:59--  https://www.yahoo.com/
Resolving www.yahoo.com (www.yahoo.com)... 206.190.36.45,
206.190.36.105, 2001:4998:c:a06::2:4008
Caching www.yahoo.com => 206.190.36.45 206.190.36.105
2001:4998:c:a06::2:4008
Connecting to www.yahoo.com (www.yahoo.com)|206.190.36.45|:443...
connected.
Created socket 3.
Releasing 0x7fdf67eecdd0 (new refcount 

Re: [squid-users] Conditional question

[James Lay]

On Sat, 2015-05-30 at 16:24 -0600, James Lay wrote:

> On Sun, 2015-05-31 at 08:45 +1200, Amos Jeffries wrote: 
> 
> > On 31/05/2015 4:48 a.m., James Lay wrote:
> > > Per the docs:
> > > 
> > > #  Conditional configuration
> > > #
> > > #   If-statements can be used to make configuration directives
> > > #   depend on conditions:
> > > #
> > > #   if 
> > > #   ... regular configuration directives ...
> > > #   [else
> > > #   ... regular configuration directives ...]
> > > #   endif
> > > #
> > > #   The else part is optional. The keywords "if", "else", and
> > > "endif"
> > > #   must be typed on their own lines, as if they were regular
> > > #   configuration directives.
> > > #
> > > #   NOTE: An else-if condition is not supported.
> > > #
> > > #   These individual conditions types are supported:
> > > #
> > > #   true
> > > #   Always evaluates to true.
> > > #   false
> > > #   Always evaluates to false.
> > > #= 
> > > #   Equality comparison of two integer numbers.
> > > 
> > > Anyone have any examples, documentation, heck ANYTHING that can show how
> > > this works?  I can't seem to find a thing besides the above.
> > 
> > Those are for process controls (SMP, named services, etc).
> > 
> > >  My goal is
> > > something like the below:
> > > 
> > > if port = 80
> > > http_access deny all
> > > else
> > > http_access allow all
> > > endif
> > > 
> > > But nothing I'm trying as the condition expression is working.  Thank
> > > you.
> > 
> > The default Squid configuration should "just work"...
> > 
> >   http_access deny !Safe_ports
> >   http_access deny CONNECT !SSL_Ports
> >   ...
> >   # this one permits the CONNECT *:443 requests to get bumped
> >   http_access allow localnet
> >   ..
> >   http_access deny all
> > 
> > If you are using any other access controls on your client traffic you
> > need to keep in mind that Squid is dealing with "CONNECT raw-IP:443 ..."
> > requests in http_access / adapted_http_access / url_rewrite_access /
> > adaptation_access / ssl_bump prior to bumping them.
> > 
> > Amos
> > ___
> > squid-users mailing list
> > squid-users@lists.squid-cache.org
> > http://lists.squid-cache.org/listinfo/squid-users
> 
> 
> Hi again Amos,
> 
> So...my method of access control might be weird.  I have a regex list
> of sites that work fine via http (say \.acer\.com).  So, I allow
> access to this list via:
> 
> acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt
> http_access allow allowed_http_sites
> http_access deny !allowed_http_sites
> 
> This works well for allowing access to the list of sitesthe lack
> of http_access allow localnet makes this happen.  With the above
> however, ssl_bumping stops working as I get:
> 
> [16:18:22 jlay@powerbook:~/test$ wget
> --ca-certificate=/etc/ssl/certs/sslsplit_ca_cert.pem -d
> https://www.msn.com
> DEBUG output created by Wget 1.16 on linux-gnu.
> 
> URI encoding = ‘UTF-8’
> --2015-05-30 16:19:46--  https://www.msn.com/
> Certificates loaded: 173
> Resolving www.msn.com (www.msn.com)... 204.79.197.203
> Caching www.msn.com => 204.79.197.203
> Connecting to www.msn.com (www.msn.com)|204.79.197.203|:443...
> connected.
> Created socket 4.
> Releasing 0x10c3ef98 (new refcount 1).
> The certificate's owner does not match hostname ‘www.msn.com’
> 
> May 30 16:19:46 analysis squid: 192.168.1.73 - - [30/May/2015:16:19:46
> -0600] "CONNECT 204.79.197.203:443 HTTP/1.1" - 200 0
> TCP_DENIED:HIER_NONE peek
> 
> Adding http_access alllow localnet makes ssl_bumping work correctly,
> but then the http_access deny !allowed_http_sites does not work.  I'm
> having a hard time getting both http and https filtering to play well
> together with one instance of squid.  I'd like to try and just go with
> one, but if I have to I'll go with two.  Anyway thanks again for
> looking...I hope I'm explaining this well.
> 
> James
> 
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


Ok I think I got it...added:

acl allow_https port 443
...
http_access allow allow_https

Now my clients are allowed full port 443 access, which gets a decision
of allow or block later on, and this also allows my "usual" http access
listwoo hoo!  I'll post the full info later.  Thanks so much.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Conditional question

[James Lay]

On Sun, 2015-05-31 at 08:45 +1200, Amos Jeffries wrote:

> On 31/05/2015 4:48 a.m., James Lay wrote:
> > Per the docs:
> > 
> > #  Conditional configuration
> > #
> > #   If-statements can be used to make configuration directives
> > #   depend on conditions:
> > #
> > #   if 
> > #   ... regular configuration directives ...
> > #   [else
> > #   ... regular configuration directives ...]
> > #   endif
> > #
> > #   The else part is optional. The keywords "if", "else", and
> > "endif"
> > #   must be typed on their own lines, as if they were regular
> > #   configuration directives.
> > #
> > #   NOTE: An else-if condition is not supported.
> > #
> > #   These individual conditions types are supported:
> > #
> > #   true
> > #   Always evaluates to true.
> > #   false
> > #   Always evaluates to false.
> > #= 
> > #   Equality comparison of two integer numbers.
> > 
> > Anyone have any examples, documentation, heck ANYTHING that can show how
> > this works?  I can't seem to find a thing besides the above.
> 
> Those are for process controls (SMP, named services, etc).
> 
> >  My goal is
> > something like the below:
> > 
> > if port = 80
> > http_access deny all
> > else
> > http_access allow all
> > endif
> > 
> > But nothing I'm trying as the condition expression is working.  Thank
> > you.
> 
> The default Squid configuration should "just work"...
> 
>   http_access deny !Safe_ports
>   http_access deny CONNECT !SSL_Ports
>   ...
>   # this one permits the CONNECT *:443 requests to get bumped
>   http_access allow localnet
>   ..
>   http_access deny all
> 
> If you are using any other access controls on your client traffic you
> need to keep in mind that Squid is dealing with "CONNECT raw-IP:443 ..."
> requests in http_access / adapted_http_access / url_rewrite_access /
> adaptation_access / ssl_bump prior to bumping them.
> 
> Amos
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


Hi again Amos,

So...my method of access control might be weird.  I have a regex list of
sites that work fine via http (say \.acer\.com).  So, I allow access to
this list via:

acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt
http_access allow allowed_http_sites
http_access deny !allowed_http_sites

This works well for allowing access to the list of sitesthe lack of
http_access allow localnet makes this happen.  With the above however,
ssl_bumping stops working as I get:

[16:18:22 jlay@powerbook:~/test$ wget
--ca-certificate=/etc/ssl/certs/sslsplit_ca_cert.pem -d
https://www.msn.com
DEBUG output created by Wget 1.16 on linux-gnu.

URI encoding = ‘UTF-8’
--2015-05-30 16:19:46--  https://www.msn.com/
Certificates loaded: 173
Resolving www.msn.com (www.msn.com)... 204.79.197.203
Caching www.msn.com => 204.79.197.203
Connecting to www.msn.com (www.msn.com)|204.79.197.203|:443...
connected.
Created socket 4.
Releasing 0x10c3ef98 (new refcount 1).
The certificate's owner does not match hostname ‘www.msn.com’

May 30 16:19:46 analysis squid: 192.168.1.73 - - [30/May/2015:16:19:46
-0600] "CONNECT 204.79.197.203:443 HTTP/1.1" - 200 0
TCP_DENIED:HIER_NONE peek

Adding http_access alllow localnet makes ssl_bumping work correctly, but
then the http_access deny !allowed_http_sites does not work.  I'm having
a hard time getting both http and https filtering to play well together
with one instance of squid.  I'd like to try and just go with one, but
if I have to I'll go with two.  Anyway thanks again for looking...I hope
I'm explaining this well.

James

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Conditional question

[James Lay]

On Sun, 2015-05-31 at 08:45 +1200, Amos Jeffries wrote:

> On 31/05/2015 4:48 a.m., James Lay wrote:
> > Per the docs:
> > 
> > #  Conditional configuration
> > #
> > #   If-statements can be used to make configuration directives
> > #   depend on conditions:
> > #
> > #   if 
> > #   ... regular configuration directives ...
> > #   [else
> > #   ... regular configuration directives ...]
> > #   endif
> > #
> > #   The else part is optional. The keywords "if", "else", and
> > "endif"
> > #   must be typed on their own lines, as if they were regular
> > #   configuration directives.
> > #
> > #   NOTE: An else-if condition is not supported.
> > #
> > #   These individual conditions types are supported:
> > #
> > #   true
> > #   Always evaluates to true.
> > #   false
> > #   Always evaluates to false.
> > #= 
> > #   Equality comparison of two integer numbers.
> > 
> > Anyone have any examples, documentation, heck ANYTHING that can show how
> > this works?  I can't seem to find a thing besides the above.
> 
> Those are for process controls (SMP, named services, etc).
> 
> >  My goal is
> > something like the below:
> > 
> > if port = 80
> > http_access deny all
> > else
> > http_access allow all
> > endif
> > 
> > But nothing I'm trying as the condition expression is working.  Thank
> > you.
> 
> The default Squid configuration should "just work"...
> 
>   http_access deny !Safe_ports
>   http_access deny CONNECT !SSL_Ports
>   ...
>   # this one permits the CONNECT *:443 requests to get bumped
>   http_access allow localnet
>   ..
>   http_access deny all
> 
> If you are using any other access controls on your client traffic you
> need to keep in mind that Squid is dealing with "CONNECT raw-IP:443 ..."
> requests in http_access / adapted_http_access / url_rewrite_access /
> adaptation_access / ssl_bump prior to bumping them.
> 
> Amos
> ___


Thanks Amosin starting from scratch I completely neglected to even
allow localnets...yugh!  Continuing on and will post my final results.

James

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Conditional question

[James Lay]

Per the docs:

#  Conditional configuration
#
#   If-statements can be used to make configuration directives
#   depend on conditions:
#
#   if 
#   ... regular configuration directives ...
#   [else
#   ... regular configuration directives ...]
#   endif
#
#   The else part is optional. The keywords "if", "else", and
"endif"
#   must be typed on their own lines, as if they were regular
#   configuration directives.
#
#   NOTE: An else-if condition is not supported.
#
#   These individual conditions types are supported:
#
#   true
#   Always evaluates to true.
#   false
#   Always evaluates to false.
#= 
#   Equality comparison of two integer numbers.

Anyone have any examples, documentation, heck ANYTHING that can show how
this works?  I can't seem to find a thing besides the above.  My goal is
something like the below:

if port = 80
http_access deny all
else
http_access allow all
endif

But nothing I'm trying as the condition expression is working.  Thank
you.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Ssl-bump deep dive (sni and access control) some success

[James Lay]

Config first:


acl localnet src 192.168.1.0/24

acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443

acl CONNECT method CONNECT

acl step1 at_step SslBump1
acl step2 at_step SslBump2

ssl_bump peek step1 all
#https_server_names.txt has \.google\.com, \.yahoo\.com, \.msn\.com
acl allowed_https_sites ssl::server_name_regex
"/opt/etc/squid/https_server_names.txt"

http_access allow all

ssl_bump bump allowed_https_sites
ssl_bump terminate !allowed_https_sites

sslproxy_cert_error allow all
sslproxy_capath /etc/ssl/certs
sslproxy_flags DONT_VERIFY_PEER 
sslproxy_options ALL

sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
sslcrtd_children 5

http_port 3128 intercept
https_port 3129 intercept ssl-bump
cert=/opt/etc/squid/certs/sslsplit_ca_cert.pem
cafile=/opt/etc/squid/certs/sslsplit_ca_cert.pem
key=/opt/etc/squid/certs/sslsplit_ca_key.pem
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslflags=NO_SESSION_REUSE

logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni %>Hs
%https://www.msn.com
DEBUG output created by Wget 1.16 on linux-gnu.

URI encoding = ‘UTF-8’
--2015-05-30 08:59:57--  https://www.msn.com/
Certificates loaded: 173
Resolving www.msn.com (www.msn.com)... 204.79.197.203
Caching www.msn.com => 204.79.197.203
Connecting to www.msn.com (www.msn.com)|204.79.197.203|:443...
connected.
Created socket 4.
Releasing 0x10503f98 (new refcount 1).

---request begin---
GET / HTTP/1.1
User-Agent: Wget/1.16 (linux-gnu)
Accept: */*
Host: www.msn.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response... 
---response begin---
HTTP/1.1 200 OK


May 30 08:59:57 analysis squid: 192.168.1.73 - - [30/May/2015:08:59:57
-0600] "CONNECT 204.79.197.203:443 HTTP/1.1" www.msn.com 200 0
TAG_NONE:ORIGINAL_DST peek
May 30 08:59:58 analysis squid: 192.168.1.73 - - [30/May/2015:08:59:58
-0600] "GET https://www.msn.com/ HTTP/1.1" www.msn.com 200 38288
TCP_MISS:ORIGINAL_DST bump

Going to a site not in the allowed_https_sites acl:

[09:02:12 jlay@powerbook:~/test$ wget
--ca-certificate=/etc/ssl/certs/sslsplit_ca_cert.pem -d
https://www.weather.com
DEBUG output created by Wget 1.16 on linux-gnu.

URI encoding = ‘UTF-8’
--2015-05-30 09:04:57--  https://www.weather.com/
Certificates loaded: 173
Resolving www.weather.com (www.weather.com)... 96.17.8.161, 96.17.8.138,
96.17.8.178, ...
Caching www.weather.com => 96.17.8.161 96.17.8.138 96.17.8.178
96.17.8.171
Connecting to www.weather.com (www.weather.com)|96.17.8.161|:443...
connected.
Created socket 4.
Releasing 0x1098c108 (new refcount 1).
GnuTLS: The TLS connection was non-properly terminated.
Closed fd 4
Unable to establish SSL connection.

May 30 09:04:57 analysis squid: 192.168.1.73 - - [30/May/2015:09:04:57
-0600] "CONNECT 96.17.8.161:443 HTTP/1.1" www.weather.com 200 0
TAG_NONE:HIER_NONE peek

However, changing http_access to http_access allow allowed_https_sites I
get:

[08:59:58 jlay@powerbook:~/test$ wget
--ca-certificate=/etc/ssl/certs/sslsplit_ca_cert.pem -d
https://www.msn.com
DEBUG output created by Wget 1.16 on linux-gnu.

URI encoding = ‘UTF-8’
--2015-05-30 09:02:12--  https://www.msn.com/
Certificates loaded: 173
Resolving www.msn.com (www.msn.com)... 204.79.197.203
Caching www.msn.com => 204.79.197.203
Connecting to www.msn.com (www.msn.com)|204.79.197.203|:443...
connected.
Created socket 4.
Releasing 0x10515f98 (new refcount 1).
The certificate's owner does not match hostname ‘www.msn.com’

May 30 09:02:12 analysis squid: 192.168.1.73 - - [30/May/2015:09:02:12
-0600] "CONNECT 204.79.197.203:443 HTTP/1.1" - 200 0
TCP_DENIED:HIER_NONE peek

Notice that peek did not get the SNI name per my %ssl::>sni in my
logging statement.  So as of now I have been unable to figure out how to
use access control with both http and https.  I can do one or the other,
but not both so far.  Of interest, redirects from http to https do not
appear to work

[08:37:39 jlay@powerbook:~/test$ wget www.yahoo.com
--2015-05-30 08:37:44--  http://www.yahoo.com/
Resolving www.yahoo.com (www.yahoo.com)... 206.190.36.45,
206.190.36.105, 2001:4998:c:a06::2:4008
Connecting to www.yahoo.com (www.yahoo.com)|206.190.36.45|:80...
connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://www.yahoo.com/ [following]
--2015-05-30 08:37:44--  https://www.yahoo.com/
Connecting to www.yahoo.com (www.yahoo.com)|206.190.36.45|:443...
connected.
ERROR: The certificate of ‘www.yahoo.com’ is not trusted.
ERROR: The certificate of ‘www.yahoo.com’ hasn't got a known issuer.

May 30 08:37:44 analysis squid: 192.168.1.73 - - [30/May/2015:08:37:44
-0600] "GET http://www.yahoo.com/ HTTP/1.1" - 301 1812
TCP_MISS:ORIGINAL_DST -
May 30 08:37:45 analysis squid: 192.168.1.73 - - [30/May/2015:08:37:45
-0600] "CONNECT 206.190.36.45:443 HTTP/1.1" www.yahoo.com 200 0
TAG_NONE:ORIGINAL_DST peek

Whereas direct does:

[08:37:45 jlay@powerbook:~/test$ wget
--ca-certi

Re: [squid-users] ssl_bump and SNI

[James Lay]

On 2015-05-29 08:57 AM, Nathan Hoad wrote:

Yes, I have it working on about a dozen deployments so far, using an
external ACL to make bumping decisions based on the SNI server name
and a few other things. No complaints from me, it Just Works.
On 29/05/2015 5:50 pm, "sp_"  wrote:


Hello,

does anyone have the working squid 3.5 with intercept + https?
I've googled a lot, but seems there is no any positive experience
with it.

--
View this message in context:


http://squid-web-proxy-cache.1019090.n4.nabble.com/ssl-bump-and-SNI-tp4670207p4671432.html

[1]
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users [2]



Links:
--
[1]
http://squid-web-proxy-cache.1019090.n4.nabble.com/ssl-bump-and-SNI-tp4670207p4671432.html
[2] http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


Nathan,

Care to post your config and external helper?  I know I'd love to see 
concrete examples.  Thank you.


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Ssl-bump deep dive (testing)

[James Lay]

So I took the advice of those here to get explicit working first, so
here's my first attempt.  My test environment is Ubuntu 15.04 Server as
the squid server with virtualbox running on it with Kali linux as the
client.  Here's my Squid 3.5.4 configure line:

/configure --prefix=/opt --enable-icap-client --with-openssl
--enable-ssl --enable-ssl-crtd --enable-linux-netfilter
--enable-follow-x-forwarded-for --with-large-files
--sysconfdir=/opt/etc/squid --enable-external-acl-helpers=none



Full squid.conf:
#
acl localnet src 192.168.1.0/24

acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443

acl CONNECT method CONNECT

http_access allow all

sslproxy_cert_error allow all
sslproxy_cert_error deny all
sslproxy_capath /etc/ssl/certs
sslproxy_flags DONT_VERIFY_PEER 
sslproxy_options ALL

sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
sslcrtd_children 5

http_port 3129 ssl-bump cert=/opt/etc/squid/certs/sslsplit_ca_cert.pem
cafile=/opt/etc/squid/certs/sslsplit_ca_cert.pem
key=/opt/etc/squid/certs/sslsplit_ca_key.pem
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslflags=NO_SESSION_REUSE

external_acl_type sni ttl=30 concurrency=10 children-max=20
children-startup=5 %ssl::>sni /opt/etc/squid/bumphelper.py

acl sni_exclusions external sni
acl tcp_level at_step SslBump1
acl client_hello_peeked at_step SslBump2

ssl_bump peek tcp_level all
ssl_bump splice client_hello_peeked sni_exclusions
ssl_bump bump all

logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %>Hs %sni %ssl::>cert_subject

access_log syslog:daemon.info mine

refresh_pattern -i (cgi-bin|\?) 0   0%  0
refresh_pattern .   0   20% 4320

coredump_dir /opt/var
#


bumphelper.py:
#
#!/usr/bin/python

import sys

while True:
req = sys.stdin.readline()

if not req:
break

id, sni = req.split()

sys.stderr.write('request %r\n' % req)
sys.stderr.flush()

if sni == 'google.com':  # bypass
sys.stdout.write('{} OK\n'.format(id))
sys.stdout.flush()
else:
sys.stdout.write('{} ERR\n'.format(id))
sys.stdout.flush()
#

The tests:
root@kali:~/test# wget -d https://www.google.com
##
DEBUG output created by Wget 1.13.4 on linux-gnu.

URI encoding = `UTF-8'
URI encoding = `UTF-8'
--2015-05-28 17:44:31--  https://www.google.com/
Connecting to 192.168.1.6:3129... connected.
Created socket 4.
Releasing 0x092c6730 (new refcount 0).
Deleting unused 0x092c6730.

---request begin---
CONNECT www.google.com:443 HTTP/1.1
User-Agent: Wget/1.13.4 (linux-gnu)

---request end---
proxy responded with: [HTTP/1.1 200 Connection established

]

---request begin---
GET / HTTP/1.1
User-Agent: Wget/1.13.4 (linux-gnu)
Accept: */*
Host: www.google.com
Connection: Close
Proxy-Connection: Keep-Alive

---request end---
Proxy request sent, awaiting response... 
---response begin---
HTTP/1.1 503 Service Unavailable
Server: squid/3.5.4
Mime-Version: 1.0
Date: Thu, 28 May 2015 23:44:33 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 3899
X-Squid-Error: ERR_SECURE_CONNECT_FAIL 32
Vary: Accept-Language
Content-Language: en
X-Cache: MISS from analysis
Via: 1.1 analysis (squid/3.5.4)
Connection: close

---response end---
503 Service Unavailable
URI content encoding = `utf-8'
2015-05-28 17:44:32 ERROR 503: Service Unavailable.


access.log entry for the above wget:
#
May 28 17:44:33 analysis squid: 192.168.1.91 - - [28/May/2015:17:44:33
-0600] "CONNECT www.google.com:443 HTTP/1.1" 200 0 TAG_NONE:HIER_DIRECT
peek www.google.com -
May 28 17:44:33 analysis squid: 192.168.1.91 - - [28/May/2015:17:44:33
-0600] "GET https://www.google.com/ HTTP/1.1" 503 4242
TAG_NONE:HIER_NONE - www.google.com -
#



sudo /opt/sbin/squid -d 1 -N -f /opt/etc/squid/squid.conf
##
2015/05/28 17:44:33| Error negotiating SSL on FD 14:
error::lib(0):func(0):reason(0) (5/-1/32)
##


I see the same type of thing for apple.com and yahoo.com.  I'm assuming
this is HSTS, but I could be wrong.  MSN however works fine with the
above:
root@kali:~/test# wget -d https://www.msn.com
##
DEBUG output created by Wget 1.13.4 on linux-gnu.

URI encoding = `UTF-8'
URI encoding = `UTF-8'
--2015-05-28 18:24:50--  https://www.msn.com/
Connecting to 192.168.1.6:3129... connected.
Created socket 4.
Releasing 0x0a6493c0 (new refcount 0).
Deleting unused 0x0a6493c0.

---request begin---
CONNECT www.msn.com:443 HTTP/1.1
User-Agent: Wget/1.13.4 (linux-gnu)

---request end---
proxy responded with: [HTTP/1.1 200 Connection established

]

---request begin---
GET / HTTP/1.1
User-Agent: Wget/1.13.4 (linux-gnu

Re: [squid-users] Ssl-bump deep dive (self-signed certs in chain)

[James Lay]

Thanks for this AmosI will try and do more experimenting this week
with more results.

James

On Tue, 2015-05-26 at 19:46 +1200, Amos Jeffries wrote:

> On 26/05/2015 4:26 a.m., James Lay wrote:
> > So following advice and instructions on this page:
> > 
> > http://wiki.squid-cache.org/Features/DynamicSslCert
> > 
> > I have set up my lab with explicit proxy by exporting http_proxy and
> > https_proxy.  After creating the self-signed root CA certificate above
> > and creating the .der file for the client, here are my results:
> > 
> > From the squid side:
> > 2015/05/25 10:02:20.161| Using certificate
> > in /opt/etc/squid/certs/SquidCA.pem
> > 2015/05/25 10:02:20.170| support.cc(1743) readSslX509CertificatesChain:
> > Certificate is self-signed, will not be chained
> > I get the below when I don't specify a CA with curl, otherwise when I do
> > I get no error:
> > 2015/05/25 09:21:02.229| Error negotiating SSL connection on FD 12:
> > error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
> 
> If that error is displayed by Squid about the clients connection. Then I
> believe it means the client is attempting to perform TLS authentication
> to Squid using the CA you installed there. Which is not possible as the
> CA is supposed to make the client trust Squid generated certs, not the
> other way around.
> 
> 
> > 
> > And from the client side:
> > root@kali:~/test# curl -v https://mail.slave-tothe-box.net
> > * About to connect() to proxy 192.168.1.9 port 3129 (#0)
> > *   Trying 192.168.1.9...
> > * connected
> > * Connected to 192.168.1.9 (192.168.1.9) port 3129 (#0)
> > * Establish HTTP proxy tunnel to mail.slave-tothe-box.net:443
> >> CONNECT mail.slave-tothe-box.net:443 HTTP/1.1
> >> Host: mail.slave-tothe-box.net:443
> >> User-Agent: curl/7.26.0
> >> Proxy-Connection: Keep-Alive
> >>
> > * Easy mode waiting response from proxy CONNECT
> > < HTTP/1.1 200 Connection established
> > < 
> > * Proxy replied OK to CONNECT request
> > * successfully set certificate verify locations:
> > *   CAfile: none
> >   CApath: /etc/ssl/certs
> > * SSLv3, TLS handshake, Client hello (1):
> > * SSLv3, TLS handshake, Server hello (2):
> > * SSLv3, TLS handshake, CERT (11):
> > * SSLv3, TLS alert, Server hello (2):
> > * SSL certificate problem: self signed certificate in certificate chain
> > * Closing connection #0
> > 
> > And testing with specifying the .der file:
> > root@kali:~/test# curl --cacert /etc/ssl/certs/SquidCA.der -v
> > https://mail.slave-tothe-box.net
> > * About to connect() to proxy 192.168.1.9 port 3129 (#0)
> > *   Trying 192.168.1.9...
> > * connected
> > * Connected to 192.168.1.9 (192.168.1.9) port 3129 (#0)
> > * Establish HTTP proxy tunnel to mail.slave-tothe-box.net:443
> >> CONNECT mail.slave-tothe-box.net:443 HTTP/1.1
> >> Host: mail.slave-tothe-box.net:443
> >> User-Agent: curl/7.26.0
> >> Proxy-Connection: Keep-Alive
> >>
> > * Easy mode waiting response from proxy CONNECT
> > < HTTP/1.1 200 Connection established
> > < 
> > * Proxy replied OK to CONNECT request
> > * error setting certificate verify locations:
> >   CAfile: /etc/ssl/certs/SquidCA.der
> >   CApath: /etc/ssl/certs
> > 
> > * Closing connection #0
> > curl: (77) error setting certificate verify locations:
> >   CAfile: /etc/ssl/certs/SquidCA.der
> >   CApath: /etc/ssl/certs
> > 
> > 
> > I can confirm that the server is using a bona-fide certificate issued
> > from StartSSL and works, so at this point I'm open to suggestions.
> > Thank you.
> 
> curl is complaining that the CA chain for the Squid-generted cert has a
> self-signed CA. This is expected and desired behaviour if the
> self-signed CA was sent by Squid.
> 
> The errors only occur when the self-signed CA is not sent by Squid, but
> using the one installed on the client.
> 
> 
> For that I believe you need to configure Squid to sign/generate using
> the intermediate certificate. The self-signed root CA not configured in
> Squid at all.
> 
> Like so:
> 
> A)
>  client Trust DB installed with self-signed root CA
> 
>  squid.conf cert= configured with intermediary CA certificate
> 
>  squid.conf cafile= configured with any other intermediary CA
> certificates (in order back to root CA, but excluding it).
> 
>  Squid generates per-connection certificate
> 
> OR:
> 
> B)
>  client Trust DB installed with self-signed roo

Re: [squid-users] ipf transparent enabled, but squid says not supported

[James Lay]

On 2015-05-27 09:45 AM, Stephen Borrill wrote:

I have:
Squid Cache: Version 3.5.4
Service Name: squid
configure options:  '--sysconfdir=/usr/pkg/etc/squid'
'--localstatedir=/var/squid' '--datarootdir=/usr/pkg/share/squid'
'--disable-strict-error-checking' '--enable-auth'
'--enable-cachemgr-hostname=localhost' '--enable-delay-pools'
'--enable-icap-client' '--enable-icmp' '--enable-poll'
'--enable-removal-policies=lru,heap'
'--enable-storeio=ufs diskd' '--with-aio' '--with-default-user=squid'
'--with-pidfile=/var/run/squid.pid' '--disable-arch-native'
'--enable-ipf-transparent' '--enable-arp-acl' '--enable-carp'
'--disable-ipv6' '--without-mit-krb5' '--without-heimdal-krb5'
'--disable-snmp' '--enable-ssl' '--with-openssl=/usr/pkg'
'--enable-auth-basic=NCSA getpwnam PAM' '--enable-auth-digest=file'
'--disable-auth-negotiate' '--enable-auth-ntlm=fake smb_lm'
'--enable-external-acl-helpers=file_userip unix_group'
'--prefix=/usr/pkg' '--build=i486--netbsdelf'
'--host=i486--netbsdelf' '--mandir=/usr/pkg/man'
'build_alias=i486--netbsdelf' 'host_alias=i486--netbsdelf'
'CC=cc' 'CFLAGS=-O2 -I/usr/include -I/usr/pkg/include'
'LDFLAGS=-L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib'
'LIBS=' 'CPPFLAGS=-I/usr/include -I/usr/pkg/include'
'CXX=c++' 'CXXFLAGS=-O2 -I/usr/include -I/usr/pkg/include'

squid.conf contains:
http_port 127.0.0.1:8006 intercept name=port_8006

Yet I see the following ev:
2015/05/27 16:02:46 kid1| WARNING: transparent proxying not supported

Same config works with earlier version of squid (3.4 and earlier).
What's changed?


Look through your config.log...I experienced a similar thing and, upon 
running my ./configure line and watching it I saw I was missing a 
library.


James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Ssl-bump deep dive (self-signed certs in chain)

[James Lay]

So following advice and instructions on this page:

http://wiki.squid-cache.org/Features/DynamicSslCert

I have set up my lab with explicit proxy by exporting http_proxy and
https_proxy.  After creating the self-signed root CA certificate above
and creating the .der file for the client, here are my results:

>From the squid side:
2015/05/25 10:02:20.161| Using certificate
in /opt/etc/squid/certs/SquidCA.pem
2015/05/25 10:02:20.170| support.cc(1743) readSslX509CertificatesChain:
Certificate is self-signed, will not be chained
I get the below when I don't specify a CA with curl, otherwise when I do
I get no error:
2015/05/25 09:21:02.229| Error negotiating SSL connection on FD 12:
error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)

And from the client side:
root@kali:~/test# curl -v https://mail.slave-tothe-box.net
* About to connect() to proxy 192.168.1.9 port 3129 (#0)
*   Trying 192.168.1.9...
* connected
* Connected to 192.168.1.9 (192.168.1.9) port 3129 (#0)
* Establish HTTP proxy tunnel to mail.slave-tothe-box.net:443
> CONNECT mail.slave-tothe-box.net:443 HTTP/1.1
> Host: mail.slave-tothe-box.net:443
> User-Agent: curl/7.26.0
> Proxy-Connection: Keep-Alive
> 
* Easy mode waiting response from proxy CONNECT
< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS alert, Server hello (2):
* SSL certificate problem: self signed certificate in certificate chain
* Closing connection #0

And testing with specifying the .der file:
root@kali:~/test# curl --cacert /etc/ssl/certs/SquidCA.der -v
https://mail.slave-tothe-box.net
* About to connect() to proxy 192.168.1.9 port 3129 (#0)
*   Trying 192.168.1.9...
* connected
* Connected to 192.168.1.9 (192.168.1.9) port 3129 (#0)
* Establish HTTP proxy tunnel to mail.slave-tothe-box.net:443
> CONNECT mail.slave-tothe-box.net:443 HTTP/1.1
> Host: mail.slave-tothe-box.net:443
> User-Agent: curl/7.26.0
> Proxy-Connection: Keep-Alive
> 
* Easy mode waiting response from proxy CONNECT
< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* error setting certificate verify locations:
  CAfile: /etc/ssl/certs/SquidCA.der
  CApath: /etc/ssl/certs

* Closing connection #0
curl: (77) error setting certificate verify locations:
  CAfile: /etc/ssl/certs/SquidCA.der
  CApath: /etc/ssl/certs


I can confirm that the server is using a bona-fide certificate issued
from StartSSL and works, so at this point I'm open to suggestions.
Thank you.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Ssl-bump deep dive (properly creating certs)

[James Lay]

On Mon, 2015-05-25 at 08:48 +1200, Jason Haar wrote:
> On 25/05/15 04:25, James Lay wrote:
> 
> > 
> > My first question is about properly creating the certs.  Looking at:
> > 
> > http://wiki.squid-cache.org/ConfigExamples/Intercept/SslBumpExplicit
> > 
> > this mentions using crtd, but as I understand it, crtd isn't
> > supported when using transparent proxies.  So, with no crtd, as I
> > understand it this is what I'll need:
> > 
> 
> 
> I don't know where you got that from, but that's not true. I think you
> are confusing the issue that when squid is used as a transparent HTTPS
> proxy, it lacks the "easy" hostname details that a formal (ie
> non-transparent) proxy has. ie when a browser asks for a secure
> website via a formal proxy, it sends
> 
> CONNECT github.com:443 HTTP/1.1
> 
> So squid knows *in advance* the server is called "github.com". So it
> connects to github.com, downloads the public key and then uses crtd to
> create a clone of it - identical except that it's signed by your
> self-created Squid CA instead of Verisign/whatever
> 
> Compare that with transparent proxy mode, where all that squid knows
> is that a browser has had it's outbound tcp port 443 traffic to
> 192.30.252.128 redirected onto it, so it doesn't know that is
> github.com. If you are using squid-3.4 or less, that's all there is to
> it - there's no way to figure out the cert name in a guaranteed
> fashion (there are hacks, but my own experience is that they can only
> work up to 95% of the time - and break for some of the largest sites).
> With squid-3.5 there is "peek" - which means squid can let the initial
> few packets through (ie act like "splice") - which is enough to see
> the client send the SNI request to the https server and get the reply.
> So "peek" allows squid to learn about the true server name of the
> https server. At that point *I think* squid creates a forged cert,
> then creates a new connection to the server, then links together the
> existing client tcp channel with the new proxy->server tcp channel and
> carries on intercepting (I think that's the outcome - there would have
> to be some extra smoke-n-mirrors in there to make that happen)
> 
> In pseudo-code, it looks like this
> 
> if http_port and "CONNECT (.*) HTTP" then sni_name=$1
> else if https_port and "peek" then sni_name=find_sni($ipaddress)
> else if https_port then sni_name=$ipaddress
> 
> 
> When all is said and done, transparent HTTPS intercept is the very
> last thing you should be working on. You need to gets squid working
> 100% as a formal proxy - and only then start looking at making that
> work in transparent mode. And you *definitely* want ssl_crtd. 
> 
> 
> 
> -- 
> Cheers
> 
> Jason Haar
> Corporate Information Security Manager, Trimble Navigation Ltd.
> Phone: +1 408 481 8171
> PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


Thanks for the great response Jason...I appreciate it.  I think maybe I
misread this:

http://wiki.squid-cache.org/Features/DynamicSslCert

"While SslBump itself works fine in transparent redirection environments
(e.g. those using WCCP or iptables), dynamic certificate generation does
not: To generate the certificate dynamically, Squid must know the server
domain name. That information is not available at the time the HTTPS
client TCP connection is intercepted and bumped. Currently, you cannot
use dynamic certificate generation for transparent connections until
bump-server-first is supported."

Is this no longer accurate with now that peek/splice has been
implemented?

Thank you.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Ssl-bump deep dive (properly creating certs)

[James Lay]

Hey all,

SoI'm sure those on the list have seen my posts a number of times,
usually all questions (sorry I'm not very helpful).  That being said,
whenever there is something I can't get to work right, or don't
understand as well as I think I should, I do kind of a deep dive into it
for about a month.  I'm going to do that now with Squid.  I have NEVER
gotten ssl-bump to work right.  I have it "sort of" working, but there
are some issues I want to address.

So I'm going to start from scratch in a lab environment using a VM as a
client, a physical machine with two nics that are bridged and run squid
as a transparent proxy, and a physical laptop as the server.

My first question is about properly creating the certs.  Looking at:

http://wiki.squid-cache.org/ConfigExamples/Intercept/SslBumpExplicit

this mentions using crtd, but as I understand it, crtd isn't supported
when using transparent proxies.  So, with no crtd, as I understand it
this is what I'll need:

Server:
Self-signed CA cert (pem) <- used as cafile= in https_port
Intermediate cert signed by the above self signed CA cert (pem) <- used
as cert= in https_port
Key file for the self-signed CA cert above (pem) <- used as key= in
https_port

Client:
Self-signed CA cert from above (pem) <- in /etc/ssl/certs for linux

Any help, advice, links that would assist in better understanding this
first step in ssl-bumping transparently would be wonderful.  Thank you.

James 

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Config audit for 3.5.3

[James Lay]

On Sat, 2015-04-25 at 14:25 +1200, Amos Jeffries wrote:

> On 25/04/2015 12:50 a.m., James Lay wrote:
> > Hey all.
> > 
> > Topic says itI'm running squid-3.5.3-20150420-r13802 and wanted to
> > see if there's anything glaring that I'm missing/have misconfigured.  My
> > setup is squid is running on a router, one nic external, one nic
> > internal.  This is running as a transparent proxy with iptables doing a
> > redirect to ports 3128 and 3129.  Config below:
> > 
> > #
> > acl localnet src 192.168.1.0/24
> > 
> > acl SSL_ports port 443
> > acl Safe_ports port 80  # http
> > acl Safe_ports port 443 # https
> > 
> > acl CONNECT method CONNECT
> > acl broken_sites dst 96.16.0.0/15
> > 
> > acl broken_sites dst 54.160.0.0/12
> > acl allowed_sites url_regex "/opt/etc/squid/url.txt"
> > acl all_others dst all
> 
> Using "dst all" is very inefficient. It requires Squid to perform DNS
> lookups just to answer "yes". Unless there is some unusual reason
> requiring that you might as well use the provided "all" ACL for faster
> operation.
> 
> 
> > acl SSL method CONNECT
> 
> This is a bit dangerous. CONNECT does not necessarily mean SSL - even
> with the port 443 restriction.  CONNECT could as easily contain a tunnel
> to email server and be pumping spam, or literally any other type of
> traffic to any other server. Spam emails, FTP, BitTorrent, and Skype are
> pretty popular protocols seen with CONNECT.
> 
> So you can easily mistake security rules about SSL and create allow
> policies that make you vulnerable to some nasty attacks.
> 
> Its also a redundant ACL definition with the default CONNECT ACL earlier.
> 
> > 
> > http_access deny !Safe_ports
> > http_access deny CONNECT !SSL_ports
> > 
> > http_access allow manager localhost
> > http_access deny manager
> > 
> > http_access allow allowed_sites
> > http_access allow broken_sites
> > 
> > http_access deny all_others 
> 
> The above being equivalent to "deny all" makes the below rules not do
> anything. I dont know yoru policy, maybe you did.
> 
> Consider whether that is what you expected/wanted to happen.
> 
> 
> > http_access allow localnet
> > http_access allow localhost
> > 
> > http_access deny all
> > icp_access deny all
> > 
> > 
> > sslproxy_cert_error allow broken_sites
> > sslproxy_cert_error deny all
> > 
> > sslproxy_options ALL
> > acl p3129 myportname 3129
> 
> This name "3129" does not match any listening port name. See below...
> 
> 
> > acl step1 at_step SslBump1
> > ssl_bump peek step1
> > #ssl_bump splice broken_sites
> > ssl_bump bump p3129
> > 
> > 
> > http_port 192.168.1.253:3128 intercept 
> 
> ... in the absence of a name= parameter the default name for tis port is
> "192.168.1.253:3128".
> 
> > https_port 192.168.1.253:3129 intercept ssl-bump
> > cert=/opt/sslsplit/sslsplit.crt key=/opt/sslsplit/sslsplitca.key
> > cafile=/opt/sslsplit/sslsplitca.pem generate-host-certificates=on
> > dynamic_cert_mem_cache_size=4MB sslflags=NO_SESSION_REUSE
> 
> ... in the absence of a name= parameter the default name for tis port is
> "192.168.1.253:3129".
> 
> Do you see the pattern?
>  set the name= parameter eplicitly or it becomes teh *string* value of
> the host:port field.
> 
> 
> > 
> > always_direct allow all
> 
> Has no use in your config.
> 
> > 
> > logformat common %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %>Hs % > Sh %ssl::>cert_subject
> 
> Bad: do not re-define built in format definitions please.
> 
> Either use the provided format, or use a different name if you need the
> custom one.
> 
> > 
> > access_log syslog:daemon.info common
> > 
> > refresh_pattern ^ftp:   144020% 10080
> > refresh_pattern ^gopher:14400%  1440
> > refresh_pattern -i (cgi-bin|\?) 0   0%  0
> > refresh_pattern .   0   20% 4320
> > 
> > icp_port 3130
> 
> You are initializing ICP port, but also configured "icp_access deny all".
> 
> To disble ICP leave remove the icp_* directives from your config.
> 
> To enable ICP, configure the icp_access to allow some sources to make
> queries.
> 
> > 
> > coredump_dir /opt/var
> > #
> &