Re: [squid-users] High CPU Usage with ssl_bump

[Alex Rousskov]

On 04/22/2016 09:19 AM, Odhiambo Washington wrote:
> Can I terminate based on time? 

Yes. You can terminate based on any information except information
contained inside HTTP messages. Time is always available.
ssl::server_name will give you an approximation of what you call "site".


> By just modifying the bits you wrote for me?

Technically yes (any configuration can be written "by just modifying
some other configuration"). In practice, the two directions
(splice/terminate versus bump/block) are so different that I recommend
the following approach instead:

1. Study the bits Amos gave you. Do not proceed until you understand
what each line means/does. Ask specific questions if needed. IMHO, if
you cannot complete this step, then you should not subject humans to
SslBump. Without this knowledge, there will be too much suffering, on
all sides.

2. Use the first template I gave you. Add your own rules so that Squid
terminates the connections you want it to terminate and splices
everything else. If you honestly complete step #1 above, then you should
be able to do this too (and, more importantly, you would be able to
troubleshoot deployment problems).

3. Test, deploy, and ask questions/file bug reports as needed.

Alex.



> On 17:45, Fri, Apr 22, 2016 Amos Jeffries wrote:
> 
> On 23/04/2016 12:39 a.m., Odhiambo Washington wrote:
> >
> > So is it possible to achieve such a non-intrusive setup, but without
> > 'terminate'?
> 
> You declared the requirement "Serve an error page.".
> 
> That is intrusive.
> 
> As Alex has said repeatedly:
>   terminate or produce an error. Pick one.
> 
> Amos
> 
> 
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
> 

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Odhiambo Washington]

Can I terminate based on time? By just modifying the bits you wrote for me?

On 17:45, Fri, Apr 22, 2016 Amos Jeffries  wrote:

> On 23/04/2016 12:39 a.m., Odhiambo Washington wrote:
> >
> > So is it possible to achieve such a non-intrusive setup, but without
> > 'terminate'?
>
> You declared the requirement "Serve an error page.".
>
> That is intrusive.
>
> As Alex has said repeatedly:
>   terminate or produce an error. Pick one.
>
> Amos
>
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Amos Jeffries]

On 23/04/2016 12:39 a.m., Odhiambo Washington wrote:
> 
> So is it possible to achieve such a non-intrusive setup, but without
> 'terminate'?

You declared the requirement "Serve an error page.".

That is intrusive.

As Alex has said repeatedly:
  terminate or produce an error. Pick one.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 


22.04.16 18:39, Odhiambo Washington пишет:
>
>
> On 22 April 2016 at 13:45, Amos Jeffries > wrote:
>
> On 22/04/2016 8:23 p.m., Odhiambo Washington wrote:
> >
> > Sure, I am really struggling to understand this. I would like to
serve
> > error pages. A complete example of this would really help. I am
thinking,
> > based on the two templates you gave and going with the one where
squid
> > intrudes, that it could be like below, but to be honest I am not
sure so
> > kindly correct me.
> >
> >
> > acl time_wastage_sites_ssl ssl::server_name .facebook.com
 .youtube.com 
> > ssl_bump splice time_wastage_sites_ssl
> > ssl_bump stare all
> > ssl_bump bump all
> > http_access allow time_wastage_sites_ssl privileged-staff
> > http_access allow time_wastage_sites_ssl privileged-clients
> > http_access allow time_wastage_sites_ssl TIMElunch
> > http_access allow time_wastage_sites_ssl TIMEafterhoursAFT
> > http_access allow time_wastage_sites_ssl TIMEafterhoursMORN
> > http_access allow time_wastage_sites_ssl TIMEsatALLDAY
> > http_access allow time_wastage_sites_ssl TIMEsundALLDAY
> > http_access deny  time_wastage_sites_ssl
> >
>
> In a file called "/etc/squid/tws":
> .facebook.com 
> .youtube.com 
>
>
> squid.conf:
>  acl time_wastage_sites_ssl  ssl::server_name "/etc/squid/tws"
>  acl time_wastage_sites_http dstdomain"/etc/squid/tws"
>
>  acl privileged_traffic any-of \
> privileged-staff privileged-clients \
> TIMElunch TIMEafterhoursAFT TIMEafterhoursMORN \
> TIMEsatALLDAY TIMEsundALLDAY
>
>  http_access allow privileged_traffic
>  http_access deny time_wastage_sites_http
>
>  ssl_bump splice privileged_traffic time_wastage_sites_ssl
>  ssl_bump stare all
>  ssl_bump bump all
>
>
>
> You can probably merge the TIME* ACLs down as well like:
>   # lunch
>   acl okay_times time ...
>   # afterhours PM
>   acl okay_times time ...
>   # afterhours AM
>   acl okay_times time ...
>   # Saturday and Sunday all day
>   acl okay_times time SA
>
> Amos
>
>
> Quoting Alex:
> "
> If you want Squid to not intrude except when terminating prohibited
traffic, then start with this sketch:
>
> >   ssl_bump terminate prohibited_traffic
> >   ssl_bump peek all
> >   ssl_bump splice all
> "
>
> So is it possible to achieve such a non-intrusive setup, but without
'terminate'?
Not only possible. This is the only solution if you do not want legal
problems. But, of course, you will forgot about high cache hit
:))
>
>
>
> --
> Best regards,
> Odhiambo WASHINGTON,
> Nairobi,KE
> +254 7 3200 0004/+254 7 2274 3223
> "Oh, the cruft."
>
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXGhxQAAoJENNXIZxhPexG+vMH/1BC4CuOMJKp9RYxHcpf/0a+
HsBW3wdCJxCMUI6gq89wCxD9FRq/7gmJIl1vM22l6zZP15JigYrUrosBaD2bjyhk
U+e8daGrOORdnxqajgggKrCOC+pBmkjlmaceU5etteb6QfkHW4sOVTxL9kF8dx1o
0/p1Dvl4LtRynsAloBhK8mr0BMhFFYSLoYipEKSBadK0mckqxAdCIyt1EQiyNAdy
aMRfPMit5KU9JhiK8R28v0c6eSiIyP0cZ7oQG1YL2DmlGOiJ6zwPBmrsDTkb7hRZ
10XPFtzAyRydVI5ca4kN+W2o/pmIc2yUEi0CLX7qUMV88PJjS4Ep7l7+QJF5/SM=
=zxK+
-END PGP SIGNATURE-



0x613DEC46.asc
Description: application/pgp-keys
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Odhiambo Washington]

On 22 April 2016 at 13:45, Amos Jeffries  wrote:

> On 22/04/2016 8:23 p.m., Odhiambo Washington wrote:
> >
> > Sure, I am really struggling to understand this. I would like to serve
> > error pages. A complete example of this would really help. I am thinking,
> > based on the two templates you gave and going with the one where squid
> > intrudes, that it could be like below, but to be honest I am not sure so
> > kindly correct me.
> >
> >
> > acl time_wastage_sites_ssl ssl::server_name .facebook.com .youtube.com
> > ssl_bump splice time_wastage_sites_ssl
> > ssl_bump stare all
> > ssl_bump bump all
> > http_access allow time_wastage_sites_ssl privileged-staff
> > http_access allow time_wastage_sites_ssl privileged-clients
> > http_access allow time_wastage_sites_ssl TIMElunch
> > http_access allow time_wastage_sites_ssl TIMEafterhoursAFT
> > http_access allow time_wastage_sites_ssl TIMEafterhoursMORN
> > http_access allow time_wastage_sites_ssl TIMEsatALLDAY
> > http_access allow time_wastage_sites_ssl TIMEsundALLDAY
> > http_access deny  time_wastage_sites_ssl
> >
>
> In a file called "/etc/squid/tws":
> .facebook.com
> .youtube.com
>
>
> squid.conf:
>  acl time_wastage_sites_ssl  ssl::server_name "/etc/squid/tws"
>  acl time_wastage_sites_http dstdomain"/etc/squid/tws"
>
>  acl privileged_traffic any-of \
> privileged-staff privileged-clients \
> TIMElunch TIMEafterhoursAFT TIMEafterhoursMORN \
> TIMEsatALLDAY TIMEsundALLDAY
>
>  http_access allow privileged_traffic
>  http_access deny time_wastage_sites_http
>
>  ssl_bump splice privileged_traffic time_wastage_sites_ssl
>  ssl_bump stare all
>  ssl_bump bump all
>
>
>
> You can probably merge the TIME* ACLs down as well like:
>   # lunch
>   acl okay_times time ...
>   # afterhours PM
>   acl okay_times time ...
>   # afterhours AM
>   acl okay_times time ...
>   # Saturday and Sunday all day
>   acl okay_times time SA
>
> Amos
>
>
Quoting Alex:
"
If you want Squid to not intrude except when terminating prohibited traffic,
then start with this sketch:

>   ssl_bump terminate prohibited_traffic
>   ssl_bump peek all
>   ssl_bump splice all
"

So is it possible to achieve such a non-intrusive setup, but without
'terminate'?



-- 
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft."
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Amos Jeffries]

On 22/04/2016 8:23 p.m., Odhiambo Washington wrote:
> 
> Sure, I am really struggling to understand this. I would like to serve
> error pages. A complete example of this would really help. I am thinking,
> based on the two templates you gave and going with the one where squid
> intrudes, that it could be like below, but to be honest I am not sure so
> kindly correct me.
> 
> 
> acl time_wastage_sites_ssl ssl::server_name .facebook.com .youtube.com
> ssl_bump splice time_wastage_sites_ssl
> ssl_bump stare all
> ssl_bump bump all
> http_access allow time_wastage_sites_ssl privileged-staff
> http_access allow time_wastage_sites_ssl privileged-clients
> http_access allow time_wastage_sites_ssl TIMElunch
> http_access allow time_wastage_sites_ssl TIMEafterhoursAFT
> http_access allow time_wastage_sites_ssl TIMEafterhoursMORN
> http_access allow time_wastage_sites_ssl TIMEsatALLDAY
> http_access allow time_wastage_sites_ssl TIMEsundALLDAY
> http_access deny  time_wastage_sites_ssl
> 

In a file called "/etc/squid/tws":
.facebook.com
.youtube.com


squid.conf:
 acl time_wastage_sites_ssl  ssl::server_name "/etc/squid/tws"
 acl time_wastage_sites_http dstdomain"/etc/squid/tws"

 acl privileged_traffic any-of \
privileged-staff privileged-clients \
TIMElunch TIMEafterhoursAFT TIMEafterhoursMORN \
TIMEsatALLDAY TIMEsundALLDAY

 http_access allow privileged_traffic
 http_access deny time_wastage_sites_http

 ssl_bump splice privileged_traffic time_wastage_sites_ssl
 ssl_bump stare all
 ssl_bump bump all



You can probably merge the TIME* ACLs down as well like:
  # lunch
  acl okay_times time ...
  # afterhours PM
  acl okay_times time ...
  # afterhours AM
  acl okay_times time ...
  # Saturday and Sunday all day
  acl okay_times time SA

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Odhiambo Washington]

On 22 April 2016 at 02:16, Alex Rousskov 
wrote:

> On 04/21/2016 03:26 PM, Odhiambo Washington wrote:
> > On 21 April 2016 at 23:14, Alex Rousskov wrote:
> > Logging aside, your latest random configuration is equivalent to
> > [...] not intercepting SSL at all, which brings
> > us back to the old question: What do you want Squid to do?
>
>
> > If I could intercept SSL and do nothing EXCEPT subject the domains to
> > time ACLs, that'd be all.
>
> You are going back to the problem we have already discussed. Please slow
> down and translate your description above into what should happen to
> user connections that match your "time ACLs".
>


*slow down mode engaged*

You have given me these two templates:

(1)
If you want Squid to not intrude except when terminating prohibited traffic,
then start with this sketch:

  ssl_bump terminate prohibited_traffic
  ssl_bump peek all
  ssl_bump splice all

I would have preffered this option, first because it doesn't involve me
installing my CA on all user devices and secondly because of no intrusion.
However I cannot figure out how to deal with this when it comes to ACLs
because '*terminate*' isn't really what I think I want. What I want is as
follows:
(a) squid receives requiest from a particular host for facebook.com. Host
is identified by MAC Address or IP
(b) squid decides (based on ACLs) if host is allowed access to facebook.com
at this time, then allows it
(c) squid throws an error message if host is not allowed access at this
time.

If I could achieve the above, I will be fine. How to craft the configs is
my trouble. I keep fumbling.


(2)
If you want Squid to intrude (where possible) and block prohibited
traffic, then install your CA certificates on all user devices and start
with this sketch:

  ssl_bump splice things_that_are_impossible_to_bump
  ssl_bump stare all
  ssl_bump bump all
  http_access deny prohibited_traffic

Now here, the CA challenge abounds. We have a guest SSID on our WLAN and
this means I have to install the CAs even for guests or redo the network to
be able to accommodate guest users browsing without being subjected to our
internal policies.



>
> * Does "subject the domains to time ACLs" mean "immediately close
> connections that match" those ACLs?
>

No.


>
> * Or does it mean "serve Squid error pages" over connections that match
> those ACLs?
>

Yes.


>
> Once you decide, apply one of the two templates provided (the two
> templates correspond to which of the two questions you answer "yes").
>
>
> > I just want the data passing through squid for me to determine who is
> > allowed to access it and at what time.
>
> Assume Squid has made that access determination you want to make, and
> the user is not allowed. Now what: Close the connection? Or serve an
> error page?
>
>
Serve an error page.

.

>
> > I do have time ACLs, [...]
>
> The specifics of your ACLs are irrelevant at this stage. You can fix
> them later once you get overall SslBump setup working the way you want.
> You can assume that there is just one ACL called "prohibited_traffic" or
> "good_traffic". Now write the rules that determine what happens to
> connections that match one of those two ACLs.
>



> If you want Squid to not intrude except when terminating prohibited
> > traffic, then start with this sketch:
> >
> >   ssl_bump terminate prohibited_traffic
> >   ssl_bump peek all
> >   ssl_bump splice all
> >
> >
> > Lemme see if I understand this. I have a problem wrapping my head around
> > 'terminate' (as a terminology, maybe)
>
> "terminate" means "close the SSL connection(s) immediately". No error
> response is sent by Squid to the user. It does not get much simpler than
> that! The browser will probably show some "secure connection could not
> be negotiated" error to the user with no usable details [because Squid
> sent nothing to the browser in this case].
>
>
That is NOT what I want. I need squid to serve an error page that "Access
is denied at this time.."
I think it's usually something like "access controls prohibit you from
access this page at this time...".


>
> > and 'prohibited_traffic' (also as a terminology).
>
> Just some ACL name. You will define that aggregate ACL later to match
> any traffic you want to prohibit. It will contain a combination of time
> and server name ACLs. Other details are not important until your SslBump
> [and http_access rules] are correct.
>

Okay.


>
> If you do not know how to aggregate ACLs, look for "any-of" and "all-of"
> in squid.conf.documented, but, again, ACL specifics are not important
> right now. They will become important at stage three. Now you are
> struggling with stage one: Deciding what to do with matching SSL
> connections (close or serve error pages).
>

Sure, I am really struggling to understand this. I would like to serve
error pages. A complete example of this would really help. I am thinking,
based on the two templates you gave and 

Re: [squid-users] High CPU Usage with ssl_bump

[Alex Rousskov]

On 04/21/2016 03:26 PM, Odhiambo Washington wrote:
> On 21 April 2016 at 23:14, Alex Rousskov wrote:
> Logging aside, your latest random configuration is equivalent to
> [...] not intercepting SSL at all, which brings
> us back to the old question: What do you want Squid to do?


> If I could intercept SSL and do nothing EXCEPT subject the domains to
> time ACLs, that'd be all.

You are going back to the problem we have already discussed. Please slow
down and translate your description above into what should happen to
user connections that match your "time ACLs".

* Does "subject the domains to time ACLs" mean "immediately close
connections that match" those ACLs?

* Or does it mean "serve Squid error pages" over connections that match
those ACLs?

Once you decide, apply one of the two templates provided (the two
templates correspond to which of the two questions you answer "yes").


> I just want the data passing through squid for me to determine who is
> allowed to access it and at what time.

Assume Squid has made that access determination you want to make, and
the user is not allowed. Now what: Close the connection? Or serve an
error page?


> I do have time ACLs, [...]

The specifics of your ACLs are irrelevant at this stage. You can fix
them later once you get overall SslBump setup working the way you want.
You can assume that there is just one ACL called "prohibited_traffic" or
"good_traffic". Now write the rules that determine what happens to
connections that match one of those two ACLs.


> If you want Squid to not intrude except when terminating prohibited
> traffic, then start with this sketch:
> 
>   ssl_bump terminate prohibited_traffic
>   ssl_bump peek all
>   ssl_bump splice all
> 
> 
> Lemme see if I understand this. I have a problem wrapping my head around
> 'terminate' (as a terminology, maybe)

"terminate" means "close the SSL connection(s) immediately". No error
response is sent by Squid to the user. It does not get much simpler than
that! The browser will probably show some "secure connection could not
be negotiated" error to the user with no usable details [because Squid
sent nothing to the browser in this case].


> and 'prohibited_traffic' (also as a terminology).

Just some ACL name. You will define that aggregate ACL later to match
any traffic you want to prohibit. It will contain a combination of time
and server name ACLs. Other details are not important until your SslBump
[and http_access rules] are correct.

If you do not know how to aggregate ACLs, look for "any-of" and "all-of"
in squid.conf.documented, but, again, ACL specifics are not important
right now. They will become important at stage three. Now you are
struggling with stage one: Deciding what to do with matching SSL
connections (close or serve error pages).

FWIW, my recommendation is to terminate/close and find other ways to
inform users about their policy violations.

Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Odhiambo Washington]

On 21 April 2016 at 23:14, Alex Rousskov 
wrote:

> On 04/21/2016 01:59 PM, Odhiambo Washington wrote:
> > On 21 April 2016 at 22:04, Amos Jeffries wrote:
> >
> > On 22/04/2016 6:20 a.m., Odhiambo Washington wrote:
> > > I have now changed to *configurations suggested specifically for
> your use
> > > case, on this email thread* :)
>
> > > acl no_ssl_interception ssl::server_name
> > > "/usr/local/etc/squid/ssl_bump_broken_sites.txt"
> > > ssl_bump splice no_ssl_interception
> > > ssl_bump stare all
> > > ssl_bump bump all
>
>
> > No the "stare" being done will prevent splice and you will see
> breakage
> > or unexpected things again.
> > You have to replace 'stare' with 'peek' AND replace 'bump' with
> > 'splice'.
>
>
> > Like below???
>
>
> > acl no_ssl_interception ssl::server_name
> > "/usr/local/etc/squid/ssl_bump_broken_sites.txt"
> > ssl_bump splice no_ssl_interception
> > ssl_bump peek all
> > ssl_bump splice all
>
>
> Logging aside, your latest random configuration is equivalent to
>
>   ssl_bump splice all
>
> which means you are better off not intercepting SSL at all, which brings
> us back to the old question: What do you want Squid to do?
>

If I could intercept SSL and do nothing EXCEPT subject the domains to time
ACLs, that'd be all. I do not need any bumping to be precise.
I just want the data passing through squid for me to determine who is
allowed to access it and at what time.

I do have time ACLs, which I use in conjunction with MAC ACLs and certain
domain, like facebook.com. Right now I rely on destdomain to identify the
domains.
With domains using SSL, I cannot use destdomain, right? Let me demonstrate
with config snippets from my squid.conf, perhaps that would explain it
better and maybe I will find a better easier way out that this ssl_bump
monster:-)

acl TIMEweekdaysALLDAY time MTWHF 09:00-17:00
acl TIMEafterhoursMORN time MTWHF 00:00-09:00
acl TIMElunch time MTWHF 12:00-13:59
acl TIMEafterhoursAFT time MTWHF 14:30-23:59
acl TIMEsatMORN time A 00:00-07:00
acl TIMEsatAFT time A 11:00-23:59
acl TIMEsatALLDAY time A 00:00-23:59
acl TIMEsundALLDAY time S 00:00-23:59

# Internet Access Regulation
# Some privileged staff - can browse unrestricted
acl privileged-staff arp "/usr/local/etc/squid/privileged_mac_addresses.txt"

# Sites staff waste all their time on - Social Networks, streaming sites, etc
acl TIMEWASTAGESITES dstdomain "/usr/local/etc/squid/time_wastage_sites.txt"

# TIMEWASTAGESITES
http_access allow TIMEWASTAGESITES privileged-staff
http_access allow TIMEWASTAGESITES TIMElunch
http_access allow TIMEWASTAGESITES TIMEafterhoursAFT
http_access allow TIMEWASTAGESITES TIMEafterhoursMORN
http_access allow TIMEWASTAGESITES TIMEsatALLDAY
http_access allow TIMEWASTAGESITES TIMEsundALLDAY
http_access deny  TIMEWASTAGESITES


Now, the above used to work until facebook.com, youtube.com, and
others switched to HTTPS. After that it wasn't possible for me to
control access to them.

So really, all I want is the ability to intercept SSL and subject it
to those time ACLs for a class of users. Nothing more.



> If you want Squid to not intrude except when terminating prohibited
> traffic, then start with this sketch:
>
>   ssl_bump terminate prohibited_traffic
>   ssl_bump peek all
>   ssl_bump splice all
>

Lemme see if I understand this. I have a problem wrapping my head around
'terminate' (as a terminology, maybe) and 'prohibited_traffic' (also as a
terminology).
So, are you saying that prohibited_traffic here is something like:

acl TIMEWASTAGE_SSL
ssl::server_name "/usr/local/etc/squid/time_wastage_SSL_sites.txt"

And in that file I have
.facebook.com
.youtube.com
...

Then I just apply my usual time ACLs..

http_access allow TIMEWASTAGESITES_SSL privileged_staff
http_access allow TIMEWASTAGESITES_SSL TIMElunch
http_access allow TIMEWASTAGESITES_SSL TIMEafterhoursAFT
http_access allow TIMEWASTAGESITES_SSL TIMEafterhoursMORN
http_access allow TIMEWASTAGESITES_SSL TIMEsatALLDAY
http_access allow TIMEWASTAGESITES_SSL TIMEsundALLDAY
http_access deny  TIMEWASTAGESITES_SSL

That is more like all I want - use *ssl::server_name* instead of *dstdomain.
* The sites are NOT prohibited per se, and 'terminate' kinda scares me. I
need to RTFM about this 'terminate' again and again until I can sing it:)



> If you want Squid to intrude (where possible) and block prohibited
> traffic, then install your CA certificates on all user devices and start
> with this sketch:
>
>   ssl_bump splice things_that_are_impossible_to_bump
>   ssl_bump stare all
>   ssl_bump bump all
>   http_access deny prohibited_traffic
>
> Alex.
>
>

I actually believe I do not need any mention of ssl_bump, but the confusion
for me arises out of TWO things:

1. In my firewall (FreeBSD PF), redirecting traffic destined to port 443
via squid wasn't an option prior to the intruduction of the SSL
interception, right?
2. In squid.conf, I did not yet 

Re: [squid-users] High CPU Usage with ssl_bump

[Alex Rousskov]

On 04/21/2016 01:59 PM, Odhiambo Washington wrote:
> On 21 April 2016 at 22:04, Amos Jeffries wrote:
> 
> On 22/04/2016 6:20 a.m., Odhiambo Washington wrote:
> > I have now changed to *configurations suggested specifically for your 
> use
> > case, on this email thread* :)

> > acl no_ssl_interception ssl::server_name
> > "/usr/local/etc/squid/ssl_bump_broken_sites.txt"
> > ssl_bump splice no_ssl_interception
> > ssl_bump stare all
> > ssl_bump bump all


> No the "stare" being done will prevent splice and you will see breakage
> or unexpected things again. 
> You have to replace 'stare' with 'peek' AND replace 'bump' with
> 'splice'.


> Like below???


> acl no_ssl_interception ssl::server_name
> "/usr/local/etc/squid/ssl_bump_broken_sites.txt"
> ssl_bump splice no_ssl_interception
> ssl_bump peek all
> ssl_bump splice all 


Logging aside, your latest random configuration is equivalent to

  ssl_bump splice all

which means you are better off not intercepting SSL at all, which brings
us back to the old question: What do you want Squid to do?


If you want Squid to not intrude except when terminating prohibited
traffic, then start with this sketch:

  ssl_bump terminate prohibited_traffic
  ssl_bump peek all
  ssl_bump splice all

If you want Squid to intrude (where possible) and block prohibited
traffic, then install your CA certificates on all user devices and start
with this sketch:

  ssl_bump splice things_that_are_impossible_to_bump
  ssl_bump stare all
  ssl_bump bump all
  http_access deny prohibited_traffic

Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Odhiambo Washington]

On 21 April 2016 at 22:04, Amos Jeffries  wrote:

> On 22/04/2016 6:20 a.m., Odhiambo Washington wrote:
> > Hi Alex,
> >
> > I have now changed to *configurations suggested specifically for your use
> > case, on this email thread* :)
> >
> >
> >
> > acl no_ssl_interception ssl::server_name
> > "/usr/local/etc/squid/ssl_bump_broken_sites.txt"
> > ssl_bump splice no_ssl_interception
> > ssl_bump stare all
> > ssl_bump bump all
> >
> > Now, suppose, as I think in my mind, bumping isn't really what I need,
> can
> > I just comment out 'ssl_bump bump all'  and sit easy or should I switch
> to
> > ssl_bump splice all ??
>
> No the "stare" being done will prevent splice and you will see breakage
> or unexpected things again.
>
> You have to replace 'stare' with 'peek' AND replace 'bump' with 'splice'.
> 
>

Like below???


acl no_ssl_interception ssl::server_name
"/usr/local/etc/squid/ssl_bump_broken_sites.txt"
ssl_bump splice no_ssl_interception
ssl_bump peek all
ssl_bump splice all


Thank you.


-- 
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft."
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Amos Jeffries]

On 22/04/2016 6:20 a.m., Odhiambo Washington wrote:
> Hi Alex,
> 
> I have now changed to *configurations suggested specifically for your use
> case, on this email thread* :)
> 
> 
> 
> acl no_ssl_interception ssl::server_name
> "/usr/local/etc/squid/ssl_bump_broken_sites.txt"
> ssl_bump splice no_ssl_interception
> ssl_bump stare all
> ssl_bump bump all
> 
> Now, suppose, as I think in my mind, bumping isn't really what I need, can
> I just comment out 'ssl_bump bump all'  and sit easy or should I switch to
> ssl_bump splice all ??

No the "stare" being done will prevent splice and you will see breakage
or unexpected things again.

You have to replace 'stare' with 'peek' AND replace 'bump' with 'splice'.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Alex Rousskov]

On 04/21/2016 08:12 AM, Odhiambo Washington wrote:

> acl no_ssl_interception ssl::server_name ...
> ssl_bump splice no_ssl_interception 
> ssl_bump stare step2
> ssl_bump splice all

You are mixing splice and stare now. There are two groups of actions:

* peek and then splice
* stare and then bump

Do not mix actions from different groups together unless you know what
you are doing.


> So basically I should just have two options, I think, no?? Like
> 
> ssl_bump stare step2
> ssl_bump splice all

Two bugs in this config:

1. It will splice everything during step #1. It is equivalent to:

   ssl_bump splice all


2. To quote the wiki page:

stare (step2): Receive server certificate while preserving the
possibility of bumping the connection. Staring at the server certificate
usually precludes future splicing of the connection.

squid.conf.documented has very similar text as well.

You are telling Squid to splice do exactly what the documentation tells
you is not usually possible.


I can understand that it may be difficult to find and interpret
documentation correctly. I can understand that it is difficult to
evaluate a given configuration correctly. What I cannot understand is
why you are not starting with configurations suggested specifically for
your use case, on this email thread.


> If one day, for some reason I want to bump, then I could change to:
> 
> ssl_bump splice no_ssl_interception
> ssl_bump stare step2
> ssl_bump bump all

Similar to #1 above, this will bump all connections not matching the
[misnamed] no_ssl_interception during step1.

The first matching action wins. During step1, that action is "bump" from
your last rule if no_ssl_interception does not match.


HTH,

Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Odhiambo Washington]

On 21 April 2016 at 16:48, Alex Rousskov 
wrote:

> On 04/21/2016 07:18 AM, Odhiambo Washington wrote:
> > Is is expected that  using ssl_bump results into high CPU usage all the
> > time?
>
> Your question is impossible to answer in general: The CPU usage levels
> depend on the amount of Squid traffic, the portion of SSL traffic in the
> overall traffic mix, the portion of step1, step2, and step3 traffic in
> the SSL traffic mix, hardware resources available to Squid, the number
> of Squid workers, and many other factors.
>
> > acl no_ssl_interception ssl::server_name ...
> > ssl_bump splice no_ssl_interception
> > ssl_bump peek step1
> > ssl_bump stare step2
>
> The above config continues to violate the specific advice given to you
> previously:
> *Do not mix "peek" and "stare" unless you have a very specific need for
> doing so.*
>

I have noted that instruction. It was actually an oversight caused by slow
understanding of the terminologies.
Once I have changed to what you advised before, the CPU usage has gone down
considerably:


acl no_ssl_interception ssl::server_name
"/usr/local/etc/squid/ssl_bump_broken_sites.txt"
ssl_bump splice no_ssl_interception
ssl_bump stare step2
#ssl_bump bump all
ssl_bump splice all

So basically I should just have two options, I think, no?? Like

ssl_bump stare step2
ssl_bump splice all

If one day, for some reason I want to bump, then I could change to:

acl no_ssl_interception ssl::server_name
"/usr/local/etc/squid/ssl_bump_broken_sites.txt"
ssl_bump splice no_ssl_interception
ssl_bump stare step2
ssl_bump bump all


Thank you so much Alex.


-- 
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft."
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Alex Rousskov]

On 04/21/2016 07:18 AM, Odhiambo Washington wrote:
> Is is expected that  using ssl_bump results into high CPU usage all the
> time?

Your question is impossible to answer in general: The CPU usage levels
depend on the amount of Squid traffic, the portion of SSL traffic in the
overall traffic mix, the portion of step1, step2, and step3 traffic in
the SSL traffic mix, hardware resources available to Squid, the number
of Squid workers, and many other factors.

> acl no_ssl_interception ssl::server_name ...
> ssl_bump splice no_ssl_interception 
> ssl_bump peek step1
> ssl_bump stare step2

The above config continues to violate the specific advice given to you
previously: Do not mix "peek" and "stare" unless you have a very
specific need for doing so.


> I think I read somewhere that 'ssl_bump splice all" is the default
> behaviour, hence why I have commented it out. All I need is just become
> a TCP tunnel without decrypting proxied traffic.

"splice all" is not the default in the latest Squids. The default there
is closer to something like "bump if the last step was 'stare' and
splice otherwise". I do not remember what the default is in your Squid
version, but, as Amos has already said, relying on _any_ default in this
complex environment is the wrong approach.

Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Odhiambo Washington]

I will put the splice explicitly and observe.

Without ssl_bump I never saw such cpu usage with squid.

However, lemme watch and also listen to feedback..


On 21 April 2016 at 16:34, Amos Jeffries  wrote:

> On 22/04/2016 1:18 a.m., Odhiambo Washington wrote:
> > Is is expected that  using ssl_bump results into high CPU usage all the
> > time?
> >
>
> Encryption adds CPU overhead, but how much depends on what your normal
> use was. I dont think any of us have a good rule-of-thumb or educated
> guess yet because Squid code has been changing so much.
>
> If its worrying you, I suggest trying your favourite profiling tools out
> and see if anything useful shows up.
>
>
> > This is squid-3.5.17
> >
> > That is what I am seeing:
> >
> > last pid: 26673;  load averages:  2.24,  2.00,  2.10
> >
> >   up 0+03:47:56  16:08:30
> > 160 processes: 2 running, 157 sleeping, 1 zombie
> > CPU: 86.1% user,  0.0% nice,  7.8% system,  3.3% interrupt,  2.7% idle
> > Mem: 843M Active, 1942M Inact, 185M Wired, 43M Cache, 89M Buf, 97M Free
> > Swap: 5900M Total, 1248K Used, 5899M Free
> >
> >   PID USERNAME   THR PRI NICE   SIZERES STATE   C   TIMEWCPU
> > COMMAND
> > 13309 squid   17  200   305M   264M uwait   0   7:38  80.86%
> > squid
> > 26088 squid1  210 12812K  5352K sbwait  1   0:04   2.49%
> > ssl_crtd
> > 26090 squid1  200 12812K  5272K sbwait  1   0:01   0.88%
> > ssl_crtd
> >
> >
> > My config has:
> >
> >
> >
> > acl no_ssl_interception ssl::server_name
> > "/usr/local/etc/squid/ssl_bump_broken_sites.txt"
> > ssl_bump splice no_ssl_interception
> > ssl_bump peek step1
> > ssl_bump stare step2
> > #ssl_bump bump all
> > #ssl_bump splice all
> >
> > I think I read somewhere that 'ssl_bump splice all" is the default
> > behaviour, hence why I have commented it out. All I need is just become a
> > TCP tunnel without decrypting proxied traffic.
>
> I wouldn't rely on the default for things like this. Squid makes a
> *guess* based on what data it has to work with on a per-connection
> basis. There is no extra cost to having it configured, Squid has to
> check the whole set anyway.
>
> Amos
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>



-- 
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft."
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Amos Jeffries]

On 22/04/2016 1:18 a.m., Odhiambo Washington wrote:
> Is is expected that  using ssl_bump results into high CPU usage all the
> time?
> 

Encryption adds CPU overhead, but how much depends on what your normal
use was. I dont think any of us have a good rule-of-thumb or educated
guess yet because Squid code has been changing so much.

If its worrying you, I suggest trying your favourite profiling tools out
and see if anything useful shows up.


> This is squid-3.5.17
> 
> That is what I am seeing:
> 
> last pid: 26673;  load averages:  2.24,  2.00,  2.10
> 
>   up 0+03:47:56  16:08:30
> 160 processes: 2 running, 157 sleeping, 1 zombie
> CPU: 86.1% user,  0.0% nice,  7.8% system,  3.3% interrupt,  2.7% idle
> Mem: 843M Active, 1942M Inact, 185M Wired, 43M Cache, 89M Buf, 97M Free
> Swap: 5900M Total, 1248K Used, 5899M Free
> 
>   PID USERNAME   THR PRI NICE   SIZERES STATE   C   TIMEWCPU
> COMMAND
> 13309 squid   17  200   305M   264M uwait   0   7:38  80.86%
> squid
> 26088 squid1  210 12812K  5352K sbwait  1   0:04   2.49%
> ssl_crtd
> 26090 squid1  200 12812K  5272K sbwait  1   0:01   0.88%
> ssl_crtd
> 
> 
> My config has:
> 
> 
> 
> acl no_ssl_interception ssl::server_name
> "/usr/local/etc/squid/ssl_bump_broken_sites.txt"
> ssl_bump splice no_ssl_interception
> ssl_bump peek step1
> ssl_bump stare step2
> #ssl_bump bump all
> #ssl_bump splice all
> 
> I think I read somewhere that 'ssl_bump splice all" is the default
> behaviour, hence why I have commented it out. All I need is just become a
> TCP tunnel without decrypting proxied traffic.

I wouldn't rely on the default for things like this. Squid makes a
*guess* based on what data it has to work with on a per-connection
basis. There is no extra cost to having it configured, Squid has to
check the whole set anyway.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 
Not necessary. May be bottleneck in OS.

21.04.16 19:25, Odhiambo Washington пишет:
> So, what could possibly be wrong with my setup, that squid consumes so much 
> CPU?
>
> On 21 April 2016 at 16:22, Yuri Voinov > wrote:
>
>
> It must not be. My most active setup has 3% CPU all time dirung peak
hours.
>
> Typical view:
>
> https://i1.someimage.com/NzM1erI.png
>
>
> 21.04.16 19:18, Odhiambo Washington пишет:
> > Is is expected that  using
>   ssl_bump results into high CPU usage all the time?
>
>
>
>   > This is squid-3.5.17
>
>
>
>   > That is what I am seeing:
>
>
>
>   > last pid: 26673;  load averages:  2.24,  2.00,
>
2.10
   

>   up 0+03:47:56  16:08:30
>
>   > 160 processes: 2 running, 157 sleeping, 1 zombie
>
>   > CPU: 86.1% user,  0.0% nice,  7.8% system,  3.3% interrupt,
>   2.7% idle
>
>   > Mem: 843M Active, 1942M Inact, 185M Wired, 43M Cache, 89M
>   Buf, 97M Free
>
>   > Swap: 5900M Total, 1248K Used, 5899M Free
>
>
>
>   >   PID USERNAME   THR PRI NICE   SIZERES STATE   C 
>   TIMEWCPU COMMAND
>
>   > 13309 squid   17  200   305M   264M uwait   0 
>   7:38  80.86% squid
>
>   > 26088 squid1  210 12812K  5352K sbwait  1 
>   0:04   2.49% ssl_crtd
>
>   > 26090 squid1  200 12812K  5272K sbwait  1 
>   0:01   0.88% ssl_crtd
>
>
>
>
>
>   > My config has:
>
>
>
>
>
>
>
>   > acl no_ssl_interception ssl::server_name
>   "/usr/local/etc/squid/ssl_bump_broken_sites.txt"
>
>   > ssl_bump splice no_ssl_interception
>
>   > ssl_bump peek step1
>
>   > ssl_bump stare step2
>
>   > #ssl_bump bump all
>
>   > #ssl_bump splice all
>
>
>
>   > I think I read somewhere that 'ssl_bump splice all" is the
>   default behaviour, hence why I have commented it out. All I need
>   is just become a TCP tunnel without decrypting proxied traffic.
>
>
>
>   > Thank you.
>
>
>
>
>
>   > --
>
>   > Best regards,
>
>   > Odhiambo WASHINGTON,
>
>   > Nairobi,KE
>
>   > +254 7 3200 0004/+254 7 2274 3223
>
>   > "Oh, the cruft."
>
>
>
>
>
>   > ___
>
>   > squid-users mailing list
>
>   > squid-users@lists.squid-cache.org

>
>   > http://lists.squid-cache.org/listinfo/squid-users
>
>
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org

> http://lists.squid-cache.org/listinfo/squid-users
>
>
>
>
> --
> Best regards,
> Odhiambo WASHINGTON,
> Nairobi,KE
> +254 7 3200 0004/+254 7 2274 3223
> "Oh, the cruft."

-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXGNa7AAoJENNXIZxhPexGQw0H+QG5jXWboYFaClEOOwwMJ3Vl
KUI6h4pnzRrcSEkMNMu0bBRytqGVXzplK5cl2U4T+scDWWpCj3pgcb1i8wITtwze
nEhvCeG9Xq6BZujvIKvQ32pe64EIl/tjstbpxxMdGLprSFre5mUXKd1NG5JghD0E
7oWYvA/hNwo/rpevzxUzU9Z+FOmgRYJ+JEGThGa86Qe+bE3B2Kxt6RAVAUbRfBcY
M/xJBX9wgeBgxncYG0IFtbJEG12X2j9Y4KXIOoh3nTpinOi7nbACF2ryr2H7z+GT
pFuNWiSnf5eSz3lEdAv+GHPZb9Fg1noD8Mxq8+P83d4BwA/L2ba/DJIweVTSEpQ=
=asuL
-END PGP SIGNATURE-



0x613DEC46.asc
Description: application/pgp-keys
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Odhiambo Washington]

So, what could possibly be wrong with my setup, that squid consumes so much
CPU?

On 21 April 2016 at 16:22, Yuri Voinov  wrote:

>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> It must not be. My most active setup has 3% CPU all time dirung peak hours.
>
> Typical view:
>
> https://i1.someimage.com/NzM1erI.png
>
>
> 21.04.16 19:18, Odhiambo Washington пишет:
> > Is is expected that  using ssl_bump results into high CPU usage all the
> time?
> >
> > This is squid-3.5.17
> >
> > That is what I am seeing:
> >
> > last pid: 26673;  load averages:  2.24,  2.00,
> 2.10
> up 0+03:47:56  16:08:30
> > 160 processes: 2 running, 157 sleeping, 1 zombie
> > CPU: 86.1% user,  0.0% nice,  7.8% system,  3.3% interrupt,  2.7% idle
> > Mem: 843M Active, 1942M Inact, 185M Wired, 43M Cache, 89M Buf, 97M Free
> > Swap: 5900M Total, 1248K Used, 5899M Free
> >
> >   PID USERNAME   THR PRI NICE   SIZERES STATE   C   TIMEWCPU
> COMMAND
> > 13309 squid   17  200   305M   264M uwait   0   7:38  80.86%
> squid
> > 26088 squid1  210 12812K  5352K sbwait  1   0:04   2.49%
> ssl_crtd
> > 26090 squid1  200 12812K  5272K sbwait  1   0:01   0.88%
> ssl_crtd
> >
> >
> > My config has:
> >
> >
> >
> > acl no_ssl_interception ssl::server_name
> "/usr/local/etc/squid/ssl_bump_broken_sites.txt"
> > ssl_bump splice no_ssl_interception
> > ssl_bump peek step1
> > ssl_bump stare step2
> > #ssl_bump bump all
> > #ssl_bump splice all
> >
> > I think I read somewhere that 'ssl_bump splice all" is the default
> behaviour, hence why I have commented it out. All I need is just become a
> TCP tunnel without decrypting proxied traffic.
> >
> > Thank you.
> >
> >
> > --
> > Best regards,
> > Odhiambo WASHINGTON,
> > Nairobi,KE
> > +254 7 3200 0004/+254 7 2274 3223
> > "Oh, the cruft."
> >
> >
> > ___
> > squid-users mailing list
> > squid-users@lists.squid-cache.org
> > http://lists.squid-cache.org/listinfo/squid-users
>
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2
>
> iQEcBAEBCAAGBQJXGNQsAAoJENNXIZxhPexGcZUIAL3zFz9UFuQdyfXFBilFQ0Gj
> 8F4HkxuJjNtCUYdb6BEwux9jBOjZpYScr8sRHRBPvIV8O4/2Z3QF7exjEW8Duj/G
> REWO3txPiE4pICD/AbdBuX8O++dvfjj46nz+lVeCH9JjGW0VoMHiyGtwGx1shSfY
> pGX0MguEGEtWp/7hxKAFbRivGuvyQ7Ogj8i9IgMBptMrRu4D3G75UO+9WmaHcpVx
> VAf1revHh+dWFWrO1k+zrWFIIFcwbR5LcrJeBYJ94scgPV3p68LC2ZpqUBZreYCM
> Koo9+Rss+Ix1rTSUkvTaoGOcMdrHJ1oMICHwyqtDMWlbDds5dAnnWXh5faNYPFk=
> =7YlO
> -END PGP SIGNATURE-
>
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
>


-- 
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft."
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High CPU Usage with ssl_bump

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 
It must not be. My most active setup has 3% CPU all time dirung peak hours.

Typical view:

https://i1.someimage.com/NzM1erI.png


21.04.16 19:18, Odhiambo Washington пишет:
> Is is expected that  using ssl_bump results into high CPU usage all the time?
>
> This is squid-3.5.17
>
> That is what I am seeing:
>
> last pid: 26673;  load averages:  2.24,  2.00, 
2.10

up 0+03:47:56  16:08:30
> 160 processes: 2 running, 157 sleeping, 1 zombie
> CPU: 86.1% user,  0.0% nice,  7.8% system,  3.3% interrupt,  2.7% idle
> Mem: 843M Active, 1942M Inact, 185M Wired, 43M Cache, 89M Buf, 97M Free
> Swap: 5900M Total, 1248K Used, 5899M Free
>
>   PID USERNAME   THR PRI NICE   SIZERES STATE   C   TIME   
WCPU COMMAND
> 13309 squid   17  200   305M   264M uwait   0   7:38 
80.86% squid
> 26088 squid1  210 12812K  5352K sbwait  1   0:04  
2.49% ssl_crtd
> 26090 squid1  200 12812K  5272K sbwait  1   0:01  
0.88% ssl_crtd
>
>
> My config has:
>
>
>
> acl no_ssl_interception ssl::server_name
"/usr/local/etc/squid/ssl_bump_broken_sites.txt"
> ssl_bump splice no_ssl_interception
> ssl_bump peek step1
> ssl_bump stare step2
> #ssl_bump bump all
> #ssl_bump splice all
>
> I think I read somewhere that 'ssl_bump splice all" is the default
behaviour, hence why I have commented it out. All I need is just become
a TCP tunnel without decrypting proxied traffic.
>
> Thank you.
>
>
> --
> Best regards,
> Odhiambo WASHINGTON,
> Nairobi,KE
> +254 7 3200 0004/+254 7 2274 3223
> "Oh, the cruft."
>
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXGNQsAAoJENNXIZxhPexGcZUIAL3zFz9UFuQdyfXFBilFQ0Gj
8F4HkxuJjNtCUYdb6BEwux9jBOjZpYScr8sRHRBPvIV8O4/2Z3QF7exjEW8Duj/G
REWO3txPiE4pICD/AbdBuX8O++dvfjj46nz+lVeCH9JjGW0VoMHiyGtwGx1shSfY
pGX0MguEGEtWp/7hxKAFbRivGuvyQ7Ogj8i9IgMBptMrRu4D3G75UO+9WmaHcpVx
VAf1revHh+dWFWrO1k+zrWFIIFcwbR5LcrJeBYJ94scgPV3p68LC2ZpqUBZreYCM
Koo9+Rss+Ix1rTSUkvTaoGOcMdrHJ1oMICHwyqtDMWlbDds5dAnnWXh5faNYPFk=
=7YlO
-END PGP SIGNATURE-



0x613DEC46.asc
Description: application/pgp-keys
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users