Re: [squid-users] sourcehash load balance

2016-10-24 Thread André Janna 

On 23/10/2016 10:35 a.m., Amos Jeffries wrote:


On 22/10/2016 12:21 a.m., André Janna wrote:

I set up a Squid proxy that forwards all requests to 2 parent caches.
I'm using Squid version 3.5.19.
My goal is that multiple connection from a client to a server should be
forwarded to the same parent, so that the server see all requests coming
from the same IP address.

I'm using the following configuration:
cache_peer squid1 parent 3128 0 no-query sourcehash
cache_peer squid2 parent 3128 0 no-query sourcehash
never_direct allow all

Looking at access.log some requests are tagged as CLOSEST_PARENT instead
of SOURCEHASH_PARENT, so it seams that Squid is not always using source
hash rule to forward requests to parent caches.
For instance:
1477046954.047   3882 10.11.2.4 TCP_TUNNEL/200 21935 CONNECT
sso.cisco.com:443 - SOURCEHASH_PARENT/10.0.33.12 -
1477046968.056 21 10.11.2.4 TCP_MISS/200 1012 POST
http://ocsp.digicert.com/ - CLOSEST_PARENT/10.0.33.13
application/ocsp-response
1477047782.038 22 10.11.2.4 TCP_MISS/204 307 GET
http://clients1.google.com/generate_204 - SOURCEHASH_PARENT/10.0.33.12 -
1477047782.045181 10.11.2.4 TCP_MISS/200 745 GET
http://tags.bluekai.com/site/2964? - CLOSEST_PARENT/10.0.33.13 image/gif

So requests from the same client are not sent to the same parent cache.
How can I force Squid to always use source hash parent selection method?

Check you setting for nonhierarchical_direct. It should be 'off'. The
default is 'on'.

Amos




Hi Amos,
I've just added "nonhierarchical_direct off" to my setting but there are 
still requests that are forwarded using a parent selection method 
different from "sourcehash".

For instance:
1477331305.998 13 10.11.0.12 TCP_MISS/200 4267 GET 
http://www.cisco.com/etc/designs/cdc/dmr/icons/arrows-grey.png - 
SOURCEHASH_PARENT/10.0.33.12 image/png
1477331306.948 10 10.11.0.12 TCP_MISS/304 609 GET 
http://platform.twitter.com/widgets.js - CLOSEST_PARENT/10.0.33.13 -


Regards,
Andre

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] sourcehash load balance

2016-10-21 Thread André Janna 
I set up a Squid proxy that forwards all requests to 2 parent caches. 
I'm using Squid version 3.5.19.
My goal is that multiple connection from a client to a server should be 
forwarded to the same parent, so that the server see all requests coming 
from the same IP address.


I'm using the following configuration:
cache_peer squid1 parent 3128 0 no-query sourcehash
cache_peer squid2 parent 3128 0 no-query sourcehash
never_direct allow all

Looking at access.log some requests are tagged as CLOSEST_PARENT instead 
of SOURCEHASH_PARENT, so it seams that Squid is not always using source 
hash rule to forward requests to parent caches.

For instance:
1477046954.047   3882 10.11.2.4 TCP_TUNNEL/200 21935 CONNECT 
sso.cisco.com:443 - SOURCEHASH_PARENT/10.0.33.12 -
1477046968.056 21 10.11.2.4 TCP_MISS/200 1012 POST 
http://ocsp.digicert.com/ - CLOSEST_PARENT/10.0.33.13 
application/ocsp-response
1477047782.038 22 10.11.2.4 TCP_MISS/204 307 GET 
http://clients1.google.com/generate_204 - SOURCEHASH_PARENT/10.0.33.12 -
1477047782.045181 10.11.2.4 TCP_MISS/200 745 GET 
http://tags.bluekai.com/site/2964? - CLOSEST_PARENT/10.0.33.13 image/gif


So requests from the same client are not sent to the same parent cache.
How can I force Squid to always use source hash parent selection method?


Regards,
André Janna
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] file descriptors leak

2015-12-15 Thread André Janna 

Em 28/11/2015 22:46, André Janna escreveu:
I took another network trace this time both at Squid and Windows 
client ends.


cache.log:
2015/11/27 11:30:55.610 kid1| SECURITY ALERT: Host header forgery 
detected on local=177.43.198.106:443 remote=192.168.64.4:61802 FD 5465 
flags=33 (local IP does not match any domain IP)


--
network trace at Squid side

client connects
11:30:55.604870 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [S], 
seq 1701554341, win 8192, options [mss 1460,nop,wscale 
8,nop,nop,sackOK], length 0
11:30:55.604992 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags 
[S.], seq 3125704417, ack 1701554342, win 29200, options [mss 
1460,nop,nop,sackOK,nop,wscale 7], length 0
11:30:55.605766 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.], 
ack 1, win 256, length 0


client sends SSL hello
11:30:55.606242 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags 
[P.], seq 1:198, ack 1, win 256, length 197
11:30:55.606306 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 198, win 237, length 0


client OS sends TCP keep-alive packets
11:31:05.607191 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.], 
seq 197:198, ack 1, win 256, length 1
11:31:05.607231 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 198, win 237, options [nop,nop,sack 1 {197:198}], length 0
11:31:15.608966 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.], 
seq 197:198, ack 1, win 256, length 1
11:31:15.609005 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 198, win 237, options [nop,nop,sack 1 {197:198}], length 0
11:31:25.614527 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.], 
seq 197:198, ack 1, win 256, length 1
11:31:25.614589 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 198, win 237, options [nop,nop,sack 1 {197:198}], length 0


client sends FIN
11:31:29.384280 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags 
[F.], seq 198, ack 1, win 256, length 0
11:31:29.421787 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 199, win 237, length 0


client OS sends TCP keep-alive packets
11:31:39.417426 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.], 
seq 198:199, ack 1, win 256, length 1
11:31:39.417489 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 199, win 237, options [nop,nop,sack 1 {198:199}], length 0
11:31:49.425366 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.], 
seq 198:199, ack 1, win 256, length 1
11:31:49.425443 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 199, win 237, options [nop,nop,sack 1 {198:199}], length 0
11:31:59.426153 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.], 
seq 198:199, ack 1, win 256, length 1
11:31:59.426233 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 199, win 237, options [nop,nop,sack 1 {198:199}], length 0
 it continues this way until I powered off Windows client after 
three hours ...



--
network trace at Windows client side

client connects
11:30:34.894242 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [S], 
seq 1701554341, win 8192, options [mss 1460,nop,wscale 
8,nop,nop,sackOK], length 0
11:30:34.898234 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags 
[S.], seq 3125704417, ack 1701554342, win 29200, options [mss 
1460,nop,nop,sackOK,nop,wscale 7], length 0
11:30:34.898298 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.], 
ack 1, win 256, length 0


client sends SSL hello
11:30:34.898712 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags 
[P.], seq 1:198, ack 1, win 256, length 197
11:30:34.899479 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 198, win 237, length 0


client OS sends TCP keep-alive packets
11:30:44.899271 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.], 
seq 197:198, ack 1, win 256, length 1
11:30:44.899986 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 198, win 237, options [nop,nop,sack 1 {197:198}], length 0
11:30:54.900495 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.], 
seq 197:198, ack 1, win 256, length 1
11:30:54.901323 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 198, win 237, options [nop,nop,sack 1 {197:198}], length 0
11:31:04.905731 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.], 
seq 197:198, ack 1, win 256, length 1
11:31:04.906560 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 198, win 237, options [nop,nop,sack 1 {197:198}], length 0


client sends FIN
11:31:08.675299 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags 
[F.], seq 198, ack 1, win 256, length 0
11:31:08.713746 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.], 
ack 199, win 237, length 0


client OS sends TCP keep-alive packets
11:31:18.708086 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.], 
seq 198:199, ack 1, win 256, length 1
11:31:18.708917 IP 177.43.198.106.443 > 192.168.64.

Re: [squid-users] file descriptors leak

2015-11-28 Thread André Janna 

Citando Amos Jeffries :


So, the first place to look is not Squid I think. But why at least 6 of
those ACK packets did not make it back to the client. That needs
resolving first to esure that the TCP level is operating correctly.

Only then if the problem remains looking at Squid, the use of port 443
indicates it is the crypto process is possibly waiting for something and
not closing the port on a 0-byte read(2) operation.



I took another network trace this time both at Squid and Windows client ends.

cache.log:
2015/11/27 11:30:55.610 kid1| SECURITY ALERT: Host header forgery  
detected on local=177.43.198.106:443 remote=192.168.64.4:61802 FD 5465  
flags=33 (local IP does not match any domain IP)


--
network trace at Squid side

client connects
11:30:55.604870 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [S],  
seq 1701554341, win 8192, options [mss 1460,nop,wscale  
8,nop,nop,sackOK], length 0
11:30:55.604992 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags  
[S.], seq 3125704417, ack 1701554342, win 29200, options [mss  
1460,nop,nop,sackOK,nop,wscale 7], length 0
11:30:55.605766 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.],  
ack 1, win 256, length 0


client sends SSL hello
11:30:55.606242 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags  
[P.], seq 1:198, ack 1, win 256, length 197
11:30:55.606306 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.],  
ack 198, win 237, length 0


client OS sends TCP keep-alive packets
11:31:05.607191 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.],  
seq 197:198, ack 1, win 256, length 1
11:31:05.607231 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.],  
ack 198, win 237, options [nop,nop,sack 1 {197:198}], length 0
11:31:15.608966 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.],  
seq 197:198, ack 1, win 256, length 1
11:31:15.609005 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.],  
ack 198, win 237, options [nop,nop,sack 1 {197:198}], length 0
11:31:25.614527 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.],  
seq 197:198, ack 1, win 256, length 1
11:31:25.614589 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.],  
ack 198, win 237, options [nop,nop,sack 1 {197:198}], length 0


client sends FIN
11:31:29.384280 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags  
[F.], seq 198, ack 1, win 256, length 0
11:31:29.421787 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.],  
ack 199, win 237, length 0


client OS sends TCP keep-alive packets
11:31:39.417426 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.],  
seq 198:199, ack 1, win 256, length 1
11:31:39.417489 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.],  
ack 199, win 237, options [nop,nop,sack 1 {198:199}], length 0
11:31:49.425366 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.],  
seq 198:199, ack 1, win 256, length 1
11:31:49.425443 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.],  
ack 199, win 237, options [nop,nop,sack 1 {198:199}], length 0
11:31:59.426153 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.],  
seq 198:199, ack 1, win 256, length 1
11:31:59.426233 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.],  
ack 199, win 237, options [nop,nop,sack 1 {198:199}], length 0
 it continues this way until I powered off Windows client after  
three hours ...



--
network trace at Windows client side

client connects
11:30:34.894242 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [S],  
seq 1701554341, win 8192, options [mss 1460,nop,wscale  
8,nop,nop,sackOK], length 0
11:30:34.898234 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags  
[S.], seq 3125704417, ack 1701554342, win 29200, options [mss  
1460,nop,nop,sackOK,nop,wscale 7], length 0
11:30:34.898298 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.],  
ack 1, win 256, length 0


client sends SSL hello
11:30:34.898712 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags  
[P.], seq 1:198, ack 1, win 256, length 197
11:30:34.899479 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.],  
ack 198, win 237, length 0


client OS sends TCP keep-alive packets
11:30:44.899271 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.],  
seq 197:198, ack 1, win 256, length 1
11:30:44.899986 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.],  
ack 198, win 237, options [nop,nop,sack 1 {197:198}], length 0
11:30:54.900495 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.],  
seq 197:198, ack 1, win 256, length 1
11:30:54.901323 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.],  
ack 198, win 237, options [nop,nop,sack 1 {197:198}], length 0
11:31:04.905731 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags [.],  
seq 197:198, ack 1, win 256, length 1
11:31:04.906560 IP 177.43.198.106.443 > 192.168.64.4.61802: Flags [.],  
ack 198, win 237, options [nop,nop,sack 1 {197:198}], length 0


client sends FIN
11:31:08.675299 IP 192.168.64.4.61802 > 177.43.198.106.443: Flags  
[F.], seq 198, ack 1, win 256, length

Re: [squid-users] file descriptors leak

2015-11-26 Thread André Janna 


Assinatura
Em 24/11/2015 00:54, Amos Jeffries escreveu:
FYI: unless you have a specific need for 3.5 you should be fine with 
the 3.4 squid3 package that is available for Jesse from Debian 
backports. The alternative is going the other way and upgrading right 
to the latest 3.5 snapshot (and/or 4.0 snapshot) to see if it is one 
of the CONNECT or TLS issues we have fixed recently. 

I'm using version 3.5 because 3.4 doesn't have ssl::server_name acl.
Debian package is not built with openssl because of licensing issues so 
I rebuilt Debian testing 3.5 source package on Debian Jessie.
This Squid installation is in production now and cannot be easily 
migrated. But I'll perform another installation for testing in the near 
future.



Neither. So it is time to move away from lsof and start using packet
capture to get a full-body packet trace to find out what exact packets
are happening on at least one affected TCP connection.

If possible identifying one of these connections from its SYN onwards
would be great, but if not then a 20min period of activity on an
existing one might still how more hints.

I did a test using a Windows laptop client with IP address 192.168.64.4, 
connected via wifi.
I browsed a few https sites until triggering Squid "local IP does not 
match any domain IP" error.
This error appeared when I was trying to open Yahoo home page. Browser 
redirected to https://br.yahoo.com/?p=us but page remained blank.
Please note that this error appears randomly: opening the same site in 
another browser tab succeeded.


cache.log:
2015/11/26 13:54:45.471 kid1| SECURITY ALERT: Host header forgery 
detected on local=206.190.56.191:443 remote=192.168.64.4:58887 FD 17244 
flags=33 (local IP does not match any domain IP)


After a couple of minutes this connection disappeared from Windows 
netstat command output. Afterward I powered off Windows laptop.


Tcpdump on Squid box:
13:54:45.410907 IP 192.168.64.4.58887 > 206.190.56.191.443: Flags [S], 
seq 1831867, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], 
length 0
13:54:45.411000 IP 206.190.56.191.443 > 192.168.64.4.58887: Flags [S.], 
seq 3695298276, ack 1831868, win 29200, options [mss 
1460,nop,nop,sackOK,nop,wscale 7], length 0
13:54:45.411630 IP 192.168.64.4.58887 > 206.190.56.191.443: Flags [.], 
ack 1, win 256, length 0
13:54:45.412490 IP 192.168.64.4.58887 > 206.190.56.191.443: Flags [P.], 
seq 1:185, ack 1, win 256, length 184
13:54:45.412573 IP 206.190.56.191.443 > 192.168.64.4.58887: Flags [.], 
ack 185, win 237, length 0
13:54:55.439709 IP 192.168.64.4.58887 > 206.190.56.191.443: Flags [.], 
seq 184:185, ack 1, win 256, length 1
13:54:55.439761 IP 206.190.56.191.443 > 192.168.64.4.58887: Flags [.], 
ack 185, win 237, options [nop,nop,sack 1 {184:185}], length 0
13:55:05.439965 IP 192.168.64.4.58887 > 206.190.56.191.443: Flags [.], 
seq 184:185, ack 1, win 256, length 1
13:55:05.440022 IP 206.190.56.191.443 > 192.168.64.4.58887: Flags [.], 
ack 185, win 237, options [nop,nop,sack 1 {184:185}], length 0
13:55:15.445667 IP 192.168.64.4.58887 > 206.190.56.191.443: Flags [.], 
seq 184:185, ack 1, win 256, length 1
13:55:15.445737 IP 206.190.56.191.443 > 192.168.64.4.58887: Flags [.], 
ack 185, win 237, options [nop,nop,sack 1 {184:185}], length 0
13:55:25.447281 IP 192.168.64.4.58887 > 206.190.56.191.443: Flags [.], 
seq 184:185, ack 1, win 256, length 1
13:55:25.447351 IP 206.190.56.191.443 > 192.168.64.4.58887: Flags [.], 
ack 185, win 237, options [nop,nop,sack 1 {184:185}], length 0
13:55:35.494936 IP 192.168.64.4.58887 > 206.190.56.191.443: Flags [.], 
seq 184:185, ack 1, win 256, length 1
13:55:35.495005 IP 206.190.56.191.443 > 192.168.64.4.58887: Flags [.], 
ack 185, win 237, options [nop,nop,sack 1 {184:185}], length 0
13:55:45.491694 IP 192.168.64.4.58887 > 206.190.56.191.443: Flags [.], 
seq 184:185, ack 1, win 256, length 1
13:55:45.491761 IP 206.190.56.191.443 > 192.168.64.4.58887: Flags [.], 
ack 185, win 237, options [nop,nop,sack 1 {184:185}], length 0
13:55:55.492158 IP 192.168.64.4.58887 > 206.190.56.191.443: Flags [.], 
seq 184:185, ack 1, win 256, length 1
13:55:55.492208 IP 206.190.56.191.443 > 192.168.64.4.58887: Flags [.], 
ack 185, win 237, options [nop,nop,sack 1 {184:185}], length 0
14:01:58.242748 IP 192.168.64.4.58887 > 206.190.56.191.443: Flags [F.], 
seq 185, ack 1, win 256, length 0
14:01:58.279916 IP 206.190.56.191.443 > 192.168.64.4.58887: Flags [.], 
ack 186, win 237, length 0


Netstat output on Squid box:
# date; netstat -tno | grep 192.168.64.4
Thu Nov 26 13:59:40 BRST 2015
tcp6   1  0 172.16.10.22:3126   192.168.64.4:58887 
CLOSE_WAIT  off (0.00/0/0)


And after 2 hours and a half netstat output is still the same:
# date; netstat -tno | grep 192.168.64.4
Thu Nov 26 16:32:37 BRST 2015
tcp6   1  0 172.16.10.22:3126   192.168.64.4:58887 
CLOSE_WAIT  off (0.00/0/0)


Squid is still using the file descriptor.
# date; lsof -n | grep 192.168.64.4
Thu Nov 26 16:33:10 BRST 2015
squid

Re: [squid-users] file descriptors leak

2015-11-23 Thread André Janna 


Assin Em 22/11/2015 16:25, Eliezer Croitoru escreveu:

Hey Andre,

There are couple things to the picture.
It's not only squid that is the "blame".
It depends on what your OS tcp stack settings are.
To verify couple things you can try to use the netstat tool.
run the command "netstat -nto" to see what is the timers status.
You can then see how long will a new connection stay in the 
established state.
It might be the squid settings but if the client is not there it could 
be because of some tcp tunable kernel settings.


Hi Eliezer and Amos,
my kernel is a regular Debian Jessie kernel using the following tcp values.
tcp_keepalive_time: 7200
tcp_keepalive_intvl: 25
tcp_keepalive_probes: 9
tcp_retries1: 3
tcp_retries2: 15
tcp_fin_timeout: 60
So in my understanding the longest timeout is set to 2 hours and a few 
minutes for keepalive connections.


Today I monitored file descriptors 23 and 24 on my box during 5 hours 
and lsof always showed:
squid  6574   proxy   23u IPv6 5320944  
0t0TCP 172.16.10.22:3126->192.168.90.35:34571 (CLOSE_WAIT)
squid  6574   proxy   24u IPv6 5327276  
0t0TCP 172.16.10.22:3126->192.168.89.236:49435 (ESTABLISHED)

while netstat always showed:
tcp6   1  0 172.16.10.22:3126 192.168.90.35:34571 
CLOSE_WAIT  6574/(squid-1)   off (0.00/0/0)
tcp6   0  0 172.16.10.22:3126 192.168.89.236:49435
ESTABLISHED 6574/(squid-1)   off (0.00/0/0)


The "off" flag in netstat output tells that for these sockets keepalive 
and retransmission timers are disabled.
Right now netstat shows 15,568 connections on squid port 3126 and only 
107 have timer set to a value other than "off".


I read that connections that are in CLOSE_WAIT state don't have any tcp 
timeout, it's Squid that must close the socket.


 About the connections in ESTABLISHED state, I monitored the connection 
to mobile device 192.168.89.236 using "tcpdump -i eth2 -n host 
192.168.89.236" during 2 hours and a half.

Tcpdump didn't record any packet and netstat is still displaying:
tcp6   1  0 172.16.10.22:3126 192.168.90.35:34571 
CLOSE_WAIT  6574/(squid-1)   off (0.00/0/0)
tcp6   0  0 172.16.10.22:3126 192.168.89.236:49435
ESTABLISHED 6574/(squid-1)   off (0.00/0/0)


So unfortunately I still don't understand why Squid or the kernel don't 
close these sockets.



Regards,
  André

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] file descriptors leak

2015-11-22 Thread André Janna 

 Citando André Janna:


Squid is still using file descriptors 12 and 14 (and a lot of others)
for the same connections as yesterday, although the mobile devices it
was connected to have not been online in our network for at least 15
hours.
 


Update: Squid released file descriptors after about 24 hours, I suppose
because expired client_lifetime which is set to 1 day default value.

Regards,
  André
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] file descriptors leak

2015-11-22 Thread André Janna 

 Citando Amos Jeffries :


CONNECT requests with tunnels can be particularly long lived, mobiles
and their applications stay active for weeks on end with few outward
signs of what is happening inside the encrypted tunnel. The only way to
be sure the connection is finished with is when one of the client or
server remote endpoints closes it.

* The port 55815 connection _was_ closed sometimes within 15 mintes of
the lsof being run.

* The port 52288 connection is still being used.

Given the timespan between those messages and the lsof, it is also
possible they were closed and reopened in between. If you have a lot of
ports in active use, then re-use of closed ones becomes more likely.
Though I suspect it is just persistence doing what it is designed to do.
You will need a more detailed trace of the entire time period to know.


11 more hours have passed since my last "lsof" and cause it's Sunday I'm
sure that no device have been connected to 192.168.x.x network at least for
15 hours.
Right now lsof output is the same as 11 hours before:

squid 32490    proxy   12u
IPv6    4065613  0t0    TCP
172.16.10.22:3126->192.168.93.113:55815 (CLOSE_WAIT)
squid 32490    proxy   14u
IPv6    4097822  0t0    TCP
172.16.10.22:3126->192.168.90.207:52288 (ESTABLISHED)
...

Squid is still using file descriptors 12 and 14 (and a lot of others) for
the same connections as yesterday, although the mobile devices it was
connected to have not been online in our network for at least 15 hours.
Is it by design?
Raising file descriptors limit is the only solution?
The maximum number of file descriptors in my installation is set to 65535.
Is there any drawback to increasing this number let's say by a factor of
ten?

Regards,
  André
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] file descriptors leak

2015-11-21 Thread André Janna 

I'm running Squid 3.5.10 on Debian Jessie and after some hours of execution
it runs out of file descriptors.
Squid is listening on port 3125, 3126 and 3127.
Port 3126 is used for intercepting, via iptables redirect, https
connections mostly from mobile devices like smartphones. On this port is
active ssl-bump but I'm not decrypting https traffic, only "peek" to get
https server host name.
Port 3125 is used for intercepting http connections of the same mobile
devices whose https traffic is intercepted on port 3126.
Port 3127 is used for clients configured to use a proxy.
Leaked file descriptors are all related to connection on port 3126 (https
intercept ssl-bump).
A sample output of lsof command gives:

    squid 32490    proxy   12u
IPv6    4065613   0t0    TCP
172.16.10.22:3126->192.168.93.113:55815 (CLOSE_WAIT)
    squid 32490    proxy   14u
IPv6    4097822   0t0    TCP
172.16.10.22:3126->192.168.90.207:52288 (ESTABLISHED)
    ...

where 172.16.10.22 is an IP address of my Squid installation and
192.168.x.x are mobile devices.
Is seems that this condition is triggered by "local IP does not match any
domain IP" error logged by Squid in cache.log, but I'm not sure if all
stuck connections are caused by this kind of error.
For the 2 connections of the sample above the related cache.log errors are:

    2015/11/21 12:57:51.229 kid1| SECURITY ALERT: Host header forgery
detected on local=23.0.163.57:443 remote=192.168.93.113:55815 FD 12
flags=33 (local IP does not match any domain IP)
    2015/11/21 13:59:44.230 kid1| SECURITY ALERT: Host header forgery
detected on local=198.144.127.162:443 remote=192.168.90.207:52288 FD 14
flags=33 (local IP does not match any domain IP)

"lsof" sample output was taken more that 10 hours after Squid logged these
errors and it shows that Squid is still holding connections open, using a
lot of file descriptors.

Regards,
    André

--- my squid.conf ---
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl CONNECT method CONNECT
acl squid-internal-static url_regex
^http://nat-academico:3127/squid-internal-static/
acl e2guardian localport 3127
follow_x_forwarded_for allow localhost
http_access allow squid-internal-static
http_access allow localhost manager
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access allow localhost
http_access allow localnet e2guardian
include /etc/squid/transparent-blacklist.conf
include /etc/squid/transparent-whitelist.conf
http_access allow transparent-whitelist-http
http_access deny transparent-blacklist-http
http_access allow localnet
http_access deny all
http_port 3127
http_port 3125 intercept
https_port 3126 cert=/etc/ssl/certs/nat-academico.crt
key=/etc/ssl/private/services.key intercept ssl-bump
acl step1 at_step SslBump1
ssl_bump peek step1 all
ssl_bump splice transparent-whitelist-https
ssl_bump terminate transparent-blacklist-https
cache_dir ufs /var/spool/squid 1 16 256
coredump_dir /var/spool/squid
refresh_pattern ^ftp:   1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern .   0   20%
4320
dns_v4_first on
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users