Re: [squid-users] Squid with Kerberos auth
Amos, so what could be causing so much load on cpu? When I run top i can see there's no swap and the squid_kerb_auth is the process that causes this high CPU usage. Is there any way i can check this helper? Here, down low, is my squid.conf. I hope you point some light because i don't believe squid runs in so such poor speed. auth_param negotiate program /etc/squid/squid-3.1.16/helpers/negotiate_auth/squid_kerb_auth/squid_kerb_auth -s HTTP/trotsky.redecamara.camara.gov.br auth_param negotiate children 4 auth_param negotiate keep_alive on # ACLs externas para buscar grupo baseado em Kerberos. external_acl_type squid_kerb_ldap ttl=3600 negative_ttl=3600 %LOGIN /etc/squid/squid-3.1.16/squid_kerb_ldap/squid_kerb_ldap -S californio.redecamara.camara.gov.br -g inter...@redecamara.camara.gov.br visible_hostname trotsky.redecamara.camara.gov.br dns_nameservers 10.1.3.5 hierarchy_stoplist cgi-bin ? #acl liberados dstdomain "/etc/squid/liberados.txt" acl ldap_group_check external squid_kerb_ldap acl AUTENTICADO proxy_auth REQUIRED http_access allow ldap_group_check # Recommended minimum configuration: # acl manager proto cache_object acl localhost src 127.0.0.1/32 ::1 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1 #acl teste external Internet teste # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl SSL_ports port 443 acl SSL_ports port 1863 acl SSL_ports port 563 acl SSL_ports port 465 acl SSL_ports port 995 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 563 # https acl Safe_ports port 465 # https acl Safe_ports port 995 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access deny manager # Deny requests to certain unsafe ports http_access deny !Safe_ports # Deny CONNECT to other than secure SSL ports http_access deny CONNECT !SSL_ports # We strongly recommend the following be uncommented to protect innocent # web applications running on the proxy server who think the only # one who can access services on "localhost" is a local user #http_access deny to_localhost # # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS acl INTRANETdst 10.0.0.0/8 acl EXE urlpath_regex -i \.exe$ acl ZIP urlpath_regex -i \.zip$ acl ARJ urlpath_regex -i \.arj$ acl MP3 urlpath_regex -i \.mp3$ acl RAR urlpath_regex -i \.rar$ acl GZ urlpath_regex -i \.gz$ acl ISO1 urlpath_regex -i \.iso\?+$ acl EXE1 urlpath_regex -i \.exe\?+$ acl ZIP1 urlpath_regex -i \.zip\?+$ acl ARJ1 urlpath_regex -i \.arj\?+$ acl MP31 urlpath_regex -i \.mp3\?+$ acl RAR1 urlpath_regex -i \.rar\?+$ acl GZ1 urlpath_regex -i \.gz\?+$ http_access allow INTRANET http_access allow localhost # And finally deny all other access to this proxy http_access deny all # Squid normally listens to port 3128 http_port 3128 cache_store_log none maximum_object_size 16384 KB minimum_object_size 0 KB maximum_object_size_in_memory 50 KB cache_swap_low 95 cache_swap_high 98 ipcache_size6000 ipcache_low 90 ipcache_high 92 fqdncache_size 6000 memory_replacement_policy heap GDSF cache_replacement_policy heap LFUDA half_closed_clients off memory_pools off cache_dir diskd /cache/squid 1 64 256 Q1=64 Q2=72 cache_mem 2048 MB # Leave coredumps in the first cache dir coredump_dir /var/spool/squid max_filedescriptors 1024 # Add any of your own refresh_pattern entries above these. refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 Regards, Wladner 2011/12/20 Amos Jeffries : > On 21/12/2011 3:03 a.m., Wladner Klimach wrote: >> >> But the problem is that i'm not running IPv6 in my network. That's why > > "Welcome to your IPv6 enabled transit network. Whether you like it, or not." > - Rob Issac, 2008. > (http://www.ausnog.net/files/ausnog-03/presentations/ausnog03-ward-IPv6_enabled_network.pdf) > > Try with -n parameter to lsof. You might get a surprise. > > The TCP "hybrid" stack can use IPv6 sockets for IPv4 traffic, this may also > be what you are seeing. Squid-3.1+ will detect stack types and use this > optimization for receiving ports if it can. > > >> I've asked if this could be a problem. And the cpu usage hiting 99% >> with only one user? Does it look like hardware limitation? When i'm >> not using authentication, the cpu usage doesn't hit 50%. > > > Unlikely with one user. > > All Squid does for auth is take the
Re: [squid-users] Squid with Kerberos auth
On 21/12/2011 3:03 a.m., Wladner Klimach wrote: But the problem is that i'm not running IPv6 in my network. That's why "Welcome to your IPv6 enabled transit network. Whether you like it, or not." - Rob Issac, 2008. (http://www.ausnog.net/files/ausnog-03/presentations/ausnog03-ward-IPv6_enabled_network.pdf) Try with -n parameter to lsof. You might get a surprise. The TCP "hybrid" stack can use IPv6 sockets for IPv4 traffic, this may also be what you are seeing. Squid-3.1+ will detect stack types and use this optimization for receiving ports if it can. I've asked if this could be a problem. And the cpu usage hiting 99% with only one user? Does it look like hardware limitation? When i'm not using authentication, the cpu usage doesn't hit 50%. Unlikely with one user. All Squid does for auth is take the tokens out of HTTP headers and relay it to the auth backend. Then add the backends reply token to the HTTP response for the client. Very minimal CPU operations in Squid, unknown amount in the backend. Maybe (max) 32KB of token copied each way, plus the HTTP bits. Amos
Re: [squid-users] Squid with Kerberos auth
On 20/12/2011 7:40 a.m., Wladner Klimach wrote: Look at this: Every 2.0s: lsof -i :3128 Mon Dec 19 16:38:22 2011 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME squid 20367 squid 12u IPv6 2474452 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4225 (ESTABLISHED) squid 20367 squid 18u IPv6 2473286 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4202 (ESTABLISHED) squid 20367 squid 22u IPv6 2474474 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4229 (ESTABLISHED) squid 20367 squid 24u IPv6 2473304 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4204 (ESTABLISHED) squid 20367 squid 28u IPv6 2473756 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4210 (ESTABLISHED) squid 20367 squid 34u IPv6 2474462 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4227 (ESTABLISHED) squid 20367 squid 38u IPv6 2474457 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4226 (ESTABLISHED) squid 20367 squid 42u IPv6 2474467 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4228 (ESTABLISHED) squid 20367 squid 44u IPv6 2474477 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4230 (ESTABLISHED) squid 20367 squid 156u IPv6 2472223 0t0 TCP *:squid (LISTEN) Is only has IPV6 conection types. Is this a problem or point a possible bottleneck ? Problem? no. Possible bottleneck? depends if there is a slow IPv6 connectivity between Squid and that remote machine (ie a tunnel with wrapping overheads). ~75% of networks have faster IPv6 connectivity than IPv4 connectivity. Amos
Re: [squid-users] Squid with Kerberos auth
Look at this: Every 2.0s: lsof -i :3128 Mon Dec 19 16:38:22 2011 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME squid 20367 squid 12u IPv6 2474452 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4225 (ESTABLISHED) squid 20367 squid 18u IPv6 2473286 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4202 (ESTABLISHED) squid 20367 squid 22u IPv6 2474474 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4229 (ESTABLISHED) squid 20367 squid 24u IPv6 2473304 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4204 (ESTABLISHED) squid 20367 squid 28u IPv6 2473756 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4210 (ESTABLISHED) squid 20367 squid 34u IPv6 2474462 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4227 (ESTABLISHED) squid 20367 squid 38u IPv6 2474457 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4226 (ESTABLISHED) squid 20367 squid 42u IPv6 2474467 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4228 (ESTABLISHED) squid 20367 squid 44u IPv6 2474477 0t0 TCP trotsky.redecamara.camara.gov.br:squid->cainf-269642.redecamara.camara.gov.br:4230 (ESTABLISHED) squid 20367 squid 156u IPv6 2472223 0t0 TCP *:squid (LISTEN) Is only has IPV6 conection types. Is this a problem or point a possible bottleneck ? 2011/12/19 Wladner Klimach : > Amos, > > how can I check if the keep alive is really working? Another thing, is > it normal behavior the cpu usage being hit 98% usage during one domain > download page? Could it be just hardware limitation? If so, this is a > very awkward situation because there's no severe load upon squid cause > only I using it. > > Regards, > > Wladner > > 2011/12/14 Amos Jeffries : >> On Wed, 14 Dec 2011 13:22:38 -0200, Wladner Klimach wrote: >>> >>> Hello, >>> >>> i'm running squid with kerberos authentication. The problem is that >>> it's runing too slow. Looks like squid is negotiating with AD every >>> URL it tries to get. Anyone could point me a way out? >> >> >> A few things: >> >> * Double-check that you have connection persistence (keep-alive) operating >> on all connections (both client and server). >> >> * Ensure that your squid is as recent as you can use, we have had ongoing >> small fixes to improve persistence across all releases this past year. >> >> * Check that the auth packets are not failing over into NTLM or older >> protocols in apps which are supposed to be on Kerberos. >> >> * Maybe also check that DNS lookups Kerberos depends on for DC location are >> responding fast with reasonable TTL. >> >> Amos >>
Re: [squid-users] Squid with Kerberos auth
Amos, how can I check if the keep alive is really working? Another thing, is it normal behavior the cpu usage being hit 98% usage during one domain download page? Could it be just hardware limitation? If so, this is a very awkward situation because there's no severe load upon squid cause only I using it. Regards, Wladner 2011/12/14 Amos Jeffries : > On Wed, 14 Dec 2011 13:22:38 -0200, Wladner Klimach wrote: >> >> Hello, >> >> i'm running squid with kerberos authentication. The problem is that >> it's runing too slow. Looks like squid is negotiating with AD every >> URL it tries to get. Anyone could point me a way out? > > > A few things: > > * Double-check that you have connection persistence (keep-alive) operating > on all connections (both client and server). > > * Ensure that your squid is as recent as you can use, we have had ongoing > small fixes to improve persistence across all releases this past year. > > * Check that the auth packets are not failing over into NTLM or older > protocols in apps which are supposed to be on Kerberos. > > * Maybe also check that DNS lookups Kerberos depends on for DC location are > responding fast with reasonable TTL. > > Amos >
Re: [squid-users] Squid with Kerberos auth
On Wed, 14 Dec 2011 13:22:38 -0200, Wladner Klimach wrote: Hello, i'm running squid with kerberos authentication. The problem is that it's runing too slow. Looks like squid is negotiating with AD every URL it tries to get. Anyone could point me a way out? A few things: * Double-check that you have connection persistence (keep-alive) operating on all connections (both client and server). * Ensure that your squid is as recent as you can use, we have had ongoing small fixes to improve persistence across all releases this past year. * Check that the auth packets are not failing over into NTLM or older protocols in apps which are supposed to be on Kerberos. * Maybe also check that DNS lookups Kerberos depends on for DC location are responding fast with reasonable TTL. Amos
[squid-users] Squid with Kerberos auth
Hello, i'm running squid with kerberos authentication. The problem is that it's runing too slow. Looks like squid is negotiating with AD every URL it tries to get. Anyone could point me a way out? Best regards, Wladner
