Guido Marino Lorenzutti wrote:
Hi people: I have a squid using ntlm to authenticate the users. I also use a external acl but im running out of ideas to make it run faster.

If I disable the ntlm everything works very well, and the cache hits increses a lot.

I found that the squids asks a lot for username and passwords to the winbind, and the winbind asks everytime to my pdc. This generates a lot of traffic between them and a high load on the pdc.

Every hit on any page, squid ask to the winbind for the username and password. Is this the expected behavior? Is there any way to reduce (caching maybe?) this? I didn't find a solution in the winbind, to stop asking the credentials to the pdc.

I have a terminal server enviroment, so where you see 69 clients the are in fact more than 500 users.

This is my output of squidclient mgr:info

Squid Object Cache: Version 2.6.STABLE5
Start Time:     Fri, 05 Feb 2010 07:21:21 GMT
Current Time:   Sat, 20 Feb 2010 03:01:08 GMT
Connection information for squid:
        Number of clients accessing cache:      69
        Number of HTTP requests received:       11790881
        Number of ICP messages received:        0
        Number of ICP messages sent:    0
        Number of queued ICP replies:   0
        Number of HTCP messages received:       0
        Number of HTCP messages sent:   0
        Request failure ratio:   0.00
        Average HTTP requests per minute since start:   552.5
        Average ICP messages per minute since start:    0.0
        Select loop called: 154266917 times, 8.300 ms avg
Cache information for squid:
        Request Hit Ratios:     5min: 50.5%, 60min: 18.3%
        Byte Hit Ratios:        5min: 14.1%, 60min: 26.3%
        Request Memory Hit Ratios:      5min: 0.0%, 60min: 10.0%
        Request Disk Hit Ratios:        5min: 19.7%, 60min: 21.9%
        Storage Swap size:      7833612 KB
        Storage Mem size:       409452 KB
        Mean Object Size:       19.26 KB
        Requests given to unlinkd:      0
Median Service Times (seconds)  5 min    60 min:
        HTTP Requests (All):   0.00919  0.03066
        Cache Misses:          0.35832  0.44492
        Cache Hits:            0.01164  0.01847
        Near Hits:             0.33943  0.37825
        Not-Modified Replies:  0.00286  0.00405
        DNS Lookups:           0.09117  0.10906
        ICP Queries:           0.00000  0.00000
Resource usage for squid:
        UP Time:        1280387.834 seconds
        CPU Time:       5238.387 seconds
        CPU Usage:      0.41%
        CPU Usage, 5 minute avg:        0.07%
        CPU Usage, 60 minute avg:       0.05%
        Process Data Segment Size via sbrk(): 561092 KB
        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 4
Memory usage for squid via mallinfo():
        Total space in arena:  561092 KB
        Ordinary blocks:       555876 KB  13964 blks
        Small blocks:               0 KB      0 blks
        Holding blocks:          1744 KB      4 blks
        Free Small blocks:          0 KB
        Free Ordinary blocks:    5215 KB
        Total in use:          557620 KB 99%
        Total free:              5215 KB 1%
        Total size:            562836 KB
Memory accounted for:
        Total accounted:       511637 KB
        memPoolAlloc calls: 1443436295
        memPoolFree calls: 1441310223
File descriptor usage for squid:
        Maximum number of file descriptors:   1024
        Largest file desc currently in use:    268
        Number of file desc currently in use:  261
        Files queued for open:                   0
        Available number of file descriptors:  763
        Reserved number of file descriptors:   100
        Store Disk files open:                   2
        IO loop method:                     epoll
Internal Data Structures:
        407686 StoreEntries
         34175 StoreEntries with MemObjects
         34170 Hot Object Cache Items
        406635 on-disk objects


This is the output of squidclient mgr:ntlmauthenticator

(warning: the avg service time is with NO users, when everyone is using it the avg service time peeks the 1000 msec. YES 1K msec).

NTLM Authenticator Statistics:
program: /usr/bin/ntlm_auth
number running: 200 of 200
requests sent: 2500498
replies received: 2500498
queue length: 0
avg service time: 19.24 msec

      #      FD     PID  # Requests     Flags      Time  Offset Request
      1      12   17113      168619               0.046       0 (none)
      2      13   17114       62644               0.055       0 (none)
      3      14   17118       31007               0.076       0 (none)
      4      15   17120       15188               0.094       0 (none)
      5      16   17121        5759               0.093       0 (none)
      6      17   17122        2845               0.071       0 (none)
      7      18   17124        1572               0.524       0 (none)
      8      19   17125         891               0.533       0 (none)
      9      21   17130         486               0.584       0 (none)
     10      22   17131         302               0.647       0 (none)
     11      23   17132         194               0.741       0 (none)
     12      24   17135         127               0.818       0 (none)
     13      25   17137          84               0.756       0 (none)
     14      26   17138          56               0.898       0 (none)
     15      27   17143          46               0.954       0 (none)
     16      28   17149          36               1.002       0 (none)
     17      29   17155          24               1.125       0 (none)
     18      30   17161          22               1.094       0 (none)
     19      31   17162          16               1.252       0 (none)
     20      32   17165          10               5.137       0 (none)
     21      33   17167           8               4.807       0 (none)
     22      34   17168           4               1.470       0 (none)
     23      35   17169           4               1.522       0 (none)
     24      36   17170           2               1.185       0 (none)
     25      37   17171           2               0.613       0 (none)
     26      38   17172           2               0.839       0 (none)
     27      39   17173           0               0.000       0 (none)


Any ideas in how to improve this scenario?

This is the squid.conf

hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
http_port 3128
#debug_options ALL,1 33,2
log_fqdn off
cache_store_log none
useragent_log none
cache_log /var/log/squid/cache_log.log
access_log /var/log/squid/access.log
error_directory /usr/share/squid/errors/Spanish
emulate_httpd_log on

offline_mode off
strip_query_terms on
httpd_suppress_version_string on

auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp --domain=MYDOMAIN
auth_param ntlm children 200
auth_param ntlm keep_alive on
authenticate_ttl 60 seconds
authenticate_ip_ttl 2 minutes
authenticate_cache_garbage_interval 10 seconds

Seems a bit extreme to be running the garbage collection 10 seconds. It happens as needed on top of this.

The defaults are measured in hours and user browsing times are usually longer than minutes.

auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic --domain=MYDOMAIN
auth_param basic children 10
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours

external_acl_type ldap_group ttl=1200 children=25 %LOGIN /usr/lib/squid/squid_ldap_group -b "GROUPDN" -f "MYFILTER" -h LDAPSERVER -v3 -S -P

negative_ttl 5 minutes

This is not really a good idea.
It will extend the period of outage for every service failure and may hose the network access to a website for 5 minutes following a single client page error.

positive_dns_ttl 5 hours
negative_dns_ttl 1 minutes

Please don't play with DNS TTLs unless you know 100% how they will affect things.

half_closed_clients off
connect_timeout 3 seconds
cache_dir aufs /var/spool/squid 9000 16 256
cache_swap_low 85
cache_swap_high 95
maximum_object_size 81920 KB
maximum_object_size_in_memory 300 KB
cache_mem 400 MB
fqdncache_size 6144
cache_replacement_policy lfuda
memory_replacement_policy lru
pipeline_prefetch off
client_persistent_connections off
server_persistent_connections off

Persistent connections are REQUIRED for NTLM and related connection-based auth to be used efficiently.

NTLM auth against the proxy requires persistent client connections, pass-thru to web servers requires both and the connection pinning feature as well.

visible_hostname myproxy.mydomain

hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY

acl all src all
acl lan_10_8 src 10.8.0.0/255.255.0.0

acl lan_10_8 src 10.8.0.0/16


acl webservers dst 10.8.50.220/255.255.255.255 10.8.50.221/255.255.255.255 10.8.50.222/255.255.255.255 10.8.50.223/255.255.255.255

acl webservers dst 10.8.50.220 10.8.50.221 10.8.50.222 10.8.50.223



acl nomsnurl dstdomain "/etc/squid/nomsn"

acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255

acl localhost src 127.0.0.1

acl SSL_ports port 443 563 1863 6667 4430
acl Safe_ports port 80          # http
acl Safe_ports port 443 563     # https, snews

acl auth proxy_auth REQUIRED
acl noinet external ldap_group noinet
acl linuxadmin external ldap_group linuxadmin
acl nomsn external ldap_group nomsn
acl dummy src 0.0.0.0/0.0.0.0

acl dummy src all


Amos
--
Please be using
  Current Stable Squid 2.7.STABLE8 or 3.0.STABLE24
  Current Beta Squid 3.1.0.16

Reply via email to