[squid-users] Squid 3.2.7 dies: icap_retry deny all
Hi My squid dies almost every day with the following entries in the cache.log I am using Centos 6.3 squid-3.2.7-1.el6.x86_64 2013/03/01 01:55:36 kid1| Starting Squid Cache version 3.2.7 for x86_64-unknown-linux-gnu... 2013/03/01 01:55:36 kid1| Process ID 2937 2013/03/01 01:55:36 kid1| Process Roles: worker 2013/03/01 01:55:36 kid1| With 65535 file descriptors available 2013/03/01 01:55:36 kid1| Initializing IP Cache... 2013/03/01 01:55:36 kid1| DNS Socket created at 0.0.0.0, FD 7 2013/03/01 01:55:36 kid1| Adding nameserver 160.85.192.100 from squid.conf 2013/03/01 01:55:36 kid1| Logfile: opening log daemon:/var/log/squid/access.log 2013/03/01 01:55:36 kid1| Logfile Daemon: opening log /var/log/squid/access.log 2013/03/01 01:55:36 kid1| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec 2013/03/01 01:55:36 kid1| Store logging disabled 2013/03/01 01:55:36 kid1| Swap maxSize 2560 + 3145728 KB, estimated 2211209 objects 2013/03/01 01:55:36 kid1| Target number of buckets: 110560 2013/03/01 01:55:36 kid1| Using 131072 Store buckets 2013/03/01 01:55:36 kid1| Max Mem size: 3145728 KB 2013/03/01 01:55:36 kid1| Max Swap size: 2560 KB 2013/03/01 01:55:36 kid1| Rebuilding storage in /var/cache/squid (clean log) 2013/03/01 01:55:36 kid1| Using Least Load store dir selection 2013/03/01 01:55:36 kid1| Set Current Directory to /var/spool/squid 2013/03/01 01:55:36 kid1| Loaded Icons. 2013/03/01 01:55:36 kid1| HTCP Disabled. 2013/03/01 01:55:36 kid1| Squid plugin modules loaded: 0 2013/03/01 01:55:36 kid1| Adaptation support is off. 2013/03/01 01:55:36 kid1| Accepting HTTP Socket connections at local=160.85.104.14:8080 remote=[::] FD 12 flags=9 2013/03/01 01:55:36 kid1| Store rebuilding is 0.90% complete 2013/03/01 01:55:38 kid1| Done reading /var/cache/squid swaplog (446056 entries) 2013/03/01 01:55:38 kid1| Finished rebuilding storage from disk. 2013/03/01 01:55:38 kid1|446054 Entries scanned 2013/03/01 01:55:38 kid1| 2 Invalid entries. 2013/03/01 01:55:38 kid1| 0 With invalid flags. 2013/03/01 01:55:38 kid1|446037 Objects loaded. 2013/03/01 01:55:38 kid1| 0 Objects expired. 2013/03/01 01:55:38 kid1| 0 Objects cancelled. 2013/03/01 01:55:38 kid1| 6 Duplicate URLs purged. 2013/03/01 01:55:38 kid1|11 Swapfile clashes avoided. 2013/03/01 01:55:38 kid1| Took 1.76 seconds (253274.55 objects/sec). 2013/03/01 01:55:38 kid1| Beginning Validation Procedure 2013/03/01 01:55:38 kid1| 262144 Entries Validated so far. 2013/03/01 01:55:38 kid1| Completed Validation Procedure 2013/03/01 01:55:38 kid1| Validated 446034 Entries 2013/03/01 01:55:38 kid1| store_swap_size = 10760564.00 KB 2013/03/01 01:55:38 kid1| storeLateRelease: released 0 objects 2013/03/01 01:56:20 kid1| Closing HTTP port 160.85.104.14:8080 2013/03/01 01:56:20 kid1| storeDirWriteCleanLogs: Starting... 2013/03/01 01:56:20 kid1| 65536 entries written so far. 2013/03/01 01:56:20 kid1|131072 entries written so far. 2013/03/01 01:56:20 kid1|196608 entries written so far. 2013/03/01 01:56:20 kid1|262144 entries written so far. 2013/03/01 01:56:20 kid1|327680 entries written so far. 2013/03/01 01:56:20 kid1|393216 entries written so far. 2013/03/01 01:56:20 kid1| Finished. Wrote 446088 entries. 2013/03/01 01:56:20 kid1| Took 0.06 seconds (7136608.70 entries/sec). FATAL: Bungled (null) line 8: icap_retry deny all Squid Cache (Version 3.2.7): Terminated abnormally. CPU Usage: 1.930 seconds = 1.050 user + 0.880 sys Maximum Resident Size: 449232 KB Page faults with physical i/o: 22 Memory usage for squid via mallinfo(): total space in arena: 95588 KB Ordinary blocks:95304 KB 37 blks Small blocks: 0 KB 1 blks Holding blocks: 38936 KB 9 blks Free Small blocks: 0 KB Free Ordinary blocks: 283 KB Total in use: 134240 KB 140% Total free: 283 KB 0% 2013/03/01 01:56:23 kid1| Starting Squid Cache version 3.2.7 for x86_64-unknown-linux-gnu... 2013/03/01 01:56:23 kid1| Process ID 2957 2013/03/01 01:56:23 kid1| Process Roles: worker 2013/03/01 01:56:23 kid1| With 65535 file descriptors available 2013/03/01 01:56:23 kid1| Initializing IP Cache... 2013/03/01 01:56:23 kid1| DNS Socket created at 0.0.0.0, FD 7 2013/03/01 01:56:23 kid1| Adding nameserver 160.85.192.100 from squid.conf 2013/03/01 01:56:23 kid1| Logfile: opening log daemon:/var/log/squid/access.log 2013/03/01 01:56:23 kid1| Logfile Daemon: opening log /var/log/squid/access.log 2013/03/01 01:56:23 kid1| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec 2013/03/01 01:56:23 kid1| Store logging disabled 2013/03/01 01:56:23 kid1| Swap maxSize 2560 + 3145728 KB, estimated 2211209 objects 2013/03/01 01:56:23 kid1| Target number of buckets: 110560 2013/03/01 01:56:23 kid1| Using 131072 Store buckets 2013/03/01 01:56:23 kid1| Max Mem size: 3145728 KB
[squid-users] squid running out of filedescriptors
Hi Today squid was suddenly running at 100% CPU and a lot of running out of filedescriptors messages in the cache.log. But if I look with squidclient it only had 989 of 65k filedescriptors open. Is there something else I need to look at? I am using squid-3.2.6 on Centos 6.3 11.Command: top PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 11338 squid 20 0 647m 531m 6488 R 99.5 6.8 4:05.12 squid 12.Command: squidclient -p 8080 -h 160.85.104.14 mgr:info | grep file Maximum number of file descriptors: 65535 Largest file desc currently in use: 1016 Number of file desc currently in use: 989 Available number of file descriptors: 64546 Reserved number of file descriptors: 64546 Store Disk files open: 7 Command: tail -100 cache.log 2013/02/20 09:38:11 kid1| WARNING! Your cache is running out of filedescriptors 2013/02/20 09:38:27 kid1| WARNING! Your cache is running out of filedescriptors 2013/02/20 09:38:43 kid1| WARNING! Your cache is running out of filedescriptors 2013/02/20 09:38:47 kid1| comm_open: socket failure: (24) Too many open files 2013/02/20 09:38:47 kid1| Reserved FD adjusted from 64542 to 64546 due to failures 2013/02/20 09:38:47 kid1| comm_open: socket failure: (24) Too many open files 2013/02/20 09:38:47 kid1| Attempt to open socket for EUI retrieval failed: (24) Too many open files noch 6 weitere solche Eintraege 2013/02/20 09:38:47 kid1| Attempt to open socket for EUI retrieval failed: (24) Too many open files 2013/02/20 09:38:47 kid1| comm_open: socket failure: (24) Too many open files 2013/02/20 09:38:47 kid1| comm_open: socket failure: (24) Too many open files 2013/02/20 09:38:47 kid1| comm_open: socket failure: (24) Too many open files 2013/02/20 09:38:47 kid1| DiskThreadsDiskFile::openDone: (24) Too many open files 2013/02/20 09:38:47 kid1| /var/cache/squid/05/09/0005097C 2013/02/20 09:38:59 kid1| WARNING! Your cache is running out of filedescriptors noch 10 weitere solche Eintraege 2013/02/20 09:41:55 kid1| WARNING! Your cache is running out of filedescriptors 2013/02/20 09:42:02 kid1| local=160.85.104.14:49011 remote=212.35.56.41:443 FD 261 flags=1: read/write failure: (110) Connection timed out 2013/02/20 09:42:11 kid1| WARNING! Your cache is running out of filedescriptors noch 5 weitere solche Eintraege 2013/02/20 09:43:47 kid1| WARNING! Your cache is running out of filedescriptors 2013/02/20 09:43:47 kid1| Failed to select source for 'http://www.fac/' 2013/02/20 09:43:47 kid1| always_direct = 0 2013/02/20 09:43:47 kid1|never_direct = 0 2013/02/20 09:43:47 kid1|timedout = 0 2013/02/20 09:43:51 kid1| Failed to select source for 'http://www.fac/favicon.ico' 2013/02/20 09:43:51 kid1| always_direct = 0 2013/02/20 09:43:51 kid1|never_direct = 0 2013/02/20 09:43:51 kid1|timedout = 0 2013/02/20 09:43:58 kid1| Failed to select source for 'http://nonexistent.yontoo.com/' 2013/02/20 09:43:58 kid1| always_direct = 0 2013/02/20 09:43:58 kid1|never_direct = 0 2013/02/20 09:43:58 kid1|timedout = 0 2013/02/20 09:44:03 kid1| WARNING! Your cache is running out of filedescriptors 2013/02/20 09:44:19 kid1| WARNING! Your cache is running out of filedescriptors 2013/02/20 09:44:35 kid1| WARNING! Your cache is running out of filedescriptors 2013/02/20 09:44:51 kid1| WARNING! Your cache is running out of filedescriptors
AW: AW: AW: AW: AW: [squid-users] Re: dns_v4_first on ignored?
That is what I guessed as well. But we can not control their DNS and the solution so far was not to check for records. It is silly for one domain but it is a quite important one that is used a lot. Not sure if there is any alternatives? I thought that squid 3.2 is doing parallel lookups to and A records? -Ursprüngliche Nachricht- Von: Amos Jeffries [mailto:squ...@treenet.co.nz] Gesendet: Dienstag, 12. Februar 2013 10:54 An: squid-users@squid-cache.org Betreff: Re: AW: AW: AW: AW: [squid-users] Re: dns_v4_first on ignored? On 12/02/2013 8:41 p.m., Sandrini Christian (xsnd) wrote: Hi I have now enabled ipv6 3: eth1: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 link/ether 00:50:56:a6:07:27 brd ff:ff:ff:ff:ff:ff inet 160.85.104.14/24 brd 160.85.104.255 scope global eth1 inet6 fe80::250:56ff:fea6:727/64 scope link valid_lft forever preferred_lft forever When I dig for record to ipv6.idrobot.net I don't get a timeout dig ipv6.idrobot.net ; DiG 9.8.2rc1-RedHat-9.8.2-0.10.rc1.el6_3.6 ipv6.idrobot.net ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NXDOMAIN, id: 34596 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0 ;; QUESTION SECTION: ;ipv6.idrobot.net. IN ;; AUTHORITY SECTION: net.900 IN SOA a.gtld-servers.net. nstld.verisign-grs.com. 1360654692 1800 900 604800 86400 ;; Query time: 17 msec ;; SERVER: 160.85.192.100#53(160.85.192.100) ;; WHEN: Tue Feb 12 08:38:40 2013 ;; MSG SIZE rcvd: 107 When I dig for record to www2.zhlex.zh.ch I get one dig www2.zhlex.zh.ch ; DiG 9.8.2rc1-RedHat-9.8.2-0.10.rc1.el6_3.6 www2.zhlex.zh.ch ;; global options: +cmd ;; connection timed out; no servers could be reached Do you have the same timout as well with that host and ipv6 running? This is a domain which is queried a lot. Yes. I traced it through three CNAME redirections to a pair of DNS servers which do not respond to any queries. # dig zhcompublicweb1.subd.djiktzh.ch @lc1.djiktzh.ch ; DiG 9.3.6-P1 zhcompublicweb1.subd.djiktzh.ch @lc1.djiktzh.ch ;; global options: printcmd ;; connection timed out; no servers could be reached # dig zhcompublicweb1.subd.djiktzh.ch @lc2.djiktzh.ch ; DiG 9.3.6-P1 zhcompublicweb1.subd.djiktzh.ch @lc2.djiktzh.ch ;; global options: printcmd ;; connection timed out; no servers could be reached Those DNS servers lc1.djiktzh.ch and lc2.djiktzh.ch are broken. Amos
[squid-users] dns_v4_first on ignored?
Hi I am using squid-3.2.6. Our network interfaces have IPV6INIT=no. We do not use ipv6. In squid.conf we have set dns_v4_first to on but it still lookups for the record on certain pages which ends in a timeout after about 2 minutes before it searches for the A record. This config works if I completely remove ipv6 kernel module but I'd rather not do that. Have I configured something wrong? We have the following config # --- # - Global Configuration # -- # Look for ipv4 first dns_v4_first on acl to_ipv6 dst ipv6 tcp_outgoing_address 160.85.104.14 !to_ipv6 # Port to listen http_port 160.85.104.14:8080 # Coredump directory coredump_dir /var/spool/squid # Cache settings cache_effective_user squid cache_effective_group squid cache_mem 3072 MB cache_dir aufs /var/cache/squid 25000 64 256 maximum_object_size_in_memory 50 KB # Mail of which will be notified when squid dies cache_mgr serviced...@zhaw.ch # Do not allow underscores in hostnames allow_underscore off # DNS Settings dns_retransmit_interval 3 seconds dns_nameservers 160.85.192.100 append_domain .zhaw.ch # Other settings hierarchy_stoplist cgi-bin ? ftp_user wwwu...@zhaw.ch request_timeout 30 seconds httpd_suppress_version_string on visible_hostname srv-app-904.zhaw.ch unique_hostname srv-app-904.zhaw.ch # -- # -- # - Define ports # -- acl SSL_ports port 443 8443 28443 50001 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT # -- # -- # - Define networks # -- acl bigip src 160.85.104.21/32# F5 forward-proxy acl monhost src 160.85.192.190/32 # mon.zhaw.ch acl snmppublic snmp_community Fast3thernet acl ZHAWnet src 160.85.0.0/16 # ZHAW acl ZHAWnet src 195.176.253.59/32 # HSWNAT acl ZHAWnet src 10.196.0.0/16 # VoIP acl ZHAWnet src 10.194.4.0/22 # HAP acl ZHAWnet src 10.194.36.0/22 # HSSAZ acl ZHAWnet src 172.28.8.0/24 # Management Netz 1 acl ZHAWnet src 172.28.9.0/24 # Management Netz 2 acl ZHAWnet src 172.28.10.0/24 # FET-DEV acl ZHAWnet src 172.28.11.0/24 # FET-TEST acl ZHAWnet src 172.28.12.0/24 # BET-DEV acl ZHAWnet src 172.28.13.0/24 # BET-TEST acl ZHAWnet src 172.28.14.0/24 # FET-VDP acl ZHAWnet src 172.28.15.0/24 # FET-VDP acl STAFFMGR src 160.85.85.0/26 acl srv-ts-057 src 160.85.186.73/32 acl srv-ts-058 src 160.85.186.74/32 acl MONZHAWCH dstdomain mon.zhaw.ch acl ZREG dstdomain zreg.zhaw.ch acl EXCLUDE dstdomain domzhwin01.zhaw.ch acl EXCLUDE dstdomain domzhwin02.zhaw.ch acl EXCLUDE dstdomain domzhwin03.zhaw.ch acl EXCLUDE dstdomain dc01.zhaw.ch acl EXCLUDE dstdomain dc02.zhaw.ch acl EXCLUDE dstdomain dc03.zhaw.ch acl EXCLUDE dstdomain dc04.zhaw.ch acl EXCLUDE dstdomain dc10.zhaw.ch acl EXCLUDE dstdomain dc11.zhaw.ch acl EXCLUDE dstdomain turtle.zhaw.ch acl EXCLUDE dstdomain zebra.zhaw.ch acl EXCLUDE dstdomain dolphin.zhaw.ch acl EXCLUDE dstdomain orca.zhaw.ch acl EXCLUDE dstdomain kangaroo.zhaw.ch acl EXCLUDE dstdomain lobster.zhaw.ch acl EXCLUDE dstdomain calamari.zhaw.ch acl EXCLUDE dstdomain warthog.zhaw.ch acl EXCLUDE dstdomain billabong.zhaw.ch acl EXCLUDE dstdomain zeus.zhaw.ch acl EXCLUDE dstdomain rhino1.zhaw.ch acl EXCLUDE dstdomain rhino2.zhaw.ch acl EXCLUDE dstdomain zhaw.zhaw.ch acl EXCLUDE dstdomain barracuda.zhaw.ch acl EXCLUDE dstdomain caesar.zhaw.ch acl EXCLUDE dstdomain octopus.zhaw.ch acl EXCLUDE dstdomain pandora.zhaw.ch acl EXCLUDE dstdomain gonzo.zhaw.ch acl PURGE method PURGE acl PUT method PUT acl PROPFIND method PROPFIND # -- # -- # - Access rules # -- # Only allow cachemgr access from localhost http_access allow localhost manager http_access deny manager # Deny requests to certain unsafe ports http_access deny !Safe_ports # Deny CONNECT to
AW: [squid-users] Re: dns_v4_first on ignored?
We only use RPM so I can not use the --disable-ipv6 parameter. -Ursprüngliche Nachricht- Von: babajaga [mailto:augustus_me...@yahoo.de] Gesendet: Montag, 11. Februar 2013 11:56 An: squid-users@squid-cache.org Betreff: [squid-users] Re: dns_v4_first on ignored? I am not using IPv6, too. So I compiled squid 3.2.7 using ./configure --disable-ipv6 -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/dns-v4-first-on-ignored-tp4658427p4658428.html Sent from the Squid - Users mailing list archive at Nabble.com.
AW: AW: [squid-users] Re: dns_v4_first on ignored?
Centos 6.3 Source: http://repo.ngtech.co.il/rpm/centos/6/x86_64/ -Ursprüngliche Nachricht- Von: Eliezer Croitoru [mailto:elie...@ngtech.co.il] Gesendet: Montag, 11. Februar 2013 12:38 An: squid-users@squid-cache.org Betreff: Re: AW: [squid-users] Re: dns_v4_first on ignored? What distro? On 2/11/2013 1:34 PM, Sandrini Christian (xsnd) wrote: We only use RPM so I can not use the --disable-ipv6 parameter. -Ursprüngliche Nachricht- Von: babajaga [mailto:augustus_me...@yahoo.de] Gesendet: Montag, 11. Februar 2013 11:56 An:squid-users@squid-cache.org Betreff: [squid-users] Re: dns_v4_first on ignored? I am not using IPv6, too. So I compiled squid 3.2.7 using ./configure --disable-ipv6 -- View this message in context:http://squid-web-proxy-cache.1019090.n4.nabble.com/dns-v4-firs t-on-ignored-tp4658427p4658428.html Sent from the Squid - Users mailing list archive at Nabble.com. -- Eliezer Croitoru http://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer at ngtech.co.il
AW: AW: AW: [squid-users] Re: dns_v4_first on ignored?
Hi Thanks for your reply. I can't really mess around with our main DNS servers. On our 3.1 squids we just disabled ipv6 module which does not sound right to me but works fine. What we see is 2013/01/30 09:52:00.296| idnsGrokReply: www2.zhlex.zh.ch query failed. Trying A now instead. We do not need any ipv6 support. I'd rather have a way to tell squid to look first for an A record. -Ursprüngliche Nachricht- Von: Eliezer Croitoru [mailto:elie...@ngtech.co.il] Gesendet: Montag, 11. Februar 2013 13:28 An: squid-users@squid-cache.org Betreff: Re: AW: AW: [squid-users] Re: dns_v4_first on ignored? My repo indeed. I dont have full IPV6 stack here but IPV6 enabled due to the necessity. It's kind of a global settings which seems to be working for almost anyone. If you do ask me I would deal with it on the DNS level rather then squid. Also take in account that there are dns which has only record for a domain. If you do have specific site that does that I would consider debugging the problem deeper to make sure the reason is not a bug. Notice that dns_v4_first may be not ignored but rather cannot be used. BIND dns can be started with -4 option to help you. just add a dns cache server to the squid instance to help it. There are other less robust forwarders which can be used only for this purpose but BIND is a very good choice. Try first and let us know how it works for you. Eliezer P.S. you need to configure BIND to use only forwarders and point it to the local shared dns server to the clients. On 2/11/2013 2:06 PM, Sandrini Christian (xsnd) wrote: Centos 6.3 Source: http://repo.ngtech.co.il/rpm/centos/6/x86_64/ -- Eliezer Croitoru http://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer at ngtech.co.il
AW: AW: AW: AW: [squid-users] Re: dns_v4_first on ignored?
Hi I have now enabled ipv6 3: eth1: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 link/ether 00:50:56:a6:07:27 brd ff:ff:ff:ff:ff:ff inet 160.85.104.14/24 brd 160.85.104.255 scope global eth1 inet6 fe80::250:56ff:fea6:727/64 scope link valid_lft forever preferred_lft forever When I dig for record to ipv6.idrobot.net I don't get a timeout dig ipv6.idrobot.net ; DiG 9.8.2rc1-RedHat-9.8.2-0.10.rc1.el6_3.6 ipv6.idrobot.net ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NXDOMAIN, id: 34596 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0 ;; QUESTION SECTION: ;ipv6.idrobot.net. IN ;; AUTHORITY SECTION: net.900 IN SOA a.gtld-servers.net. nstld.verisign-grs.com. 1360654692 1800 900 604800 86400 ;; Query time: 17 msec ;; SERVER: 160.85.192.100#53(160.85.192.100) ;; WHEN: Tue Feb 12 08:38:40 2013 ;; MSG SIZE rcvd: 107 When I dig for record to www2.zhlex.zh.ch I get one dig www2.zhlex.zh.ch ; DiG 9.8.2rc1-RedHat-9.8.2-0.10.rc1.el6_3.6 www2.zhlex.zh.ch ;; global options: +cmd ;; connection timed out; no servers could be reached Do you have the same timout as well with that host and ipv6 running? This is a domain which is queried a lot. -Ursprüngliche Nachricht- Von: Amos Jeffries [mailto:squ...@treenet.co.nz] Gesendet: Dienstag, 12. Februar 2013 01:10 An: squid-users@squid-cache.org Betreff: Re: AW: AW: AW: [squid-users] Re: dns_v4_first on ignored? On 12/02/2013 12:17 p.m., Eliezer Croitoru wrote: I gave you an option to install on the squid server a BIND cache server wasn't talking about your main DNS server. Note the you can always use a secondary dns instance to serve this purpose to filter responses. On 2/11/2013 2:48 PM, Sandrini Christian (xsnd) wrote: Hi Thanks for your reply. I can't really mess around with our main DNS servers. On our 3.1 squids we just disabled ipv6 module which does not sound right to me but works fine. I suggest to not disable v6 and work with it if you can. What we see is 2013/01/30 09:52:00.296| idnsGrokReply: www2.zhlex.zh.ch query failed. Trying A now instead. We do not need any ipv6 support. I'd rather have a way to tell squid to look first for an A record. Please take your time to file a bug-report in the bugzilla: http://bugs.squid-cache.org describe the problem and add any logs you can into the report to help the development team track and fix it. It seems like a *big* issue to me since this points about dns_v4_first failure. No. A bug report will not make any difference here. dns_v4_first is about the sorting the results found, not the lookup order. is faster than A in most networks, so we perform that lookup first in 3.1. This was altered in 3.2 to perform happy-eyeballs parallel lookups anyway so most bugs in the lookup code of 3.1 will be closed as irrelevant. Note that the current supported release is now 3.3.1. Try to use the BIND solution I am using. I have been logging my dns server and it seems like squid 3.HEAD tries to resolve A before but tries to resolve after A record. You can try to remove manually ipv6 address from lo and other devices to make sure there is no v6 address initialized by centos scripts. In my testing server the system starts with lo adapter inet6 addr: ::1/128 Scope:Host and also on another devices with a local auto v6 address. so remove them and try restarting squid service to see what is going on. This is VERY likely to be the problem. Squid tests for IPv6 ability automatically by opening a socket on a private IP address, if that works the socket options are noted and used. There is no way for Squid to identify in advance of opening upstream connections whether the NIC the kernel chooses to use will be v6-enabled or not. Notice that the method used to disable IPv6 was to simply not assign IPv6 address to the NIC, nothing at the sockets layer was actually disabled. So every NIC needs to be checked and disabled individually as well, and any sub-system loading IPv6 functionality into the kernel also needs disabling as well. (Warning: soapbox) The big question is, why disable in the first place? v6 is faster and more efficient than v4 when you get it going properly. And one he*l of a lot easier to administrate. If any of your upstreams supply native connections it is well worth taking the option up. If not there is always 6to4 or other tunnel types that can be built right to the proxy box to get IPv6 at only a small initial latency on the SYN packet (ping 192.88.99.1 to see what 6to4 adds for you). Note that these are IPv6 connectivity initiated from the proxy to the Internet *only*, so firewall alterations are minimal to get Squid v6-enabled. Amos
WG: [squid-users] Squid processing very slow on some pdf
Hi due to George's hint about DNS resolution I found the solution As we are not using ipv6 I have disabled ipv6 on the server and added the following line to the squid.conf # Force ipv4 acl to_ipv6 dst ipv6 tcp_outgoing_address 160.85.104.12 !to_ipv Not it works like a charm! Thanks George. -Ursprüngliche Nachricht- Von: Sandrini Christian (xsnd) Gesendet: Mittwoch, 30. Januar 2013 11:30 An: 'George Herbert' Betreff: AW: [squid-users] Squid processing very slow on some pdf Hi I do see the following line in the cache.log when I enable debugging 2013/01/30 09:52:00.296| idnsGrokReply: www2.zhlex.zh.ch query failed. Trying A now instead. There is actually a timeout. # host www2.zhlex.zh.ch www2.zhlex.zh.ch is an alias for zhcompublicweb1.djiktzh.ch. zhcompublicweb1.djiktzh.ch is an alias for zhcompublicweb1.subd.djiktzh.ch. zhcompublicweb1.subd.djiktzh.ch has address 195.65.218.66 ;; connection timed out; no servers could be reached Host zhcompublicweb1.subd.djiktzh.ch not found: 3(NXDOMAIN) How could I fix that? How comes it works smooth without squid? Doesn't it have to resolve the name the same way as squid does? -Ursprüngliche Nachricht- Von: George Herbert [mailto:george.herb...@gmail.com] Gesendet: Mittwoch, 30. Januar 2013 11:05 An: Sandrini Christian (xsnd) Cc: squid-users@squid-cache.org Betreff: Re: [squid-users] Squid processing very slow on some pdf On first impression from this data? Check DNS resolution from the Squid to that hostname. It sounds like a timeout / retry / recursion fail in progress... George William Herbert Sent from my iPhone On Jan 29, 2013, at 11:54 PM, Sandrini Christian \(xsnd\) x...@zhaw.ch wrote: Hi We are using an f5 appliance that is loadbalancing http request to 3 squid servers. We use squid 3.1.10. When I want to open a pdf file of a certain domain it takes several minutes for 160kb. If I open the pdf without going through the proxy it is very quick. We have seen this problem only on the pdf of the following domain http://www2.zhlex.zh.ch/appl/zhlex_r.nsf/0/62FABE8867570E44C1257A21003 2892E/$file/414.252.3_29.1.08_77.pdf This is in the access.log. Squid takes 115 seconds to handle the request. 1359524314.374 115810 160.85.85.46 TCP_HIT/200 111028 GET http://www2.zhlex.zh.ch/appl/zhlex_r.nsf/0/62FABE8867570E44C1257A21003 2892E/$file/414.252.3_29.1.08_77.pdf - NONE/- application/pdf No logs have been written to cache.log during that time. I have captured the network traffic from the squidbox to www2.zhlex.zh.ch to find out the time squid takes to get the pdf. It does it in less than a second. tcpdump -i eth1 host www2.zhlex.zh.ch tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 06:30:45.491906 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [S], seq 1530511587, win 5840, options [mss 1460,nop,nop,sackOK], length 0 06:30:45.494241 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [S.], seq 3031868726, ack 1530511588, win 64240, options [mss 1380,nop,nop,sackOK], length 0 06:30:45.494259 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 1, win 5840, length 0 06:30:45.494353 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [P.], seq 1:519, ack 1, win 5840, length 518 06:30:45.524850 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [P.], seq 1:290, ack 519, win 63722, length 289 06:30:45.524864 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 290, win 6432, length 0 06:30:45.541484 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 290:1670, ack 519, win 63722, length 1380 06:30:45.541493 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 1670, win 9660, length 0 06:30:45.541603 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 1670:3050, ack 519, win 63722, length 1380 06:30:45.541612 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 3050, win 12420, length 0 06:30:45.541709 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 3050:4430, ack 519, win 63722, length 1380 06:30:45.541718 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 4430, win 15180, length 0 06:30:45.543929 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 4430:5810, ack 519, win 63722, length 1380 06:30:45.543937 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 5810, win 17940, length 0 06:30:45.544053 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 5810:7190, ack 519, win 63722, length 1380 06:30:45.544062 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 7190, win 20700, length 0 06:30:45.544162 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 7190:8570, ack 519, win 63722, length 1380 06:30:45.544170 IP srv-app-902.zhaw.ch.34179
[squid-users] Squid processing very slow on some pdf
Hi We are using an f5 appliance that is loadbalancing http request to 3 squid servers. We use squid 3.1.10. When I want to open a pdf file of a certain domain it takes several minutes for 160kb. If I open the pdf without going through the proxy it is very quick. We have seen this problem only on the pdf of the following domain http://www2.zhlex.zh.ch/appl/zhlex_r.nsf/0/62FABE8867570E44C1257A210032892E/$file/414.252.3_29.1.08_77.pdf This is in the access.log. Squid takes 115 seconds to handle the request. 1359524314.374 115810 160.85.85.46 TCP_HIT/200 111028 GET http://www2.zhlex.zh.ch/appl/zhlex_r.nsf/0/62FABE8867570E44C1257A210032892E/$file/414.252.3_29.1.08_77.pdf - NONE/- application/pdf No logs have been written to cache.log during that time. I have captured the network traffic from the squidbox to www2.zhlex.zh.ch to find out the time squid takes to get the pdf. It does it in less than a second. tcpdump -i eth1 host www2.zhlex.zh.ch tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 06:30:45.491906 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [S], seq 1530511587, win 5840, options [mss 1460,nop,nop,sackOK], length 0 06:30:45.494241 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [S.], seq 3031868726, ack 1530511588, win 64240, options [mss 1380,nop,nop,sackOK], length 0 06:30:45.494259 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 1, win 5840, length 0 06:30:45.494353 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [P.], seq 1:519, ack 1, win 5840, length 518 06:30:45.524850 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [P.], seq 1:290, ack 519, win 63722, length 289 06:30:45.524864 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 290, win 6432, length 0 06:30:45.541484 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 290:1670, ack 519, win 63722, length 1380 06:30:45.541493 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 1670, win 9660, length 0 06:30:45.541603 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 1670:3050, ack 519, win 63722, length 1380 06:30:45.541612 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 3050, win 12420, length 0 06:30:45.541709 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 3050:4430, ack 519, win 63722, length 1380 06:30:45.541718 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 4430, win 15180, length 0 06:30:45.543929 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 4430:5810, ack 519, win 63722, length 1380 06:30:45.543937 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 5810, win 17940, length 0 06:30:45.544053 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 5810:7190, ack 519, win 63722, length 1380 06:30:45.544062 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 7190, win 20700, length 0 06:30:45.544162 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 7190:8570, ack 519, win 63722, length 1380 06:30:45.544170 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 8570, win 23460, length 0 06:30:45.544303 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 8570:9950, ack 519, win 63722, length 1380 06:30:45.544308 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 9950, win 26220, length 0 06:30:45.544372 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 9950:11330, ack 519, win 63722, length 1380 06:30:45.544381 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 11330, win 28980, length 0 06:30:45.544531 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 11330:12710, ack 519, win 63722, length 1380 06:30:45.544541 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 12710, win 33120, length 0 06:30:45.546216 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 12710:14090, ack 519, win 63722, length 1380 06:30:45.546226 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 14090, win 35880, length 0 06:30:45.546332 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 14090:15470, ack 519, win 63722, length 1380 06:30:45.546341 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 15470, win 38640, length 0 06:30:45.546463 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 15470:16850, ack 519, win 63722, length 1380 06:30:45.546472 IP srv-app-902.zhaw.ch.34179 195.65.218.66.http: Flags [.], ack 16850, win 41400, length 0 06:30:45.546585 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 16850:18230, ack 519, win 63722, length 1380 06:30:45.548946 IP 195.65.218.66.http srv-app-902.zhaw.ch.34179: Flags [.], seq 32030:33410, ack 519, win 63722, length 1380 06:30:45.549036 IP 195.65.218.66.http
