[squid-users] Problem with ntlm_auth: dying
Hello, I have running Squid Cache: Version 3.0.STABLE13 with ntlm auth, using samba-3.2.10 and winbind, also SquidGuard 1.4 Since a few days I am detecting that squid is going down (then restart again without any problem) with the error: 2009/05/06 12:59:33| assertion failed: comm.cc:572: fdc_table[fd].active == 1 [2009/05/06 12:59:34, 1] utils/ntlm_auth.c:manage_squid_request(2167) fgets() failed! dying. errno=1 (Operation not permitted) I really don't know what`s going wrong. It usually occurs when I do a squid reload. If you need more information please say to me. The complete cache.log of the error is: 2009/05/06 12:59:16| storeDirWriteCleanLogs: Starting... 2009/05/06 12:59:16| 65536 entries written so far. ... 2009/05/06 12:59:17| 1245184 entries written so far. 2009/05/06 12:59:17| Finished. Wrote 1280543 entries. 2009/05/06 12:59:17| Took 0.86 seconds (1489072.75 entries/sec). 2009/05/06 12:59:17| logfileRotate: /var/log/squid/access.log 2009/05/06 12:59:17| Pinger socket opened on FD 11 2009/05/06 12:59:17| helperOpenServers: Starting 30 'squidGuard' processes 2009/05/06 12:59:19| helperStatefulOpenServers: Starting 50 'ntlm_auth' processes 2009/05/06 12:59:24| helperOpenServers: Starting 5 'ntlm_auth' processes 2009/05/06 12:59:24| helperOpenServers: Starting 10 'wbinfo_group.pl' processes 2009/05/06 12:59:25| Reconfiguring Squid Cache (version 3.0.STABLE13)... 2009/05/06 12:59:25| FD 121 Closing HTTP connection 2009/05/06 12:59:25| Closing Pinger socket on FD 11 2009/05/06 12:59:25| Processing Configuration File: /etc/squid/squid.conf (depth 0) 2009/05/06 12:59:25| cache_cf.cc(346) squid.conf:122 unrecognized: ' #' 2009/05/06 12:59:25| Initializing https proxy context 2009/05/06 12:59:25| Store logging disabled 2009/05/06 12:59:25| User-Agent logging is disabled. 2009/05/06 12:59:25| Referer logging is disabled. 2009/05/06 12:59:25| DNS Socket created at 0.0.0.0, port 42477, FD 8 2009/05/06 12:59:25| Adding nameserver 172.28.1.90 from /etc/resolv.conf 2009/05/06 12:59:25| Adding nameserver 172.28.1.91 from /etc/resolv.conf 2009/05/06 12:59:25| Adding domain iipp.int from /etc/resolv.conf 2009/05/06 12:59:25| helperOpenServers: Starting 30 'squidGuard' processes 2009/05/06 12:59:28| helperStatefulOpenServers: Starting 50 'ntlm_auth' processes 2009/05/06 12:59:32| helperOpenServers: Starting 5 'ntlm_auth' processes 2009/05/06 12:59:32| helperOpenServers: Starting 10 'wbinfo_group.pl' processes 2009/05/06 12:59:33| Accepting HTTP connections at 0.0.0.0, port 3128, FD 181. 2009/05/06 12:59:33| HTCP Disabled. 2009/05/06 12:59:33| Pinger socket opened on FD 216 2009/05/06 12:59:33| Configuring Parent 192.168.113.4/8080/0 2009/05/06 12:59:33| Loaded Icons. 2009/05/06 12:59:33| Ready to serve requests. 2009/05/06 12:59:33| assertion failed: comm.cc:572: fdc_table[fd].active == 1 [2009/05/06 12:59:34, 1] utils/ntlm_auth.c:manage_squid_request(2167) fgets() failed! dying. errno=1 (Operation not permitted) [2009/05/06 12:59:34, 1] utils/ntlm_auth.c:manage_squid_request(2167) fgets() failed! dying. errno=1 (Operation not permitted) [2009/05/06 12:59:34, 1] utils/ntlm_auth.c:manage_squid_request(2167) fgets() failed! dying. errno=1 (Operation not permitted) 2009/05/06 12:59:37| Starting Squid Cache version 3.0.STABLE13 for i686-suse-linux-gnu... 2009/05/06 12:59:37| Process ID 8297 2009/05/06 12:59:37| With 4096 file descriptors available 2009/05/06 12:59:37| DNS Socket created at 0.0.0.0, port 59654, FD 7 2009/05/06 12:59:37| Adding nameserver 172.28.1.90 from /etc/resolv.conf 2009/05/06 12:59:37| Adding nameserver 172.28.1.91 from /etc/resolv.conf 2009/05/06 12:59:37| Adding domain iipp.int from /etc/resolv.conf 2009/05/06 12:59:37| helperOpenServers: Starting 30 'squidGuard' processes 2009/05/06 12:59:38| helperStatefulOpenServers: Starting 50 'ntlm_auth' processes 2009/05/06 12:59:39| helperOpenServers: Starting 5 'ntlm_auth' processes 2009/05/06 12:59:39| helperOpenServers: Starting 10 'wbinfo_group.pl' processes 2009/05/06 12:59:39| User-Agent logging is disabled. 2009/05/06 12:59:39| Referer logging is disabled. 2009/05/06 12:59:39| Unlinkd pipe opened on FD 107 2009/05/06 12:59:39| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec 2009/05/06 12:59:39| Store logging disabled 2009/05/06 12:59:39| Swap maxSize 2560 KB, estimated 1969230 objects 2009/05/06 12:59:39| Target number of buckets: 98461 2009/05/06 12:59:39| Using 131072 Store buckets 2009/05/06 12:59:39| Max Mem size: 16384 KB 2009/05/06 12:59:39| Max Swap size: 2560 KB 2009/05/06 12:59:39| Version 1 of swap file with LFS support detected... 2009/05/06 12:59:39| Rebuilding storage in /var/log/squid/cache (CLEAN) 2009/05/06 12:59:39| Using Least Load store dir selection 2009/05/06 12:59:39| Current Directory is / 2009/05/06 12:59:39| Loaded Icons. 2009/05/06 12:59:39| Accepting HTTP connections
Re: [squid-users] Problem with ntlm_auth: dying
Gonzalo PG wrote: Hello, I have running Squid Cache: Version 3.0.STABLE13 with ntlm auth, using samba-3.2.10 and winbind, also SquidGuard 1.4 Since a few days I am detecting that squid is going down (then restart again without any problem) with the error: 2009/05/06 12:59:33| assertion failed: comm.cc:572: fdc_table[fd].active == 1 [2009/05/06 12:59:34, 1] utils/ntlm_auth.c:manage_squid_request(2167) fgets() failed! dying. errno=1 (Operation not permitted) I really don't know what`s going wrong. It usually occurs when I do a squid reload. If you need more information please say to me. The complete cache.log of the error is: 2009/05/06 12:59:16| storeDirWriteCleanLogs: Starting... 2009/05/06 12:59:16| 65536 entries written so far. ... 2009/05/06 12:59:17| 1245184 entries written so far. 2009/05/06 12:59:17| Finished. Wrote 1280543 entries. 2009/05/06 12:59:17| Took 0.86 seconds (1489072.75 entries/sec). 2009/05/06 12:59:17| logfileRotate: /var/log/squid/access.log 2009/05/06 12:59:17| Pinger socket opened on FD 11 2009/05/06 12:59:17| helperOpenServers: Starting 30 'squidGuard' processes 2009/05/06 12:59:19| helperStatefulOpenServers: Starting 50 'ntlm_auth' processes 2009/05/06 12:59:24| helperOpenServers: Starting 5 'ntlm_auth' processes 2009/05/06 12:59:24| helperOpenServers: Starting 10 'wbinfo_group.pl' processes 2009/05/06 12:59:25| Reconfiguring Squid Cache (version 3.0.STABLE13)... 2009/05/06 12:59:25| FD 121 Closing HTTP connection 2009/05/06 12:59:25| Closing Pinger socket on FD 11 2009/05/06 12:59:25| Processing Configuration File: /etc/squid/squid.conf (depth 0) 2009/05/06 12:59:25| cache_cf.cc(346) squid.conf:122 unrecognized: ' #' 2009/05/06 12:59:25| Initializing https proxy context 2009/05/06 12:59:25| Store logging disabled 2009/05/06 12:59:25| User-Agent logging is disabled. 2009/05/06 12:59:25| Referer logging is disabled. 2009/05/06 12:59:25| DNS Socket created at 0.0.0.0, port 42477, FD 8 2009/05/06 12:59:25| Adding nameserver 172.28.1.90 from /etc/resolv.conf 2009/05/06 12:59:25| Adding nameserver 172.28.1.91 from /etc/resolv.conf 2009/05/06 12:59:25| Adding domain iipp.int from /etc/resolv.conf 2009/05/06 12:59:25| helperOpenServers: Starting 30 'squidGuard' processes 2009/05/06 12:59:28| helperStatefulOpenServers: Starting 50 'ntlm_auth' processes 2009/05/06 12:59:32| helperOpenServers: Starting 5 'ntlm_auth' processes 2009/05/06 12:59:32| helperOpenServers: Starting 10 'wbinfo_group.pl' processes 2009/05/06 12:59:33| Accepting HTTP connections at 0.0.0.0, port 3128, FD 181. 2009/05/06 12:59:33| HTCP Disabled. 2009/05/06 12:59:33| Pinger socket opened on FD 216 2009/05/06 12:59:33| Configuring Parent 192.168.113.4/8080/0 2009/05/06 12:59:33| Loaded Icons. 2009/05/06 12:59:33| Ready to serve requests. 2009/05/06 12:59:33| assertion failed: comm.cc:572: fdc_table[fd].active == 1 [2009/05/06 12:59:34, 1] utils/ntlm_auth.c:manage_squid_request(2167) fgets() failed! dying. errno=1 (Operation not permitted) [2009/05/06 12:59:34, 1] utils/ntlm_auth.c:manage_squid_request(2167) fgets() failed! dying. errno=1 (Operation not permitted) [2009/05/06 12:59:34, 1] utils/ntlm_auth.c:manage_squid_request(2167) fgets() failed! dying. errno=1 (Operation not permitted) 2009/05/06 12:59:37| Starting Squid Cache version 3.0.STABLE13 for i686-suse-linux-gnu... Thank you for this report. Firstly, can you replicate this with 3.0.STABLE15? There have been a few ntlm_auth handling fixes recently. By 'usually occurs when I do a squid reload' do you have any actual trace from it happening without a reload operation? And it there any difference visible? I suspect this is another side effect of an FD issue we are aware of that occurs only during reconfigure and shutdown. The fix for that one is taking a while and may need to be a part of 3.1. Amos -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE15 Current Beta Squid 3.1.0.7
Re: [squid-users] Problem with ntlm_auth: dying
Hi Amos, Thanks you for your soon replay, I've been looking through the logs and it only happens when doing a reload. About updating to STABLE 15, I need to plan an stop of the internet service for all the organization where I work, so it's no so easy, but if you think it can resolve the problem I do it. Sorry for the question, but what is an FD issue? Again thank you and your partners, you are doing a great job. Gontzal
Re: [squid-users] Problem with ntlm_auth: dying
Gonzalo PG wrote: Hi Amos, Thanks you for your soon replay, I've been looking through the logs and it only happens when doing a reload. About updating to STABLE 15, I need to plan an stop of the internet service for all the organization where I work, so it's no so easy, but if you think it can resolve the problem I do it. Sorry for the question, but what is an FD issue? FD == file descriptor. Squid uses FD during its working. We know that when it does not close all helpers on a reconfigure/shutdown the remaining open ones hang onto FD and can crash at the final stage of close. I've looked at the code thats breaking and its not fixed yet. So unless an upgrade is easy its probably not worth it for you just for testing this. It's probably fixed in 3.1, but I don't want to go as far as to recommend 3.1 for use in any seriously important places yet. I will have to consult with the other dev who may know more about this particular piece of code before we do anything. I will get back to you on this. Amos -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE15 Current Beta Squid 3.1.0.7
Re: [squid-users] Problem with ntlm_auth: dying
Amos Jeffries wrote: Gonzalo PG wrote: Hi Amos, Thanks you for your soon replay, I've been looking through the logs and it only happens when doing a reload. About updating to STABLE 15, I need to plan an stop of the internet service for all the organization where I work, so it's no so easy, but if you think it can resolve the problem I do it. Sorry for the question, but what is an FD issue? FD == file descriptor. Squid uses FD during its working. We know that when it does not close all helpers on a reconfigure/shutdown the remaining open ones hang onto FD and can crash at the final stage of close. I've looked at the code thats breaking and its not fixed yet. So unless an upgrade is easy its probably not worth it for you just for testing this. It's probably fixed in 3.1, but I don't want to go as far as to recommend 3.1 for use in any seriously important places yet. I will have to consult with the other dev who may know more about this particular piece of code before we do anything. I will get back to you on this. Amos On a little bit more investigation... If you have half_closed_clients set to on or missing from your config can you try setting it to off and see if that solves the problem? Amos -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE15 Current Beta Squid 3.1.0.7
Re: [squid-users] adding content to cache
Squid indexes content by its URL, so in all honesty, the best way to get an object into squid's storage is to just request it through the proxy. This is easily scriptable via the curl and wget command-line tools, or frameworks like perl's LWP. -C On May 9, 2009, at 10:04 PM, Laurent Luce wrote: Actually, I am looking at a way of adding it directly to the squid cache. Basically, take the file and add it to the cache. I am looking into patching Squid to provide an API to do that. How complicated do you think it is if I want to add the file content along with the metadata directly into the cache ? Laurent - Original Message From: Jeff Pang pa...@arcor.de To: Laurent Luce laurentluc...@yahoo.com Cc: squid-users@squid-cache.org Sent: Monday, May 4, 2009 9:39:40 PM Subject: Re: [squid-users] adding content to cache Laurent Luce: I am looking for a way to manually add content to the cache. Is there an API to do that ? For example, I have the following file image.gif and I want to add it to the proxy cache so it can be served from there when needed. You could use a tool like wget to pass requests through Squid then the object will be cached if it is cachable. wget has some good arguments like -p or -m which even can be used to cache the whole site. -- Jeff Pang DingTong Technology www.dtonenetworks.com
[squid-users] Squid 2.X for Windows 7
Hi, Recently we found out that Squid 2.X is not compatible with Windows 7. It does however run when I do the Windows VISTA compatibility mode. However, we can't use it that way for my work. Can anyone please let me know if there is a roadmap for Windows 7 support and when it will be available? Thanks Balaji
[squid-users] How to set different delay_initial_bucket_level for different pools
Hi, can this line appear more than once in squid.conf? delay_initial_bucket_level 100 Say, at the top of each delay pool definition block? I'm trying to give different init ial buckets to different pools. Regards Dayo
[squid-users] R: [squid-users] Squid 2.X for Windows 7
Hi, -Messaggio originale- Da: Balaji Ganesan [mailto:bgane...@venturiwireless.com] Inviato: lunedì 11 maggio 2009 18.55 A: squid-users@squid-cache.org Oggetto: [squid-users] Squid 2.X for Windows 7 Hi, Recently we found out that Squid 2.X is not compatible with Windows 7. It does however run when I do the Windows VISTA compatibility mode. However, we can't use it that way for my work. Can anyone please let me know if there is a roadmap for Windows 7 support and when it will be available? Thanks Balaji Sure, Windows 7 support is already into 2.HEAD, and it will be available in the next 2.7 STABLE release. Regards Guido Serassio Acme Consulting S.r.l. - Microsoft Certified Partner Via Lucia Savarino, 110098 - Rivoli (TO) - ITALY Tel. : +39.011.9530135 Fax. : +39.011.9781115 Email: i...@acmeconsulting.it WWW: http://www.acmeconsulting.it/
[squid-users] TProxy not faking source address.
Hello, I'm trying to get TProxy 4.1 to work as outlined here: http://wiki.squid-cache.org/Features/Tproxy4 namely under Ubuntu 9.04 stable/testing mix with the following: linux-image-2.6.28-11-server 2.6.28-11.42 iptables 1.4.3.2-2ubuntu1 squid-3.1.0.7.tar.bz2 from original sources Squid has been built this way: $ /usr/local/squid/sbin/squid -v Squid Cache: Version 3.1.0.7 configure options: '--enable-linux-netfilter' --with-squid=/home/guessed/squid-3.1.0.7 --enable-ltdl-convenience (myself I only gave it --enable-linux-netfilter) squid.conf is pretty much whatever 'make install' created, with my changes given at the end, after the blank line: acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localnet http_access deny all http_port 3128 hierarchy_stoplist cgi-bin ? refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 coredump_dir /usr/local/squid/var/cache cache_dir ufs /usr/local/squid/var/cache 100 16 256 cache_mem 16 MB http_port 3129 tproxy visible_hostname tproxy Then I did: iptables -t mangle -N DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT #Use DIVERT to prevent existing connections going through TPROXY twice: iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT #Mark all other (new) packets and use TPROXY to pass into Squid: iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 #On each boot startup set: echo 1 /proc/sys/net/ipv4/ip_forward ran squid -z and launched squid. My topology: desktop where I sit: one link has address 192.168.0.1/24, the other to the Internet Squid box: one link: 192.168.0.184/24 (bridged VMware interface on the same box as desktop), the other link is custom VMware interface 192.168.1.1/24 The client box: single interface 192.168.1.2/24 So, the squid box is directly connected to the outside on the one side, and to the client on the other. My desktop's routing knows to reach the client through the Squid box, and vice versa, so the port 80 traffic under consideration flows through the Squid box in both ways. Now, after I do this on the client: $ telnet 192.168.0.1 80 GET / HTTP/1.0 (correct webpage output) Connection closed by foreign host. Nevertheless, in 192.168.0.1's webserver's logs I can see 192.168.0.184 connecting, not the TProxied 192.168.1.2, as if working under the plain ole interception proxying I've been trying to get rid of! Why?! Counters on the Squid box do get bumped: $ sudo iptables -t mangle -L -v -n Chain PREROUTING (policy ACCEPT 163 packets, 21851 bytes) pkts bytes target prot opt in out source destination 2274 214K DIVERT tcp -- * * 0.0.0.0/0 0.0.0.0/0 socket 16 920 TPROXY tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 TPROXY redirect 0.0.0.0:3129 mark 0x1/0x1 ... Chain DIVERT (1 references) pkts bytes target prot opt in out source destination 2274 214K MARK all -- * * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0x 2274 214K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 Thanks for any tips.
[squid-users] %{Referer} or %{Referrer} header to external helper?
Dist,
In pre 3.0 Squid versions, I used the %{Referer} header to an external
helper, I 'seem' to be having an issue with this Request Header in
3.0... just wondering if anyone else has had an issue OR can confirm
that it works?
Thanks List,
--
Louis Gonzales
BSCS EMU 2003
HP Certified Professional
louis.gonza...@linuxlouis.net
