Re: [squid-users] CDN / JS 503 Service Unavailable
Hello Yes we tried disabling ufdbguard. I first suspected it was doing something with proxy tunnels but it made no difference. The dev has moved the .js file to another site so theres no pressure. I'll look at spinning up a basic squid and work through the config, see if i can identify whats or where the issue is. Thanks for your help On 19 August 2014 01:30, Eliezer Croitoru elie...@ngtech.co.il wrote: On 08/18/2014 10:37 AM, Paul Regan wrote: @Eliezer - Sorry to say the acl lines made no difference. Can I use any of the debugging options to get deeper into this? Well it depends on the 503 content. it can be a network issue or an application level issue. Since you can download the file using wget from the proxy machine it seems like a proxy settings error but there is an option that you are using some wrong settings. For me and many others it do work with proper proxy settings. Have you tried to remove the ufdbguard from the settings? It might be the reason.. Eliezer
Re: [squid-users] CDN / JS 503 Service Unavailable
Hello @Eliezer - Sorry to say the acl lines made no difference. Can I use any of the debugging options to get deeper into this? @Amos - Maybe my British sarcasm was lost on an international audience ;) ... This config is an organic one which in parts has existed through a number of SA and versions so thanks for the audit and we will take a look at each suggestion. Re performance. Only 200 (ish) users and no issues reported. Thats not to say it can't be improved and people accept what they have as the norm. The only problem we have (that we know about!) is the .js caching of this cloudflare site. Paul On 17 August 2014 07:03, Amos Jeffries squ...@treenet.co.nz wrote: On 15/08/2014 11:22 p.m., Paul Regan wrote: Urg, thats like standing front of the class for everyone to stare! If you are not able to take constructive criticisms, sysadmin is probably not the best ine of work for you :-) I see you seem to have found the problem. So consider these a free audit. here you go : cache_effective_user squid url_rewrite_program /usr/sbin/ufdbgclient -l /var/ufdbguard/logs url_rewrite_children 64 acl localnet src subnet acl eu-edge-IP src subnet acl eu-buscon-edge-IP src subnet acl eu-inet-dmz src subnet acl na-subnet src subnet acl na-inet-dmz src subnet acl na-buscon-edge-IP src subnet acl st-buscon-vpc src subnet acl eu-mfmgt src subnet acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT hosts_file /etc/hosts dns_nameservers server server server http_access deny !Safe_ports http_access deny CONNECT !SSL_ports acl infrastructure src http_access allow localhost manager http_access allow infrastructure manager http_access deny manager acl mo-whitelist dstdomain /etc/squid/mo-whitelist http_access allow mo-whitelist acl mo-blockedsites dstdomain /etc/squid/mo-blockedsites deny_info http://urlrestricted_content_blockedsites.html mo-blockedsites http_access deny mo-blockedsites acl mo-blockedkeywords urlpath_regex /etc/squid/mo-blockedkeywords deny_info http://urlrestricted_content_keywords.html mo-blockedkeywords http_access deny mo-blockedkeywords acl mo-nocache dstdomain /etc/squid/mo-nocache no_cache deny mo-nocache The correct name for that directive is cache, has been since Squid-2.4. As in, what you should have there is: cache deny mo-nocache acl mo-blockedIP src /etc/squid/mo-blockedIP acl mo-allowURLs dstdomain src /etc/squid/mo-allowURLs http_access allow mo-blockedIP mo-allowURLs http_access deny mo-blockedIP deny_info http://urlrestricted_content_blockedip.html mo-blockedIP acl mo-allowNYIP src /etc/squid/mo-allowNYIP http_access allow mo-allowNYIP http_access allow na-subnet mo-allowURLs http_access deny na-subnet deny_info http://urlrestricted_content_subnet.html na-subnet http_access allow localnet http_access deny st-buscon-vpc http_access allow eu-edge-IP http_access allow eu-inet-dmz http_access allow eu-buscon-edge-IP http_access allow na-inet-dmz http_access allow na-buscon-edge-IP http_access allow eu-mfmgt acl ftp proto FTP always_direct allow ftp acl purge method PURGE http_access allow purge localhost http_access deny purge Hmm.. What you have here is a pure forward-proxy configuration. If you need to purge things from the cache of a forward-proxy then it is caching badly/wrong. I know that Squid does cache some things badly, but we have taken great pains to ensure that those cases are conservative. The wrong cases shoudl all take form of dropping things which should have been kept, rather than storing things which should have been dropped. Are you perhase finding that you need to manually erase content permitted into cache by the refresh rules with override-expire ignore-no-store ignore-private. Ignoring private and no-store in particular are very dangerous... think Captcha images, username in image form for embeded session display, company private information, etc. http_access allow localhost http_access deny all http_port 8080 cache_dir aufs /squid-cache 39322 16 256 cache_replacement_policy heap LFUDA cache_swap_low 96 cache_swap_high 98 cache_mem 256 MB maximum_object_size 64 KB It's a little unclear why you are limiting cached objects to 64KB while refresh patterns also force archive and binary executable types to be cached. You have 40.25 GB of cache space available. maximum_object_size_in_memory 20 KB quick_abort_min 0 KB quick_abort_max 0 KB memory_pools off Have you tested performance with these on recently? HTH Amos
Re: [squid-users] CDN / JS 503 Service Unavailable
Hi .. Yes, works fine. On 15 August 2014 01:53, Eliezer Croitoru elie...@ngtech.co.il wrote: Have you tried to run wget on the proxy itself without any proxy settings? Eliezer On 08/14/2014 01:22 PM, Paul Regan wrote: Proxy tunneling failed: Service Unavailable Unable to establish SSL connection. If I put the script on another non site everything works fine. So this seems more like an issue with the CDN provider, and maybe Squid. But its got me stumped and my GoogleFu is failing so putting it out there to see if anyone has any clever ideas. Paul
Re: [squid-users] CDN / JS 503 Service Unavailable
Urg, thats like standing front of the class for everyone to stare! here you go : cache_effective_user squid url_rewrite_program /usr/sbin/ufdbgclient -l /var/ufdbguard/logs url_rewrite_children 64 acl localnet src subnet acl eu-edge-IP src subnet acl eu-buscon-edge-IP src subnet acl eu-inet-dmz src subnet acl na-subnet src subnet acl na-inet-dmz src subnet acl na-buscon-edge-IP src subnet acl st-buscon-vpc src subnet acl eu-mfmgt src subnet acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT hosts_file /etc/hosts dns_nameservers server server server http_access deny !Safe_ports http_access deny CONNECT !SSL_ports acl infrastructure src http_access allow localhost manager http_access allow infrastructure manager http_access deny manager acl mo-whitelist dstdomain /etc/squid/mo-whitelist http_access allow mo-whitelist acl mo-blockedsites dstdomain /etc/squid/mo-blockedsites deny_info http://urlrestricted_content_blockedsites.html mo-blockedsites http_access deny mo-blockedsites acl mo-blockedkeywords urlpath_regex /etc/squid/mo-blockedkeywords deny_info http://urlrestricted_content_keywords.html mo-blockedkeywords http_access deny mo-blockedkeywords acl mo-nocache dstdomain /etc/squid/mo-nocache no_cache deny mo-nocache acl mo-blockedIP src /etc/squid/mo-blockedIP acl mo-allowURLs dstdomain src /etc/squid/mo-allowURLs http_access allow mo-blockedIP mo-allowURLs http_access deny mo-blockedIP deny_info http://urlrestricted_content_blockedip.html mo-blockedIP acl mo-allowNYIP src /etc/squid/mo-allowNYIP http_access allow mo-allowNYIP http_access allow na-subnet mo-allowURLs http_access deny na-subnet deny_info http://urlrestricted_content_subnet.html na-subnet http_access allow localnet http_access deny st-buscon-vpc http_access allow eu-edge-IP http_access allow eu-inet-dmz http_access allow eu-buscon-edge-IP http_access allow na-inet-dmz http_access allow na-buscon-edge-IP http_access allow eu-mfmgt acl ftp proto FTP always_direct allow ftp acl purge method PURGE http_access allow purge localhost http_access deny purge http_access allow localhost http_access deny all http_port 8080 cache_dir aufs /squid-cache 39322 16 256 cache_replacement_policy heap LFUDA cache_swap_low 96 cache_swap_high 98 cache_mem 256 MB maximum_object_size 64 KB maximum_object_size_in_memory 20 KB quick_abort_min 0 KB quick_abort_max 0 KB memory_pools off client_db off half_closed_clients off buffered_logs on logfile_rotate 21 coredump_dir /squid-cache refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 10080 90% 43200 override-expire ignore-no-store ignore-private refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90% 432000 override-expire ignore-no-store ignore-private refresh_pattern -i \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|pptx|doc|docx|tiff)$ 10080 90% 43200 override-expire ignore-no-store ignore-private refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 ftp_user ftpuser@domain.com ftp_passive off ftp_sanitycheck off ftp_epsv off ftp_eprt off On 15 August 2014 11:24, Eliezer Croitoru elie...@ngtech.co.il wrote: On 08/15/2014 11:50 AM, Rafael Akchurin wrote: Hello Paul, I presume you are using SSL bump? Raf. squid.conf is our friend, care to share it Paul? In any case it seems like the service is unavailable to the proxy settings. Eliezer
[squid-users] CDN / JS 503 Service Unavailable
Dev is trying to use a remote script in their page : https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.9.1/jquery-ui.min.js Which is failing through the proxy (3.3.6) w/ a 503 Service Unavailable, either in code browser and cli. Nothing obvious in the squid logs, I see the connection made. 1408011292.226 11 10.2.76.1 TCP_MISS/503 0 CONNECT cdnjs.cloudflare.com:443 - HIER_NONE/- - a wget debug : ---request end--- proxy responded with: [HTTP/1.1 503 Service Unavailable Server: squid/3.3.6 Mime-Version: 1.0 Date: Thu, 14 Aug 2014 10:14:52 GMT Content-Type: text/html Content-Length: 3115 X-Squid-Error: ERR_CONNECT_FAIL 101 Vary: Accept-Language Content-Language: en Proxy tunneling failed: Service Unavailable Unable to establish SSL connection. If I put the script on another non site everything works fine. So this seems more like an issue with the CDN provider, and maybe Squid. But its got me stumped and my GoogleFu is failing so putting it out there to see if anyone has any clever ideas. Paul
Re: [squid-users] 3.1.16 FTP problems ..
Thanks for the reply, will open the bugzilla .. Some further info is I just tried the same using a 3.0 stable13 server with the same results. I haven't run a wire trace but cache.log reports the same error : ftpDataRead: read error: (107) Transport endpoint is not connected Paul On 7 November 2011 22:37, Amos Jeffries squ...@treenet.co.nz wrote: On Mon, 7 Nov 2011 14:37:24 +, Paul Regan wrote: New install of 3.1.16, FTP appears to be configured correctly but hence the mail group post its not working as expected .. Using ftp.emea.ibm.com as the test destination. Active FTP from proxy server command line works fine. Via a browser squidclient The following error was encountered while trying to retrieve the URL: ftp://ftp.emea.ibm.com/ Read Error The system returned: [No Error] An error condition occurred while reading data from the network. Please retry your request. cache.log : 2011/11/07 12:45:20| ftpDataRead: read error: (107) Transport endpoint is not connected Wire capture : 12:45:20.395751 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: S 2543509559:2543509559(0) win 5840 mss 1460,sackOK,timestamp 853154020 0,nop,wscale 7 12:45:20.414004 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: S 429684985:429684985(0) ack 2543509560 win 65535 mss 1380,nop,wscale 5 12:45:20.414012 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: . ack 1 win 46 12:45:20.444260 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 1:78(77) ack 1 win 16384 snip Lots of packets on control channel. Looks normal. The handshake, login, file location etc. happen about here. 12:45:20.608768 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P ack 78 win 16384 12:45:20.818002 IP ftp.emea.ibm.com.ftp-data proxyserver.domain.36703: S 3997244124:3997244124(0) win 65535 mss 1380,nop,wscale 5 Server connects in with a data connection. You configured ftp_passive OFF. So this is expected. 12:45:20.818022 IP proxyserver.domain.36703 ftp.emea.ibm.com.ftp-data: S 2004820984:2004820984(0) ack 3997244125 win 5840 mss 1460,nop,wscale 7 12:45:20.835744 IP ftp.emea.ibm.com.ftp-data proxyserver.domain.36703: . ack 1 win 11520 12:45:20.835804 IP ftp.emea.ibm.com.ftp-data proxyserver.domain.36703: P 1:692(691) ack 1 win 11520 12:45:20.835809 IP proxyserver.domain.36703 ftp.emea.ibm.com.ftp-data: . ack 692 win 57 12:45:20.835812 IP ftp.emea.ibm.com.ftp-data proxyserver.domain.36703: FP 692:692(0) ack 1 win 11520 FIN-ACK on data channel from server after 692 bytes transferred. 12:45:20.835899 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 1108:1129(21) ack 78 win 16384 12:45:20.836193 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: F 78:78(0) ack 1129 win 63 FIN on the control channel from Squid. Data file is received and connections can close. 12:45:20.836210 IP proxyserver.domain.36703 ftp.emea.ibm.com.ftp-data: R 1:1(0) ack 693 win 57 12:45:20.853549 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P ack 79 win 16384 12:45:20.853576 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 1129:1163(34) ack 79 win 16384 Server keeps throwing data at Squid down the control channel after FIN. 12:45:20.853586 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: R 2543509638:2543509638(0) win 0 Squid responds to the extra data with RST. 12:45:20.853603 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: FP 1163:1163(0) ack 79 win 16384 12:45:20.853606 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: R 2543509638:2543509638(0) win 0 More packets from the server. More RST from Squid. Only 17.41 ms have elapsed, so its within bounds of network lag. Overall this looks like proper TCP closing sequence. Except that the data channel is closed first. Before any messages on the control channel arrive to inform Squid that the transfer is completed. Can you report this with the above details in bugzilla please so this does not get lost. I think we could work around it by checking the expected object size on data closure and skip the error if they match. Amos
[squid-users] 3.1.16 FTP problems ..
New install of 3.1.16, FTP appears to be configured correctly but hence the mail group post its not working as expected .. Using ftp.emea.ibm.com as the test destination. Active FTP from proxy server command line works fine. Via a browser squidclient The following error was encountered while trying to retrieve the URL: ftp://ftp.emea.ibm.com/ Read Error The system returned: [No Error] An error condition occurred while reading data from the network. Please retry your request. cache.log : 2011/11/07 12:45:20| ftpDataRead: read error: (107) Transport endpoint is not connected Wire capture : 12:45:20.395751 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: S 2543509559:2543509559(0) win 5840 mss 1460,sackOK,timestamp 853154020 0,nop,wscale 7 12:45:20.414004 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: S 429684985:429684985(0) ack 2543509560 win 65535 mss 1380,nop,wscale 5 12:45:20.414012 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: . ack 1 win 46 12:45:20.444260 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 1:78(77) ack 1 win 16384 12:45:20.444290 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: . ack 78 win 46 12:45:20.444295 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 78:145(67) ack 1 win 16384 12:45:20.444299 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: . ack 145 win 46 12:45:20.462391 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 145:595(450) ack 1 win 16384 12:45:20.462396 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: . ack 595 win 54 12:45:20.462464 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: P 1:17(16) ack 595 win 54 12:45:20.480374 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P ack 17 win 16384 12:45:20.517610 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 595:639(44) ack 17 win 16384 12:45:20.517667 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: P 17:41(24) ack 639 win 54 12:45:20.535681 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P ack 41 win 16384 12:45:20.536375 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 639:699(60) ack 41 win 16384 12:45:20.536392 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 699:721(22) ack 41 win 16384 12:45:20.536405 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: . ack 721 win 54 12:45:20.554833 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 721:1043(322) ack 41 win 16384 12:45:20.554873 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: P 41:49(8) ack 1043 win 63 12:45:20.572753 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P ack 49 win 16384 12:45:20.572777 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 1043:1086(43) ack 49 win 16384 12:45:20.572838 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: P 49:72(23) ack 1086 win 63 12:45:20.591096 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P ack 72 win 16384 12:45:20.591100 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 1086:1108(22) ack 72 win 16384 12:45:20.591130 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: P 72:78(6) ack 1108 win 63 12:45:20.608768 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P ack 78 win 16384 12:45:20.818002 IP ftp.emea.ibm.com.ftp-data proxyserver.domain.36703: S 3997244124:3997244124(0) win 65535 mss 1380,nop,wscale 5 12:45:20.818022 IP proxyserver.domain.36703 ftp.emea.ibm.com.ftp-data: S 2004820984:2004820984(0) ack 3997244125 win 5840 mss 1460,nop,wscale 7 12:45:20.835744 IP ftp.emea.ibm.com.ftp-data proxyserver.domain.36703: . ack 1 win 11520 12:45:20.835804 IP ftp.emea.ibm.com.ftp-data proxyserver.domain.36703: P 1:692(691) ack 1 win 11520 12:45:20.835809 IP proxyserver.domain.36703 ftp.emea.ibm.com.ftp-data: . ack 692 win 57 12:45:20.835812 IP ftp.emea.ibm.com.ftp-data proxyserver.domain.36703: FP 692:692(0) ack 1 win 11520 12:45:20.835899 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 1108:1129(21) ack 78 win 16384 12:45:20.836193 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: F 78:78(0) ack 1129 win 63 12:45:20.836210 IP proxyserver.domain.36703 ftp.emea.ibm.com.ftp-data: R 1:1(0) ack 693 win 57 12:45:20.853549 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P ack 79 win 16384 12:45:20.853576 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: P 1129:1163(34) ack 79 win 16384 12:45:20.853586 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: R 2543509638:2543509638(0) win 0 12:45:20.853603 IP ftp.emea.ibm.com.ftp proxyserver.domain.60769: FP 1163:1163(0) ack 79 win 16384 12:45:20.853606 IP proxyserver.domain.60769 ftp.emea.ibm.com.ftp: R 2543509638:2543509638(0) win 0 Its not so obvious from the tcpdump (easier on wireshark) but the 3rd packet from bottom is a [RST] from the proxy server. Comparing that to a capture during an FTP connection from the command line (same server) then that equiv packet is an [ACK] - I can make the pcap file available if need be .. I dont know if thats the actual issue but its a difference between a failure and a success which I've homed in on otherwise I'm spinning my wheels on this problem
