Re: [squid-users] users bypassing rules.. Help!?
I had the same problem, and after some discussion with the experts I believe there's nothing we can do to block user for using another http-proxy-service. The best thing I can do is to add proxy phrase in my regex, it blocks many http-proxy-service. and because my ISP is in Indonesia, user will have a severe performance when they're using such http-proxy-service since most of them are located in US :-) Nyoman Bogi Aditya Karna IM Telkom http://imtelkom.ac.id --- On Sun, 7/12/09, Roland Roland r_o_l_a_...@hotmail.com wrote: From: Roland Roland r_o_l_a_...@hotmail.com Subject: [squid-users] users bypassing rules.. Help!? To: squid-users@squid-cache.org Date: Sunday, July 12, 2009, 12:21 PM Hello, for a while now.. almost 3 weeks I've been using an ACL tht matches a specific file content with url_regex in this file there's facebook, and a few other sites that I don't want users to access. users have found a way to bypass these restrictions by using online sites that supports such a thing.. like using google translate service to translate sites which by default would be blocked.. or simply using other online websites that masks such a usage... anyone has a better way for me to block such sites? thanks in advance, Roland
Re: [squid-users] squid becomes very slow during peak hours
I use Squid 2.7.STABLE4 on FreeBSD7.1 with these settings in squid.conf (explained by Amos about 3 months ago): maximum_object_size 16384 KB minimum_object_size 0 KB maximum_object_size_in_memory 512 KB cache_replacement_policy lru memory_replacement_policy lru cache_dir aufs /webcache 163840 256 256 and it works fine with hundreds of users (HIT ratio is above 75%) and yes, I also change my dedicated harddrive (for webcache) from SATA to IDE, since my problem was the Squid is rejecting all request when it's so busy calculating the cached object. hope it helps Nyoman Bogi Aditya Karna IM Telkom http://imtelkom.ac.id --- On Tue, 6/30/09, Chris Robertson crobert...@gci.net wrote: From: Chris Robertson crobert...@gci.net Subject: Re: [squid-users] squid becomes very slow during peak hours To: squid-users@squid-cache.org Date: Tuesday, June 30, 2009, 5:25 PM goody goody wrote: Hi there, I am running squid 2.5 on freebsd 7, As Adrian said, upgrade. 2.6 (and 2.7) support kqueue under FreeBSD. and my squid box respond very slow during peak hours. my squid machine have twin dual core processors, 4 ram and following hdds. Filesystem Size Used Avail Capacity Mounted on /dev/da0s1a 9.7G 241M 8.7G 3% / devfs 1.0K 1.0K 0B 100% /dev /dev/da0s1f 73G 35G 32G 52% /cache1 /dev/da0s1g 73G 2.0G 65G 3% /cache2 /dev/da0s1e 39G 2.5G 33G 7% /usr /dev/da0s1d 58G 6.4G 47G 12% /var below are the status and settings i have done. i need further guidance to improve the box. last pid: 50046; load averages: 1.02, 1.07, 1.02 up 7+20:35:29 15:21:42 26 processes: 2 running, 24 sleeping CPU states: 25.4% user, 0.0% nice, 1.3% system, 0.8% interrupt, 72.5% idle Mem: 378M Active, 1327M Inact, 192M Wired, 98M Cache, 112M Buf, 3708K Free Swap: 4096M Total, 20K Used, 4096M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 49819 sbt 1 105 0 360M 351M CPU3 3 92:43 98.14% squid 487 root 1 96 0 4372K 2052K select 0 57:00 3.47% natd 646 root 1 96 0 16032K 12192K select 3 54:28 0.00% snmpd SNIP pxy# iostat tty da0 pass0 cpu tin tout KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 126 12.79 5 0.06 0.00 0 0.00 4 0 1 0 95 pxy# vmstat procs memory page disks faults cpu r b w avm fre flt re pi po fr sr da0 pa0 in sy cs us sy id 1 3 0 458044 103268 12 0 0 0 30 5 0 0 273 1721 2553 4 1 95 Those statistics show wildly different utilization. The first (top, I assume) shows 75% idle (or a whole CPU in use). The next two show 95% idle (in effect, one CPU 20% used). How close (in time) were the statistics gathered? some lines from squid.conf cache_mem 256 MB cache_replacement_policy heap LFUDA memory_replacement_policy heap GDSF cache_swap_low 80 cache_swap_high 90 cache_dir diskd /cache2 6 16 256 Q1=72 Q2=64 cache_dir diskd /cache1 6 16 256 Q1=72 Q2=64 cache_log /var/log/squid25/cache.log cache_access_log /var/log/squid25/access.log cache_store_log none half_closed_clients off maximum_object_size 1024 KB if anyother info required, i shall provide. The types (and number) of ACLs in use would be of interest as well. Regards, .Goody. Chris
Re: [squid-users] Having trouble figuring out delay pools
(and again, if network is idle users should get more bandwidth.) Again, delay pools do not really allow for this. You'd have to look into a fair queuing algorithm for your firewall to do this properly. Delay Pools and Firewall is just a traffic shaping not a Bandwidth Management System (BMS) which can share idle bandwidth from upper class. to do this (sharing idle bandwidth) you need to use 3rd party application like CBQ (Class Based Queueing) or HTB (Hierarchial Token Bucket) and usually combination with your firewall. you may check which BMS is available for your platform, but I need to tell you this, BMS will eat up your resources (processor, memory, hard drive transaction, etc). Nyoman Bogi Aditya Karna IM Telkom http://imtelkom.ac.id
Re: [squid-users] Should i enable mikrotik bandwidth sharing or leave it to Squid?
I never play with mikrotik, but if your information is correct that mikrotik is able to share idle bandwidth than I suggest you to activate the PCQ. Delay Pools in Squid is not (CMIIW) a Bandwidth Management System which can share/borrow idle bandwidth from another class, it's simply a Traffic Shaping which is faster, easier, and not resource dependent. try to check the resource usage in mikrotik (CPU, Memory, Cache, etc) and compare it if you enable the Delay Pools in Squid. hope it helps - Nyoman Bogi Aditya Karna IM Telkom http://www.imtelkom.ac.id - --- On Sun, 7/19/09, Mark Lodge mlodg...@gmail.com wrote: From: Mark Lodge mlodg...@gmail.com Subject: [squid-users] Should i enable mikrotik bandwidth sharing or leave it to Squid? To: squid-users@squid-cache.org Date: Sunday, July 19, 2009, 9:05 PM My wireless setup is as follows Client PC --- Mikrotik-Squid Cache Server--Internet Do you suggest that I should configure PCQ (equal/fair bandwidth sharing/distribution) on the Mikrotik routerboard or should i leave the bandwidth sharing to Squid as suggested by Amos? Thanks Mark
[squid-users] Dual Link Connection
so sorry if this topic already discussed earlier, but can squid do traffic balancing between 2 or more ISP? +---+ +--| ISP 1 | +--+ +---+ | +---+ | intranet |---| Squid |--| +--+ +---+ | +---+ +--| ISP 2 | +---+ ISP-1 and ISP-2 provide static IP Public but with different subnet, and we would like the Squid automatically use both Internet link efficiently. please tell us how. thanks in advance. Nyoman Bogi Aditya Karna IM Telkom http://imtelkom.ac.id
[squid-users] transparent proxy for CONNECT method
dear squid-users, it's been long I've accepted the fact that transparent proxy will not work for CONNECT method because of security issues (considered as man-in-the-middle attack). but perhaps there's a way to get around this problem? because everyone will stuck with using GMail, YahooMail, etc since they're all using HTTPS for signing in. -- Nyoman Bogi Aditya Karna IM Telkom http://www.imtelkom.ac.id --
Re: [squid-users] delay_pools
Dear Maksim, first of all you'll need to attach your squid.conf without that we can only guess. but this is a simple example for delay pool i used, it create 2 pools, 1 for faculty (b...@32kbps) and 1 for students (b...@128kbps): acl faculty src 172.16.1.0/255.255.255.0 acl students src 172.16.0.0/255.255.224.0 delay_pools 2 delay_class 1 2 delay_class 2 2 delay_access 1 allow faculty delay_access 1 deny all delay_access 2 allow students delay_access 2 deny all delay_parameters 1 256000/256000 4000/4000 delay_parameters 2 256000/256000 16000/16000 -- Nyoman Bogi Aditya Karna IM Telkom http://www.imtelkom.ac.id -- --- On Fri, 3/27/09, Maksim Filenko maksim.file...@local-global.com.ua wrote: From: Maksim Filenko maksim.file...@local-global.com.ua Subject: [squid-users] delay_pools To: squid-users@squid-cache.org Date: Friday, March 27, 2009, 10:35 AM Hi everyone! I've stuck with shaping issues. squid.exe -v Squid Cache: Version 2.7.STABLE4 configure options: --enable-win32-service --enable-storeio='ufs aufs null coss' --enable-default-hostsfile=none --enable-removal-policies='heap lru' --enable-snmp --enable-htcp --disable-wccp --disable-wccpv2 --enable-useragent-log --enable-referer-log --enable-cache-digests --enable-auth='basic ntlm digest negotiate' --enable-basic-auth-helpers='LDAP NCSA mswin_sspi' --enable-negotiate-auth-helpers=mswin_sspi --enable-ntlm-auth-helpers='mswin_sspi fakeauth' --enable-external-acl-helpers='mswin_lm_group ldap_group' --enable-large-cache-files --enable-digest-auth-helpers='password LDAP eDirectory' --enable-forw-via-db --enable-follow-x-forwarded-for --enable-delay-pools --enable-arp-acl --prefix=c:/squid Compiled as Windows System Service. Here's what I've got in log: 2009/03/27 15:00:20| Reconfiguring Squid Cache (version 2.6.STABLE19)... 2009/03/27 15:00:20| FD 11 Closing HTTP connection 2009/03/27 15:00:20| FD 16 Closing SNMP socket 2009/03/27 15:00:20| FD 14 Closing ICP connection 2009/03/27 15:00:20| FD 15 Closing HTCP socket 2009/03/27 15:00:20| FD 17 Closing SNMP socket 2009/03/27 15:00:20| Cache dir 'c:/squid/var/cache' size remains unchanged at 102400 KB 2009/03/27 15:00:20| parseConfigFile: line 2972 unrecognized: 'delay_pools 5' 2009/03/27 15:00:20| parseConfigFile: line 2974 unrecognized: 'delay_class 1 1' 2009/03/27 15:00:20| parseConfigFile: line 2975 unrecognized: 'delay_class 2 1' 2009/03/27 15:00:20| parseConfigFile: line 2976 unrecognized: 'delay_class 3 2' 2009/03/27 15:00:20| parseConfigFile: line 2977 unrecognized: 'delay_class 4 1' 2009/03/27 15:00:20| parseConfigFile: line 2978 unrecognized: 'delay_class 5 1' 2009/03/27 15:00:20| parseConfigFile: line 2980 unrecognized: 'delay_access 1 allow media' 2009/03/27 15:00:20| parseConfigFile: line 2981 unrecognized: 'delay_access 1 deny all' 2009/03/27 15:00:20| parseConfigFile: line 2982 unrecognized: 'delay_access 2 allow leechers' 2009/03/27 15:00:20| parseConfigFile: line 2983 unrecognized: 'delay_access 2 deny all' 2009/03/27 15:00:20| parseConfigFile: line 2984 unrecognized: 'delay_access 3 allow limited' 2009/03/27 15:00:20| parseConfigFile: line 2985 unrecognized: 'delay_access 3 deny all' 2009/03/27 15:00:20| parseConfigFile: line 2986 unrecognized: 'delay_access 4 allow office_net' 2009/03/27 15:00:20| parseConfigFile: line 2987 unrecognized: 'delay_access 4 deny all' 2009/03/27 15:00:20| parseConfigFile: line 2988 unrecognized: 'delay_access 5 allow unlim_ip' 2009/03/27 15:00:20| parseConfigFile: line 2989 unrecognized: 'delay_access 5 deny all' 2009/03/27 15:00:20| parseConfigFile: line 2991 unrecognized: 'delay_parameters 1 16000/16000' 2009/03/27 15:00:20| parseConfigFile: line 2992 unrecognized: 'delay_parameters 2 16000/16000' 2009/03/27 15:00:20| parseConfigFile: line 2993 unrecognized: 'delay_parameters 3 32000/32000 8000/8000' 2009/03/27 15:00:20| parseConfigFile: line 2994 unrecognized: 'delay_parameters 4 128000/128000 # 1Mbit to all' 2009/03/27 15:00:20| parseConfigFile: line 2995 unrecognized: 'delay_parameters 5 -1/-1 # 2Mbit to all' 2009/03/27 15:00:20| User-Agent logging is disabled. 2009/03/27 15:00:20| Referer logging is disabled. 2009/03/27 15:00:20| DNS Socket created at 0.0.0.0, port 48960, FD 10
Re: [squid-users] access to cache is very high
thanks for the advice Amos but I've just reset all objects in my cache (thus it's empty now) and the CPU utilization has decreased dramatically (up to 17%), but clients still get (rarely) error that the proxy is refusing connection. 2009/03/22 13:34:42| httpAccept: FD 8: accept failure: (53) Software caused connection abort 2009/03/22 15:16:49| httpAccept: FD 8: accept failure: (53) Software caused connection abort I'll try your advice in my testlab first before go into live server. thanks in advance. --- On Fri, 3/20/09, Amos Jeffries squ...@treenet.co.nz wrote: From: Amos Jeffries squ...@treenet.co.nz Subject: Re: [squid-users] access to cache is very high To: nyoman karna balique8...@yahoo.com Cc: squid-users@squid-cache.org Date: Friday, March 20, 2009, 10:19 PM nyoman karna wrote: guys, I've been using squid for 5 years in my educational institution (IM Telkom - Telkom Institute of Management) but this is the first time i encounter this problem. need your expertise. I'm using HP Proliant ML110 G5 with 320GB of SATA disk. I'm using squid 2.6 stable 16 (FreeBSD 6.3) with 100GB of squid-cache. when the cache was not yet full, there's no problem whatsoever but when the cache is full (but still 90% of the mountpoint), squid is always busy validating its objects and refusing all client connection. this is part of file /usr/local/etc/squid/squid.conf maximum_object_size 16384 KB minimum_object_size 0 KB maximum_object_size_in_memory 8 KB cache_replacement_policy lru memory_replacement_policy lru cache_dir ufs /webcache 10 64 256 this is the capture when the squid refusing connection -- (using top) -- CPU states: 39.5% user, 0.0% nice, 55.3% system, 5.3% interrupt, 0.0% idle PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 77059 squid 1 123 0 186M 184M RUN 0:10 86.26% squid -- (using systat) -- /0% /10 /20 /30 /40 /50 /60 /70 ad4 MB/sX tps|XX -- (cache.log) -- 2009/03/20 09:54:44| httpAccept: FD 11: accept failure: (53) Software caused connection abort UFS as the slowest IO process in Squid, is not the best for large caches. Since you are on FreeBSD try diskd instead. No change of cache is needed to alter ufs-diskd as the storage manager type. Average HTTP object size these days is between 64KB and 128 KB. Your max in-memory size of 8KB is causing a lot of objects to be disk-saved without need. Also check the garbage collection range you have set: http://www.squid-cache.org/Doc/config/cache_swap_low/ http://www.squid-cache.org/Doc/config/cache_swap_high/ Amos -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE13 Current Beta Squid 3.1.0.6
[squid-users] access to cache is very high
guys, I've been using squid for 5 years in my educational institution (IM Telkom - Telkom Institute of Management) but this is the first time i encounter this problem. need your expertise. I'm using HP Proliant ML110 G5 with 320GB of SATA disk. I'm using squid 2.6 stable 16 (FreeBSD 6.3) with 100GB of squid-cache. when the cache was not yet full, there's no problem whatsoever but when the cache is full (but still 90% of the mountpoint), squid is always busy validating its objects and refusing all client connection. this is part of file /usr/local/etc/squid/squid.conf maximum_object_size 16384 KB minimum_object_size 0 KB maximum_object_size_in_memory 8 KB cache_replacement_policy lru memory_replacement_policy lru cache_dir ufs /webcache 10 64 256 this is the capture when the squid refusing connection -- (using top) -- CPU states: 39.5% user, 0.0% nice, 55.3% system, 5.3% interrupt, 0.0% idle PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 77059 squid 1 1230 186M 184M RUN 0:10 86.26% squid -- (using systat) -- /0%/10/20/30/40/50/60/70 ad4 MB/sX tps|XX -- (cache.log) -- 2009/03/20 09:54:44| httpAccept: FD 11: accept failure: (53) Software caused connection abort
