[squid-users] [PATCH] Squid 3.1.4 not building under OpenBSD 4.6
Hello!
Yesterday i tried to build the new version of Squid on OpenBSD 4.6.
Unfortunately there's some problem in src/mem.cc. new_pool_limit is of
type size_t which is a long unsigned int. In line 359 a -1 is written
into new_pool_limit. Newer g++ versions seems to accept this as valid
but OpenBSD's g++ 3.3.5 complains about it.
Attached is a small patch to fix this.
PS: Your bugzilla site seems to be down at moment. I'm just getting some
Perl error messages.
-- Matthias
gcc of OpenBSD does not like the original assignment.
Newer versions of gcc seem to have no problem.
The explicit cast fixes this error for gcc 3.3.5 in OpenBSD
--- src/mem.cc.orig Tue Jun 1 17:24:03 2010
+++ src/mem.cc Tue Jun 1 17:24:22 2010
@@ -356,7 +356,7 @@
else {
if (Config.MemPools.limit == 0)
debugs(13, 1, "memory_pools_limit 0 has been chagned to
memory_pools_limit none. Please update your config");
-new_pool_limit = -1;
+new_pool_limit = static_cast(-1);
}
#if 0
[squid-users] Squid 3.1.x uses considerably more memory than 2.7.x or 3.0.x
Hi guys. I'm using squid 3.1.4 and since of first 3.1 stable release I'm suffering from memory leak problems too. My production server is a CentOS 5.4 32 bits (kernel 2.6.18-164.6.1.el5). It has 2GB of RAM (512MB reserved to squid cache_mem) and 6GB of disk space reserved to squid's disk cache. In approximately 24h, squid consumes all the available memory of my server and swap starts to be consumed. I'm currently restarting squid once a day in order to workaround the problem. Follow attached a squidclient output made some hours before all the server memory be consumed. HTTP/1.0 200 OK Server: squid Mime-Version: 1.0 Date: Wed, 02 Jun 2010 02:22:12 GMT Content-Type: text/plain Expires: Wed, 02 Jun 2010 02:22:12 GMT Last-Modified: Wed, 02 Jun 2010 02:22:12 GMT X-Cache: MISS from proxyweb.ipasgo.go.gov.br Via: 1.0 proxyweb.ipasgo.go.gov.br (squid) Proxy-Connection: close Squid Object Cache: Version 3.1.4 Start Time: Tue, 01 Jun 2010 02:59:02 GMT Current Time: Wed, 02 Jun 2010 02:22:12 GMT Connection information for squid: Number of clients accessing cache: 1 Number of HTTP requests received: 1607991 Number of ICP messages received:0 Number of ICP messages sent:0 Number of queued ICP replies: 0 Number of HTCP messages received: 0 Number of HTCP messages sent: 0 Request failure ratio: 0.00 Average HTTP requests per minute since start: 1146.0 Average ICP messages per minute since start:0.0 Select loop called: 21704724 times, 3.879 ms avg Cache information for squid: Hits as % of all requests: 5min: 1.3%, 60min: 2.5% Hits as % of bytes sent:5min: 99.3%, 60min: 93.0% Memory hits as % of hit requests: 5min: 100.0%, 60min: 100.0% Disk hits as % of hit requests: 5min: 0.0%, 60min: 0.0% Storage Swap size: 5662188 KB Storage Swap capacity: 90.0% used, 10.0% free Storage Mem size: 420444 KB Storage Mem capacity: 80.7% used, 19.3% free Mean Object Size: 21.29 KB Requests given to unlinkd: 0 Median Service Times (seconds) 5 min60 min: HTTP Requests (All): 0.00463 0.00463 Cache Misses: 0.00307 0.28853 Cache Hits:0.0 0.0 Near Hits: 0.0 0.0 Not-Modified Replies: 0.0 0.0 DNS Lookups: 0.00860 0.00860 ICP Queries: 0.0 0.0 Resource usage for squid: UP Time:84190.610 seconds CPU Time: 2395.902 seconds CPU Usage: 2.85% CPU Usage, 5 minute avg:0.05% CPU Usage, 60 minute avg: 0.05% Process Data Segment Size via sbrk(): 1081804 KB Maximum Resident Size: 0 KB Page faults with physical i/o: 2522 Memory usage for squid via mallinfo(): Total space in arena: 1081936 KB Ordinary blocks: 1079110 KB 7695 blks Small blocks: 0 KB 0 blks Holding blocks: 3492 KB 7 blks Free Small blocks: 0 KB Free Ordinary blocks:2825 KB Total in use: 1082602 KB 100% Total free: 2825 KB 0% Total size:1085428 KB Memory accounted for: Total accounted: 550778 KB 51% memPool accounted: 550778 KB 51% memPool unaccounted: 534649 KB 49% memPoolAlloc calls: 307930462 memPoolFree calls: 308780650 File descriptor usage for squid: Maximum number of file descriptors: 4096 Largest file desc currently in use: 92 Number of file desc currently in use: 46 Files queued for open: 0 Available number of file descriptors: 4050 Reserved number of file descriptors: 100 Store Disk files open: 0 Internal Data Structures: 266030 StoreEntries 57596 StoreEntries with MemObjects 57595 Hot Object Cache Items 266004 on-disk objects
Re: [squid-users] interesting challenge
Le mardi 1 juin 2010 21:04:06, Amos Jeffries a écrit : > On Tue, 1 Jun 2010 19:35:47 -0500, Luis Daniel Lucio Quiroz > wrote: > > http://www.appliansys.com/products/cachebox/caching_appliances_video_cachin > g.html > > > i wonder if this is possible in 3.1 > > "So, what have we done? [...] Created custom extensions to Squid, the > caching engine used by CACHEBOX,"... > > 3.1 lacks the storeurl_rewrite feature which makes this setup easy. > A pre-cache ICAP processor could alter the requested URL to something > local and have the same effect in a more complex way. > > Anyone want to assist with porting the storeurl_rewrite code to Squid-3 ? > > Amos icap then cause i dont have c++ enoguht skills just to confirm, with icap i need to set in a standar url so squid may cache them in one common url
Re: [squid-users] interesting challenge
On Tue, 1 Jun 2010 19:35:47 -0500, Luis Daniel Lucio Quiroz wrote: > http://www.appliansys.com/products/cachebox/caching_appliances_video_caching.html > > i wonder if this is possible in 3.1 "So, what have we done? [...] Created custom extensions to Squid, the caching engine used by CACHEBOX,"... 3.1 lacks the storeurl_rewrite feature which makes this setup easy. A pre-cache ICAP processor could alter the requested URL to something local and have the same effect in a more complex way. Anyone want to assist with porting the storeurl_rewrite code to Squid-3 ? Amos
Re: [squid-users] Problem: parseHttpRequest: Requestheader contains NULL characters
On Tue, 1 Jun 2010 13:07:28 -0400, Pedro Silva wrote: > Hi, > > In my cache.log following lines appear. > > 2010/06/01 12:11:10| parseHttpRequest: Requestheader contains NULL > characters > 2010/06/01 12:11:10| parseHttpRequest: Unsupported method ' ' > 2010/06/01 12:11:10| clientReadRequest: FD 131 (192.168.0.14:54629) > Invalid Request > > These appear to see a site that does streaming. Looking for > information should I add or modify > the line in my squid.conf extension_methods. The problem is not > knowing what to add parameters > to accept streaming audio / video. > > Can that be a problem?. Can you help me solve this problem? For most streaming it is not possible to solve in Squid. The problem is non-HTTP protocol garbage being pushed into port 80. You need to get the client agents configured to use the proxy properly or to use their native protocol ports and not port 80. The exception being SHOUTcast streaming which natively sends an almost-HTTP protocol over port 80. Support has been added to Squid-3.1, with an early experimental support also in Squid-2.7 via the upgrade_http0.9 directive. Amos
[squid-users] interesting challenge
http://www.appliansys.com/products/cachebox/caching_appliances_video_caching.html i wonder if this is possible in 3.1
Re: [squid-users] FTP authenticated using explorer 8
On Tue, 01 Jun 2010 17:21:58 +0200, Henrik Nordström wrote: > tis 2010-06-01 klockan 15:12 +0200 skrev Luciano Mannucci: >> Hello, >> >> a user of mine is complaining that with IE 8 he cannot open URLS like >> ftp://u...@site.tld though my cache (the same works quite well with >> chrome) on his Windows seven PC. > > The official standard is ftp://user:passw...@host/ > > How well browsers supports other forms varies greatly between browsers. > Also varies between Squid. 3.1+ will run the HTTP authentication stuff for Basic auth when FTP needs a login and one is not provided as above. Amos
Re: [squid-users] creat a groupe in squid and squidGuard
On Tue, 1 Jun 2010 12:05:46 -0700 (PDT), squidACL wrote: > Good day, > > I have squid and squidGuard installed with domaine controller , i need to > create groups in this (some groups have access to intenet the other no) I > need to use the groups and users that i have in my active directory > > it s possible to do this? > > I will be thankful if you can help me . http://wiki.squid-cache.org/SquidFaq/SquidAcl#Using_Proxy_Authentication references ... http://wiki.squid-cache.org/Features/Authentication references ... http://wiki.squid-cache.org/ConfigExamples/Authenticate/NtlmWithGroups Amos
Re: [squid-users] SQUID makes few slow connections (see video)
On Tue, 1 Jun 2010 23:50:35 +0200, Tytus Rogalewski wrote: > I thik that this is something to do it with many requests, because > half of images are loaded imiedietly (with hits MISS) , and rest are > loaded after few seconds (also MISSED) - missed because i have erased > cache. and is page is cached - it works fine. Only NEW requests are > working slowly Sounds like normal network delays. Horribly long though one there being 12s. As asked earlier are you forward-proxying or reverse-proxying with this Squid? ie, are you providing access to a local web server where these pages are hosted? or just trying to get internal clients good access to the external site? >> === >> SQUID SETTINGS >> >> http_port 8123 transparent >> cache_mgr tytan...@gmail.com >> visible_hostname sky-link.net >> error_directory /usr/share/squid/errors/Polish_clean >> >> cache_mem 1500 MB # ilosc ramu >> memory_pools on >> memory_pools_limit 100 MB >> cache_swap_low 94 >> cache_swap_high 98 >> fqdncache_size 51200 >> ipcache_size 51200 >> ipcache_low 90 >> ipcache_high 95 >> pipeline_prefetch on mumble. This _might_ be affecting lag time since pipelining accepts a second request and starts it's timers before the first request has finished and the second can actually start being handled. Squid will still be able to handle a small number of client-pipelined requests with this turned off. >> minimum_object_size 0 KB >> maximum_object_size 5 KB >> maximum_object_size_in_memory 512 KB >> half_closed_clients on >> shutdown_lifetime 2 seconds >> #ident_timeout 1 seconds >> cache_replacement_policy heap LFUDA >> memory_replacement_policy heap lru >> #cache_dir aufs /mnt/dysk/squid 10 256 256 >> cache_dir aufs /mnt/dysk/squid 2 32 256 >> #quick_abort_min 200 KB >> #quick_abort_max 200 KB >> quick_abort_min 0 KB >> quick_abort_max 0 KB >> >> ### OPCJE ### >> icp_port 0 >> htcp_port 0 >> icp_access deny all >> htcp_access deny all >> snmp_port 0 >> snmp_access deny all >> >> ### LOGI ### >> access_log /var/log/squid/access.log >> cache_log /var/log/squid/cache.log >> cache_store_log /var/log/squid/store.log store.log is not very useful for most and wastes disk IO resources. You can probably set this to: cache_store_log none >> log_fqdn on # pelne nazwy domen w logach Forces rDNS lookups to log things. This can add lag. >> #debug_options ALL,1 >> >> ie_refresh on >> cache_effective_user squid >> cache_effective_group squid >> negative_ttl 1 minutes >> positive_dns_ttl 24 hours >> negative_dns_ttl 1 minutes Storing DNS results for longer than the authoritive website allows can lead you to the lag problem, if the web server IP has changed and Squid struggles to find a bad route to the old IP. >> >> ## ACL >> acl manager proto cache_object >> acl localhost src 127.0.0.1/255.255.255.255 >> acl sky-link src 192.168.0.0/255.255.255.0 #Nasza LAN >> acl SSL_ports port 443 563 >> acl Safe_ports port 21 80 442 563 70 210 280 488 591 777 1025-65535 >> acl CONNECT method CONNECT >> #acl MCONN maxconn 300 >> # Youtube ACL >> acl youtube_domains >> dstdomain .youtube.com .googlevideo.com .ytimg.com .wrzuta.pl >> >> ### ADL Blokada Reklam ### >> acl allowlist_urlregex url_regex "/etc/squid/acl/ >> allowlist_urlregex.acl" >> acl allowlist_dstdomain dstdomain "/etc/squid/acl/ >> allowlist_dstdomain.acl" >> acl denylist_urlregex url_regex "/etc/squid/acl/denylist_urlregex.acl" >> acl denylist_dstdomain dstdomain "/etc/squid/acl/ >> denylist_dstdomain.acl" >> >> ## HTTP ACCESS >> http_access allow allowlist_dstdomain >> http_access allow allowlist_urlregex >> http_access deny denylist_urlregex >> http_access deny denylist_dstdomain Optimization Tip #1: place regex rules as late as possible. Meaning: http_access deny denylist_dstdomain http_access deny denylist_urlregex >> http_access allow sky-link >> http_access allow manager localhost >> http_access deny manager >> http_access deny CONNECT !SSL_ports >> http_access deny !Safe_ports >> http_access deny youtube_domains You access lists can be re-ordered to gain a fair bit of speed on some requests. Not sure if its related to the ones showing you problems though... # These rules are the security boundaries preventing nasty abusive behaviour # by your clients intentionally or when infected. # They also form the second level of DDoS protection for your Squid. http_access allow manager localhost http_access deny manager http_access deny CONNECT !SSL_ports http_access deny !Safe_ports I have to assume that since you are using "transparent" option, your proxy is forward-only. (It's a giant security breach of your network to do reverse-proxy that way.) http_access deny !sky-link http_access allow allowlist_dstdomain http_access allow allowlist_urlregex http_access deny denylist_dstdomain http_access deny denylist_urlregex http_access allow sky-link http_access deny all Note how the youtube rule is now missing, it was not hav
Re: [squid-users] rtmp protocol in squid is possible ?
On Tue, 1 Jun 2010 15:00:31 -0400, Pedro Silva wrote: > Hi, > > You may spend as rtmp streaming through squid, I need access to > videoconferencing and no one can access > > How I can do this?. Is it possible with Squid? > > My system: > Centos 5.4 > Squid 2.6 Stable Not directly. Only ICY / SHOUTcast streaming is supported like that so far. If your RTMP client applications have HTTP proxy support they can be configured and use the CONNECT method. You will then just have to add the RTMP server port to the Safe_Ports and SSL_Ports access lists in your Squid. Amos
RE: [squid-users] Accessing OWA or Sharepoint through Squid 3.1.0.17
On Tue, 1 Jun 2010 11:25:35 -0500, "Johnson, S" wrote: > More information based on the searches I've done... > > I'm using transparent mode on the squid proxy (without auth). Well, > I've got an AUP page set up for the users to agree to but no > LDAP/AD/NTLM auth is being performed on this proxy. You said you had port 80 and port 443 configured with the proxy. This does match you above statement that it's working "transparent". Or did you mean some other meaning of the word "transparent" than "NAT interception"? > > However, I tried the other squid proxy with ntlm_auth and it works a-ok. > I'm really drawing a blank here... Stretching for a long-shot you could try with explicit connection-auth=on flag to the http_port line. Though. Middle-ware proxies really should just work with these. The only special config is needed to reverse-proxy OWA. Amos
Re: [squid-users] squid_session logs
tis 2010-06-01 klockan 16:10 -0400 skrev Winfield Henry: > Is there a way to see the logs generated by squid_session after setting = > the -b flag? The file exists, but is not readable. > Thanks, W It's a BDB key-value database, not a log. It only contains timestamps when each session was last updated, indexed by their key as you have specified in squid.conf. The ttl parameter determines session update frequency. Regards Henrik
[squid-users] SQUID makes few slow connections (see video)
I thik that this is something to do it with many requests, because half of images are loaded imiedietly (with hits MISS) , and rest are loaded after few seconds (also MISSED) - missed because i have erased cache. and is page is cached - it works fine. Only NEW requests are working slowly (Read below full problem) here is more info: -- sky-link squid # squidclient -h 192.168.0.1 -p 8123 mgr:info HTTP/1.0 200 OK Server: squid/3.0.STABLE20 Mime-Version: 1.0 Date: Tue, 01 Jun 2010 21:47:50 GMT Content-Type: text/plain Expires: Tue, 01 Jun 2010 21:47:50 GMT Last-Modified: Tue, 01 Jun 2010 21:47:50 GMT X-Cache: MISS from sky-link.net X-Cache-Lookup: MISS from sky-link.net:8123 Via: 1.0 sky-link.net (squid/3.0.STABLE20) Proxy-Connection: close Squid Object Cache: Version 3.0.STABLE20 Start Time: Tue, 01 Jun 2010 21:47:46 GMT Current Time: Tue, 01 Jun 2010 21:47:50 GMT Connection information for squid: Number of clients accessing cache: 1 Number of HTTP requests received: 0 Number of ICP messages received:0 Number of ICP messages sent:0 Number of queued ICP replies: 0 Number of HTCP messages received: 0 Number of HTCP messages sent: 0 Request failure ratio: 0.00 Average HTTP requests per minute since start: 0.0 Average ICP messages per minute since start:0.0 Select loop called: 2945 times, 1.105 ms avg Cache information for squid: Hits as % of all requests: 5min: 0.0%, 60min: 0.0% Hits as % of bytes sent:5min: -0.0%, 60min: -0.0% Memory hits as % of hit requests: 5min: 0.0%, 60min: 0.0% Disk hits as % of hit requests: 5min: 0.0%, 60min: 0.0% Storage Swap size: 12692 KB Storage Swap capacity: 0.0% used, 100.0% free Storage Mem size: 108 KB Storage Mem capacity:0.0% used, 100.0% free Mean Object Size: 6.36 KB Requests given to unlinkd: 0 Median Service Times (seconds) 5 min60 min: HTTP Requests (All): 0.0 0.0 Cache Misses: 0.0 0.0 Cache Hits:0.0 0.0 Near Hits: 0.0 0.0 Not-Modified Replies: 0.0 0.0 DNS Lookups: 0.0 0.0 ICP Queries: 0.0 0.0 Resource usage for squid: UP Time:3.255 seconds CPU Time: 0.040 seconds CPU Usage: 1.23% CPU Usage, 5 minute avg:0.00% CPU Usage, 60 minute avg: 0.00% Process Data Segment Size via sbrk(): 4244 KB Maximum Resident Size: 54512 KB Page faults with physical i/o: 0 Memory usage for squid via mallinfo(): Total space in arena:4520 KB Ordinary blocks: 4514 KB 2 blks Small blocks: 0 KB 0 blks Holding blocks: 5916 KB 10 blks Free Small blocks: 0 KB Free Ordinary blocks: 5 KB Total in use: 10430 KB 100% Total free: 5 KB 0% Total size: 10436 KB Memory accounted for: Total accounted: 1969 KB 19% memPool accounted: 1969 KB 19% memPool unaccounted: 8466 KB 81% memPoolAlloc calls: 9684 memPoolFree calls: 4422 File descriptor usage for squid: Maximum number of file descriptors: 1024 Largest file desc currently in use: 15 Number of file desc currently in use:9 Files queued for open: 0 Available number of file descriptors: 1015 Reserved number of file descriptors: 100 Store Disk files open: 0 Internal Data Structures: 2024 StoreEntries 27 StoreEntries with MemObjects 26 Hot Object Cache Items 1997 on-disk objects --- 2010/6/1 Tytus Rogalewski : > Hi guys, i have problem with slow performance in my squid > When i enter the page WITHOUT squid, pages are loaded in ~1 sec, > but when i turn on SQUID - page load in 70% in 1 sec, and then few > images are loading very slowly > > I made flash movie - you can see how this slow connevting VIA squid works > http://sky-link.net/temp/squid/squid_performance.html (10 MB - i have > slow connection so you need to wait few mins :) > > > Server Hardware: > Core2Duo E4600 3.0 Ghz > 4 GB ramu > Dysk Szajsung 640GB (w fstabie dopisany noatime 0 0) > > Server Soft: > Linux Gentoo > squid-3.0.19 or 3.0.20 is the same thing > > > > Squid, access.log > You can see in second collumn when did image loaded fully. (1000 = 1 sec) > Look at 3 last rows, 3 sec,m 6 sec ened even 12 sec ? wtf ? > > 1275413778.282 401 192.168.0.2 TCP_MISS/200 20538 GET > http
[squid-users] squid_session logs
Hello, Is there a way to see the logs generated by squid_session after setting = the -b flag? The file exists, but is not readable. Thanks, W -- Winfield Winfield Henry (336) 506-4136 Information Systems Alamance Community College
[squid-users] creat a groupe in squid and squidGuard
Good day, I have squid and squidGuard installed with domaine controller , i need to create groups in this (some groups have access to intenet the other no) I need to use the groups and users that i have in my active directory it s possible to do this? I will be thankful if you can help me . Thank you -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/creat-a-groupe-in-squid-and-squidGuard-tp2239038p2239038.html Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] rtmp protocol in squid is possible ?
Hi, You may spend as rtmp streaming through squid, I need access to videoconferencing and no one can access How I can do this?. Is it possible with Squid? My system: Centos 5.4 Squid 2.6 Stable Thank you, -- Pedro
Re: [squid-users] SQUID makes few slow connections (see video)
Did you check if you are hitting the FD limit? An output of "General Runtime Information" from the "cachemgr.cgi" would provide some better information. Also, it appears you are using squid to provide local users better access to the internet at large (also known as forward proxy). In that case, you might want to use the COSS file storage offered by the 2.x series. We use the latest stable 2.7STABLE9 with great results. Regards HASSAN On Wed, Jun 2, 2010 at 00:22, Tytus Rogalewski wrote: > > Hi guys, i have problem with slow performance in my squid > When i enter the page WITHOUT squid, pages are loaded in ~1 sec, > but when i turn on SQUID - page load in 70% in 1 sec, and then few > images are loading very slowly > > I made flash movie - you can see how this slow connevting VIA squid works > http://sky-link.net/temp/squid/squid_performance.html (10 MB - i have > slow connection so you need to wait few mins :) > > > Server Hardware: > Core2Duo E4600 3.0 Ghz > 4 GB ramu > Dysk Szajsung 640GB (w fstabie dopisany noatime 0 0) > > Server Soft: > Linux Gentoo > squid-3.0.19 or 3.0.20 is the same thing > > > > Squid, access.log > You can see in second collumn when did image loaded fully. (1000 = 1 sec) > Look at 3 last rows, 3 sec,m 6 sec ened even 12 sec ? wtf ? > > 1275413778.282 401 192.168.0.2 TCP_MISS/200 20538 GET > http://allegro.pl/76700_dodatki_i_ozdoby.html? - DIRECT/193.23.48.134 > text/html > 1275413778.417 41 192.168.0.2 TCP_MISS/200 2845 GET > http://img16.allegroimg.pl/photos/64x48/10/52/33/52/1052335222 - > DIRECT/194.0.251.215 image/jpeg > 1275413778.426 48 192.168.0.2 TCP_MISS/200 1955 GET > http://img10.allegroimg.pl/photos/64x48/10/47/74/82/1047748223 - > DIRECT/194.0.251.214 image/jpeg > 1275413778.436 53 192.168.0.2 TCP_MISS/200 2124 GET > http://img14.allegroimg.pl/photos/64x48/10/47/88/44/1047884413 - > DIRECT/194.0.251.215 image/jpeg > 1275413778.447 61 192.168.0.2 TCP_MISS/200 2564 GET > http://img04.allegroimg.pl/photos/64x48/10/60/13/27/1060132706 - > DIRECT/194.0.251.213 image/jpeg > 1275413778.461 70 192.168.0.2 TCP_MISS/200 1880 GET > http://img17.allegroimg.pl/photos/64x48/10/48/29/44/1048294443 - > DIRECT/194.0.251.215 image/jpeg > 1275413778.473 82 192.168.0.2 TCP_MISS/200 1811 GET > http://img17.allegroimg.pl/photos/64x48/10/48/29/45/1048294559 - > DIRECT/194.0.251.215 image/jpeg > 1275413778.487 95 192.168.0.2 TCP_MISS/200 1906 GET > http://img17.allegroimg.pl/photos/64x48/10/48/29/46/1048294688 - > DIRECT/194.0.251.215 image/jpeg > 1275413778.503 73 192.168.0.2 TCP_MISS/200 2226 GET > http://img10.allegroimg.pl/photos/64x48/10/47/74/92/1047749248 - > DIRECT/194.0.251.214 image/jpeg > 1275413778.511 124 192.168.0.2 TCP_MISS/200 2012 GET > http://img11.allegroimg.pl/photos/64x48/10/48/20/58/1048205827 - > DIRECT/194.0.251.214 image/jpeg > 1275413778.526 137 192.168.0.2 TCP_MISS/200 2475 GET > http://img11.allegroimg.pl/photos/64x48/10/48/21/13/1048211380 - > DIRECT/194.0.251.214 image/jpeg > 1275413778.541 0 192.168.0.2 TCP_DENIED/403 353 GET > http://allegro.hit.gemius.pl/_1275413783164/rexdot.gif? - NONE/- > text/html > 1275413778.549 41 192.168.0.2 TCP_MISS/200 2422 GET > http://img10.allegroimg.pl/photos/64x48/10/47/74/92/1047749229 - > DIRECT/194.0.251.214 image/jpeg > 1275413778.600 47 192.168.0.2 TCP_MISS/200 480 GET > http://www.google-analytics.com/__utm.gif? - DIRECT/209.85.129.101 > image/gif > 1275413778.615 55 192.168.0.2 TCP_MISS/200 2692 GET > http://img10.allegroimg.pl/photos/64x48/10/48/11/40/1048114081 - > DIRECT/194.0.251.214 image/jpeg > 1275413778.659 42 192.168.0.2 TCP_MISS/200 2754 GET > http://img10.allegroimg.pl/photos/64x48/10/48/11/40/1048114095 - > DIRECT/194.0.251.214 image/jpeg > 1275413778.669 289 192.168.0.2 TCP_MISS/200 2488 GET > http://img10.allegroimg.pl/photos/64x48/10/47/74/92/1047749270 - > DIRECT/194.0.251.214 image/jpeg > 1275413778.678 298 192.168.0.2 TCP_MISS/200 2274 GET > http://img10.allegroimg.pl/photos/64x48/10/47/74/92/1047749255 - > DIRECT/194.0.251.214 image/jpeg > 1275413778.688 305 192.168.0.2 TCP_MISS/200 2747 GET > http://img11.allegroimg.pl/photos/64x48/10/52/72/84/1052728463 - > DIRECT/194.0.251.214 image/jpeg > 1275413778.697 318 192.168.0.2 TCP_MISS/200 1962 GET > http://img10.allegroimg.pl/photos/64x48/10/47/74/82/1047748228 - > DIRECT/194.0.251.214 image/jpeg > 1275413778.711 334 192.168.0.2 TCP_MISS/200 2148 GET > http://img14.allegroimg.pl/photos/64x48/10/47/74/07/1047740767 - > DIRECT/194.0.251.215 image/jpeg > 1275413778.724 62 192.168.0.2 TCP_MISS/200 2298 GET > http://img10.allegroimg.pl/photos/64x48/10/48/11/41/1048114124 - > DIRECT/194.0.251.214 image/jpeg > 1275413778.736 359 192.168.0.2 TCP_MISS/200 2206 GET > http://img06.allegroimg.pl/photos/64x48/10/47/47/49/1047474911 - > DIRECT/194.0.251.213 image/jpeg > 1275413778.749 79 192.168.0.2 TCP_MISS/200 2562
[squid-users] SQUID makes few slow connections (see video)
Hi guys, i have problem with slow performance in my squid When i enter the page WITHOUT squid, pages are loaded in ~1 sec, but when i turn on SQUID - page load in 70% in 1 sec, and then few images are loading very slowly I made flash movie - you can see how this slow connevting VIA squid works http://sky-link.net/temp/squid/squid_performance.html (10 MB - i have slow connection so you need to wait few mins :) Server Hardware: Core2Duo E4600 3.0 Ghz 4 GB ramu Dysk Szajsung 640GB (w fstabie dopisany noatime 0 0) Server Soft: Linux Gentoo squid-3.0.19 or 3.0.20 is the same thing Squid, access.log You can see in second collumn when did image loaded fully. (1000 = 1 sec) Look at 3 last rows, 3 sec,m 6 sec ened even 12 sec ? wtf ? 1275413778.282401 192.168.0.2 TCP_MISS/200 20538 GET http://allegro.pl/76700_dodatki_i_ozdoby.html? - DIRECT/193.23.48.134 text/html 1275413778.417 41 192.168.0.2 TCP_MISS/200 2845 GET http://img16.allegroimg.pl/photos/64x48/10/52/33/52/1052335222 - DIRECT/194.0.251.215 image/jpeg 1275413778.426 48 192.168.0.2 TCP_MISS/200 1955 GET http://img10.allegroimg.pl/photos/64x48/10/47/74/82/1047748223 - DIRECT/194.0.251.214 image/jpeg 1275413778.436 53 192.168.0.2 TCP_MISS/200 2124 GET http://img14.allegroimg.pl/photos/64x48/10/47/88/44/1047884413 - DIRECT/194.0.251.215 image/jpeg 1275413778.447 61 192.168.0.2 TCP_MISS/200 2564 GET http://img04.allegroimg.pl/photos/64x48/10/60/13/27/1060132706 - DIRECT/194.0.251.213 image/jpeg 1275413778.461 70 192.168.0.2 TCP_MISS/200 1880 GET http://img17.allegroimg.pl/photos/64x48/10/48/29/44/1048294443 - DIRECT/194.0.251.215 image/jpeg 1275413778.473 82 192.168.0.2 TCP_MISS/200 1811 GET http://img17.allegroimg.pl/photos/64x48/10/48/29/45/1048294559 - DIRECT/194.0.251.215 image/jpeg 1275413778.487 95 192.168.0.2 TCP_MISS/200 1906 GET http://img17.allegroimg.pl/photos/64x48/10/48/29/46/1048294688 - DIRECT/194.0.251.215 image/jpeg 1275413778.503 73 192.168.0.2 TCP_MISS/200 2226 GET http://img10.allegroimg.pl/photos/64x48/10/47/74/92/1047749248 - DIRECT/194.0.251.214 image/jpeg 1275413778.511124 192.168.0.2 TCP_MISS/200 2012 GET http://img11.allegroimg.pl/photos/64x48/10/48/20/58/1048205827 - DIRECT/194.0.251.214 image/jpeg 1275413778.526137 192.168.0.2 TCP_MISS/200 2475 GET http://img11.allegroimg.pl/photos/64x48/10/48/21/13/1048211380 - DIRECT/194.0.251.214 image/jpeg 1275413778.541 0 192.168.0.2 TCP_DENIED/403 353 GET http://allegro.hit.gemius.pl/_1275413783164/rexdot.gif? - NONE/- text/html 1275413778.549 41 192.168.0.2 TCP_MISS/200 2422 GET http://img10.allegroimg.pl/photos/64x48/10/47/74/92/1047749229 - DIRECT/194.0.251.214 image/jpeg 1275413778.600 47 192.168.0.2 TCP_MISS/200 480 GET http://www.google-analytics.com/__utm.gif? - DIRECT/209.85.129.101 image/gif 1275413778.615 55 192.168.0.2 TCP_MISS/200 2692 GET http://img10.allegroimg.pl/photos/64x48/10/48/11/40/1048114081 - DIRECT/194.0.251.214 image/jpeg 1275413778.659 42 192.168.0.2 TCP_MISS/200 2754 GET http://img10.allegroimg.pl/photos/64x48/10/48/11/40/1048114095 - DIRECT/194.0.251.214 image/jpeg 1275413778.669289 192.168.0.2 TCP_MISS/200 2488 GET http://img10.allegroimg.pl/photos/64x48/10/47/74/92/1047749270 - DIRECT/194.0.251.214 image/jpeg 1275413778.678298 192.168.0.2 TCP_MISS/200 2274 GET http://img10.allegroimg.pl/photos/64x48/10/47/74/92/1047749255 - DIRECT/194.0.251.214 image/jpeg 1275413778.688305 192.168.0.2 TCP_MISS/200 2747 GET http://img11.allegroimg.pl/photos/64x48/10/52/72/84/1052728463 - DIRECT/194.0.251.214 image/jpeg 1275413778.697318 192.168.0.2 TCP_MISS/200 1962 GET http://img10.allegroimg.pl/photos/64x48/10/47/74/82/1047748228 - DIRECT/194.0.251.214 image/jpeg 1275413778.711334 192.168.0.2 TCP_MISS/200 2148 GET http://img14.allegroimg.pl/photos/64x48/10/47/74/07/1047740767 - DIRECT/194.0.251.215 image/jpeg 1275413778.724 62 192.168.0.2 TCP_MISS/200 2298 GET http://img10.allegroimg.pl/photos/64x48/10/48/11/41/1048114124 - DIRECT/194.0.251.214 image/jpeg 1275413778.736359 192.168.0.2 TCP_MISS/200 2206 GET http://img06.allegroimg.pl/photos/64x48/10/47/47/49/1047474911 - DIRECT/194.0.251.213 image/jpeg 1275413778.749 79 192.168.0.2 TCP_MISS/200 2562 GET http://img10.allegroimg.pl/photos/64x48/10/48/11/41/1048114135 - DIRECT/194.0.251.214 image/jpeg 1275413778.762 83 192.168.0.2 TCP_MISS/200 2429 GET http://img10.allegroimg.pl/photos/64x48/10/48/12/11/1048121165 - DIRECT/194.0.251.214 image/jpeg 1275413778.773395 192.168.0.2 TCP_MISS/200 2564 GET http://img04.allegroimg.pl/photos/64x48/10/59/76/37/1059763730 - DIRECT/194.0.251.213 image/jpeg 1275413778.785 72 192.168.0.2 TCP_MISS/200 2130 GET http://img10.allegroimg.pl/photos/64x48/10/48/25/20/1048252001 - DIRECT/194.0.251.214 image/jpeg 1275413778.796 71 192.168.0.2 TCP_MISS/200 2152 GET http://img10.allegroimg.pl/photos/64x48/10/48/25/24/1048252448 - DIRECT
RE: [squid-users] high iowait on cache_swap_low
tis 2010-06-01 klockan 17:59 + skrev sameer khan: > yea i am think about put one or two more drives in with same space > spread on 4 drives, but that is the last option. is coss any better > and is it stable for squid 2.7 ? COSS in 2.7 is considered production quality. Regards Henrik
RE: [squid-users] high iowait on cache_swap_low
hey yea i am think about put one or two more drives in with same space spread on 4 drives, but that is the last option. is coss any better and is it stable for squid 2.7 ? thanks again > Subject: RE: [squid-users] high iowait on cache_swap_low > From: hen...@henriknordstrom.net > To: khanza...@hotmail.com > CC: squid-users@squid-cache.org > Date: Tue, 1 Jun 2010 19:39:28 +0200 > > tis 2010-06-01 klockan 17:38 + skrev sameer khan: > >> thanks for replying, any solution to the problem ? > > More disks? > > COSS storage type (unfortunately not stable in Squid-3) > > Regards > Henrik > _ http://clk.atdmt.com/UKM/go/195013117/direct/01/
Re: [squid-users] How to avoid the exceptions at the browser
tis 2010-06-01 klockan 10:53 -0500 skrev Juan Cardoza: > Is there a way we can send the URL to the proxy server and then the > proxy identify the URL and send it back to the internal network and find > the correct direccion. The idea to do this is to avoid the domain > request process, because sometime it takes a lot of time that can be > eliminate if the proxy manage the request. Sure. There is no big difference between internal or external servers to the proxy. Just make sure it has connectivity to the servers and know how to resolve their names to IP. Regards Henrik
RE: [squid-users] high iowait on cache_swap_low
tis 2010-06-01 klockan 17:38 + skrev sameer khan: > thanks for replying, any solution to the problem ? More disks? COSS storage type (unfortunately not stable in Squid-3) Regards Henrik
RE: [squid-users] high iowait on cache_swap_low
hey henrik, thanks for replying, any solution to the problem ? > Subject: Re: [squid-users] high iowait on cache_swap_low > From: hen...@henriknordstrom.net > To: khanza...@hotmail.com > CC: squid-users@squid-cache.org > Date: Tue, 1 Jun 2010 17:19:55 +0200 > > tis 2010-06-01 klockan 12:38 + skrev sameer khan: > >> We are using 2 x 320GB SATA2 with reiserfs 3.6. The machine experiences high >> iowait when the cache reaches cache_swap_low. What we have observed that it >> is constantly stuck at cache_swap_low. If we purge the cache it starts to >> work fine again until cache hits cache_swap_low mark. > > That's because the I/O load almost doubles then.. removing content from > a filesystem is nearly as expensive as writing it in the first place. > > Regards > Henrik > _ http://clk.atdmt.com/UKM/go/19780/direct/01/ Do you have a story that started on Hotmail? Tell us now
[squid-users] Problem: parseHttpRequest: Requestheader contains NULL characters
Hi, In my cache.log following lines appear. 2010/06/01 12:11:10| parseHttpRequest: Requestheader contains NULL characters 2010/06/01 12:11:10| parseHttpRequest: Unsupported method ' ' 2010/06/01 12:11:10| clientReadRequest: FD 131 (192.168.0.14:54629) Invalid Request These appear to see a site that does streaming. Looking for information should I add or modify the line in my squid.conf extension_methods. The problem is not knowing what to add parameters to accept streaming audio / video. Can that be a problem?. Can you help me solve this problem? My system: CentOS release 5.4 (Final) Linux kernel 2.6.18-164.6.1.el5 squid-2.6.STABLE21-3.el5 Thanks, -- Pedro Silva
Re: [squid-users] Re: Joomla DB authentication support hits Squid! :)
Le mardi 1 juin 2010 10:23:05, Henrik Nordström a écrit : > tis 2010-06-01 klockan 00:23 -0500 skrev Luis Daniel Lucio Quiroz: > > strict was removed because at least in mandriva perl, it was complaining > > about that some variables didnt exists > > Which variabes? > > Regards > Henrik I dont remember, i will check them in a while
RE: [squid-users] Accessing OWA or Sharepoint through Squid 3.1.0.17
More information based on the searches I've done... I'm using transparent mode on the squid proxy (without auth). Well, I've got an AUP page set up for the users to agree to but no LDAP/AD/NTLM auth is being performed on this proxy. However, I tried the other squid proxy with ntlm_auth and it works a-ok. I'm really drawing a blank here...
[squid-users] How to avoid the exceptions at the browser
Hello all At this moment I am using the exceptions at the browsers to avoid send the request to the proxy server for internal URLs. Is there a way we can send the URL to the proxy server and then the proxy identify the URL and send it back to the internal network and find the correct direccion. The idea to do this is to avoid the domain request process, because sometime it takes a lot of time that can be eliminate if the proxy manage the request. Kind regards c...@rm@N
Re: [squid-users] Re: Joomla DB authentication support hits Squid! :)
tis 2010-06-01 klockan 00:23 -0500 skrev Luis Daniel Lucio Quiroz: > strict was removed because at least in mandriva perl, it was complaining > about > that some variables didnt exists Which variabes? Regards Henrik
Re: [squid-users] FTP authenticated using explorer 8
tis 2010-06-01 klockan 15:12 +0200 skrev Luciano Mannucci: > Hello, > > a user of mine is complaining that with IE 8 he cannot open URLS like > ftp://u...@site.tld though my cache (the same works quite well with > chrome) on his Windows seven PC. The official standard is ftp://user:passw...@host/ How well browsers supports other forms varies greatly between browsers. Regards Henrik
Re: [squid-users] high iowait on cache_swap_low
tis 2010-06-01 klockan 12:38 + skrev sameer khan: > We are using 2 x 320GB SATA2 with reiserfs 3.6. The machine experiences high > iowait when the cache reaches cache_swap_low. What we have observed that it > is constantly stuck at cache_swap_low. If we purge the cache it starts to > work fine again until cache hits cache_swap_low mark. That's because the I/O load almost doubles then.. removing content from a filesystem is nearly as expensive as writing it in the first place. Regards Henrik
[squid-users] Accessing OWA or Sharepoint through Squid 3.1.0.17
I'm using Squid and Dansguardian to block and cache sites. Everything works great but accessing OWA or Sharepoint related site with authentication doesn't work. I get a great non-descript error in IE: Internet Explorer cannot display the webpage". If I try using Chrome, the login prompt just keeps on showing up. I've got 80 and 443 configured through squid. Has anyone else run into this issue and know what's going on? Thanks. sj
[squid-users] FTP authenticated using explorer 8
Hello, a user of mine is complaining that with IE 8 he cannot open URLS like ftp://u...@site.tld though my cache (the same works quite well with chrome) on his Windows seven PC. He gets: ERROR Cache Access Denied While trying to retrieve the URL: ftp://u...@www.example.it/ The following error was encountered: .Cache Access Denied. Sorry, you are not currently allowed to request: ftp://u...@www.example.it/from this cache until you have authenticated yourself. You need to use Netscape version 2.0 or greater, or Microsoft Internet Explorer 3.0, or an HTTP/1.1 compliant browser for this to work. Please contact the cache administrator if you have difficulties authenticating yourself or change your default password. Generated Tue, 01 Jun 2010 12:52:17 GMT by belisama.example.it (squid/2.7.STABLE9) Instead of the request of user and password that comes with Chrome. Using FireFox 3.6.3 he first gets a warning window saying "wou are about to access www.example.it as user user, is that what you want?". By clicking "OK" he gets: ERROR The requested URL could not be retrieved An FTP authentication failure occurred while trying to retrieve the URL: ftp://u...@www.example.it/ Squid sent the following FTP command: PASS and then received this reply Login authentication failed Your cache administrator is n...@example.it. Generated Tue, 01 Jun 2010 13:04:25 GMT by belisama.example.it (squid/2.7.STABLE9) Is that normal? My squid.conf file goes: http_port 3128 hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex cgi-bin \? no_cache deny QUERY cache_mem 64 MB maximum_object_size 65536 KB minimum_object_size 1 KB cache_dir ufs /var/cache/squid 51200 16 256 cache_access_log /var/log/squid/access.log cache_store_log none ftp_user sq...@example.it auth_param basic children 5 auth_param basic realm Squid proxy-caching web server auth_param basic credentialsttl 2 hours auth_param basic casesensitive off refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern . 0 20% 4320 positive_dns_ttl 2 hours request_timeout 15 minutes pconn_timeout 1200 seconds acl all src 0.0.0.0/0.0.0.0 acl inside src 192.168.0.0/16 10.0.101.0/24 acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 563 2087 8443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 563 # https, snews acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT acl ftp proto ftp http_access allow manager localhost http_access deny manager http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localhost http_access allow inside http_access allow ftp http_access deny all http_reply_access allow all icp_access allow all cache_mgr n...@example.it cache_effective_user squid visible_hostname belisama.example.it coredump_dir /var/cache/squid luciano. -- /"\ /Via A. Salaino, 7 - 20144 Milano (Italy) \ / ASCII RIBBON CAMPAIGN / PHONE : +39 2 485781 FAX: +39 2 48578250 X AGAINST HTML MAIL/ E-MAIL: posthams...@sublink.sublink.org / \ AND POSTINGS/ WWW: http://www.mannucci.ORG/
[squid-users] high iowait on cache_swap_low
hey We are using 2 x 320GB SATA2 with reiserfs 3.6. The machine experiences high iowait when the cache reaches cache_swap_low. What we have observed that it is constantly stuck at cache_swap_low. If we purge the cache it starts to work fine again until cache hits cache_swap_low mark. We are using: cache_swap_low 94 cache_swap_high 95 Our settup <*> Squid Cache: Version 2.7.STABLE6 configure options: '--enable-async-io' '--enable-icmp' '--enable-snmp' '--enable-cache-digests' '--enable-follow-x-forwarded-for' '--enable-storeio=aufs,ufs,null,diskd,coss' '--enable-removal-policies=heap,lru' '--with-maxfd=65535' '--disable-ident-lookups' '--enable-truncate' '--enable-linux-tproxy' '--enable-linux-tproxy' '--enable-large-cache-files' <*> Kernel 2.6.31.13 <*> Debain lenny 5.04 <*> ~300 requests/sec machine is setup in bridge mode with tproxy transparent, with ip configured on br0 interface. any help will be much appreciated. thanks in advance _ http://clk.atdmt.com/UKM/go/19780/direct/01/ Do you have a story that started on Hotmail? Tell us now
Re: [squid-users] squid rewrite & squidguard
On Mon, May 31, 2010 at 11:25 PM, Joseph L. Casale wrote: > Check the first two directives in your conf, see who can write > there. Hello, Thanks for your reply, I have some rights issues but even when i assign the right permissions (ie. squid), it still cannot read them: 2010-06-01 10:31:19 [17307] New setting: dbhome: /var/lib/squidguard 2010-06-01 10:31:19 [17307] New setting: logdir: /var/log/squid 2010-06-01 10:31:19 [17307] init domainlist /var/lib/squidguard/blacklists/ads/domains 2010-06-01 10:31:19 [17307] loading dbfile /var/lib/squidguard/blacklists/ads/domains.db 2010-06-01 10:31:19 [17307] Error db_open: Permission denied 2010-06-01 10:31:19 [17307] Going into emergency mode I'll mail the squidguard list, thanks for your help. Steph
[squid-users] squid-3,1.4 - dead sibling
Dear list, I upgraded to 3.1.4 but since then all my siblings seems to be down :( $ ./squidclient -h myserver -p myport mgr:server_list . . Status : Down . . This is strange because i got plenty of icpv2 (3130) traffic over the wire (tcpdump). It (still) works well with 3.1.3. Didn't change anything... Any idea regards, Jan
