Re: [squid-users] Re: access.log issues with squid 3.2.0.15
On 09.03.2012 18:45, YJZ wrote: I had access_log stdio:/Applications/oss/logs/access.log squid which worked fine. Today, I switched to: logformat customfmt %tl access_log stdio:/Applications/oss/logs/access-customfmt.log customfmt based on the suggestion in http://squid-web-proxy-cache.1019090.n4.nabble.com/Date-time-format-in-access-log-td1458569.html. This doesn't seem to have any effect. In fact, squid continues to log to /Applications/oss/logs/access.log in the squid native format. I've tried with or without the stdio: part. I did restart or -k reconfigure squid. Am I missing something obvious? /Applications/oss/logs/access.log vs /Applications/oss/logs/access-customfmt.log perhapse? Amos
[squid-users] squid 3.2.0.16: heavy load, huge store.log and WARNING: Disk space over limit: 246385708.00 KB 5120000 KB
Dear
I have upgraded my squid 3.2.0.15 to the squid 3.2.0.16
My server have a load between 4 to 10
ps aux
squid10495 69.2 0.8 878308 34624 ?Dl 14:41 21:26
(squid-3) -sYC -f /etc/squid3/squid.conf
squid10496 19.5 0.8 877012 36300 ?Sl 14:41 6:02
(squid-1) -sYC -f /etc/squid3/squid.conf
root 16870 0.0 0.0 849032 2296 ?Ss 13:38 0:00
/usr/sbin/squid -sYC -f /etc/squid3/squid.conf
squid16872 0.0 0.3 853476 12624 ?S13:38 0:03
(squid-coord-5) -sYC -f /etc/squid3/squid.conf
squid17707 22.4 0.9 879772 39120 ?Sl 14:43 6:21
(squid-2) -sYC -f /etc/squid3/squid.conf
squid26988 0.3 0.7 864476 30760 ?S15:10 0:00
(squid-4) -sYC -f /etc/squid3/squid.conf
In cache.log there is many events :
2012/03/09 14:38:15 kid3| WARNING: Disk space over limit: 220212628.00
KB 512 KB
2012/03/09 14:38:26 kid3| WARNING: Disk space over limit: 222510356.00
KB 512 KB
2012/03/09 14:38:37 kid3| WARNING: Disk space over limit: 225427248.00
KB 512 KB
What does it means ?
I have many space left on the disk:
/dev/sda1 452G 202G 228G 48% /
The store.log is increased to more than 10Go with these events :
1331289752.882 RELEASE -1 AEB65290D03E08DD782A337A15C479A4
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 535E450BD0DE1D710ADC738CE0E08FF1
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 903D01C5B90A27D5F7D200D87A3FCA7A
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 2FB0D200A038346A01B76B76C15EB5D3
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 48725D7EF8A3D6A90E06171969B285EA
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 49B074F3E611694B420267BC9A55DB84
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 E5AD82B621949034DC703BC3BAC74456
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 B4968A3D96D2F7CD6BC44CAD213565DF
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 86EC4961EB11634BF1FAA208C0643E9C
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 6B24147A383CFC7E9C66B55735DEC5E3
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 366BEA0201808D4A6F2849A22E62FC43
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 A7D34E3E97C37AE16EBF7C7DAF4A444F
? ? ? ? ?/? ?/? ? ?
1331289752.883 RELEASE -1 F52AEDCF0ADE71BE0E6E5D713212D76A
? ? ? ? ?/? ?/? ? ?
1331289752.883 RELEASE -1 7BFD18551EC24A8E3610A79BE3ECDC28
? ? ? ? ?/? ?/? ? ?
1331289752.883 RELEASE -1 0272A0D991F5C6CBB91A1DDDC218B505
? ? ? ? ?/? ?/? ? ?
Here it is my caches configuration file :
#- Multiple cpus --
workers 4
if ${process_number} = 1
cache_dir aufs /var/cache/squid2-1 4000 128 512
endif
if ${process_number} = 2
cache_dir aufs /var/cache/squid2-2 4000 128 512
endif
if ${process_number} = 3
cache_dir aufs /var/cache/squid2-3 4000 128 512
endif
if ${process_number} = 4
cache_dir aufs /var/cache/squid2-4 4000 128 512
endif
#--
cache_dir aufs /var/cache/squid 1000 16 256
# - OTHER CACHES
Re: [squid-users] enabling X-Authenticated-user
On 9/03/2012 6:16 p.m., Brett Lymn wrote: On Thu, Mar 08, 2012 at 10:37:01AM +1030, Brett Lymn wrote: 1) The credentials being passed to the upstream are not rewritten - if I decode the basic auth it has my real password going to the upstream. And scratch this one too... if I use: cache_peer upstream.proxy parent 8080 7 login=*:password no-query default along with the external acl the username rewrite happens[1] so now the silly upstream logging actually works for both basic kerberos authentication. [1] see line 1628 in http.cc - there is a check for peer_login == * and then it checks if there is an external ecl rewrite for the login details. Just below it on line 1644 was the case I was referring to where the username and password are set by the helper. But the * case will suit as well. Thanks for the patience help Amos - I got there in the end. Huzzah for happy endings :) Amos
Re: [squid-users] SQUID TPROXY not working when URL is hosted on the same machine running SQUID
Hi Amos, Thanks a lot for your help. Your suggestion of redirecting using cache-peering worked. I did cache-peering with the same squid instance (on a different port) and from then on sent to our captive portal. That way, didnt have to change any URL rewriting logic. Best Regards, Vignesh On Wed, Mar 7, 2012 at 4:43 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 6/03/2012 6:50 a.m., Vignesh Ramamurthy wrote: Hello, We are using squid to transparently proxy the traffic to a captive portal that is residing on the same machine as the squid server. The solution was working based on a NAT REDIRECT . We are moving the solution to TPROXY based now as part of migration to IPv6. The TPROXY works fine in intercepting traffic and also successfully able to allow / deny traffic to IPv6 sites. We are facing a strange issue when we try to access a URL in the same machine that hosts the squid server. The acces hangs and squid is not able to connect to the URL. We are having AOL webserver to host the webpage. As a workaround you can use the cache_peer no-tproxy option to get Squid to use its own IP when contacting that local server. It can still use the X-Forwarded-For header to get the client IP. I'm not too clear on the details, but I think it has something to do with the packets not actually going through routing or some layers of the handling TPROXY needs when shifting between processes on the same machine. If you want to learn the details and get it going please contact the netfilter people to find out whats happening to the packets once they leave Squid. Amos
Re: [squid-users] squid 3.2.0.16: heavy load, huge store.log and WARNING: Disk space over limit: 246385708.00 KB 5120000 KB
On 10/03/2012 12:28 a.m., David Touzeau wrote:
Dear
I have upgraded my squid 3.2.0.15 to the squid 3.2.0.16
My server have a load between 4 to 10
This load will be a side effect of the erasures underway. See below...
ps aux
squid10495 69.2 0.8 878308 34624 ?Dl 14:41 21:26
(squid-3) -sYC -f /etc/squid3/squid.conf
squid10496 19.5 0.8 877012 36300 ?Sl 14:41 6:02
(squid-1) -sYC -f /etc/squid3/squid.conf
root 16870 0.0 0.0 849032 2296 ?Ss 13:38 0:00
/usr/sbin/squid -sYC -f /etc/squid3/squid.conf
squid16872 0.0 0.3 853476 12624 ?S13:38 0:03
(squid-coord-5) -sYC -f /etc/squid3/squid.conf
squid17707 22.4 0.9 879772 39120 ?Sl 14:43 6:21
(squid-2) -sYC -f /etc/squid3/squid.conf
squid26988 0.3 0.7 864476 30760 ?S15:10 0:00
(squid-4) -sYC -f /etc/squid3/squid.conf
In cache.log there is many events :
2012/03/09 14:38:15 kid3| WARNING: Disk space over limit: 220212628.00
KB 512 KB
2012/03/09 14:38:26 kid3| WARNING: Disk space over limit: 222510356.00
KB 512 KB
2012/03/09 14:38:37 kid3| WARNING: Disk space over limit: 225427248.00
KB 512 KB
What does it means ?
It means you have a cache_dir configured for 5000 MB of space. Something
has made Squid worker #3 identify that it has over 210 GB of data on disk.
Resulting in urgent purging files to make room for new traffic. You can
see that in store.log below.
I have many space left on the disk:
/dev/sda1 452G 202G 228G 48% /
This is not about total free space on disk. It is about free space in
the small part of disk you have configured Squid to use as cache.
The store.log is increased to more than 10Go with these events :
1331289752.882 RELEASE -1 AEB65290D03E08DD782A337A15C479A4
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 535E450BD0DE1D710ADC738CE0E08FF1
? ? ? ? ?/? ?/? ? ?
snip
Here it is my caches configuration file :
#- Multiple cpus --
workers 4
if ${process_number} = 1
cache_dir aufs /var/cache/squid2-1 4000 128 512
endif
if ${process_number} = 2
cache_dir aufs /var/cache/squid2-2 4000 128 512
endif
if ${process_number} = 3
cache_dir aufs /var/cache/squid2-3 4000 128 512
endif
if ${process_number} = 4
cache_dir aufs /var/cache/squid2-4 4000 128 512
endif
#--
cache_dir aufs /var/cache/squid 1000 16 256
# - OTHER CACHES
From the above it appears that each worker has roughtly 4GB, and they
all share a 1 GB store. ~5GB for each, with a total of *only* 15.6 GB of
disk space permitted to be used. Yet your disk listing earlier said
around 200 GB was used.
This looks a lot like one of the side effects of disk corruption fixed
in 3.2.0.15. Did you have the bug 3441 fixes in your previous Squid?
Amos
Re: [squid-users] problem compiling squid-3.2.0.16-20120308-r11536 on OS X 10.4.11
On 9/03/2012 7:56 p.m., YJZ wrote: Back on OS X 10.4.11, squid-3.2.0.16-20120308-r11536 fails to compile with either gcc 4.0.1 or 4.2.1, but squid-3.2.0.16 from March 6th compiles fine with the same ./configure flags: Nothing changed in the Squid code between those two bundles. Just one include for defining symbols was added to a socket I/O module which you are not even building. Amos
[squid-users] sarg-reports monthly does not work
Hi,
I have a problem with monthly reports of squid using sarg
sarg-reports monthly
It does not work at all, and I dont know where to look for answers -
fe in log there is nothing concerning this issue.
Daily and weekly reports works perfect - with no problems.
I need some detective investigation or advices.
My logrotate /etc/logrotate.d/squid:
/var/log/squid/*.log {
monthly
compress
delaycompress
rotate 2
missingok
nocreate
sharedscripts
#added
prerotate
test ! -x /usr/sbin/sarg-maint || /usr/sbin/sarg-maint
endscript
postrotate
test ! -e /var/run/squid.pid || /usr/sbin/squid -k rotate
endscript
}
Bartosz.
[squid-users] Problem with swf streams from www.tv4play.se behind squid
Hello! We can't access swf streams from www.tv4play.se behind squid since about a month ago. Earlier we could at least play some of them, but now it seems that none of them work through squid. We haven't changed anything in the squid version or squid configuration that would cause this. We have contacted their support, but they see no problems on their side. I'm debugging the problem now (with ALL,9), but the size of the debug is enormous so it's hard to find anything relevant in there. Is there some debug setting I should rather use instead of ALL,9, to be able to locate the problem more easily? Our current squid version is 3.1, and I have tried both with 3.1.16 and 3.1.19. We run squid in FreeBSD. I have tried running squid both with and without caching active, same problem. I will try squid 2.7 later. Does anyone know of any problems with swf through squid, either in general or specifically from www.tv4play.se? (These streams are only available from Sweden, so they are unfortunately not available for general testing.) Thanks! -- Peter Olssonp...@leissner.se
Re: [squid-users] squid 3.2.0.16: heavy load, huge store.log and WARNING: Disk space over limit: 246385708.00 KB 5120000 KB
Thanks Amos
to answer :
I have many space left on the disk:
/dev/sda1 452G 202G 228G 48% /
This is not about total free space on disk. It is about free space in
the small part of disk you have configured Squid to use as cache.
Sure but /var/cache is stored on / partition
There is only one partition that allowing to use 400G so there is a
minimal of 200GB free
What did you recommend ? increase all caches in the configuration file ?
From the above it appears that each worker has roughtly 4GB, and they
all share a 1 GB store. ~5GB for each, with a total of *only* 15.6 GB of
disk space permitted to be used. Yet your disk listing earlier said
around 200 GB was used.
This looks a lot like one of the side effects of disk corruption fixed
in 3.2.0.15. Did you have the bug 3441 fixes in your previous Squid?
I did not know about the fix you mention the latest version was
squid-3.2.0.15-20120302-r11519
Do you recommend cleaning a rebuilding caches ?
Le 09/03/2012 13:35, Amos Jeffries a écrit :
On 10/03/2012 12:28 a.m., David Touzeau wrote:
Dear
I have upgraded my squid 3.2.0.15 to the squid 3.2.0.16
My server have a load between 4 to 10
This load will be a side effect of the erasures underway. See below...
ps aux
squid10495 69.2 0.8 878308 34624 ?Dl 14:41 21:26
(squid-3) -sYC -f /etc/squid3/squid.conf
squid10496 19.5 0.8 877012 36300 ?Sl 14:41 6:02
(squid-1) -sYC -f /etc/squid3/squid.conf
root 16870 0.0 0.0 849032 2296 ?Ss 13:38 0:00
/usr/sbin/squid -sYC -f /etc/squid3/squid.conf
squid16872 0.0 0.3 853476 12624 ?S13:38 0:03
(squid-coord-5) -sYC -f /etc/squid3/squid.conf
squid17707 22.4 0.9 879772 39120 ?Sl 14:43 6:21
(squid-2) -sYC -f /etc/squid3/squid.conf
squid26988 0.3 0.7 864476 30760 ?S15:10 0:00
(squid-4) -sYC -f /etc/squid3/squid.conf
In cache.log there is many events :
2012/03/09 14:38:15 kid3| WARNING: Disk space over limit:
220212628.00 KB 512 KB
2012/03/09 14:38:26 kid3| WARNING: Disk space over limit:
222510356.00 KB 512 KB
2012/03/09 14:38:37 kid3| WARNING: Disk space over limit:
225427248.00 KB 512 KB
What does it means ?
It means you have a cache_dir configured for 5000 MB of space.
Something has made Squid worker #3 identify that it has over 210 GB of
data on disk.
Resulting in urgent purging files to make room for new traffic. You
can see that in store.log below.
I have many space left on the disk:
/dev/sda1 452G 202G 228G 48% /
This is not about total free space on disk. It is about free space in
the small part of disk you have configured Squid to use as cache.
The store.log is increased to more than 10Go with these events :
1331289752.882 RELEASE -1 AEB65290D03E08DD782A337A15C479A4
? ? ? ? ?/? ?/? ? ?
1331289752.882 RELEASE -1 535E450BD0DE1D710ADC738CE0E08FF1
? ? ? ? ?/? ?/? ? ?
snip
Here it is my caches configuration file :
#- Multiple cpus --
workers 4
if ${process_number} = 1
cache_dir aufs /var/cache/squid2-1 4000 128 512
endif
if ${process_number} = 2
cache_dir aufs /var/cache/squid2-2 4000 128 512
endif
if ${process_number} = 3
cache_dir aufs /var/cache/squid2-3 4000 128 512
endif
if ${process_number} = 4
cache_dir aufs /var/cache/squid2-4 4000 128 512
endif
#--
cache_dir aufs /var/cache/squid 1000 16 256
# - OTHER CACHES
From the above it appears that each worker has roughtly 4GB, and they
all share a 1 GB store. ~5GB for each, with a total of *only* 15.6 GB
of disk space permitted to be used. Yet your disk listing earlier said
around 200 GB was used.
This looks a lot like one of the side effects of disk corruption fixed
in 3.2.0.15. Did you have the bug 3441 fixes in your previous Squid?
Amos
[squid-users] Help with a tcp_miss/200 issue
Hello all, I am trying to improve caching/acceleration on a series of wordpress sites. Almost all objects are being cached at this point other than the page HTML itself. All I am getting there is TCP_MISS/200 log lines. The request is a GET for the URL http://planetphotoshop.com At the moment my response header is: Cache-Controlmax-age=60 Cneonctionclose Connectionkeep-alive Content-Encodinggzip Content-Length15339 Content-Typetext/html; charset=UTF-8 DateFri, 09 Mar 2012 13:58:01 GMT ServerApache/2.2.15 (CentOS) VaryAccept-Encoding Via1.0 squid1.kelbymediagroup.com (squid) X-CacheMISS from squid1.kelbymediagroup.com X-Cache-LookupMISS from squid1.kelbymediagroup.com:80 X-Pingbackhttp://planetphotoshop.com/xmlrpc.php I dont see anything preventing caching Any thoughts or ideas? Thank you in advance for the help. James
Re: [squid-users] Problem with swf streams from www.tv4play.se behind squid
Sorry, forgot to mention that these streams work fine through a Microsoft TMG proxy at the same network position as one of our squid servers, so there is no network problem. And I also should mention that we run squid in proxy mode with clients configured either through wpad or with a static proxy server. No transparency. Peter Olsson On Fri, Mar 09, 2012 at 03:24:45PM +0100, Peter Olsson wrote: Hello! We can't access swf streams from www.tv4play.se behind squid since about a month ago. Earlier we could at least play some of them, but now it seems that none of them work through squid. We haven't changed anything in the squid version or squid configuration that would cause this. We have contacted their support, but they see no problems on their side. I'm debugging the problem now (with ALL,9), but the size of the debug is enormous so it's hard to find anything relevant in there. Is there some debug setting I should rather use instead of ALL,9, to be able to locate the problem more easily? Our current squid version is 3.1, and I have tried both with 3.1.16 and 3.1.19. We run squid in FreeBSD. I have tried running squid both with and without caching active, same problem. I will try squid 2.7 later. Does anyone know of any problems with swf through squid, either in general or specifically from www.tv4play.se? (These streams are only available from Sweden, so they are unfortunately not available for general testing.) Thanks! -- Peter Olssonp...@leissner.se
Re: [squid-users] Problem with swf streams from www.tv4play.se behind squid
On Fri, Mar 9, 2012 at 4:24 PM, Peter Olsson p...@leissner.se wrote: Hello! Hi. We can't access swf streams from www.tv4play.se behind squid since about a month ago. Earlier we could at least play some of them, but now it seems that none of them work through squid. We haven't changed anything in the squid version or squid configuration that would cause this. We have contacted their support, but they see no problems on their side. Does anyone know of any problems with swf through squid, either in general or specifically from www.tv4play.se? (These streams are only available from Sweden, so they are unfortunately not available for general testing.) For what its worth I don't have any problem streaming some of those videos but I am using v3.2.0.16. I streamed the Big Brother one for awhile with no problems. Thx -- .warren
[squid-users] Probably solved: Problem with swf streams from www.tv4play.se behind squid
I set forwarded_for off in squid.conf, and this seems to solve the problem. I haven't tried in our production squids yet, but a lab squid is working now. Peter Olsson On Fri, Mar 09, 2012 at 03:46:19PM +0100, Peter Olsson wrote: Sorry, forgot to mention that these streams work fine through a Microsoft TMG proxy at the same network position as one of our squid servers, so there is no network problem. And I also should mention that we run squid in proxy mode with clients configured either through wpad or with a static proxy server. No transparency. Peter Olsson On Fri, Mar 09, 2012 at 03:24:45PM +0100, Peter Olsson wrote: Hello! We can't access swf streams from www.tv4play.se behind squid since about a month ago. Earlier we could at least play some of them, but now it seems that none of them work through squid. We haven't changed anything in the squid version or squid configuration that would cause this. We have contacted their support, but they see no problems on their side. I'm debugging the problem now (with ALL,9), but the size of the debug is enormous so it's hard to find anything relevant in there. Is there some debug setting I should rather use instead of ALL,9, to be able to locate the problem more easily? Our current squid version is 3.1, and I have tried both with 3.1.16 and 3.1.19. We run squid in FreeBSD. I have tried running squid both with and without caching active, same problem. I will try squid 2.7 later. Does anyone know of any problems with swf through squid, either in general or specifically from www.tv4play.se? (These streams are only available from Sweden, so they are unfortunately not available for general testing.) Thanks! -- Peter Olssonp...@leissner.se
[squid-users] Squid 2.7 problem with url
Hello Going though squid in transparent mode This url is loaded as text in the browser: http://200.123.152.55/cgi-bin/ft.exe?q=images/edgedesk-logo-big.gifsite=clientes Without squid loads an ugly images, but it loads. I have used wget -S to see server headers, and they both put the content-type. Now I notice that using squid even says Lenght, but connecting to server doesn't. Does anybody have any clue? Thanks in advance, Sebastian Ex: With squid: root@cache:/var/tmp/test# wget -S http://200.123.152.55/cgi-bin/ft.exe?q=images/edgedesk-logo-big.gifsite=clientes; --2012-03-09 17:41:13-- http://200.123.152.55/cgi-bin/ft.exe?q=images/edgedesk-logo-big.gifsite=clientes Connecting to 127.0.0.1:31280... connected. Proxy request sent, awaiting response... HTTP/1.0 200 OK Date: Fri, 09 Mar 2012 19:55:34 GMT Server: Apache/2.0.54 (Fedora) Content-Type: image/gif Age: 2739 Content-Length: 4421 X-Cache: HIT from cache.netfly.com.ar X-Cache-Lookup: HIT from cache.netfly.com.ar:31280 Via: 1.1 cache:31280 (Lusca/LUSCA_FMI) Connection: close Length: 4421 (4.3K) [image/gif] Saving to: “ft.exe?q=images%2Fedgedesk-logo-big.gifsite=clientes.3” 100%[] 4,421 --.-K/s in 0s Without: root@cache:/var/tmp/test# wget -S http://200.123.152.55/cgi-bin/ft.exe?q=images/edgedesk-logo-big.gifsite=clientes; --2012-03-09 17:42:50-- http://200.123.152.55/cgi-bin/ft.exe?q=images/edgedesk-logo-big.gifsite=clientes Connecting to 200.123.152.55:80... connected. HTTP request sent, awaiting response... HTTP/1.1 200 OK Date: Fri, 09 Mar 2012 20:38:28 GMT Server: Apache/2.0.54 (Fedora) Connection: close Content-Type: image/gif Length: unspecified [image/gif] Saving to: “ft.exe?q=images%2Fedgedesk-logo-big.gifsite=clientes.4” [ = ] 4,421 --.-K/s in 0s 2012-03-09 17:42:50 (19.3 MB/s) - “ft.exe?q=images%2Fedgedesk-logo-big.gifsite=clientes.4” saved [4421]
Re: [squid-users] squid 3.2.0.16: heavy load, huge store.log and WARNING: Disk space over limit: 246385708.00 KB 5120000 KB
On 10/03/2012 3:27 a.m., David Touzeau wrote:
Thanks Amos
to answer :
I have many space left on the disk:
/dev/sda1 452G 202G 228G 48% /
This is not about total free space on disk. It is about free space in
the small part of disk you have configured Squid to use as cache.
Sure but /var/cache is stored on / partition
There is only one partition that allowing to use 400G so there is a
minimal of 200GB free
What did you recommend ? increase all caches in the configuration file ?
I was just pointing out that you have configured Squid to use no more
than 17GB of *cache*. So the disk having 200GB free is not relevant to
how full the cache is.
To make an analogy, using a 56Kbps modem on a broadband enabled phone
line is not going to get you Mbps speeds. You get at most the maximum of
what the modem is capable.
From the above it appears that each worker has roughtly 4GB, and they
all share a 1 GB store. ~5GB for each, with a total of *only* 15.6 GB
of disk space permitted to be used. Yet your disk listing earlier said
around 200 GB was used.
This looks a lot like one of the side effects of disk corruption fixed
in 3.2.0.15. Did you have the bug 3441 fixes in your previous Squid?
I did not know about the fix you mention the latest version was
squid-3.2.0.15-20120302-r11519
Do you recommend cleaning a rebuilding caches ?
Um. I must be confused about what you were saying then.
Were you actually running squid-3.2.0.15 (or one of the dated
bundles) previously when this too big cache was created?
*If* you were running anything older than 3.2.0.15 or 3.1.19 release
bundles, then you need to erase the swap.state files from each cache_dir
(or the whole thing) while Squid is shitdown. Then start Squid. You will
get one 'DIRTY' rebuild, possibly with erasures, then things should work
again fine.
If you were running 3.2.0.15 while that cache was created, something
else must be going on. I'm not sure what though.
Amos
Le 09/03/2012 13:35, Amos Jeffries a écrit :
On 10/03/2012 12:28 a.m., David Touzeau wrote:
Dear
I have upgraded my squid 3.2.0.15 to the squid 3.2.0.16
My server have a load between 4 to 10
This load will be a side effect of the erasures underway. See below...
ps aux
squid10495 69.2 0.8 878308 34624 ?Dl 14:41 21:26
(squid-3) -sYC -f /etc/squid3/squid.conf
squid10496 19.5 0.8 877012 36300 ?Sl 14:41 6:02
(squid-1) -sYC -f /etc/squid3/squid.conf
root 16870 0.0 0.0 849032 2296 ?Ss 13:38 0:00
/usr/sbin/squid -sYC -f /etc/squid3/squid.conf
squid16872 0.0 0.3 853476 12624 ?S13:38 0:03
(squid-coord-5) -sYC -f /etc/squid3/squid.conf
squid17707 22.4 0.9 879772 39120 ?Sl 14:43 6:21
(squid-2) -sYC -f /etc/squid3/squid.conf
squid26988 0.3 0.7 864476 30760 ?S15:10 0:00
(squid-4) -sYC -f /etc/squid3/squid.conf
In cache.log there is many events :
2012/03/09 14:38:15 kid3| WARNING: Disk space over limit:
220212628.00 KB 512 KB
2012/03/09 14:38:26 kid3| WARNING: Disk space over limit:
222510356.00 KB 512 KB
2012/03/09 14:38:37 kid3| WARNING: Disk space over limit:
225427248.00 KB 512 KB
What does it means ?
It means you have a cache_dir configured for 5000 MB of space.
Something has made Squid worker #3 identify that it has over 210 GB
of data on disk.
Resulting in urgent purging files to make room for new traffic. You
can see that in store.log below.
I have many space left on the disk:
/dev/sda1 452G 202G 228G 48% /
This is not about total free space on disk. It is about free space in
the small part of disk you have configured Squid to use as cache.
The store.log is increased to more than 10Go with these events :
1331289752.882 RELEASE -1
AEB65290D03E08DD782A337A15C479A4 ? ? ? ?
?/? ?/? ? ?
1331289752.882 RELEASE -1
535E450BD0DE1D710ADC738CE0E08FF1 ? ? ? ?
?/? ?/? ? ?
snip
Here it is my caches configuration file :
#- Multiple cpus --
workers 4
if ${process_number} = 1
cache_dir aufs /var/cache/squid2-1 4000 128 512
endif
if ${process_number} = 2
cache_dir aufs /var/cache/squid2-2 4000 128 512
endif
if ${process_number} = 3
cache_dir aufs /var/cache/squid2-3 4000 128 512
endif
if ${process_number} = 4
cache_dir aufs /var/cache/squid2-4 4000 128 512
endif
#--
cache_dir aufs /var/cache/squid 1000 16 256
# - OTHER CACHES
From the above it appears that each worker has roughtly 4GB, and they
all share a 1 GB store. ~5GB for each, with a total of *only* 15.6 GB
of disk space permitted to be used. Yet your disk listing earlier
said around 200 GB was used.
This looks a lot like one of the side effects of disk corruption
fixed in 3.2.0.15. Did you have the bug 3441 fixes in your previous
Squid?
Amos
Re: [squid-users] Probably solved: Problem with swf streams from www.tv4play.se behind squid
On 10/03/2012 5:55 a.m., Peter Olsson wrote: I set forwarded_for off in squid.conf, and this seems to solve the problem. I haven't tried in our production squids yet, but a lab squid is working now. Perhapse this will help their technicians detect the problem. If changing that header fixed it for you then the issue is clearly something at their end in its processing. I've been noticing this on a few sites now. For my clients it seems to be web systems which break on IPv6 addresses in the HTTP headers. X-Forwarded-For included. SquirrelMail seems to be the common culprit with its send button handler being broken. So, are your clients contacting Squid from IPv6 addresses? Amos
Re: [squid-users] Squid 2.7 problem with url
On 10/03/2012 9:48 a.m., Iojan Sebastian wrote: Hello Going though squid in transparent mode This url is loaded as text in the browser: http://200.123.152.55/cgi-bin/ft.exe?q=images/edgedesk-logo-big.gifsite=clientes Without squid loads an ugly images, but it loads. I have used wget -S to see server headers, and they both put the content-type. Now I notice that using squid even says Lenght, but connecting to server doesn't. Does anybody have any clue? Clue #1: Via: 1.1 cache:31280 (Lusca/LUSCA_FMI) This appears to be Lusca. Not Squid. Lusca is a commercial branch of Squid-2.7. Please contact Xenion for Lusca support. Amos
Re: [squid-users] Squid 2.7 problem with url
Clue #1: Via: 1.1 cache:31280 (Lusca/LUSCA_FMI) This appears to be Lusca. Not Squid. Lusca is a commercial branch of Squid-2.7. Please contact Xenion for Lusca support. Amos Sorry, downloaded from google code, compiled, and installed, didn't know that it was commercial and/or it was far away from squid. Thanks Sebastian
Re: [squid-users] Squid 2.7 problem with url
Please use Squid 3.x series, 2.7 is ancient. On 03/09/2012 08:18 PM, Iojan Sebastian wrote: Clue #1: Via: 1.1 cache:31280 (Lusca/LUSCA_FMI) This appears to be Lusca. Not Squid. Lusca is a commercial branch of Squid-2.7. Please contact Xenion for Lusca support. Amos Sorry, downloaded from google code, compiled, and installed, didn't know that it was commercial and/or it was far away from squid. Thanks Sebastian
Re: [squid-users] Re: access.log issues with squid 3.2.0.15
I had access_log stdio:/Applications/oss/logs/access.log squid which worked fine. Today, I switched to: logformat customfmt %tl access_log stdio:/Applications/oss/logs/access-customfmt.log customfmt based on the suggestion in http://squid-web-proxy-cache.1019090.n4.nabble.com/Date-time-format-in-access-log-td1458569.html. This doesn't seem to have any effect. In fact, squid continues to log to /Applications/oss/logs/access.log in the squid native format. I've tried with or without the stdio: part. I did restart or -k reconfigure squid. Am I missing something obvious? /Applications/oss/logs/access.log vs /Applications/oss/logs/access-customfmt.log perhapse? Amos Nah, I double-checked that. In fact, squid still logs in epoch timestamps with the following config: logformat customfmt %tl access_log stdio:/Applications/oss/logs/access.log customfmt Or it only logs to access.log, with the following: logformat customfmt %tl access_log stdio:/Applications/oss/logs/access-customfmt.log customfmt access_log stdio:/Applications/oss/logs/access.log squid It almost as if squid is using a cached copy of its previous squid.conf. -- NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone! Jetzt informieren: http://mobile.1und1.de/?ac=OM.PW.PW003K20328T7073a
