Re: [squid-users] access.log
On Mon, 28 Feb 2011 23:45:23 +0100, marco wrote: Howdi again, does someone know an online Page for checkout the access.log like squid sarg style ? There is none. one of my costumer use ipfire, and there is no way to get squid sarg on it. (no gcc compiler on ipfire, nothing...) Not usually a problem as long as ssh is available. You can scp the file over to another box or configure squid to log to another box acting as log server. Amos
Re: [squid-users] netdbExchangeHandleReply: corrupt data, aborting
On Mon, 28 Feb 2011 11:39:42 +, Alex Sharaz wrote: Sent this out a while back. Don't think I got any replies. Anyway, Still happening but now with squid 3.1.10/3.1.11 I'd like to do a phased upgrade to 3.1.x but don;t want to try it if I'm still getting these netdb errors This is nothing to be overly worried about. All it means is that the receiving Squid will have less data to work with than a full transfer would have given it. NetDB is susceptible to architecture differences "corrupting" its expected data. The message is output on detection and the unknown entries are dropped. In your log you can see 821 entries were still accepted. If you see this happening between two Squid-3.1 please report as a bug and it will be looked at and fixed eventually. A copy of the "corrupted" netdb object would be useful in the report to see why squid-3 rejects its 822nd entry. HTH Amos
[squid-users] access.log
Howdi again, does someone know an online Page for checkout the access.log like squid sarg style ? one of my costumer use ipfire, and there is no way to get squid sarg on it. (no gcc compiler on ipfire, nothing...) the customer want to see, what pages the employer entered and if they play an online Game -.- The privacy is not the problem thx, regards - marco
Re: [squid-users] Squid 3.1 reverse proxy to OWA on IIS7
On Mon, 28 Feb 2011 16:18:27 -, Gordon McKee wrote: Hi The "GET / HTTP/1.1" returns: GET / HTTP/1.1 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* Accept-Language: en-us Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) Host: www.optimalprofit.com Connection: Close :) I hope not. That is the initial request. and the "GET /images/op-hwynit-ad1.gif HTTP/1.1" to pull an image file returns: HTTP/1.0 200 OK Content-Type: image/gif Content-Encoding: gzip Last-Modified: Wed, 08 Dec 2004 15:34:12 GMT Accept-Ranges: bytes ETag: "a0d3e25d3bddc41:0" Vary: Accept-Encoding Server: Microsoft-IIS/7.0 X-Powered-By: ASP.NET Date: Mon, 28 Feb 2011 16:13:28 GMT Content-Length: 264171 X-Cache: MISS from kursk.gdmckee.home Via: 1.0 kursk.gdmckee.home (squid/3.1.11) Connection: close I have tried the telnet codes to access the OWA folder and the scripts come back very fast and the images take for every. Not sure what is going wrong. It's 258 KB after compression and not being cached. Size may have something to do with it if the scripts are much smaller. Amos
Re: [squid-users] Frustrating "Invalid Request" Reply
On Mon, 28 Feb 2011 16:51:54 +0200, Ümit Kablan wrote: Hi, Sorry for the late reply, Enter the full phrase and hit enter: [192.168.1.10 -> 192.168.1.120] GET /search?hl=tr&source=hp&biw=1280&bih=897&q=ertex&aq=2&aqi=g10&aql=&oq=ert&fp=3405898bc8895081&tch=1&ech=1&psi=_LBrTd6iFM-o8QPm5P3tDA12989033090755&safe=active HTTP/1.1 Host: www.google.com.tr Proxy-Connection: keep-alive Referer: http://www.google.com.tr/ Accept: */* User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.224 Safari/534.10 Accept-Encoding: gzip,deflate,sdch Accept-Language: tr-TR,tr;q=0.8,en-US;q=0.6,en;q=0.4 Accept-Charset: ISO-8859-9,utf-8;q=0.7,*;q=0.3 Cookie: NID=44=WDrVJT3IHROI8LLhYljiGzpNonvug9envnNeEoo6qdVxw1B1eHwarlfgZgODzoTsj7i7QGza5luXEqgQuFx7eWduz3Pcc-8IFrLp8tTyIaJC9VgyXEyQAv0qBQD3Dxm9; PREF=ID=e5ce72ddfd5e542a:U=0163fee991eaa35b:FF=0:TM=1298386459:LM=1298903279:S=6Sakp_hgUHZXMW1W [192.168.1.120 -> 192.168.1.10] HTTP/1.0 400 Bad Request Server: squid/2.7.STABLE8 Date: Mon, 28 Feb 2011 14:30:43 GMT Content-Type: text/html Content-Length: 2044 X-Squid-Error: ERR_INVALID_REQ 0 X-Cache: MISS from kiemserver X-Cache-Lookup: NONE from kiemserver:3128 Via: 1.0 kiemserver:3128 (squid/2.7.STABLE8) Connection: close Last is the weird part. It crops the full url and it thinks it is talking directly to the origin as you already said. Or I am skipping something obvious. I'm still convinced this is some form of configuration mistake somewhere. Lets step through this piece by piece in detail and see if anything appears. Which browser are you using to test with? What proxy settings are entered into its control panel? What does the client hosts file contain? What does the client resolv.conf or equivalent Windows network connection settings contain as gateway router, domain, and DNS servers? Amos
RE: [squid-users] Squid Memory always increasing
Thank You Amos. This was helpful. I went through wiki page related to Squid memory usage which helped me understand more about Squid Memory usage. Thanks! Regards, Saurabh -Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Sunday, February 27, 2011 10:10 AM To: squid-users@squid-cache.org Subject: Re: [squid-users] Squid Memory always increasing On 26/02/11 16:06, Saurabh Agarwal wrote: > Thanks Amos! One more question. > > When there is no load on Squid after a period of heavy load will Squid memory > footprint won't go down? I think it should. Are there some ways other than > "memory_pools off" config to make Squid free the earlier malloc'ed memory. > Squid should always be "freeing" memory as soon as it is finished with. Allocating and freeing is not to be confused with the memory footprint, which always can only grow. Squid allocates from the OS as much as needed. More when more is needed. When pools are on Squid will retain internal pools to locate the currently available/"free" memory tuned to exact object sizes Squid uses for re-use by the objects in later needs. With pools turned off in Squid this is returned for the OS to maintain such pools via its less well tuned algorithms. The OS will still report a memory footprint equal to the largest *ever* allocated amount used by Squid. Which should grow to a max and hold there. The wiki page on memory usage explains what Squid uses RAM for. To give you a good idea whether the amount is reasonable for your Squids maximum traffic load. And to allow you to roughly budget the system RAM. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.11 Beta testers wanted for 3.2.0.5
Re: [squid-users] Squid 3.1 reverse proxy to OWA on IIS7
Hi The "GET / HTTP/1.1" returns: GET / HTTP/1.1 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* Accept-Language: en-us Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) Host: www.optimalprofit.com Connection: Close and the "GET /images/op-hwynit-ad1.gif HTTP/1.1" to pull an image file returns: HTTP/1.0 200 OK Content-Type: image/gif Content-Encoding: gzip Last-Modified: Wed, 08 Dec 2004 15:34:12 GMT Accept-Ranges: bytes ETag: "a0d3e25d3bddc41:0" Vary: Accept-Encoding Server: Microsoft-IIS/7.0 X-Powered-By: ASP.NET Date: Mon, 28 Feb 2011 16:13:28 GMT Content-Length: 264171 X-Cache: MISS from kursk.gdmckee.home Via: 1.0 kursk.gdmckee.home (squid/3.1.11) Connection: close I have tried the telnet codes to access the OWA folder and the scripts come back very fast and the images take for every. Not sure what is going wrong. Many thanks Gordon -Original Message- From: Amos Jeffries Sent: Monday, February 28, 2011 12:19 AM To: squid-users@squid-cache.org Subject: Re: [squid-users] Squid 3.1 reverse proxy to OWA on IIS7 On Sun, 27 Feb 2011 17:19:33 -, Gordon McKee wrote: Hi I had FreeBSD 6.3 and squid 2.6 running fine reverse proxying my OWA server. I have now upgraded to FreeBSD 8 and squid 3.1 as the old software was getting rather old. I have copied the config file off the old server on the the new server. All is working except OWA. The images come down very very slowly (it does work really slowly). I was thinking it might be a DNS issue, but if I telnet (outside network) to www.optimalprofit.com 80 and enter: GET / HTTP/1.1 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* Accept-Language: en-us Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) Host: www.optimalprofit.com Connection: Close the page comes back really fast, but if I telnet to www.optimalprofit.com 80 and enter (to get a gif file off the server): GET /images/op-hwynit-ad1.gif HTTP/1.1 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* Accept-Language: en-us Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) Host: www.optimalprofit.com Connection: Close the text comes back really really slowly. I am not sure what is wrong as even my mobile connects in and over active sync and picks up my emails just fine. I have tried different browsers and they don't make any difference. What are the reply headers for each of these tests? Amos
Re: [squid-users] Frustrating "Invalid Request" Reply
Hi, Sorry for the late reply, 2011/2/25 Amos Jeffries : > On 25/02/11 22:53, Ümit Kablan wrote: >> >> 2011/2/24 Amos Jeffries: >>> >>> On Wed, 23 Feb 2011 12:32:56 +0200, Ümit Kablan wrote: 2011/2/22 Amos Jeffries : > > On Tue, 22 Feb 2011 17:24:39 +0200, Ümit Kablan wrote: >> >> 2011/2/21 Amos Jeffries wrote: >>> >>> On Mon, 21 Feb 2011 16:19:53 +0200, Ümit Kablan wrote: --- GET /search?hl=tr&source=hp&biw=1276&bih=823&q=eee+ktu&aq=0&aqi=g10&aql=&oq=eee&fp=64d53dfd7a69225a&tch=3&ech=1ψ=6UBOTbHmCtah_Aa2haXRDw12969740590425&wrapid=tlif129697480915821&safe=active HTTP/1.1 >>> >>> Note the missing http://domain details in the URL. This is not a >>> browser->proxy HTTP request. It is a browsers->origin request. >>> >>> IIRC interception of this type of request does not work in Windows, >>> since >>> the kernel NAT details are not available without proprietary >>> third-party >>> network drivers. Look at WPAD configuration of the localnet browsers >>> instead, that way they will send browser->proxy requests nicely. >> >> Exactly! The working requests are all starting with http://domain/ as >> you mentioned. (I must say I couldn't capture loopback network packets >> ... > > Squid needs to be configured via the http_port to know what mode/type > of > traffic it is going to receive. The browsers need to be sending the > right > type as well. I have - http_port 3128 - in my configuration. Do I miss something? >>> >>> Yes. But you keep omitting the details of *how* browsers are getting to >>> squid, so we can't tell if you are attempting to run a transparent proxy >>> or >>> a reverse proxy. Two very different configurations both in Squid and in >>> the >>> network underneath. >>> >>> Please confirm your network layout and traffic flows including software >>> which is involved on each related machine. >>> >> >> My network has 20+ machines all connecting to internet individually >> through ONE adsl modem in my network (those are connected to each >> other with a switch). My browsers are configured to use the squid >> proxy explicitly (so I think it has nothing to to with transparency) >> > > Okay. Then it is VERY weird that they would be behaving as if the proxy were > an origin server and not a proxy. None of the major browsers or thousands of > other agents out there display that type of confusion. > >>> >>> You say this Squid is on Windows where interception type of transparent >>> proxy is not possible for free, but keep mentioning the public website >>> google as working. >> >> Actually I was trying to stress on the weird problem I encountered to >> help shed some light on the problem. >> >>> >>> I suspect you are trying to perform NAT interception on a separate box to >>> Squid. Which is highly dangerous. >>> >> >> I think NAT inspection you mentioned is not executed on the XP machine >> where squid is running, yes. But I am not sharing my internet >> connection through that windows machine. I just want clients (those >> browsers configured to use proxy) use the internal proxy. > > If the NAT anywhere is forwarding packets to Squid it would display like > this inside Squid. > > > Check for NAT (sometimes called port forwarding) rules on that box > mentioning the Squid box. Remove any found. > > As an experiment you can also add an full firewall block of HTTP traffic > coming out of the network form anywhere except the Squid box. If the > browsers are correctly configured and going > browser->squid->firewall->Internet then the client will not even notice the > firewall block. Amos, I couldn't make that experiment you defined but I installed wireshark on that client machine (192.168.1.120) to sniff the network conversation with the proxy (192.168.1.10). Here is what I got: Enter the search engine: [192.168.1.10 -> 192.168.1.120] GET http://www.google.com/ HTTP/1.1 Host: www.google.com Proxy-Connection: keep-alive Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.224 Safari/534.10 Accept-Encoding: gzip,deflate,sdch Accept-Language: tr-TR,tr;q=0.8,en-US;q=0.6,en;q=0.4 Accept-Charset: ISO-8859-9,utf-8;q=0.7,*;q=0.3 Cookie: NID=44=gkt-jx_qa_J60q_7Kh4Js1k6NWv6AiHLRZ9CS-rvoyYOmqzicK-QCaJ0G6i0NEWMU_ZMLkbmSi3SM1lY87Wa-4xbeSbMW587mgMopt52Ft63oWkorPWy1qT2lT7yOkh_; PREF=ID=35a4f1ae7230beb1:U=b17222c86da2e9a2:FF=0:TM=1298386458:LM=1298903279:S=lsWVEGvnUbx5O1tO Start typing a phase and it tries to autocomplete: [192.168.1.10 -> 192.168.1.120] GET http://clients1.google.com.tr/complete/search?hl=tr&client=hp&q=ert&cp=3 HTTP/1.1 Host: clients1.google.com.tr Proxy-Connection: keep-alive Referer: http://www.google.com.tr/ Accept: */* User-Agent: Mozil
[squid-users] netdbExchangeHandleReply: corrupt data, aborting
Sent this out a while back. Don't think I got any replies. Anyway, Still happening but now with squid 3.1.10/3.1.11 I'd like to do a phased upgrade to 3.1.x but don;t want to try it if I'm still getting these netdb errors Rgds Alex Hi, For a while now I've been running a squid 2.7stable7 service here (just upgraded to stable9) and thought I'd try out the 3.1.4 build on my test web cache. Although the test cache is linked into my production cache cluster as a sibling the universtiy access the cache service via a serveriron hardware load balancer which load balances traffic over all my 2.7.STABLE9 boxes. I access the test cache directly. Since this morning, when i upgraded to 3.1.4 I've been seeing the following in the 3.1.4 cache.log file 2010/06/21 12:14:12| storeLateRelease: released 0 objects 2010/06/21 12:14:33| netdbExchangeHandleReply: corrupt data, aborting 2010/06/21 12:14:33| netdbExchangeHandleReply: corrupt data, aborting 2010/06/21 12:14:41| netdbExchangeHandleReply: corrupt data, aborting 2010/06/21 12:18:59| Detected DEAD Sibling: wwwcache3-east.hull.ac.uk 2010/06/21 12:18:59| Detected DEAD Sibling: wwwcache4-east.hull.ac.uk 2010/06/21 12:18:59| Detected DEAD Sibling: wwwcache1-west.hull.ac.uk 2010/06/21 12:18:59| Detected DEAD Sibling: wwwcache3-west.hull.ac.uk 2010/06/21 12:18:59| Detected DEAD Sibling: wwwcache1-east.hull.ac.uk 2010/06/21 12:18:59| Detected REVIVED Sibling: wwwcache1-west.hull.ac.uk 2010/06/21 12:18:59| Detected REVIVED Sibling: wwwcache3-west.hull.ac.uk 2010/06/21 12:18:59| Detected REVIVED Sibling: wwwcache3-east.hull.ac.uk 2010/06/21 12:18:59| Detected REVIVED Sibling: wwwcache1-east.hull.ac.uk 2010/06/21 12:18:59| Detected REVIVED Sibling: wwwcache4-east.hull.ac.uk 2010/06/21 12:54:11| NETDB state saved; 821 entries, 3 msec 2010/06/21 12:54:45| netdbExchangeHandleReply: corrupt data, aborting 2010/06/21 12:54:45| netdbExchangeHandleReply: corrupt data, aborting 2010/06/21 12:54:52| netdbExchangeHandleReply: corrupt data, aborting 2010/06/21 13:11:28| Detected DEAD Sibling: wwwcache4-east.hull.ac.uk 2010/06/21 13:11:28| Detected DEAD Sibling: wwwcache2-west.hull.ac.uk 2010/06/21 13:11:28| Detected REVIVED Sibling: wwwcache4-east.hull.ac.uk 2010/06/21 13:11:28| Detected REVIVED Sibling: wwwcache2-west.hull.ac.uk 2010/06/21 13:40:18| netdbExchangeHandleReply: corrupt data, aborting 2010/06/21 13:40:25| netdbExchangeHandleReply: corrupt data, aborting 2010/06/21 13:40:26| netdbExchangeHandleReply: corrupt data, aborting 2010/06/21 13:55:15| NETDB state saved; 821 entries, 3 msec Don't think I've seen this before. Web cache configs available if necessary. Anyone else trying to mix 2.7 and 3.1 siblings? Rgds
Re: [squid-users] what does this warning means?
Solved!!
I realized that at the same time of the warnings, i have at access.log
the next entries:
[28/Feb/2011:08:34:59 +0100] "POST
http://activate.pdfcreator-toolbar.org/toolbar/activate.php HTTP/0.0"
400 1733 NONE:NONE
[28/Feb/2011:08:34:59 +0100] "POST
http://activate2.pdfcreator-toolbar.org/toolbar/activate.php HTTP/0.0"
400 1733 NONE:NONE
So it is a toolbar that sometimes PDF Creator installs, and it's
trying to make those connections. Just uninstall it and everything ok.
Thanks Amos for putting me on the track.
2011/2/25 Amos Jeffries :
> On 25/02/11 22:39, Gontzal wrote:
>>
>> Hi list,
>>
>> I always have this messages on my cache.log, but i've never been
>> worried about them, it is just curiosity to know what this means and
>> if I can solve it:
>>
>> The message is:
>>
>> 2011/02/25 09:53:53| WARNING: HTTP header contains NULL characters
>> {Accept: */*^M
>> Content-Type: application/x-www-form-urlencoded}
>>
>
> Exactly what is says. The HTTP headers contain a NULL character.
> Older Squid will only display one {} section with the NULL byte being right
> after the last displayed character. 3.x will display two {} sections with
> the text "NULL" in between to indicate the problem better.
>
>
> Squid should be aborting the request unanswered and closing the TCP link
> involved. This is sign of an attack on the HTTP service, although it can be
> done by badly broken software unintentionally.
>
> In this case the Content-Type indicates the headers came from some client
> agent. I don't think its a browser since they are usually sending correct
> HTTP requests.
>
> If you have time it is worth tracking down where these come from and seeing
> what can be done to fix the source.
>
> Amos
> --
> Please be using
> Current Stable Squid 2.7.STABLE9 or 3.1.11
> Beta testers wanted for 3.2.0.5
>
Re: [squid-users] Squid - Read Error
On 28/02/11 19:16, Mohamed Adhil wrote: Dear Amos, I changed negative_ttl setting is at "0 seconds". I did trace also( trace root is workig fine) but sitll i am getting same read error. What I mean by packet trace is a complete record of the packets used during one such broken transaction. Usually this is generated by tcpdump with the -s0 option. Using that you will be able to see where the RST came from and hopefully what was sent (or not sent) that caused it. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.11 Beta testers wanted for 3.2.0.5
Re: [squid-users] tproxy- bridge mode vs routing mode
On 28/02/11 20:42, jiluspo wrote: both uses 2 ethernet. I tested both with ICMP & UDP latency and found out routing mode has higher latency compared to bridge. difference about .300ms. does routing really causes delay? There is no difference to Squid. What you are seeing is kernel layer overheads. Whether or not these matter is left to the opinion of the box administrator. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.11 Beta testers wanted for 3.2.0.5
