Re: [squid-users] Problem at loading advert with Squid 2.7 & 3.1
On Tue, Dec 27, 2011 at 10:37 AM, Amos Jeffries wrote: > On Mon, 26 Dec 2011 18:21:25 +0100, feralert wrote: >> >> Dear all, >> >> Squid is not loading an advert in a web page frame which loads fine >> when using a direct connection to the internet. >> The versions used are 2.7.STABLE9-2.1 and 3.1.6-1.2 both in a debian >> squezze with default configuration) >> >> The url the frame tries to load is: >> >> >> >> http://frame.cool.com/ad-frame/#&ad_wrap=ad-1&ad_url=http://ad.doubleclick.net/adj/site011.opus/home;Slot=Leaderboard;Pos=Top;Page=home;LoggedIn=No;tile=1;sz=728x90;&ad_timestamp=13249166700149 >> >> >> In squids log file I only see a line for >> 'http://frame.cool.com/ad-frame/': >> >> 1324916528.019 5405 192.168.5.237 TCP_REFRESH_MISS/200 445 GET >> http://frame.cool.com/ad-frame/ - DIRECT/67.228.247.179 text/html >> >> But no sight of a ''http://ad.doubleclick.net/adj/site011.opus/hom"e >> request. >> >> >> If I load the page in my computer (with a direct connection to the >> internet) and watch http traffic with the "http fox" firefox >> extension, I can see both requests. >> Hi Amos and all, > > The browser appears to have a bug. You demonstrated only one URL. > "http://frame.cool.com/ad-frame/"; I forgot to mention that I tried loading the page with firefox, chromium and IE and got the same results with all of them. And all of them are able to load the page fine when using a direct connection to the internet (without proxy). > > The '#' character is reserved for browser internal use. On a page view like > this the browsers move the top of the visible display area down to the part > of the page labeled > "&ad_wrap=ad-1&ad_url=http://ad.doubleclick.net/adj/site011.opus/home;Slot=Leaderboard;Pos=Top;Page=home;LoggedIn=No;tile=1;sz=728x90;&ad_timestamp=13249166700149";. > That label is not itself a URL. > > So... there is no second URL to Squid, unless there is a completely separate > lookup triggered by something else you did not mention. Even so, I see that my first message wasn't somehow displayed properly and might not be clear enough. Where it says 'But no sight of a ''"e request.', I meant to say: 'But no sight of a http://ad.doubleclick.net/adj/site011.opus/home request', meaning that there is a GET request for the first url but not for the second (as seen when not using squid but a direct connection to the internet). I have taken traces of the HTTP traffic with tcpdump, which I show below. The first one is using squid 2.7, which fails to load the ad and the second with the direct connection to the internet, which works flawlessly. As can be seen, when using squid, there is no data in the reply from the server, whereas there is some in the reply to the same request when using the direct connection, could the 'not requesting second url' error be due to this ?? - Squid 2.7: Host: frame.cool.com User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: es-es,es;q=0.8,en-us;q=0.5,en;q=0.3 Accept-Encoding: gzip, deflate Proxy-Connection: keep-alive Referer: http://www.cool.com/ Cookie: cool_all=24042dc2a0e328135316d197a53f45e8; cool_l=%7B%22n%22%3A4%2C%22d%22%3A1%7D; manifest=%7B%22pth%22%3A%7B%22rpge%22%3A%22home%22%7D%7D; __qca=P0-613459998-1324917196001; __utma=215533653.1105349504.1324917196.1324917196.1324917196.1; __utmz=215533653.1324917196.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmv=215533653.home HTTP/1.0 200 OK Server: nginx/1.0.4 Content-Type: text/html; charset=UTF-8 X-Powered-By: PHP/5.3.6 Expires: Tue, 27 Dec 2011 16:22:04 GMT Last-Modified: Mon, 26 Dec 2011 16:22:04 GMT Cache-Control: public, max-age=86400 Date: Mon, 26 Dec 2011 16:22:04 GMT X-Varnish: 1092645737 Age: 6176 Content-Length: 0 X-Cache: HIT from coruscant X-Cache-Lookup: HIT from coruscant:3128 Via: 1.1 varnish, 1.1 coruscant:3128 (squid/2.7.STABLE9) Connection: keep-alive Proxy-Connection: keep-alive - Direct conection to internet: (First, the frame request) GET /ad-frame/ HTTP/1.1 Host: frame.cool.com User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: es-es,es;q=0.8,en-us;q=0.5,en;q=0.3 Accept-Encoding: gzip, deflate Connection: keep-alive Referer: http://www.cool.com/ Cookie: cool_all=0395f7770045f92f9af22ee6079164ed; cool_l=%7B%22c%22%3A%22ES%22%2C%22n%22%3A1%2C%22d%22%3A1%7D; manifest=%7B%22pth%22%3A%7B%22rpge%22%3A%22home%22%7D%7D; __qca=P0-698414843-1324923294457 HTTP/1.1 200 OK Server: nginx/1.0.4 Co
[squid-users] Problem at loading advert with Squid 2.7 & 3.1
Dear all, Squid is not loading an advert in a web page frame which loads fine when using a direct connection to the internet. The versions used are 2.7.STABLE9-2.1 and 3.1.6-1.2 both in a debian squezze with default configuration) The url the frame tries to load is: http://frame.cool.com/ad-frame/#&ad_wrap=ad-1&ad_url=http://ad.doubleclick.net/adj/site011.opus/home;Slot=Leaderboard;Pos=Top;Page=home;LoggedIn=No;tile=1;sz=728x90;&ad_timestamp=13249166700149 In squids log file I only see a line for 'http://frame.cool.com/ad-frame/': 1324916528.019 5405 192.168.5.237 TCP_REFRESH_MISS/200 445 GET http://frame.cool.com/ad-frame/ - DIRECT/67.228.247.179 text/html But no sight of a ''http://ad.doubleclick.net/adj/site011.opus/hom"e request. If I load the page in my computer (with a direct connection to the internet) and watch http traffic with the "http fox" firefox extension, I can see both requests. Any help would be highly appreciated. Cheers! Fred. "UNIX is very simple, it just needs a genius to understand its simplicity." -- Dennis Ritchie, D.E.P.
[squid-users] TCP_MEM_HIT long elapsed time
Hi all, Maybe a dump question: why does it take so long for some TCP_MEM_HITS to 'show up', for example i got this: Dec 21 17:37:15 42721 192.X.X.X TCP_MEM_HIT/200 68873 GET http://example.com if TCP_MEM_HIT is "a valid copy of the requested object was in the cache and it was in memory, thus avoiding disk accesses." then I would seem logical it should come up almost immediately, but it seems it doesn't, anyone kind enough to explain me why? Cheers, Fred. "UNIX is very simple, it just needs a genius to understand its simplicity." -- Dennis Ritchie, D.E.P.
Re: [squid-users] TCP_MISS/504 when accesing www.citibank.com
On Sat, Nov 12, 2011 at 12:57 PM, Amos Jeffries wrote: > On 12/11/2011 1:02 a.m., feralert wrote: >> >> On Fri, Nov 11, 2011 at 1:22 AM, Amos Jeffries >> wrote: >>> >>> On 11/11/2011 4:16 a.m., feralert wrote: >>>> >>>> As a workaround (thanks to Radoslaw, who came up with the idea) I have >>>> done this: >>>> >>>> I added an entry for www.citibank.com pointing to the working server >>>> ip into /etc/hosts, then added "hosts_file /etc/hosts" into >>>> squid.conf. >>>> >>>> It works for the time being, but is not a final solution. >>> >>> NP: You should not have had to mark /etc/hosts in squid.conf (one less >>> place >>> to undo later), unless the file is in a strange location. >> >> I'm not sure if I undestand your tip. >> Do you mean that I can get rid of "/etc/hosts" argument to the >> "hosts_file" directive because that is actually the default setting >> for "hosts_file"?, or you mean that I can get rid of the whole >> directive since squid already takes into account the entries in >> /etc/hosts by default? > > Yes. > > You should not need the whole line because it is the default value. > > Removing just the value will tell Squid *not* to use the hosts file at all > (or kill Squid with a bungled config, one or the other). > > > Amos > Thank you Amos for the explanation and your help. >From last Friday both servers pointing to www.citibank.com are answering HTTP 1.0 petitions as they should, and therefore the problem is gone. This confirms it wasn't a squid 2.7 issue but citibank's, although I haven't got confirmation from them because they didn't care to answer my emails. Cheers, Fred.
Re: [squid-users] TCP_MISS/504 when accesing www.citibank.com
As a workaround (thanks to Radoslaw, who came up with the idea) I have done this: I added an entry for www.citibank.com pointing to the working server ip into /etc/hosts, then added "hosts_file /etc/hosts" into squid.conf. It works for the time being, but is not a final solution. "UNIX is very simple, it just needs a genius to understand its simplicity." -- Dennis Ritchie, D.E.P. On Wed, Nov 9, 2011 at 9:54 PM, feralert wrote: > Thanks Amos, > > A dig to www.citibank.com gives two different ips, changing one for > the other after a short period of time, one of them works fine and > squid 2.7 is able to get the page and the other one fails. > > Using tcpdump I have taken captures for both, while using debian > package for squid 2.7.stable9, these are the results: > > In both of them squid sends a "GET HTTP/1.0": > > 1) The one that works replays with a "HTTP/1.1 301 Moved > Permanently", that TCP session is close and another one is open squid > now asking for the new URL, finally being able to retrieve it and > serve it fine. > > 2) The other one doesnt reply at all to the inital "GET HTTP/1.0" > petition and therefore there are a few TCP retransmissions of it until > it receives a FIN,ACK packet from the server. > > I tried the same but using squid 3.1.6 instead, again in both servers, > and it works in both!. The difference being that squid 3.1 instead of > sending a"GET HTTP/1.0" sends a "GET HTTP/1.1", getting the "HTTP/1.1 > 301" response in both cases. So, to the light of these results, is > seems that the second server (the one that fails) does not > understand/talk HTTP/1.0, dont you think? is this possible?. > > > Thank you, > Fred. > > > > "UNIX is very simple, it just needs a genius to understand its simplicity." > -- Dennis Ritchie, D.E.P. > > > > On Wed, Nov 9, 2011 at 5:37 AM, Amos Jeffries wrote: >> On 9/11/2011 12:29 a.m., feralert wrote: >>> >>> Hi all, >>> >>> I'm new on the list so hi everyone and please do excuse my english. >>> >>> And now down to bussiness :) >>> >>> Im having trouble accessing www.citibank.com with squid 2.7. I'm >>> actually trying from two different platforms (Debian Lenny linux box >>> running 2.7.STABLE3 and NetBSD running 2.7.STABLE7), using different >>> connections to the internet and both getting weird results. >>> >>> >>> * ) With the NetBSD machine: >>> >>> What i usually get in the browser is a " (110) Connection timed out", >>> and a "TCP_MISS/504" in the servers logs. >>> >>>> While trying to retrieve the URL: http://www.citibank.com/ >>>> The following error was encountered: >>>> Connection to 192.193.219.58 Failed >>>> The system returned: >>>> (110) Connection timed out >>>> The remote host or network may be down. Please try the request again. >>> >>> 03:16:39 120355 10.5.5.236 TCP_MISS/504 1454 GET >>> http://www.citibank.com/ - DIRECT/192.193.219.58 text/html >>> >>> >>> But sometimes (ina very few rare occasions) it gets through and I see >>> the following: >>> >>> 1320746405.173 15615 10.5.5.236 TCP_MISS/200 6985 CONNECT >>> metrics1.citibank.com:443 - DIRECT/63.140.40.2 - >>> >>> And in the browser I see that I get redirected to >>> "https://online.citibank.com/US/Welcome.c";. From there on I have no >>> trouble surfing the website. Also, if I try go directly to >>> "https://online.citibank.com"; works every time. >>> >>> >>> >>> * ) With the linux one: >>> >>> With this one I never gotten through (maybe I havent tried as many >>> times), I also get the "Connection timed out" and "TCP_MISS/504", and >>> also a"Zero Sized Reply" and a "TCP_MISS/502": >>> >>> The browser shows: >>> >>>> While trying to retrieve the URL: http://www.citibank.com/ >>>> The following error was encountered: >>>> Zero Sized Reply >>>> Squid did not receive any data for this request. >>> >>> And in the logs I get: >>> 03:01:03 150396 10.5.5.236 TCP_MISS/502 1334 GET >>> http://www.citibank.com/ - DIRECT/192.193.103.222 text/html >>> 03:03:35 151215 10.5.5.236 TCP_MISS/504 1477 GET >>> http://www.citibank.com/favicon.ico - DIRECT/192.193.103.222 text/html >>> >>> >> >> These seems to be all symptoms of either TCP connection setup problems or >> ICMP blocking. The "Zero Sized Reply" hints that it is more likely MTU >> problems and ICMP blocking somewhere. Where Squid can locate and send data >> to the server, but nothing comes back (ie the packet sent was too big, but >> the ICMP reply telling the Squid machine to send smaller packts never got >> delivered). >> >> Amos >> >
Re: [squid-users] TCP_MISS/504 when accesing www.citibank.com
Thanks Amos, A dig to www.citibank.com gives two different ips, changing one for the other after a short period of time, one of them works fine and squid 2.7 is able to get the page and the other one fails. Using tcpdump I have taken captures for both, while using debian package for squid 2.7.stable9, these are the results: In both of them squid sends a "GET HTTP/1.0": 1) The one that works replays with a "HTTP/1.1 301 Moved Permanently", that TCP session is close and another one is open squid now asking for the new URL, finally being able to retrieve it and serve it fine. 2) The other one doesnt reply at all to the inital "GET HTTP/1.0" petition and therefore there are a few TCP retransmissions of it until it receives a FIN,ACK packet from the server. I tried the same but using squid 3.1.6 instead, again in both servers, and it works in both!. The difference being that squid 3.1 instead of sending a"GET HTTP/1.0" sends a "GET HTTP/1.1", getting the "HTTP/1.1 301" response in both cases. So, to the light of these results, is seems that the second server (the one that fails) does not understand/talk HTTP/1.0, dont you think? is this possible?. Thank you, Fred. "UNIX is very simple, it just needs a genius to understand its simplicity." -- Dennis Ritchie, D.E.P. On Wed, Nov 9, 2011 at 5:37 AM, Amos Jeffries wrote: > On 9/11/2011 12:29 a.m., feralert wrote: >> >> Hi all, >> >> I'm new on the list so hi everyone and please do excuse my english. >> >> And now down to bussiness :) >> >> Im having trouble accessing www.citibank.com with squid 2.7. I'm >> actually trying from two different platforms (Debian Lenny linux box >> running 2.7.STABLE3 and NetBSD running 2.7.STABLE7), using different >> connections to the internet and both getting weird results. >> >> >> * ) With the NetBSD machine: >> >> What i usually get in the browser is a " (110) Connection timed out", >> and a "TCP_MISS/504" in the servers logs. >> >>> While trying to retrieve the URL: http://www.citibank.com/ >>> The following error was encountered: >>> Connection to 192.193.219.58 Failed >>> The system returned: >>> (110) Connection timed out >>> The remote host or network may be down. Please try the request again. >> >> 03:16:39 120355 10.5.5.236 TCP_MISS/504 1454 GET >> http://www.citibank.com/ - DIRECT/192.193.219.58 text/html >> >> >> But sometimes (ina very few rare occasions) it gets through and I see >> the following: >> >> 1320746405.173 15615 10.5.5.236 TCP_MISS/200 6985 CONNECT >> metrics1.citibank.com:443 - DIRECT/63.140.40.2 - >> >> And in the browser I see that I get redirected to >> "https://online.citibank.com/US/Welcome.c";. From there on I have no >> trouble surfing the website. Also, if I try go directly to >> "https://online.citibank.com"; works every time. >> >> >> >> * ) With the linux one: >> >> With this one I never gotten through (maybe I havent tried as many >> times), I also get the "Connection timed out" and "TCP_MISS/504", and >> also a"Zero Sized Reply" and a "TCP_MISS/502": >> >> The browser shows: >> >>> While trying to retrieve the URL: http://www.citibank.com/ >>> The following error was encountered: >>> Zero Sized Reply >>> Squid did not receive any data for this request. >> >> And in the logs I get: >> 03:01:03 150396 10.5.5.236 TCP_MISS/502 1334 GET >> http://www.citibank.com/ - DIRECT/192.193.103.222 text/html >> 03:03:35 151215 10.5.5.236 TCP_MISS/504 1477 GET >> http://www.citibank.com/favicon.ico - DIRECT/192.193.103.222 text/html >> >> > > These seems to be all symptoms of either TCP connection setup problems or > ICMP blocking. The "Zero Sized Reply" hints that it is more likely MTU > problems and ICMP blocking somewhere. Where Squid can locate and send data > to the server, but nothing comes back (ie the packet sent was too big, but > the ICMP reply telling the Squid machine to send smaller packts never got > delivered). > > Amos >
[squid-users] TCP_MISS/504 when accesing www.citibank.com
Hi all, I'm new on the list so hi everyone and please do excuse my english. And now down to bussiness :) Im having trouble accessing www.citibank.com with squid 2.7. I'm actually trying from two different platforms (Debian Lenny linux box running 2.7.STABLE3 and NetBSD running 2.7.STABLE7), using different connections to the internet and both getting weird results. * ) With the NetBSD machine: What i usually get in the browser is a " (110) Connection timed out", and a "TCP_MISS/504" in the servers logs. >While trying to retrieve the URL: http://www.citibank.com/ >The following error was encountered: >Connection to 192.193.219.58 Failed >The system returned: >(110) Connection timed out >The remote host or network may be down. Please try the request again. 03:16:39 120355 10.5.5.236 TCP_MISS/504 1454 GET http://www.citibank.com/ - DIRECT/192.193.219.58 text/html But sometimes (ina very few rare occasions) it gets through and I see the following: 1320746405.173 15615 10.5.5.236 TCP_MISS/200 6985 CONNECT metrics1.citibank.com:443 - DIRECT/63.140.40.2 - And in the browser I see that I get redirected to "https://online.citibank.com/US/Welcome.c";. From there on I have no trouble surfing the website. Also, if I try go directly to "https://online.citibank.com"; works every time. * ) With the linux one: With this one I never gotten through (maybe I havent tried as many times), I also get the "Connection timed out" and "TCP_MISS/504", and also a"Zero Sized Reply" and a "TCP_MISS/502": The browser shows: > While trying to retrieve the URL: http://www.citibank.com/ > The following error was encountered: > Zero Sized Reply >Squid did not receive any data for this request. And in the logs I get: 03:01:03 150396 10.5.5.236 TCP_MISS/502 1334 GET http://www.citibank.com/ - DIRECT/192.193.103.222 text/html 03:03:35 151215 10.5.5.236 TCP_MISS/504 1477 GET http://www.citibank.com/favicon.ico - DIRECT/192.193.103.222 text/html Machines are: - Linux box (debian lenny). # squid -v Squid Cache: Version 2.7.STABLE3 configure options: '--prefix=/usr' '--exec_prefix=/usr' '--bindir=/usr/sbin' '--sbindir=/usr/sbin' '--libexecdir=/usr/lib/squid' '--sysconfdir=/etc/squid' '--localstatedir=/var/spool/squid' '--datadir=/usr/share/squid' '--enable-async-io' '--with-pthreads' '--enable-storeio=ufs,aufs,coss,diskd,null' '--enable-linux-netfilter' '--enable-arp-acl' '--enable-epoll' '--enable-removal-policies=lru,heap' '--enable-snmp' '--enable-delay-pools' '--enable-htcp' '--enable-cache-digests' '--enable-underscores' '--enable-referer-log' '--enable-useragent-log' '--enable-auth=basic,digest,ntlm,negotiate' '--enable-negotiate-auth-helpers=squid_kerb_auth' '--enable-carp' '--enable-follow-x-forwarded-for' '--with-large-files' '--with-maxfd=65536' 'i386-debian-linux' 'build_alias=i386-debian-linux' 'host_alias=i386-debian-linux' 'target_alias=i386-debian-linux' 'CFLAGS=-Wall -g -O2' 'LDFLAGS=' 'CPPFLAGS=' - NetBSD 5.0.2 machine: [root@prometeo ~]# squid -v Squid Cache: Version 2.7.STABLE7 configure options: '--sysconfdir=/usr/pkg/etc/squid' '--localstatedir=/var/squid' '--datarootdir=/usr/pkg/share/squid' '--enable-auth=basic,digest,ntlm' '--enable-cachemgr-hostname=localhost' '--enable-delay-pools' '--enable-icmp' '--enable-removal-policies=lru,heap' '--enable-poll' '--enable-underscores' '--enable-storeio=ufs null aufs coss diskd' '--with-aio' '--enable-ipf-transparent' '--enable-carp' '--enable-snmp' '--enable-ssl' '--with-openssl=/usr' '--enable-basic-auth-helpers=getpwnam MSNT NCSA YP PAM' '--enable-digest-auth-helpers=password' '--enable-ntlm-auth-helpers=fakeauth SMB' '--enable-external-acl-helpers=ip_user unix_group' '--prefix=/usr/pkg' '--build=i386--netbsdelf' '--host=i386--netbsdelf' '--mandir=/usr/pkg/man' 'build_alias=i386--netbsdelf' 'host_alias=i386--netbsdelf' 'CC=cc' 'CFLAGS=-O2 -pthread -I/usr/include' 'LDFLAGS=-L/usr/lib -Wl,-R/usr/lib -pthread -Wl,-R/usr/pkg/lib' 'LIBS=' 'CPPFLAGS=-I/usr/include' I can post both squid.conf files if requested. Any hints? Cheers, Fred.
