On 12.01.2012 04:25, Andreas Schulz wrote:
Hi folks,
we have a very special problem with our proxy environment. It
concerns
only ONE uri http://www.mediaassetbox.com/client/escada. Other uris
are
working properly.
Unfortunately this is a very bad uri because it works only with
flash.
Nevertheless our customer is working with it and we have a
performance
issue.
If page starts to load it need approx. 60-70 seconds until the blue
progress bar under the login field disappears.
If I use another proxy product - eg. IWSS - the page loads in about
30
seconds. Also with direct internet connection we have this value ...
So far so good - strange behaviour starts after working on the
problem.
Starting strace on the squid process - the performance increases to
direct
internet connection speed.
Next we started debugging in squid itself - ALL,3 - without strace -
the
performance increases again. Starting with debug section 0 we found
out
that 'debug_options 5,3' (or 5,5 ...) increases the performance as
fast
as a direct connection.
So doing I/O to a disk log somehow speeds up TCP throughput? strange
This sounds a bit like the speed problems we see with very low traffic
rates. When the I/O loops get very few requests through they end up
pausing in 10ms time chunks each processing cycle to prevent CPU
overload doing lots of processing on very small amounts of bytes.
Taking a wild guess; the debug log I/O might be raising the number of
total I/O being handled each second over that low-speed bump. Unlikely
but possible.
What we already did without success
- disable ipv6 in os
- strip configuration to minimum
- using a cache_peer parent configuration (the IWSS proxy)
- tried to find out, which systemcalls 'increases' the squid (see
statistics below)
Now some details about the system:
- OS - Debian Squeeze - Linux xxx 2.6.32-5-amd64 #1 SMP Thu Nov 3
03:41:26 UTC 2011 x86_64 GNU/Linux
- Squid - 3.1.6-1.2+squeeze2
This release is getting a bit old now and has a few I/O buffering bugs
in it that may be related.
Please try the 3.1.18 Debian package from Wheezy / testing repositories
(may require some dependency updates as well).
We can also provide HTTPFox (Firefox extension) lines for fast and
slow
connections.
We searched the mailing list and found
http://www.mail-archive.com/squid-users@squid-cache.org/msg33267.html
-
but there was no really helpful information. Other entries doesn't
match.
We collected the strace statistics only for this session:
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
97.66 0.004015 1 3209 epoll_wait
1.24 0.000051 0 145 145 connect
145 connect() calls in 0.05 ms, all failing? does not seem right at
all.
Given the time measure I don't think its related, but probably worth
knowing and fixing. Did the section 5 trace show what was going on here?
0.71 0.000029 0 870 recvmsg
0.22 0.000009 0 912 epoll_ctl
0.17 0.000007 0 299 getsockname
0.00 0.000000 0 484 2 read
0.00 0.000000 0 494 write
0.00 0.000000 0 444 close
0.00 0.000000 0 435 socket
0.00 0.000000 0 16 7 accept
0.00 0.000000 0 290 sendto
0.00 0.000000 0 290 bind
0.00 0.000000 0 290 setsockopt
0.00 0.000000 0 145 getsockopt
0.00 0.000000 0 616 fcntl
0.00 0.000000 0 1 getrusage
------ ----------- ----------- --------- --------- ----------------
100.00 0.004111 8940 154 total
Our squid config:
***********
pid_filename /var/run/squid3-special.pid
http_port 8081
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
QUERY can die.
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
Add here:
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
<snip>
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny CONNECT !SSL_ports
http_access deny !CONNECT !Safe_ports
http_access allow all
Eeek! nearly unlimited access to the whole Internet. Why?
icp_access deny all
#debug_options 5,5
***********
network layout is:
client -> firewall -> proxy -> firewall -> internet
Does anyone has an idea what could be the cause for this strange
behaviour?
--
Andreas Schulz
HTH
Amos
