Re: [squid-users] squid3 keeps many idle connections
On Thu, 23 Oct 2008 23:39:01 +1300 Amos Jeffries <[EMAIL PROTECTED]> wrote: > Malte Schröder wrote: > > On Thu, 23 Oct 2008 01:10:58 +0200 > > Henrik Nordstrom <[EMAIL PROTECTED]> wrote: > > > >> On ons, 2008-10-22 at 11:31 +0200, Malte Schröder wrote: > >> Not normal. > >> > >> Squid version? > > > > 3.0.STABLE10 > > > >> And how did you measure these? You are not counting TIME_WAIT sockets > >> are you? > > > > by getting cache_object://localhost/filedescriptors and checking > > sockets in ESTABLISHED state. > > > > I attached four files with the output of the filedescriptor page. > > "start" is the state directly after the launch of squid, 1 is the state > > after loading slashdot.org, 2 after reloading slashdot and 3 after > > an additional reload. I increased pconn_timeout to 5 minutes so the > > effect becomes more visible. The parent is a squid 2.6.STABLE20. > > > > Okay, this is getting into stuff for squid-dev. cc'd. > > If you are able build your own squid this patch will at least lead an > indication of whats going on with the idle FD. Patch applied, filedescriptors after some reloads on slashdot attached. > > Amos Active file descriptors: File Type Tout Nread * Nwrite * Remote AddressDescription -- - -- 5 Log 0 00 /var/log/squid3/cache.log 7 File 0 0 1131 /var/log/squid3/access_mod.log 8 Socket0 0 44 .0DNS Socket 9 Pipe 0 00unlinkd -> squid 10 Socket0 00 .0HTTP Socket 11 Socket0 00 .0HTTP Socket 12 Pipe 0 00squid -> unlinkd 13 Socket0 00 .0HTTP Socket 14 Socket0 00 .0SNMP Port 15 Socket4 422*1676 127.0.0.1.8086 idle connection for test_parent:8086 16 Socket4 423*1614 127.0.0.1.8086 idle connection for test_parent:8086 17 Socket4 423*1682 127.0.0.1.8086 idle connection for test_parent:8086 18 Socket4 424*1664 127.0.0.1.8086 idle connection for test_parent:8086 19 Socket4 422*1676 127.0.0.1.8086 idle connection for test_parent:8086 20 Socket4 425*1616 127.0.0.1.8086 idle connection for test_parent:8086 21 Socket4 425*1616 127.0.0.1.8086 idle connection for test_parent:8086 22 Socket 15 0*1488 10.248.7.86.80 http://squid3-test/cgi-bin/cachemgr3.cgi?host=localhost&port=70 23 Socket 1440 1860* 127.0.0.1.33298 Reading next request 25 Socket4 424*1608 127.0.0.1.8086 idle connection for test_parent:8086 27 Socket4 425*1610 127.0.0.1.8086 idle connection for test_parent:8086 28 Socket4 424*1608 127.0.0.1.8086 idle connection for test_parent:8086 29 Socket4 425*1604 127.0.0.1.8086 idle connection for test_parent:8086 30 Socket4 425*1610 127.0.0.1.8086 idle connection for test_parent:8086 31 Socket4 425*1610 127.0.0.1.8086 idle connection for test_parent:8086 32 Socket4 425*1676 127.0.0.1.8086 idle connection for test_parent:8086 33 Socket4 422*1684 127.0.0.1.8086 idle connection for test_parent:8086 34 Socket4 423*1692 127.0.0.1.8086 idle connection for test_parent:8086 35 Socket4 425*1604 127.0.0.1.8086 idle connection for test_parent:8086 36 Socket4 423*1706 127.0.0.1.8086 idle connection for test_parent:8086 37 Socket4 425*1610 127.0.0.1.8086 idle connection for test_parent:8086 38 Socket4 423*1614 127.0.0.1.8086 idle connection for test_parent:8086 39 Socket4 425*1634 127.0.0.1.8086 idle connection for test_parent:8086 40 Socket4 424*1664 127.0.0.1.8086 idle connection for test_parent:8086 41 Socket4 425*1634 127.0.0.1.8086 idle connection for test_parent:8086 42 Socket4 425*1628 127.0.0.1.8086 idle connection for test_parent:8086 43 Socket4 425*1634 127.0.0.1.8086 idle connection for test_parent:8086 44 Socket4 425*1640 127.0.0.1.8086 idle connection for test_parent:8086 45 Socket4 425*1652 127.0.0.1.8086 idle connection for test_parent:8086 46 Socket4 425*1654 127.0.0.1.8086 idle connection for test_parent:8086 47 Socket4 425*1646 127.0.0.1.8086 idle connection for test_parent:8086 48 Socket4 425*1652 127.0.0.1.8086 idle connection for test_parent:8086 49 Socket4 425*1640 1
Re: [squid-users] squid3 keeps many idle connections
Malte Schröder wrote:
On Thu, 23 Oct 2008 01:10:58 +0200
Henrik Nordstrom <[EMAIL PROTECTED]> wrote:
On ons, 2008-10-22 at 11:31 +0200, Malte Schröder wrote:
Not normal.
Squid version?
3.0.STABLE10
And how did you measure these? You are not counting TIME_WAIT sockets
are you?
by getting cache_object://localhost/filedescriptors and checking
sockets in ESTABLISHED state.
I attached four files with the output of the filedescriptor page.
"start" is the state directly after the launch of squid, 1 is the state
after loading slashdot.org, 2 after reloading slashdot and 3 after
an additional reload. I increased pconn_timeout to 5 minutes so the
effect becomes more visible. The parent is a squid 2.6.STABLE20.
Okay, this is getting into stuff for squid-dev. cc'd.
If you are able build your own squid this patch will at least lead an
indication of whats going on with the idle FD.
Amos
--
Please use Squid 2.7.STABLE4 or 3.0.STABLE9
=== modified file 'src/pconn.cc'
--- src/pconn.cc 2008-07-12 15:11:10 +
+++ src/pconn.cc 2008-10-23 10:34:18 +
@@ -235,7 +235,7 @@
IdleConnList *list;
const char *aKey;
-LOCAL_ARRAY(char, desc, FD_DESC_SZ);
+char desc[FD_DESC_SZ];
if (fdUsageHigh())
{
@@ -262,7 +262,7 @@
list->push(fd);
assert(!comm_has_incomplete_write(fd));
-snprintf(desc, FD_DESC_SZ, "%s idle connection", host);
+snprintf(desc, FD_DESC_SZ, "idle connection for %s", aKey);
fd_note(fd, desc);
debugs(48, 3, "PconnPool::push: pushed FD " << fd << " for " << aKey);
}
Re: [squid-users] squid3 keeps many idle connections
On Thu, 23 Oct 2008 01:10:58 +0200 Henrik Nordstrom <[EMAIL PROTECTED]> wrote: > On ons, 2008-10-22 at 11:31 +0200, Malte Schröder wrote: > Not normal. > > Squid version? 3.0.STABLE10 > > And how did you measure these? You are not counting TIME_WAIT sockets > are you? by getting cache_object://localhost/filedescriptors and checking sockets in ESTABLISHED state. I attached four files with the output of the filedescriptor page. "start" is the state directly after the launch of squid, 1 is the state after loading slashdot.org, 2 after reloading slashdot and 3 after an additional reload. I increased pconn_timeout to 5 minutes so the effect becomes more visible. The parent is a squid 2.6.STABLE20. > > Regards > Henrik > Active file descriptors: File Type Tout Nread * Nwrite * Remote AddressDescription -- - -- 2 Log 0 00 /var/log/squid3/cache.log 6 File 0 0 2811 /var/log/squid3/access_mod.log 7 Pipe 0 00unlinkd -> squid 8 Socket0 110 110 .0DNS Socket 9 Socket0 00 .0HTTP Socket 10 Pipe 0 00squid -> unlinkd 11 Socket0 00 .0HTTP Socket 12 Socket0 00 .0HTTP Socket 14 Socket0 00 .0SNMP Port 15 File 0 00 /var/spool/squid3/swap-state-var.spool.squid3 16 Socket 1440 699* 0 127.0.0.1.57972 Reading next request 17 Socket 15 0*1488 10.248.7.86.80 http://squid3-test/cgi-bin/cachemgr3.cgi?host=localhost&port=70 18 Socket 1440 1860* 127.0.0.1.57973 Reading next request - Diese E-Mail wurde digital signiert. Sie koennen anhand der Signatur nachvollziehen, dass die E-Mail wirklich von Vattenfall stammt und dass deren Inhalt auf dem Transportweg nicht manipuliert wurde. Weitere Informationen ueber die eingesetzten Signaturen finden Sie auf unserer Website unter http://www.vattenfall.de/pki. - Active file descriptors: File Type Tout Nread * Nwrite * Remote AddressDescription -- - -- 2 Log 0 00 /var/log/squid3/cache.log 6 File 0 0 8007 /var/log/squid3/access_mod.log 7 Pipe 0 00unlinkd -> squid 8 Socket0 301 177 .0DNS Socket 9 Socket0 00 .0HTTP Socket 10 Pipe 0 00squid -> unlinkd 11 Socket0 00 .0HTTP Socket 12 Socket0 00 .0HTTP Socket 14 Socket0 00 .0SNMP Port 15 File 0 00 /var/spool/squid3/swap-state-var.spool.squid3 16 Socket4 422*1676 127.0.0.1.8086 test_parent idle connection 17 Socket4 423*1682 127.0.0.1.8086 test_parent idle connection 18 Socket 14404547*2685 127.0.0.1.57975 Reading next request 19 Socket4 425*1616 127.0.0.1.8086 test_parent idle connection 20 Socket 14404488*2685 127.0.0.1.57976 Reading next request 21 Socket4 424*1608 127.0.0.1.8086 test_parent idle connection 22 Socket 14403790*2150 127.0.0.1.57977 Reading next request 23 Socket4 425*1610 127.0.0.1.8086 test_parent idle connection 24 Socket 14403773*2147 127.0.0.1.57978 Reading next request 25 Socket4 425*1604 127.0.0.1.8086 test_parent idle connection 26 Socket 14404574*2686 127.0.0.1.57979 Reading next request 27 Socket4 425*1610 127.0.0.1.8086 test_parent idle connection 28 Socket4 425*1676 127.0.0.1.8086 test_parent idle connection 29 Socket4 422*1684 127.0.0.1.8086 test_parent idle connection 30 Socket4 423*1692 127.0.0.1.8086 test_parent idle connection 31 Socket 14403858*2148 127.0.0.1.57980 Reading next request 32 Socket4 423*1706 127.0.0.1.8086 test_parent idle connection 33 Socket 14403087*1610 127.0.0.1.57981 Reading next request 34 Socket 14403043*1611 127.0.0.1.57982 Reading next request 35 Socket4 423*1680 127.0.0.1.8086 tes
Re: [squid-users] squid3 keeps many idle connections
On ons, 2008-10-22 at 11:31 +0200, Malte Schröder wrote: > Hello, > Squid3 seems to keep a LOT (over a thousand) idle connections to its > parent proxy. Not normal. Squid version? And how did you measure these? You are not counting TIME_WAIT sockets are you? Regards Henrik signature.asc Description: This is a digitally signed message part
Re: [squid-users] squid3 keeps many idle connections
No, it's a parent cache to another squid (2.7.STABLE5). It talks to a WebWasher content filter. On Wed, 22 Oct 2008 14:39:17 +0200 "Itzcak Pechtalt" <[EMAIL PROTECTED]> wrote: > From: "Itzcak Pechtalt" <[EMAIL PROTECTED]> > To: "Malte Schröder" <[EMAIL PROTECTED]> > Cc: squid-users@squid-cache.org > Subject: Re: [squid-users] squid3 keeps many idle connections > Date: Wed, 22 Oct 2008 14:39:17 +0200 > > Hi, > If you use tranparent cache, you will have several connections open > per client IP. > > Itzcak > > On Wed, Oct 22, 2008 at 11:31 AM, Malte Schröder <[EMAIL PROTECTED]> wrote: > > Hello, > > Squid3 seems to keep a LOT (over a thousand) idle connections to its > > parent proxy. To me it seems as if doesn't properly reuse existing > > connections. Is there a way to find out what's going on? From what I > > can see there are not more than about two dozens requests at the > > same time. I already reduced pconn_timeout to 10 seconds to reduce the > > number of connections that are open to around 100. > > > > Kind regards > > Malte > >
Re: [squid-users] squid3 keeps many idle connections
Hi, If you use tranparent cache, you will have several connections open per client IP. Itzcak On Wed, Oct 22, 2008 at 11:31 AM, Malte Schröder <[EMAIL PROTECTED]> wrote: > Hello, > Squid3 seems to keep a LOT (over a thousand) idle connections to its > parent proxy. To me it seems as if doesn't properly reuse existing > connections. Is there a way to find out what's going on? From what I > can see there are not more than about two dozens requests at the > same time. I already reduced pconn_timeout to 10 seconds to reduce the > number of connections that are open to around 100. > > Kind regards > Malte >
[squid-users] squid3 keeps many idle connections
Hello, Squid3 seems to keep a LOT (over a thousand) idle connections to its parent proxy. To me it seems as if doesn't properly reuse existing connections. Is there a way to find out what's going on? From what I can see there are not more than about two dozens requests at the same time. I already reduced pconn_timeout to 10 seconds to reduce the number of connections that are open to around 100. Kind regards Malte
