more profiling

2006-09-15 Thread Adrian Chadd
I've done some more profiling. Same test setup, current squid3 source. perfmon output: Samples Self % Total % Function 18915 26.09% 26.09% MemPool::clean(long) 9266 12.78% 38.86% Me

Re: more profiling

2006-09-15 Thread Adrian Chadd
This is with cache_mem 512 MB btw. On Fri, Sep 15, 2006, Adrian Chadd wrote: > I've done some more profiling. Same test setup, current squid3 source. > > perfmon output: > > > Samples Se

Re: more profiling

2006-09-15 Thread Henrik Nordstrom
fre 2006-09-15 klockan 16:25 +0800 skrev Adrian Chadd: > I've done some more profiling. Same test setup, current squid3 source. > > perfmon output: > > > Samples Self % Total % Functio

Re: more profiling

2006-09-18 Thread Adrian Chadd
> I'd on other hand would like to actually see it being used on purpose, for > once ;) Yeah, I promise I'll get around to doing it before I curse the allocator. :) That said, adding in the intelligence needed to bzero or not bzero the buffers would make a noticable impact to the current code. Al

Re: more profiling

2006-09-18 Thread andre
On 15 Sep 2006 at 10:45, Henrik Nordstrom wrote: > fre 2006-09-15 klockan 16:25 +0800 skrev Adrian Chadd: > > I've done some more profiling. Same test setup, current squid3 source. > >

Re: more profiling

2006-09-18 Thread andre
> On Fri, Sep 15, 2006, Adrian Chadd wrote: > > I've done some more profiling. Same test setup, current squid3 source. > > > > perfmon output: > > > > > > Samples

Re: more profiling

2006-09-18 Thread Adrian Chadd
Here's the latest 5 minute snapshot. Squid-3 has hit the store mem limit and is now having to delete in-memory objects to make room for incoming requests. (This is using datacomm-1. I wanted to see the worst-case performance - ie, squid has to keep removing objects without getting any benefit fr

Re: more profiling

2006-09-18 Thread Adrian Chadd
Here's the hourly snapshot. Adrian Last 1 hour averages: (Cumulated time: 25411206816740, 2782.06 sec) Probe NameEvents cumulated time best case average worst case Rate / sec % in int PROF_UNACCOUNTED 105984696 1728110104996 0 16305 424

Re: more profiling

2006-09-18 Thread Adrian Chadd
And here's why clean/getStats are being called so often! #0 0x08106a5a in MemPool::clean (this=0x846f7e0, maxage=55) at MemPool.cc:651 #1 0x08105632 in MemPool::getStats (this=0x846f7e0, stats=0xbf845450) at MemPool.cc:720 #2 0x080b5ca8 in mem_node::InUseCount () at mem_node.cc:82 #3 0x0

Re: more profiling

2006-09-19 Thread Adrian Chadd
On Tue, Sep 19, 2006, Adrian Chadd wrote: > You did say the getStats() call was very expensive. > > Aha! Could someone please beat me to fixing this? I need to attend to studies > for a few days. Too late! http://www.creative.net.au/diffs/20060919-squid3-mempools.diff This seems to be doing th

Re: more profiling

2006-09-19 Thread Andres Kroonmaa
On 19 Sep 2006 at 13:17, Adrian Chadd wrote: > Here's the hourly snapshot. > > Adrian > > Last 1 hour averages: (Cumulated time: 25411206816740, 2782.06 sec) > > Probe NameEvents cumulated time best case average > worst case Rate / sec % in int > > PROF_UNACCOUNTED

Re: more profiling

2006-09-19 Thread Adrian Chadd
On Tue, Sep 19, 2006, Andres Kroonmaa wrote: > Is that all? or did you paste only part of probes? > When I used it on my prod caches, I enabled quite alot > more probes. When the profiling feature was included, > there were other concurrent changes (eg chunked mempools) > and in submitted patch ma

Re: more profiling

2006-09-19 Thread Adrian Chadd
On Tue, Sep 19, 2006, Andres Kroonmaa wrote: > Is that all? or did you paste only part of probes? > When I used it on my prod caches, I enabled quite alot > more probes. When the profiling feature was included, > there were other concurrent changes (eg chunked mempools) > and in submitted patch ma

Re: more profiling

2006-09-19 Thread Gonzalo Arana
On 9/19/06, Adrian Chadd <[EMAIL PROTECTED]> wrote: On Tue, Sep 19, 2006, Andres Kroonmaa wrote: ... > Since then quite alot of changes have happened, so I'd > suggest to look at the gprof stats to decide what funcs > to probe with hires prof and add them. Yeah, I'm thinking that too. Is

Re: more profiling

2006-09-19 Thread Adrian Chadd
On Tue, Sep 19, 2006, Gonzalo Arana wrote: > Is hires profiling *that* heavy? I've used it in my production squids > (while I've used squid3) and the overhead was neglible. It doesn't seem to be that heavy. > There is a comment in profiling.h claiming that rdtsc (for x86 arch) > stalls CPU pipes

Re: more profiling

2006-09-19 Thread Gonzalo Arana
On 9/19/06, Adrian Chadd <[EMAIL PROTECTED]> wrote: On Tue, Sep 19, 2006, Gonzalo Arana wrote: > There is a comment in profiling.h claiming that rdtsc (for x86 arch) > stalls CPU pipes. That's not what Intel documentation says (page 213 > -numbered as 4-209- of the Intel Architecture Software

Re: more profiling

2006-09-19 Thread Adrian Chadd
On Tue, Sep 19, 2006, Gonzalo Arana wrote: > >Cute! It'd still be a good idea to explicitly state beginning/end where > >appropriate. What might be nice is a "i was deallocated at the end of the > >function rather than being deallocated explicitly" counter so things > >could be noted? > > I don't

Re: more profiling

2006-09-19 Thread Gonzalo Arana
On 9/19/06, Adrian Chadd <[EMAIL PROTECTED]> wrote: On Tue, Sep 19, 2006, Gonzalo Arana wrote: > >Bout the only really crinkly point I see atm is the zero-sized reply > >stuff. I have a sneaking sense that the forwarder code is still slightly > >broken. > > Nothing the squid-guru-team c

Re: more profiling

2006-09-19 Thread Adrian Chadd
On Tue, Sep 19, 2006, Gonzalo Arana wrote: > >Hey, want to join? :) > > Tempting invitation :) but for the moment I'll be commited to > another projects (I believe for 2 months or so). But once I finish > on-going projects, I'd love to join. > > I manage about 20 squid servers for a hundred

Re: more profiling

2006-09-19 Thread Andres Kroonmaa
On 19 Sep 2006 at 21:10, Adrian Chadd wrote: > On Tue, Sep 19, 2006, Gonzalo Arana wrote: > > > Is hires profiling *that* heavy? I've used it in my production squids > > (while I've used squid3) and the overhead was neglible. > > It doesn't seem to be that heavy. hires profiling was designed for

Re: more profiling

2006-09-19 Thread Gonzalo Arana
On 9/19/06, Andres Kroonmaa <[EMAIL PROTECTED]> wrote: On 19 Sep 2006 at 21:10, Adrian Chadd wrote: > On Tue, Sep 19, 2006, Gonzalo Arana wrote: > > > Is hires profiling *that* heavy? I've used it in my production squids > > (while I've used squid3) and the overhead was neglible. > > It doesn't s

Re: more profiling

2006-09-19 Thread Andres Kroonmaa
On 19 Sep 2006 at 14:12, Gonzalo Arana wrote: > On 9/19/06, Andres Kroonmaa <[EMAIL PROTECTED]> wrote: > > > On Tue, Sep 19, 2006, Gonzalo Arana wrote: > > > > > > > There is a comment in profiling.h claiming that rdtsc (for x86 arch) > > > > stalls CPU pipes. That's not what Intel documentation

Re: more profiling

2006-09-19 Thread Henrik Nordstrom
tis 2006-09-19 klockan 21:10 +0300 skrev Andres Kroonmaa: > Because gprof call graph is determinate, but profile > information is statistical approximation. For vast > majority of cases its good enough. For eg. in this > case it seems that gprof wasn't that much off afterall > as Adrian found a bu