Re: [PATCH] Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 18:58 -0600 skrev Alex Rousskov: > In general, the caller should forget about the call after making it > (i.e., after scheduling it). We can make comm different, but I am not > sure it is a good idea from the clarity/simplicity point of view. Which kind of rules out using ca

Re: Problem with CVS pserver?

2008-04-08 Thread Amos Jeffries
Benno Rice wrote: I've started getting this today: > cvs -z9 -d :pserver:[EMAIL PROTECTED]:/squid co squid open /dev/null failed Operation not supported Has something broken on the CVS server? The central repository has been migrated from CVS to BZR. Only the developers mirror still provid

Re: [MERGE] eCAP support, part 1: Loadable modules and ICAP-independent Squid core.

2008-04-08 Thread Adrian Chadd
On Tue, Apr 08, 2008, Alex Rousskov wrote: > > I'd like to see the pconn stuff with AsyncCalls sorted out first before > > more potentially disruptive stuff is included. > > Why wait? The two issues are unrelated, being worked on by different > folks, and the suggested changes are relatively beni

Re: [MERGE] eCAP support, part 1: Loadable modules and ICAP-independent Squid core.

2008-04-08 Thread Alex Rousskov
On Wed, 2008-04-09 at 13:07 +0800, Adrian Chadd wrote: > On Tue, Apr 08, 2008, Alex Rousskov wrote: > > Hi, > > > > FYI: I will commit this tomorrow if there are still no objections. > > I'd like to see the pconn stuff with AsyncCalls sorted out first before > more potentially disruptive stuf

Re: Suggested 3.0 merge candidates

2008-04-08 Thread Amos Jeffries
Tsantilas Christos wrote: Alex Rousskov wrote: On Tue, 2008-04-08 at 17:04 +0200, Henrik Nordstrom wrote: tis 2008-04-08 klockan 13:23 +1200 skrev Amos Jeffries: +1. I'm just waiting on you all to agree that its tested enough. If you want to do the merge yourself Henrik, I'm okay with that.

Re: [MERGE] eCAP support, part 1: Loadable modules and ICAP-independent Squid core.

2008-04-08 Thread Adrian Chadd
On Tue, Apr 08, 2008, Alex Rousskov wrote: > Hi, > > FYI: I will commit this tomorrow if there are still no objections. I'd like to see the pconn stuff with AsyncCalls sorted out first before more potentially disruptive stuff is included. Adrian

Re: [MERGE] eCAP support, part 1: Loadable modules and ICAP-independent Squid core.

2008-04-08 Thread Alex Rousskov
Hi, FYI: I will commit this tomorrow if there are still no objections. Thanks, Alex. On Fri, 2008-04-04 at 23:53 -0600, Alex Rousskov wrote: > Hello, > > I am done with the first part of eCAP work and would like to commit > it to Squid3 trunk. The first part includes (a) initial suppor

Problem with CVS pserver?

2008-04-08 Thread Benno Rice
I've started getting this today: > cvs -z9 -d :pserver:[EMAIL PROTECTED]:/squid co squid open /dev/null failed Operation not supported Has something broken on the CVS server? -- Benno Rice [EMAIL PROTECTED]

Re: Suggested 3.0 merge candidates

2008-04-08 Thread Amos Jeffries
> tis 2008-04-08 klockan 13:23 +1200 skrev Amos Jeffries: > >> Is there a security problem with them being set? >> IMO its just a cleanup otherwise (already voted those not to go back). > > It's not even a source change, just a bzr attributes cleanup. But the > current situation may confuse some un

Re: TPROXY support in Squid 3

2008-04-08 Thread Amos Jeffries
> On Tue, 2008-04-08 at 15:15 +0200, Henrik Nordstrom wrote: >> tis 2008-04-08 klockan 17:59 +1200 skrev Amos Jeffries: >> > Currently: >> >fde::flags::transparent == 'intercept/non-intercept' >> >fde::flags::tproxy == real-transparent/non-transparent >> >(new) COMM_TRANSPARENT == real-

Re: [PATCH] Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Alex Rousskov
On Wed, 2008-04-09 at 01:51 +0200, Henrik Nordstrom wrote: > tis 2008-04-08 klockan 12:55 -0600 skrev Alex Rousskov: > > BTW, if somebody commits that patch, please polish the reason phrase in > > the cancel() call to something more specific. > > The submitted pach didn't work for two reasons. In

Re: [PATCH] Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Alex Rousskov
On Wed, 2008-04-09 at 02:05 +0200, Henrik Nordstrom wrote: > tis 2008-04-08 klockan 17:55 -0600 skrev Alex Rousskov: > > is worth the bugs and complexity we are _removing_. Also, the future SMP > > support may operate within the same nothing-is-immediate problem space. > > In other words, let's ge

Re: [PATCH] Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 22:23 +0200, Henrik Nordstrom wrote: > I still think the cbdata model is generally preferrable. Much less to > keep track of. For pconn it would need one cbdata object per idle fd, > and having only that object invalidated when the fd is taken out of > the pconn queue for wh

Re: commSetTimeout looks odd.. maybe

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 18:04 -0600 skrev Alex Rousskov: > Yes, it is setting the fd member of the call[back]. Yes, I realized that after sending the message. The syntax is very ugly, but I do not have a better proposal at the moment. I'll guess I get used to it eventually... Regards Henrik

Re: [PATCH] Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 17:55 -0600 skrev Alex Rousskov: > is worth the bugs and complexity we are _removing_. Also, the future SMP > support may operate within the same nothing-is-immediate problem space. > In other words, let's get used to it! :-) Note: For SMP to work out you want as few hops be

Re: commSetTimeout looks odd.. maybe

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 22:38 +0200, Henrik Nordstrom wrote: > in commSetTimeout there is the following fragment which looks very odd > to me > > if (callback != NULL) { > typedef CommTimeoutCbParams Params; > Params ¶ms = GetCommParams(callback); > params

Re: [PATCH] Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 22:23 +0200, Henrik Nordstrom wrote: > What worries me a bit is that this introduces a queue where invocation > was always immediate before. So we have a different problem space to > deal with in queued events where we before had to deal with recursive > events.. It's a subtl

Re: [PATCH] Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 12:55 -0600 skrev Alex Rousskov: > BTW, if somebody commits that patch, please polish the reason phrase in > the cancel() call to something more specific. The submitted pach didn't work for two reasons. In fact this cancellation of comm scheduled calls doesn't work at all.

commSetTimeout looks odd.. maybe

2008-04-08 Thread Henrik Nordstrom
in commSetTimeout there is the following fragment which looks very odd to me if (callback != NULL) { typedef CommTimeoutCbParams Params; Params ¶ms = GetCommParams(callback); params.fd = fd; F->timeoutHandler = callback; } Does this

Re: [PATCH] Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 20:46 +0300 skrev Tsantilas Christos: > AsyncCalls are refcounted so believed that with "ccb->callback = NULL" > the AsyncCall which the ccb->callback point released. But yes if it is > refcounted by an other object (for example in AsyncCalls queues,) > it is not released

Re: [PATCH] Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 20:46 +0300, Tsantilas Christos wrote: > Henrik Nordstrom wrote: > > So any proposals on how we would go about fixing comm_read_cancel? > > > > Hmm.. on reading that code commio_cancel_callback looks a bit odd in > > trunk. Why is callback set to NULL twice, and what is real

Re: [PATCH] Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Tsantilas Christos
Henrik Nordstrom wrote: > So any proposals on how we would go about fixing comm_read_cancel? > > Hmm.. on reading that code commio_cancel_callback looks a bit odd in > trunk. Why is callback set to NULL twice, and what is really the purpose > of this function now that the actual callback is in an

Re: TPROXY support in Squid 2

2008-04-08 Thread Henrik Nordstrom
ons 2008-04-09 klockan 00:19 +0800 skrev Adrian Chadd: > On Tue, Apr 08, 2008, Adrian Chadd wrote: > > > > Take 3: > > http://www.creative.net.au/diffs/20080408-tproxy-fix-2.diff using -3 instead of -2 looks better... but incomplete. You need to add the n

Re: Suggested 3.0 merge candidates

2008-04-08 Thread Tsantilas Christos
Alex Rousskov wrote: > On Tue, 2008-04-08 at 17:04 +0200, Henrik Nordstrom wrote: >> tis 2008-04-08 klockan 13:23 +1200 skrev Amos Jeffries: >> >>> +1. I'm just waiting on you all to agree that its tested enough. If you >>> want to do the merge yourself Henrik, I'm okay with that. >> Doesn't matter

[PATCH] Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 07:47 -0600 skrev Alex Rousskov: > I believe there is a better option that was added to handle specifically > to handle these kind of problems: > > d) Cancel the call when it is no longer expected. That's also an option, not at all far from 'a'. > This is already supporte

Re: [squid-users] About Cache Digest

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 08:57 -0600 skrev Alex Rousskov: > IIRC, one-bucket-at-a-time was too slow in environments that cared more > about hits than optimizing miss performance. Folks wanted a knob to > control the trade-off. Ok. We should change that to be buckets based, and default to 1. The cur

Re: TPROXY support in Squid 3

2008-04-08 Thread Adrian Chadd
On Tue, Apr 08, 2008, Adrian Chadd wrote: > Take 3: http://www.creative.net.au/diffs/20080408-tproxy-fix-2.diff * restored the global flag which indicates "tproxy"ness; renamed to enable_spoof_srv * moved the tproxy "bind" stuff into comm.c, with a flag to comm_openex()

Re: Squid Throughput/Performance Testing

2008-04-08 Thread Bradley Kite
On 08/04/2008, Henrik Nordstrom <[EMAIL PROTECTED]> wrote: > tis 2008-04-08 klockan 13:57 +0100 skrev Bradley Kite: > > > Any hints/tips that you can recomend on a low budget? I've looked at > > Web Polygraph, and in a lab environment this seems excellent, but I > > think it requires more hardwar

Re: Squid Throughput/Performance Testing

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 13:57 +0100 skrev Bradley Kite: > Any hints/tips that you can recomend on a low budget? I've looked at > Web Polygraph, and in a lab environment this seems excellent, but I > think it requires more hardware than I have available to me. There is a number of other tools simple

Re: Suggested 3.0 merge candidates

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 17:04 +0200, Henrik Nordstrom wrote: > tis 2008-04-08 klockan 13:23 +1200 skrev Amos Jeffries: > > > +1. I'm just waiting on you all to agree that its tested enough. If you > > want to do the merge yourself Henrik, I'm okay with that. > > Doesn't matter for me who merges the

Re: TPROXY support in Squid 3

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 22:17 +0800 skrev Adrian Chadd: > > > * make sure upstream/peer-forwarded requests aren't thrown to the tproxy > > > code. > > > I read the current code as "don't do that"; its the behaviour I'd like to > maintain. We can always add the functionality later. Yes, it is no

Re: Suggested 3.0 merge candidates

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 13:23 +1200 skrev Amos Jeffries: > +1. I'm just waiting on you all to agree that its tested enough. If you > want to do the merge yourself Henrik, I'm okay with that. Doesn't matter for me who merges the backport. It's a trivial bzr merge with the backport already done.. >

Re: TPROXY support in Squid 3

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 15:15 +0200, Henrik Nordstrom wrote: > tis 2008-04-08 klockan 17:59 +1200 skrev Amos Jeffries: > > Currently: > >fde::flags::transparent == 'intercept/non-intercept' > >fde::flags::tproxy == real-transparent/non-transparent > >(new) COMM_TRANSPARENT == real-transpa

Re: Squid Throughput/Performance Testing

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 13:57 +0100, Bradley Kite wrote: > I'm interested in the performance of squid, and in particular, how do > the developers optimise/benchmark/stress-test the code - I'd like to > use similar ideas for evaluation. > > Any hints/tips that you can recomend on a low budget? I've

Re: Squid Throughput/Performance Testing

2008-04-08 Thread Adrian Chadd
Polygraph takes a while to setup right. It depends on how hard you wish to hit the proxy. I also use httperf/apachebench in specific configurations to stress certain parts of the Squid codebase. Adrian On Tue, Apr 08, 2008, Bradley Kite wrote: > Hi all. > > I'm interested in the performance

Re: TPROXY support in Squid 3

2008-04-08 Thread Adrian Chadd
with placeholders for tproxy4/freebsd.) > > > > http://www.creative.net.au/diffs/20080408-tproxy-fix-2.diff > > Comments: > > commTransparentRemote needs to move into comm_openex. You can't fit > tproxy4/freebsd in a reasnoable manner otherwise. Yup, that'd be

Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 11:39 +0200, Henrik Nordstrom wrote: > - There is two events that may happen, 'A' and 'B'. > - In the processing of one interest of the other is deregistered with > the silent assumption that after that it won't get invoked, but the > object it's related to still exists. > -

Re: TPROXY support in Squid 3

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 14:28 +0800 skrev Adrian Chadd: > Take 2: includes initial modularisation (untested; I'll test it at home > this weekend when I get my tproxy box back online) and configure magic > (with placeholders for tproxy4/freebsd.) > > http://www.creative.n

Re: TPROXY support in Squid 3

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 17:59 +1200 skrev Amos Jeffries: > Currently: >fde::flags::transparent == 'intercept/non-intercept' >fde::flags::tproxy == real-transparent/non-transparent >(new) COMM_TRANSPARENT == real-transparent > > Their use is currently good for what they do. A small re-na

Re: Suggested 3.0 merge candidates

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 13:23 +1200 skrev Amos Jeffries: > Is there a security problem with them being set? > IMO its just a cleanup otherwise (already voted those not to go back). It's not even a source change, just a bzr attributes cleanup. But the current situation may confuse some unexperience

Squid Throughput/Performance Testing

2008-04-08 Thread Bradley Kite
Hi all. I'm interested in the performance of squid, and in particular, how do the developers optimise/benchmark/stress-test the code - I'd like to use similar ideas for evaluation. Any hints/tips that you can recomend on a low budget? I've looked at Web Polygraph, and in a lab environment this se

Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 23:50 +1200 skrev Amos Jeffries: > If its not easy and its not simple it means more bugs. I don't think 'c' > is a good choice here without a full ground-up recode. Nay on that from > me on general principles. True. I view 'c' as more of a last resort if the other turn out

Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Amos Jeffries
Henrik Nordstrom wrote: mån 2008-04-07 klockan 23:41 -0600 skrev Alex Rousskov: Can you post cache.log lines with full debugging enabled? I would be happy to help with the queue review, but it would speed things up if I do not have to go through the complicated steps required to reproduce the pr

Re: pconn.cc assert index >= 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
mån 2008-04-07 klockan 23:41 -0600 skrev Alex Rousskov: > Can you post cache.log lines with full debugging enabled? I would be > happy to help with the queue review, but it would speed things up if I > do not have to go through the complicated steps required to reproduce > the problem. The cache.l