Re: TPROXY support in Squid 3

2008-04-08 Thread Adrian Chadd
Take 2: includes initial modularisation (untested; I'll test it at home this weekend when I get my tproxy box back online) and configure magic (with placeholders for tproxy4/freebsd.) http://www.creative.net.au/diffs/20080408-tproxy-fix-2.diff TODO: * pull out the capabilities stuff, removing

Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
mån 2008-04-07 klockan 23:41 -0600 skrev Alex Rousskov: Can you post cache.log lines with full debugging enabled? I would be happy to help with the queue review, but it would speed things up if I do not have to go through the complicated steps required to reproduce the problem. The cache.log

Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Amos Jeffries
Henrik Nordstrom wrote: mån 2008-04-07 klockan 23:41 -0600 skrev Alex Rousskov: Can you post cache.log lines with full debugging enabled? I would be happy to help with the queue review, but it would speed things up if I do not have to go through the complicated steps required to reproduce the

Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 23:50 +1200 skrev Amos Jeffries: If its not easy and its not simple it means more bugs. I don't think 'c' is a good choice here without a full ground-up recode. Nay on that from me on general principles. True. I view 'c' as more of a last resort if the other turn out

Squid Throughput/Performance Testing

2008-04-08 Thread Bradley Kite
Hi all. I'm interested in the performance of squid, and in particular, how do the developers optimise/benchmark/stress-test the code - I'd like to use similar ideas for evaluation. Any hints/tips that you can recomend on a low budget? I've looked at Web Polygraph, and in a lab environment this

Re: Suggested 3.0 merge candidates

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 13:23 +1200 skrev Amos Jeffries: Is there a security problem with them being set? IMO its just a cleanup otherwise (already voted those not to go back). It's not even a source change, just a bzr attributes cleanup. But the current situation may confuse some unexperienced

Re: TPROXY support in Squid 3

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 17:59 +1200 skrev Amos Jeffries: Currently: fde::flags::transparent == 'intercept/non-intercept' fde::flags::tproxy == real-transparent/non-transparent (new) COMM_TRANSPARENT == real-transparent Their use is currently good for what they do. A small re-naming

Re: TPROXY support in Squid 3

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 14:28 +0800 skrev Adrian Chadd: Take 2: includes initial modularisation (untested; I'll test it at home this weekend when I get my tproxy box back online) and configure magic (with placeholders for tproxy4/freebsd.) http://www.creative.net.au/diffs/20080408-tproxy-fix

Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 11:39 +0200, Henrik Nordstrom wrote: - There is two events that may happen, 'A' and 'B'. - In the processing of one interest of the other is deregistered with the silent assumption that after that it won't get invoked, but the object it's related to still exists. - Both

Re: TPROXY support in Squid 3

2008-04-08 Thread Adrian Chadd
.) http://www.creative.net.au/diffs/20080408-tproxy-fix-2.diff Comments: commTransparentRemote needs to move into comm_openex. You can't fit tproxy4/freebsd in a reasnoable manner otherwise. Yup, that'd be part of the follow up work. This first cut is to break out what is there; the next

Re: Squid Throughput/Performance Testing

2008-04-08 Thread Adrian Chadd
Polygraph takes a while to setup right. It depends on how hard you wish to hit the proxy. I also use httperf/apachebench in specific configurations to stress certain parts of the Squid codebase. Adrian On Tue, Apr 08, 2008, Bradley Kite wrote: Hi all. I'm interested in the performance of

Re: Squid Throughput/Performance Testing

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 13:57 +0100, Bradley Kite wrote: I'm interested in the performance of squid, and in particular, how do the developers optimise/benchmark/stress-test the code - I'd like to use similar ideas for evaluation. Any hints/tips that you can recomend on a low budget? I've

Re: TPROXY support in Squid 3

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 15:15 +0200, Henrik Nordstrom wrote: tis 2008-04-08 klockan 17:59 +1200 skrev Amos Jeffries: Currently: fde::flags::transparent == 'intercept/non-intercept' fde::flags::tproxy == real-transparent/non-transparent (new) COMM_TRANSPARENT == real-transparent

Re: Suggested 3.0 merge candidates

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 13:23 +1200 skrev Amos Jeffries: +1. I'm just waiting on you all to agree that its tested enough. If you want to do the merge yourself Henrik, I'm okay with that. Doesn't matter for me who merges the backport. It's a trivial bzr merge with the backport already done..

Re: TPROXY support in Squid 3

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 22:17 +0800 skrev Adrian Chadd: * make sure upstream/peer-forwarded requests aren't thrown to the tproxy code. I read the current code as don't do that; its the behaviour I'd like to maintain. We can always add the functionality later. Yes, it is not something

Re: Suggested 3.0 merge candidates

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 17:04 +0200, Henrik Nordstrom wrote: tis 2008-04-08 klockan 13:23 +1200 skrev Amos Jeffries: +1. I'm just waiting on you all to agree that its tested enough. If you want to do the merge yourself Henrik, I'm okay with that. Doesn't matter for me who merges the

Re: Squid Throughput/Performance Testing

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 13:57 +0100 skrev Bradley Kite: Any hints/tips that you can recomend on a low budget? I've looked at Web Polygraph, and in a lab environment this seems excellent, but I think it requires more hardware than I have available to me. There is a number of other tools simpler

Re: Squid Throughput/Performance Testing

2008-04-08 Thread Bradley Kite
On 08/04/2008, Henrik Nordstrom [EMAIL PROTECTED] wrote: tis 2008-04-08 klockan 13:57 +0100 skrev Bradley Kite: Any hints/tips that you can recomend on a low budget? I've looked at Web Polygraph, and in a lab environment this seems excellent, but I think it requires more hardware than I

Re: TPROXY support in Squid 3

2008-04-08 Thread Adrian Chadd
On Tue, Apr 08, 2008, Adrian Chadd wrote: Take 3: http://www.creative.net.au/diffs/20080408-tproxy-fix-2.diff * restored the global flag which indicates tproxyness; renamed to enable_spoof_srv * moved the tproxy bind stuff into comm.c, with a flag to comm_openex() * changed forward.c to try

Re: [squid-users] About Cache Digest

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 08:57 -0600 skrev Alex Rousskov: IIRC, one-bucket-at-a-time was too slow in environments that cared more about hits than optimizing miss performance. Folks wanted a knob to control the trade-off. Ok. We should change that to be buckets based, and default to 1. The

[PATCH] Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 07:47 -0600 skrev Alex Rousskov: I believe there is a better option that was added to handle specifically to handle these kind of problems: d) Cancel the call when it is no longer expected. That's also an option, not at all far from 'a'. This is already supported by

Re: Suggested 3.0 merge candidates

2008-04-08 Thread Tsantilas Christos
Alex Rousskov wrote: On Tue, 2008-04-08 at 17:04 +0200, Henrik Nordstrom wrote: tis 2008-04-08 klockan 13:23 +1200 skrev Amos Jeffries: +1. I'm just waiting on you all to agree that its tested enough. If you want to do the merge yourself Henrik, I'm okay with that. Doesn't matter for me who

Re: TPROXY support in Squid 2

2008-04-08 Thread Henrik Nordstrom
ons 2008-04-09 klockan 00:19 +0800 skrev Adrian Chadd: On Tue, Apr 08, 2008, Adrian Chadd wrote: Take 3: http://www.creative.net.au/diffs/20080408-tproxy-fix-2.diff using -3 instead of -2 looks better... but incomplete. You need to add the new ipt_* files to the EXTRA_squid_SOURCES

Re: [PATCH] Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Tsantilas Christos
Henrik Nordstrom wrote: So any proposals on how we would go about fixing comm_read_cancel? Hmm.. on reading that code commio_cancel_callback looks a bit odd in trunk. Why is callback set to NULL twice, and what is really the purpose of this function now that the actual callback is in an

Re: [PATCH] Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 20:46 +0300, Tsantilas Christos wrote: Henrik Nordstrom wrote: So any proposals on how we would go about fixing comm_read_cancel? Hmm.. on reading that code commio_cancel_callback looks a bit odd in trunk. Why is callback set to NULL twice, and what is really the

Re: [PATCH] Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 20:46 +0300 skrev Tsantilas Christos: AsyncCalls are refcounted so believed that with ccb-callback = NULL the AsyncCall which the ccb-callback point released. But yes if it is refcounted by an other object (for example in AsyncCalls queues,) it is not released and

commSetTimeout looks odd.. maybe

2008-04-08 Thread Henrik Nordstrom
in commSetTimeout there is the following fragment which looks very odd to me if (callback != NULL) { typedef CommTimeoutCbParams Params; Params params = GetCommParamsParams(callback); params.fd = fd; F-timeoutHandler = callback; }

Re: [PATCH] Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 12:55 -0600 skrev Alex Rousskov: BTW, if somebody commits that patch, please polish the reason phrase in the cancel() call to something more specific. The submitted pach didn't work for two reasons. In fact this cancellation of comm scheduled calls doesn't work at all.

Re: [PATCH] Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 22:23 +0200, Henrik Nordstrom wrote: What worries me a bit is that this introduces a queue where invocation was always immediate before. So we have a different problem space to deal with in queued events where we before had to deal with recursive events.. It's a subtle

Re: commSetTimeout looks odd.. maybe

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 22:38 +0200, Henrik Nordstrom wrote: in commSetTimeout there is the following fragment which looks very odd to me if (callback != NULL) { typedef CommTimeoutCbParams Params; Params params = GetCommParamsParams(callback);

Re: [PATCH] Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 17:55 -0600 skrev Alex Rousskov: is worth the bugs and complexity we are _removing_. Also, the future SMP support may operate within the same nothing-is-immediate problem space. In other words, let's get used to it! :-) Note: For SMP to work out you want as few hops

Re: commSetTimeout looks odd.. maybe

2008-04-08 Thread Henrik Nordstrom
tis 2008-04-08 klockan 18:04 -0600 skrev Alex Rousskov: Yes, it is setting the fd member of the call[back]. Yes, I realized that after sending the message. The syntax is very ugly, but I do not have a better proposal at the moment. I'll guess I get used to it eventually... Regards Henrik

Re: [PATCH] Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Alex Rousskov
On Tue, 2008-04-08 at 22:23 +0200, Henrik Nordstrom wrote: I still think the cbdata model is generally preferrable. Much less to keep track of. For pconn it would need one cbdata object per idle fd, and having only that object invalidated when the fd is taken out of the pconn queue for

Re: [PATCH] Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Alex Rousskov
On Wed, 2008-04-09 at 02:05 +0200, Henrik Nordstrom wrote: tis 2008-04-08 klockan 17:55 -0600 skrev Alex Rousskov: is worth the bugs and complexity we are _removing_. Also, the future SMP support may operate within the same nothing-is-immediate problem space. In other words, let's get used

Re: [PATCH] Re: pconn.cc assert index = 0, async call queue madness

2008-04-08 Thread Alex Rousskov
On Wed, 2008-04-09 at 01:51 +0200, Henrik Nordstrom wrote: tis 2008-04-08 klockan 12:55 -0600 skrev Alex Rousskov: BTW, if somebody commits that patch, please polish the reason phrase in the cancel() call to something more specific. The submitted pach didn't work for two reasons. In fact

Re: TPROXY support in Squid 3

2008-04-08 Thread Amos Jeffries
On Tue, 2008-04-08 at 15:15 +0200, Henrik Nordstrom wrote: tis 2008-04-08 klockan 17:59 +1200 skrev Amos Jeffries: Currently: fde::flags::transparent == 'intercept/non-intercept' fde::flags::tproxy == real-transparent/non-transparent (new) COMM_TRANSPARENT == real-transparent

Re: Suggested 3.0 merge candidates

2008-04-08 Thread Amos Jeffries
tis 2008-04-08 klockan 13:23 +1200 skrev Amos Jeffries: Is there a security problem with them being set? IMO its just a cleanup otherwise (already voted those not to go back). It's not even a source change, just a bzr attributes cleanup. But the current situation may confuse some

Problem with CVS pserver?

2008-04-08 Thread Benno Rice
I've started getting this today: cvs -z9 -d :pserver:[EMAIL PROTECTED]:/squid co squid open /dev/null failed Operation not supported Has something broken on the CVS server? -- Benno Rice [EMAIL PROTECTED]

Re: Suggested 3.0 merge candidates

2008-04-08 Thread Amos Jeffries
Tsantilas Christos wrote: Alex Rousskov wrote: On Tue, 2008-04-08 at 17:04 +0200, Henrik Nordstrom wrote: tis 2008-04-08 klockan 13:23 +1200 skrev Amos Jeffries: +1. I'm just waiting on you all to agree that its tested enough. If you want to do the merge yourself Henrik, I'm okay with that.

Re: [MERGE] eCAP support, part 1: Loadable modules and ICAP-independent Squid core.

2008-04-08 Thread Alex Rousskov
On Wed, 2008-04-09 at 13:07 +0800, Adrian Chadd wrote: On Tue, Apr 08, 2008, Alex Rousskov wrote: Hi, FYI: I will commit this tomorrow if there are still no objections. I'd like to see the pconn stuff with AsyncCalls sorted out first before more potentially disruptive stuff is

Re: [MERGE] eCAP support, part 1: Loadable modules and ICAP-independent Squid core.

2008-04-08 Thread Adrian Chadd
On Tue, Apr 08, 2008, Alex Rousskov wrote: I'd like to see the pconn stuff with AsyncCalls sorted out first before more potentially disruptive stuff is included. Why wait? The two issues are unrelated, being worked on by different folks, and the suggested changes are relatively benign