Re: [squid-users] Squid Future (was Re: [squid-users] Squid-2, Squid-3, roadmap)

2008-03-23 Thread Adrian Chadd
On Sat, Mar 22, 2008, Henrik Nordstrom wrote: > Squid-3 is different and uses a splay tree for the memory nodes of the > object, and should behave a lot better in this regard. The bounds are probably saner but the runtime hit for small objects is noticable. The real solution is a tree for offset

Re: [squid-users] Squid Future (was Re: [squid-users] Squid-2, Squid-3, roadmap)

2008-03-22 Thread Henrik Nordstrom
On Sat, 2008-03-22 at 20:14 +1300, Amos Jeffries wrote: > > http://www.mail-archive.com/squid-users@squid-cache.org/msg52509.html > > Hmm, not sure exactly what Adrian as planned there, beyond changing the > underlying malloc/calloc system of squid to something else. > Added it to the 'undocument

Re: [squid-users] Squid Future (was Re: [squid-users] Squid-2, Squid-3, roadmap)

2008-03-22 Thread Amos Jeffries
Chris Woodfield wrote: For our purposes (reverse proxy usage) we don't see any missing features from squid 3 that we would need - however, we'd like to see the code base mature some more before we trust it in production. Same reason that smart folks don't deploy new Cisco IOS trains until it hi

Re: [squid-users] Squid Future (was Re: [squid-users] Squid-2, Squid-3, roadmap)

2008-03-20 Thread Chris Woodfield
For our purposes (reverse proxy usage) we don't see any missing features from squid 3 that we would need - however, we'd like to see the code base mature some more before we trust it in production. Same reason that smart folks don't deploy new Cisco IOS trains until it hits the 3rd or 4th r

Re: [squid-users] Squid Future (was Re: [squid-users] Squid-2, Squid-3, roadmap)

2008-03-16 Thread Adrian Chadd
On Sun, Mar 16, 2008, Nick Duda wrote: > The only reason I haven't upgraded beyond the current stable 2.6 code is that > some third part companies (like Secure Computing, who we use as a Squid > plugin) only supports certain versions of squid. I haven't even played with > 3.0 because of this. I

RE: [squid-users] Squid Future (was Re: [squid-users] Squid-2,Squid-3, roadmap)

2008-03-16 Thread Adam Carter
My 2c WRT 2 v 3 etc; - We currently run commercial proxies and are looking to replace them with squid boxes, however recent list discussion is making me a little nervous. I would have used 2.6 for performance (need to support 10K users) and for - Secure Computing's Smartfilter. It currently runs

RE: [squid-users] Squid Future (was Re: [squid-users] Squid-2, Squid-3, roadmap)

2008-03-16 Thread Nick Duda
, March 16, 2008 9:25 PM To: Adrian Chadd Cc: squid-users@squid-cache.org Subject: Re: [squid-users] Squid Future (was Re: [squid-users] Squid-2, Squid-3, roadmap) On Mon, 2008-03-17 at 10:18 +0900, Adrian Chadd wrote: > At the end of the day, I'd rather see something that an increasing >

Re: [squid-users] Squid Future (was Re: [squid-users] Squid-2, Squid-3, roadmap)

2008-03-16 Thread Robert Collins
On Mon, 2008-03-17 at 10:18 +0900, Adrian Chadd wrote: > At the end of the day, I'd rather see something that an increasing number of > people > on the Internet will use and - I won't lie here - whatever creates a self > sustaining > project, both from community and financial perspectives. I ag

[squid-users] Squid Future (was Re: [squid-users] Squid-2, Squid-3, roadmap)

2008-03-16 Thread Adrian Chadd
Just to summarise the discussion, both public and private. * Squid-3 is receiving the bulk of the active core Squid developers' focus; * Squid-2 won't be actively developed at the moment by anyone outside of paid commercial work; * I've been asked (and agreed at the moment) to not push any big c

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-11 Thread Alex Rousskov
On Mon, 2008-03-10 at 22:38 -0800, Michael Puckett wrote: > I'll come back to one of Mark's earlier points then which seems to have > been lost. What will decide on adoption of -2 or -3 is the "killer app". That point has not been lost. There are two distinct problems here: First, "the killer ap

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-11 Thread Alex Rousskov
On Tue, 2008-03-11 at 15:19 +0900, Adrian Chadd wrote: > I shouldn't have been the one that tried to pull some sensible > direction and feedback into the development group ... I shouldn't have > had to try and kick Squid-3 developers along to do simple things like > regression testing and local be

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-11 Thread Adrian Chadd
I believe that we have to look far past the current Squid-2 and Squid-3 versions and think something much .. well, "better". My initial Squid-2 roadmap proposal (which is still in the Wiki) aimed to take the codebase forward whilst beginning a modularisation project (which is in there; just read b

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-10 Thread Michael Puckett
I'll come back to one of Mark's earlier points then which seems to have been lost. What will decide on adoption of -2 or -3 is the "killer app". Developers roadmaps or sponsors notwithstanding. I think the point was raised that neither roadmap was especially compelling or seemed to yet contain

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-10 Thread Adrian Chadd
On Mon, Mar 10, 2008, Alex Rousskov wrote: > > WRT responsible sponsoring: I'm willing to pay a (reasonable) premium > > to get the things that I pay to get into -2 into -3 as well, > > Thank you, and I am sure many sponsors would do the same if the > trade-offs are explained to them correctly.

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-10 Thread Alex Rousskov
On Tue, 2008-03-11 at 10:57 +1100, Mark Nottingham wrote: > It might be good to have a *little* more process around how ones > sponsors changes in Squid -- whether -2 or -3 -- to assure that > there's coordination between the feature sets, and that the entire > Squid developer community is b

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-10 Thread Mark Nottingham
Thanks, Alex -- that's actually quite helpful. It might be good to have a *little* more process around how ones sponsors changes in Squid -- whether -2 or -3 -- to assure that there's coordination between the feature sets, and that the entire Squid developer community is bought into the cha

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-07 Thread Adrian Chadd
If anything, I hope that this particular discussion realigns the Squid-3 developers with the sorts of things the community wants. On Sat, Mar 08, 2008, Amos Jeffries wrote: > Some of us do. At least 5% of the net and growing. When 3.x rolls out > the capability fully you can expect a small jump

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-07 Thread Amos Jeffries
Dodd, Tony wrote: -Original Message- From: Mark Nottingham [mailto:[EMAIL PROTECTED] Well, that's a bit of a straw-man, isn't it? AIUI 3 *is* already 2 re- coded into C++. Never mind the question of why that's necessary; indeed, I think a lot of people's discomfort is centred on the fact

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-07 Thread Amos Jeffries
Dodd, Tony wrote: -Original Message- From: Amos Jeffries [mailto:[EMAIL PROTECTED] 3.0 was about parity with needs. It failed some in that regard. 3.1 is about making up that failure plus some. Is seamless IPv6, SSL control, and weighted round-robin not enough of a killer app for you

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-07 Thread Alex Rousskov
Below you will find my personal comments on a few hand-picked thoughts (from various posters) that I consider important on this thread: On Thu, 2008-03-06 at 08:44 -0800, Michael Puckett wrote: > If there is one killer app that tops all other functionality > additions it would be to multi-thread

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-06 Thread Alex Rousskov
On Wed, 2008-02-27 at 14:30 +1100, Mark Nottingham wrote: > Specifically, a few questions for the developers of Squid: > ... The following is the response from 5 out of 7 Squid Core members. This is not an official response from Squid Core, the group steering the Squid Project as a whole. > anec

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-06 Thread Adrian Chadd
On Fri, Mar 07, 2008, Mark Nottingham wrote: > Ideally, you'd avoid locking as much as possible; e.g., have a pool of > threads for disk access (as now with aufs), a pool for header parsing, > a pool for forward requests, and so on. I don't think it's a good idea > at all to re-architect squi

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-06 Thread Mark Nottingham
Ideally, you'd avoid locking as much as possible; e.g., have a pool of threads for disk access (as now with aufs), a pool for header parsing, a pool for forward requests, and so on. I don't think it's a good idea at all to re-architect squid into a thread-per-connection model or anything; j

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-06 Thread Marcus Kool
Steve, adzapper uses a very large amount of CPU time compared to other redirectors. On my box Squid uses 6.5 times more CPU time than the redirector (ufdbGuard). Marcus Steve Snyder wrote: On Thursday 06 March 2008 11:05:24 am Adrian Chadd wrote: Well, the way I'd approach it is to first get

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-06 Thread Michael Puckett
Well, I think that Mark is right on the money here. If there is one killer app that tops all other functionality additions it would be to multi-thread Squid so that it can perform on multi-cores. You wanted input from your users on the -2 or -3 roadmaps here it is. With the current implementati

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-06 Thread Steve Snyder
On Thursday 06 March 2008 11:05:24 am Adrian Chadd wrote: > Well, the way I'd approach it is to first get an idea of how to throw > things into 'threads', and probably draft and craft a basic event loop > and submission queue for "stuff" to happen across threads. > > Then "Squid" can run as one thr

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-06 Thread Adrian Chadd
Well, the way I'd approach it is to first get an idea of how to throw things into 'threads', and probably draft and craft a basic event loop and submission queue for "stuff" to happen across threads. Then "Squid" can run as one thread, and CPU intensive stuff can happen via message queues to other

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-06 Thread Chris Woodfield
I'll readily admit that I Am Not A Developer, but I'm wondering if this could be something that could be worked incrementally - finding easy-to-cleave-off subsystems that can be moved to separate threads similarly to how asyncio was. The most obvious one I can think of is the front-end clie

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-06 Thread Adrian Chadd
On Wed, Mar 05, 2008, Michael Puckett wrote: > Mark Nottingham wrote: > > > >A killer app for -3 would be multi-core support (and the perf > >advantages that it would bring), or something else that the > >re-architecture makes possible that isn't easy in -2. AIUI, though, > >that isn't the case;

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-05 Thread Michael Puckett
Mark Nottingham wrote: A killer app for -3 would be multi-core support (and the perf advantages that it would bring), or something else that the re-architecture makes possible that isn't easy in -2. AIUI, though, that isn't the case; i.e., -3 doesn't make this significantly easier. Absolutel

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-05 Thread Mark Nottingham
BTW, eCAP *is* interesting; it just looks really tentative at this point, and the perf/stability issues overshadow it to some degree. Now, if you released Python bindings for eCAP, *that* would be interesting. Also, multi-core would make eCAP that much more powerful; as it is, servers like

RE: [squid-users] Squid-2, Squid-3, roadmap

2008-03-05 Thread Dodd, Tony
> -Original Message- > From: Amos Jeffries [mailto:[EMAIL PROTECTED] > 3.0 was about parity with needs. It failed some in that regard. > 3.1 is about making up that failure plus some. > Is seamless IPv6, SSL control, and weighted round-robin not enough of a > killer app for you? > SSL co

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-05 Thread Mark Nottingham
On 06/03/2008, at 12:28 PM, Amos Jeffries wrote: stale-if-error stale-while-revalidate - Um, so why did you (the sponsor for these two I believe) not also request their addition in -3 for future-proofing your install app? Because -3 isn't on our roadmap, for the reasons cited. If it appears

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-05 Thread Adrian Chadd
On Thu, Mar 06, 2008, Amos Jeffries wrote: > 3.1 is about making up that failure plus some. > Is seamless IPv6, SSL control, and weighted round-robin not enough of a > killer app for you? The trouble is Amos, I'm reasonably confident I can get sponsorship for porting enough of those to Squid-2 fo

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-05 Thread Amos Jeffries
> > On 05/03/2008, at 1:39 PM, Amos Jeffries wrote: > >>> Well, >>> >>> I am interested in speed, features and ICAP. >>> So I like -2 and -3 to merge. >>> >>> It seems to me that for the sake of being polite with each other >>> we do not want to call the -2 / -3 issue a fork, but effectively >>> it

RE: [squid-users] Squid-2, Squid-3, roadmap

2008-03-05 Thread Dodd, Tony
> -Original Message- > From: Mark Nottingham [mailto:[EMAIL PROTECTED] > > Well, that's a bit of a straw-man, isn't it? AIUI 3 *is* already 2 re- > coded into C++. Never mind the question of why that's necessary; > indeed, I think a lot of people's discomfort is centred on the fact > that

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-05 Thread Mark Nottingham
On 05/03/2008, at 1:39 PM, Amos Jeffries wrote: Well, I am interested in speed, features and ICAP. So I like -2 and -3 to merge. It seems to me that for the sake of being polite with each other we do not want to call the -2 / -3 issue a fork, but effectively it really is a fork. So here is m

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-04 Thread Amos Jeffries
> Well, > > I am interested in speed, features and ICAP. > So I like -2 and -3 to merge. > > It seems to me that for the sake of being polite with each other > we do not want to call the -2 / -3 issue a fork, but effectively > it really is a fork. > > So here is my question back to the main maintai

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-04 Thread François Cami
Hi, We've been testing Squid 3. 2.X is out of the question since we need ICAP. Our 3.0STABLE1 build with backported icap-related patches from 3.0-current is stable enough for us (no crashes, no weird behaviour). What I would personally like to see is full HTTP 1.1 compliance and a more complete I

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-04 Thread Steve Snyder
On Tuesday 04 March 2008 7:36:50 am Adrian Chadd wrote: > Hi everyone, > > I'm quite disappointed in the lack of feedback from the community over > this. Its hard to figure out what people want if noone speaks up, so > this is your time to speak up. I see nothing attractive in Squid v3.0. I don't

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-04 Thread Marcus Kool
Well, I am interested in speed, features and ICAP. So I like -2 and -3 to merge. It seems to me that for the sake of being polite with each other we do not want to call the -2 / -3 issue a fork, but effectively it really is a fork. So here is my question back to the main maintainers: do you wan

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-03-04 Thread Adrian Chadd
Hi everyone, I'm quite disappointed in the lack of feedback from the community over this. Its hard to figure out what people want if noone speaks up, so this is your time to speak up. Adrian On Wed, Feb 27, 2008, Mark Nottingham wrote: > Hello Squid folk, > > I maintain Yahoo!'s internal bu

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-02-29 Thread Adrian Chadd
(I'm going to try and raise these during the London meeting if I can get in with Skype or something similar.) On Thu, Feb 28, 2008, Robert Collins wrote: > Folk will scratch their own itches - thats open source for you. I know > I'd really prefer it if features being added are *primarily* added t

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-02-28 Thread Robert Collins
On Wed, 2008-02-27 at 14:30 +1100, Mark Nottingham wrote: > >* Besides the availability of *CAP and ESI -- which are very > specialised, and of interest only to a subset of Squid users -- is > there any user-visible benefit to switching to -3? class 4 delay pools, some request tagging stu

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-02-27 Thread Ric
On Feb 26, 2008, at 7:30 PM, Mark Nottingham wrote: Hello Squid folk, I maintain Yahoo!'s internal build of Squid, and serve as a resource for the various Y! properties that use it. We currently only use Squid-2, and don't have plans to migrate to Squid-3; although ESI, ICAP as well as eC

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-02-27 Thread john allspaw
riginal Message > From: Tony Dodd <[EMAIL PROTECTED]> > To: Mark Nottingham <[EMAIL PROTECTED]> > Cc: Squid Users > Sent: Wednesday, February 27, 2008 4:50:26 AM > Subject: Re: [squid-users] Squid-2, Squid-3, roadmap > > Mark, thanks for raising these questions, we at L

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-02-27 Thread Adrian Chadd
Just so people aren't left wondering why there's been no response - the best way to approach this is being discussed amongst the core developers at the moment. It just co-incides with three of them - Alex, Henrik and Robert- in transit to London. That said, if anyone else has anything to offer on

Re: [squid-users] Squid-2, Squid-3, roadmap

2008-02-27 Thread Tony Dodd
Mark, thanks for raising these questions, we at Last.FM are facing the same issues as Yahoo! is wrt squid-2/-3. In answer to your final question in the list, I've tested -3 on a number of servers over a number of time-periods during the past few months. Unfortunately, the missing features found w

[squid-users] Squid-2, Squid-3, roadmap

2008-02-26 Thread Mark Nottingham
Hello Squid folk, I maintain Yahoo!'s internal build of Squid, and serve as a resource for the various Y! properties that use it. We currently only use Squid-2, and don't have plans to migrate to Squid-3; although ESI, ICAP as well as eCAP look interesting, there are too many critical fea