Re: the wheel of httpd-dev life is surely slowing down, solutions please
--On Thursday, November 13, 2003 1:51 PM -0500 Jim Jagielski [EMAIL PROTECTED] wrote: I'm confused then... I had thought that there were API differences (within httpd and apr) between the 2 trees in able to support some of the new features. I suspect that the biggest pain for a 2.0-2.2 migration (if we keep the delta roughly where we're at now) will be syncing with the deprecated APR functions, but that's more about APR going 1.0 than httpd going to 2.2. -- justin
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Although it's probably a little late to be responding to innuendo, the bare minimum points that need a response; At 12:36 AM 11/14/2003, [EMAIL PROTECTED] wrote: There was a lot going down 'offline' and things were just being 'announced' on the forum. That's the way development often happens. Pound on code for a week and present it when it's working. You are reading just a little too much cloak and dagger into things. What you neglected to mention was that this now-famous '303 - Second Street - San Francisco' in-person huddle that you are talking about took place at ( you guessed it ) Covalent Corporate headquarters. Yup - they hosted it, although had 1/2 the committers been in the same area on the east cost, our friends in Raleigh were happy to open their meeting room. How long after that meeting was it that they hired you? You were then ( still? ) in Nebraska, yes? I started consulting for them sometime after that meeting, when Ryan appealed for my help. I joined Covalent that following March. And nope, never NE, I've lived in Chicago most of my life, but for a few year sabbatical in upstate NY. Had never even met any Apache or Covalent folk in person till that meeting. I'm proud that I've been a contributor before Covalent reinsert text snipped for trollbait and will remain ( proud ) even if I find other opportunities at some point. /reinsert Funny wording. Do you mean you aren't proud to be a contributor now that you ARE at Covalent? Nope, I mean exactly that, I played with Apache for the heck of it and got alot out of the experience. It reassured me that C is not dead (another parallel discussion that's getting a little silly :-) And I'm equally proud to be part of Covalent, and proud of what Covalent employees have contributed to Apache, and the Apache 2.0 version. Bill And with that minimum response... ignore class=troll/
Re: the wheel of httpd-dev life is surely slowing down, solutions please
At 12:36 AM 11/14/2003, [EMAIL PROTECTED] wrote: 'Individual' attempts to contribute are getting IGNORED and the last few words of this message thread's subject line are just asking to hear from the powers that be what they intend to do about that ( solutions please ? ). Yes, I think that's the page everyone is on. How to solve this. ( and before you come back with the standard Are YOU willing to review patches, then? at least I'll be honest and say there's NO WAY right now or for the forseeable future. Unlike you... I am NOT paid to work on Apache and I just don't have the time. [...] Actually, I invest far more than 40 hours in my work and Apache time, and I know of many others (from other companies) that do likewise. But you just hit the nail on the head, NO WAY. Nobody suggested that the core committers must single-handedly review the wealth of patches that are offered on this list and added to bugzilla! How can we communicate to this list, bugzilla users and others, that all it takes is to vote on the patch (in bugzilla, or by posting to this list) that the proposed solution actually solves the problem, and didn't break anything else in the process?!? It works for me are the four sweetest words to read when any of us are spending a free hour looking for patches to consider and apply! Besides, I'm also about 100% sure you don't want me reviewing patches for Apache. Newcomers can go read some archived messages if they are curious about the history there. ) If the casual reader is wondering why Kevin is unfortunately not been welcomed with open arms, by all means do go back over those archives. Then again, one doesn't need to scroll back more than the past day or few. Forgive in advance, Kevin, if I now tune you out. Bill
Re: the wheel of httpd-dev life is surely slowing down, solutions please
[EMAIL PROTECTED] wrote: The 'other' not-so-dedicated-but-certainly-interested developers felt 'shut out' of the 2.0 development cycle because it was obvious a lot of it was taking place 'off line' and nothing was being documented so they couldn't really get a good handle on what was going on in order to make a contribution. So they were mostly 'waiting in the wings' for a lot of the 2.0 DESIGN level decisions and concepts to become clear so they could 'jump in'. When I reworked mod_proxy for v2.0, it was one of the first modules to jump in and fully utilise the filter mechanism. Admittedly it took a while to get my head around the filter stuff, but once I had got it, it was fine. All the v2.0 mod_proxy development happened over weekends, and not on company time (a lot of it in a hotel room in Alsjo in Stockholm, but anyway). I feel the architecture jump from v1.3 to v2.0 was worth the effort, as the previous one handler at a time nonsense went out the window, replaced with a possiblity of choose from these filters, and interchange them at will in new and interesting ways. The split of mod_proxy into mod_proxy and mod_cache would not have been possible without filters. I think the key thing is this: Apache v2.0 works, and better fulfils the requirements of people doing web hosting. Less development means there are less itches, and less itches means Apache v2.0 is a closer fit to user requirements than v1.3 was. Regards, Graham --
Re: the wheel of httpd-dev life is surely slowing down, solutions please
On Wed, Nov 12, 2003 at 05:51:46PM -0800, Justin Erenkrantz wrote: Creating roles like this is just as bad as having chunks of httpd owned by one particular developer. (See below) I don't think you understand the role of 'patch manager.' On the projects I'm involved with, that person's only responsibility is for ensuring that the patches posted to the mailing list are entered into the right tool. I think that can easily be a volunteer, rotating job as you don't need to do anything more than that. I don't know what this hypothetical tool looks like, but why wouldn't the person submitting the patch just use the tool directly? Seriously though, until we actually have a tool that we want to use, this part of the discussion is moot. In the patch manager role, nothing would preclude others from adding patches to the right tool. But, having someone whose responsibility it is to ensure that patches get inputted into the tool on a timely basis is goodness. On other projects I'm involved with, that person isn't an even a committer. They don't have to understand what the patch does - just make sure it is recorded and allow for annotation. And, it doesn't have to be on a 'timely' basis - once a week, you go through and ensure that all [PATCH] messages are in the right patch management tool. This lessens the number of dropped patches that we'd have. (Bugzilla sucks at this, so I think we'd need a new tool.) Bugzilla wasn't designed for this, so yes, it sucks at it. I still maintain that the person submitting the patch should simply enter it into the patch system directly. Even easier would be a patch system that monitored the list for [PATCH] emails and tracked those automatically. We might come up with some standard syntax to help the patch tracking system know what version of httpd (or apr or whatever) the patch applies to. Woah woah woah. Are you discouraging people from thinking about big changes for the 2.2 timeframe? If someone has a revolutionary idea for the next generation of Apache HTTPD, who will stand in their way? Not I. Frankly, yes. I think our changes that we already have in the tree is about 'right' for 2.2 (no big architecture changes, but lots of modules have been rewritten and improved). It's just that no one has time or desire to shepherd 2.2 to a release. And, I think we need APR 1.0 out the door before 2.2 can be rationally discussed. [tangential issue] Why does it take some much time and desire to make a release? Shouldn't that be one of the easier things to do around here? If we did any major changes from this point that require modules to rewrite themselves, we'd need to go to 3.0 per our versioning rules. And, I'd *strongly* discourage that. I don't think it's in anyone's best interest to revamp our architecture at this stage. We don't need to give a moving target to our poor module developers. Only by producing a series of high-quality releases can we ensure that module writers will be confident in writing against 2.0 or 2.2. If we come out with 3.x now, I think we'll just end up hurting ourselves in the long run even worse than we are now. Let me pose a different angle: If our module API is so broken that we need to change it in an incompatible way, then don't you think the module authors would rather have the improved interface rather than being stuck with the broken one? In other words, the only time we actually talk about making drastic changes is when they are warranted. Therefore, being conservative about API changes merely serves to discourage creative development and that is _a bad thing_. -aaron
Re: the wheel of httpd-dev life is surely slowing down, solutions please
MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1) wrote: Disclaimer : Not targetting any one individual I don't think there is target here, i consider it more as an open discussion. I have a question to the people have lots of time to write such long mails and responses - why can't you instead spend that time to review patches and give feedback ? it's 9pm here and i am still at work, i was just taking a little time to answer before quit. i agree it's a long mail because english is not easy for me. I try to give feedback about all happen in modules and experimental code in httpd 2.0 But when you give feedback, review code, and post patch and nothing happen then... It's not easy to continue this way =) Maybe my feedback and mails aren't good, so in this case i understand more... but with no answer, it's hard to think something about my work. It sure will improve the life of httpd-dev. I am not sure continuing this way will improve much the life of httpd-dev. And core team will receive more and more mail they will not be able to answer in a reasonnable time. I don't think the problem is in reviewing and giving feedback, but how this is used and what will happen then... Thanks -Madhu -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Thursday, November 13, 2003 10:47 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions please Hi all... I just have to jump in here since the topic is fascinating... ...and I think there's an opportunity here to review something that has contributed to the 'slow down' at httpd-dev which no one has seemed to grasp (yet). I will call it... The Covalent Factor. If you look at what has REALLY happened in the past 3 years ( yes... going back that far since it's now 4 or 5 years since 2.0 became a real blib on the radar ) there's no question that there was this intense period of development and 'new' things were happening at a fast rate. As more and more developers got interested in getting 2.0 cranked out the (limited) resources all got eaten up in the 2.0 development cycle and 1.3 development virtually shut down. It was even 'officially' shut down long before 2.0 was ready ( 1.3 officially went into maintenance mode only ). So now you had lots of legacy developers ( albeit... lots of weekend-warriors, too, but WW's are the heartbeat of Open Source ) who knew 1.3 very well but were now totally put out to pasture. Very few of them 'came over' to the 2.0 development. The majority of the developers for 2.0 were the 'paid to play' kind that were either being paid to directly work on Apache ( I'll get to the Covalent factor in a second ) or, at least, were being paid to do something else but no one seemed to mind if they sat there at their real jobs all day and did Apache development. They might not admit to being directly paid to work on Apache but when that's what you are doing all day and still bringing home a paycheck then that's exactly what that is. The 'other' not-so-dedicated-but-certainly-interested developers felt 'shut out' of the 2.0 development cycle because it was obvious a lot of it was taking place 'off line' and nothing was being documented so they couldn't really get a good handle on what was going on in order to make a contribution. So they were mostly 'waiting in the wings' for a lot of the 2.0 DESIGN level decisions and concepts to become clear so they could 'jump in'. Well... we all know that it was a looong time before some of the 2.0 design concepts became 'clear' enough for a not-paid-to-work-on-apache person to 'jump in'. There were basically only 2 big changes to Apache 2.0 versus 1.x. 1. MPM. Get out of the 'prefork' only 'go fork thyself' design model. 2. ( Came later ) Add some I/O filtering and get rid of BUFF. Well, these are both fine goals to have but they are not for the faint-of-heart ( or within the grasp of most weekend-warriors ). That brings me to The Covalent Factor. Is anyone going to deny that at a certain point in the 2.0 development cycle the PRIMARY driving force was a group of people that ALL worked for one company? ( Covalent ). That's certainly the way it looked. ...and these guys were crankin'... I mean they were MOTIVATED, because Covalent's entire corporate life revolves around Apache and they were actually high level deals going on with Compaq ( and other corporate entities? ) that all revolved around getting Apache 2.0 out the door. At one point one of the Covalent guys ( I believe it was that Randy Bloom fellow? ) was pretty much the ONLY person who had any idea how the new 'filtering' was even SUPPOSED to work. It was quite some time before he even finished thinking it through and it went through (too?) many re-works to even keep a good grasp on it. I remember the feeling for MONTHS was something like... let's just see what Covalent comes up with and let them put their stamp of approval
RE: the wheel of httpd-dev life is surely slowing down, solutions please
-Original Message- From: Matthieu Estrade [mailto:[EMAIL PROTECTED] [SNIP] But when you give feedback, review code, and post patch and nothing happen then... It's not easy to continue this way =) Maybe my feedback and mails aren't good, so in this case i understand more... but with no answer, it's hard to think something about my work. Exactly my point. When you post a patch, and somebody replies back with some feedback, it's like a sense of satisfaction for me. Atleast, I know somebody out there (perhaps not a person with commit access) is giving some importance to my mail and replying back. Some times, I don't even care if the patch gets committed - as long as somebody reviews it. The reason : I need to know if I'm doing something wrong :). Imagine sending mails OR posting patches and NOT getting any replies at all. I've experienced that. The common answer you get is try and try again - and one day the light will shine upon you. I've got that answer a lot of times (it's in the archives if you have the patience to dig in). The question was intended to those people who talk a lot, but neither post patches NOR give reviews/feedback, but jump in with long mails only when there's a debate going on. And core team will receive more and more mail they will not be able to answer in a reasonnable time. Why wait for the core team to answer the questions - why should others NOT pick up the role of answering the questions ? -Madhu (guilty on my own point)
Re: the wheel of httpd-dev life is surely slowing down, solutions please
In a message dated 11/13/2003 12:53:42 PM Central Standard Time, [EMAIL PROTECTED] writes: By the by... Covalent signs my paycheck. And if you look at 1.3, you'll see that I've been pretty key on staying on top of it. Kind of blows away your theory, don't it? Nope
Re: the wheel of httpd-dev life is surely slowing down, solutions please
At 12:47 PM 11/13/2003, [EMAIL PROTECTED] wrote: If you look at what has REALLY happened in the past 3 years ( yes... going back that far since it's now 4 or 5 years since 2.0 became a real blib on the radar ) there's no question that there was this intense period of development and 'new' things were happening at a fast rate. Without a doubt this period of development was abnormally intense for any five year old open source project. Good point. As more and more developers got interested in getting 2.0 cranked out the (limited) resources all got eaten up in the 2.0 development cycle and 1.3 development virtually shut down. It was even 'officially' shut down long before 2.0 was ready ( 1.3 officially went into maintenance mode only ). That's an interesting point. Most of my early (independent) contributions, about 600 dev hours worth, were entirely focused on making 1.3 work under Win32. And you are dead on, some of my work was accepted, and on other points, I was asked hold on, that's a goal in 2.0. There was a little difference though, there really was very little 2.0 anything at that time for me to point my efforts at. So now you had lots of legacy developers ( albeit... lots of weekend-warriors, too, but WW's are the heartbeat of Open Source ) who knew 1.3 very well but were now totally put out to pasture. Nay, they were gone long before I started using/working with/hacking Apache in 2000. Even mod_ssl and OpenSSL were put to bed. The old guard had moved on, and few folks paid much attention to the bug queue. Those that did were overwhelmed by some requests that just wouldn't fit well into the architecture of 1.3. Not to mention that 1.3's core was a twisted mess of platform quirks. It still is, that's why it's orphaned. Hard on old timers, sure, but for newcomers the difference between reading 1.3 and 2.0 is night and day. Yes filters are difficult to grok, but the rest of the entire server is much more simple to follow. Perhaps some will be motivated to make filters, the most difficult 2.0 feature, a little easier to use or understand. Justin already made some progress on this front, and it continues to evolve. The 'other' not-so-dedicated-but-certainly-interested developers felt 'shut out' of the 2.0 development cycle because it was obvious a lot of it was taking place 'off line' and nothing was being documented so they couldn't really get a good handle on what was going on in order to make a contribution. Hehe, of all of your silly perceptions in this note, I'm picking up on this one for the benefit of newer list readers. In the bad old days of the 2.0 dev cycle, very little ever happened that wasn't tracked on the mailing list. Today, there is parallel activity on channels like #apr (of irc.freenode.net), and your comment reinforces the point that not all of that activity can be healthy for the wider community. Unless it is digested and explained to the readership of dev@ and in the maillist archives for posterity and future explanation. At one point one of the Covalent guys ( I believe it was that Randy Bloom fellow? ) was pretty much the ONLY person who had any idea how the new 'filtering' was even SUPPOSED to work. It was quite some time before he even finished thinking it through and it went through (too?) many re-works to even keep a good grasp on it. Hehe, this ties into my point in the prior paragraph. EVERYTHING on the filters was nailed down on the list. What happened? Two camps had two different end goals. They did not see eye to eye on the design. When it got to the point that there was no resolution to be found, I suggested that a face to face of everyone interested would be terrific. Note I was an independent, not paid for webserver apps but actually a database systems dude who liked playing with the web. And I was tired of waiting for the discussion to end (and sorta confused as were most observers.) About 20+ folks, five different firms, some independents, some by phone, sat down to watch Ryan and Greg duke out the design one bullet point at a time. It was amazing, wish that someone had a brought a camcorder :-) And what resulted was a design that satisfied *everyones* requirements. Details and skeletons were posted to the list in realtime by some observers. What do you call this? An impromptu mini-hackathon. And it worked to move forward on a very difficult-to-follow concept. My only real point here ( and the way all this relates to the current thread ) is that maybe it's time to acknowledge that what is happening now is what will always happen to a major development project if you let too many of your eggs go into the same 'corporate' basket. I'm gonna just close with this observation - Apache was always driven by two forces, academic and business. By ISPs who needed a stable platform. By independent coders who earned a living doing systems. By companies who needed a stable and trusted base platform. And Guess What? There is
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Hi Bill... This is Kevin... William Rowe wrote... We value individual contributions here, not corporate affiliation. We means ASF, right? If so... then I think you just nailed the whole point of this thread, if I am reading the original poster's concerns correctly. There doesn't CURRENTLY seem to be much evidence that what you just said is TRUE. 'Individual' attempts to contribute are getting IGNORED and the last few words of this message thread's subject line are just asking to hear from the powers that be what they intend to do about that ( solutions please ? ). Requiring people to post and re-post and re-post patches until they are blue in the face and/or need to start shouting from mountaintops ( or find a 'backdoor' or 'inside track' into the 'real inner cirlce' like the last guy finally did ) is no way to 'walk the walk' that you just talked ( We value individual contributions ). Prove it. Make it EASY to contribute... not a nightmare. I think that's all the guy is asking for. ( and before you come back with the standard Are YOU willing to review patches, then? at least I'll be honest and say there's NO WAY right now or for the forseeable future. Unlike you... I am NOT paid to work on Apache and I just don't have the time. Besides, I'm also about 100% sure you don't want me reviewing patches for Apache. Newcomers can go read some archived messages if they are curious about the history there. ) Nobody has ever gotten a 'pass' into the Apache HTTP Server for their employment or their employers efforts. Methinks thou dost protest too much. I never suggested any such thing. I never came near saying that Covalent or any of it's myriad Apache developers were getting 'free passes' to anything. It still takes votes to do things at Apache and even though there are times when the vote count includes mostly Covalent people ( like when Apache 2.0 was released too early ) there is always the VETO as a safety catch. It was never exercised because as far as anyone knows nothing un-toward was going on. Apache is stil a 'meritocracy'... but isn't that part of what the guy who started this thread is complaining about? He tried for 4 months to even get a leg up on the bottom rung of the 'meritocracy' and no one gave a shit. I would say that's a short-circuit to the whole scheme itself. The powers that be stay the powers that be unless 'new' people are being give the chance to show what THEY can do. The fact that folks, such as Brad and Madhu, are committers and PMC members is a result of their personal dedication to the project and that the project is proud to count them as members, regardless of current employment status. Then that means there was a time when they first tried to post a patch as well. What was their experience? Good, bad, or ugly? How did they get a 'leg up' in the meritocracy? I'm not asking these questions for myself... It's no mystery to me and I know exactly how it's done and how many rungs there are on that ladder. I am asking the questions on behalf of the original poster and I think the answers might fill out the thread subject. If you look at what has REALLY happened in the past 3 years ( yes... going back that far since it's now 4 or 5 years since 2.0 became a real blib on the radar ) there's no question that there was this intense period of development and 'new' things were happening at a fast rate. Without a doubt this period of development was abnormally intense for any five year old open source project. Good point. ...and let's hope anyone that's even paying attention to this thread isn't expecting Apache development to always be that way. Sometimes ( like now ) it just becomes 'dog work' and the flash and glitter is hard to find. There have NEVER been enough warm bodies at Apache and there never will be. I think the red flags have gone up because it's starting to appear as if NO ONE is answering the phone at all anymore. That's a problem. As more and more developers got interested in getting 2.0 cranked out the (limited) resources all got eaten up in the 2.0 development cycle and 1.3 development virtually shut down. It was even 'officially' shut down long before 2.0 was ready ( 1.3 officially went into maintenance mode only ). That's an interesting point. Most of my early (independent) contributions, about 600 dev hours worth, were entirely focused on making 1.3 work under Win32. I know. From some of your other comments it might appear that your memory is failing a bit, Bill, but you must remember me. I had Apache running under Windows using the (free) Borland compiler about 2 years before you appeared but the concern about making Apache for Windows REAL and going to head to head with Microsoft only cranked up after that Mindspring article appeared and showed that IIS was beating the pants off of pre-fork Apache and the UNIX boys got all pissed off. That was the 2x4 that got the mule's attention. My patches to compile Apache under Windows using
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Mads Toftum wrote: On Wed, Nov 12, 2003 at 05:10:24PM -0800, Aaron Bannert wrote: My main requirement is that the bug tracking system be fully-accessible through email. Having a full web interface is great, but not at the expense of usable offline replies to bug reports. RT could be what you're looking for - http://www.bestpractical.com/rt An example is http://rt.cpan.org Seconded. The perl project uses RT with a great success (http://rt.perl.org/) __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: the wheel of httpd-dev life is surely slowing down, solutions please
I've quite a few ideas and opinions about why things might be quiet these days. I'd recommend against taking any of these ideas too seriously. Here is an idea: we have gotten out in front of the users. Products have features and overtime they get more and more features. Users have some ability to absorb features and overtime their ability to do so rises. For young products the users are ahead of the product (What do you mean it doesn't to CGI scripting!?). For older products the mismatch gets smaller and smaller (I need massive virtual hosting!). My joke about this is that the process is a perfect behaviorist training loop. Young products get strong feedback which trains their sponsors to listen to customer demand and add features. Overtime that feedback peter's out, which trains them to listen more and more carefully. Finally they get to the point where they listen to the wind and they hear feature demands in it's ghostly moans. That's call market research. :-) Commercial products tend to overshoot the users. Consider Microsoft's office products! Open source is less given to overshoot. If features only go in because a user volunteered to do the work that acts; to some degree to temper the chance of overshoot. Open source can still overshoot. Exceptional users may add features that are rarely needed. Firms, lead by market research or other in-house demands, may volunteer. So, one theory is that we have overshot the user's ability to absorb new features. Some amount of overshoot is to be expected on any major release. Solutions? Well if you buy this model - and like I said it's only one - then the trick is to aid users in climbing the learning curve. Figuring out where the user demand is and then helping to bridge the gap using the new features. Helping all the complementary products absorb the new stuff. This may seem like an argument that we have filled out our ecological niche; but it's slightly different than that. The niche isn't fixed, it is a free variable as well. - ben
RE: the wheel of httpd-dev life is surely slowing down, solutions please
Good response - ask the customer what he wants and then help him achieve it. It all starts with stability - compatibility - performance. The ASF has a tough job ahead of it, getting millions of users to change. Not an easy task in today's environment Peter -Original Message- From: Ben Hyde [mailto:[EMAIL PROTECTED] Sent: Thursday, November 13, 2003 7:24 AM To: [EMAIL PROTECTED] Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions please I've quite a few ideas and opinions about why things might be quiet these days. I'd recommend against taking any of these ideas too seriously. Here is an idea: we have gotten out in front of the users. Products have features and overtime they get more and more features. Users have some ability to absorb features and overtime their ability to do so rises. For young products the users are ahead of the product (What do you mean it doesn't to CGI scripting!?). For older products the mismatch gets smaller and smaller (I need massive virtual hosting!). My joke about this is that the process is a perfect behaviorist training loop. Young products get strong feedback which trains their sponsors to listen to customer demand and add features. Overtime that feedback peter's out, which trains them to listen more and more carefully. Finally they get to the point where they listen to the wind and they hear feature demands in it's ghostly moans. That's call market research. :-) Commercial products tend to overshoot the users. Consider Microsoft's office products! Open source is less given to overshoot. If features only go in because a user volunteered to do the work that acts; to some degree to temper the chance of overshoot. Open source can still overshoot. Exceptional users may add features that are rarely needed. Firms, lead by market research or other in-house demands, may volunteer. So, one theory is that we have overshot the user's ability to absorb new features. Some amount of overshoot is to be expected on any major release. Solutions? Well if you buy this model - and like I said it's only one - then the trick is to aid users in climbing the learning curve. Figuring out where the user demand is and then helping to bridge the gap using the new features. Helping all the complementary products absorb the new stuff. This may seem like an argument that we have filled out our ecological niche; but it's slightly different than that. The niche isn't fixed, it is a free variable as well. - ben
Re: the wheel of httpd-dev life is surely slowing down, solutions please
On Nov 12, 2003, at 8:51 PM, Justin Erenkrantz wrote: I think our changes that we already have in the tree is about 'right' for 2.2 (no big architecture changes, but lots of modules have been rewritten and improved). It's just that no one has time or desire to shepherd 2.2 to a release. And, I think we need APR 1.0 out the door before 2.2 can be rationally discussed. If we did any major changes from this point that require modules to rewrite themselves, we'd need to go to 3.0 per our versioning rules. And, I'd *strongly* discourage that. I don't think it's in anyone's best interest to revamp our architecture at this stage. We don't need to give a moving target to our poor module developers. Only by producing a series of high-quality releases can we ensure that module writers will be confident in writing against 2.0 or 2.2. If we come out with 3.x now, I think we'll just end up hurting ourselves in the long run even worse than we are now. -- justin Another point to consider... With 2.2, module writers will need to worry about *3* versions of Apache. Commercial entities which have *just* gotten around to porting their 1.3 modules for 2.0 will likely not bother with 2.2 modules for awhile. There's no easy answer... Maybe dev on 2.0 is slow simply because it's obvious that it's a dead end. Once 2.1 happened, there was a real concern in putting effort into 2.0 because the gravestone was already being carved for it. I do think that some sort of improved communication between bugs and developers would be good. In general, once a developer is aware of a bug and/or patch, things get done.
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Hi all... I just have to jump in here since the topic is fascinating... ...and I think there's an opportunity here to review something that has contributed to the 'slow down' at httpd-dev which no one has seemed to grasp (yet). I will call it... The Covalent Factor. If you look at what has REALLY happened in the past 3 years ( yes... going back that far since it's now 4 or 5 years since 2.0 became a real blib on the radar ) there's no question that there was this intense period of development and 'new' things were happening at a fast rate. As more and more developers got interested in getting 2.0 cranked out the (limited) resources all got eaten up in the 2.0 development cycle and 1.3 development virtually shut down. It was even 'officially' shut down long before 2.0 was ready ( 1.3 officially went into maintenance mode only ). So now you had lots of legacy developers ( albeit... lots of weekend-warriors, too, but WW's are the heartbeat of Open Source ) who knew 1.3 very well but were now totally put out to pasture. Very few of them 'came over' to the 2.0 development. The majority of the developers for 2.0 were the 'paid to play' kind that were either being paid to directly work on Apache ( I'll get to the Covalent factor in a second ) or, at least, were being paid to do something else but no one seemed to mind if they sat there at their real jobs all day and did Apache development. They might not admit to being directly paid to work on Apache but when that's what you are doing all day and still bringing home a paycheck then that's exactly what that is. The 'other' not-so-dedicated-but-certainly-interested developers felt 'shut out' of the 2.0 development cycle because it was obvious a lot of it was taking place 'off line' and nothing was being documented so they couldn't really get a good handle on what was going on in order to make a contribution. So they were mostly 'waiting in the wings' for a lot of the 2.0 DESIGN level decisions and concepts to become clear so they could 'jump in'. Well... we all know that it was a looong time before some of the 2.0 design concepts became 'clear' enough for a not-paid-to-work-on-apache person to 'jump in'. There were basically only 2 big changes to Apache 2.0 versus 1.x. 1. MPM. Get out of the 'prefork' only 'go fork thyself' design model. 2. ( Came later ) Add some I/O filtering and get rid of BUFF. Well, these are both fine goals to have but they are not for the faint-of-heart ( or within the grasp of most weekend-warriors ). That brings me to The Covalent Factor. Is anyone going to deny that at a certain point in the 2.0 development cycle the PRIMARY driving force was a group of people that ALL worked for one company? ( Covalent ). That's certainly the way it looked. ...and these guys were crankin'... I mean they were MOTIVATED, because Covalent's entire corporate life revolves around Apache and they were actually high level deals going on with Compaq ( and other corporate entities? ) that all revolved around getting Apache 2.0 out the door. At one point one of the Covalent guys ( I believe it was that Randy Bloom fellow? ) was pretty much the ONLY person who had any idea how the new 'filtering' was even SUPPOSED to work. It was quite some time before he even finished thinking it through and it went through (too?) many re-works to even keep a good grasp on it. I remember the feeling for MONTHS was something like... let's just see what Covalent comes up with and let them put their stamp of approval on it and then we'll see if we understand it. ...and that's pretty much how the entire SECOND primary goal of Apache 2.0 was achieved. Covalent just did it and expected everyone else to 'go along'... and they did. The point of all this is not to FLAME anyone or any corporate entity. No one can deny that Covalent has been crucial to the life and times ( and success ) of Apache itself and they should be proud of the contribution(s) they have made to OSS. Covalent was started by one of the original Apache guys and some of the best developers left around here still work for Covalent. I think William Rowe still does, for example, and he's still pretty much the only human being who seems to understand the Windows version of Apache... but the legacy flood of 'covalent' email addresses showing up at httpd-dev has virtually vanished so it's hard to tell what what with that anymore. There is nothing in the rules of Open Source that says you can't crank up a company that makes money off of Open Source software. Indeed... if you look at almost ALL of the major Open Source projects you will see the same sort of 'Covalent' deal going on whereby the people that know that OSS best have found a way to get paid to work on it. My only real point here ( and the way all this relates to the current thread ) is that maybe it's time to acknowledge that what is happening now is what will always happen to a major development project if you let too many of your
Re: the wheel of httpd-dev life is surely slowing down, solutions please
On Thu, Nov 13, 2003 at 10:29:55AM -0500, Jim Jagielski wrote: Another point to consider... With 2.2, module writers will need to worry about *3* versions of Apache. Commercial entities which have *just* gotten around to porting their 1.3 modules for 2.0 will likely not bother with 2.2 modules for awhile. Well, 2.2 modules for the most part should be source-compatible with 2.0. Certain subsystems have changed dramatically (auth being the prime example I can think of), but I wouldn't expect every module to have to change to support 2.2. (This is, again, taking my perspective that what we have in HEAD is roughly what should be in 2.2 modulo testing.) Ideally, once we start producing 2.2 releases on a regular basis, we won't be producing 2.0 releases on such a regular basis. -- justin
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Justin Erenkrantz wrote: Well, 2.2 modules for the most part should be source-compatible with 2.0. Certain subsystems have changed dramatically (auth being the prime example I can think of), but I wouldn't expect every module to have to change to support 2.2. (This is, again, taking my perspective that what we have in HEAD is roughly what should be in 2.2 modulo testing.) I'm confused then... I had thought that there were API differences (within httpd and apr) between the 2 trees in able to support some of the new features. -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
RE: the wheel of httpd-dev life is surely slowing down, solutions please
Disclaimer : Not targetting any one individual I have a question to the people have lots of time to write such long mails and responses - why can't you instead spend that time to review patches and give feedback ? It sure will improve the life of httpd-dev. Thanks -Madhu -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Thursday, November 13, 2003 10:47 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions please Hi all... I just have to jump in here since the topic is fascinating... ...and I think there's an opportunity here to review something that has contributed to the 'slow down' at httpd-dev which no one has seemed to grasp (yet). I will call it... The Covalent Factor. If you look at what has REALLY happened in the past 3 years ( yes... going back that far since it's now 4 or 5 years since 2.0 became a real blib on the radar ) there's no question that there was this intense period of development and 'new' things were happening at a fast rate. As more and more developers got interested in getting 2.0 cranked out the (limited) resources all got eaten up in the 2.0 development cycle and 1.3 development virtually shut down. It was even 'officially' shut down long before 2.0 was ready ( 1.3 officially went into maintenance mode only ). So now you had lots of legacy developers ( albeit... lots of weekend-warriors, too, but WW's are the heartbeat of Open Source ) who knew 1.3 very well but were now totally put out to pasture. Very few of them 'came over' to the 2.0 development. The majority of the developers for 2.0 were the 'paid to play' kind that were either being paid to directly work on Apache ( I'll get to the Covalent factor in a second ) or, at least, were being paid to do something else but no one seemed to mind if they sat there at their real jobs all day and did Apache development. They might not admit to being directly paid to work on Apache but when that's what you are doing all day and still bringing home a paycheck then that's exactly what that is. The 'other' not-so-dedicated-but-certainly-interested developers felt 'shut out' of the 2.0 development cycle because it was obvious a lot of it was taking place 'off line' and nothing was being documented so they couldn't really get a good handle on what was going on in order to make a contribution. So they were mostly 'waiting in the wings' for a lot of the 2.0 DESIGN level decisions and concepts to become clear so they could 'jump in'. Well... we all know that it was a looong time before some of the 2.0 design concepts became 'clear' enough for a not-paid-to-work-on-apache person to 'jump in'. There were basically only 2 big changes to Apache 2.0 versus 1.x. 1. MPM. Get out of the 'prefork' only 'go fork thyself' design model. 2. ( Came later ) Add some I/O filtering and get rid of BUFF. Well, these are both fine goals to have but they are not for the faint-of-heart ( or within the grasp of most weekend-warriors ). That brings me to The Covalent Factor. Is anyone going to deny that at a certain point in the 2.0 development cycle the PRIMARY driving force was a group of people that ALL worked for one company? ( Covalent ). That's certainly the way it looked. ...and these guys were crankin'... I mean they were MOTIVATED, because Covalent's entire corporate life revolves around Apache and they were actually high level deals going on with Compaq ( and other corporate entities? ) that all revolved around getting Apache 2.0 out the door. At one point one of the Covalent guys ( I believe it was that Randy Bloom fellow? ) was pretty much the ONLY person who had any idea how the new 'filtering' was even SUPPOSED to work. It was quite some time before he even finished thinking it through and it went through (too?) many re-works to even keep a good grasp on it. I remember the feeling for MONTHS was something like... let's just see what Covalent comes up with and let them put their stamp of approval on it and then we'll see if we understand it. ...and that's pretty much how the entire SECOND primary goal of Apache 2.0 was achieved. Covalent just did it and expected everyone else to 'go along'... and they did. The point of all this is not to FLAME anyone or any corporate entity. No one can deny that Covalent has been crucial to the life and times ( and success ) of Apache itself and they should be proud of the contribution(s) they have made to OSS. Covalent was started by one of the original Apache guys and some of the best developers left around here still work for Covalent. I think William Rowe still does, for example, and he's still pretty much the only human being who seems to understand the Windows version of Apache... but the legacy flood of 'covalent' email addresses showing up at httpd-dev has virtually vanished so it's hard to tell what what with that anymore. There is nothing in the rules of Open Source
Re: the wheel of httpd-dev life is surely slowing down, solutions please
hi, 2d). CRT seemed to come as a replacement for design discussions. It's very easy to observe from the traffic numbers: Please excuse the total ignorance of passive Apache-Dev readers, but these abbreviations were new to me. I've found then im the Apache Glossary, though, and provide them to all who didn't know them ei- ther: http://incubator.apache.org/learn/glossary.html#CommitThenReview http://incubator.apache.org/learn/glossary.html#ReviewThenCommit Not trying to start a flamewar, but could the Jakarta Project have had an influence the decline? Hal Flynn's [1] points out quite well that for most server-sided applications Java (and it's clone .NET) provide a viable platform for secure applications with negligible impact on performance. And with all the fuss around J2EE (JBoss vs. Geronimo) a new feature in Apache might not catch as much attention in the community anymore. OpenSource is a development model which bases on peer-based ego-gratification and if the incentives to work on Apache aren't that high anymore, the Apache httpd server might not be able to attract as many developers anymore. Or to put it in other words: It's not anymore cool to work on Apache. uuhmm .. asbestos or not? No intention to provoke, seriously. We're all grownup people and our kids are going to read these mailinglists in CS-history classes, so behave, please ;) [1] http://www.theregister.co.uk/content/4/33859.html -daniel
Re: the wheel of httpd-dev life is surely slowing down, solutions please
On Mon, Nov 10, 2003 at 05:14:35PM -0800, Stas Bekman wrote: == 1) Bugs searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=REOPENEDproduct=Apache+httpd-2.0 Suggestion: make bugzilla CC bug-reports to the dev list. Most developers won't just go and check the bugs database to see if there is something to fix, both because they don't have time and because there are too many things to fix. Posting the reports to the list raises a chance for the developer to have a free minute and may be to resolve the bug immediately or close it. Would anyone else be in favor of exploring other forms of bug-tracking? The old system (gnats) was completely email-oriented, which was good and bad. Bugzilla may be more usable for web-oriented users, but it it definitely more difficult to use offline (ie. impossible). Perhaps we could try out the bug tracking system used by the debian project (http://www.debian.org/Bugs/). == 2) Lack of Design In my personal opinion, the move to CTR from RTC had very bad implications on the design of httpd 2.1. 2a). Design of new on-going features (and changes/fixes) of the existing features is not discussed before it's put to the code. When it's committed it's usually too late to have any incentive to give a design feedback, after all it's already working and we are too busy with other things, so whatever. I agree with the premise but not your conclusion. Design is being adversely affected when people make large design-changing commits. These people should be posting their design changes (code and all) to the mailing list before committing. However, if people are making commits and others aren't reviewing those commits, then the responsibility falls to the reviewer. Nobody should be obligated to hold on to a patch because other people don't have the time to review. The worst part is that it's now easy to sneak in code which otherwise would never be accepted (and backport it to 2.0). I don't have any examples, but I think the danger is there. I have not seen any evidence of this. 2b). As a side-effect when someone asks a design question (e.g. me) it's not being answered because people tell: we are in the CRT mode, go ahead code it and commit it. But if the poster doesn't know what's the right design, if she did she won't have asked in first place. There is not a fine line between those types of changes that warrant discussion and those that can be simply committed, so this is a difficult issue to address. It is not uncommon for design questions to go unanswered on [EMAIL PROTECTED], and it has been that way as long as I can remember. Patches, on the other hand, speak loudest. 2c). Future design. There is no clear roadmap of where do we go with Apache 2.1. People scratch their itches with new ideas which is cool, but if there was a plan to work on things, this may have invited new developers to scratch their itches. Make proposals (or better yet, add them to the 2.1 STATUS file. :) I would personally like to see: a) httpd.conf rewrite - normalized syntax (structured, maybe even *gasp* XML) - normalized parsing in apache (defined passes, hooks, restart semantics, etc) - other ways to retrieve a config (not just from a file, eg. LDAP) b) sendfile improvements for 64bit memory addressing machines (eg. can we mmap/sendfile a bunch of DVD images at the same time and not crash? Do we see improvements?) c) simplified filters (it's been too long since I've thought about this, but basicly writing filters is too difficult and should be easier). 2d). CRT seemed to come as a replacement for design discussions. It's very easy to observe from the traffic numbers: I think it's actually the opposite. The amount of design discussions in general has dramatically decreased since RTC went into effect in 2.0. I recall very few discussions around 2.1 in general. == 3). Contributions I don't have numbers to support my clause, but I have a strong feeling that nowadays we see a much smaller number of posts with contributions from non-developers, since most contributions are simply ignored and it's a known thing among Apache community that posting fixes and suggestions to httpd-dev list is usually a waste of time. (This is based on my personal experience and discussions with other developers who aren't httpd-core coders). I realize that I'm not entirely correct saying that, since some things are picked up, so I apologize to those folks who do try hard to pick those things. The obvious problem is that most of those things are unsexy, usually don't fit someones itch and they sometimes require a lot of communication overhead with the
Re: the wheel of httpd-dev life is surely slowing down, solutions please
On Mon, Nov 10, 2003 at 05:50:46PM -0800, Justin Erenkrantz wrote: --On Monday, November 10, 2003 17:14:35 -0800 Stas Bekman [EMAIL PROTECTED] wrote: I believe Sander's suggested that we start a patch manager role (discussion at AC Hackathon!). A few other projects I'm involved with do this. It'd be goodness, I think. But, the problem is finding a volunteer willing to shepherd patches. -1 Creating roles like this is just as bad as having chunks of httpd owned by one particular developer. (See below) Having a patch tracking tool would be a much better solution, in my opinion. a. certain (most?) chunks of httpd lack owners. If httpd was to have I really dislike this. The least maintained modules are those that had 'owners.' When those owners left, they didn't send a notice saying, 'we're leaving.' The code just rotted. I firmly believe that httpd's group ownership is the right model. Only recently have those modules that had 'owners' been cleaned up. (Yay for nd!) +1 b. similar to the root rotation we have on the infrastructure, I suggest to have a voluntary weekly rotation of httpd-dev list champion +1 -1 How is this different than owners above? You can't expect developers on this list to think ahead of time when they might have extra time to contribute to this project. Most of the time it is on a whim late at night when all of the important chores are taken care of. If you ask for people to sign up for a slotted amount of time, you'll only end up excluding people who don't work on Apache at their day job. Additionally, I'd say that we're best off *slowing* down 2.x development to let module authors write modules against 2.0. Changing the world in 2.2 would be detrimental, I think. Woah woah woah. Are you discouraging people from thinking about big changes for the 2.2 timeframe? If someone has a revolutionary idea for the next generation of Apache HTTPD, who will stand in their way? Not I. -aaron
Re: the wheel of httpd-dev life is surely slowing down, solutions please
On 13.11.2003, at 01:29, Aaron Bannert wrote: On Mon, Nov 10, 2003 at 05:14:35PM -0800, Stas Bekman wrote: == 1) Bugs searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries http://nagoya.apache.org/bugzilla/buglist.cgi? bug_status=NEWbug_status=REOPENEDproduct=Apache+httpd-2.0 Suggestion: make bugzilla CC bug-reports to the dev list. Most developers won't just go and check the bugs database to see if there is something to fix, both because they don't have time and because there are too many things to fix. Posting the reports to the list raises a chance for the developer to have a free minute and may be to resolve the bug immediately or close it. Would anyone else be in favor of exploring other forms of bug-tracking? The old system (gnats) was completely email-oriented, which was good and bad. Bugzilla may be more usable for web-oriented users, but it it definitely more difficult to use offline (ie. impossible). Perhaps we could try out the bug tracking system used by the debian project (http://www.debian.org/Bugs/). I'd be very in favour of exploring other forms of bug-tracking. For example we'll have a full replication of BugZilla in Jira (/me hides) in the near future here on ASF hardware (see http://nagoya.apache.org/jira/ for a preview). I've to admit that I really like it and especially as a patch-tracking tool it could be useful, at least IMHO. Btw, Scarab (http://nagoya.apache.org/scarab/issues) is also available but I doubt that it is used by a single project ;-) Cheers, Erik
Re: the wheel of httpd-dev life is surely slowing down, solutions please
On Thu, Nov 13, 2003 at 02:04:31AM +0100, Erik Abele wrote: I'd be very in favour of exploring other forms of bug-tracking. For example we'll have a full replication of BugZilla in Jira (/me hides) in the near future here on ASF hardware (see http://nagoya.apache.org/jira/ for a preview). I've to admit that I really like it and especially as a patch-tracking tool it could be useful, at least IMHO. Btw, Scarab (http://nagoya.apache.org/scarab/issues) is also available but I doubt that it is used by a single project ;-) My main requirement is that the bug tracking system be fully-accessible through email. Having a full web interface is great, but not at the expense of usable offline replies to bug reports. (Do either of these bug tracking systems support this?) -aaron
Re: the wheel of httpd-dev life is surely slowing down, solutions please
On 13.11.2003, at 02:10, Aaron Bannert wrote: On Thu, Nov 13, 2003 at 02:04:31AM +0100, Erik Abele wrote: I'd be very in favour of exploring other forms of bug-tracking. For example we'll have a full replication of BugZilla in Jira (/me hides) in the near future here on ASF hardware (see http://nagoya.apache.org/jira/ for a preview). I've to admit that I really like it and especially as a patch-tracking tool it could be useful, at least IMHO. Btw, Scarab (http://nagoya.apache.org/scarab/issues) is also available but I doubt that it is used by a single project ;-) My main requirement is that the bug tracking system be fully-accessible through email. Having a full web interface is great, but not at the expense of usable offline replies to bug reports. Okay, I can understand that. (Do either of these bug tracking systems support this?) Hmm, don't know, but I don't think so... Is bugzilla able to be fully controlled through email? Cheers, Erik -aaron
Re: the wheel of httpd-dev life is surely slowing down, solutions please
On Wed, Nov 12, 2003 at 05:10:24PM -0800, Aaron Bannert wrote: My main requirement is that the bug tracking system be fully-accessible through email. Having a full web interface is great, but not at the expense of usable offline replies to bug reports. RT could be what you're looking for - http://www.bestpractical.com/rt An example is http://rt.cpan.org vh Mads Toftum -- `Darn it, who spiked my coffee with water?!' - lwall
Re: the wheel of httpd-dev life is surely slowing down, solutions please
On Thu, Nov 13, 2003 at 02:17:01AM +0100, Erik Abele wrote: My main requirement is that the bug tracking system be fully-accessible through email. Having a full web interface is great, but not at the expense of usable offline replies to bug reports. Okay, I can understand that. (Do either of these bug tracking systems support this?) Hmm, don't know, but I don't think so... Is bugzilla able to be fully controlled through email? Nope, but it sucks for other reasons too. ;) We used to use gnats until someone suggested we try Bugzilla. I suggested today that we might look into using the homegrown bug tracking system used by debian. It is email based with a web interface. -aaron
Re: the wheel of httpd-dev life is surely slowing down, solutions please
--On Wednesday, November 12, 2003 16:38:11 -0800 Aaron Bannert [EMAIL PROTECTED] wrote: I believe Sander's suggested that we start a patch manager role (discussion at AC Hackathon!). A few other projects I'm involved with do this. It'd be goodness, I think. But, the problem is finding a volunteer willing to shepherd patches. -1 Creating roles like this is just as bad as having chunks of httpd owned by one particular developer. (See below) I don't think you understand the role of 'patch manager.' On the projects I'm involved with, that person's only responsibility is for ensuring that the patches posted to the mailing list are entered into the right tool. I think that can easily be a volunteer, rotating job as you don't need to do anything more than that. How is this different than owners above? You can't expect developers on this list to think ahead of time when they might have extra time to contribute to this project. Most of the time it is on a whim late at night when all of the important chores are taken care of. If you ask for people to sign up for a slotted amount of time, you'll only end up excluding people who don't work on Apache at their day job. In the patch manager role, nothing would preclude others from adding patches to the right tool. But, having someone whose responsibility it is to ensure that patches get inputted into the tool on a timely basis is goodness. On other projects I'm involved with, that person isn't an even a committer. They don't have to understand what the patch does - just make sure it is recorded and allow for annotation. And, it doesn't have to be on a 'timely' basis - once a week, you go through and ensure that all [PATCH] messages are in the right patch management tool. This lessens the number of dropped patches that we'd have. (Bugzilla sucks at this, so I think we'd need a new tool.) Woah woah woah. Are you discouraging people from thinking about big changes for the 2.2 timeframe? If someone has a revolutionary idea for the next generation of Apache HTTPD, who will stand in their way? Not I. Frankly, yes. I think our changes that we already have in the tree is about 'right' for 2.2 (no big architecture changes, but lots of modules have been rewritten and improved). It's just that no one has time or desire to shepherd 2.2 to a release. And, I think we need APR 1.0 out the door before 2.2 can be rationally discussed. If we did any major changes from this point that require modules to rewrite themselves, we'd need to go to 3.0 per our versioning rules. And, I'd *strongly* discourage that. I don't think it's in anyone's best interest to revamp our architecture at this stage. We don't need to give a moving target to our poor module developers. Only by producing a series of high-quality releases can we ensure that module writers will be confident in writing against 2.0 or 2.2. If we come out with 3.x now, I think we'll just end up hurting ourselves in the long run even worse than we are now. -- justin
Re: the wheel of httpd-dev life is surely slowing down, solutions please
At 07:14 PM 11/10/2003, Stas Bekman wrote: I have several reasons to believe that the wheel of httpd-dev life is slowing down and something has to be done to get this wheel up to the speed like in the good old days. The following observation are listed in no particular order. I've also tried to offer suggestions how to resolve problems. == 1) Bugs searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=REOPENEDproduct=Apache+httpd-2.0 Suggestion: make bugzilla CC bug-reports to the dev list. Most developers won't just go and check the bugs database to see if there is something to fix, both because they don't have time and because there are too many things to fix. Posting the reports to the list raises a chance for the developer to have a free minute and may be to resolve the bug immediately or close it. == 2) Lack of Design In my personal opinion, the move to CTR from RTC had very bad implications on the design of httpd 2.1. 2a). Design of new on-going features (and changes/fixes) of the existing features is not discussed before it's put to the code. When it's committed it's usually too late to have any incentive to give a design feedback, after all it's already working and we are too busy with other things, so whatever. The worst part is that it's now easy to sneak in code which otherwise would never be accepted (and backport it to 2.0). I don't have any examples, but I think the danger is there. 2b). As a side-effect when someone asks a design question (e.g. me) it's not being answered because people tell: we are in the CRT mode, go ahead code it and commit it. But if the poster doesn't know what's the right design, if she did she won't have asked in first place. 2c). Future design. There is no clear roadmap of where do we go with Apache 2.1. People scratch their itches with new ideas which is cool, but if there was a plan to work on things, this may have invited new developers to scratch their itches. 2d). CRT seemed to come as a replacement for design discussions. It's very easy to observe from the traffic numbers: http://marc.theaimsgroup.com/?l=apache-httpd-dev 2003-11-01 - 2003-12-01 (118 messages) 2003-10-01 - 2003-11-01 (336 messages) 2003-09-01 - 2003-10-01 (275 messages) 2003-08-01 - 2003-09-01 (264 messages) 2003-07-01 - 2003-08-01 (321 messages) 2003-06-01 - 2003-07-01 (430 messages) 2003-05-01 - 2003-06-01 (450 messages) 2003-04-01 - 2003-05-01 (359 messages) 2003-03-01 - 2003-04-01 (696 messages) 2003-02-01 - 2003-03-01 (573 messages) 2003-01-01 - 2003-02-01 (546 messages) 2002-12-01 - 2003-01-01 (436 messages) 2002-11-01 - 2002-12-01 (538 messages) 2002-10-01 - 2002-11-01 (763 messages) 2002-09-01 - 2002-10-01 (894 messages) 2002-08-01 - 2002-09-01 (815 messages) 2002-07-01 - 2002-08-01 (721 messages) 2002-06-01 - 2002-07-01 (916 messages) 2002-05-01 - 2002-06-01 (1028 messages) 2002-04-01 - 2002-05-01 (1380 messages) 2002-03-01 - 2002-04-01 (1094 messages) 2002-02-01 - 2002-03-01 (1155 messages) Quiz: based on this input, tell which date CRT was introduced at. You don't need a fancy graph to see a clear decline in the last 1.5 years. Quite a few people suggested that this is due to the market decline. I won't doubt them, but it's quite possible that the market hasn't changed to the worst in the last 1.5 years (it is more likely that we should have seen this dip in 2001-2002, which I can't see from the numbers at the above URL). The cvs commit average rate hasn't changed much, which shows that development continues though mainly behind the scenes. http://marc.theaimsgroup.com/?l=apache-cvsr=1w=2 == 3). Contributions I don't have numbers to support my clause, but I have a strong feeling that nowadays we see a much smaller number of posts with contributions from non-developers, since most contributions are simply ignored and it's a known thing among Apache community that posting fixes and suggestions to httpd-dev list is usually a waste of time. (This is based on my personal experience and discussions with other developers who aren't httpd-core coders). I realize that I'm not entirely correct saying that, since some things are picked up, so I apologize to those folks who do try hard to pick those things. The obvious problem is that most of those things are unsexy, usually don't fit someones itch and they sometimes require a lot of communication overhead with the reporter/contributor just to understand what exactly is going on. Solutions: a. certain (most?) chunks of httpd lack owners. If httpd was to have clear owners of certain code bases then it'll be easy to take care of bug reports and
Re: the wheel of httpd-dev life is surely slowing down, solutions please
At 07:14 PM 11/10/2003, Stas Bekman wrote: I have several reasons to believe that the wheel of httpd-dev life is slowing down and something has to be done to get this wheel up to the speed like in the good old days. The following observation are listed in no particular order. I've also tried to offer suggestions how to resolve problems. Always good to discuss, but httpd is about folks contributing and the team reviewing patches. When there are lots of activity, things move quickly. Summer is a slow period for alot of projects. Stability slows things down too (and yes, 2.0.48 is far more stable than anything released before on the 2.0 branch.) 1) Bugs searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=REOPENEDproduct=Apache+httpd-2.0 Yes - these are important, I'm the first to admit I have a number I track that I haven't resolved. And more eyes are needed :) However... Suggestion: make bugzilla CC bug-reports to the dev list. Most developers won't just go and check the bugs database to see if there is something to fix, both because they don't have time and because there are too many things to fix. Posting the reports to the list raises a chance for the developer to have a free minute and may be to resolve the bug immediately or close it. Nope, it assures a good number of interested contributors SHUT DOWN their subscriptions to httpd-dev. Trust me, I subscribe to bug traffic as well. Guess which of the two I read daily? If I didn't split them, I would never be caught up on the developer discussions. (Even those I don't understand completely I do read.) 2) Lack of Design In my personal opinion, the move to CTR from RTC had very bad implications on the design of httpd 2.1. You seem to be misinformed here... Apache 2.0 was ALWAYS CTR until we decided to put a roadblock in the way of breaking the stable version, and encourage that experimentation to move to an unstable branch. RTC was only introduced after the last ApacheCon. And very very resisted, at that. 2a). Design of new on-going features (and changes/fixes) of the existing features is not discussed before it's put to the code. When it's committed it's usually too late to have any incentive to give a design feedback, after all it's already working and we are too busy with other things, so whatever. However, Show me the code remains the mantra of httpd. So continuing to use CTR, the code is in place. With httpd-test/perl-framework the coder can ever verify that their new change doesn't break any tests that are already defined. The worst part is that it's now easy to sneak in code which otherwise would never be accepted (and backport it to 2.0). I don't have any examples, but I think the danger is there. That's always been true in httpd. We didn't lock 1.3 until 2.0 was nearing completion. We didn't lock 2.0 until 2.1 was available for folks to hack on. It is VERY hard to get code into 2.0. That code is RTC. Gratuitous changes and new features aren't encouraged. Time to move on to 2.1 for all of the great and wonderful things our web server aught to do. In fact, it's why I've suggested point blank that the 'which pass at the startup file' patch you offered will never belong in 2.0 - anyone designing for an arbitrary 2.0 can't even count on such a feature. But anyone using the future 2.2 release should trust that it is present. 2b). As a side-effect when someone asks a design question (e.g. me) it's not being answered because people tell: we are in the CRT mode, go ahead code it and commit it. But if the poster doesn't know what's the right design, if she did she won't have asked in first place. Nothing stops anyone from posting up code first. Many of us post up code when we just aren't sure if it's the one best answer - and although it takes folks a while sometimes to review, the dialog is useful. But that discussion shouldn't be an obstacle to writing code. 2c). Future design. There is no clear roadmap of where do we go with Apache 2.1. People scratch their itches with new ideas which is cool, but if there was a plan to work on things, this may have invited new developers to scratch their itches. The roadmap is this. When httpd-2.1 solves enough issues that 2.0 cannot (such as a major auth reorganization, perhaps auth credential chains, and hopefully some improvements to the filtering schema and file system provider) AND it can pass the regressions as gracefully as 2.0 - we are ready to start with some alphas. All someone has to do is speak up and say it's time. No fixed lists of features set in stone, no. If that *must have* feature isn't done, then it moves on to 2.3 to await inclusion in 2.4. It all depends on what folks can dedicate their time to. 2d). CRT seemed to come as a replacement for design discussions. It's very easy to observe from the traffic numbers: [snip] Quiz: based on this input, tell which date CRT was
RE: the wheel of httpd-dev life is surely slowing down, solutions please
Something else to think about... What's the differentiator in the market place between 1.x and 2.x? (hint: it's not a feature list) If I was to go out and buy Apache (Covalent) apart from some management tools (features) what's the biggest differentiator between the old version (public domain 1.x) and the new version (2.x.) Invariably three things come to mind - performance, stability, compatibility? So is Apache 2.x faster or slower than 1.x? Has anyone shown definitively that 2.x can serve pages faster than 1.x? and if so by what percentage or factor? Looks like 2.1 is going to be more stable than the old 2.x versions but is it more stable than 1.x? Finally compatibility? Can I run my 1.x modules unmodified in a 2.x environment? HTTPD dev life is slowing down - simply fixing bugs and adding new features is not the way to differentiate, there's probably 50,000 users of 2.x while there's millions and millions of 1.x users. What's it going to take to shift them to 2.x? Not features (only the minority of users now care) - not compatibility (although it helps) - my bet is raw performance. So, anyone got any hard data that shows Apache 2.x serving pages factors faster than 1.x? Regards, Peter -Original Message- From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 11, 2003 2:00 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions please At 07:14 PM 11/10/2003, Stas Bekman wrote: I have several reasons to believe that the wheel of httpd-dev life is slowing down and something has to be done to get this wheel up to the speed like in the good old days. The following observation are listed in no particular order. I've also tried to offer suggestions how to resolve problems. Always good to discuss, but httpd is about folks contributing and the team reviewing patches. When there are lots of activity, things move quickly. Summer is a slow period for alot of projects. Stability slows things down too (and yes, 2.0.48 is far more stable than anything released before on the 2.0 branch.) 1) Bugs searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=REO PENEDproduct=Apache+httpd-2.0 Yes - these are important, I'm the first to admit I have a number I track that I haven't resolved. And more eyes are needed :) However... Suggestion: make bugzilla CC bug-reports to the dev list. Most developers won't just go and check the bugs database to see if there is something to fix, both because they don't have time and because there are too many things to fix. Posting the reports to the list raises a chance for the developer to have a free minute and may be to resolve the bug immediately or close it. Nope, it assures a good number of interested contributors SHUT DOWN their subscriptions to httpd-dev. Trust me, I subscribe to bug traffic as well. Guess which of the two I read daily? If I didn't split them, I would never be caught up on the developer discussions. (Even those I don't understand completely I do read.) 2) Lack of Design In my personal opinion, the move to CTR from RTC had very bad implications on the design of httpd 2.1. You seem to be misinformed here... Apache 2.0 was ALWAYS CTR until we decided to put a roadblock in the way of breaking the stable version, and encourage that experimentation to move to an unstable branch. RTC was only introduced after the last ApacheCon. And very very resisted, at that. 2a). Design of new on-going features (and changes/fixes) of the existing features is not discussed before it's put to the code. When it's committed it's usually too late to have any incentive to give a design feedback, after all it's already working and we are too busy with other things, so whatever. However, Show me the code remains the mantra of httpd. So continuing to use CTR, the code is in place. With httpd-test/perl-framework the coder can ever verify that their new change doesn't break any tests that are already defined. The worst part is that it's now easy to sneak in code which otherwise would never be accepted (and backport it to 2.0). I don't have any examples, but I think the danger is there. That's always been true in httpd. We didn't lock 1.3 until 2.0 was nearing completion. We didn't lock 2.0 until 2.1 was available for folks to hack on. It is VERY hard to get code into 2.0. That code is RTC. Gratuitous changes and new features aren't encouraged. Time to move on to 2.1 for all of the great and wonderful things our web server aught to do. In fact, it's why I've suggested point blank that the 'which pass at the startup file' patch you offered will never belong in 2.0 - anyone designing for an arbitrary 2.0 can't even count on such a feature. But anyone using the future 2.2 release should trust that it is present. 2b). As a side-effect when someone asks a design question
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Stas Bekman wrote: 1) Bugs searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=REOPENEDproduct=Apache+httpd-2.0 yes, far too many :( Suggestion: make bugzilla CC bug-reports to the dev list. Most developers won't just go and check the bugs database to see if there is something to fix, both because they don't have time and because there are too many things to fix. Posting the reports to the list raises a chance for the developer to have a free minute and may be to resolve the bug immediately or close it. That would reduce the number of subscribers to dev@, which is the opposite of what we need. At least we have a monthy summary posted to dev@ which reminds folks that there is a bug db. Also, it is my interpretation of bug db traffic that Joshua and Andre' already jump on the PRs that can be resolved immediately. Many of the outstanding PRs are quite difficult to resolve. Working the bug db is a noble effort, but many in our community have no time to work it. The way to get more time being spent on the bug db is to take the kind of actions that increase the number of httpd developers. 2) Lack of Design 2a). Design of new on-going features (and changes/fixes) of the existing features is not discussed before it's put to the code. When it's committed it's usually too late to have any incentive to give a design feedback, after all it's already working and we are too busy with other things, so whatever. We leave it up to the developer to judge whether some design issue needs to be discussed first. If something is committed prior to being discussed, then the person with the objection must start the discussion at that point. This is a tool against stagnation. If somebody has the time/energy to work on something, we can't put up a roadblock just because nobody else has the time/skills to discuss it with them. The only roadblock is the one to protect users of stable releases. The worst part is that it's now easy to sneak in code which otherwise would never be accepted (and backport it to 2.0). I don't have any examples, but I think the danger is there. There is a barrier to getting things backported to 2.0 as a protection against possible drawbacks of C-T-R. If three people are in favor of putting something in the stable branch, mistakes can still happen, but I don't see how we can refer to such a backport as sneaking in code. 2b). As a side-effect when someone asks a design question (e.g. me) it's not being answered because people tell: we are in the CRT mode, go ahead code it and commit it. But if the poster doesn't know what's the right design, if she did she won't have asked in first place. Alternate opinion: If a question is not answered, it is because nobody has time/skills. As mentioned above, I see the C-T-R mode for 2.1-dev as a tool that helps work-around a lack of energy/skills on the list. 2c). Future design. There is no clear roadmap of where do we go with Apache 2.1. People scratch their itches with new ideas which is cool, but if there was a plan to work on things, this may have invited new developers to scratch their itches. 2d). CRT seemed to come as a replacement for design discussions. It's very easy to observe from the traffic numbers: As Bill said, it has been this way for some time. At some point we created a stable branch for 2.0-dev, but 2.1-dev is handled in the same way that 2.x was handled all along. 3). Contributions I don't have numbers to support my clause, but I have a strong feeling that nowadays we see a much smaller number of posts with contributions from non-developers I can believe it. Solutions: a. certain (most?) chunks of httpd lack owners. If httpd was to have clear owners of certain code bases then it'll be easy to take care of bug reports and contributions/patches. Even though httpd is collectively owned, it doesn't preclude champions in certain areas, who can see that good things happen to the areas they overlook. b. similar to the root rotation we have on the infrastructure, I suggest to have a voluntary weekly rotation of httpd-dev list champion whose responsibility is to make sure that most (all?) bug-reports and contributions are dealts with: assigned to those who those champions who know how to fix/review/adopt things (See (a) above), asking for more input if needed, etc. You don't have to know httpd as your five fingers to be able to do a great rewarding job. Personally I'm very disinterested in a system that puts certain responsibility on a single person for an area of the code or for a period of time. Work/life issues arise from time to time that force folks to disappear for a stretch. To the greatest extent possible, any policies/procedures we have should have the goal of channelling volunteer efforts instead of placing certain responsibilities with a single person. It would be good to have a single place other than the
Re: the wheel of httpd-dev life is surely slowing down, solutions please
hi, 2d). CRT seemed to come as a replacement for design discussions. It's very easy to observe from the traffic numbers: Please excuse the total ignorance of passive Apache-Dev readers, but these abbreviations were new to me. I've found then im the Apache Glossary, though, and provide them to all who didn't know them ei- ther: http://incubator.apache.org/learn/glossary.html#CommitThenReview http://incubator.apache.org/learn/glossary.html#ReviewThenCommit Not trying to start a flamewar, but could the Jakarta Project have had an influence on the decline? Hal Flynn's [1] points out quite well that for most server-sided applications Java (and it's clone .NET) provide a viable platform for secure applications with negligible impact on performance. And with all the fuss around J2EE (JBoss vs. Geronimo) a new feature in Apache might not catch as much attention in the community anymore. OpenSource is a development model which bases on peer-based ego-gratification and if the incentives to work on Apache aren't that high anymore, the Apache httpd server might not be able to attract as many developers anymore. Or to put it in other words: It's not anymore cool to work on Apache. uuhmm .. asbestos or not? No intention to provoke, seriously. We're all grownup people and our kids are going to read these mailinglists in CS-history classes, so behave, please ;) [1] http://www.theregister.co.uk/content/4/33859.html -daniel
RE: the wheel of httpd-dev life is surely slowing down, solutions please
It's not anymore cool to work on Apache. You nailed it - because no one knows where it's going. Where's the focus, what does Apache really want to be, whose leading the charge? I've been following this forum a long, long time and the change in the last 2 years has been the most dramatic - the old guard has gone, there is little leadership and even less reason to do anything. It takes a tremendous amount of work to build a quality software project and sadly there is little enthusiasm to really improve Apache. One reason is obvious - with 66% of the market you're a monopoly (close) and we've all seen what happens when competition disappears from the market place. Regards, Peter -Original Message- From: Daniel Lorch [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 11, 2003 7:10 AM To: [EMAIL PROTECTED] Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions please hi, 2d). CRT seemed to come as a replacement for design discussions. It's very easy to observe from the traffic numbers: Please excuse the total ignorance of passive Apache-Dev readers, but these abbreviations were new to me. I've found then im the Apache Glossary, though, and provide them to all who didn't know them ei- ther: http://incubator.apache.org/learn/glossary.html#CommitThenReview http://incubator.apache.org/learn/glossary.html#ReviewThenCommit Not trying to start a flamewar, but could the Jakarta Project have had an influence on the decline? Hal Flynn's [1] points out quite well that for most server-sided applications Java (and it's clone .NET) provide a viable platform for secure applications with negligible impact on performance. And with all the fuss around J2EE (JBoss vs. Geronimo) a new feature in Apache might not catch as much attention in the community anymore. OpenSource is a development model which bases on peer-based ego-gratification and if the incentives to work on Apache aren't that high anymore, the Apache httpd server might not be able to attract as many developers anymore. Or to put it in other words: It's not anymore cool to work on Apache. uuhmm .. asbestos or not? No intention to provoke, seriously. We're all grownup people and our kids are going to read these mailinglists in CS-history classes, so behave, please ;) [1] http://www.theregister.co.uk/content/4/33859.html -daniel
Re: the wheel of httpd-dev life is surely slowing down, solutions please
hi, I'm not sure http-dev is the place to flam ASF and its commiters. I don't think it was Peter's intention to flame anyone. The ASF has done a great job to deliver a fantastic, widely-deployed webserver. Consindering though that Apache 2 is mostly a refactored 1.3.x, which doesn't provide anything spectacularly new. Yes improved X and better Y, but hey, it still is a webserver and it doesn't make coffee for you. If I can get my name into the headlines [1] when writing Java-Stuff, hell, then I'll do it. Read The Cathedral The Bazaar from ESR [2]. It provides a good insight into the social structures of the community: motivations, incentives, .. It's not always the money, you know ;) Market share? Who cares! Customers? Who cares! I want my name to be a three letter acronym everyone recognizes! RMS? ESR? [1] http://apache.slashdot.org/apache/03/11/10/2057218.shtml http://www.theserverside.com/home/thread.jsp?thread_id=22337 [2] http://www.oreilly.com/catalog/cathbazpaper/ -daniel
RE: the wheel of httpd-dev life is surely slowing down, solutions please
There is no flame - just a couple of points and a request for data. If you want to improve something, you should provide solutions, not critics Certainly - early next year you will see them. Here are some current performance stats with some new technology we're working on. Configuration ToolElapsed Time (secs) Data Transfer Rate (KB/sec)Requests per Second Requests per Minute Performance Gain Factor Apache Apache Bench38.735 882.92 2581.64 154,898 1.0 Cyclone Proxy Cache + Apache Apache Bench15.663 2387.79 6384.47 383,068 2.47 Apache Zeus Bench 39.961 855.83 2502.44 150,146.4 1.0 Squid + Apache Zeus Bench 28.910 1314.42 3459.01 207,540.6 1.38 Cyclone Proxy Cache + Apache Zeus Bench 15.176 2464.42 6589.35 395,361 2.63 Cyclone Proxy Cache (Tuned Parser) + Apache Zeus Bench 13.505 2769.34 7404.67 444,280.2 2.95 Cyclone Proxy Cache (4 Tuned Functions) + Apache Zeus Bench 13.006 2875.6 7688.76 461,325.6 3.07 These numbers were obtained using a single processor Itanium® 1.0Ghz (Madison) chip. By tuning certain HTTP string handling functions we have seen up to a factor 11 performance improvement. Our next benchmark is due by year end. Essentially we will be adding one more line for the stats above. The goal is very simple - transmit greater than 1 million requests in a single minute on a single processor Itanium 1Ghz machine. A factor 10 performance improvement. A single processor Deerfield Itanium® chip costs $744 - our solution doesn't require a current OS, nor hard drive to operate - it scales to multiple chips and can support a cache of up to 1 terabyte of RAM Revolution is for new players, carefully crafted evolutions are for the Mass Yep Support for a 1TB cache, no hard drive, no current OS required, and the ability to pump data faster than any other platform on the planet should do the trick. Only thing left is to get the Itanium® platform into a single 1RU box at sub $5,000. I doubt we will have to wait long for that. Long live the revolution Regards, Peter -Original Message- From: Henri Gomez [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 11, 2003 7:41 AM To: [EMAIL PROTECTED] Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions please Peter J. Cranstone a écrit : It's not anymore cool to work on Apache. You nailed it - because no one knows where it's going. Where's the focus, what does Apache really want to be, whose leading the charge? I've been following this forum a long, long time and the change in the last 2 years has been the most dramatic - the old guard has gone, there is little leadership and even less reason to do anything. It takes a tremendous amount of work to build a quality software project and sadly there is little enthusiasm to really improve Apache. One reason is obvious - with 66% of the market you're a monopoly (close) and we've all seen what happens when competition disappears from the market place. I'm not sure http-dev is the place to flam ASF and its commiters. If you want to improve something, you should provide solutions, not critics. HTTPD have 66% of the market and that's great to see that an OpenSource solution is well behind M$. Sun, Oracle and majors corps have stopped dreaming having 50% of market share some years ago. At least we could say, Apache Software Foundation does it and maintain its leading position. How ? - By producing solutions like HTTPD which are stable, full featured and works on so many platforms. Revolution is for new players, carefully crafted evolutions are for the mass.
Re: the wheel of httpd-dev life is surely slowing down, solutions please
* Daniel Lorch [EMAIL PROTECTED] wrote: If I can get my name into the headlines [1] when writing Java-Stuff, hell, then I'll do it. Read The Cathedral The Bazaar from ESR [2]. It provides a good insight into the social structures of the community: motivations, incentives, .. It's not always the money, you know ;) Market share? Who cares! Customers? Who cares! I want my name to be a three letter acronym everyone recognizes! RMS? ESR? Sure, but that's your way. It's not mine, for example. My personal intention is to make the software better. That's it. I don't need to read my name in headlines. Why did I start contributing? Because I've found errors. Why did I start cleaning up Bugzilla-Reports? Because I searched something and didn't find through because there was too much noise. That's all. Forget my name. Use that great piece of open source software and learn from it. I love good and clean code. So my most contributions clean up code and try to improve it. In fact, most people will never know about this, because it's just behind the scenes. Apart from the above I don't like Java very much ;-) nd
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Daniel Lorch a écrit : hi, I'm not sure http-dev is the place to flam ASF and its commiters. I don't think it was Peter's intention to flame anyone. The ASF has done a great job to deliver a fantastic, widely-deployed webserver. Consindering though that Apache 2 is mostly a refactored 1.3.x, which doesn't provide anything spectacularly new. Yes improved X and better Y, but hey, it still is a webserver and it doesn't make coffee for you. Yes, HTTPD 2.0 is still a webserver but the refactory was impressive and it use APR which is a wonderfull API to hide Operating System specifics. If I can get my name into the headlines [1] when writing Java-Stuff, hell, then I'll do it. Read The Cathedral The Bazaar from ESR [2]. It provides a good insight into the social structures of the community: motivations, incentives, .. It's not always the money, you know ;) Market share? Who cares! Customers? Who cares! I want my name to be a three letter acronym everyone recognizes! RMS? ESR? [1] http://apache.slashdot.org/apache/03/11/10/2057218.shtml http://www.theserverside.com/home/thread.jsp?thread_id=22337 [2] http://www.oreilly.com/catalog/cathbazpaper/ I know well 'The Cathedral The Bazaar' and it's not allways money, but who speak about money here. I just speak about market shares, in term of users base. And when more than 66% of users trust your product you couldn't just make permanent revolution. If you don't care about customers, in our case HTTPD users, I'm not sure you understand that even if OSS is a software production model, it's final goal is to produce real software, for real users, and more users are using your software more responsability you have. If someone want to produce software for its own use, without users considerations, well it shouldn't works in communities and certainly not with ASF commiters.
Re: the wheel of httpd-dev life is surely slowing down, solutions please
Peter J. Cranstone a crit : There is no flame - just a couple of points and a request for data. If you want to improve something, you should provide solutions, not critics Certainly - early next year you will see them. Here are some current performance stats with some new technology we're working on. Configuration Tool Elapsed Time (secs) Data Transfer Rate (KB/sec) Requests per Second Requests per Minute Performance Gain Factor Apache Apache Bench 38.735 882.92 2581.64 154,898 1.0 Cyclone Proxy Cache + Apache Apache Bench 15.663 2387.79 6384.47 383,068 2.47 Apache Zeus Bench 39.961 855.83 2502.44 150,146.4 1.0 Squid + Apache Zeus Bench 28.910 1314.42 3459.01 207,540.6 1.38 Cyclone Proxy Cache + Apache Zeus Bench 15.176 2464.42 6589.35 395,361 2.63 Cyclone Proxy Cache (Tuned Parser) + Apache Zeus Bench 13.505 2769.34 7404.67 444,280.2 2.95 Cyclone Proxy Cache (4 Tuned Functions) + Apache Zeus Bench 13.006 2875.6 7688.76 461,325.6 3.07 These numbers were obtained using a single processor Itanium 1.0Ghz (Madison) chip. By tuning certain HTTP string handling functions we have seen up to a factor 11 performance improvement. Our next benchmark is due by year end. Essentially we will be adding one more line for the stats above. The goal is very simple - transmit greater than 1 million requests in a single minute on a single processor Itanium 1Ghz machine. A factor 10 performance improvement. A single processor Deerfield Itanium chip costs $744 - our solution doesn't require a current OS, nor hard drive to operate - it scales to multiple chips and can support a cache of up to 1 terabyte of RAM Revolution is for new players, carefully crafted evolutions are for the Mass Yep Support for a 1TB cache, no hard drive, no current OS required, and the ability to pump data faster than any other platform on the planet should do the trick. Only thing left is to get the Itanium platform into a single 1RU box at sub $5,000. I doubt we will have to wait long for that. Long live the revolution Regards, Well the http tuning of string handling is a known factor of optimization, just study tomcat 3.2, 3.3 and Coyote 1.1 and you'll that it could still be optimized. BTW, if you post these benchmarks on the httpd-dev list, should I assume you'll give ASF your optimizedtuned algorythms ? Do you known that IBM does some nice optimization using FRCA on its Apache 2.0 implementation on iSeries ? A proof that Apache 2.0 is a great platform for such games ;)
Re[2]: the wheel of httpd-dev life is surely slowing down, solutions please
Read The Cathedral The Bazaar from ESR [2]. It provides a good insight into the social structures of the community: motivations, incentives, .. It's not always the money, you know ;) Market share? Who cares! Customers? Who cares! I want my name to be a three letter acronym everyone recognizes! RMS? ESR? How pity. Hunting for honor is just a short term motivation. What about the next hype? Will you jump on the new train? This is not what makes HTTP. This doesn't bring the market share. This doesn't bring the trust, customers have. It may lead into some good features - mostly maintained by others after some time. Sorry, this is your motivation. It's ok for you. But don't believe, this drives everybody. For myself, I joined the project, because I have been sad to tell people where to find the right thing within the documentation. I want a good software to be used. This is why I does not code. There are lots of programmers. But less documentation writers. I do not need my name written on the sky. Sure, honor is nice. But good teamwork is nicer. And THIS is the discussion about. How to improve the teamwork to get better results. Just my 2 cents Kess
Re: the wheel of httpd-dev life is surely slowing down, solutions please
On Tue, Nov 11, 2003 at 06:02:36AM -0700, Peter J. Cranstone wrote: So, anyone got any hard data that shows Apache 2.x serving pages factors faster than 1.x? Yes, plenty :) ftp.heanet.ie serves about 1 million requests, well over a terabyte of data per day and maintains an average of about 220Mbps for http. It's roughly 10 times less latent, and serves about 5 times more per second under 2.0 than 1.3, and I benchmarked that in the early 2.0 days. Performance under 2.x has improved since. If I could use sendfile (my hardware is still broken in IPv6) I'd see a much bigger increase in performance. 2.x knocks the socks off of 1.x. I've benchmarked many other transitions, and though the improvement is smaller for dynamic content I've never seen the numbers get worse. Of course this is all going from 1.x prefork to a 2.0 worker mpm. More importantly though; 2.x has IPv6 support. And whilst many people reading this mail may think IPv6 is an obscure requirement, many of us are in parts of internet where it's de facto. We couldn't even consider rolling out an app which didn't have reliable IPv6 support. Many people are finding themselves in such parts of the internet, at an increasing rate. As an asside; It's been my experience that most of industry doesn't rate application performance all that highly in evaluation criteria (mainly because buying beefier hardware is an easier solution). -- Colm MacCárthaighPublic Key: [EMAIL PROTECTED]
Re: the wheel of httpd-dev life is surely slowing down, solutions please
On Mon, Nov 10, 2003 at 05:14:35PM -0800, Stas Bekman wrote: 3). Contributions I don't have numbers to support my clause, but I have a strong feeling that nowadays we see a much smaller number of posts with contributions from non-developers More facetious than anything else, I'm going to hijack this part to make a small suggestion; give non-committers the option of not having their e-mail in the CHANGES/STATUS files, or at least of having them obfuscated in some fashion. The ammount of spam I get due to the httpd CHANGES file is simply unreal, most of it nonsense - with the forged from headers of other users of this list. That's about the only thing that ever makes me think twice about posting a patch/contrib. PS. This is not a complaint, it's my own silly fault for never remembering to ask someone about this. -- Colm MacCárthaighPublic Key: [EMAIL PROTECTED]
RE: the wheel of httpd-dev life is surely slowing down, solutions please
Well the http tuning of string handling is a known factor of optimization You're right - nothing new about optimizing string handling - just doing it BTW, if you post these benchmarks on the HTTPd-dev list, should I assume you'll give ASF your optimized tuned algorithms ? I wouldn't assume anything at this point - however if you remember correctly we did give the ASF mod_gzip (last time I checked even Google was compressing their output), however there will be an open source contribution at some point. Do you known that IBM does some nice optimization using FRCA on its Apache 2.0 implementation on iSeries Great - where are the benchmarks, as I said in my earlier post what's the differentiator between 1.x with 66% of the market and 2.x with 0% of the market? Remember your audience - it either has to make me money or save me money. If I'm going to implement 2.x then I should be able to see a return on the time I've invested either through a hardware/performance improvement or productivity. We all know why 2.x is struggling - the bar was set with 1.x and 2.x failed to move it more than about 1 inch. Peter -Original Message- From: Henri Gomez [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 11, 2003 8:25 AM To: [EMAIL PROTECTED] Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions please Peter J. Cranstone a écrit : There is no flame - just a couple of points and a request for data. If you want to improve something, you should provide solutions, not critics Certainly - early next year you will see them. Here are some current performance stats with some new technology we're working on. Configuration ToolElapsed Time (secs) Data Transfer Rate (KB/sec) Requests per Second Requests per MinutePerformance Gain Factor ApacheApache Bench38.735 882.92 2581.64 154,898 1.0 Cyclone Proxy Cache + ApacheApache Bench15.663 2387.79 6384.47 383,068 2.47 ApacheZeus Bench 39.961 855.83 2502.44 150,146.4 1.0 Squid + ApacheZeus Bench 28.910 1314.42 3459.01 207,540.6 1.38 Cyclone Proxy Cache + ApacheZeus Bench 15.176 2464.42 6589.35 395,361 2.63 Cyclone Proxy Cache (Tuned Parser) + ApacheZeus Bench 13.505 2769.34 7404.67 444,280.2 2.95 Cyclone Proxy Cache (4 Tuned Functions) + ApacheZeus Bench 13.006 2875.6 7688.76 461,325.6 3.07 These numbers were obtained using a single processor Itanium® 1.0Ghz (Madison) chip. By tuning certain HTTP string handling functions we have seen up to a factor 11 performance improvement. Our next benchmark is due by year end. Essentially we will be adding one more line for the stats above. The goal is very simple - transmit greater than 1 million requests in a single minute on a single processor Itanium 1Ghz machine. A factor 10 performance improvement. A single processor Deerfield Itanium® chip costs $744 - our solution doesn't require a current OS, nor hard drive to operate - it scales to multiple chips and can support a cache of up to 1 terabyte of RAM Revolution is for new players, carefully crafted evolutions are for the Mass Yep Support for a 1TB cache, no hard drive, no current OS required, and the ability to pump data faster than any other platform on the planet should do the trick. Only thing left is to get the Itanium® platform into a single 1RU box at sub $5,000. I doubt we will have to wait long for that. Long live the revolution Regards, Well the http tuning of string handling is a known factor of optimization, just study tomcat 3.2, 3.3 and Coyote 1.1 and you'll that it could still be optimized. BTW, if you post these benchmarks on the httpd-dev list, should I assume you'll give ASF your optimizedtuned algorythms ? Do you known that IBM does some nice optimization using FRCA on its Apache 2.0 implementation on iSeries ? A proof that Apache 2.0 is a great platform for such games ;)
RE: the wheel of httpd-dev life is surely slowing down, solutions please
Thanks for the stat - our environment supports throughputs 1Gbps on a single processor. Near linear scalability is expected with additional chips up to 256 CPU's. I agree with your comments on IPv6 - it's already here - might as well embrace the horror. Regards, Peter -Original Message- From: Colm MacCarthaigh,,, [mailto:[EMAIL PROTECTED] On Behalf Of Colm MacCarthaigh Sent: Tuesday, November 11, 2003 9:53 AM To: [EMAIL PROTECTED] Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions please On Tue, Nov 11, 2003 at 06:02:36AM -0700, Peter J. Cranstone wrote: So, anyone got any hard data that shows Apache 2.x serving pages factors faster than 1.x? Yes, plenty :) ftp.heanet.ie serves about 1 million requests, well over a terabyte of data per day and maintains an average of about 220Mbps for http. It's roughly 10 times less latent, and serves about 5 times more per second under 2.0 than 1.3, and I benchmarked that in the early 2.0 days. Performance under 2.x has improved since. If I could use sendfile (my hardware is still broken in IPv6) I'd see a much bigger increase in performance. 2.x knocks the socks off of 1.x. I've benchmarked many other transitions, and though the improvement is smaller for dynamic content I've never seen the numbers get worse. Of course this is all going from 1.x prefork to a 2.0 worker mpm. More importantly though; 2.x has IPv6 support. And whilst many people reading this mail may think IPv6 is an obscure requirement, many of us are in parts of internet where it's de facto. We couldn't even consider rolling out an app which didn't have reliable IPv6 support. Many people are finding themselves in such parts of the internet, at an increasing rate. As an asside; It's been my experience that most of industry doesn't rate application performance all that highly in evaluation criteria (mainly because buying beefier hardware is an easier solution). -- Colm MacCárthaighPublic Key: [EMAIL PROTECTED]
the wheel of httpd-dev life is surely slowing down, solutions please
I have several reasons to believe that the wheel of httpd-dev life is slowing down and something has to be done to get this wheel up to the speed like in the good old days. The following observation are listed in no particular order. I've also tried to offer suggestions how to resolve problems. == 1) Bugs searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=REOPENEDproduct=Apache+httpd-2.0 Suggestion: make bugzilla CC bug-reports to the dev list. Most developers won't just go and check the bugs database to see if there is something to fix, both because they don't have time and because there are too many things to fix. Posting the reports to the list raises a chance for the developer to have a free minute and may be to resolve the bug immediately or close it. == 2) Lack of Design In my personal opinion, the move to CTR from RTC had very bad implications on the design of httpd 2.1. 2a). Design of new on-going features (and changes/fixes) of the existing features is not discussed before it's put to the code. When it's committed it's usually too late to have any incentive to give a design feedback, after all it's already working and we are too busy with other things, so whatever. The worst part is that it's now easy to sneak in code which otherwise would never be accepted (and backport it to 2.0). I don't have any examples, but I think the danger is there. 2b). As a side-effect when someone asks a design question (e.g. me) it's not being answered because people tell: we are in the CRT mode, go ahead code it and commit it. But if the poster doesn't know what's the right design, if she did she won't have asked in first place. 2c). Future design. There is no clear roadmap of where do we go with Apache 2.1. People scratch their itches with new ideas which is cool, but if there was a plan to work on things, this may have invited new developers to scratch their itches. 2d). CRT seemed to come as a replacement for design discussions. It's very easy to observe from the traffic numbers: http://marc.theaimsgroup.com/?l=apache-httpd-dev 2003-11-01 - 2003-12-01 (118 messages) 2003-10-01 - 2003-11-01 (336 messages) 2003-09-01 - 2003-10-01 (275 messages) 2003-08-01 - 2003-09-01 (264 messages) 2003-07-01 - 2003-08-01 (321 messages) 2003-06-01 - 2003-07-01 (430 messages) 2003-05-01 - 2003-06-01 (450 messages) 2003-04-01 - 2003-05-01 (359 messages) 2003-03-01 - 2003-04-01 (696 messages) 2003-02-01 - 2003-03-01 (573 messages) 2003-01-01 - 2003-02-01 (546 messages) 2002-12-01 - 2003-01-01 (436 messages) 2002-11-01 - 2002-12-01 (538 messages) 2002-10-01 - 2002-11-01 (763 messages) 2002-09-01 - 2002-10-01 (894 messages) 2002-08-01 - 2002-09-01 (815 messages) 2002-07-01 - 2002-08-01 (721 messages) 2002-06-01 - 2002-07-01 (916 messages) 2002-05-01 - 2002-06-01 (1028 messages) 2002-04-01 - 2002-05-01 (1380 messages) 2002-03-01 - 2002-04-01 (1094 messages) 2002-02-01 - 2002-03-01 (1155 messages) Quiz: based on this input, tell which date CRT was introduced at. You don't need a fancy graph to see a clear decline in the last 1.5 years. Quite a few people suggested that this is due to the market decline. I won't doubt them, but it's quite possible that the market hasn't changed to the worst in the last 1.5 years (it is more likely that we should have seen this dip in 2001-2002, which I can't see from the numbers at the above URL). The cvs commit average rate hasn't changed much, which shows that development continues though mainly behind the scenes. http://marc.theaimsgroup.com/?l=apache-cvsr=1w=2 == 3). Contributions I don't have numbers to support my clause, but I have a strong feeling that nowadays we see a much smaller number of posts with contributions from non-developers, since most contributions are simply ignored and it's a known thing among Apache community that posting fixes and suggestions to httpd-dev list is usually a waste of time. (This is based on my personal experience and discussions with other developers who aren't httpd-core coders). I realize that I'm not entirely correct saying that, since some things are picked up, so I apologize to those folks who do try hard to pick those things. The obvious problem is that most of those things are unsexy, usually don't fit someones itch and they sometimes require a lot of communication overhead with the reporter/contributor just to understand what exactly is going on. Solutions: a. certain (most?) chunks of httpd lack owners. If httpd was to have clear owners of certain code bases then it'll be easy to take care of bug reports and contributions/patches. Even though httpd is collectively owned, it
Re: the wheel of httpd-dev life is surely slowing down, solutions please
--On Monday, November 10, 2003 17:14:35 -0800 Stas Bekman [EMAIL PROTECTED] wrote: Suggestion: make bugzilla CC bug-reports to the dev list. Most developers won't just go and check the bugs database to see if there Why can't they subscribe to [EMAIL PROTECTED] I don't think throwing all of the bugs on dev@ is going to help matters. I'd just add a procmail rule to toss them back into my [EMAIL PROTECTED] folder. In my personal opinion, the move to CTR from RTC had very bad implications on the design of httpd 2.1. CTR is still in effect in 2.1. Only RTC is in effect for 2.0, but, IMHO, that's to try to cover our butts that people don't break the server. I believe that RTC is very good for maintenance branches that shouldn't break anything (like our stated goal for 2.0), and CTR is good for development branches (2.1). 2a). Design of new on-going features (and changes/fixes) of the existing features is not discussed before it's put to the code. When it's committed it's usually too late to have any incentive to give a design feedback, after all it's already working and we are too busy with other things, so whatever. Do you have an example? The worst part is that it's now easy to sneak in code which otherwise would never be accepted (and backport it to 2.0). I don't have any examples, but I think the danger is there. I'm sorry, but when has this happened? 2b). As a side-effect when someone asks a design question (e.g. me) it's not being answered because people tell: we are in the CRT mode, go ahead code it and commit it. But if the poster doesn't know what's the right design, if she did she won't have asked in first place. We've been in CTR for the 'open' branches. You have commit access, so you have been entrusted with doing the right thing. If you do something wrong, someone will eventually chime in - as it is CTR. ;-) 2c). Future design. There is no clear roadmap of where do we go with Apache 2.1. People scratch their itches with new ideas which is cool, but if there was a plan to work on things, this may have invited new developers to scratch their itches. I don't believe there was ever a plan for 2.0 either. ;-) I think 2.0 GA happened because, well, um, a few people that would have stopped it were out of town. Yet, it happened, so at that point, we started to be committed to what 2.0 was. When we adopted the binary compatibility rules, we were fixed even further as to what 2.0 was. Until those points, I don't think any of us had a plan for 2.0... I think the binary compatibility rules were essential and, in retrospect, we should have had them in place before we went GA in 2.0. Yet, we know better now. 2.2 shouldn't have those same mistakes when we do that. 2d). CRT seemed to come as a replacement for design discussions. It's very easy to observe from the traffic numbers: Not sure why you think there's a correlation here. Statistics can be misleading. =) You don't need a fancy graph to see a clear decline in the last 1.5 years. Quite a few people suggested that this is due to the market decline. I won't doubt them, but it's quite possible that the market People change jobs or whatnot. Committers are only expected to contribute when they have time. It's not like we're all getting paid to work on this. I'm certainly not. I confess that I don't have the time I used to, but I'm also involved 'behind the scenes' within the ASF more than I was a few years ago. So, perhaps my time committment to the ASF is the same, but it's spread out amongst other projects/worthy causes. Working on the same thing for years on end can be a tad disconcerting. We all need a break at times. I don't have numbers to support my clause, but I have a strong feeling that nowadays we see a much smaller number of posts with contributions from non-developers, since most contributions are simply ignored and Perhaps. I believe Sander's suggested that we start a patch manager role (discussion at AC Hackathon!). A few other projects I'm involved with do this. It'd be goodness, I think. But, the problem is finding a volunteer willing to shepherd patches. a. certain (most?) chunks of httpd lack owners. If httpd was to have I really dislike this. The least maintained modules are those that had 'owners.' When those owners left, they didn't send a notice saying, 'we're leaving.' The code just rotted. I firmly believe that httpd's group ownership is the right model. Only recently have those modules that had 'owners' been cleaned up. (Yay for nd!) b. similar to the root rotation we have on the infrastructure, I suggest to have a voluntary weekly rotation of httpd-dev list champion +1 c. turn the dealing with contributions and bugs into a sexy thing. Everybody wants to feed their ego, there is no better way to make your ego happy then getting a positive feedback from a person you have helped to resolve a bug or handhold to provide a good patch for the new feature, they spent so
