Disclaimer : Not targetting any one individual I have a question to the people have lots of time to write such long mails and responses - why can't you instead spend that time to review patches and give feedback ?
It sure will improve the life of httpd-dev. Thanks -Madhu -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Thursday, November 13, 2003 10:47 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions please Hi all... I just have to jump in here since the topic is fascinating... ...and I think there's an opportunity here to review something that has contributed to the 'slow down' at httpd-dev which no one has seemed to grasp (yet). I will call it... "The Covalent Factor". If you look at what has REALLY happened in the past 3 years ( yes... going back that far since it's now 4 or 5 years since 2.0 became a real blib on the radar ) there's no question that there was this intense period of development and 'new' things were happening at a fast rate. As more and more developers got interested in getting 2.0 cranked out the (limited) resources all got eaten up in the 2.0 development cycle and 1.3 development virtually shut down. It was even 'officially' shut down long before 2.0 was ready ( 1.3 officially went into maintenance mode only ). So now you had lots of legacy developers ( albeit... lots of weekend-warriors, too, but WW's are the heartbeat of Open Source ) who knew 1.3 very well but were now totally put out to pasture. Very few of them 'came over' to the 2.0 development. The majority of the developers for 2.0 were the 'paid to play' kind that were either being paid to directly work on Apache ( I'll get to the Covalent factor in a second ) or, at least, were being paid to do something else but no one seemed to mind if they sat there at their real jobs all day and did Apache development. They might not admit to being directly paid to work on Apache but when that's what you are doing all day and still bringing home a paycheck then that's exactly what that is. The 'other' not-so-dedicated-but-certainly-interested developers felt 'shut out' of the 2.0 development cycle because it was obvious a lot of it was taking place 'off line' and nothing was being documented so they couldn't really get a good handle on what was going on in order to make a contribution. So they were mostly 'waiting in the wings' for a lot of the 2.0 DESIGN level decisions and concepts to become clear so they could 'jump in'. Well... we all know that it was a looong time before some of the 2.0 design concepts became 'clear' enough for a not-paid-to-work-on-apache person to 'jump in'. There were basically only 2 big changes to Apache 2.0 versus 1.x. 1. MPM. Get out of the 'prefork' only 'go fork thyself' design model. 2. ( Came later ) Add some I/O filtering and get rid of BUFF. Well, these are both fine goals to have but they are not for the faint-of-heart ( or within the grasp of most weekend-warriors ). That brings me to "The Covalent Factor". Is anyone going to deny that at a certain point in the 2.0 development cycle the PRIMARY driving force was a group of people that ALL worked for one company? ( Covalent ). That's certainly the way it looked. ...and these guys were crankin'... I mean they were MOTIVATED, because Covalent's entire corporate life revolves around Apache and they were actually high level deals going on with Compaq ( and other corporate entities? ) that all revolved around getting Apache 2.0 out the door. At one point one of the Covalent guys ( I believe it was that Randy Bloom fellow? ) was pretty much the ONLY person who had any idea how the new 'filtering' was even SUPPOSED to work. It was quite some time before he even finished thinking it through and it went through (too?) many re-works to even keep a good grasp on it. I remember the feeling for MONTHS was something like... "let's just see what Covalent comes up with and let them put their stamp of approval on it and then we'll see if we understand it." ...and that's pretty much how the entire SECOND primary goal of Apache 2.0 was achieved. Covalent just did it and expected everyone else to 'go along'... and they did. The point of all this is not to FLAME anyone or any corporate entity. No one can deny that Covalent has been crucial to the life and times ( and success ) of Apache itself and they should be proud of the contribution(s) they have made to OSS. Covalent was started by one of the original Apache guys and some of the best developers left around here still work for Covalent. I think William Rowe still does, for example, and he's still pretty much the only human being who seems to understand the Windows version of Apache... but the legacy flood of 'covalent' email addresses showing up at httpd-dev has virtually vanished so it's hard to tell what what with that anymore. There is nothing in the rules of Open Source that says you can't crank up a company that makes money off of Open Source software. Indeed... if you look at almost ALL of the major Open Source projects you will see the same sort of 'Covalent' deal going on whereby the people that know that OSS best have found a way to get paid to work on it. My only real point here ( and the way all this relates to the current thread ) is that maybe it's time to acknowledge that what is happening now is what will always happen to a major development project if you let too many of your eggs go into the same 'corporate' basket. If nothing else comes out of this thread maybe it should just be a resolve to never let any future OSS development cycles have 'too many eggs in the same basket'. Corporations are motivated by contracts, deadlines, promises and $$$$. Their 'motivations' and goals can co-exist with OSS goals but it can lead to situations where TOO MANY of the primary developers all end up working for the same company and OSS project goals and one-company's corporate goals can get too entangled to unravel. I think that pretty much already happened at httpd-dev right under everyone's noses. When Apache 2.0 was first released... it was a disaster. The person who called for the release 'out of nowhere' worked for Covalent... and within just a few hours he had the required 2 or 3 'yes' votes to initiate the release... and 2 of those 3 votes were also from Covalent employees. I think it can be safely said that 'Covalent' initiated/approved the release of Apache 2.0 and they did it so fast ( In just one day ) and without any chance for discussion that you have to wonder to this day whether they all just walked out of a meeting or something having 'decided' to do it because that's what fit their contract schedule(s). Someone or something was waiting for 2.0 to just be offically out of 'Beta' no matter WHAT shape it was actually in. DISCLAIMER: I don't know any of this for a fact. This is just the way it 'looked' from simply following (public) httpd-dev mail messages. I have no idea what really went/goes on at Covalent. I think even the Covalent developers who initiated the release were suprised at how 'not ready' it really was. Very little had even been tested and it was simply not even usuable for quite some time. There was a lot of major functionality that wasn't even tested for the FIRST time until AFTER the freakin' thing was officially released... and some (crucial) features weren't even close to working at all. This created a real feeling of disappoinment throughout the entire Apache community... developers and users alike. After all that hard work... it went out the door with a whimper and was actually nothing short of embarassing. The silence over that whole 'not-ready-for-prime-time' too-early release of Apache 2.0 was DEAFENING. You could have heard a pin drop at httpd-dev for quite a while. I think people were just in shock or too digusted to even say anything. Then the horror set in and people just got down to work solving the problems... but for a while there it looked like the httpd-dev list just came to grinding halt. I think you lost a lot people ( developers included ) right then and there. There was then another flurry of activity to get the ( now released ) Apache 2.0 into a state where you could even call it usable... and after that is when the whole deal seemed to just put the brakes on. I don't know if it was because of disgust over the whole way it went down, or development burnout, or just that one particular company that had an enormous amount of control ( too much ? ) over the product simply 'got what it wanted'... but it sure looks like a lot of people have just 'walked away' from Apache 2.0. I mean... where is Covalent now? Where are all those guys who had so much influence on what this Server has now become and who knew it so well? Have they all just vanished? Why aren't they helping to FINISH what they so enthusiastically STARTED? There are still messages floating by on httpd-dev even as we speak regarding the 'filtering' and it seems like you are hard-pressed to find anyone left at httpd-dev who REALLY understands it even now. You see a lot of "I think it works like this" and "maybe we forgot to do this" and "have to go and see how that works" responses but rarely anything concrete. It's weird. There are STILL some legacy 1.3 developers who don't have their heads around MPM and BUCKET BRIGADES well enough to 'come back' and join the party again. You might even consider re-opening the 1.3 tree for development again. I would. I never understood why it was 'closed' in the first place. I find it hard to believe that one of the most influenial OSS projects in the world wouldn't be able to develop a new version in the background and still keep the development of the current version alive. Covalent wasn't interested in seeing 1.3 advance but that shouldn't have made any difference. 1.3 development should have stayed 'open' to 'keep the fires burning'. Reality would also dicatate keeping the 1.3 development 'alive' for patches and improvements. Even now... almost 2 years after it was released... there are VERY few people even interested in Apache 2.0 and what it has to offer. I will bet you that some 80 percent of your 'customers' wouldn't have the faintest idea what MPM means or what replacing BUFF means. They just wouldn't get it. Everyone understands SPEED, of course, but despite a few postings in this regard on this thread itself there isn't much concrete information out there about whether Apache 2.0 really is faster. I will bet that if you just run it in good 'ole pre-fork and simply add a filter or two that it is either just the same as 1.3 or even SLOWER. Your high-end, high-dollar, high-transaction customers are, of course, going to understand what is different about Apache 2.0 and why they should give it a shot but that's still going to just be a fraction of your actual user-base. The only way, at this point, to get the majority of your customer base to switch to Apache 2.0 and help iron out all the remaining bugs is if some serious security issue arises with 1.3 and you guys REFUSE to add a patch to 1.3. That would FORCE people who don't even care about it to start using 2.0. Sounds a little nasty... but that's what you could do and, indeed, might be the ONLY way to crank up the usage on 2.0 and... as a consequence... the development and improvement of it. Go ahead and shut down even 'maintenance' mode on 1.3 ( No more patches of any kind, security included ) and just sit back and wait. When it comes... you will hardly have the bandwidth to handle the number of people that are 'finally' switching to 2.0. The reason things have 'slowed down' is because there isn't any 'jazz' at the moment. If no one is using 2.0 and discovering new bugs that need to be squashed and everyone seems happy with 1.3... then where's the problem? It's time to go lie on the beach and be proud of your success. You have 2 Servers that seem to work fine and no one is even asking for anything 'new' other than minor tweaks here and there. Life is good (??). It's like being a fireman but nothing is catching fire... ...but even most fireman don't take the job to sit around the station and eat pizza and watch soap operas... they joined up to put out fires. Software development is about problem solving. That's the 'jazz' that most people get out of it. It's that feeling that you got your head around something that most people would never understand and maybe noticed something that no one else has noticed that makes guys hang drywall for 5 days a week and then post patches to OSS projects on the weekend. It's fun. It makes you forget for a while that you actually do just hang drywall most of the time. So when any development cycle 'slows down' the first question to ask is... 1. Where are the problems that need to be solved? I don't think most people perceive the 'problems' as intently anymore. Apache 2.0 is out there and has been freely available for quite some time to anyone who is interested in using it ( apparently not many ) and anyone who is actually using 2.0 doesn't seem to be complaining about it much... and Apache 1.3 development is permanently dead. So where's the fire? Later... Kevin
