hep
Hep pease unscripe !!
El-Kabong HTML Parser -- has m0v3d
Aight, so since this has moved elsewhere, I thought I'd tell the people who may have initially been interested in the code. You can now grab it here: http://ekhtml.sf.net Should be handy for creating Apache filters that want to mangle content before shipping it to the browser. -- Jon
E-Kabong resolution: Re: acceptance of El-Kabong into APR
Jeff, I cc'd the dev@ lists, since the original proposition was made there, and the public following the discussion should know the resolution. My comments at the end. On Wed, Sep 11, 2002 at 07:45:21PM +, Jeff Trawick wrote: Hi Jon, As you well know, it has taken a while to reach a conclusion on the donation of the El-Kabong codebase. Even after determining that it was a contribution worthy of a home in the ASF, it was hard to determine where exactly it should be placed. Eventually, the majority of the APR Project Management Committee (PMC) voted for it to be placed in apr-util/html. Another major concern was with granting you commit privileges to the code once it was transferred to an ASF CVS repository. Unfortunately, we do not feel comfortable with granting you commit privileges at the present time. We are happy for you to submit patches and we will make a special effort to handle them in an expeditious manner. We hope that we will eventually be able to grant you commit privileges, but before that happens we need a stronger indication of your respect for the community and its processes. Concerns were expressed about your ultimatum for accepting El-Kabong within a certain timeframe. We feel that any time required initially for considering the donation and where to place it were important for its long-term success as part of the ASF, and that your ultimatum was another indication of a lack of respect or understanding for the community. We need to have a more constructive relationship with you over a period of time before we can grant you commit access. While we believe that the El-Kabong codebase is a valuable contribution that we would like to pick up, we also recognize the possibility that you may have a decreased incentive to work on it further if you are initially denied commit access to the repository when it moves to the ASF. If this is the case, we understand that it may be best for the codebase to live elsewhere if it would be harmed by a lack of contributions from its original author, and we look to you for further guidance on this issue. Respectfully, the APR PMC --- Jeff Trawick | [EMAIL PROTECTED] I've decided to host the project elsewhere. It would be extremely frustrating to require you, the ASF, to review code I'm patching to code I initially wrote, given that I know that code better than anyone. Moreover, I would have no review privileges for code being committed to e-k, which seems like very irresponsible code management. As originally stated, I would be the initial maintainer of the code. Since this is not the case, it seems that the ASF has accepted part of the deal, and discarded the rest. I'd ask that you not include e-k, since we haven't come to a satisfactory conclusion. -- Jon
Re: E-Kabong resolution: Re: acceptance of El-Kabong into APR
On Wed, Sep 11, 2002 at 04:25:00PM -0700, Greg Stein wrote: On Wed, Sep 11, 2002 at 01:55:35PM -0700, Jon Travis wrote: Snip I've decided to host the project elsewhere. It would be extremely frustrating to require you, the ASF, to review code I'm patching to code I initially wrote, given that I know that code better than anyone. Moreover, I would have no review privileges for code being committed to e-k, which seems like very irresponsible code management. As Jeff indicated, we had hoped that you would want to work *with* us on moving E-K forward. Your participation on the APR development list, reviewing of commits, suggesting new directions, and providing patches to improve the system are/were quite welcome. Your comment about host the project elsewhere is interesting. We never viewed this as a way for you to host code at the ASF. Instead, this was about two things: 1) a donation of code from a company (Covalent) 2) providing an individual (Jon Travis) with commit access to apr-util #1 is correct. #2 is not -- rather it was about Jon Travis having maintainer privileges to wherever it ended up (not necessarily apr-util). I've repeatedly stated that moving stuff to a different project was fine. Read over the emails -- it's all there in whatever colors your mail reader renders it.. ;-) The two are separable items with different issues and requirements. As originally stated, I would be the initial maintainer of the code. Since this is not the case, it seems that the ASF has accepted part of the deal, and discarded the rest. Again, the two issues are separate. It is unfortunate that the ASF did not communicate this critical piece of information to you, though. A lot of headache may have been avoided if we simply said, A code donation does not imply commit access. Each will be evaluated on their own merits. That's fine -- but in this case I said that I would be the initial maintainer. If you didn't want the one, you shouldn't have taken the other. I'd ask that you not include e-k, since we haven't come to a satisfactory conclusion. We have already executed the paperwork with Covalent for the software grant. This I know -- I filled it all out.. ;-) I don't have a particular read on what the PMC is going to choose to do with the code -- whether it will go into apr-util/html/, or whether will be skipped. We do have the choice, but I'm not sure which will be chosen. You certainly do, and I hope you act responsibly in your decision. -- Jon
Re: El-Kabong -- HTML Parser
On Mon, Sep 09, 2002 at 11:21:04PM -0700, Greg Stein wrote: On Mon, Sep 09, 2002 at 01:33:25PM -0700, Jon Travis wrote: Ok, since I'm not seeing any activity towards getting this integrated, I'd like to set a deadline. This would help me out, since it gives direction as to where the project can go, as well as the ASF since political discussion shouldn't weigh down the process. It will just save us all a lot of time energy. Anyway, I'd like to give an additional week to the ASF to deal with the code. Next Monday, if it hasn't been decided I'll look into other options. Jon -- The HTTP Server Project, the APR Project, and the ASF will not abide by your threats of taking the code elsewhere. We will not be subject to coercive behavior, nor do we look kindly upon the attempt. The ASF is about working together. Your message certainly does not mesh with our concepts of community and respect. The ASF is apparently not about working together, since I (and everyone else who is not on the PMC list) have been entirely left out of all this conversation which is going on behind closed doors. I am not trying to coerce anyone -- I'm merely trying to put a limit on the maximum amount of time that this will be debated. That is a perfectly valid, and responsible thing to do. Frankly, there is no need or driving force for us to accept code donations. That means it isn't possible to hold this over our heads. We can easily choose to ignore the whole thing, with no real loss to our fundamental ideals and to our communities. It seems as though that is exactly what you've done -- ignored it. I am constantly probing for information as to where this stands, both via these lists and asking on the #apr IRC channel. You're interpretation if holding this over your head is fairly outrageous, since Apache exists perfectly fine without this contribution. As of this moment, I'm going to recommend that the ASF will work directly with Covalent management regarding the donation. The other members involved may certainly choose to continue working through you [to Covalent], as I do not speak for them, but I seriously doubt that will be the case. Your behavior does not bode well for your continued involvement in the process. That's quite the attack (and one which fits in well with your concepts of community and respect?) . All decisions which have been made about this project (Open sourcing it, setting a deadline, etc.) have been dictated by management at Covalent. If you want to persuade your management to publish the code through a different owner, or under a different license, then please feel free. If Covalent management chooses to not donate the code to the ASF, then so be it -- that is their choice. But take your coercive tactics elsewhere. Regards, Greg Stein Chairman, Apache Software Foundation These are not coercive tactics. These are processes which are beneficial to both the ASF and Covalent. I cannot continually monitor the progress of this project for eternity. I'm astonished that this deadline email has caused such a response. This sets an extremely bad precedent for other companies (or anyone for that matter) who wants to contribute to the ASF. Personally (Covalent hat off), it's a bummer that this is your response to the donation. I was the one who originally proposed it to management, they agreed to it, and now I've gotten involved in all kinds of politics and inflamatory emails. That's a long way from being excited about contritributing to the ASF, and sadly seems like more trouble than it's worth. -- Jon
Re: El-Kabong -- HTML Parser
On Tue, Sep 10, 2002 at 01:57:06AM -0700, Rasmus Lerdorf wrote: The ASF is apparently not about working together, since I (and everyone else who is not on the PMC list) have been entirely left out of all this conversation which is going on behind closed doors. Which closed doors are those? There has been discussion on the dev list and on the board list. Both of which are public lists that you can subscribe to. All I know of is the PMC list (which is private), but discussion on board (which is also private) is news to me. http://www.apache.org/foundation/mailinglists.html#foundation-board -- Jon
Re: El-Kabong -- HTML Parser
On Tue, Sep 10, 2002 at 09:47:01AM -0700, Scott Hess wrote: [I am not an Apache contributor, merely a lurker, but...] On Tue, 10 Sep 2002, Jon Travis wrote: These are not coercive tactics. These are processes which are beneficial to both the ASF and Covalent. I cannot continually monitor the progress of this project for eternity. I'm astonished that this deadline email has caused such a response. This sets an extremely bad precedent for other companies (or anyone for that matter) who wants to contribute to the ASF. Personally (Covalent hat off), it's a bummer that this is your response to the donation. I was the one who originally proposed it to management, they agreed to it, and now I've gotten involved in all kinds of politics and inflamatory emails. That's a long way from being excited about contritributing to the ASF, and sadly seems like more trouble than it's worth. As I said earlier: if all you want is to contribute the code, put a compatible open source license on it and put it on a publicly accessable website, somewhere. We decided to give the ASF the first shot at it if they wanted the project. It seemed to fit in well with things they did, and we have roots in the ASF. From following the thread, I get the feeling you don't want to contribute it, you want someone to take ownership of it. A couple points: Untrue. If you look at my very first email on this issue (the initial proposal), I site that I will be the initial maintainer. I certainly have the most experience with the code, and certainly some amount of passion for it, regardless of the simplicity. 1) Everyone here has a real-life job. 2) Many of those jobs don't involve Apache directly. 3) Anyone who's writing code has their own pet projects they want done. 4) Anyone without a pet project has a choice of dozens/hundreds of abandoned/unmaintained projects to work on. 5) Integration work is hard work. If you really want the ASF to pull this project into the Apache core, your best bet is to volunteer to integrate it and write some example code. After all, you're the one with the code, you're the one who wants to contribute it to the community. This is the point they are debating. There is no consensus that it will be integrated. It either would be, or it would be a seperate project, and that has not yet been decided. They have already agreed that the code is good (I've already sent it to people to review ). I'm more than happy to integrate it wherever they see it fit. As I've said in other emails (to this list and privately to members) I would definitely be involved. -- Jon
Re: El-Kabong -- HTML Parser
Time for another ping. It's been 2 weeks. Any word? -- Jon On Mon, Aug 26, 2002 at 08:32:16PM -0700, Jon Travis wrote: Hi all... Jon Travis here... Covalent has written a pretty keen HTML parser (called el-kabong) which we'd like to offer to the ASF for inclusion in APR-util (or whichever other umbrella it fits under.) It's faster than anything I can find, provides a SAX stylee interface, uses APR for most of its operations (hash tables, etc.), and has a pretty nice testsuite. We use it in our code to re-write HTML on the fly. I would be the initial maintainer of the code. Please voice any interest, thanks. -- Jon
Re: El-Kabong -- HTML Parser
Ok, since I'm not seeing any activity towards getting this integrated, I'd like to set a deadline. This would help me out, since it gives direction as to where the project can go, as well as the ASF since political discussion shouldn't weigh down the process. It will just save us all a lot of time energy. Anyway, I'd like to give an additional week to the ASF to deal with the code. Next Monday, if it hasn't been decided I'll look into other options. -- Jon On Mon, Sep 09, 2002 at 10:36:21AM -0700, Jon Travis wrote: Time for another ping. It's been 2 weeks. Any word? -- Jon On Mon, Aug 26, 2002 at 08:32:16PM -0700, Jon Travis wrote: Hi all... Jon Travis here... Covalent has written a pretty keen HTML parser (called el-kabong) which we'd like to offer to the ASF for inclusion in APR-util (or whichever other umbrella it fits under.) It's faster than anything I can find, provides a SAX stylee interface, uses APR for most of its operations (hash tables, etc.), and has a pretty nice testsuite. We use it in our code to re-write HTML on the fly. I would be the initial maintainer of the code. Please voice any interest, thanks. -- Jon
Re: El-Kabong -- HTML Parser
It's possible that if it goes elsewhere that it would be under a different license. That's of course contingent on the decision from the ASF. -- Jon On Mon, Sep 09, 2002 at 01:50:18PM -0700, Scott Hess wrote: I'm not sure I understand what your goal is, here. The discussion seems to be +1 for including your parser somewhere in some Apache project in the future, there's just no clear concensus on where. Is there any reason you can't just release your project under the ASF license and be done with it? Later, scott On Mon, 9 Sep 2002, Jon Travis wrote: Ok, since I'm not seeing any activity towards getting this integrated, I'd like to set a deadline. This would help me out, since it gives direction as to where the project can go, as well as the ASF since political discussion shouldn't weigh down the process. It will just save us all a lot of time energy. Anyway, I'd like to give an additional week to the ASF to deal with the code. Next Monday, if it hasn't been decided I'll look into other options. -- Jon On Mon, Sep 09, 2002 at 10:36:21AM -0700, Jon Travis wrote: Time for another ping. It's been 2 weeks. Any word? -- Jon On Mon, Aug 26, 2002 at 08:32:16PM -0700, Jon Travis wrote: Hi all... Jon Travis here... Covalent has written a pretty keen HTML parser (called el-kabong) which we'd like to offer to the ASF for inclusion in APR-util (or whichever other umbrella it fits under.) It's faster than anything I can find, provides a SAX stylee interface, uses APR for most of its operations (hash tables, etc.), and has a pretty nice testsuite. We use it in our code to re-write HTML on the fly. I would be the initial maintainer of the code. Please voice any interest, thanks. -- Jon
Re: El-Kabong -- HTML Parser
My comments inline: On Tue, Sep 03, 2002 at 02:53:03PM -0400, [EMAIL PROTECTED] wrote: There are currently two possible avenues. 1) The code goes into apr-util. 2) The code goes into a sandbox project. The APR option is faster, but there is some misgivings about whether it belongs in apr-util. The vote was done, and it seems to be accepted, but Greg was keeping tally, so I don't have the exact numbers about where it would go. I _think_, and I could be wrong, that it would be put in apr-util/html as a separate piece of apr-util. The second option will take a bit longer, because the sandbox project will need to be created first. I have tried to answer without letting any of my personal opinions show in the message, because that has caused some problems before. The real question now, is given those two options, which would you prefer. Not saying that your preference is the only factor i the decision, but it should be taken into account. Either one is fine to me. Integrating the code into apr-util is probably an easier setup, but will require more work to adapt to the build system and change the symbols (and of course I'm quite liking the name 'el-kabong' ;-)). There are also some people questioning why we are moving so quickly on this. The general feeling is that we should find the best fit before taking the code. If you are in a rush, then that would change things, but the understanding was just that you wanted to be kept in the loop about what is happening. Keep pinging, but the conversation is on-going, and very active, so there is little chance that it won't happen. It is really just a matter of time now. Ryan I'm not in a rush, I just like to know where things stand. Since this discussion is seemingly happening off-list, I can't differentiate between no discussion or a heated one. I'd prefer this to be on-list, as I think it does affect the users of APR, and it would allow me to monitor the progress here. -- Jon
Re: El-Kabong -- HTML Parser
Any word on this? -- Jon On Mon, Aug 26, 2002 at 08:32:16PM -0700, Jon Travis wrote: Hi all... Jon Travis here... Covalent has written a pretty keen HTML parser (called el-kabong) which we'd like to offer to the ASF for inclusion in APR-util (or whichever other umbrella it fits under.) It's faster than anything I can find, provides a SAX stylee interface, uses APR for most of its operations (hash tables, etc.), and has a pretty nice testsuite. We use it in our code to re-write HTML on the fly. I would be the initial maintainer of the code. Please voice any interest, thanks. -- Jon
Re: El-Kabong -- HTML Parser
On Thu, Aug 29, 2002 at 06:42:39PM +0200, Dirk-Willem van Gulik wrote: On Thu, 29 Aug 2002, Jon Travis wrote: Any word on this? These things take time... and it pays off to do them well. There is absolutely no rush. Just wanted a word. More often than not, when something stops being discussed on the list, it drops off the face of the earth. I should know, I have many patches who've taken such a fall.. ;-) -- Jon
Re: El-Kabong -- HTML Parser
On Thu, Aug 29, 2002 at 11:29:24AM -0700, Aaron Bannert wrote: On Thu, Aug 29, 2002 at 02:24:28PM -0400, Ryan Bloom wrote: +1 from me, I prefer APR actually. I am really uncomfortable with this going under the APR project. As things stand right now, it just doesn't fit with what we have stated our goals to be. If you want to change our stated goals, then go ahead and do it. Just committing code that doesn't fit with our goals isn't the way to do that. (I will defer answering this for an apr-only discussion.) I will make one exception to that statement. If it lands inside of APR-util, under the XML directory, and it is made to work with the XML parser, I can accept that landing spot. As it fits in closer with our goals (I think). Jim, I can't decide if this is what you meant or not. I'm +1 on integrating it into our XML stuff. I consider it to be equivalent to apr-util, so either we put it inside apr-util, or we create a new APR subproject or sub-library for it. I'm not keen on integrating it into the APR XML layer for a few reasons: 1 - APR's XML is not SAX-stylee. El-Kabong is. That isn't to say that E-K couldn't get a full object model interface, but it doesn't have it now. 2 - XML and HTML, while related, have several large differences which won't make a nice API (IMO). 3 - El-Kabong is quite speedy, and throwing another layer of indirection on top of it isn't particularly appealing. -- Jon
Re: El-Kabong -- HTML Parser
On Thu, Aug 29, 2002 at 02:30:35PM -0700, Sander van Zoest wrote: On Thu, 29 Aug 2002, Jon Travis wrote: On Thu, Aug 29, 2002 at 11:29:24AM -0700, Aaron Bannert wrote: On Thu, Aug 29, 2002 at 02:24:28PM -0400, Ryan Bloom wrote: I will make one exception to that statement. If it lands inside of APR-util, under the XML directory, and it is made to work with the XML parser, I can accept that landing spot. As it fits in closer with our goals (I think). Jim, I can't decide if this is what you meant or not. I'm +1 on integrating it into our XML stuff. I consider it to be equivalent to apr-util, so either we put it inside apr-util, or we create a new APR subproject or sub-library for it. I'm not keen on integrating it into the APR XML layer for a few reasons: 1 - APR's XML is not SAX-stylee. El-Kabong is. That isn't to say that E-K couldn't get a full object model interface, but it doesn't have it now. Expat is a stream based parsers that is pretty similar to SAX2. It isn't a DOM xml parser. What does this have to do with anything? Expat is uninvolved here. I was talking about the APR XML API, which may-or-may-not wrap Expat. That API is DOM style. -- Jon
Re: El-Kabong -- HTML Parser
On Tue, Aug 27, 2002 at 09:47:58AM -0700, daniel wrote: A couple of notes on the parser: - It is pretty lightweight and self contained - This HTML parser can be used for a multitude of applications, in Apache 2.0 filter modules. The filter processes content generated by Apache or proxied content and can rewrite URLs embedded in HTML pages: a) URL rewriting on the fly for cookieless tracking and single sign-on b) Allow ProxyPassReverse to modify not only HTTP headers but look inside proxied content and rewrite hardcoded URLs c) Strip banners or malicious javascript before it reaches the client Those are some possibilities that having a fast, lightweight parser allows. Jon, maybe you can post the source somewhere so people can have a look and play with it Daniel I can't publicly post the source under the ASF license until it has been accepted (which is a chicken egg issue). I can, however, distribute to individuals on a restricted basis for evaluation for acceptance. -- Jon
Re: El-Kabong -- HTML Parser
Well, if people are agreeing to this, can we get someone involved in the HTTPD project (non-Covalent affiliated) to review and approve/decline? Volunteers? -- Jon On Tue, Aug 27, 2002 at 01:22:03PM -0400, Jim Jagielski wrote: I didn't mean in the same tree at all! :) But, as you said, a subproj under HTTPD Jon Travis wrote: I personally think it belongs as some kind of sub-project to httpd, but not in the same tree. -- Jon On Tue, Aug 27, 2002 at 12:43:17PM -0400, Jim Jagielski wrote: Aaron Bannert wrote: On Tue, Aug 27, 2002 at 11:02:47AM -0400, Ryan Bloom wrote: I would prefer that this became it's own project either under the httpd project or the APR project. I don't believe that it should be a part of the APR-util library however. The functionality provided by el-kabong seems like a good fit within APR. I think it would be fine if el-kabong became a new subproject of APR. Personally, I think it's very cool that E-K is being folded into the ASF. I think the httpd project is a better fit however, simply because APR should be protocol ignorant. -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: Apache Worm
On Sun, Jun 30, 2002 at 11:24:30PM +0200, [EMAIL PROTECTED] wrote: On Sun, 30 Jun 2002, Pier Fumagalli wrote: Rasmus Lerdorf [EMAIL PROTECTED] wrote: I assume everyone has seen this? http://dammit.lt/apache-worm/ Me and Fede are running through the decompiled assembly code right now... Will let you know what we find out (it looks kinda odd from the look of it). I found several. Source under private cover. You mean apache-worm.c, posted on that URL 5 lines up from your reply? That's been out there for a while. -- Jon
Re: libexpat
On Mon, May 20, 2002 at 09:54:16PM -0500, William A. Rowe, Jr. wrote: At 09:43 PM 5/20/2002, you wrote: On Mon, 20 May 2002, William A. Rowe, Jr. wrote: Context? httpd links in expat, perl extension links against a different version of expat. both have the same symbol names, and they are not binary compatible. perl extension resolves symbols to the httpd version. kaboom. its been an issue for years with 1.3, you'll find plenty in the modperl archives on it. Can we somehow draw one from the two? In all fairness, I'm surprized this is much of an issue on Win32... we don't have a flat namespace. If you dig into libhttpd and libaprutil, you should discover no expat symbols exported (the depends utility should let you see any exports/used symbols.) Of course that's win32 (or OS X) ... on Unix I can imagine this is hellish, no matter how we try solving it. This same stuff comes up every few months. Python modules had the same issue with PCRE stuff (surprised mod_perl doesn't have that issue). -- Jon
Pseudo-bug with PCRE Apache
Apache uses PCRE for a few regexp operations, but external modules which try to use other symbols (such as get_substring) el-kabong because Apache isn't importing it. So, should: a) all these modules be linking in all of the PCRE symbols into themselves or b) Apache include PCRE symbols as part of the exports.c 'hack' variables. I personally prefer 'b'. -- Jon
Re: cvs commit: httpd-2.0 CHANGES
On Mon, Nov 12, 2001 at 11:16:52PM -0800, Ryan Bloom wrote: On Monday 12 November 2001 11:06 pm, [EMAIL PROTECTED] wrote: jwoolley01/11/12 23:06:42 Modified:.CHANGES Log: I was originally just going to s/commans/commas/, and then I got carried away and rewrote half the paragraph. sigh Hey, I never said I could write well. I rely on MS Word to fix my grammer and ^^^ spelling. I just have to get the concepts on paper usually. :-) | | Ryan | | This email apparently wasn't run through MS Word? ;-) -+ -- Jon
Re: More Dos -- Large GETs
I'm checking my httpd.conf file to see if there is something weird there. I am indeed sending a valid GET with a huge content-length. -- Jon On Tue, Oct 30, 2001 at 08:15:44PM -0500, Cliff Woolley wrote: On Tue, 30 Oct 2001, Cliff Woolley wrote: O... I think I see the problem. In your test, you actually did a real HTTP request and then had a really big request body. ... Is that right, Jon? I just reread the original message and I see that's not what Jon was doing. But still, what happens if you do this? I'll have to go try that --Cliff -- Cliff Woolley [EMAIL PROTECTED] Charlottesville, VA
Re: More Dos -- Large GETs
On Wed, Oct 31, 2001 at 10:43:12AM -0600, William A. Rowe, Jr. wrote: From: Jon Travis [EMAIL PROTECTED] Sent: Wednesday, October 31, 2001 10:18 AM I'm checking my httpd.conf file to see if there is something weird there. I am indeed sending a valid GET with a huge content-length. To which handler, may we ask? That might make a difference, thought it should not. Just to '/'. -- Jon
Re: More Dos -- Large GETs
Ok, I've tracked it a bit further. This may be one of those 'Doctor it hurts when I do this' types of things, but I was using an old 2.0 config file that I had, and that was causing this. Anyway, I gradually picked off lines from my config file vs. the distributed one, and it looks like if you comment out the following 2 lines: AddHandler type-map var DirectoryIndex index.html index.html.var Within the Directory for the htdocs, then the memory will go through the roof with the method I described. -- Jon On Wed, Oct 31, 2001 at 10:12:47AM -0500, Greg Ames wrote: Jon Travis wrote: Nope. I just allocated 1MB of 'x's and sent that buffer a couple hundred times. It was the httpd process which was growing, not my test program. This was with Apache 2.0 HEAD, BTW, and 100% reproducable for me. I think you have my fix to modules/http/http_request.c, right? If not, ap_get_client_block can wreak havoc when a request body exists and we do an internal redirect, such as when the GET is to a directory. Also, do you send an empty line before the 'x's, or could this be the problem Aaron noticed? Greg
Re: More Dos -- Large GETs
On Tue, Oct 30, 2001 at 03:46:14PM -0500, Jeff Trawick wrote: Jon Travis [EMAIL PROTECTED] writes: It's possible to make Apache eat up all available memory on a system by sending a GET with a large Content-Length (like several hundred MB), and then sending that content. This is as of HEAD about 5 minutes ago. Maybe the problem is your client implementation? You didn't by any chance get a mongo buffer to hold the request body did you? I just sent a GET w/ 500,000,000-byte body and didn't suffer. strace showed that server process was pulling in 8K at a time... lots of CPU between client and server but no swap fest. Nope. I just allocated 1MB of 'x's and sent that buffer a couple hundred times. It was the httpd process which was growing, not my test program. This was with Apache 2.0 HEAD, BTW, and 100% reproducable for me. -- Jon
Re: DoS on POSTS
Like I said in my follow up post to my original, you don't even need to post the data to actually have this occur. I telneted to the server, and let it sit there for like 47 minutes before I killed it. I never had it time out. -- Jon On Fri, Oct 26, 2001 at 11:51:59AM -0700, Ryan Bloom wrote: On Thursday 25 October 2001 08:52 pm, Ryan Bloom wrote: It seems that there is a possibility for DoS on Apache servers when doing a POST. On search.apache.org, I can send the following request: PUT / HTTP/1.1 Host: search.apache.org:80 Content-Length: 1000 newline here And just let it sit there forever. search.apache.org is running 2.0.24, and I'm running out of CVS and seeing the same behaviour. Seems bogus to me. Well, after a few weeks of meaning to look into this, I finally have. Jon, you are 100% correct that this does happen. The problem is the handle_map_file handler. I have begun to track it down, but what is happening, is that the first request fails after the timeout is hit. The error page is requested, and that gets sent back to the ap_internal_redirect, but the content-length is still set, so the second request is hosed. Then we end up in an endless loop. I haven't really looked at how to fix this yet, and I have to write a part of my book tonight, but the first step is identifying the problem. This goes away if you remove all of the .var files from the config file BTW. I would suggest that if we don't fix ASAP, those lines should be removed from the apache.org site, and this MUST be fixed before we release the next beta. Had more time to look at this. It looks like we actually will timeout given enough time, but by default that time limit is like 10 minutes. I think this can be fixed by setting the content-length to 0 when we go to serve error pages. I am attempting this now-ish. Ryan __ Ryan Bloom[EMAIL PROTECTED] Covalent Technologies [EMAIL PROTECTED] --
DoS on POSTS
It seems that there is a possibility for DoS on Apache servers when doing a POST. On search.apache.org, I can send the following request: PUT / HTTP/1.1 Host: search.apache.org:80 Content-Length: 1000 newline here And just let it sit there forever. search.apache.org is running 2.0.24, and I'm running out of CVS and seeing the same behaviour. Seems bogus to me. -- Jon
Re: New post-log-transaction hook?
On Wed, Sep 19, 2001 at 12:16:24PM -0700, Ryan Bloom wrote:
On Wednesday 19 September 2001 11:37 am, William A. Rowe, Jr. wrote:
From: Greg Stein [EMAIL PROTECTED]
Sent: Wednesday, September 19, 2001 1:26 PM
On Wed, Sep 19, 2001 at 01:52:12PM -0400, Rodent of Unusual Size wrote:
Greg Stein wrote:
It isn't a bug. Cleanups are for just wrapping things up,
not doing work.
If that's the authoritative answer, then we need to provide
a supported way for 'doing work' at cleanup time.
You might not even be able to open and use that file if
your pool is n the process of being destroyed.
That sounds like utter tripe. If you can't depend on the
pool lasting at least until your cleanup routine ends,
then the whole cleanup mechanism is seriously borked. AFAIK
it isn't, so I think the above assertion *is*.
The problem is cross-dependency between the cleanup actions. One can
destroy something another cleanup needs. If you restrict their actions to
simple things, then the cross-dependencies are (hopefully!) removed.
Really? No. Cleanups are run as a LIFO stack. Anything that existed when
something was added to the pool must exist when that something is removed
from the pool.
IMHO, we need to make subpool scrubbing an actual LIFO cleanup as well, so
that will also be true of subpools.
Considering how we use pools for dynamic libraries and the rest, it's
absolutely vital that they are unspun from the pool in LIFO order of their
creation.
I agree with Bill. Having reviewed the code quite deeply yesterday, pool
cleanups follow a very clean rule, and registering a cleanup from within
a cleanup will always work if done correctly. If you have data in a pool
BZzzzt. The attached code registers a cleanup from within a cleanup, and
does so 'correctly'. See the program attached at the bottom, which behaves
incorrectly. It is simple code, but not knowing that a given
function registers a cleanup can cause major problems (leaking
file descriptors, etc. eventually). The file should contain 'Cleanup',
because the cleanup of the file should flush the buffer -- that
cleanup is never run, though.
when the cleanup is registered, it is gauranteed to be there when the cleanup
is run.
Anything else is completely broken.
#include apr.h
#include apr_file_io.h
static apr_status_t my_cleanup(void *cbdata){
apr_pool_t *p = cbdata;
apr_file_t *file;
apr_file_open(file, /tmp/bonk,
APR_WRITE | APR_CREATE | APR_TRUNCATE | APR_BUFFERED,
APR_OS_DEFAULT, p);
apr_file_printf(file, Cleanup);
return APR_SUCCESS;
}
int main(int argc, char *argv[]){
apr_pool_t *pool;
apr_initialize();
apr_pool_create(pool, NULL);
apr_pool_cleanup_register(pool, pool, my_cleanup, NULL);
apr_pool_destroy(pool);
apr_terminate();
return 0;
}
Re: New post-log-transaction hook?
On Tue, Sep 18, 2001 at 02:20:35PM -0500, William A. Rowe, Jr. wrote: From: Ryan Bloom [EMAIL PROTECTED] Sent: Tuesday, September 18, 2001 11:44 AM On Tuesday 18 September 2001 08:17 am, William A. Rowe, Jr. wrote: Why not let the MPM register the lingerclose with APR_HOOK_MIDDLE in the post_connection hook? That way, if Jon's (or any other author's) intent is to work before the lingering close, then it can be APR_HOOK_FIRST. Otherwise register it APR_HOOK_LAST. It shouldn't be a hook. This should just be done with a pool cleanup. Hooks aren't the answer to every problem in the server. Doing something after a specific action, like the close of the connection should be done by registering a pool cleanup. Fix the bug that you can't register a cleanup within a cleanup, and Jon's problem goes away completely, because he can use the cleanup that he is already using. The pool cleanup has one disadvantage (assuming the register cleanup within cleanup bug is fixed), the order of cleanups is a strict LIFO. There _may_ be an advantage to an orderable hook. At this point I agree, fix the register cleanup in cleanup bug, let Jon experiment with that solution, and then argue the merits for a new hook. I've got a workaround for that right now (which seems to work fine). I create a new pool within my cleanup, and destroy it before I exit. -- Jon
New post-log-transaction hook?
I've got a bit of code that needs to run after a connection to a client has been closed. Right now I can (kind of) spoof this by setting the keepalive for the client to 0, and registering a cleanup on the request_req pool. Unfortunately the code in there is somewhat bulky, so any subsequent cleanups that it registers will never get called (apparently registering a cleanup within a cleanup no workie). I'd like to propose a new hook which gets run after connection to the client has been closed. (in my case, I run some cleanup code which can take a while, and would like the client to be on its way). Any thoughts? -- Jon
Re: New post-log-transaction hook?
On Mon, Sep 17, 2001 at 07:01:21PM -0400, Cliff Woolley wrote: On Mon, 17 Sep 2001, Jon Travis wrote: I've got a bit of code that needs to run after a connection to a client has been closed. Right now I can (kind of) spoof this by setting the keepalive for the client to 0, and registering a cleanup on the request_req pool. Unfortunately the code in there is somewhat bulky, so any subsequent cleanups that it registers will never get called (apparently registering a cleanup within a cleanup no workie). I'd like to propose a new hook which gets run after connection to the client has been closed. (in my case, I run some cleanup code which can take a while, and would like the client to be on its way). Why can't you just register a cleanup on c-pool instead of r-pool? Well, I can register a cleanup on either pool, create a new subpool and destroy it myself before I return from the cleanup. That would solve all the problems, but it does seem really hackish (and an abuse of the cleanups). -- Jon
Re: New post-log-transaction hook?
I tried setting keepalive == 0 in the handler, and doing my ju-ju in the log_transaction phase. The client was still hanging around. -- Jon On Mon, Sep 17, 2001 at 04:11:58PM -0700, Ryan Bloom wrote: On Monday 17 September 2001 03:52 pm, Jon Travis wrote: Why can't you do it in the log_transaction phase. Assuming this is not a keepalive connection, the client will be gone by the time that phase is run. If this is a keep-alive transaction, then you won't save anything by adding another phase. Ryan I've got a bit of code that needs to run after a connection to a client has been closed. Right now I can (kind of) spoof this by setting the keepalive for the client to 0, and registering a cleanup on the request_req pool. Unfortunately the code in there is somewhat bulky, so any subsequent cleanups that it registers will never get called (apparently registering a cleanup within a cleanup no workie). I'd like to propose a new hook which gets run after connection to the client has been closed. (in my case, I run some cleanup code which can take a while, and would like the client to be on its way). Any thoughts? -- Jon -- __ Ryan Bloom[EMAIL PROTECTED] Covalent Technologies [EMAIL PROTECTED] --
