Re: do we still want sendfile enabled with our default conf files?
funny, I took the list of exceptions to be so large and hard to maintain that it made more sense to go with Jeff's original idea of just disabling sendfile by default unless a user specifically decided to enable it. I just had to debug a problem for a friend with sendfile on Linux. I don't know what caused the problem, but disabling sendfile solved it immediately. Seems to me that until our sendfile support is better, we should err on the side of always sending the data correctly instead of absolutely as fast as possible. I would much rather have APR default to not using the native sendfile, and only enable native sendfile when we have a lot of evidence that it does work correctly. Ryan On Fri, 18 Mar 2005 08:07:01 -0800, Justin Erenkrantz <[EMAIL PROTECTED]> wrote: > --On Friday, March 18, 2005 5:59 AM -0500 Jeff Trawick <[EMAIL PROTECTED]> > wrote: > > > ...snip, snip... > > AIX: > > > > Doesn't really fail in the normal sense of not putting the right data > > on the wire, but can trigger a kernel memory issue if some kernel > > tuning is incorrect. So always fail if the APR_SENDFILE_AUTODETECT is > > on. (This kernel tuning is irrelevant unless sendfile or more obscure > > TCP usage is actually occuring, so the tuning issue has typically been > > there all along without hurting anything.) > > Is the kernel turning incorrect on AIX by default? Will this be fixed in some > future releases? You could do lots of things to corrupt your kernel by tuning > in other ways - so unless this is by default, I can't see why we should block > this. > > > ...snip, snip... > > +1 to this list of exceptions and adding a new flag called APR_SENDFILE_CHECK > (or APR_SENDFILE_AUTODETECT) to apr_socket_sendfile. -- justin > -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: do we still want sendfile enabled with our default conf files?
That's fine. Pay attention to what I suggested. Default to
non-native sendfile, until we have know that it works. If you have an
OS that you know for a fact does sendfile correctly, then that would
be a case where we know that it works.
Instead of
#ifndef LINUX|WIN32|AIX|HPUX
sendfile()
#else
fake_sendfile()
#endif
do:
#ifdef SomeOSThatWorks
sendfile(.)
#else
fake_sendfile()
#endif
Since the beginning of APR, we thought that we knew how to get
sendfile working on most of our supported OS'. That's why we included
sendfile support. It turns out though, that for many of those
platforms, sendfile just doesn't work in some situations. When we
have a platform without the bugs, then yes we should use sendfile, but
for all other cases I care infinately more about correctness than
speed.
Ryan
On Fri, 18 Mar 2005 08:23:18 -0800, Justin Erenkrantz
<[EMAIL PROTECTED]> wrote:
> --On Friday, March 18, 2005 11:12 AM -0500 Ryan Bloom <[EMAIL PROTECTED]>
> wrote:
>
> > funny, I took the list of exceptions to be so large and hard to
> > maintain that it made more sense to go with Jeff's original idea of
> > just disabling sendfile by default unless a user specifically decided
> > to enable it. I just had to debug a problem for a friend with
> > sendfile on Linux. I don't know what caused the problem, but
> > disabling sendfile solved it immediately. Seems to me that until our
> > sendfile support is better, we should err on the side of always
> > sending the data correctly instead of absolutely as fast as possible.
>
> I absolutely refuse to punish users who are using good OSes because some OSes
> are brain-dead. This is exactly the role that APR is meant to fill: if we
> know of conditions where it is unsafe to use sendfile, we won't use it unless
> explicitly told so by the user.
>
> The minimal check can be:
>
> if (flags & APR_SENDFILE_CHECK) {
> #ifdef LINUX || WIN32 || AIX || HPUX
> return APR_ENOTIMPL;
> #endif
> }
>
> As people determine what conditions sendfile is safe (or causes problems),
> then we can add those.
>
> Feel free to advocate Linux always returning APR_ENOTIMPL for sendfile - I
> don't care. However, blocking sendfile on non-buggy OSes is not a solution
> that I am willing to sign off on. -- justin
>
--
Ryan Bloom
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Re: mod_actions 1.32 patch never made it to 2.0
On Tue, 14 Dec 2004 17:13:23 +0100, André Malo <[EMAIL PROTECTED]> wrote: > * Ryan Bloom <[EMAIL PROTECTED]> wrote: > > > I have a couple of pretty big issues with this response. > > > > 1) You have a configuration in Apache 1.3 that doesn't work in Apache > > 2.0, but the config directives don't have to be changed at all. This > > is something that we worked really hard not to do in 2.0. There > > should never be a config in 1.3 that just gives the wrong results in > > 2.0 without any way for the user to understand why. > > Ah? That's what one would call a bug. While breaking the behaviour in > 1.3.9/1.3.10 nobody even thought about this issue. It's *still not > documented that it was broken*. And a lot of users suffered of it, including > me. Suffered how? How exactly did a change that made the code accept more configs break your config? Also, it isn't that nobody thought of this when making the change to 1.3. Looking through the mailing list archives, I see Ben Laurie specifically had a problem with Location and mod_actions that Manoj fixed. I haven't found the whole thread about the problem, just the RM notes about it. So, we have a bug that was fixed in 1.3 that was reintroduced in 2.0, and 2.0 is solving the problem the completely opposite way. Instead of defaulting to doing what 1.3 does, you default to the opposite position. That is what I am saying is so wrong here. Pick the same default as 1.3, and allow the option to override that default. > > > 2) In choosing to default to the 404, you have broken anybody who > > wants to share 1.3 and 2.0 config snippets. > > [...] > see above. > Additionally 1.3 and 2.0 *are* different, so this is null argument at all. I'm sorry, but no it is not. I know something about this, and we spent a lot of time and energy trying to ensure that a config that worked in 1.3 worked the same way in 2.0. We jumped through hoops to ensure that a handler configured as it would be in 1.3 would work even if the handler was moved to a filter. There should not be any examples of a config directive that has the exact same syntax in 1.3 and 2.0 and different behavior. That is what we are talking about here. The directive has the _same_ syntax in the two trees, thus it should act the same way. > > 4) This isn't documented anywhere at all. If you are going to break > > old configs, then please add some docs to the docs-2.0 tree. > > Sure. It's not incorporated yet. It was voted recently and I'm going to > commit it soon. And I'm asking that you reconsider doing that so that 2.0 and 1.3 can interoperate more cleanly. You aren't going to change how 1.3 handles this situation, and 2.1 hasn't been released, and the change hasn't been back-ported to 2.0. So, the 2.x tree should do what 1.3 does, and your flag should allow you to override. Ryan -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
mod_actions 1.32 patch never made it to 2.0
A co-worker of mine pointed out that the following works in Apache
1.3, but not 2.0 if the location /foo, doesn't exist on the disk:
AddHandler foobar /cgi-bin/printenv
SetHandler foobar
This patch ports this behavior forward into 2.0.
Index: modules/mappers/mod_actions.c
===
--- modules/mappers/mod_actions.c (revision 111773)
+++ modules/mappers/mod_actions.c (working copy)
@@ -163,11 +163,6 @@
if ((t = apr_table_get(conf->action_types,
action ? action : ap_default_type(r {
script = t;
- if (r->finfo.filetype == 0) {
- ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r,
- "File does not exist: %s", r->filename);
- return HTTP_NOT_FOUND;
- }
}
if (script == NULL)
I'm not subscribed to the list anymore, but please let me know when
this is committed.
Ryan
--
Ryan Bloom
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Re: using APR_STATUS_IS_SUCCESS
Basically, the macro is wrong and needs to be removed. The contract that _all_ APR API's live up to is that on a successful result, they must return APR_SUCCESS. The reason we chose to use 0 as success is simple: 1) Most platforms can check for equality with 0 faster than they can check for any other integer equality. 2) It makes checking for success _really_ easy, because all APIs use the same value for success, there is no guessing or research needed, if the result wasn't 0, then the function didn't succeed. It didn't necessarily fail, because there are status codes that aren't full success and aren't failures, but more research is needed. 3) It provides you an opportunity to have a lot of different values for errors and statuses without having to use a separate variable. I assumed that the original addition of the macro was so that success was handled like any other result code, ie you always use a macro. If the reason was so that some functions could return non-zero success codes, then the macro definately needs to go, because that is a really bad idea. Ryan On Wed, 28 Jul 2004 13:47:20 -0700, Geoffrey Young <[EMAIL PROTECTED]> wrote: > cross-posted to [EMAIL PROTECTED] > > Garrett Rooney wrote: > > Geoffrey Young wrote: > > > >> hi all > >> > >> I was just in garrett's APR talk here at oscon and he was mentioning the > >> APR_STATUS_IS_SUCCESS macro, which I found interesting since httpd > >> only uses > >> it in a few places, opting for a direct comparison to APR_SUCCESS > >> instead. > >> > >> should we move to APR_STATUS_IS_SUCCESS in all places? can someone who > >> groks the nuances of the macro add some insight here? > > > > > > This is actually something I was wondering about as I wrote the > > presentation. Neither Apache or Subversion use APR_STATUS_IS_SUCCESS > > everywhere, but I wonder if we should, since if you look at the > > definition of the macro there are cases when it's more than just (s) == > > APR_SUCCESS. > > just another note, I grep'd the code for "rc == APR_SUCCESS" and it looks > like not even APR is using the macro everywhere... > > --Geoff > -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: PHP POST handling
On Tue, 1 Oct 2002, William A. Rowe, Jr. wrote: > At 01:12 PM 10/1/2002, Greg Stein wrote: > >For PHP, we said "make it a filter [so the source can come from anywhere]". > >I think we really should have said "for GET requests, allow it to be > >processed by PHP." The POST, PROPFIND, COPY, etc should all be possible to handle >by PHP, which means that PHP also needs a handler. > > Agreed, if you write a PHP script we better allow you to PROPFIND > or COPY the puppy, in addition to POST. These are two different statements, if I am reading both correctly. Please correct me if I am not. Will, you are saying that if we have a PHP script, then we need to be able to do all DAV operations on the script. Greg, you are saying that a PHP script needs to be able to satisfy a DAV request (meaning that the PHP code actually copies the resource, or generates the PROPFIND data). Assuming I am reading the two statements correctly, I agree with Will, but not with Greg. There is a major difference between satisfying a COPY or PROPFIND request and generating a page that has accepted POST data. A filter will never be able to satisfy COPY or PROPFIND, because those are actions that should be done in the handler phase. However, having two ways to read the PHP script from disk (default_handler and php_handler), and run the page through the interpreter doesn't make sense. That is why PHP was re-written as a filter, to allow it to take advantage of ANY back-end data store that we have. > >Heck, PHP should also be able to handle a GET request. For example, it > >should be able to retrieve the content from a database, and then shove that > >into the filter stack. > > > >IOW, PHP is really a handler *and* a filter. It can handle the generation of > >content, but it can also process generated content when that content is a > >PHP script. I think I am missing something here. PHP doesn't handle content generation. It never has. In Apache 1.3, PHP could read a script from the disk and interpret it. In Apache 2.0, PHP _should_ be able to read a script from a bucket and interpret it. (The fact that it doesn't right now, is not really germane to this discussion). >From my reading of the statement above, you want people to be able to write handlers in PHP, which would find another page or script in a database and send it down the filter stack. That can't be done right now, PHP can't write handlers that way, at least not that I am aware of. This BTW, is why mod_perl has both concepts, handlers and filters. Handlers are used as content endpoints, they generate data. Filters are used to modify data that was already generated. Please let me know if I have misunderstood anything in this mail. Everything I have said above is based on my reading of the message, and I tried to point out where I may have not understood what the original author was saying. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: Speaking of pipes from cgis causing trouble...
On Tue, 1 Oct 2002, Paul J. Reder wrote: > I've been working on the caching code and ran across a core dump... > > A particular file contains an SSI call to a cgi. The cgi causes a pipe > bucket to pass down the chain. cache_in_filter tries to save the bucket > away and core dumps. > > Since a pipe bucket can be of any length, and could take any amount of > time to complete, I would assume that the cache code should decide not > to cache any response with a pipe bucket, correct? Not necessarily. The cache code should stream the data to the cache, and allow the data to also stream to the core_output_filter. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: PHP POST handling (was: cvs commit: httpd-2.0/server core.c)
On Tue, 1 Oct 2002, Greg Stein wrote: > On Tue, Oct 01, 2002 at 12:29:53PM -0400, [EMAIL PROTECTED] wrote: > > On 1 Oct 2002 [EMAIL PROTECTED] wrote: > > > > > gstein 2002/10/01 09:24:41 > > > > > > Modified:server core.c > > > Log: > > > Fix bug in the default handler. POST is not allowed on regular files. > > > The resource must be handled by something *other* than the default > > > handler. > > > > -1. This is going to break PHP. PHP is a filter now, which means that > > the page is served by the default_handler. Since PHP requests are allowed > > to use POST, this is now broken. > > > > As I said before, the bug is in mod_dav, and must be fixed there. > > That will get fixed. I keep saying that, but you keep harping on "default > handler isn't broken". Bunk. > > The default_handler is broken. If you POST to a resource, it returns the > resource. That isn't right. > > But the PHP point is a good one. So how do we prevent a POST from returning > the resource, yet make it available for PHP? > > I think that we have a model problem here. For a POST method, you need > something to *handle* the POST. Somebody actually needs to accept the posted > content, do something with it, and then *generate* a response. That response > might actually have other filter processing to apply to it. > > It almost seems like PHP needs to act as a handler for POST requests (to > deal with the posted content), and as a filter for GET requests. > > Does that seem right? No, it doesn't seem right. The default_handler is designed to deliver a file from disk down to the next filter. In the case of filters, we have extended the handler to work for POST and GET requests, because that is how filters work. The assumption is that if it is a POST request, and the default_handler gets the request, then there is a filter that will actually run the request. This works, and has worked since we introduced filters, and converted some handlers to filters. The bug is in mod_dav. It is trying to handle too many requests. There was no bug in the default_handler. Which is why I am against putting a fix in for it. As for: "And I *already* said there is a separate issue/bug. You don't have to keep beating the damned horse and continue to disagree to try and show that you're the guy with all the answers. It's really annoying. You could just say, "yah. default_handler does have a bug, but both need to be fixed [but I already fixed the mod_dav one]"." Because I don't believe there was a bug in the default_handler, it was working as designed. The default_handler assumes that it will be run last, and that other modules will have an opportunity to serve the request if they are supposed to. I am not trying to show that I have all the answers, but in this case, I did the research, and I found the bug. And, I do have a VERY strong opinion about how it should be solved. Filters introduced some intricacies that weren't in the server before. If we leave the change in that was made to the default_handler, nothing will be fixed. If we add a new handler that handles POST requests for PHP, what does that get us? It wouldn't have stopped the bug that we are discussing. At best, it would have changed the response the server gave to the user. However, that is only if the request was for a CGI script. In the case of PHP, php requests were working before my fix, because the default_handler was doing it's job. With the fix to the default_handler that is in place now and without the mod_dav fix, both CGI and PHP are broken. Without the default_handler fix and without the mod_dav fix, Just CGI are broken. With just the mod_dav fix, and without the default_handler fix, CGI and PHP work. With both fixes, CGI work and PHP are broken. You want to fix PHP scripts now by adding a new handler for PHP. Would that be specific to PHP? If so, then there are two ways to run PHP scripts. No thanks. If not, then I fail to see what is being fixed. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: POST
On Tue, 1 Oct 2002, Greg Stein wrote: > On Tue, Oct 01, 2002 at 11:03:16AM -0400, Ryan Bloom wrote: > > On Mon, 30 Sep 2002, Greg Stein wrote: > >... > > > For this particular case, the bug is in default_handler(). Plain and simple. > > > There is no reason for a POST request to return the file contents. Yes, the > > > system should also call the thing as a CGI script, in this case, but that > > > doesn't excuse the default handler. > > > > No Greg, I'm sorry, but the bug has nothing to do with the > > default_handler. Plain and simple. If mod_dav wasn't in the server, the > > default_handler would never be called, because mod_cgi would have been > > given an opportunity to handle the request. The bug is in mod_dav, and no > > fix anyplace else will actually solve the problem. > > mod_dav causes the bug in default_handler to be exposed. Nope. The default_handler relies on other handlers to run first, so that it only gets the requests it is supposed to get. Even if we change the default_handler to only serve GET requests, the bug still exists, because the bug is in mod_dav. > A secondary issue is how mod_dav alters the dav-handler in a way which > disables POST to a CGI. You've fixed this latter issue (altho it breaks the > RFC 2518 requirement of checking locks before allowing a POST). I think we > should figure out a different hook to use for that check. > > While the "fixups" hook isn't really intended for this, it would seem a good > place to do the check. mod_dav already hooks it, so that should be fine. The fixups hook is definately intended for this. The handler phase is only intended for actually generating content. Mod_dav isn't generating content for a POST request, thus it shouldn't be trying to handle it in the handler phase. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: POST
On Mon, 30 Sep 2002, Greg Stein wrote: > On Mon, Sep 30, 2002 at 06:53:09PM -0400, Ryan Bloom wrote: > >... > > The problem is that the default_handler shouldn't be involved. Because > > mod_dav is now replacing the r->handler field for ALL requests, things > > Woah! *NOT* "all" requests. Only those under the URL namespace which has > been assigned to mod_dav. It does not just go out and blast r->handler > willy-nilly. You have specifically enabled DAV for the URL space in > question. > > >... > > If those two things are done, then we could have two handlers for the same > > resource. However, mod_dav shouldn't just be taking over all requests and > > assuming that they were meant for the core server. Doing so means that > > all generators are broken if DAV is loaded in the server. > > It does not "just be taking over all requests." It is handling requests for > the space that you've assigned to mod_dav. > > For this particular case, the bug is in default_handler(). Plain and simple. > There is no reason for a POST request to return the file contents. Yes, the > system should also call the thing as a CGI script, in this case, but that > doesn't excuse the default handler. No Greg, I'm sorry, but the bug has nothing to do with the default_handler. Plain and simple. If mod_dav wasn't in the server, the default_handler would never be called, because mod_cgi would have been given an opportunity to handle the request. The bug is in mod_dav, and no fix anyplace else will actually solve the problem. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: [PATCH] core_output_filter
I have committed a slightly modified version of this patch.
Ryan
On Fri, 20 Sep 2002, Ryan Morgan wrote:
>
> I have a quick question about some of the logic in the core_output_filter.
>
> When the core_output_filter sends data to the network (via writev_it_all,
> emulate_sendfile, or sendfile_it_all), we check the apr_status_t and log
> an error when rv != APR_SUCCESS. However, apache only sets c->aborted if
> the status code was ECONNABORTED, ECONNRESET, or EPIPE.
>
> Is there a reason we only set c->aborted for these specific return values?
> I think it may make more sense to always set c->aborted if we encounter an
> error. This flag is the only way modules will know if something in the
> filter chain went wrong.
>
> A patch is attached to always set c->aborted on network send errors.
> (although I have only run into this problem with APR_TIMEUP errors, so
> we could also just add a APR_STATUS_IS_TIMEUP(rv) check)
>
> -Ryan
>
> Index: core.c
> ===
> RCS file: /home/cvspublic/httpd-2.0/server/core.c,v
> retrieving revision 1.205
> diff -u -r1.205 core.c
> --- core.c 16 Sep 2002 13:11:55 - 1.205
> +++ core.c 20 Sep 2002 20:46:52 -
> @@ -3910,11 +3910,7 @@
> if (more)
> apr_brigade_destroy(more);
>
> -if (APR_STATUS_IS_ECONNABORTED(rv)
> -|| APR_STATUS_IS_ECONNRESET(rv)
> -|| APR_STATUS_IS_EPIPE(rv)) {
> -c->aborted = 1;
> -}
> +c->aborted = 1;
>
> /* The client has aborted, but the request was successful. We
> * will report success, and leave it to the access and error
>
--
___
Ryan Bloom [EMAIL PROTECTED]
550 Jean St
Oakland CA 94610
---
Re: POST
On Mon, 30 Sep 2002, Greg Stein wrote: > On Sun, Sep 29, 2002 at 10:33:08PM -0700, Justin Erenkrantz wrote: > > On Mon, Sep 30, 2002 at 01:17:55AM -0400, Ryan Bloom wrote: > > > Because 2.0.42 always displays script source for CGI scripts that use > > > POST, I believe that we should put that notice on our main site, and stop > > > suggesting 2.0.42 for production use. > > > > I could not reproduce your problem in my tests. Do you have a clear > > reproduction case? (POSTing to a DAV resource for me yields a 404.) > > > > Is it somehow related to mounting a DAV repository at / and the > > server getting confused about the /cgi-bin/ dir? -- justin > > > In any case, Ryan suggested that mod_dav shouldn't set r->handler for > methods that it isn't going to handle. That is arguable. The resource *is* > located within the URL space designated as being under mod_dav's control. > > The bug here is default_handler()'s serving up POST'd documents. > > A secondary issue is possibly refining mod_dav's handling. But that is a > *very* deep problem. The specified resource might not be in the filesystem, > so it would be *invisible* to Apache if mod_dav doesn't handle it. The grey > area is when you're working with mod_dav_fs. It may be time to bite the bullet and abstract out the FS from the core server. It would be a PITA, and it would mean a major re-factoring of DAV code, because mod_dav wouldn't have the repository code anymore. Instead, it would be a part of Apache itself. I believe that is the only way that you are actually going to solve this problem cleanly though. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: POST
On Mon, 30 Sep 2002, Greg Stein wrote: > On Mon, Sep 30, 2002 at 09:36:50AM -0400, [EMAIL PROTECTED] wrote: > > On Sun, 29 Sep 2002, Justin Erenkrantz wrote: > > > > > On Mon, Sep 30, 2002 at 01:17:55AM -0400, Ryan Bloom wrote: > > > > Because 2.0.42 always displays script source for CGI scripts that use > > > > POST, I believe that we should put that notice on our main site, and stop > > > > suggesting 2.0.42 for production use. > > > > > > I could not reproduce your problem in my tests. Do you have a clear > > > reproduction case? (POSTing to a DAV resource for me yields a 404.) > > > > > > Is it somehow related to mounting a DAV repository at / and the > > > server getting confused about the /cgi-bin/ dir? -- justin > > > > Nope, any script that is under DAV control will post the source to the > > user. > > Hunh? > > The POST code in mod_dav checks for DAV locks on the resource (per RFC > 2518). If there is a lock, then it returns an appropriate error. Otherwise, > it just returns DECLINED and lets (presumably) the default handler deal with > the POST request. > > The bug is located in server/core.c::default_handler(). It serves up the > content of a file that you POST to. That isn't right. > > Of course, we also have the problem that we can't have more than one > handler. We could restore the RFC 2518 POST check by putting that into a > different processing phase. Not sure which, tho... (it needs to be after the > authentication, at least) The problem is that the default_handler shouldn't be involved. Because mod_dav is now replacing the r->handler field for ALL requests, things broke. Basically, mod_cgi had been set as the handler, but the mod_dav came in and took over. Then, when mod_dav was done, it passed the request over to the core server, which shouldn't be seeing this request. If mod_dav is going to take over all reqeusts, then it needs to save a copy of the old r->handler field so that it can be reset appropriately, and it needs to be registered as APR_HOOK_REALLY_FIRST for the handler phase. If those two things are done, then we could have two handlers for the same resource. However, mod_dav shouldn't just be taking over all requests and assuming that they were meant for the core server. Doing so means that all generators are broken if DAV is loaded in the server. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: POST
On Sun, 29 Sep 2002, Jerry Baker wrote: > Ryan Bloom says: > > There is already a bug filed. It works if you don't have DAV enabled for > > that CGI location. I am hoping to look at that today. > > Is there a way to at least stop Apache from giving the script source to > the viewer without disabling CGI or DAV? According to my reading of the code, no it isn't possible. However, I have just commmitted a fix for this. I am hoping that one of the DAV experts will review my fix for correctness, but it is what we did until a few weeks ago. However, from my reading of the dav_module, I have a major concern. The module is currently trying to handle every type of request. But, that is wrong, it isn't how modules are supposed to behave. Mod_dav should only be setting the handler field for requests that it knows it can serve correctly. Because 2.0.42 always displays script source for CGI scripts that use POST, I believe that we should put that notice on our main site, and stop suggesting 2.0.42 for production use. Mod_dav developers, please check my commit. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: POST
On Sun, 29 Sep 2002, Jerry Baker wrote: > Ryan Bloom says: > > There is already a bug filed. It works if you don't have DAV enabled for > > that CGI location. I am hoping to look at that today. > > Is there a way to at least stop Apache from giving the script source to > the viewer without disabling CGI or DAV? I haven't found one yet, but I am just now looking at code. Ryan _______ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: POST
There is already a bug filed. It works if you don't have DAV enabled for that CGI location. I am hoping to look at that today. Ryan On Sun, 29 Sep 2002, Jerry Baker wrote: > For some reason POST doesn't work with the latest HEAD. I can't say when > it stopped working, but I can tell you that GET works. If I have a form > and set the action to GET, the script runs flawlessly. If I set the same > form's action to POST, Apache returns the source of the Perl script as > if it were text/plain. I was surprised that Apache was giving out the > source code to my scripts, and now I'm afraid to imagine for how long it > has been doing this. > > -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: How Do I Create A Per-Worker Pool?
On Thu, 26 Sep 2002, Charles Reitzel wrote: > Thanks for telling me what I need to know: "you can't get there from here." > > I don't want to start a philosophical debate, but it is a common idiom > (yea, verily a "pattern") in multi-threaded programming to avoid contention > by duplicating singletons in each thread. This can be done either by using > platform-specific thread-local storage or, portably, by passing in the > needed singleton (or a place to put it) into the worker thread. > > A mutex-free, per-thread pool would serve this purpose perfectly. Change > the word "thread" to "worker" in the above, and you have made it portable > across MPM models. > > A per-thread pool would assist a great deal in bringing many Apache 1.x > modules over to 2.x. In essence, you move the globals into the per-thread > pool, and the remaining application semantics are unchanged. > > You can't "just add synchronization" to most > applications/modules. Usually, you need to redesign much of it to make it > work without killing performance. > > Convinced? Unfortunately nope. You are confusing a programming model with pools. Pools are about variable life-time. In the case of what you are trying to do, it is possible already. All you need to do, is do the initalization in the child_init phase, and save each version of the data in a table that is accessible by the whole process. The problem with a qorker_rec, is that it doesn't fit with the Apache model. Yes, it would be possible to add the pointer to the request_Rec, but it isn't necessary. Everything in Apache is centered around the request, not the thread/process that is serving the data. By adding a worker_rec, we are actually encouraging programming practices that we don't want, like copying a bunch of global data into a worker_pool. We would much rather that modules were written with thread-safety in mind, so while it is possible to do, it isn't the right solution. Also, we did have a thread_pool at one time that worked for threaded MPMs. The problem was that it just wasn't ever used. This makes sense, because pools are about lifetime, and in all of the MPMs (except perchild), threads have the same lifetime as processes by definition. I hope that makes sense. Ryan
Re: [PATCH] mod_mime and virtual requests
> > Hmm. We might not even need the fixup. Now that *every* handler is executed > > nowadays, we can probably just remove the concept of "dav-handler" and move > > those checks in type_checker/fixups into dav_handler() itself. > > Hmm. Perhaps. I'll take a look at it. -- justin Small problem with that. Not all handlers are run, only the first handler to do something is run. Also, we shouldn't be moving stuff into handlers, we should be moving stuff out of them. The idea of handlers is to actually generate the data. If mod_dav is doing checks, those should be done before the handler, so that the dav_handler can return ASAP. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: El-Kabong -- HTML Parser
On Thu, 29 Aug 2002, Aaron Bannert wrote: > On Thu, Aug 29, 2002 at 02:24:28PM -0400, Ryan Bloom wrote: > > > +1 from me, I prefer APR actually. > > > > I am really uncomfortable with this going under the APR project. As > > things stand right now, it just doesn't fit with what we have stated our > > goals to be. > > > > If you want to change our stated goals, then go ahead and do it. Just > > committing code that doesn't fit with our goals isn't the way to do that. > > (I will defer answering this for an apr-only discussion.) > > > I will make one exception to that statement. If it lands inside of > > APR-util, under the XML directory, and it is made to work with the XML > > parser, I can accept that landing spot. As it fits in closer with our > > goals (I think). Jim, I can't decide if this is what you meant or not. > > I'm +1 on integrating it into our XML stuff. I consider it to be > equivalent to apr-util, so either we put it inside apr-util, or > we create a new APR subproject or sub-library for it. I should also mention that I completely do not see this as equivalent to apr-util. I reserve the right to ask for this project to be removed from APR after the APR project has decided on it's stated goals. That does not mean that it would be removed from the ASF (assuming my request is approved), only that it would need to find a new home within the ASF umbrella. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: What evil lurks beneath macros... (an AIX xlc optimizer bug)
On Fri, 16 Aug 2002, Cliff Woolley wrote: > On Fri, 16 Aug 2002, Ryan Bloom wrote: > > > A better question IMHO, is whether any of those macros can be made less > > complex. > > It's a good question, but IMO the answer is no. The ring macros are very > tight and easy to read... like I said, they're about four lines each. > The brigade macros are, for the most part, one line wrappers around those. > I don't see how they could get any less complicated than they are. The problem is that many of the macros are implemented in terms of other macros. For example, the INSERT_TAIL macro seems overly complex. It may not be, but it is not the easiest macro to chase down. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: What evil lurks beneath macros... (an AIX xlc optimizer bug)
On Fri, 16 Aug 2002, Cliff Woolley wrote: > On Fri, 16 Aug 2002, Ian Holsman wrote: > > > I'm just wondering if there is any kind of measureable performance > > benefit in keeping these as a macro vs putting them in a function > > (possibly inline if the compiler can support it). > > I'm quite sure there is a performance benefit, though admittedly I don't > have numbers on hand to support that at the moment. > > It's as simple as this: many of the operations, once you get rid of the > typecasts and other noise, optimize down to just a few adds/subtracts and > a few assignments. Four lines of code in most cases. Adding the overhead > of a function call (actually probably two or three function calls) for > each operation would be substantial. > > It just looks messy when you expand the macros because it all gets > squished on one line and because lots of noise gets shoved in. But you > could say that about most macros when read in fully expanded form. I agree 100%. These macros are relatively simple, just doing a bunch of casting and pointer manipulation. The cost of putting those in functions would most likely be very high. A better question IMHO, is whether any of those macros can be made less complex. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: Re[2]: perchild on FreeBSD 5?
> In view of this "It's impossible to tell which libs are thread safe" > wouldn't it perhaps make sense to have a perchild version that uses > processed instead of threads? I sure know that this would result in a > insane amount of sleeping processes in a massive Virtualhosting setup > but in my tests a very low end 500Mhz FreeBSD 4.6 box with 256MB RAM had > no problem whatsoever with 2000 sleeping Apache processes belonging to 250 > DISTINCT instances of 1.3.26, all based upon different config files > and running on different ports . Swap wasn't used and in idle state, > the 2000 processes consumed less than 1.5% CPU... While it is technicalloy possible to create a preforking server that operates like perchild, it isn't a simple thing to do. I would even go so far as to say that it isn't something you really want. If you want that behavior, the best solution is to just run multiple Apache servers. I looked at every MPM before creating perchild, and the only one that made a lot of sense for perchild was a static number of processes with a dynamic number of threads per process. If you have a dynamic number of processes, then the parent process must always determine how many child processes with each uid/gid exist currently. If you are going to do that, you are better off to just run two or three instances of prefork with an IP alias. Ryan _______ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: [PATCH] UDP support
I don't believe that this should be incorporated into the core server. The core of Apache is an HTTP server, which means that it should only know how to listen on TCP sockets. The support is there however, for other socket types to be added to the server. For the UDP case, your protocol module should implement it's own listen directive, to allow people to add UDP sockets for your protocol module. Then, the protocol module can set the accept_function pointer in the listen_rec, which in this case should be set to unixd_udp_connect. Now, when the server returns from apr_poll, it will automatically call the correct function for the current socket type. For the actual network I/O, your protocol module will need to have it's own filter to replace the core_output and core_input filters. This makes sense, because the standard filters are very much tuned towards what we expect in the HTTP world. If you replace those functions, you can tune them to give the best possible performance for your protocols. The two places that still need to be fixed for this to work are adding sockets to the listen_rec list and the lingering_close logic. Both of these can be fixed by adding a hook to the server. Adding listeners to the listen_rec list _may_ be possible in the open_logs phase, which is where the core server does it, but I haven't tried to implement that yet. If it can be doine in the open_logs phase, then you won't need another hook for it. The lingering close logic should just be replaced by a simple hook that the core server calls. The core can add lingering_close to that hook, and everything should continue to work. Please let me know if any of this doesn't make sense. I like the idea of allowing the core to be used for UDP-based protocols, but I want to be sure that it is done the best way possible. Ryan On Wed, 14 Aug 2002, Tony Toyofuku wrote: > Hi, > > Many months ago I submitted a patch for UDP support within Apache 2.0. This > a resubmission of that patch, > which allows for UDP packets to work with Unix versions of Apache 2.0. > Here's what I wrote then: > > This patch adds UDP support to unix versions of Apache 2.0. > > This patch is set to add UDP support to the prefork MPM; however it should > be > trivial to add UDP support to other MPMs (changing the MPM_ACCEPT_FUNC > definition in mpm.h, and adding the UDP_LISTEN_COMMANDS line in the MPM > source > code). > > Here's how it works: > > 1. At configuration time, there's a new directive "UDPListen". This is just > like the normal "Listen" directive, but it sets up a UDP "listener". It > sits > in the httpd.conf file, and looks like this (where "8021" is the port > number): > > UDPListen 8021 > > 2. Since there's no notion of accept() on a UDP socket, there's a new > abstraction layer between the accept system call, named unixd_pop_socket. > If > the incoming request is UDP, the socket gets routed to a UDP version of the > "unixd_accept" function. If the request is TCP, it gets send to the > existing > "unixd_accept" function. > > 3. The network I/O is now done using recvfrom() & sendmsg, since UDP must > know > the port/address of the client. Additionally, rather than using sendfile() > for > the UDP requests, emulate_sendfile is used instead. This is required since > sendfile() won't work with connection-less sockets. > > That's pretty much it. > > Although the UDP transport layer will work for HTTP, for me the value of UDP > > is to use Apache 2.0 with its new multiple protocol support. In this way, > I can write an Apache protocol module to communicate with the legacy UDP > systems that I've got to support. > ><> <> <> <> > > I've included a modified version of one of the APR UDP test apps, and its > Makefile to exercise the server. > > Tony > > -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: Re[2]: perchild on FreeBSD 5?
On Tue, 13 Aug 2002, Gabriel Ambuehl wrote: > -BEGIN PGP SIGNED MESSAGE- > > Hello Ryan, > > Tuesday, August 13, 2002, 6:10:21 PM, you wrote: > > > > Assuming that FreeBSD 5 solves the threading problems with FreeBSD, > then > > That's what we've been told by the people working on 5.0 at least. I > dunno whether they actually understand the issue though as they claim > that FreeBSD 4.X supports threading, too. > > > yes, Perchild will work on that platform. The problem right now is > that > > PErchild doesn't work at all. I am hoping to have time to work on > > PErchild on Thursday or Friday to finish the work on that MPM. > > On another note: will it already work on NetBSD or OpenBSD once it > will be back in service on Linux? I don't honestly know. The problem is that there are mulitple ways to pass file descriptors between processes based on the Unix that you are using. Assuming FreeBSD, NetBSD, and OpenBSD all have good thread libraries and that they all support the fd passing mechanism that Liux uses, yes everything will work. IF they don't support the same fd passing logic, then it is a simple matter of implementing the other fd passing logic, which should be easy to do. Assuming I get Perchild working this week, I will try to get it working on a BSD machine that I have as well. If I can't make it work on my BSD machine (because it is FreeBSD), then I will most likely need an account on a Net or Open BSD box. I have recently lost access to most of my different types of machines, and while I'll get some access to new boxes starting next week, it won't be specifically for Apache work. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: perchild on FreeBSD 5?
Assuming that FreeBSD 5 solves the threading problems with FreeBSD, then yes, Perchild will work on that platform. The problem right now is that PErchild doesn't work at all. I am hoping to have time to work on PErchild on Thursday or Friday to finish the work on that MPM. Ryan On Tue, 13 Aug 2002, Gabriel Ambuehl wrote: > Hello, > I've more or less accepted that perchild on FreeBSD 4.X isn't going to > happen (as sad as it is, I always considered it to be THE feature [1] in > 2.0 that would warrant an upgrade for us) but what I'd like to know is > if there is any chance to see perchild on FreeBSD 5 which gets wholly > new threading and SMP libs? > > > My company would happily pay someone a few thousand US$ to come up > with a working perchild implementation on FreeBSD 5 and from what > I've gathered on the FreeBSD mailing lists, there might be other > parties that would contribute to the funding, too. We haven't got any > reasonable in-house knowhow to contribute with code but we'd surely > help beta testing. > > > > > [1] Name based vhosts with PHP scripts running under the UID > of the user. Great for all ISPs out there. > > > Regards, > Gabriel > -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: user field of conn-rec in change from apache 13 to 20
The user field is in the request_rec now, because users are a request-based entity in HTTP. Ryan On Mon, 12 Aug 2002, Harrie Hazewinkel wrote: > HI, > > Can someone point me to the equivalent in the Apache 2.0.x versions > of the user field that was in the Apache 1.3.x in hte conn_rec?? > > thanks by advance, > > Harrie > -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
RE: filename-to-filename hook
The hook was not added for Apache 2.0. If the docs were changed from 1.3 to state that it was added, then it is a bug. Ryan -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] > -Original Message- > From: Werner Schalk [mailto:[EMAIL PROTECTED]] > Sent: Thursday, August 08, 2002 4:11 PM > To: [EMAIL PROTECTED] > Subject: filename-to-filename hook > > Hello, > > I have already asked the people of > the users mailing list but nobody > did answer my question so I gonna > ask it here: > > In the docs is written that it is > not needed to add the [PT] option > to a RewriteRule, when passing it > through another module because > apache2 now supports a filename-to-filename > hook. How can I enable it? > > See http://httpd.apache.org/docs-2.0/mod/mod_rewrite.html#rewritemap, > there is written: > For Apache hackers > If the current Apache API had a filename-to-filename hook additionally > to the URI-to-filename hook then we wouldn't need this flag! > But without such a hook this flag is the only solution. > The Apache Group has discussed this problem and will add such > a hook in Apache version 2.0. > > > Bye and thanks, > Werner.
RE: cvs commit: httpd-2.0/build httpd_roll_release
> > > Even if we don't build it, this is extremely good practice that the > folks > > > rolling and releasing the tarball TAG the apr-iconv tree in sync with > > > the current apr and apr-util trees.. > > > >I completely disagree. The problem is that the httpd_roll_release > >script is for rolling httpd releases, not APR releases. This change > >doesn't help people realize that they have to tag APR-iconv before they > >can release httpd. > > Amazing that we tag APR at all, no? That APR gets tagged with Apache, is a side-effect of not having released APR yet, nothing more. In time, we won't tag APR with an Apache tag. > > I really agree with Cliff, the change to pull > >apr-iconv out of APR is annoying, and it is going to cause problems. I > >understand that it is the "best" solution we have right now, it is still > >a bad solution. > > Of course it is bad. That's why I suggest a separate tarball for iconv. > > But it doesn't matter, we need trees in-sync, so apr-iconv must be tagged > with apr's tags, from here forwards. If you want to do that as an rtag, > that would be fine too. The other thing, is that httpd_roll_release doesn't do the tag, it simply checks out the code that has already been tagged. Ryan
RE: cvs commit: httpd-2.0/build httpd_roll_release
> From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]] > > At 11:42 AM 8/1/2002, you wrote: > >ianh2002/08/01 09:42:33 > > > > Modified:buildhttpd_roll_release > > Log: > > we need apr-iconv now > > Even if we don't build it, this is extremely good practice that the folks > rolling and releasing the tarball TAG the apr-iconv tree in sync with > the current apr and apr-util trees.. I completely disagree. The problem is that the httpd_roll_release script is for rolling httpd releases, not APR releases. This change doesn't help people realize that they have to tag APR-iconv before they can release httpd. I really agree with Cliff, the change to pull apr-iconv out of APR is annoying, and it is going to cause problems. I understand that it is the "best" solution we have right now, it is still a bad solution. Ryan
RE: SSLMutex directive, PR 8122
> This PR points out a problem with the current SSLMutex handling in > mod_ssl and provides a patch to resolve it. > > I'd rather see a different solution: > > 1) give SSLMutex the same syntax as AcceptMutex, except that SSLMutex >has an additional choice: none. > 2) add SSLMutexFile which is processed like the LockFile directive > > This is a migration problem for people using 2.0.x with mod_ssl, but > for the long term I suspect it will be beneficial not to bundle the > optional lock file name with the mutex mechanism. > > Comments? ++1. Ryan
RE: quick_handler hook is completely bogus.
I agree with everything that Brian said, and he put it far better than I could have. I would be MUCH happier with the quick_handler phase if it occurred after the AAA phases. Potentially, this could be achieved, and the performance could be improved by simply having the cache module register a map_to_storage function, which checked to see if the page was in the cache. If it is, then no other map_to_storage function is required (assuming enough state is saved in the cache to re-authenticate the request). If it isn't, then the server moves on to the next function. This has the advantage of being secure, and it also removes the biggest performance problem, the directory and location walks. Ryan -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] > -Original Message- > From: Eidelman, Brian [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, July 31, 2002 8:01 AM > To: '[EMAIL PROTECTED]' > Subject: RE: quick_handler hook is completely bogus. > > Hi All, > > As an access control module developer I feel compelled to weigh in here. > > First, I would like to point out that you cannot determine whether a page > was password protected based on the Apache configuration files including > .htaccess. Modules such as the one we develop, which is widely deployed, > plug into the access hook and use more robust notions of access policies. > > The only safe way you can determine what content is safe to cache is by > having a robust set of configurable rules in your caching module (or > proxy) > that allows a site administrator to set rules that determine what is safe > to > cache. These rules can be as simple as "cache these URLS" or as complex > as > "cache all URLS that end with .gif when no cookies are sent inbound or set > outbound and when there is no cache control header telling us not to > cache." > > The caching done in mod_cache and in reverse proxy caches are different > from > browser caches and forward proxy caches from an access control point of > view > in that it is within the scope and responsibility of the enterprise that > owns the content being cached. To put it another way, mod_cache and > rproxy > caches are responding to real requests made back to the server where as > browser caches and forward proxy caches are not. > > I agree with Graham's notion that any hook designed primarily for caching > should be moved to after the access control hooks. I think the direct > performance impact will be minimal and easily made up for by the fact that > you will be able to cache more content since you will no longer be > constrained by whether or not the content is protected (only by whether or > not it is dynamic for each user/request). > > Additionally, I'd like to point out that total reliance on cache control > headers is not a good option. Unfortunately, the reality is that most > dynamic web applications are unaware of these headers and incapable of > setting them or honoring them. At the same time, an access control module > may not want to set a do not cache header on every request because it > might > be acceptable (as described above) for a browser cache to cache a piece of > content but not for mod_cache or a reverse proxy cache. Enterprises must > strike a delicate balance here. This balance is usually reached by using > cache control headers to communicate with the end user and use other > intelligent cooperation for the enterprises internal caching mechanisms. > > Stepping back a little to discuss the hook in general rather than > mod_cache, > I agree strongly with Ryan that it is dangerous and to some extent > conceptually flawed. A well designed API that uses callbacks should allow > people to plug in to do the work that was designed to happen in that hook > without having to worry too much about whether or not some other module > that > plugs in somewhere else is going to circumvent that hook. Even if the > quick_handler hook is being used appropriately and safely by one module, > that does not justify the power given to it.
RE: quick_handler hook is completely bogus.
> From: Bill Stoddard [mailto:[EMAIL PROTECTED]] > > > I'm approaching this from a caching perspective, so when a module uses > > quick_handler for non-caching mechanisms, my comments do not apply but > > here's an option: > > > > FWIW, quick_handler was added to the server to enable more efficient > content > caching. Graham modified mod_proxy to use it (and it seems a reasonable > use > to me). This use alone justifies the existence of the quick_handler hook > IMHO. I must be missing something: C:\opensource\httpd-2.0\modules\proxy>grep "quick_handler" * grep: CVS: Is a directory grep: Debug: Is a directory mod_proxy doesn't use the quick_handler. Ryan
RE: quick_handler hook is completely bogus.
> > 1) If I have a page that I have served and it gets put in the cache, > > then it will be served out of the quick_handler phase. However, if I > > then add or modify a .htaccess file to deny access to that page, > > then my changes won't be honored until the page expires from the > > cache. This is a security hole, because I don't know of anyway to > > invalidate cached pages. (This one if from a conversation with > > wrowe). [ I guess it might be possible to clear the cache with a > > graceful restart. ] > > How does this differ from the document being cached anywhere > else? Such as in squid, or a proxy, or the client's cache? Depending > upon the cache-control fields in the original response header, > the cache engine may not even do a conditional GET. I can accept that argument. Although, from a user's point of view, I would consider them different specifically because in the cache module, everything required to serve the page is in the same place. > > 2) If I have a page that uses access checking to ensure that only > > certain people can request the page, the cache_filter will put it > > in the quick handler. > > I thought the caching modules didn't cache anything that required > either access or auth/authz checking. FirstBill? I read through the code, and I see where the auth/authz decision is made. However, I can't see where the access control decision is made. If it is there, then I would be more than happy to remove this issue. > > 3) It isn't possible for a module author to circumvent the > > quick_handler phase. If I write a module that doesn't want to > > allow the quick_handler phase, for security reasons, I can't > > enforce it. > > How can a module author disallow *any* phase? That's a core > function, not up to modules to decide.. In every other hook in the server, I can generally return some value that makes the phase stop processing, while allowing the request to finish. For many phases, that code is DONE, for others it is OK or DECLINED. With quick_handler, there is nothing I can do. Ryan
quick_handler hook is completely bogus.
I realize that this is a strong statement, but I believe that I can back it up. My reasons for not liking this hook at all: 1) If I have a page that I have served and it gets put in the cache, then it will be served out of the quick_handler phase. However, if I then add or modify a .htaccess file to deny access to that page, then my changes won't be honored until the page expires from the cache. This is a security hole, because I don't know of anyway to invalidate cached pages. (This one if from a conversation with wrowe). [ I guess it might be possible to clear the cache with a graceful restart. ] 2) If I have a page that uses access checking to ensure that only certain people can request the page, the cache_filter will put it in the quick handler. However, the page may not be allowed to people who will request it from the cache. I may be wrong about this one, but I see how the cache disallows pages that require authentication. I do not see how it can disallow caching of pages that require access_checking. 3) It isn't possible for a module author to circumvent the quick_handler phase. If I write a module that doesn't want to allow the quick_handler phase, for security reasons, I can't enforce it. While I understand that we are giving people a lot of rope and asking them to use it wisely, this phase gives too much rope, and invites people to hang themselves. I believe that this hook should be removed, and all content should be served out of the handler phase. If we are looking to remove some request phases, then we should make it possible to avoid individual phases when serving requests, not completely skip all of them. Ryan ------ Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: ldap
> From: Brad Nicholes [mailto:[EMAIL PROTECTED]] > > Being Novell which is the leading provider of directory services, we > obviously have a great interest in LDAP. What if I were able to get the > Novell LDAP developers to step up and support AUTH_LDAP? Would that be > enough to get AUTH_LDAP put back into the mainstream? My own opinion, no it wouldn't. Generally, we require a person who has commit access to the tree to own the module. However, like I said, I have a huge conflict of interests, so I am not standing in the way of this. Ryan
RE: ldap
Mod_proxy wasn't added back to the server until the developers had proven that there was a development community around it, and most of the bugs had been fixed. The same must be true for ldap before it can be added to the base distribution. Also, as a counter-point to this. Adding a module to the core discourages other people from implementing the same basic functionality. While that is usually a good thing, there are a LOT of versions of auth_ldap for 1.3, each with its own advantages and disadvantages. I know of at least 1 other auth_ldap for 2.0 (proprietary, by Covalent), would any of those modules been created if auth_ldap was in the core. Now, I am trying to stay out of this discussion, because I have an obvious conflict of interests, but I did want to give people something to think about. Ryan -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] > -Original Message- > From: Brad Nicholes [mailto:[EMAIL PROTECTED]] > Sent: Monday, July 29, 2002 8:21 AM > To: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: Re: ldap > >I see the same thing happening to LDAP. For the most part it has > been ignored. If it is considered to be unstable at this point, why not > put it in /experimental with the other modules that are considered to be > "not yet ready for prime-time" but still very useful? In this way, it > will get the exposure that it needs, documentation can continue (BTW > where did the docs go??) and when it is stable, it can be moved into the > mainstream. > > Brad > > Brad Nicholes > Senior Software Engineer > Novell, Inc., the leading provider of Net business solutions > http://www.novell.com > > >>> [EMAIL PROTECTED] Monday, July 29, 2002 6:03:26 AM >>> > [EMAIL PROTECTED] wrote: > > > People didn't want it to be a part of the core more because of module > bloat. > > As Aaron says, there is no reason to add all these modules to the > core > > only to have to release them on the same schedule - I like it as a > sub project. > > When proxy was a subproject, it received no exposure, which was > detrimental to the project as a whole. Bugs were only fixed when proxy > > went mainstream again. Subprojects mean more work for end-users, and > avoiding end-user work is better than avoiding developer work. > > Regards, > Graham > -- > - > [EMAIL PROTECTED] > "There's a moon > over Bourbon Street > tonight..."
RE: daedalus is running httpd-2.0.pre40
> > I would be in favor of never installing them on an upgrade. They are > > useless on a production machine that already has a configuration. They > > are meant as DEFAULT values to help people get up and running. > > And they also provide examples of how things are done. When those > things change on an upgrade -- such as adding the wrappers -- > it makes excellent sense for them to be provided. Okay, I keep asking for the REASON for installing these files. I have heard a couple: 1) People want to be able to get back to the standard config files after they have modified them. -- Cool. Back up the files before you modify them. Having us re-install the -std.conf files on upgrades is not a solution to this problem, because it means that you need to re-install to get the default files. 2) People want to be able to compare previous default config files with new ones. -- Fine, but the solution being proposed doesn't allow for this, because the default config file is over written by the upgrade step. The only solution to this is to add the version to the end of the file, which means that we just keep on adding new files. No thanks. 3) People want to see examples of how we think the config should look. -- Fine, but that is an example, and belongs as a part of the manual. Don't install the -std.conf files in conf/ at all if the goal is for them to be examples, put them in the manual directory. If there is another goal, then lets here it, but right now, we are doing something to solve a problem with an external shell script. With all of the input for why this would be a good thing, the solution we are proposing doesn't ACTUALLY solve any of the problem being proposed. > > Is there ANY other software package anywhere that actually re-installs > > the default configuration files on an upgrade? > > Stop handwaving. No-one is suggesting overwriting httpd.conf. Re-read that paragraph. In no place did I say we were. I said we were re-installing the default config files on an upgrade, which means that after I get my conf/ directory down to: httpd.conf and mime.types, after the upgrade, I now have those files plus httpd-std.conf. And httpd-std.conf DOESN'T have any bearing to my site, and it will always be installed. Sorry, that is bogus. Ryan
RE: daedalus is running httpd-2.0.pre40
> From: [EMAIL PROTECTED] [mailto:trawick@rdu88-250- > > Nor do I want spurious -std > > files copying in there to confuse matters. > > Some of us want the -std files though. From time to time I (and That is the difference between developers and users. I want the -std files on my DEVELOPER machines, and I have tricks to get them. I don't want them anywhere near my PRODUCTION machines, because they get in the way. > potentially many other people who start with Apache's basic > httpd.conf) compare httpd.conf and httpd-std.conf to see if I want to > merge in some of the Apache changes. > > I don't think anybody would be in favor of not providing > httpd-std.conf; perhaps the issue is just where to put it when the > user does an install (from binbuild or from make install it should > work the same). I would be in favor of never installing them on an upgrade. They are useless on a production machine that already has a configuration. They are meant as DEFAULT values to help people get up and running. Is there ANY other software package anywhere that actually re-installs the default configuration files on an upgrade? Ryan
RE: daedalus is running httpd-2.0.pre40
> From: gregames [mailto:gregames] On Behalf Of Greg Ames > Ryan Bloom wrote: > > > You are working around a problem in your script in the Apache > > upgrade step. > > no sir. It would have been a piece of cake to change my perl script to > insure > that conf/ has enough stuff so the server can start. But I couldn't take > the > easy way out with a clear conscience because other people are likely to > get > burned in a similar manner. Come off it. That is insane. You have a broken script, fix it. Nobody else will get burned this way, as proof, this is how 1.3 has worked for years, and there were never any complaints. Anybody who has an existing configuration who wants to copy that config to a new server MUST copy all of the config files. You script didn't. That is broken. Period, end of story. > > Now, on EVERY production machine I have, I will always have > > *-std.conf files. > > Another way to word this is that whenever you do a make install your > standard > configuration files are updated to match the current code. Why would I possibly want that? The -std files have no relationship to my server. None. I may have started with them years ago, but my current config doesn't look anything like them. > > Why is that a good thing? > > Most of the time I use daedalus's config files or something that closely > resembles them. Those are complex, but don't include every possible > combination > of config directives (thank goodness). Every once in a while I need to > drop > back to a simpler config to do benchmarking, or something with new config If you are doing benchmarking on deadalus, then we have issues. Since we are talking about your automation scripts on daedalus that is the only thing you could mean, right? > features to work on a problem for instance. It's nice to be able to grab > one of > the -std.conf files, make minimal/no changes, and have a working server > with > guaranteed current config directives in short order. Fine, so grab one, but DON'T put it in MY production conf/ directory please. I don't care where you keep a copy of those files, but they don't belong in my product conf/ directory. > If you read this whole thread, you'll see that I'm not the only one who > likes > having current -std.conf files available. They worked this way for ages. > I > don't recall seeing any complaints about this behavior until yesterday. I don't care how many people like it, it is wrong (besides, if that argument help any water at all, I could point how that in the last few days, there have been at least three users disagreeing with the current code). It has only worked this way since 2.0, and we didn't spend a whole lot of time on the install phases until recently. As for complaints. If there hadn't been at least one (mine!), I wouldn't have changed the install-conf target to match 1.3 weeks ago. Ryan
RE: daedalus is running httpd-2.0.pre40
> From: gregames [mailto:gregames] On Behalf Of Greg Ames > > David Shane Holden wrote: > > > I agree with Ryan wholeheartedly here. > > > > Here's an idea... > > If conf/ exist, copy httpd.conf, magic, and mime.types (These are basic > > files that all conf/ should have, right?). If conf/ does not exist, > copy > > everything. > > uhhh, that clobbers httpd.conf, and they'd tar and feather us for sure. > But if > we leave out that piece, it's close to what's happening now: > > . make a conf/ directory if it doesn't already exist > . if mime.types or magic don't already exist, copy them > . always copy in *-std.conf (httpd-std.conf and ssl-std.conf for now) with > appropriate substitutions for paths and modules etc learned during > ./configure > . look for files whose names match the *-std.conf files copied above with > the > "-std" piece of the name removed (i.e., httpd.conf and ssl.conf). If they > don't > exist, copy them from the corresponding *-std.conf files. Yep, that is the current algorithm, and it doesn't make any sense at all. You are working around a problem in your script in the Apache upgrade step. Once the first installation is done, the conf/ directory doesn't belong to us anymore. If the directory exists, then leave it alone. Now, on EVERY production machine I have, I will always have *-std.conf files. Why is that a good thing? For people with a working server, this change doesn't make sense. Ryan
RE: daedalus is running httpd-2.0.pre40
> > The situation that you are trying to protect against is most likely not > > standard. My question, is why isn't your automation just copying the > > whole conf/ directory? > > Keep in mind that I'm like the proverbial canary in the coal mine. If the > canary suddenly keels over when entering the mine, there's not much point > in > lecturing it to get more exercise, eat healthier, etc. > > I believe that the way make install in HEAD works with conf/ is > satisfactory. I don't, but I am not going to argue anymore. I will simply say that the way things work now, I am going to have a bunch of useless files sitting in the conf/ directory of all of my production machines, because every time I upgrade Apache, I will get all of the files that I have deleted before. The conf/ directory is mine as a user. An initial installation copies some default files around, because that is nice for us to do. Subsequent installations should leave the directory alone, because that directory is mine. The same way we leave the cgi-bin, htdocs, and error directories alone. The only thing an upgrade should do, is to touch binaries and manuals. Everything else is owned by the user. Oh well. Ryan
RE: [PATCH] We have sysvsem on OpenBSD
> From: Aaron Bannert [mailto:[EMAIL PROTECTED]] > > On Wed, Jul 17, 2002 at 10:31:44AM -0400, Jeff Trawick wrote: > > does everybody agree that this is preferable? > > Why isn't this being detected by autoconf? SysV semaphore support > isn't perfect yet and has some problems. Because this is Apache 1.3 which doesn't use autoconf. :-) Ryan
RE: cvs commit: httpd-2.0 acinclude.m4 CHANGES
> From: Aaron Bannert [mailto:[EMAIL PROTECTED]] > > > Look for OpenSSL libraries in /usr/lib64. > ... > > for p in $ap_ssltk_base/lib /usr/local/openssl/lib \ > > - /usr/local/ssl/lib /usr/local/lib /usr/lib /lib; do > > + /usr/local/ssl/lib /usr/local/lib /usr/lib /lib > /usr/lib64; do > > If we just told people to add the right include and lib paths to their > CPPFLAGS and LDFLAGS variables, then we wouldn't have to add any > special case directories to our autoconf routines... *sigh* That would only work if we looked in those variable for the files. But, we don't, so we have to special case this stuff. We should re-write big portions of the detection logic so that we don't have ANY hard-coded paths, but I am not volunteering for that. Ryan
RE: [PATCH] 64bit compiler issues
> From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]] > > At 07:23 PM 7/15/2002, Ryan Bloom wrote: > > >We could force the size, by using apr_int32_t. The problem that he is > >having, is that pointers on _most_ 64-bit machines (Windows is a notable > >exception, there may be others), are 64-bits long. But we are using > >int's, which are 32-bits for the pointers. We have the added problem > >that throughout the code, we pass in integers for void *'s. :-( > > Transposed that statement ;-/ > > Pointers on Win64 are 64 bits long, just like you expected. > > Both int and long remain 32 bits long, unlike what you might expect > (and certainly different from Unix.) Ah. Okay, thanks for correcting me. > When we mean a void*, we need to spell out void*. If we need to > pass it through an integer, prove it, and we will consider an > apr_intptr_t type, that could be nothing more complicated than > a union of the appropriate int type and void* for a given platform. Unfortunately, it isn't something we need to do, just something we have done throughout the code for convenience.:-( Ryan
RE: [PATCH] 64bit compiler issues
> From: Greg Stein [mailto:[EMAIL PROTECTED]] > > On Mon, Jul 15, 2002 at 02:19:06PM +0200, Peter Poeml wrote: > > Hi, > > > > building httpd-2.0.39 on x86_64 (AMD's upcoming 64 bit architecture) > > there are a few compiler warnings, e.g. due to misfitting type casts. > > > > While some of the warnings can be ignored, I believe that the attached > > patch fixes the relevant issues. > > Nope, sorry. All of those values (namespace IDs) are integer sized. Making > them a "long" is just too big. > > What is the specific error that you were seeing? I would guess that the > problem can be fixed a bit differently, without changing the type of those > indexes. We could force the size, by using apr_int32_t. The problem that he is having, is that pointers on _most_ 64-bit machines (Windows is a notable exception, there may be others), are 64-bits long. But we are using int's, which are 32-bits for the pointers. We have the added problem that throughout the code, we pass in integers for void *'s. :-( Just specify the exact size you want, and this problem should go away. Ryan
Re: Faster time_now() [Was: Why not POSIX time_t?]
On Mon, 15 Jul 2002, William A. Rowe, Jr. wrote: > At 10:03 PM 7/14/2002, Ryan Bloom wrote: > > >BTW, this whole conversation started because we wanted to speed up > >Apache. Has anybody considered taking a completely different tack to > >solve this problem? > > > >I know there is a patent on this, but I am willing to ignore it, and I > >am pretty sure that we can get the patent owner to let us use it. (I > >won't say who owns the patent, but if the owner wants to stand up, it > >will be obvious why I say this). We could create a separate thread to > >keep track of the current time every second. That way, the computation > >is completely removed from request processing it is just a memory > >access. On platforms without threads, we just go back to getting the > >time during request processing. That will hurt performance for those > >platforms, but I am not too concerned about that. > > Such code, of course, certainly doesn't belong in apr. That's a higher > level construct that would fit well in http, or any other app that needs > such performance tweaks. The app would have to decide how to live > if it doesn't have threads available. > > And there is the matter of IP :-) Of course this doesn't belong in APR, but the only reason the whole apr_time_t discussion came up was to fix a performance problem in Apache. If that performance problem hadn't come up, would we have even looked at changing the current implementation? Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
RE: content-length filter and r->bytes_sent
> From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]] > > At 02:51 AM 7/14/2002, you wrote: > >Currently, the content-length filter attempts to compute the length > >of the entire response before passing any data on to the next filter, > >and it sets request_rec->bytes_sent to the computed content-length. > > > > * r->bytes_sent is used by mod_log_config as a count of bytes sent > > to the client (minus response header). But the value that's > > computed in the C-L filter isn't necessarily equal to the number > > of bytes sent, as there may be additional filters between the > > C-L filter and the client that affect the output (the chunking > > filter, for example, or mod_deflate). > > AFAICT, > > We do _not_ care about bytes manipulated after body content processing > for r->bytes_sent. If you want to create a new r->bytes_transmitted field > tracking the actually bytes sent over the wire, that's fine. When folks > check their logs, they expect to see the number of bytes of content in > that count, which provides some level of confirmation that the response > was handled properly (if my index.html file is 1474 bytes, I'm looking > that > all 1474 bytes are sent.) > > r->bytes_transmitted would be very interesting to me [looking at SSL > connection traffic, for example.] But it shouldn't replace r->bytes_sent. In 1.3, r->bytes_sent was the number of bytes actually sent to the client. That is what it should continue to be in 2.0. Ryan
RE: Tests failing????
> From: Ian Holsman [mailto:[EMAIL PROTECTED]] > > Pier Fumagalli wrote: > > Ryan Bloom <[EMAIL PROTECTED]> wrote: > > > > > >>I know that there has been some talk of doing a release next week. I > >>just ran my test suite, and we seem to be failing some mod_access > >>checks. I think it has to do with the table re-write, but I haven't > >>looked at it in enough detail to be 100% sure. Is anybody else seeing > >>this problem? > > yes.. I saw the same thing on my gentoo-linux box (which is what you ran > on right) > when I ran it on a more 'popular' linux distribution it ran without > error, so I'm chalking this down to gentoo ;( Okay, that's scary. I'll look into it in more detail now. Ryan
RE: cvs commit: httpd-2.0/server/mpm/beos beos.c
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] > > Modified:server/mpm/beos beos.c > Log: > Adjust the sizes of the pollsets we create/use so that we work again. > > With the poll change we seem to have improved performance. :) This change scares me a bit. The code is supposed to take care of adding the 1 for you. Ryan
Tests failing????
I know that there has been some talk of doing a release next week. I just ran my test suite, and we seem to be failing some mod_access checks. I think it has to do with the table re-write, but I haven't looked at it in enough detail to be 100% sure. Is anybody else seeing this problem? Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA
apr_poll() done.
Thanks to Will doing the work for Windows, the apr_poll() move is done. Most of the current platforms are actually using a select() based implementation, so instead of duplicating logic again, every platform except for OS/2 is using the Unix poll.c file. The only implementation that is likely to work currently is the Unix implementation. The test suite does have a testpoll.c, which can tell you if the implementation is working, although it is hard to determine if the tests were successful. The important lines are the ones that give the socket status. There should be five of these lines, and they should look like: Socket 0Socket 1Socket 2 No wait No wait No wait No wait No wait POLLIN! No wait POLLIN! No wait No wait POLLIN! POLLIN! POLLIN! No wait POLLIN! The other test that you should run is testpipe. Remember that apr_file_read and apr_file_write use apr_wait_for_io_or_timeout, and that now uses apr_poll(). This means that if the apr_poll() implementation doesn't work on your platform, CGI scripts and any other pipe-based request will fail. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA
RE: Apache 2 instruction count profile (head as of ~15:00 EDT July 10)
> From: Bill Stoddard [mailto:[EMAIL PROTECTED]] > > This is with Ryan's poll patch and some of my patches to mod_cache and > mod_mem_cache (which I will publish later on). Unfortuanetely the results > are difficult to compare with earlier results because my test tree was > just > too polluted with patches for Ryan's patch to cleanly apply. Still, the > overall direction is good (7.5% reduction in instruction count). The > difference between Jeff's wait_for_timeout and Ryan's is noise in this > comparison. However, I suspect the apr_poll() is better in general and is > contributing to the improvement. Based on these numbers, I would like to commit the new apr_poll() today. This is likely to break everything but Unix for a few days however. Once the code is committed, there are some obvious performance improvements that can still be made. The first, is to use a static array for small poll sets, or use alloca on systems that have it. The second is to go through Apache and remove the functions that used to setup poll sets. That will actually have an impact on all of the Unix MPMs, because we will finally be able to stop re-inserting the same socket into the pollset for every request. Does anybody have a problem with this plan? I would like to commit late this evening, which will give me enough time to fix Windows tonight (hopefully). Ryan
RE: Port in Makefile.in
You are. That's because we used to do the PORT replacement based on uid. Hm. If I had caught that when I removed Port, this discussion would never have happened. That is enough reason in my mind to add the feature back in. Ryan > We still seem to be pretending we have a Port directive in Apache 2.0. > There's a superfluous -e 's#@@Port@@#$(PORT)#g' handling of the config > files, if I'm reading Makefile.in right.
RE: Port 80 vs 8080 when not SU.
> From: Justin Erenkrantz [mailto:[EMAIL PROTECTED]] > > On Wed, Jul 10, 2002 at 03:12:07PM -0400, Jim Jagielski wrote: > > Have there been any complaints about how 1.3 has been doing it for > > ages? A 'make install; /bin/apachectl start' no matter who does > > the building has "always" resulted in at least a somewhat functional > > server. I don't see the reason for stopping a traditional behavior > > (and a possible expectation from the community) without more > > compelling reasons. > > On the contrary, I don't believe we have an obligation to follow > 1.3 here. I think what we are doing for 2.0 makes the most sense. > > The default is the default. Please don't muddle things based on > who is compiling it. -- Justin While I agree that changing the port based on the user who is compiling the server is a bit backwards, the argument about how things are done on Nagoya is handwaving IMNSHO. We have fixed our installation step to preserve existing config files, so if you compile as a non-root user, and install over the top of an existing installation, your port won't change. This only has to do with how the server is configured the FIRST time the server is installed. Ryan
RE: Auth checker - long term goal..
I still believe that everything that is currently in ROADMAP can and
should be implemented in 2.0.
Ryan
--
Ryan Bloom [EMAIL PROTECTED]
645 Howard St. [EMAIL PROTECTED]
San Francisco, CA
> -Original Message-
> From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, July 10, 2002 10:37 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Auth checker - long term goal..
>
> At 12:07 PM 7/10/2002, Aaron Bannert wrote:
> >On Wed, Jul 10, 2002 at 09:39:29AM -0700, Ryan Bloom wrote:
> > > I'm sorry, but that is completely bogus. If the API needs to
change
> to
> > > make things better, then change the API. Stop trying to open a
new
> dev
> > > branch when the current one is still moving forward quickly. We
have
> > > this discussion every few weeks now, and every few weeks the 2.1
repo
> > > gets shot down, because these changes belong in 2.0.
> >
> >I don't recall any strong opinions on this other than from you and
> >OtherBill.
> >
> >My feeling is somewhere between. We shouldn't rush off and branch 2.1
if
> >we don't have any specific goals to solve, nor should we be forcing
major
> >changes upon our 2.0 users. The point of inflection comes when
someone
> >produces a patch for 2.0 that we aren't quite ready to swallow. As
soon
> >as that happens we have a perfect excuse for a branch.
>
> The list is in ROADMAP. Every item there was effectively vetoed for
the
> current development tree as too radical an overhaul. Each was pointed
to
> "the next version, we are {too close to|already for|already have the
ga}
> release".
> Improve the ROADMAP. Spell out what 2.1/3.0 will offer.
>
> Things like needing to track r->openfile instead of r->filename,
needing
> to
> follow a new convention to write auth modules {splitting authn/authz
into
> smaller useful chunks, but with no back-compat}, proving pushback as a
> more effective authoring and performance filtering model (that
accomodates
> both input and output filters in the same schema), async
cross-threaded
> requests, and so forth.
>
> >S -1 for 2.1 until we have such a patch.
>
> I agree we aren't ready for 2.1 until 2.0 is stable and relatively bug
> free.
> I thought someone a year and a half ago actually threw one out there
> for some of the auth, but I too want the group to stay focused on
making
> 2.0 a serious threat to 1.3 and other servers. Without breaking
existing
> 3rd party modules beyond rebuilding, and occasional API changes, that
> are absolutely required. API changes that break 3rd party 2.0
modules,
> just "because it's better|cooler|faster", are bogus now that we are
GA.
>
> Bill
>
RE: Auth checker - long term goal..
> From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, July 10, 2002 9:31 AM > To: [EMAIL PROTECTED] > Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: RE: Auth checker - long term goal.. > > At 11:22 AM 7/10/2002, Ryan Bloom wrote: > > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] > > > > > > > > > On Wed, 10 Jul 2002, Ryan Bloom wrote: > > > > > > > > user foo" checks. 'require group' can stay in mod_auth or > > > > > go into a mod_auth_group. > > > > > > > > Didn't we decide to take this approach like a year ago? > > > > > > Hmm - been asleep as usual - if so - I'd love to make that split right > > > away ! I feel a patch coming... > > > >I am pretty sure that splitting auth from authz is even listed in the > >STATUS file. > > It is in fact in the ROADMAP file. See there. > > Then feel free to decorate with more details. I agree with Dirk, it's > too much to ask all porters/module authors to rework their entire > stratagem in the production branch. > > It's time for 2.1. We don't need to make .1 a total overhaul, simply > address the things we've wanted for some time, that are likely to > inflict new bugs on 3rd party modules. I'm sorry, but that is completely bogus. If the API needs to change to make things better, then change the API. Stop trying to open a new dev branch when the current one is still moving forward quickly. We have this discussion every few weeks now, and every few weeks the 2.1 repo gets shot down, because these changes belong in 2.0. Ryan
RE: Auth checker - long term goal..
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] > > > On Wed, 10 Jul 2002, Ryan Bloom wrote: > > > > user foo" checks. 'require group' can stay in mod_auth or > > > go into a mod_auth_group. > > > > Didn't we decide to take this approach like a year ago? > > Hmm - been asleep as usual - if so - I'd love to make that split right > away ! I feel a patch coming... I am pretty sure that splitting auth from authz is even listed in the STATUS file. Ryan
RE: Auth checker - long term goal..
> ->Or alternatively mod_auth is split into a mod_auth and a > a mod_require. The first does the File based username > checking; the latter does the "require valid-user", "require > user foo" checks. 'require group' can stay in mod_auth or > go into a mod_auth_group. Didn't we decide to take this approach like a year ago? > With that out of the way; In the long term may want to do the same for > groups checking which now fundamentally must happen in each module as > group findings are not shared. > > I.e. just like we have a r->user have a r->groups field (a table rather > than a char*) which lists the groups the user is part off. (Though I > recognize that there will always be group auth modules which need to to > the reverse; i.e. check if a user is in a list of groups - or, for these > cases, an additional check_group_access handler mirroring the user_access > stage). > > Any thoughds ? Note that part of the above problem is perception causing > the duplication of mod_auth because of the file association. > > Dw > -- > Dirk-Willem van Gulik > > Ad *: We'd need to touch up the error messages of mod_auth a little. > Ad **: Which of course would need another hook... and run method in > apache 3.0. There is no reason to wait for Apache 3.0 to add a new hook. The whole reason for the hook abstraction was to make adding hooks painless for module authors. Ryan
RE: Port 80 vs 8080 when not SU.
> From: Lars Eilebrecht [mailto:[EMAIL PROTECTED]] > > According to Ravindra Jaju: > > > How about an extra echo: > > > > if [ "x`$aux/getuid.sh`" != "x0" -a "x$port" = "x" ]; then > > conf_port="8080" > > echo "Non-root process. Server will run on port $conf_port" > > fi > > +1 The problem with this, is that it gets lost in the noise of autoconf. Whenever we try to put informative messages like this in the configure output, nobody seems it. Ryan
RE: cvs commit: httpd-2.0/server protocol.c
> fix folding when the continuation charater is a blank > Reported by:one of Jeff T's regression test cases Can this be added to the standard test case? Ryan
RE: PATH_INFO in A2?
> From: Justin Erenkrantz [mailto:[EMAIL PROTECTED]] > > On Sat, Jul 06, 2002 at 03:11:20PM -0400, [EMAIL PROTECTED] wrote: > > We just added a new function for all input filters to allow this to be > > done (Justin referenced it in his reply). However, that function > doesn't > > solve the problem, because there should be an ap_filter_is_dynamic(r) > that > > hides the implementation details for all filters. > > I don't believe that mod_include would want AcceptPathInfo on. > Only PHP would. So, I don't know what ap_filter_is_dynamic() would > buy here (other than setting r->no_local_copy to 1). -- Justin ap_filter_is_dynamic wouldn't replace the init function, it would be a simple function that the init functions could call to easily setup the filters. Ryan
RE: conserving file descriptors vs. ap_save_brigade()
> From: Justin Erenkrantz [mailto:[EMAIL PROTECTED]] > > On Sat, Jul 06, 2002 at 08:25:18AM -0700, Ryan Bloom wrote: > > > From: Justin Erenkrantz [mailto:[EMAIL PROTECTED]] > > > Of course, in the common case of a static file with no filters, we > > > already know the content-length (default handler sets it). > > > > > > IIRC, I've brought up skipping the C-L filter when we already know > > > the C-L (as defined by r->headers_out), but that was not met with > > > approval. > > > > Who didn't approve that? I was under the impression that we did skip > > the C-L filter if we already had a C-L, and it was the filters > > responsibility to remove the C-L if it was changing it. > > <[EMAIL PROTECTED]> > > I don't know who might have said that. =) -- Justin Pay attention to what that message says please. It says "The filter should always be in the stack, and it should always collect information." It doesn't say "The filter should always be touching the C-L for the response." We use the data collected by the c-L filter in other places, so we should always try to compute the C-L in the filter. However, we should NEVER buffer unless we absolutely have to. Ryan
RE: PATH_INFO in A2?
> From: Rasmus Lerdorf [mailto:[EMAIL PROTECTED]] > > > > > Make sure you have AcceptPathInfo On > > > > > > Argh! Why the heck is that off by default? > > > > It's on by default for dynamic pages, but there is no way that Apache > > can tell that a page is going to be served by PHP, so it is off for what > > Apache thinks are static pages. > > What is a dynamic page if not a PHP page? Like I said, Apache doesn't know if a file on disk is meant for PHP or not. The best way to fix this would be for mod_php to set the value if the filter is added for the request. I agree, it would be cool if Apache could set this correctly based on the filters that have been added for the request. Ryan
RE: PATH_INFO in A2?
> From: Rasmus Lerdorf [mailto:[EMAIL PROTECTED]] > > On Sat, 6 Jul 2002, Dale Ghent wrote: > > > On Sat, 6 Jul 2002, Rasmus Lerdorf wrote: > > > > | 2.0.40-dev built on June 23rd > > > > Make sure you have AcceptPathInfo On > > Argh! Why the heck is that off by default? It's on by default for dynamic pages, but there is no way that Apache can tell that a page is going to be served by PHP, so it is off for what Apache thinks are static pages. Ryan
RE: conserving file descriptors vs. ap_save_brigade()
> From: Justin Erenkrantz [mailto:[EMAIL PROTECTED]] > > On Sat, Jul 06, 2002 at 01:03:42AM -0700, Brian Pane wrote: > > As it's currently implemented, the C-L filter is trying to compute > > the C-L on everything by default. It only gives up in a few cases: > > Of course, in the common case of a static file with no filters, we > already know the content-length (default handler sets it). > > IIRC, I've brought up skipping the C-L filter when we already know > the C-L (as defined by r->headers_out), but that was not met with > approval. Who didn't approve that? I was under the impression that we did skip the C-L filter if we already had a C-L, and it was the filters responsibility to remove the C-L if it was changing it. Ryan
RE: conserving file descriptors vs. ap_save_brigade()
> From: Brian Pane [mailto:[EMAIL PROTECTED]] > > On Wed, 2002-07-03 at 15:26, Ryan Bloom wrote: > > How big a problem is this really? Most of the time, the content-length > > filter isn't supposed to actually buffer the brigade. It should only be > > doing the buffering if we are doing a keepalive request and we can't do > > chunking. > > I'm seeing the buffering in non-keepalive tests, though. Then we have a bug. The C-L filter should only be trying to compute the C-L if we MUST have one on the response, and we don't have to have one in the non-keepalive case. Ryan
RE: conserving file descriptors vs. ap_save_brigade()
How big a problem is this really? Most of the time, the content-length filter isn't supposed to actually buffer the brigade. It should only be doing the buffering if we are doing a keepalive request and we can't do chunking. My own opinion is that if we are in that situation, we are potentially best served by just turning off keepalives. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA > -Original Message- > From: Brian Pane [mailto:[EMAIL PROTECTED]] > Sent: Thursday, July 04, 2002 11:44 AM > To: [EMAIL PROTECTED] > Subject: conserving file descriptors vs. ap_save_brigade() > > I'm working on a fix to keep file buckets from being mmaped when > they're set aside. (The motivation for this is to eliminate the > bogus mmap+memcpy+munmap that happens when a client requests a > file smaller than 8KB over a keep-alive connection.) > > The biggest problem I've found is that the scalability of the > content-length filter depends on a side-effect of the current > mmap-on-file-setaside semantics. > > Consider the case of an SSI file that includes 10 non-SSI html > files of 500 bytes each. As content streams through the output > filters, the content-length filter sets aside each bucket until > it sees EOS. Currently, the setaside of each of the 10 file > buckets turns the file bucket into an mmap bucket. The good > news is that this keeps us from running out of file descriptors > if a threaded MPM is handling a hundred requests like this at > once. The bad news is that the mmap causes a performance > degradation. > > The solutions I've thought of so far are: > > 1. Change the content-length filter to send its saved brigade >on to the next filter if it contains more than one file >bucket. (This basically would match the semantics of >the core output filter, which starts writing to the socket >if it finds any files at all in the brigade.) > Pros: We can eliminate the mmap for file bucket setaside > and not have to worry about running out of file descriptors. > Cons: We lose the ability to compute the content length > in this example. (I'm so accustomed to web servers not > being able to report a content-length on SSI requests > that this wouldn't bother me, though.) > > 2. Create two variants of ap_save_brigade(): one that does a >setaside of the buckets for use in core_output_filter(), and >another that does a read of the buckets (to force the mmap >of file buckets) for use in ap_content_length_filter(). > Pros: Allows us to eliminate the mmap on small keepalive > requests, and doesn't limit our ability to report a > content-length on SSI requests. > Cons: We'll still have a performance problem for a lot of > SSI requests due to the mmap in the content-length filter. > > > I like option 1 the best, unless anyone can think of a third option > that would work better. > > --Brian >
RE: [Patch] htpasswd doesn't add a newline to the end of an entry
Committed, thanks. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA > -Original Message- > From: Thom May [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, July 02, 2002 10:49 AM > To: HTTPD Dev List > Subject: [Patch] htpasswd doesn't add a newline to the end of an entry > > Uh, as the title really. I guess this is a bug - it certainly isn't the > old > behaviour, as far as I can tell. > Cheers, > -Thom > -- > Thom May -> [EMAIL PROTECTED] > > *sigh* you'd think a distribution composed of 6000 > packages distributed across 13 different architectures (in various stages > between pre-alpha and release quality), maintained by 700 amateurs with > often > conflicting goals who're globally distributed and have rarely met each > other > -- you'd think a distribution like that would be simpler... > > Index: htpasswd.c > === > RCS file: /home/cvspublic/httpd-2.0/support/htpasswd.c,v > retrieving revision 1.49 > diff -u -u -r1.49 htpasswd.c > --- htpasswd.c 19 Jun 2002 17:31:19 - 1.49 > +++ htpasswd.c 2 Jul 2002 17:51:14 - > @@ -236,6 +236,7 @@ > strcpy(record, user); > strcat(record, ":"); > strcat(record, cpw); > +strcat(record, "\n"); > return 0; > } >
RE: How to achieve zero-copy when reading request headers?
> From: Brian Pane [mailto:[EMAIL PROTECTED]] > > On Sun, 2002-06-30 at 20:57, Justin Erenkrantz wrote: > > On Fri, Jun 28, 2002 at 07:01:34PM -0700, Ryan Bloom wrote: > > > Have you looked at which of these are causing the most performance > > > problems? It seems to me that the easiest thing to do, would be to > use > > > a persistent brigade, which removes two steps from this. > > > > I'd really prefer that we decouple this persistent brigade from the > > request_rec. We used to have it coupled (req_cfg->bb) and it made > > the code too confusing as other places started to rely on this > > brigade. So, I'd like to not go back down that road and figure out > > a better scope for such a persistent brigade (a filter context seems > > about right for the scope). > > The brigade can't be in a filter context, because then it wouldn't be > accessible from ap_rgetline_core(). The request_rec is the only place > where we could easily put this brigade. We just need to name it > something like "temp_working_brigade" and document the fact that any > function that puts things in that brigade must clear them out before > it exits. First, please don't put it in the request_rec, it should be placed inside of core_request_config. In fact, if you look at that structure, it already has a brigade specifically for this function. Second, please don't rename that brigade. If people can't read the docs to determine that they aren't supposed to use that brigade, then let them shoot themselves in the foot. In order to find that brigade, they have to read through http_core.h, and they have to define CORE_PRIVATE in their module. Just put a note in the docs that specifically state that 99.9% of module authors shouldn't use that brigade, and that if they do then most likely their filter won't work in all situations. Or, and this is probably better, remove the damned CORE_PRIVATE from our header files. Move those definitions into server/*.h, and stop installing those definitions on user's machines. That will make it FAR more obvious that those structures are private to the core and shouldn't be used by modules. Ryan
RE: How to achieve zero-copy when reading request headers?
> From: Justin Erenkrantz [mailto:[EMAIL PROTECTED]] > > On Fri, Jun 28, 2002 at 07:01:34PM -0700, Ryan Bloom wrote: > > Have you looked at which of these are causing the most performance > > problems? It seems to me that the easiest thing to do, would be to use > > a persistent brigade, which removes two steps from this. > > I'd really prefer that we decouple this persistent brigade from the > request_rec. We used to have it coupled (req_cfg->bb) and it made > the code too confusing as other places started to rely on this > brigade. So, I'd like to not go back down that road and figure out > a better scope for such a persistent brigade (a filter context seems > about right for the scope). No. Please use the core_request_config->bb variable for this. That structure is protected by CORE_PRIVATE. The vector in the request_rec was specifically designed for this kind of thing, so not using it because people don't understand the API is completely bogus. Document the API better if that is the problem. > > Until we know where the performance problems are in the current model I > > believe that it makes little sense to redesign the whole model. I think > > you can stream-line this by just re-using the same brigade for every > > header and by moving the copy down to the HTTP_IN filter. > > HTTP_IN is designed to read the entity - not the headers. HTTP_IN is > only active after the request has been setup and the headers have > been read. At this stage, HTTP_IN isn't active, so I'm slightly > confused what you'd move to HTTP_IN. Yes, you would need to move where HTTP_IN is located in order for the design to work. > As far as connection level filters having access to the request, you > are right that they do. However, I don't believe that the connection > filters should be modifying its request_rec since it may change on > each invocation of the filter and something strikes me as wrong if a > connection filter plays with the request - to me, that means it is a > request filter. Of course they shouldn't be modifying the request_rec directly. I read all of the messages as though you were still going to pass the data up to ap_getline() to do anything with the data, not modify the request_rec inside the filter. -1 on using the filter like that, because it removes any chance for any other input filter to re-write the headers as they are read. Ryan
RE: How to achieve zero-copy when reading request headers?
> From: Brian Pane [mailto:[EMAIL PROTECTED]] > > One of the biggest remaining performance problems in the httpd is > the code that scans HTTP requests. In the current implementation, > read_request_line() and ap_get_mime_headers() call ap_rgetline_core(), > which has to: >- create a temporary brigade >- call ap_get_brigade() to get the next line of input from the > core input filter, which in turn has to scan for LF and split > the bucket >- copy the content into a buffer, >- destroy the temp brigade >- call itself recursively in the (rare) folding case > And all this happens for every line of the request header. Have you looked at which of these are causing the most performance problems? It seems to me that the easiest thing to do, would be to use a persistent brigade, which removes two steps from this. The copying into a buffer is required, because you have no way of knowing how many buckets were used in getting the header line from the client. The only way to solve this, is to do the copy in the HTTP_IN filter. That would work just fine, and would remove the copy in the common case. Then, in the getline() function, you can remove the memcpy all together. I don't really care about the recursive case, because is almost never happens. > We're creating a ton of temporary brigades and temporary buckets > in this process, plus registering and unregistering pool cleanup > callbacks and doing at least one memcpy operation per line. > > I'd like to streamline this processing so that ap_read_request() > can do something like this: > > - get the input brigade from the input filter stack > - loop through the brigade and parse the HTTP header > - Note: in the common case, the whole header will > be in the first bucket No, please don't do this. Currently, it is VERY easy to write a filter that operates on the headers, because you can rely on the code separating the headers into individual brigades. I personally have a filter that modifies the request line, and this model would make that filter MUCH harder to write. > - split the bucket where the request header ends, > remove the bucket(s) containing the header from > the brigade, and hand the remainder of the brigade > back to the filter stack This requires push-back, which has been discussed and vetoed, because it adds a lot of complexity with very little gain. Until we know where the performance problems are in the current model I believe that it makes little sense to redesign the whole model. I think you can stream-line this by just re-using the same brigade for every header and by moving the copy down to the HTTP_IN filter. Ryan
RE: How to achieve zero-copy when reading request headers?
> > stack in ap_read_request() (and remove once we've parsed the > > request header): let this filter do the scanning for the various > > forms of HTTP header, remove the header from the brigade, and > > leave the rest of the brigade for subsequent input filters to > > handle. > > It'd have to be a connection-level filter to ensure that we don't > lose pipelined-requests since we read too much. But, since we are > a connection-level filter, that means you don't have access to > the request. So, you'd have to split it out into two filters: > > - One connection-level filter that will split on a CRLFCRLF and > stash the rest away for the next read on this connection. > - One request-level filter that will expect a CRLFCRLF-segmented > brigade and process the headers and then get out of the way. Connection level filters do have access to the request. I don't understand why you don't think they do. Ryan
RE: 2.0 book
It's being printed now, should be in stores in a week or two. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA > -Original Message- > From: Aryeh Katz [mailto:[EMAIL PROTECTED]] > Sent: Thursday, June 27, 2002 8:18 AM > To: [EMAIL PROTECTED] > Subject: Re: 2.0 book > > > On Thu, 27 Jun 2002, Aryeh Katz wrote: > > > > > I saw that Ryan Bloom is (was) scheduled to release a book on the > > > 2.0 server. Does that book have information for developers, or is it > > > intended for sysadmins? > > > > It definitely has information for developers. Lots of it. :) > > > Do you know when it it's going to be released? > Aryeh > > --- > Aryeh Katz > VASCO > www.vasco.com
RE: core_output_filter buffering for keepalives? Re: Apache 2.0 Numbers
> >>-1 on buffering across requests, because the performance problems > >>caused by the extra mmap+munmap will offset the gain you're trying > >>to achieve with pipelining. > >> > >> > > > >Wait a second. Now you want to stop buffering to fix a completely > >differeny bug. The idea that we can't keep a file_bucket in the brigade > >across requests is only partially true. Take a closer look at what we > >are doing and why when we convert the file to an mmap. That logic was > >added so that we do not leak file descriptors across requests. > > > >However, there are multiple options to fix that. The first, and easiest > >one is to just have the core_output_filter call apr_file_close() after > >it has sent the file. The second is to migrate the apr_file_t to the > >conn_rec's pool if and only if the file needs to survive the request's > >pool being killed. Because we are only migrating file descriptors in > >the edge case, this shouldn't cause a big enough leak to cause a > >problem. > > > > Migrating the apr_file_t to the conn_rec's pool is an appealing > solution, but it's quite dangerous. With that logic added to the > current 8KB threshold, it would be too easy to make an httpd run > out of file descriptors: Send a pipeline of a few hundred requests > for some tiny file (e.g., a small image). They all get setaside > into the conn_rec's pool. Then send a request for something > that takes a long time to process, like a CGI. Run multiple > copies of this against the target httpd at once, and you'll be > able to exhaust the file descriptors for a threaded httpd all > too easily. That is why we allow people to control how many requests can be sent on the same connection. Or, you can just have a limit on the number of file descriptors that you are willing to buffer. And, the pipe_read function should be smart enough that if we don't get any data off of the pipe, for say 30 seconds, then we flush whatever data we currently have. > >>That's not the problem that I care about. The problem that matters > >>is the one that happens in the real world, as a side-effect of the > >>core_output_filter() code trying to be too clever: > >> - Client opens a keepalive connection to httpd > >> - Cclient requests a file smaller than 8KB > >> - core_output_filter(), upon seeing the file bucket followed by EOS, > >>decides to buffer the output because it has less than 8KB total. > >> - There isn't another request ready to be read (read returns EAGAIN) > >>because the client isn't pipelining its connections. So we then > >>do the writev of the file that we've just finished buffering. > > > >But, this case only happens if the headers + the file are less than 8k. > >If the file is >10k, then this problem doesn't actually exist at all. > >As I said above, there are better ways to fix this than removing all > >ability to pipeline responses. > > > > We're not removing the ability to pipeline responses. You are removing a perfectly valid optimization to stop us from sending a lot of small packets across pipelined responses. Ryan
RE: core_output_filter buffering for keepalives? Re: Apache 2.0 Numbers
> From: Bill Stoddard [mailto:[EMAIL PROTECTED]] > > > From: Brian Pane [mailto:[EMAIL PROTECTED]] > > > Ryan Bloom wrote: > > > >>From: Brian Pane [mailto:[EMAIL PROTECTED]] > > > >>Ryan Bloom wrote: > > Wait a second. Now you want to stop buffering to fix a completely > > differeny bug. The idea that we can't keep a file_bucket in the brigade > > across requests is only partially true. Take a closer look at what we > > are doing and why when we convert the file to an mmap. That logic was > > added so that we do not leak file descriptors across requests. > > > > However, there are multiple options to fix that. The first, and easiest > > one is to just have the core_output_filter call apr_file_close() after > > it has sent the file. The second is to migrate the apr_file_t to the > > conn_rec's pool if and only if the file needs to survive the request's > > pool being killed. Because we are only migrating file descriptors in > > the edge case, this shouldn't cause a big enough leak to cause a > > problem. > > > > > >You are trying to solve a problem that doesn't exist in the real > > world > > > >IIUIC. Think it through. The problem is that if the page is <1k and > > > >you request that page 8 times on the same connection, then Apache > > will > > > >buffer all of the responses. How often are those conditions going to > > > >happen in the real world? > > > > > > > > > > That's not the problem that I care about. The problem that matters > > > is the one that happens in the real world, as a side-effect of the > > > core_output_filter() code trying to be too clever: > > > - Client opens a keepalive connection to httpd > > > - Cclient requests a file smaller than 8KB > > > - core_output_filter(), upon seeing the file bucket followed by EOS, > > > decides to buffer the output because it has less than 8KB total. > > > - There isn't another request ready to be read (read returns EAGAIN) > > > because the client isn't pipelining its connections. So we then > > > do the writev of the file that we've just finished buffering. > > > > But, this case only happens if the headers + the file are less than 8k. > > If the file is >10k, then this problem doesn't actually exist at all. > > As I said above, there are better ways to fix this than removing all > > ability to pipeline responses. > > > > > Aside from slowing things down for the user, this hurts the > > scalability > > > of the httpd (mmap and munmap don't scale well on multiprocessor > > boxes). > > > > > > What we should be doing in this case is just doing the write > > immediately > > > upon seeing the EOS, rather than penalizing both the client and the > > > server. > > > > By doing that, you are removing ANY benefit to using pipelined requests > > when serving files. Multiple research projects have all found that > > pipelined requests show a performance benefit. In other words, you are > > fixing a performance problem by removing another performance enhancer. > > > > Ryan > > > > Ryan, > Solve the problem to enable setting aside the open fd just long enough to > check for a > pipelined request will nearly completely solve the worst part (the > mmap/munmap) of this > problem. On systems with expensive syscalls, we can do browser detection > and dynamically > determine whether we should attempt the pipelined optimization or not. Not > many browsers > today support pipelining requests, FWIW. That would be a trivial change. I'll have a patch posted for testing later today. Ryan
RE: core_output_filter buffering for keepalives? Re: Apache 2.0 Numbers
> From: Brian Pane [mailto:[EMAIL PROTECTED]] > Ryan Bloom wrote: > > >>From: Brian Pane [mailto:[EMAIL PROTECTED]] > >> > >>Ryan Bloom wrote: > >> > >> > >> > >>>I think we should leave it alone. This is the difference between > >>>benchmarks and the real world. How often do people have 8 requests > >>> > >>> > >in a > > > > > >>>row that total less than 8K? > >>> > >>>As a compromise, there are two other options. You could have the > >>>core_output_filter refuse to buffer more than 2 requests, or you > >>> > >>> > >could > > > > > >>>have the core_output_filter not buffer if the full request is in the > >>>buffer. > >>> > >>> > >>> > >>In your second option, do you mean "full response" rather than "full > >>request"? If so, it's equivalent to what I was proposing yesterday: > >>send the response for each request as soon as we see EOS for that > >> > >> > >request. > > > > > >>I like that approach a lot because it keeps us from doing an extra > >>mmap/memcpy/memunmap write before the write in the real-world case > >>where a client sends a non-pipelined request for a small (<8KB) file > >>over a keepalive connection. > >> > >> > > > >I do mean full response, and that is NOT equivalent to sending the > >response as soon as you see the EOS for that request. EVERY request > >gets its own EOS. If you send the response when you see the EOS, then > >you will have removed all of the buffering for pipelined requests. > > > > -1 on buffering across requests, because the performance problems > caused by the extra mmap+munmap will offset the gain you're trying > to achieve with pipelining. Wait a second. Now you want to stop buffering to fix a completely differeny bug. The idea that we can't keep a file_bucket in the brigade across requests is only partially true. Take a closer look at what we are doing and why when we convert the file to an mmap. That logic was added so that we do not leak file descriptors across requests. However, there are multiple options to fix that. The first, and easiest one is to just have the core_output_filter call apr_file_close() after it has sent the file. The second is to migrate the apr_file_t to the conn_rec's pool if and only if the file needs to survive the request's pool being killed. Because we are only migrating file descriptors in the edge case, this shouldn't cause a big enough leak to cause a problem. > >You are trying to solve a problem that doesn't exist in the real world > >IIUIC. Think it through. The problem is that if the page is <1k and > >you request that page 8 times on the same connection, then Apache will > >buffer all of the responses. How often are those conditions going to > >happen in the real world? > > > > That's not the problem that I care about. The problem that matters > is the one that happens in the real world, as a side-effect of the > core_output_filter() code trying to be too clever: > - Client opens a keepalive connection to httpd > - Cclient requests a file smaller than 8KB > - core_output_filter(), upon seeing the file bucket followed by EOS, > decides to buffer the output because it has less than 8KB total. > - There isn't another request ready to be read (read returns EAGAIN) > because the client isn't pipelining its connections. So we then > do the writev of the file that we've just finished buffering. But, this case only happens if the headers + the file are less than 8k. If the file is >10k, then this problem doesn't actually exist at all. As I said above, there are better ways to fix this than removing all ability to pipeline responses. > Aside from slowing things down for the user, this hurts the scalability > of the httpd (mmap and munmap don't scale well on multiprocessor boxes). > > What we should be doing in this case is just doing the write immediately > upon seeing the EOS, rather than penalizing both the client and the > server. By doing that, you are removing ANY benefit to using pipelined requests when serving files. Multiple research projects have all found that pipelined requests show a performance benefit. In other words, you are fixing a performance problem by removing another performance enhancer. Ryan
RE: core_output_filter buffering for keepalives? Re: Apache 2.0 Numbers
> From: Brian Pane [mailto:[EMAIL PROTECTED]] > > Ryan Bloom wrote: > > >I think we should leave it alone. This is the difference between > >benchmarks and the real world. How often do people have 8 requests in a > >row that total less than 8K? > > > >As a compromise, there are two other options. You could have the > >core_output_filter refuse to buffer more than 2 requests, or you could > >have the core_output_filter not buffer if the full request is in the > >buffer. > > > > In your second option, do you mean "full response" rather than "full > request"? If so, it's equivalent to what I was proposing yesterday: > send the response for each request as soon as we see EOS for that request. > I like that approach a lot because it keeps us from doing an extra > mmap/memcpy/memunmap write before the write in the real-world case > where a client sends a non-pipelined request for a small (<8KB) file > over a keepalive connection. I do mean full response, and that is NOT equivalent to sending the response as soon as you see the EOS for that request. EVERY request gets its own EOS. If you send the response when you see the EOS, then you will have removed all of the buffering for pipelined requests. You are trying to solve a problem that doesn't exist in the real world IIUIC. Think it through. The problem is that if the page is <1k and you request that page 8 times on the same connection, then Apache will buffer all of the responses. How often are those conditions going to happen in the real world? Now, take it one step further please. The real problem is how AB is measuring the results. If I send 3 requests, and Apache buffers the response, how is AB measuring the time? Does it start counting at the start of every request, or is the time just started at start of the first request? Perhaps a picture: Start of requestresponse received 0 5 10 15 20 25 30 35 ---> request 1 ---> request 2 -> request 3 --> request 4 Did the 4 requests take 35 seconds total, or 85 seconds? I believe that AB is counting this as 85 seconds for 4 requests, but Apache is only taking 35 seconds for the 4 requests. Ryan
RE: state of mpm perchild
There is no date set, because this is all volunteer work. It will not be production quality for a number of months. I hope to have it working again by the end of the week. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA -Original Message- From: Nick De Decker [mailto:[EMAIL PROTECTED]] Sent: Monday, June 24, 2002 10:15 AM To: [EMAIL PROTECTED] Subject: state of mpm perchild Hi guys, Any progress on the mpm perchild for linux ? I'm setting up a production webserver and really would like to use this mpm. Any date set ? Regards Nick De Decker
Re: Query: bugs 8712 and 10156
That is just Ken's way of getting stuff onto the development list and stating that he hasn't done anything with it yet. :-) Ryan On 24 Jun 2002, Larry Rosenman wrote: > On Mon, 2002-06-24 at 11:36, Rodent of Unusual Size wrote: > > No acked. > > Huh? > > > > -- > > #kenP-)} > > > > Ken Coar, Sanagendamgagwedweinini http://Golux.Com/coar/ > > Author, developer, opinionist http://Apache-Server.Com/ > > > > "Millennium hand and shrimp!" > > > > > > > From: Larry Rosenman <[EMAIL PROTECTED]> > > To: [EMAIL PROTECTED] > > Subject: Query: bugs 8712 and 10156 > > Date: 23 Jun 2002 15:19:31 -0500 > > > > I submitted 8712 a month or more ago, and have gotten NO feedback at > > all. > > > > I just submitted 10156 and wonder what it would take to get the patch > > into the next release. > > > > LER > > -- > > Larry Rosenman http://www.lerctr.org/~ler > > Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED] > > US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749 > > > > > > ----- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
RE: Apache 2.0 Numbers
> From: Rasmus Lerdorf [mailto:[EMAIL PROTECTED]] > > > As it happens, DONE is defined to be -2. :-) > > Ok, I will use that, but 'DONE' doesn't really give the impression of > being a fatal error return value. I know. It's original use was for use during request processing, when a module wanted to be sure that it was the last function run for a specific hook. Basically this value ensured that no other functions were run for that hook. Ryan
RE: Apache 2.0 Numbers
> From: Rasmus Lerdorf [mailto:[EMAIL PROTECTED]] > > On Mon, 24 Jun 2002, Cliff Woolley wrote: > > On Mon, 24 Jun 2002, Rasmus Lerdorf wrote: > > > > > Hrm.. Nope. doing 'return DECLINED' from the post_config phase does > not > > > stop the server from starting. I have this: > > > > I thought you were supposed to return HTTP_INTERNAL_SERVER_ERROR. > > In include/http_config.h it says: > > /** > * Run the post_config function for each module > * @param pconf The config pool > * @param plog The logging streams pool > * @param ptemp The temporary pool > * @param s The list of server_recs > * @return OK or DECLINED on success anything else is a error > */ > > So I guess I need to return 'anything else' > > Trying this, ie. returning -2 it does the job. But this seems a little > vague. Should we perhaps have a #define FATAL -2 or something similar so > I don't get stepped on later on if someone decides to use -2 for something > else? As it happens, DONE is defined to be -2. :-) Ryan
RE: Apache 2.0 Numbers
> From: Cliff Woolley [mailto:[EMAIL PROTECTED]] > > On Mon, 24 Jun 2002, Rasmus Lerdorf wrote: > > > Hrm.. Nope. doing 'return DECLINED' from the post_config phase does > not > > stop the server from starting. I have this: > > I thought you were supposed to return HTTP_INTERNAL_SERVER_ERROR. No. That implies that you have an actual HTTP error. You don't, this is during config processing, not request processing. Yes, that value will work, but it is incorrect semantically. Ryan
RE: Apache 2.0 Numbers
My bad. Post_config is a run_all. If you return DONE the server won't
start. This is what the MPMs do if the socket is already taken.
Ryan
--
Ryan Bloom [EMAIL PROTECTED]
645 Howard St. [EMAIL PROTECTED]
San Francisco, CA
> -Original Message-
> From: Rasmus Lerdorf [mailto:[EMAIL PROTECTED]]
> Sent: Monday, June 24, 2002 8:34 AM
> To: Ryan Bloom
> Cc: [EMAIL PROTECTED]
> Subject: RE: Apache 2.0 Numbers
>
>
>
> On Mon, 24 Jun 2002, Rasmus Lerdorf wrote:
>
> > > > What is the correct way to fail in a filter post_config? Do I
> return
> > > -1
> > > > from it if my filter finds a fatal error? I can't use
> ap_log_rerror()
> > > at
> > > > this point, right? How would I log the reason for the failure?
> > >
> > > I'm confused by the question, but I'll try to answer. If you mean
the
> > > post_config phase, then you can use ap_log_error or ap_log_perror.
If
> > > you want to stop the server from starting, just return DECLINED.
> >
> > Right, I found ap_log_error. It was the return value I was looking
for.
> > None of the example filter modules had a fatal error check at the
config
> > phase. So returning a -1 is the correct way to stop the server from
> > starting. Thanks.
>
> Hrm.. Nope. doing 'return DECLINED' from the post_config phase does
not
> stop the server from starting. I have this:
>
> static int
> php_apache_server_startup(apr_pool_t *pconf, apr_pool_t *plog,
apr_pool_t
> *ptemp, server_rec *s)
> {
> void *data = NULL;
> const char *userdata_key = "apache2filter_post_config";
> #ifndef ZTS
> int threaded_mpm;
>
> ap_mpm_query(AP_MPMQ_IS_THREADED, &threaded_mpm);
> if(threaded_mpm) {
> ap_log_error(APLOG_MARK, APLOG_CRIT, 0, s, "Apache is running
a
> threaded MPM, but your PHP Module is not compiled to be threadsafe.
You
> need to recompile PHP.");
> return DECLINED;
> }
> #endif
> ...
> }
>
> ...
>
> ap_hook_pre_config(php_pre_config, NULL, NULL, APR_HOOK_MIDDLE);
> ap_hook_post_config(php_apache_server_startup, NULL, NULL,
> APR_HOOK_MIDDLE);
>
> And in my log I get:
>
> [Mon Jun 24 08:27:23 2002] [crit] Apache is running a threaded MPM,
but
> your PHP Module is not compiled to be threadsafe. You need to
recompile
> PHP.
> [Mon Jun 24 08:27:23 2002] [crit] Apache is running a threaded MPM,
but
> your PHP Module is not compiled to be threadsafe. You need to
recompile
> PHP.
> [Mon Jun 24 08:27:23 2002] [notice] Apache/2.0.40-dev (Unix)
configured --
> resuming normal operations
RE: Apache 2.0 Numbers
> From: Rasmus Lerdorf [mailto:[EMAIL PROTECTED]] > > > Runtime is harder, but you can just use ap_mpm_query to get the MPMs > > characteristics. This won't give you the MPM name, but it will let you > > know if the MPM is threaded or not. > > What is the correct way to fail in a filter post_config? Do I return -1 > from it if my filter finds a fatal error? I can't use ap_log_rerror() at > this point, right? How would I log the reason for the failure? I'm confused by the question, but I'll try to answer. If you mean the post_config phase, then you can use ap_log_error or ap_log_perror. If you want to stop the server from starting, just return DECLINED. Ryan
RE: Apache 2.0 Numbers
> From: Rasmus Lerdorf [mailto:[EMAIL PROTECTED]] > > > > > It would be nice > > > > if there was an apxs flag that would return the MPM type. > > > > > > +1 > > > > There is. -q will query for any value in config_vars.mk, and MPM_NAME > > is in that file. So `apxs -q MPM_NAME` will return the configured MPM > > type. > > Ah right. Is there a way to check at runtime as well? I've added a PHP > configure check now to the apache2filter sapi module so it will come up > non-threaded by default if it sees Apache2-prefork. Just a bit worried > about someone changing their MPM after the fact, so perhaps a runtime > check is needed as well. Runtime is harder, but you can just use ap_mpm_query to get the MPMs characteristics. This won't give you the MPM name, but it will let you know if the MPM is threaded or not. Ryan
RE: Query: bugs 8712 and 10156
> From: Justin Erenkrantz [mailto:[EMAIL PROTECTED]] > > At some point, Larry Rosenman <[EMAIL PROTECTED]> excited the electrons: > > I submitted 8712 a month or more ago, and have gotten NO feedback at > > all. > > FreeBSD is packaging their version with mod_ssl. We don't include > mod_ssl with 1.3. We have no control over the configuration that > FreeBSD decides to use. I would refer back to the FreeBSD port > maintainer and help them come up with a configuration that isn't > susceptable to this problem. I'm not really sure how we can fix > this problem considering that the ASF has no control over mod_ssl > or FreeBSD's configurations that they use. The ASF distribution > does not contain in its default configuration. > > As a hint to the FreeBSD maintainer, the idea here is to always > load the mod_ssl library and then conditionally execute the SSL > directives. I do not believe that loading DSOs conditionally is > what we intended with the construct. Actually loading the module inside the is exactly what was meant. The bug here is in apxs not FreeBSD. I considered answering this PR the same way you did Justin, but it is our problem. The problem is how apxs is adding modules. Currently, apxs looks for the last LoadModule in the config file, and just adds a new line after it. If the last module you load in your config is mod_ssl, then you are susceptible to this problem. The solution is to have a line like: #LoadModule goes here and always key off of that line. Ryan
RE: worker MPM shutdown
> From: Brian Pane [mailto:[EMAIL PROTECTED]] > > On Sat, 2002-06-22 at 17:01, Ryan Bloom wrote: > > I believe that the problem is platform specific. The reason that loop > > was added, was to allow for graceless shutdown on linux. On non-linux > > platforms, killing the main thread kills the whole process, but on linux > > this doesn't work. The point of closing the sockets was to force the > > worker threads to finish ASAP so that the process could die. > > Why not just pthread_kill() all the workers if we're running > on Linux? That would be a much more direct way of stopping > them than closing the sockets. Because asynchronous cancellation of threads sucks, and APR doesn't support it. Most OSes leak memory when you use pthread_kill(). Ryan
RE: core_output_filter buffering for keepalives? Re: Apache 2.0 Numbers
I think we should leave it alone. This is the difference between benchmarks and the real world. How often do people have 8 requests in a row that total less than 8K? As a compromise, there are two other options. You could have the core_output_filter refuse to buffer more than 2 requests, or you could have the core_output_filter not buffer if the full request is in the buffer. Removing the buffering is not the correct solution, because it does have a negative impact in the real world. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA > -Original Message- > From: Brian Pane [mailto:[EMAIL PROTECTED]] > Sent: Sunday, June 23, 2002 9:38 PM > To: [EMAIL PROTECTED] > Subject: core_output_filter buffering for keepalives? Re: Apache 2.0 > Numbers > > On Sun, 2002-06-23 at 20:58, Brian Pane wrote: > > > For what it's worth, I just tried the test case that you posted. On my > > test system, 2.0 is faster when I run ab without -k, and 1.3 is faster > > when I run with -k. > > I studied this test case and found out why 2.0 runs faster in the > non-keepalive case and slower in the non-keepalive case. It's > because of the logic in core_output_filter() that tries to avoid > small writes when c->keepalive is set. In Rasmus's test, the file > size is only 1KB, so core_output_filter reads in and buffers the > contents of 8 requests before it finally reaches the 8KB threshold > and starts calling writev. 1.3, in contrast, writes out each > request's response immediately, with no buffering. > > I'm somewhat inclined to remove the buffering in this case, and > let core_output_filter send the response as soon as it sees EOS > for each request, even if that means sending a small packet. I > just tried this in my working tree, and it does speed up 2.0 for > this particular test case. On the other hand, I'm not a fan of > small write calls in general. > > Anyone else care to offer an opinion on whether we should remove > the buffering in this situation? > > --Brian >
RE: Apache 2.0 Numbers
> > It would be nice > > if there was an apxs flag that would return the MPM type. > > +1 There is. -q will query for any value in config_vars.mk, and MPM_NAME is in that file. So `apxs -q MPM_NAME` will return the configured MPM type. Ryan
RE: CAN-2002-0392 : what about older versions of Apache?
> From: Aaron Bannert [mailto:[EMAIL PROTECTED]] > > On Sun, Jun 23, 2002 at 05:09:05PM -0700, Roy Fielding wrote: > > I have re-uploaded a patch to fix the problem on all versions of > > httpd 1.2.0 through 1.3.22. This time I added the four lines that > > check for a negative return value from atol, even though there has > > been no evidence of any such error in the standard C libraries. > > > > To the person who deleted my prior patch: You just wasted > > my Sunday afternoon. Even if the patch didn't, by some stretch of > > your imagination, suffice for the broken atol case, you prevented > > people from protecting themselves against a published exploit script > > that doesn't even use content-length as an attack. Do not remove > > my patch unless you replace it with a better fix that is known to > > apply for that version and compile on all platforms. > > > > -1 to any additions of ap_strtol to prior versions of Apache. > > That introduced more problems than it fixed. There is no reason > > to work around the operating system when a simple fix to our own > > code is necessary and sufficient to solve the problem. > > > I don't remember seeing any +1's for this patch on the list. > > Please remove this patch until one can be made that addresses the same > issues with the proxy code (which also uses get_chunk_size()). The proxy didn't use that code until it supported HTTP 1.1, which didn't happen until 1.3.24. Roy is right, removing this patch is completely bogus. Ryan
RE: worker MPM shutdown
I believe that the problem is platform specific. The reason that loop was added, was to allow for graceless shutdown on linux. On non-linux platforms, killing the main thread kills the whole process, but on linux this doesn't work. The point of closing the sockets was to force the worker threads to finish ASAP so that the process could die. Ryan > From: Brian Pane [mailto:[EMAIL PROTECTED]] > > During the worker MPM non-graceful shutdown, the signal_threads() > function attempts to close all open sockets. > > I have two major objections to this: > > 1) It's not necessarily safe to close a socket that another thread >is using. Note that apr_socket_close() calls the pool cleanup >on the pool from which the socket was allocated--bad news if >one of the active worker threads happens to be, say, registering >a new cleanup in the same pool at the same time. > > 2) It appears to be contributing to the fact that non-graceful >shutdown doesn't work. Without the socket shutdown loop, >the child processes shut down promptly. > > As I understand it, the motivation for closing the sockets during > shutdown was to try to fix a race condition in which an active > worker thread might be trying to write to a socket at the same > time that the underlying pool was being destroyed (due to the > recursive destruction of APR's global_pool during apr_terminate()). > > If so, I think the right solution is to add a way to create > parentless pools that aren't implicitly added as children to > the global pool, so that a worker thread's pool won't disappear > before that thread does. Is there any specific reason why we're > not doing this already? > > Thanks, > --Brian >
RE: cvs commit: httpd-2.0/modules/generators mod_cgi.c mod_cgid.c
> From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]] > Sent: Saturday, June 22, 2002 9:45 AM > To: [EMAIL PROTECTED] > Subject: Re: cvs commit: httpd-2.0/modules/generators mod_cgi.c mod_cgid.c > > Looking for both faults (passing the next filter when we are doing > lookups, > and passing NULL when we are going to serve the content) ... it really > seems that; > > D:\clean\httpd-2.0\modules\mappers\mod_negotiation.c(2550): > sub_req = ap_sub_req_lookup_file(variant->file_name, r, NULL); > > SHOULD be setting the next filter. Anyone want to dig on this? I am missing something. The third argument has nothing to do with whether the request will be served. It has to do with how the filter stack is initialized in the sub-request. If the argument is NULL, the sub-requests filters are set to the protocol filters of the original request. If the argument isn't NULL, then the sub-request's filters are initialized to the filter passed in. In either case, the data can either be sent or not. Please revert the doc portions of your last change, they are incorrect. The code is safe to leave, because the requests aren't actually served. However, mod_negotiation is 100% correct, and should NOT be changed. Ryan > > Bill > > >wrowe 2002/06/22 09:32:45 > > > > Modified:include http_request.h > >modules/generators mod_cgi.c mod_cgid.c > > Log: > > Note the changed meaning of the NULL next_filter argument to the > > ap_sub_req_lookup() family, and fix a few oddball cases (those are, > > PATH_TRANSLATED reference issues.) >
RE: 2.0.38-39 lockup problem ?
> From: gregames [mailto:gregames] On Behalf Of Greg Ames > > "Paul J. Reder" wrote: > > > > This looks exactly like the problem that Allan and I ran into when you > > tried to send a request to http://foo.bar.org:443 (i.e. insecure request > > over the secure port). It tried to generate an error and went into an > > infinte loop. > > Can you try that with current HEAD and let us know what happens? While the problems were similar, they were not caused by the same code, and the SSL problem would not have been touched at all by this patch. The problem with mod_ssl was fixed a while ago, and it never touched the ap_die code. Basically, in mod_ssl's post_read_request phase, we check for a flag to determine if the request was HTTP over the HTTPS port. The problem is that if we find the flag, then we do an internal redirect, which also runs the post_read_request function. Because the flag wasn't cleared, we just returned another error, which caused another internal_redirect, which ran the same phase, etc. Ryan
Re: Linux and the worker model
They should both be there. The version in mpm_common is for MOST mpms, but worker uses a different set of functions. If you worker MPM headers are actually defining AP_MPM_USES_POD, then there is a problem with the source. The version in CVS does not define this for worker. Ryan On Thu, 20 Jun 2002, Jean-Jacques Clar wrote: > I am building 2.0.39, but it was the same thing with 2.0.35. > > >>> [EMAIL PROTECTED] 06/20/02 10:26AM >>> > On Thu, Jun 20, 2002 at 09:09:45AM -0600, Jean-Jacques Clar wrote: > > There are errors when building 2.0.x on Linux RH7.3 with the worker > > Which version of 2.0 exactly? > > -aaron > -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
RE: I asked this on the freebsd-general list...
To the best of my knowledge, we have told FreeBSD that their threading support doesn't work with Apache. In general, we have found inconsistent return values with a lot of the networking APIs when threading was enabled. There are rumors that these are fixed in 5.0, but I haven't tested it. Suffice to say, we worked for a long time to get apache.org to run a threaded MPM, and we always found that pages were being truncated for no reason. After digging into it, we found that the problem was the freebsd network layer when threads were enabled. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA > -Original Message- > From: Rick Kukiela [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, June 18, 2002 8:44 AM > To: apache-dev > Subject: I asked this on the freebsd-general list... > > In the last episode (Jun 17), Rick Kukiela said: > > I asked on the apache-dev list about when perchild processing will work > on > > freebsd and this was the response i got: > > > > "It will never work with FreeBSD 4.6. Perchild requires a good threading > > library, and FreeBSD doesn't have one." > > > > What is being done about this? > > THEIR REPLY: > Unless they provide more details (i.e. proof that it's bad), there's > not much that can be done. The userland threads system is good enough > to run MySQL, mozilla, openoffice, and who knows how many other pieces > of software. > > -- > Dan Nelson > [EMAIL PROTECTED] > > > So what's the deal here? can some one tell them their threads suck so it > can > be fixed? > > Thanks > Rick >
RE: cvs commit: httpd-2.0/support htpasswd.c
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] > > rbb 2002/06/16 08:52:16 > > Modified:.CHANGES >support htpasswd.c > Log: > Finish the htpasswd port to APR. This brings the file checking code to > APR. I am done changing htpasswd, but I should mention that I severely dislike the current file checking code. For people who don't know, we do a series of checks at the beginning of running the code to determine if we can read/write/update the password file before we ever actually try to open it. IMHO, that is completely incorrect. We should do the checks while opening the file, and fail cleanly. As things stand now, if we want to add more file checks, that logic is just going to get uglier and uglier, and there is the real chance that people will forget to update those checks if they add features to htpasswd. Unfortunately, the code is somewhat ugly to read, and I don't have anymore time this morning. If I have time when I get home tonight, I will look at refactoring the code to flow a bit cleaner. However, if anybody would like to beat me to it, feel free. Ryan
RE: cvs commit: httpd-2.0/docs/error/include bottom.html
> From: Aaron Bannert [mailto:[EMAIL PROTECTED]] > > On Sat, Jun 15, 2002 at 11:02:18AM -0400, Joshua Slive wrote: > > [EMAIL PROTECTED] wrote: > > >rbb 2002/06/15 00:01:25 > > > > > > Modified:docs/error/include bottom.html > > > Log: > > > Comment out the SERVER_STRING variable from our default error > documents. > > > Some people do not like having this information in their error pages, > and > > > it makes sense to not do it by default. If users want this back, > they > > > can uncomment it. > > > > > > PR: 9319 > > > > Personally, I think this is silly. The server signature on error pages > > is there for a good reason: helping people debug problems, especially > > with requests that pass through proxies, etc. > > I agree, and the same logic above applies in reverse: > > If an admin doesn't want to reveal the server string in the > error document, they can remove that part themselves. With one major difference. We provide server configuration directives to stop this stuff from being displayed. Whether they are correct or not, many admins do believe that they are improving security by not exposing this information. The problem is that you can change the config and not affect the default error pages that we ship. If you want to get the information, then it is easy to add back. However, I would simply suggest that the default error documents should not be included in the default config. Include the files, comment the config, and this issue goes away. As things stand right now, most admins have no clue that we have replaced the default Apache error documents which is why putting information that they _may_ want to keep private in them is completely wrong. Ryan
RE: cvs commit: httpd-2.0/modules/mappers mod_userdir.c
> From: Aaron Bannert [mailto:[EMAIL PROTECTED]]
>
> On Sat, Jun 15, 2002 at 07:20:59AM -, [EMAIL PROTECTED] wrote:
> > 1.50 +5 -5 httpd-2.0/modules/mappers/mod_userdir.c
> >
> > Index: mod_userdir.c
> >
===
> > RCS file: /home/cvs/httpd-2.0/modules/mappers/mod_userdir.c,v
> > retrieving revision 1.49
> > retrieving revision 1.50
> > diff -u -r1.49 -r1.50
> > --- mod_userdir.c 31 May 2002 16:59:13 - 1.49
> > +++ mod_userdir.c 15 Jun 2002 07:20:59 - 1.50
> > @@ -347,12 +347,12 @@
> > r->pool)) == APR_SUCCESS
> > || rv ==
> APR_INCOMPLETE))) {
> >r->filename = apr_pstrcat(r->pool, filename, dname,
> NULL);
> > - /* XXX: Does this walk us around FollowSymLink rules?
> > +/* XXX: Does this walk us around FollowSymLink rules?
> > * When statbuf contains info on r->filename we can
save
> a syscall
> > - * by copying it to r->finfo
> > - */
> > - if (*userdirs && dname[0] == 0)
> > - r->finfo = statbuf;
> > + * by copying it to r->finfo
> > + */
> > +if (*userdirs && dname[0] == 0)
> > +r->finfo = statbuf;
> >
> >/* For use in the get_suexec_identity phase */
> >apr_table_setn(r->notes, "mod_userdir_user", w);
>
> Is this anything more than an intentation change, and if so did you
mean
> to include it in the same commit with the shtml AddType fix?
It is just indentation, and no I didn't mean to include it, but since it
is safe, I left it there.
Ryan
RE: questions from my trivial patch
> From: Brian Degenhardt [mailto:[EMAIL PROTECTED]] > > The functions for ap_hook_open_logs, ap_hook_post_config, and > ap_hook_pre_config will fail if DECLINE is returned. Is that > intentional? Yes. This is because those are configuration phases. If a module tries to do something in those phases and can't, then the server should not continue to run, because something wasn't configured properly. If the server did continue, then it would most likely not behave the way it was expected to. > In the quick_handler, returning HTTP_mumble will result in that status > being returned in a real request, but it will fallback to the real > handler in a subrequest. Is that intentional? I don't know, but it sounds completely incorrect. Bill, can you shed any light here? > What's the difference between adding your filters inside your > ap_hook_insert_filter hook versus calling ap_register_*_filter right > inside of the register_hooks callback? You are confusing registering and adding hooks. You should always register the hooks in register_hooks, but adding the hook to the request should be done during request processing. Basically registering a filter lets the server know that the filter exists, but it doesn't enable it for any requests. Adding the filter enables it for that request. Ryan
RE: bogus methods return 500 not 501
Just looked at Apache 1.3, we used to return 501. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA > -Original Message- > From: Cliff Woolley [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, June 12, 2002 6:09 PM > To: [EMAIL PROTECTED] > Subject: bogus methods return 500 not 501 > > > And what's with this? > > root@deepthought:/root# telnet cvs.apache.org 80 > Trying 63.251.56.143... > Connected to cvs.apache.org. > Escape character is '^]'. > GRRR / HTTP/1.0 > Host: cvs.apache.org > > HTTP/1.1 500 Internal Server Error > Date: Thu, 13 Jun 2002 01:08:42 GMT > Server: Apache/2.0.37 (Unix) > Vary: accept-language > Accept-Ranges: bytes > Content-Length: 761 > Connection: close > Content-Type: text/html; charset=ISO-8859-1 > Expires: Thu, 13 Jun 2002 01:08:42 GMT > > > > ...snip... > > > Note: a request of simply "GRRR" rather than "GRRR / HTTP/1.0" also return > 500. I'd guess that one should be 400, maybe 501, but 500 doesn't seem > right. > > sigh. > > --Cliff
