Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
William A. Rowe, Jr. wrote: I've been working with the 2.4 authn/z stuff a bit lately and what I keep tripping over is that the default authorization merge rule uses OR logic. For example, if I enable mod_access_compat and put in a traditional: I wonder if anyone would offer a fastfeather talk next week on wed or thurs - it's only 15 minutes - to introduce what's upcoming in 2.4? I won't be there, but here's a recap of the issue for discussion. (Caveat: I may be missing something important!) With 2.2 and prior versions, one can do something like: Require valid-user Require user admin The logic which is then applied is: 1) For all requests under /htdocs, except those under /htdocs/admin, require any valid user. 2) For all requests under /htdocs/admin, require the "admin" user. With 2.4, unless I'm missing something, the same configuration produces the logic: 1) For all requests under /htdocs, except those under /htdocs/admin, require any valid user. 2) For all requests under /htdocs/admin, require any valid user OR require the user "admin". Of course this grants any valid user access. To get the old behaviour, you seem to need to add "AuthzMergeRules Off" to the second . I just tested versions of this configuration with 2.2 and 2.4 and I think I'm describing the situation correctly. Assuming I am, I fear this will surprise a lot of people who think they've secured their systems after upgrading. It certainly caught me short. Perhaps the default AuthzMergeRules setting should be Off rather than On, at least when merging across configuration blocks? Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Chris Darroch wrote: I've been working with the 2.4 authn/z stuff a bit lately and what I keep tripping over is that the default authorization merge rule uses OR logic. For example, if I enable mod_access_compat and put in a traditional: I wonder if anyone would offer a fastfeather talk next week on wed or thurs - it's only 15 minutes - to introduce what's upcoming in 2.4? Bill
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On Apr 3, 2008, at 12:32 PM, Brad Nicholes wrote: It wouldn't surprise me, which is why we need to get a 2.3-beta out there for testing. That would be good as well... that way we can determine how solid the existing impl is, so when the new stuff is added we know the "old" stuff is still good :)
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
William A. Rowe, Jr. wrote: I'd -1 a 2.4.0 release today, because nobody has even bothered to make a candidate for 2.3-dev. Auth logic changes break most if not all third party auth modules (broke an auth feature in mod_ftp). Not talking about commercial modules but every third party auth extension out there. I've been working with the 2.4 authn/z stuff a bit lately and what I keep tripping over is that the default authorization merge rule uses OR logic. For example, if I enable mod_access_compat and put in a traditional: Order deny,allow Deny from all it doesn't take effect, because the default top-level contains "Require all granted" and since that succeeds for all requests, everything else is short-circuited by the OR merge logic. So at a minimum I seem to have to put in an "AuthzMergeRules Off" to get things to DWIM. I fear that might trip up a significant percentage of people upgrading ... perhaps a "AuthzMergeRules Off" in the default httpd.conf would be sufficient, but my experience with mod_authz_dbd suggested that I needed to put it in a lot of places to get stuff working the way I intended (e.g., the example config in the mod_authz_dbd manual page in the trunk docs). Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Nick Kew wrote: But before that, we need a vision of where we're going, and how to get there without breaking what we've got. * server_conf goes away. Modules have zero or more "conf" sections, essentially today's misnamed dir_conf, which are initialized and merged as they are today. Either using simple-wrappers or custom. * we lose nothing, any module can rerun the conf merge conditionally (see mod_proxy today, or method, or the new such that there is no run time penalty for vhosts. * particular merges should remain, as they are today, as a modular assemblage of features. If your server will never serve files, there is no reason to compile in mod_filesystem e.g. I don't see any such vision in this discussion. Go back to my post from last night. Not saying there's agreement, just some vision. I know folks are thinking "wow, I want to put my favorite [Lua] language parser into httpd core!" Well that's fine for some users, but at the very same time, you have a huge crowd of perl users, mod_macro users, etc. Based on httpd's design, you are SUPPOSED to be able to plug whichever wrapper you like into the core. Overloading the core is taking httpd into an entirely different direction. Just look at the effort we've already expended is splitting cache from proxy, auth components from one another, proxy capabilities from one monolithic module. For that matter, plug in an XML syntax parser. We aren't quite there because we don't have the concept of named-arguments. Fix that across the entire spectrum of modules. E.g. instead of TAKE2, we have some tuple for alias such as (TAKE2, "uri-path", "file-path", NULL) which will facilitate more deliberate parsers and conf authoring. And that is without breaking today's syntax. Bill
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
>>> On 4/3/2008 at 8:23 AM, in message <[EMAIL PROTECTED]>, Plüm, Rüdiger, VF-Group <[EMAIL PROTECTED]> wrote: > >> -Ursprüngliche Nachricht- >> Von: Jim Jagielski >> Gesendet: Donnerstag, 3. April 2008 16:07 >> An: dev@httpd.apache.org >> Betreff: 2.4 (Was: Re: Configuration Issues to Address [was >> Re: Dynamic configuration for the hackathon?]) >> >> Another good topic of discussion: >> >> Time for a 2.4 release? I wouldn't mind pushing that along >> and get some of the feature-set of 2.4 out before we do too >> much ripping with the inevitable delays associated with that :) >> > > I know that I am always devils advocate and a brakeman regarding this, > but we should keep in mind the following: > > 1. After the rewrite of the authz code to a provider based model we still > fail >the basic authz tests in the perl framework. This is a clear showstopper >and needs to be fixed first. Yes, I also should have a had a more closer >look on what Brad (no blame game intended against anyone as I failed to > do >proper review back then) did there in the past to highlight issues > earlier, >but my gut feeling tells me that there are still some surprises in this > code >regarding bugs and the configuration syntax. > It wouldn't surprise me, which is why we need to get a 2.3-beta out there for testing. I've tried to make sure that the holes where filled with the authz refactor, but it is likely that something will be missing until more wide spread testing is done. The perl framework problems were discussed on the list a couple of months ago http://mail-archives.apache.org/mod_mbox/httpd-dev/200801.mbox/[EMAIL PROTECTED] The current tests don't apply anymore because authz has moved from hook based to provider based with logic operators added. If we need to rework something outside of the tests themselves, then that needs to be identified. As far as breaking existing authz modules, it is a similar situation when authn moved from hooks to providers in 2.2. All of the standard Apache authz modules have already been refactored. This issue is third parties will have to refactor their authz modules for 2.4. Brad
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On Thu, 03 Apr 2008 11:13:31 -0500 "William A. Rowe, Jr." <[EMAIL PROTECTED]> wrote: > The hope. Those admins who refuse to let their junior admins use that > directive should have a level of control over their outward facing > heavily-loaded machines :) The logic is approximately cloned from , and just differs in what it evaluates. If we're talking about any substantial config changes, then the whole location_walk and merges should be on the table. But before that, we need a vision of where we're going, and how to get there without breaking what we've got. I don't see any such vision in this discussion. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Justin Erenkrantz wrote: On Thu, Apr 3, 2008 at 8:53 AM, Akins, Brian <[EMAIL PROTECTED]> wrote: Very rough draft. But this is not necessarily slow... ;) Right. Even then, the user/admin may be willing to burn CPU cycles anyway to get a simpler config. Plus, if they were to use mod_rewrite, they've already blown a huge chunk of CPU cycles! =) -- justin Yes! I'm not against offering slow features :) I'm only antagonistic towards replacing the fast ones, from today. FYI - Files and Directory should be entirely moved out of core into our default filesystem provider module. Only host/location/method should be part of the core (well, host perhaps in the http layer). The
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
Akins, Brian wrote: On 4/3/08 11:38 AM, "William A. Rowe, Jr." <[EMAIL PROTECTED]> wrote: But that *doesn't* mean I don't want it... simply not to replace directory, file, location or method. Keep in mind you wouldn't have your ErrorLog opened at startup time, as this is too variant Unless I'm mistaken, there is nothing that really stops us from making all log related directives from being per-dir (assuming they have "real" names at startup). Using mod_log_config as an example, it opens all of it's logs as root at start, and we are able to select one at run time via env. That could just as easily be done via a per-dir merge. (Note: I am ignorant of the stuff between the errolog directive until it gets to the actually logging stuff, so I may be way off. ) You are right; provided that we pre-merge SOME of the layers of dir configs (e.g. we pre-merge all of the vhost-related dir configs). And remember a merge of A+null or null+A is fast. So if we break up a very very big dir config structure into three, e.g. those directives that are very frequently tweaked, those which are sometimes tweaked and those we don't expect to change at all (at least, not after server-merges at startup) the overall hit of dir_merge (now three different functions for this hypothetical modules) will be much less.
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Nick Kew wrote: is of course a crusty old relative. Limit is unrelated, it's fundamentally borked (directive must know it is participating in a limit-ed section, cannot overly multiple limit-ed sections because that directive has never created a conf section, and there is no exception thrown when a non-participating directive is nested within a limit). This is why I'm not 'fixing'
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On Thu, Apr 3, 2008 at 4:31 PM, Mads Toftum <[EMAIL PROTECTED]> wrote: > On Thu, Apr 03, 2008 at 10:06:50AM -0400, Jim Jagielski wrote: > > Time for a 2.4 release? I wouldn't mind pushing that along > > and get some of the feature-set of 2.4 out before we do too > > much ripping with the inevitable delays associated with that :) > > Is there really enough news in trunk to warrant the overhead of > maintaining yet another branch? tbh. I'd much rather see work going > towards 3.x ;) > > vh > > Mads Toftum > -- > http://soulfood.dk > I'm wondering the same. -- ~Jorge
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On Thu, Apr 3, 2008 at 8:53 AM, Akins, Brian <[EMAIL PROTECTED]> wrote: > Very rough draft. But this is not necessarily slow... ;) Right. Even then, the user/admin may be willing to burn CPU cycles anyway to get a simpler config. Plus, if they were to use mod_rewrite, they've already blown a huge chunk of CPU cycles! =) -- justin
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
On 4/3/08 11:38 AM, "William A. Rowe, Jr." <[EMAIL PROTECTED]> wrote: > But that *doesn't* mean I don't want it... simply not to replace directory, > file, location or method. Keep in mind you wouldn't have your ErrorLog > opened at startup time, as this is too variant Unless I'm mistaken, there is nothing that really stops us from making all log related directives from being per-dir (assuming they have "real" names at startup). Using mod_log_config as an example, it opens all of it's logs as root at start, and we are able to select one at run time via env. That could just as easily be done via a per-dir merge. (Note: I am ignorant of the stuff between the errolog directive until it gets to the actually logging stuff, so I may be way off. ) -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
On Thu, 03 Apr 2008 11:22:00 -0400 "Akins, Brian" <[EMAIL PROTECTED]> wrote: > > DocumentRoot /www/cnn > ServerAdmin [EMAIL PROTECTED] > etc That basically comes out of what I committed this morning. Well, up to a point: it only applies to the per-dir config. > Maybe no need for Directory, location, etc, either... > I'm thinking more of displacing tortuous mod_rewrite-based configs than any of the old containers (except possibly the much-misunderstood-and-abused ). -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On 4/3/08 11:38 AM, "William A. Rowe, Jr." <[EMAIL PROTECTED]> wrote: >> >> ... >> > > Slow Not if the parsing is done at config time and HTTP_Method is handle by a provider. Some pseudo code: At config time, the parser would do something like: parse_provider *prov; void *data; prov = ap_lookup_provider("config_parse", "HTTP_Method", "0.1"); data = prov->init(conf_pool, "HTTP_Method", TOKEN_EQUAL, "GET") /*the provider may do something like*/ typedef struct { parse_token token; int method_number; } method_data; void *method_init(apr_pool_t *pool, const char *key, parse_token token, const char *arg) { method_data *data = apr_pcalloc(pool, sizeof(method_data)); data->token = token; /*need to check if we only handle === or something */ if(strcacecmp(arg, "GET")) { data->method_number = M_GET; } return data; /*the parser stores this data with the node*/ At run time, then when running this node from the cached parse tree, it may call something like: node->prov->exec(r, nod->data) /*the provider runs something like*/ int method_exec(reuqest_rec *r, method_data *data) { if(data->method_number == r->method_number) return 1; return 0; } Very rough draft. But this is not necessarily slow... ;) -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On Thu, 03 Apr 2008 11:25:56 -0400 "Akins, Brian" <[EMAIL PROTECTED]> wrote: > On 4/3/08 10:47 AM, "William A. Rowe, Jr." <[EMAIL PROTECTED]> > wrote: > > > I'll commit the > > > ... > May work already (not tested) if Rewrite is active (so method is available as an env var). Certainly on the radar. is of course a crusty old relative. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Akins, Brian wrote: On 4/3/08 10:47 AM, "William A. Rowe, Jr." <[EMAIL PROTECTED]> wrote: I'll commit the ... Slow
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
Akins, Brian wrote: On 4/2/08 5:56 PM, "William A. Rowe, Jr." <[EMAIL PROTECTED]> wrote: I'm pondering this... if we drop "per-server" ... yet retain the ability for authors to factor their config info into related config sections... Yes... Bcs what IO am imagining is something like what I've posted before: DocumentRoot /www/cnn ServerAdmin [EMAIL PROTECTED] etc Foo bar Maybe no need for Directory, location, etc, either... horribly inefficient, on a 10-fold order of magnitude. But that *doesn't* mean I don't want it... simply not to replace directory, file, location or method. Keep in mind you wouldn't have your ErrorLog opened at startup time, as this is too variant (unless the ErrorLog code became really very clever - although dynamic patterns in the ErrorLog file name could also make that impossible) ... but where you are very cautious about your log file directive, and it's not a mass vhosting server, this really wouldn't prove to be an issue. Bill
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On 4/3/08 10:47 AM, "William A. Rowe, Jr." <[EMAIL PROTECTED]> wrote: > I'll commit the ... ;) -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
On 4/2/08 6:07 PM, "William A. Rowe, Jr." <[EMAIL PROTECTED]> wrote: > we can finish these out, opening logs with > full privileges. Other merges will happen at run time (or be optimized > when we can accomplish this) per-request. We already "fake" per-dir logs with the env stuff in mod_log_config. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
On 4/2/08 5:56 PM, "William A. Rowe, Jr." <[EMAIL PROTECTED]> wrote: > I'm pondering this... if we drop "per-server" ... yet retain the ability > for authors to factor their config info into related config sections... Yes... Bcs what IO am imagining is something like what I've posted before: DocumentRoot /www/cnn ServerAdmin [EMAIL PROTECTED] etc Foo bar Maybe no need for Directory, location, etc, either... -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
On 4/2/08 5:50 PM, "William A. Rowe, Jr." <[EMAIL PROTECTED]> wrote: > ixnay on the run-time intensive, slow down the server sorts of changes. > httpd continues to become slower as it becomes more powerful. I know you > are the first one to raise your hand and point out when we are doing too > much processing for too simple a request. Isn't this what modules are for? > > Perhaps you could elaborate? Yes, there is always a fine line between configuration and performance, I suppose. Basically, at the heart of what I'm imagining is everything is per_dir (no real per-server configs) and the "fancy configuration stuff" really boils down to per-dir config merges. The "parser" or "language" needs to be small and quick and do most of it's heavy lifting in post-config (parse and cache the tree, don't do the full string parsing at run time). Sorta how mod_rewrite works now... The If/Else stuff would probably be able to do this.. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Plüm wrote: 2. My feeling regarding the usage of 2.2 is that since about 6 month we are getting track as commercial 3rd parties now supply modules for httpd 2.2. This means that will have to maintain one more stable branch for quite some time and to be honest currently we effectively manage only one since 1.3 and also 2.0 seem to be pretty abandoned. I am not quite sure if we have enough resources to do this. Perhaps 1.3 is maintained, perhaps not, just depending on if there are enough developers who care to provide minor patches. But it really shouldn't be a distraction. Once folks accept 2.2 as a clean replacement for 2.0 (we are essentially there now, I think, perhaps after one or two more point-bumps), 2.0 should fall off the radar entirely. This 2.2 becomes stable, so we are really only talking about supporting a 2.2 and a 2.4 for a while, then 2.4 and a 3.0. I'd -1 a 2.4.0 release today, because nobody has even bothered to make a candidate for 2.3-dev. Auth logic changes break most if not all third party auth modules (broke an auth feature in mod_ftp). Not talking about commercial modules but every third party auth extension out there. The only thing I'd like to change before we push 2.3-dev onto our module developer community is to enhance the methods logic (64 is now insufficient when you add DAV + FTP + {whatever} into the method stew) and drop the directive already for the directive. These have some subtle auth-changes that a few auth module authors need to compensate for. (The third party auth modules that won't change are the ones that are already broken for ... who won't have to do anything special in support of the directive.) I'll commit the changes before the hackathon so that folks can corner me in person, dev@ list feedback will also be welcome as always. Just to prove I'm not being a prick with respect to Lua support, I'm working on as an add in module ;-) Bill
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On Thu, Apr 03, 2008 at 10:06:50AM -0400, Jim Jagielski wrote: > Time for a 2.4 release? I wouldn't mind pushing that along > and get some of the feature-set of 2.4 out before we do too > much ripping with the inevitable delays associated with that :) Is there really enough news in trunk to warrant the overhead of maintaining yet another branch? tbh. I'd much rather see work going towards 3.x ;) vh Mads Toftum -- http://soulfood.dk
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
> -Ursprüngliche Nachricht- > Von: Jim Jagielski > Gesendet: Donnerstag, 3. April 2008 16:07 > An: dev@httpd.apache.org > Betreff: 2.4 (Was: Re: Configuration Issues to Address [was > Re: Dynamic configuration for the hackathon?]) > > Another good topic of discussion: > > Time for a 2.4 release? I wouldn't mind pushing that along > and get some of the feature-set of 2.4 out before we do too > much ripping with the inevitable delays associated with that :) > I know that I am always devils advocate and a brakeman regarding this, but we should keep in mind the following: 1. After the rewrite of the authz code to a provider based model we still fail the basic authz tests in the perl framework. This is a clear showstopper and needs to be fixed first. Yes, I also should have a had a more closer look on what Brad (no blame game intended against anyone as I failed to do proper review back then) did there in the past to highlight issues earlier, but my gut feeling tells me that there are still some surprises in this code regarding bugs and the configuration syntax. 2. My feeling regarding the usage of 2.2 is that since about 6 month we are getting track as commercial 3rd parties now supply modules for httpd 2.2. This means that will have to maintain one more stable branch for quite some time and to be honest currently we effectively manage only one since 1.3 and also 2.0 seem to be pretty abandoned. I am not quite sure if we have enough resources to do this. Regards Rüdiger
2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
>>> On 4/3/2008 at 8:06 AM, in message <[EMAIL PROTECTED]>, Jim Jagielski <[EMAIL PROTECTED]> wrote: > Another good topic of discussion: > > Time for a 2.4 release? I wouldn't mind pushing that along > and get some of the feature-set of 2.4 out before we do too > much ripping with the inevitable delays associated with that :) Please let's get 2.4 out. It would be great to finally have the new Authz configuration logic see the light of day along with other functionality that has been sitting around for a while. Brad
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
Jorge Schrauwen wrote: ... if we had a config finalize, modules who were prepared to declare their config (e.g. mod_vhost declaring the per-host directory merges "completed") then as-root, we can finish these out, opening logs with full privileges. Other merges will happen at run time (or be optimized when we can accomplish this) per-request. So does a setup like this make it possible for the processes/thread handling the request to change to the correct UID/GID before reading/writing files? Just something that popped into my head when reading this. No. Once the httpd engine finishes the config phase altogether, we continue to drop from root to the desired UID/GID and that process must not have the privilege to change these again. The request engine ... which is the container where exploits are targeted, must remain secure.
2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Another good topic of discussion: Time for a 2.4 release? I wouldn't mind pushing that along and get some of the feature-set of 2.4 out before we do too much ripping with the inevitable delays associated with that :)
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
> ... if we had a config finalize, modules who were prepared to declare > their config (e.g. mod_vhost declaring the per-host directory merges > "completed") then as-root, we can finish these out, opening logs with > full privileges. Other merges will happen at run time (or be optimized > when we can accomplish this) per-request. So does a setup like this make it possible for the processes/thread handling the request to change to the correct UID/GID before reading/writing files? Just something that popped into my head when reading this. -- ~Jorge
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
William A. Rowe, Jr. wrote: William A. Rowe, Jr. wrote: -I have to write a good bit of code before a module is configurable. (I'm lazy. Very lazy.) Agreed - see my first point. One interesting point; why do we keep per-server and per-dir sections? Perhaps it's time for a single simpler-to-use mechanic which can represent either or both (essentially deprecate per-server in 3.0). I'm pondering this... if we drop "per-server" ... yet retain the ability for authors to factor their config info into related config sections... ... we could divide the frequently-merged and infrequently-merged options into two or more groups; the overall merge calls would run more quickly, but the flexibility would be greatly enhanced. Override-able ->document_root, anyone? And taking this one step further - addressing issues such as the fact that we all want certain merged-sections (e.g. vhosts) to be run as root early in the process, but be flexible enough to override a bit later on... ... if we had a config finalize, modules who were prepared to declare their config (e.g. mod_vhost declaring the per-host directory merges "completed") then as-root, we can finish these out, opening logs with full privileges. Other merges will happen at run time (or be optimized when we can accomplish this) per-request. It's not looking trivial, but it seems doable.
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
William A. Rowe, Jr. wrote: -I have to write a good bit of code before a module is configurable. (I'm lazy. Very lazy.) Agreed - see my first point. One interesting point; why do we keep per-server and per-dir sections? Perhaps it's time for a single simpler-to-use mechanic which can represent either or both (essentially deprecate per-server in 3.0). I'm pondering this... if we drop "per-server" ... yet retain the ability for authors to factor their config info into related config sections... ... we could divide the frequently-merged and infrequently-merged options into two or more groups; the overall merge calls would run more quickly, but the flexibility would be greatly enhanced. Override-able ->document_root, anyone? Bill
Re: Dynamic configuration for the hackathon?
Graham Leggett wrote: Jim Jagielski wrote: This reminds me: a serf BOF or session would, I think, go over quite well :) A question that has been on my mind for a bit, is "what does serf intend to replace", and "why is it better?". The impression I have so far is that somehow what we have now is suboptimal, and that serf somehow does it better, but exactly what is suboptimal, and exactly how serf does it better has yet to be clarified. Someone please consider offering a title/abstract/speaker/bio on a fast paced introduction to the advantages of serf as a 15 minute FFT preso, so that we can all understand the answers to Graham's questions. [EMAIL PROTECTED] - for either wed or thurs afternoon.
Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
Akins, Brian wrote: The biggest problems I have with current system are: -Every module does things differently Within limits this will remain true. But we are missing a host of very trivial simplifications for the casual module developer, and reinvent the same wheel module after module. I'm going to propose some changes to the default TAKE[0-9]+ macro and support functions to illustrate how much more useful these can be, and offer some new functions that make it really simple to * define constraints (an int, sure, but in what range?) * tag default-value, and error out if a non-default value is replaced by the user (e.g. LogLevel debug LogLevel error later on) -No real per-request configuration. Some modules use env to do some of this. ixnay on the run-time intensive, slow down the server sorts of changes. httpd continues to become slower as it becomes more powerful. I know you are the first one to raise your hand and point out when we are doing too much processing for too simple a request. Isn't this what modules are for? Perhaps you could elaborate? -I have to write a good bit of code before a module is configurable. (I'm lazy. Very lazy.) Agreed - see my first point. One interesting point; why do we keep per-server and per-dir sections? Perhaps it's time for a single simpler-to-use mechanic which can represent either or both (essentially deprecate per-server in 3.0). If we do what niq suggests (which, if we stick with current config system is fine) is that it just adds another layer on top of all the existing issues. With lua, for example you could make modules Lua modules... Maybe could do same in perl?? That's why I'm not a fan of all lua all the time. But to do what we all want, I bet we'll need to refine the config API, and simplify it for adding pluggable semantic engines (lua, perl, simply 'sed' etc). My opinion (which is worthless, I know) is to pick one way and do it. Lua is easy to learn and satisfies most (all?) of our requirements. And if Brian M. and I can learn it, anyone can ;) Most importantly, if I am building an embedded httpd, I don't want all of this extra crap. httpd was famous for how lean it could be, are we all ready to throw out that advantage?
Re: Dynamic configuration for the hackathon?
Issac Goldstand wrote: We're not talking about fresh users, we're talking about existing users. Fresh users have to deal with one learning curve or another anyway. I'm not talking about fresh users either. Matt
Re: Dynamic configuration for the hackathon?
There's a couple of conflicting demands by our users, and spending time on the users mailing list and on the IRC channel is a great way to see this first-hand. They don't want to learn a new syntax. And they want a new syntax that lets them do what they mean. And they're very frustrated when they can't do things that are obviously what everyone wants, and have to resort to mod_rewrite. Now, I'm very fond of mod_rewrite, because, after all, I wrote a book, and royalties from that book buy me my toys. But the time spent answering the same questions again and again on IRC lead me to believe that there are certain features that would gain us a lot of appreciation from our users - the folks who, at least to me, make this stuff worth doing. Easy for me to say, since I just document what the rest of you guys actually do. I recognize that it looks easy from the cheap seats. Virtual hosts are just one example of something that's amazingly hard (for beginners) to get right, and very easy to get wrong. And when it's wrong, it's unpredictably wrong, in ways that are hard to anticipate. On Mar 26, 2008, at 09:06, Nick Kew wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. ... Directives applicable per-request (anything, subject to per-directive context checking) ++1. This would solve an enormous number of the user problems that we experience on the help channels, which, I have to assume, are but the tip of the iceberg of problems experienced out there in the real world. It's amazing, sometimes, what absurd lengths folks have to go to, to solve the problems that they've created. A noble objective: render mod_rewrite obsolete :-) ++1 -- What the world needs is more geniuses with humility, there are so few of us left. Oscar Levant
Re: Dynamic configuration for the hackathon?
On 4/2/08 8:44 AM, "Rich Bowen" <[EMAIL PROTECTED]> wrote: > May I request, if mod_wombat becomes a standard module, that it be given a > name not quite so calculated to make the newbie disable it without a second > glance. I think the reason wombat was chosen is because mod_lua was taken. In reality, I suppose there should be a base lua module, then a configuration module based upon it and a "hook handler" module based upon it as well. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On Mar 31, 2008, at 13:46, Issac Goldstand wrote: Make mod_wombat a standard module and part of the default moduleset May I request, if mod_wombat becomes a standard module, that it be given a name not quite so calculated to make the newbie disable it without a second glance. I mean, wombats are cute and all, and it's a delightful name, but nobody knows what it means. -- Whatever you do will be insignificant, but it is very important that you do it. Mahatma Ghandi
Re: Dynamic configuration for the hackathon?
On Mar 31, 2008, at 13:31, Paul Querna wrote: Just look at SSLRequire, Rewrite*, MPM Process/Thread Management, Filter chaining, large Auth{N,Z} chains, and more. Imagine them not sucking. That would be lovely. Truly. -- Happiness isn't something you experience; it's something you remember. Oscar Levant
Re: Dynamic configuration for the hackathon?
On Tue, Apr 1, 2008 at 6:20 AM, Jim Jagielski <[EMAIL PROTECTED]> wrote: > IMO, we work best when we feel empowered to scratch our itches... > As we've also seen, sometimes all it takes is someone to create > a rough framework of an implementation for people to get excited > by it and jump right on in, adding, extending and tuning it. > > This reminds me: a serf BOF or session would, I think, go over > quite well :) Lieven and I are going to be focusing on cutting a serf release during the Hackathon. I promised that we'd get a new serf release out before Subversion 1.5 final comes out. =P If folks are interested in a serf BOF or 'beer session' at the bar, I'm sure we could swing something. =) -- justin
Re: Dynamic configuration for the hackathon?
Jim Jagielski wrote: I'd prefer optimum runtime and let that drive how it gets exposed to the admin, rather than the reverse... And then we can see if that pain is worth it :) +1 to this as a guiding principle. I know our administrators would, above all else, like a standard way to build up configuration files from templates. Perhaps that could be mod_macro packaged in the main distribution, or some kind of "configulator" utility, or yet something else. Dynamic (re)configuration issues are a lesser concern for them, I think; it would be great if whatever is implemented could be largely optional so it can be loaded only if necessary. Good luck at the hackathon! Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: Dynamic configuration for the hackathon?
On 4/1/08 11:21 AM, "Torsten Foertsch" <[EMAIL PROTECTED]> wrote: > On Tue 01 Apr 2008, Akins, Brian wrote: >> In pseudo config, like niq is suggesting, you could have something like: >> >> >> #cnn specific stuff here... >> DocumentRoot /htdocs/cnn >> CutomLog "|/usr/bin/logger cnn" my_format >> ErrorLog /var/log/cnn.error >> > > I don't like that. I think there are security considerations why logfiles are > opened from the parent process as root. But with other logging mechanisms > that provide write-only semantics it is good. In my setup the apache logs to > a named pipe to a process outside the chroot. The example wasn't about logs, it was just an example of how, if you wanted, could do virtual hosts using niq's if/else style. Nothing says that the logs wouldn't be opened in parent by root. If log config was per dir (which we "emulate" with env variables) this would "just work." > As I understood it the main problem with the current mod_rewrite based config > is that it is too complex. The new language has to watch out not to end at > the same place. One thing that I think is messy is the use of subprocess_env > to pass information from module to module and even from administrator to > module: no-gzip, force-gzip, downgrade-1.0, nokeepalive, redirect-carefully > etc. +1 to all that... -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On Tue 01 Apr 2008, Akins, Brian wrote: > In pseudo config, like niq is suggesting, you could have something like: > > > #cnn specific stuff here... > DocumentRoot /htdocs/cnn > CutomLog "|/usr/bin/logger cnn" my_format > ErrorLog /var/log/cnn.error > I don't like that. I think there are security considerations why logfiles are opened from the parent process as root. But with other logging mechanisms that provide write-only semantics it is good. In my setup the apache logs to a named pipe to a process outside the chroot. To really create vhosts on the fly I think a new hook in the MPM would be good that is called from a configuration provider module. It reconfigures the parent apache and does a graceful restart. This way almost anything can be reconfigured. A question is whether the provider should send changes to the apache or a complete new config. In the former case we'd need something like UnListen localhost:80 CloseErrorLog ... DeleteVirtualHost localhost:80 In the end the server_rec would go away. We have one server with a list of loaded modules, a PidFile and an AcceptMutex that is listening on a list of ports. The rest is configurable this way: SSLCertificateFile ... Protocol http # expecting HTTP to be spoken on the wire Timeout 10 ErrorLog ... Or rather the request is passed to the config module that checks localport and localaddr and issues the SSLCertificateFile directive. Then it checks the Host-header ... As for dynamic request configuration, I'd wish some configuration provider with intelligent conftree caching. That provider then implements a language as it likes, LUA, Perl, Apache-style ..., ... It generates a list of directives that is compiled into a conftree. As I understood it the main problem with the current mod_rewrite based config is that it is too complex. The new language has to watch out not to end at the same place. One thing that I think is messy is the use of subprocess_env to pass information from module to module and even from administrator to module: no-gzip, force-gzip, downgrade-1.0, nokeepalive, redirect-carefully etc. Torsten
Re: Dynamic configuration for the hackathon?
On Apr 1, 2008, at 10:32 AM, Matthew M. Burke wrote: Graham Leggett wrote: The trouble is, if I want to solve problem A ("configure the server"), and I find out that before I can solve problem A ("configure the server") I need to first solve problem B ("learn a new language"), that is a big incentive to just ignore the new server and stick with httpd v2. It makes no difference in the world how "easy" Lua is to learn, "no learning required" will always beat "easy to learn" hands down every single time. That's just marketing! Configuring the server is not a "no learning required" activity, particularly if you are using mod_rewrite! Matt Graham's point is, I think, that for most setups, although ugly, the config does not require a lot of thought. Compare, for example, a "typical" Apache config to sendmail's sendmail.cf file :)
Re: Dynamic configuration for the hackathon?
Matthew M. Burke wrote: Graham Leggett wrote: The trouble is, if I want to solve problem A ("configure the server"), and I find out that before I can solve problem A ("configure the server") I need to first solve problem B ("learn a new language"), that is a big incentive to just ignore the new server and stick with httpd v2. It makes no difference in the world how "easy" Lua is to learn, "no learning required" will always beat "easy to learn" hands down every single time. That's just marketing! Configuring the server is not a "no learning required" activity, particularly if you are using mod_rewrite! We're not talking about fresh users, we're talking about existing users. Fresh users have to deal with one learning curve or another anyway. Issac
Re: Dynamic configuration for the hackathon?
Graham Leggett wrote: The trouble is, if I want to solve problem A ("configure the server"), and I find out that before I can solve problem A ("configure the server") I need to first solve problem B ("learn a new language"), that is a big incentive to just ignore the new server and stick with httpd v2. It makes no difference in the world how "easy" Lua is to learn, "no learning required" will always beat "easy to learn" hands down every single time. That's just marketing! Configuring the server is not a "no learning required" activity, particularly if you are using mod_rewrite! Matt
Re: [OT] Dynamic configuration for the hackathon?
Jorge Schrauwen wrote: On Tue, Apr 1, 2008 at 4:13 PM, Issac Goldstand <[EMAIL PROTECTED]> wrote: > > Downside: > - perl isn't very easy and userfriendly I think that the downside is the fact that perl interpreters suck up RAM, not the easiness factor. It's probably not significantly easier/harder than lua, but it's big and clunky Well perl did scare the *bleep* out of my class mates when they where looking what I was doing during the break. Isn't that why we love it so much? :) Issac
Re: Dynamic configuration for the hackathon?
On Tue, Apr 1, 2008 at 4:13 PM, Issac Goldstand <[EMAIL PROTECTED]> wrote: > > > > Jorge Schrauwen wrote: > > Solutions propose: > > - lua in the core or atleast in a module > > - mod_perl > > mod_perl exists already. We're looking to replace it because... (see below) > I'm quite aware that it exists, I ment that it is a possible solution to the problem. (I'm actually using it... although restarts are still needed to load the new vhosts) > > > > > Downside: > > - perl isn't very easy and userfriendly > > I think that the downside is the fact that perl interpreters suck up > RAM, not the easiness factor. It's probably not significantly > easier/harder than lua, but it's big and clunky > Well perl did scare the *bleep* out of my class mates when they where looking what I was doing during the break. Then again, I do most of my system scripts in perl ^^ >Issac > > -- ~Jorge
Re: Dynamic configuration for the hackathon?
Jim Jagielski wrote: On Apr 1, 2008, at 9:36 AM, Jorge Schrauwen wrote: Let me try and summarize this then: Problem: The httpd configuration is to static for some users (e.g. large host providers) they want to have a more dynamic system. IMO, based on feedback from people I've dealt with with my Covalent/SpringSource hat on, the biggest problem for these dynamic hosts is adding/changing/deleting SSL vhosts more than anything else :) Actually, wearing my company's hat, our need is actually more for the dynamic config (including vhosts if we could plug one in via a config API) that can be modified at run-time (and not only after [graceful] reloads). Issac
Re: Dynamic configuration for the hackathon?
Jorge Schrauwen wrote: Solutions propose: - lua in the core or atleast in a module - mod_perl mod_perl exists already. We're looking to replace it because... (see below) Downside: - perl isn't very easy and userfriendly I think that the downside is the fact that perl interpreters suck up RAM, not the easiness factor. It's probably not significantly easier/harder than lua, but it's big and clunky Issac
Re: Dynamic configuration for the hackathon?
On 4/1/08 9:40 AM, "Torsten Foertsch" <[EMAIL PROTECTED]> wrote: > I know one can do that. But a VHost has a server_rec, maybe a separate > error_log and access_log, etc. Those cannot be created at request time. That > is what I meant. Well I was hacking around with the idea that the selection of vhost is a hook, rather than how it is now. Basically, what I did was protect the vhost list with a mutex, so you can add/delete (well not fully delete) vhosts. Combine that with a "global" logger (ie, piped logger to syslog-ng) and the access and error logs are "created" on the fly as well. In the end, it was way too hacky and I haven't thought about it much in well over a year. In pseudo config, like niq is suggesting, you could have something like: #cnn specific stuff here... DocumentRoot /htdocs/cnn CutomLog "|/usr/bin/logger cnn" my_format ErrorLog /var/log/cnn.error H Would just have to move a bunch of stuff to per_dir and make sure all of our directory config creation and merging is very fast... This IF/else stuff could be done in a module and not in core for testing purposes. Not quite where I'd like the config to be, but this may actually be obtainable in near future. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On Apr 1, 2008, at 9:40 AM, Torsten Foertsch wrote: On Tue 01 Apr 2008, Jim Jagielski wrote: On Apr 1, 2008, at 5:21 AM, Torsten Foertsch wrote: You cannot add virtual servers on the fly Hmmm let's see now. If we have a default Vhost that all non- matched name-based hosts get directed to configured, then a mod_perl based handler can be adjusted to look at the Server header and do "different stuff" based on what it's set to... envision a simple dynamic mapping. I know one can do that. But a VHost has a server_rec, maybe a separate error_log and access_log, etc. Those cannot be created at request time. That is what I meant. Gotcha.
Re: Dynamic configuration for the hackathon?
On Apr 1, 2008, at 9:36 AM, Jorge Schrauwen wrote: Let me try and summarize this then: Problem: The httpd configuration is to static for some users (e.g. large host providers) they want to have a more dynamic system. IMO, based on feedback from people I've dealt with with my Covalent/SpringSource hat on, the biggest problem for these dynamic hosts is adding/changing/deleting SSL vhosts more than anything else :)
Re: Dynamic configuration for the hackathon?
On Apr 1, 2008, at 9:34 AM, Graham Leggett wrote: Jim Jagielski wrote: This reminds me: a serf BOF or session would, I think, go over quite well :) A question that has been on my mind for a bit, is "what does serf intend to replace", and "why is it better?". The impression I have so far is that somehow what we have now is suboptimal, and that serf somehow does it better, but exactly what is suboptimal, and exactly how serf does it better has yet to be clarified. Basically, with serf it's easier to get away from the 1 request/response cycle being tied to a specific thread/process. serf provides the async aspects "for free" and abstracts it out for us.
Re: Dynamic configuration for the hackathon?
On Tue 01 Apr 2008, Jim Jagielski wrote: > On Apr 1, 2008, at 5:21 AM, Torsten Foertsch wrote: > > You cannot add virtual servers on the fly > > Hmmm let's see now. If we have a default Vhost that all non-matched > name-based hosts get directed to configured, then a mod_perl based > handler can be adjusted to look at the Server header and do "different > stuff" based on what it's set to... envision a simple dynamic > mapping. I know one can do that. But a VHost has a server_rec, maybe a separate error_log and access_log, etc. Those cannot be created at request time. That is what I meant. Torsten
Re: Dynamic configuration for the hackathon?
Let me try and summarize this then: Problem: The httpd configuration is to static for some users (e.g. large host providers) they want to have a more dynamic system. Where they can configure things on a request basis and add vhosts and such without restarting httpd. Solutions propose: - lua in the core or atleast in a module - mod_perl Downside: - perl isn't very easy and userfriendly - overhead of having this be re-evaluated a lot - lue -> users to lazy to learn new language I'm sure I missed a lot since I seem to be missing some older messages so feel free to ignore this add to it or whatever. -- ~Jorge
Re: Dynamic configuration for the hackathon?
Jim Jagielski wrote: This reminds me: a serf BOF or session would, I think, go over quite well :) A question that has been on my mind for a bit, is "what does serf intend to replace", and "why is it better?". The impression I have so far is that somehow what we have now is suboptimal, and that serf somehow does it better, but exactly what is suboptimal, and exactly how serf does it better has yet to be clarified. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Dynamic configuration for the hackathon?
IMO, we work best when we feel empowered to scratch our itches... As we've also seen, sometimes all it takes is someone to create a rough framework of an implementation for people to get excited by it and jump right on in, adding, extending and tuning it. This reminds me: a serf BOF or session would, I think, go over quite well :)
Re: Dynamic configuration for the hackathon?
On Mar 31, 2008, at 2:15 PM, Roy T. Fielding wrote: Users like mod_rewrite for many reasons, but I think mostly because it does so many things that almost every Apache hosting provider needs to have it installed and usable in .htaccess Except for web hosting companies, and some special cases, at least in my experience people avoid .htaccess files like the plague. :)
Re: Dynamic configuration for the hackathon?
On Apr 1, 2008, at 2:17 AM, Justin Erenkrantz wrote: On Mon, Mar 31, 2008 at 7:41 PM, William A. Rowe, Jr. <[EMAIL PROTECTED]> wrote: I sympathize, but this doesn't reflect the addition of blocks... those blocks can be trivially implemented as a loadable module ;-) As I grok it, the point would be throw out our ridiculous config syntax entirely (or at best write a compatibility module or a converter to the new format) and expose a real config API (hello providers! *chuckle*) and then build a pure LUA-based config format on top of that. I'd be definitely curious to see what would come of that - 'cuz really it can't be much worse than the garbage we're stuck with now. -- justin Agreed... historically, of course, that alone has not be reason enough. I recall many many "discussions", esp with Dean, about the overhead of folding such a "complex" parser into httpd compared to having that done externally (m4 anyone? :) ) and out of process...
Re: Dynamic configuration for the hackathon?
Akins, Brian wrote: My opinion (which is worthless, I know) is to pick one way and do it. Lua is easy to learn and satisfies most (all?) of our requirements. And if Brian M. and I can learn it, anyone can ;) The trouble is, if I want to solve problem A ("configure the server"), and I find out that before I can solve problem A ("configure the server") I need to first solve problem B ("learn a new language"), that is a big incentive to just ignore the new server and stick with httpd v2. It makes no difference in the world how "easy" Lua is to learn, "no learning required" will always beat "easy to learn" hands down every single time. The Apache config already supports the concepts of a "global" config, and a per-request config (.htaccess), these basic things could easily form the building blocks of a pluggable config mechanism, in whatever language you like. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Dynamic configuration for the hackathon?
On Apr 1, 2008, at 5:21 AM, Torsten Foertsch wrote: You cannot add virtual servers on the fly Hmmm let's see now. If we have a default Vhost that all non-matched name-based hosts get directed to configured, then a mod_perl based handler can be adjusted to look at the Server header and do "different stuff" based on what it's set to... envision a simple dynamic mapping.
Re: Dynamic configuration for the hackathon?
Can we 1st determine exactly what pain-point we're trying to solve here? Or, at least, prioritize them? It seems to me that if the main issue is runtime re-configuration during the request/response phases, then that really forces us into something which must be very lean, mean and FAST. By extension, this also provides to us a way to really clean up and standardize our current config mess. If however, our focus would be to make our config files cleaner and prettier, then I fear that by the time that trickles down to how that interacts with the actual runtime processing, we'll get nasty performance. I'd prefer optimum runtime and let that drive how it gets exposed to the admin, rather than the reverse... And then we can see if that pain is worth it :)
Re: Dynamic configuration for the hackathon?
On 4/1/08 5:35 AM, "Issac Goldstand" <[EMAIL PROTECTED]> wrote: > I don't think we're even talking about on-the-fly stuff for the "Lua" > parser engine, so Perl can do everything that Lua can. I am. The biggest problems I have with current system are: -Every module does things differently -No real per-request configuration. Some modules use env to do some of this. -I have to write a good bit of code before a module is configurable. (I'm lazy. Very lazy.) If we do what niq suggests (which, if we stick with current config system is fine) is that it just adds another layer on top of all the existing issues. With lua, for example you could make modules Lua modules... Maybe could do same in perl?? My opinion (which is worthless, I know) is to pick one way and do it. Lua is easy to learn and satisfies most (all?) of our requirements. And if Brian M. and I can learn it, anyone can ;) -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
William A. Rowe, Jr. wrote: -0.5 PLEASE not in the core. Make mod_wombat a standard module and part of the default moduleset, whatever, but let's not make more core dependencies, please?!? -0.99 - agreed. Perl is perfectly happy having blocks as modular behaviors... I've noticed a trend in the last few years of building on the core (and folks rightfully accused me of growing mod_proxy core when new directives are rightfully part of mod_proxy_{whatever}). Let's focus on keeping it in useful pieces, even if they are built by default. -0.99 - agree with wrowe. I looked at the option of supporting different configuration providers a while back, I was researching the idea that configuration could be stored in LDAP instead of flat files, and it seemed pretty straightforward (didn't have time to explore further). What I did find is that it is quite possible to support pluggable configuration options, it isn't difficult: your configuration module must somehow render config lines which are pumped into the core. How those lines come into existence, whether sourced from flat file, rows in a database, or rendered by a programming language, is up to you. One further thing PLEASE not the default. I don't have the time in my day to learn a whole programming language just to configure my webserver. The problem that needs solving is to find an acceptable tradeoff between allowing the mod_rewrite fans to do complex stuff, while at the same time letting us mere mortals do the simple stuff as well without any drama. Please don't lose sight of that. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Dynamic configuration for the hackathon?
On Mon, 31 Mar 2008 11:15:38 -0700 "Roy T. Fielding" <[EMAIL PROTECTED]> wrote: > On Mar 31, 2008, at 10:39 AM, Paul Querna wrote: > > Just read the mod_rewrite docs: > > http://httpd.apache.org/docs/2.2/rewrite/rewrite_tech.html#InternalAPI > > > > They are already exposing internals to "users'. Not, erm, yer average user ... > > Embed Lua. That is orthogonal to the question I'm addressing. You're going for more powerful; I'm going for more usable! > Unfortunately, after last year's experience of being the only server > person around who wasn't working on a Joost release, ... or tied up in training events ... >I decided to > delay my arrival until Tuesday rather than attend the hackathon. Heh. For the first time, I have the hackathon and no conflicting committment! > Please have fun, finish the server, and let me know what happened > when I arrive; I'll try to squeeze it into my keynote for Friday. ;-) Please sir, we broke it! -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Dynamic configuration for the hackathon?
Torsten Foertsch wrote: On Tue 01 Apr 2008, Paul Querna wrote: William A. Rowe, Jr. wrote: -0.99 - agreed. Perl is perfectly happy having blocks as modular behaviors... I've noticed a trend in the last few years of building on the core (and folks rightfully accused me of growing mod_proxy core when new directives are rightfully part of mod_proxy_{whatever}). Yes, but the root of the problem even with blocks is that they have almost no way to change the behavior of existing modules -- like you can via configuration -- and instead for the most part, they reimplment entire C modules in Perl, or any other language, rather than binding to the configuration, and change that based on some other inputs. I disagree. In the mod_perl API you can almost entirely configure a single request. You can hook maptostorage and add there directives for other modules via $r->add_config. Anything that can be configured in .htaccess or can be done that way as well. One can even change DocumentRoot (prefork-only) for a single request. The PerlMapToStorageHandler can then decide whether to skip the standard maptostorage (directory walk and file walk) or not. It would be good for such a perl-configured request to be able to skip the location-walk that follows maptostorage. But if the admin wants to do that he can simply avoid blocks. You cannot add virtual servers on the fly or redirect error_log. But I don't expect that to become feasible in the new config language since those are things that are done at server startup. I don't think we're even talking about on-the-fly stuff for the "Lua" parser engine, so Perl can do everything that Lua can. As people have mentioned, though, Perl is bloated (but +1 for mp2.1 to include plugs for this new config API :)) Then the existing configuration file, a new lua system, or anything else, could be written in terms of that, rather than the current system where each language reinvents the modules it wants to control. I agree that a configuration language like lua is good but it doesn't necessarily have to be in core. It can be a module as mod_perl shows. I think if we take Paul's idea of a new pluggable API, we'll have a lot of happy people. We can have a lua implementation, a perl implementation, and IMHO we should retain some C implementation to the crappy config that people (users) have now and are used to working with (and bear in mind that there are STILL folks out there in 1.3-land because of what we did to the httpd-2 config). It would make everyone happy by having a C implementation that can be used immediately without any new dependencies (and looks just like the existing engine does), a Lua implementation which can come out-of-the-box with mod_wombat (and we can even make that the "ASF recommended" approach effective immediately as long as we keep everything compatible at least until httpd-3 is ready), and mod_perl (and friends) can provide other interfaces. My EUR 0.02, Issac
Re: Dynamic configuration for the hackathon?
On Tue 01 Apr 2008, Paul Querna wrote: > William A. Rowe, Jr. wrote: > > -0.99 - agreed. Perl is perfectly happy having blocks as modular > > behaviors... I've noticed a trend in the last few years of building on > > the core (and folks rightfully accused me of growing mod_proxy core when > > new directives are rightfully part of mod_proxy_{whatever}). > > Yes, but the root of the problem even with blocks is that they > have almost no way to change the behavior of existing modules -- like > you can via configuration -- and instead for the most part, they > reimplment entire C modules in Perl, or any other language, rather than > binding to the configuration, and change that based on some other inputs. I disagree. In the mod_perl API you can almost entirely configure a single request. You can hook maptostorage and add there directives for other modules via $r->add_config. Anything that can be configured in .htaccess or can be done that way as well. One can even change DocumentRoot (prefork-only) for a single request. The PerlMapToStorageHandler can then decide whether to skip the standard maptostorage (directory walk and file walk) or not. It would be good for such a perl-configured request to be able to skip the location-walk that follows maptostorage. But if the admin wants to do that he can simply avoid blocks. You cannot add virtual servers on the fly or redirect error_log. But I don't expect that to become feasible in the new config language since those are things that are done at server startup. > Then the existing configuration file, a new lua system, or anything > else, could be written in terms of that, rather than the current system > where each language reinvents the modules it wants to control. I agree that a configuration language like lua is good but it doesn't necessarily have to be in core. It can be a module as mod_perl shows. Torsten
Re: Dynamic configuration for the hackathon?
On Mon, Mar 31, 2008 at 7:41 PM, William A. Rowe, Jr. <[EMAIL PROTECTED]> wrote: > I sympathize, but this doesn't reflect the addition of blocks... > those blocks can be trivially implemented as a loadable module ;-) As I grok it, the point would be throw out our ridiculous config syntax entirely (or at best write a compatibility module or a converter to the new format) and expose a real config API (hello providers! *chuckle*) and then build a pure LUA-based config format on top of that. I'd be definitely curious to see what would come of that - 'cuz really it can't be much worse than the garbage we're stuck with now. -- justin
Re: Dynamic configuration for the hackathon?
On Mon, Mar 31, 2008 at 11:15 AM, Roy T. Fielding <[EMAIL PROTECTED]> wrote: > Unfortunately, after last year's experience of being the only server > person around who wasn't working on a Joost release, *hides* > I decided to delay > my arrival until Tuesday rather than attend the hackathon. Please have > fun, finish the server, and let me know what happened when I arrive; For other reasons, I too won't arrive until Tuesday morning. I look forward to seeing what happens on Monday. =P -- justin
Re: Dynamic configuration for the hackathon?
Paul Querna wrote: William A. Rowe, Jr. wrote: Issac Goldstand wrote: Paul Querna wrote: Akins, Brian wrote: On 3/26/08 9:06 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having blocks of code. No need to invent a new language when a perfectly fine one exists... +1, and embed Lua in the core, and a dozen problems just like this are solved. -0.5 PLEASE not in the core. Make mod_wombat a standard module and part of the default moduleset, whatever, but let's not make more core dependencies, please?!? -0.99 - agreed. Perl is perfectly happy having blocks as modular behaviors... I've noticed a trend in the last few years of building on the core (and folks rightfully accused me of growing mod_proxy core when new directives are rightfully part of mod_proxy_{whatever}). Yes, but the root of the problem even with blocks is that they have almost no way to change the behavior of existing modules -- like you can via configuration -- and instead for the most part, they reimplment entire C modules in Perl, or any other language, rather than binding to the configuration, and change that based on some other inputs. The few that can change behaviors, ie via ENV vars, are special cases in every modules case, and do not make a difference in the larger picture. Not completely true. mod_perl provides several ways to bind to the configuration. Consider this bit of Perl magic - this is some mp1 code, running in Perl-land, not in a block, though it could as easily be run there... package Apache::ReadConfig; no strict; $Location{$UploadScript} = { Options => '+ExecCGI', PerlHeaderParserHandler => $namespace."::upload_jit_handler", }; $Location{$UploadMeter} = { Options => '+ExecCGI', PerlHeaderParserHandler => $namespace."::meter_jit_handler", }; $Location{$UploadForm} = { Options => '+ExecCGI', PerlHeaderParserHandler => $namespace."::form_jit_handler", }; Granted, you're still limited to the underlying config parser, so I'm not arguing with the new config API idea (at least as I understood it yesterday), but this should be treated as a pluggable config engine which everyone can access; not as a switch from one config parser to another. Issac
Re: Dynamic configuration for the hackathon?
Paul Querna wrote: Then the existing configuration file, a new lua system, or anything else, could be written in terms of that, rather than the current system where each language reinvents the modules it wants to control. I sympathize, but this doesn't reflect the addition of blocks... those blocks can be trivially implemented as a loadable module ;-)
Re: Dynamic configuration for the hackathon?
William A. Rowe, Jr. wrote: Issac Goldstand wrote: Paul Querna wrote: Akins, Brian wrote: On 3/26/08 9:06 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having blocks of code. No need to invent a new language when a perfectly fine one exists... +1, and embed Lua in the core, and a dozen problems just like this are solved. -0.5 PLEASE not in the core. Make mod_wombat a standard module and part of the default moduleset, whatever, but let's not make more core dependencies, please?!? -0.99 - agreed. Perl is perfectly happy having blocks as modular behaviors... I've noticed a trend in the last few years of building on the core (and folks rightfully accused me of growing mod_proxy core when new directives are rightfully part of mod_proxy_{whatever}). Yes, but the root of the problem even with blocks is that they have almost no way to change the behavior of existing modules -- like you can via configuration -- and instead for the most part, they reimplment entire C modules in Perl, or any other language, rather than binding to the configuration, and change that based on some other inputs. The few that can change behaviors, ie via ENV vars, are special cases in every modules case, and do not make a difference in the larger picture. The core of what I want is a better configuration API. Then the existing configuration file, a new lua system, or anything else, could be written in terms of that, rather than the current system where each language reinvents the modules it wants to control. -Paul
Re: Dynamic configuration for the hackathon?
Issac Goldstand wrote: Paul Querna wrote: Akins, Brian wrote: On 3/26/08 9:06 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having blocks of code. No need to invent a new language when a perfectly fine one exists... +1, and embed Lua in the core, and a dozen problems just like this are solved. -0.5 PLEASE not in the core. Make mod_wombat a standard module and part of the default moduleset, whatever, but let's not make more core dependencies, please?!? -0.99 - agreed. Perl is perfectly happy having blocks as modular behaviors... I've noticed a trend in the last few years of building on the core (and folks rightfully accused me of growing mod_proxy core when new directives are rightfully part of mod_proxy_{whatever}). Let's focus on keeping it in useful pieces, even if they are built by default.
Re: Dynamic configuration for the hackathon?
On Mar 31, 2008, at 10:39 AM, Paul Querna wrote: Just read the mod_rewrite docs: http://httpd.apache.org/docs/2.2/rewrite/rewrite_tech.html#InternalAPI They are already exposing internals to "users'. "Users" want customization. We should just do it right, and stop hacking around the central problem. Expose the structures. Embed Lua. Users like mod_rewrite for many reasons, but I think mostly because it does so many things that almost every Apache hosting provider needs to have it installed and usable in .htaccess, and thus is one of the few solutions that can be taught in a FAQ to powerless users). I like customizations, but I also like servers that stay up forever and do most of their configuration-based thinking outside the critical path of request processing. My worry about Lua is that run-time procedural customizations are far less efficient than precompiled regex tables. In other words, we need both, though we'd be better served if we can do everything in a single, sensible syntax. Unfortunately, after last year's experience of being the only server person around who wasn't working on a Joost release, I decided to delay my arrival until Tuesday rather than attend the hackathon. Please have fun, finish the server, and let me know what happened when I arrive; I'll try to squeeze it into my keynote for Friday. ;-) Roy
Re: Dynamic configuration for the hackathon?
Paul Querna wrote: Issac Goldstand wrote: I think the right approach is to first change the internal configuration API. Make it a real API, not a series of callbacks with filepointers and strings in them. Once we have that, we can write language bindings for all of them, and all languages are on an equal footing. My personal belief is that Lua should be compiled into the core, and enabled by default. This does not really mean a hard dependency. I would *prefer* that it can still be compiled out, and yes, while it would be hard to configure without a 'configuration provider', alternatives could be written in any language, or even with XML input as others have wanted for years, since we now fixed configuration to be an API. That, I'd happily +1 Issac
Re: Dynamic configuration for the hackathon?
Issac Goldstand wrote: Paul Querna wrote: Akins, Brian wrote: On 3/26/08 9:06 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having blocks of code. No need to invent a new language when a perfectly fine one exists... +1, and embed Lua in the core, and a dozen problems just like this are solved. -0.5 PLEASE not in the core. Make mod_wombat a standard module and part of the default moduleset, whatever, but let's not make more core dependencies, please?!? Why not? I look at it this way: We are already making up a "fake" language in our configuration file. If I get voted down (which is still pretty likely especially if all the pro-lua'ers will be at the hackathon, whereas I won't :)) at least No decisions will be made at the hackathon that will not be reflected on the mailing list. consider trying to limit the embedded interpreter to the parent process and preventing it from being inherited by the children, if possible (to remove completely unnecessary bloating) I think the right approach is to first change the internal configuration API. Make it a real API, not a series of callbacks with filepointers and strings in them. Once we have that, we can write language bindings for all of them, and all languages are on an equal footing. My personal belief is that Lua should be compiled into the core, and enabled by default. This does not really mean a hard dependency. I would *prefer* that it can still be compiled out, and yes, while it would be hard to configure without a 'configuration provider', alternatives could be written in any language, or even with XML input as others have wanted for years, since we now fixed configuration to be an API. -Paul
Re: Dynamic configuration for the hackathon?
On 3/31/08 1:39 PM, "Paul Querna" <[EMAIL PROTECTED]> wrote: > We should just do it right, and stop hacking around the central problem. > > Expose the structures. > > Embed Lua. +1, but you already knew that... Also, mod_wombat, as such, goes away if Lua is embedded. We may have a module that sits on top of the embedded lua to give mod_wombat like features (handlers). Lua is not perl. Comparisons between embedding lua (the core of mod_wombat, not the handler stuff to mod_perl is just not the same. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On 3/31/08 1:46 PM, "Issac Goldstand" <[EMAIL PROTECTED]> wrote: > if possible (to > remove completely unnecessary bloating) Lua != perl Lua < perl (size wise by an order of magnitude) > And in > addition, the learning curve to learn to use these powerful directives > is still optional I disagree. It's hard currently bcs every module does their config differently... Some use blocks, some don't. Some support expressions ($stuff, $ENV{foo}), some don't, etc. With Lua (or magic slim interpreter) module author just writes a bit of glue code, no more complicated than the current module "boot-strap" stuff, and it all just works. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
Paul Querna wrote: Akins, Brian wrote: On 3/26/08 9:06 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having blocks of code. No need to invent a new language when a perfectly fine one exists... +1, and embed Lua in the core, and a dozen problems just like this are solved. -0.5 PLEASE not in the core. Make mod_wombat a standard module and part of the default moduleset, whatever, but let's not make more core dependencies, please?!? If I get voted down (which is still pretty likely especially if all the pro-lua'ers will be at the hackathon, whereas I won't :)) at least consider trying to limit the embedded interpreter to the parent process and preventing it from being inherited by the children, if possible (to remove completely unnecessary bloating) Every complicated 'directive' is trying to be a programing language in the config file, but they aren't. Just look at SSLRequire, Rewrite*, MPM Process/Thread Management, Filter chaining, large Auth{N,Z} chains, and more. Imagine them not sucking. Agreed. That's why we have modules like mod_wombat and mod_perl which give you *better* directives. More flexible directives. And in addition, the learning curve to learn to use these powerful directives is still optional (ok - rewrite's a pretty damn big curve itself, but many of the other above items aren't anywhere near as bad). Issac
Re: Dynamic configuration for the hackathon?
Nick Kew wrote: On Thu, 27 Mar 2008 08:17:01 -0400 "Akins, Brian" <[EMAIL PROTECTED]> wrote: On 3/27/08 3:58 AM, "Torsten Foertsch" <[EMAIL PROTECTED]> wrote: So I was going to reimplement it based on mod_wombat some time this year. The nice thing about lua, in addition to being lightweight, is that most of the url mapping/rewriting can be simple lua statements. if string.match(r.uri, '/something') then r.filename = '/www/that/path' end Fine for users who want to hack their own server. Like . But r.filename is the kind of innards we really don't want to expose to the typical mod_rewrite user! I disagree. Just read the mod_rewrite docs: http://httpd.apache.org/docs/2.2/rewrite/rewrite_tech.html#InternalAPI They are already exposing internals to "users'. "Users" want customization. We should just do it right, and stop hacking around the central problem. Expose the structures. Embed Lua. -Paul
Re: Dynamic configuration for the hackathon?
Akins, Brian wrote: On 3/26/08 9:06 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having blocks of code. No need to invent a new language when a perfectly fine one exists... +1, and embed Lua in the core, and a dozen problems just like this are solved. Every complicated 'directive' is trying to be a programing language in the config file, but they aren't. Just look at SSLRequire, Rewrite*, MPM Process/Thread Management, Filter chaining, large Auth{N,Z} chains, and more. Imagine them not sucking. No offense intended to the respective authors of each -- they are all complicated things, and I've helped prolong their lives. I'll be at the hackathon :-) -Paul
Re: Dynamic configuration for the hackathon?
On 3/27/08 9:00 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: > /Lua> > > Fine for users who want to hack their own server. Like . Every play with lighttpd? It's almost the same way... Of course typical lighthttpd user is a "hacker." > But r.filename is the kind of innards we really don't want > to expose to the typical mod_rewrite user! We already expose a lot. You can, indirectly set r->filename with mod_rewrite currently. >> And if the more "complicated" modules had a little lua "glue": >> >> if string.match(r.uri, '/something') then >> mod_cache:cacheable( r ) >> end > > A fine recipe for users shooting themselves in the foot, PHP-style. > How would you propose to make that work without hackage to > existing modules? I don't. Hence the "glue." >> If one were so inclined, the entire configuration could be lua. Just >> define and register these functions that need to run per request. > > That'll go alongside DrBacchus's "You can do everything with > mod_rewrite" :-) If you had lua (or whatever) in configs, you don't need mod_rewrite, alias, etc... Anyway, back to your suggestion of If's and per request configs, I'm +1, especially if it's easily extensible and doesn't parse the tree on every single request. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On Thu, 27 Mar 2008 08:17:01 -0400 "Akins, Brian" <[EMAIL PROTECTED]> wrote: > On 3/27/08 3:58 AM, "Torsten Foertsch" <[EMAIL PROTECTED]> > wrote: > > > So I was going to reimplement it based on mod_wombat some > > time this year. > > > The nice thing about lua, in addition to being lightweight, is that > most of the url mapping/rewriting can be simple lua statements. > > > if string.match(r.uri, '/something') then > r.filename = '/www/that/path' > end > Fine for users who want to hack their own server. Like . But r.filename is the kind of innards we really don't want to expose to the typical mod_rewrite user! > And if the more "complicated" modules had a little lua "glue": > > if string.match(r.uri, '/something') then > mod_cache:cacheable( r ) > end A fine recipe for users shooting themselves in the foot, PHP-style. How would you propose to make that work without hackage to existing modules? > If one were so inclined, the entire configuration could be lua. Just > define and register these functions that need to run per request. That'll go alongside DrBacchus's "You can do everything with mod_rewrite" :-) -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Dynamic configuration for the hackathon?
On 3/27/08 3:58 AM, "Torsten Foertsch" <[EMAIL PROTECTED]> wrote: > So I was going to reimplement it based on mod_wombat some > time this year. The nice thing about lua, in addition to being lightweight, is that most of the url mapping/rewriting can be simple lua statements. if string.match(r.uri, '/something') then r.filename = '/www/that/path' end And if the more "complicated" modules had a little lua "glue": if string.match(r.uri, '/something') then mod_cache:cacheable( r ) end If one were so inclined, the entire configuration could be lua. Just define and register these functions that need to run per request. My $.02 US (which isn't much these days...) -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
I used to use mod_macro, then I moved to mod_perl but like you said. mod_perl is great (well, more okay than great) for dynamic configurations that change/get generated on start and not per request. A new more flexible alternative would be awsome. Jorge (on vacation) On Thu, Mar 27, 2008 at 8:58 AM, Torsten Foertsch <[EMAIL PROTECTED]> wrote: > On Wed 26 Mar 2008, Akins, Brian wrote: > > > There seems to be a demand for dynamic per-request configuration, > > > as evidenced by the number of users hacking it with mod_rewrite, > > > and the other very limited tools available. Modern mod_rewrite > > > usage commonly looks like programming, but it's not designed as > > > a programming language. Result: confused and frustrated users. > > > > This is what I had in mind when I suggested having blocks of code. > > No need to invent a new language when a perfectly fine one exists... > > As Issac pointed out something similar can be done with blocks at > the > cost of having mod_perl in core. Those are not evaluated evaluated > per-request. > > But based on mod_perl there is Apache2::Translation that does per-request > configuration. It hooks uri translation, maptostorage and fixup to do the > job. Again it needs a perl interpreter in core and hence doesn't work well > with threaded MPMs. So I was going to reimplement it based on mod_wombat > some > time this year. > > I just wanted to add these $0.02 to the discussion. > > Torsten > -- ~Jorge
Re: Dynamic configuration for the hackathon?
On Wed 26 Mar 2008, Akins, Brian wrote: > > There seems to be a demand for dynamic per-request configuration, > > as evidenced by the number of users hacking it with mod_rewrite, > > and the other very limited tools available. Modern mod_rewrite > > usage commonly looks like programming, but it's not designed as > > a programming language. Result: confused and frustrated users. > > This is what I had in mind when I suggested having blocks of code. > No need to invent a new language when a perfectly fine one exists... As Issac pointed out something similar can be done with blocks at the cost of having mod_perl in core. Those are not evaluated evaluated per-request. But based on mod_perl there is Apache2::Translation that does per-request configuration. It hooks uri translation, maptostorage and fixup to do the job. Again it needs a perl interpreter in core and hence doesn't work well with threaded MPMs. So I was going to reimplement it based on mod_wombat some time this year. I just wanted to add these $0.02 to the discussion. Torsten
Re: Dynamic configuration for the hackathon?
On 3/26/08 1:14 PM, "Nick Kew" <[EMAIL PROTECTED]> wrote: > we have to parse a string before we have Remote_IP. > Once we have that, sure, our evaluation function can dispatch > to the Remote_IP handler. Of course. I was getting ahead of my self... > You seem to be looking a little further than my proposal went. > Which is kind-of why it would be good to hackathonise this:-) True. Let me digest this some more... -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On Wed, 26 Mar 2008 12:42:51 -0400 "Akins, Brian" <[EMAIL PROTECTED]> wrote: > On 3/26/08 10:31 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: > > > Straightforward: conditions on headers, method (obsoletes ), > > request line, env, CGI vars. With the option to disable conditional > > stuff for speed. > > In mod_include, we parse into a tree on every request. For the > configuration, we should probably just parse it at startup and "run" > it on every request. Indeed - hence the parse/eval separation in the proposed API. > Also, currently, ap_expr is string specific, it would be nice if this > was provider based. Not sure of the exact interface, but it would be > extendable for other types of comparisons, for example. Well, we always start from a string. Later when it's tokenised we can, and indeed do, dispatch to a provider (in mod_include's case, functions called "handle_foo" for keyword foo). > typedef struct { >apr_status_t (*init_expr)(apr_pool_t *p, const char *lvalue, const > char *rvalue, int expr, void**data); > apr_status_t (*eval_expr)(reuqest_rec *r, void *data); > } ap_expr_provider_t; That's no use at top level, because > So this expresssion, at startup: > > > ... > > > Would call the provider registered for "Remote_IP" as: we have to parse a string before we have Remote_IP. Once we have that, sure, our evaluation function can dispatch to the Remote_IP handler. You seem to be looking a little further than my proposal went. Which is kind-of why it would be good to hackathonise this:-) -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Dynamic configuration for the hackathon?
On 3/26/08 12:42 PM, "Akins, Brian" <[EMAIL PROTECTED]> wrote: > Thoughts? Of course, it will not work exactly as I have said because we have to take stuff like variable substitution into account, etc. Was just thinking out loud... -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On 3/26/08 10:31 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: > Straightforward: conditions on headers, method (obsoletes ), > request line, env, CGI vars. With the option to disable conditional > stuff for speed. In mod_include, we parse into a tree on every request. For the configuration, we should probably just parse it at startup and "run" it on every request. Also, currently, ap_expr is string specific, it would be nice if this was provider based. Not sure of the exact interface, but it would be extendable for other types of comparisons, for example. typedef struct { apr_status_t (*init_expr)(apr_pool_t *p, const char *lvalue, const char *rvalue, int expr, void**data); apr_status_t (*eval_expr)(reuqest_rec *r, void *data); } ap_expr_provider_t; So this expresssion, at startup: ... Would call the provider registered for "Remote_IP" as: provider->init_expr(conf->pool, "Remote_IP", "10.189.", AP_EXPR_REGEX, &data); The provider would construct what ever struct it needs, in this case, to do partial ip address matching, shove that into a struct and return it via the data argument; And then, at request time, we would run: provider->eval_expr(r, data) Where data is what was returned by init. This returns basically true or false. The string stuff could be easily integrated and provided "by default." The nice thing, is all of this could be then used in mod_include as well, as well as any other modules that use ap_expr. Thoughts? -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On Wed, 26 Mar 2008 10:15:05 -0400 "Akins, Brian" <[EMAIL PROTECTED]> wrote: > As to your suggestion: > > So basically, the per_dir merge would use this mechanism instead of > what it does now (file walk, location walk)> (or in addition to??) > > Something like: > > > SetEnv coolstuff > > Set something different > > foo bar > >something completely different > Sort-of. There's a question of ordering: merge-config happens before some of the vars we'd like to make available are available, so we'd have a bit more work to make Cookie{baz} work (as opposed to parsing a CGI-style HTTP_COOKIE variable). But that's basically the kind of thing. Oh, and there's no inherent reason it shouldn't apply per-host config too. > (Horrible, example I know). If it were easy to extend the expresions > (ie, I want to implement (Cache == yes/no) and stuff like ENV{key} > were made to work, I'm all for it. Straightforward: conditions on headers, method (obsoletes ), request line, env, CGI vars. With the option to disable conditional stuff for speed. Higher-level stuff like *evaluating* caching or cookie headers ... maybe sometime, but one could argue that's the point where or makes more sense. > It *should* be fairly easy to test this out with the current system > (ala Proxy blocks). Hopefully, yes. Oh, and since ap_expr is a prerequisite for this, it would be great if folks could review at least the API part (ap_expr.h) of what I posted earlier. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Dynamic configuration for the hackathon?
On 3/26/08 9:53 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: > I'm not talking about inventing a new language. Those who want one > have some options already, as noted below ... Right. I was just "throwing it out there," so to speak. I'm not opposed to what you are saying, just wondering if we would/should take it to the next level. As to your suggestion: So basically, the per_dir merge would use this mechanism instead of what it does now (file walk, location walk)> (or in addition to??) Something like: SetEnv coolstuff Set something different foo bar something completely different (Horrible, example I know). If it were easy to extend the expresions (ie, I want to implement (Cache == yes/no) and stuff like ENV{key} were made to work, I'm all for it. It *should* be fairly easy to test this out with the current system (ala Proxy blocks). -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On Wed, 26 Mar 2008 15:39:53 +0200 Issac Goldstand <[EMAIL PROTECTED]> wrote: > > > > Akins, Brian wrote: > > On 3/26/08 9:06 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: > > > >> There seems to be a demand for dynamic per-request configuration, > >> as evidenced by the number of users hacking it with mod_rewrite, > >> and the other very limited tools available. Modern mod_rewrite > >> usage commonly looks like programming, but it's not designed as > >> a programming language. Result: confused and frustrated users. > > > > > > This is what I had in mind when I suggested having blocks of > > code. No need to invent a new language when a perfectly fine one > > exists... I'm not talking about inventing a new language. Those who want one have some options already, as noted below ... > > FWIW, it's done with blocks too (I do some funky things that > way), BUT I'm not sure if those are parsed per-request as I think > Nick is suggesting. Neither am I, FWIW. > Also, many times people don't want to bloat > their processes with a fully-fleged interpreter That is much more of a consideration. As I said, the basic idea is to provide a much simpler rationalisation for the kind of things people struggle to do with mod_rewrite et al. >(again, I'm building > on my mod_perl experience here - I know that the shared Perl objects > are pretty clunky, and not sure if mod_wombat looks the same). AFAICT mod_wombat just provides Lua bindings (some of them stubs) for hooks exported from the core. Nothing for configuration. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Dynamic configuration for the hackathon?
Akins, Brian wrote: On 3/26/08 9:06 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having blocks of code. No need to invent a new language when a perfectly fine one exists... FWIW, it's done with blocks too (I do some funky things that way), BUT I'm not sure if those are parsed per-request as I think Nick is suggesting. Also, many times people don't want to bloat their processes with a fully-fleged interpreter (again, I'm building on my mod_perl experience here - I know that the shared Perl objects are pretty clunky, and not sure if mod_wombat looks the same). Right now it doesn't look like I'll be in Amsterdam... Issac
Re: Dynamic configuration for the hackathon?
On 3/26/08 9:06 AM, "Nick Kew" <[EMAIL PROTECTED]> wrote: > There seems to be a demand for dynamic per-request configuration, > as evidenced by the number of users hacking it with mod_rewrite, > and the other very limited tools available. Modern mod_rewrite > usage commonly looks like programming, but it's not designed as > a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having blocks of code. No need to invent a new language when a perfectly fine one exists... -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Dynamic configuration for the hackathon?
There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. We could make simple changes to mod_rewrite itself: for example, a container would at least bring users basic block structuring. But so long as the block context applies only to mod_rewrite, it remains ad-hoc tinkering with the problem. I'm wondering what it would take to get us to something like: Directives applicable per-request (anything, subject to per-directive context checking) (ideally with and ) Clearly an container has to create a configuration record that'll be merged if and only if the condition is satisfied by a request. The condition should have access to headers_in and subprocess_env (with client info stuff included), as well as the request line. As further step we could consider evaluating the contents per-request, with support for variable interpolation and backreferences using ap_expr. A noble objective: render mod_rewrite obsolete :-) Anyone fancy spending some hackathon time on this in Amsterdam? -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/