Re: mpm-itk and upstream Apache, once again
On Sun, Nov 11, 2012 at 08:25:08AM -0500, Jeff Trawick wrote: > I'll have a look again "soon." Like my “soon”s, this :-) > Earlier I couldn't think of a more efficient or direct mechanism that makes > sense as an API I guess it depends on how mpm-itk specific you want it to be. One could maybe have a post-htaccess hook that gets run with the status, but that would probably be less useful for a hypothetical other module? > IIRC, mpm-itk only needs to call stat() on the file to check for > permissions, and since (in the successful case) the file is about to be > opened I assume that the performance degradation is minimal. I could probably change to stat, but it would be a bit complicated to get right, since often you can stat a file without opening it, and ap_pcfg_* had lots of stuff I didn't want to delve into there and then :-) Maybe access() would be better than stat(), and good enough. (There's a small race window if the permissions are changed between access() and open(), but I guess we can live with that.) Do you think it would be possible to get these hooks into 2.4.4 or 2.4.5? /* Steinar */ -- Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
I'll have a look again "soon." Earlier I couldn't think of a more efficient or direct mechanism that makes sense as an API IIRC, mpm-itk only needs to call stat() on the file to check for permissions, and since (in the successful case) the file is about to be opened I assume that the performance degradation is minimal. (It has been a while since I looked at the code.) On Thu, Nov 8, 2012 at 4:22 PM, Steinar H. Gunderson wrote: > On Thu, Nov 08, 2012 at 08:53:12PM +0100, Steinar H. Gunderson wrote: > >> I've looked at this now; sorry for the long delay. It would seem it is > not > >> sufficient for removing the patches from server/config.c (which exit if > >> .htaccess files cannot be opened); or am I misunderstanding something? > > Sorry, I'm just confused. Back to coding... > > Yes, indeed, it works. Out-of-tree-built mpm-itk, yay! > > But it's not ideal, since I don't get the status of the ap_pcfg_openfile() > call in the htaccess hook (since it's before that), and I also can't > return a > file pointer. The only way I can see to do this in mpm-itk is to run > ap_pcfg_openfile() myself to see if it goes through or not, and then close > it > and return DECLINED if the call was successful. That would seem somewhat > inefficient. > > /* Steinar */ > -- > Homepage: http://www.sesse.net/ > -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: mpm-itk and upstream Apache, once again
On Thu, Nov 08, 2012 at 08:53:12PM +0100, Steinar H. Gunderson wrote: >> I've looked at this now; sorry for the long delay. It would seem it is not >> sufficient for removing the patches from server/config.c (which exit if >> .htaccess files cannot be opened); or am I misunderstanding something? > Sorry, I'm just confused. Back to coding... Yes, indeed, it works. Out-of-tree-built mpm-itk, yay! But it's not ideal, since I don't get the status of the ap_pcfg_openfile() call in the htaccess hook (since it's before that), and I also can't return a file pointer. The only way I can see to do this in mpm-itk is to run ap_pcfg_openfile() myself to see if it goes through or not, and then close it and return DECLINED if the call was successful. That would seem somewhat inefficient. /* Steinar */ -- Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
On Thu, Nov 08, 2012 at 08:51:50PM +0100, Steinar H. Gunderson wrote: > I've looked at this now; sorry for the long delay. It would seem it is not > sufficient for removing the patches from server/config.c (which exit if > .htaccess files cannot be opened); or am I misunderstanding something? Sorry, I'm just confused. Back to coding... /* Steinar */ -- Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
On Mon, Sep 24, 2012 at 08:44:21AM -0400, Jeff Trawick wrote: > I went ahead and committed this to trunk as r1389339. Hopefully this > completes the ability to enable mpm-itk without patches to httpd core. I've looked at this now; sorry for the long delay. It would seem it is not sufficient for removing the patches from server/config.c (which exit if .htaccess files cannot be opened); or am I misunderstanding something? /* Steinar */ -- Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
On Mon, Sep 24, 2012 at 08:44:21AM -0400, Jeff Trawick wrote: >> Attached is a patch that adds a hook called just before htaccess is >> opened. See if you can use that to resolve the remaining issue. > I went ahead and committed this to trunk as r1389339. Hopefully this > completes the ability to enable mpm-itk without patches to httpd core. Hi, Thanks for doing this. I'm currently a bit hindered from testing it out, but I'll get back to you with the results once things settle down a bit :-) /* Steinar */ -- Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
On Fri, Sep 21, 2012 at 8:29 AM, Jeff Trawick wrote:
> On Sun, Aug 5, 2012 at 11:05 AM, Jeff Trawick wrote:
>> On Sun, Aug 5, 2012 at 11:00 AM, Steinar H. Gunderson
>> wrote:
>>> On Wed, Aug 01, 2012 at 01:58:16PM -0400, Jeff Trawick wrote:
Your post-perdir-config patch has been committed to trunk with r1368121.
>>>
>>> Thanks!
>>>
Attached is a patch to trunk that allows you to hook in to the stat
calls from directory walk. Call apr_stat() like core_dirwalk_stat()
but check for APR_STATUS_IS_EACCES(rv) and decide whether to run
lingering close and exit. Let us know how that goes.
You still need the parse-htaccess patch for now.
>>>
>>> I backported this to 2.4.2, and changed mpm-itk to hook into that function
>>> with the following hook:
>>>
>>> static apr_status_t itk_dirwalk_stat(apr_finfo_t *finfo, request_rec *r,
>>>apr_int32_t wanted)
>>> {
>>> apr_status_t status = apr_stat(finfo, r->filename, wanted, r->pool);
>>> if (ap_has_irreversibly_setuid && APR_STATUS_IS_EACCES(status)) {
>>>ap_log_rerror(APLOG_MARK, APLOG_WARNING, status, r,
>>> "Couldn't read %s, closing connection.",
>>> r->filename);
>>>ap_lingering_close(r->connection);
>>>clean_child_exit(0);
>>> }
>>> return status;
>>> }
>>>
>>> Seems to work great, from my limited testing. As an extra bonus, I can
>>> easily
>>> call clean_child_exit() (which runs more cleanup hooks) instead of exit(),
>>> since this is in the MPM's own .c file.
>>
>> Great! I'll do something about the remaining patch "before long".
>
> It has been a while :)
>
> The dirwalk_stat hook has now been committed:
> http://svn.apache.org/viewvc?view=revision&revision=1388447
>
> Attached is a patch that adds a hook called just before htaccess is
> opened. See if you can use that to resolve the remaining issue.
I went ahead and committed this to trunk as r1389339. Hopefully this
completes the ability to enable mpm-itk without patches to httpd core.
>
>>
>>>
>>> /* Steinar */
>>> --
>>> Homepage: http://www.sesse.net/
>>
>>
>>
>> --
>> Born in Roswell... married an alien...
>> http://emptyhammock.com/
>
>
>
> --
> Born in Roswell... married an alien...
> http://emptyhammock.com/
--
Born in Roswell... married an alien...
http://emptyhammock.com/
Re: mpm-itk and upstream Apache, once again
A very interesting read for myself. The hat I am wearing is
"admin/implementer" of AIX RBAC (role based access control) where the whole
application is running as a non-root id (or non-superuser if you prefer).
The kernel privileges granted include setting "privileges" and accessing
"all" files - which I shall be looking at to limit/manage removing the
privileges and seeing what "breaks".
If a seteuid() call is made - could it do that to root? must look into that
(not so much httpd, but RBAC mechanisms in general).
My goal with an RBACed configuration of httpd is that a non-superuser could
fully administer httpd - start/stop service; edit key config files.
So the interesting part of this read - maybe httpd has "unknown to me"
needs to be "more super". And I may even have a reason to actually look at
parts of the code :)
On Fri, Sep 21, 2012 at 2:29 PM, Jeff Trawick wrote:
> On Sun, Aug 5, 2012 at 11:05 AM, Jeff Trawick wrote:
> > On Sun, Aug 5, 2012 at 11:00 AM, Steinar H. Gunderson
> > wrote:
> >> On Wed, Aug 01, 2012 at 01:58:16PM -0400, Jeff Trawick wrote:
> >>> Your post-perdir-config patch has been committed to trunk with
> r1368121.
> >>
> >> Thanks!
> >>
> >>> Attached is a patch to trunk that allows you to hook in to the stat
> >>> calls from directory walk. Call apr_stat() like core_dirwalk_stat()
> >>> but check for APR_STATUS_IS_EACCES(rv) and decide whether to run
> >>> lingering close and exit. Let us know how that goes.
> >>>
> >>> You still need the parse-htaccess patch for now.
> >>
> >> I backported this to 2.4.2, and changed mpm-itk to hook into that
> function
> >> with the following hook:
> >>
> >> static apr_status_t itk_dirwalk_stat(apr_finfo_t *finfo, request_rec
> *r,
> >>apr_int32_t wanted)
> >> {
> >> apr_status_t status = apr_stat(finfo, r->filename, wanted,
> r->pool);
> >> if (ap_has_irreversibly_setuid && APR_STATUS_IS_EACCES(status)) {
> >>ap_log_rerror(APLOG_MARK, APLOG_WARNING, status, r,
> >> "Couldn't read %s, closing connection.",
> >> r->filename);
> >>ap_lingering_close(r->connection);
> >>clean_child_exit(0);
> >> }
> >> return status;
> >> }
> >>
> >> Seems to work great, from my limited testing. As an extra bonus, I can
> easily
> >> call clean_child_exit() (which runs more cleanup hooks) instead of
> exit(),
> >> since this is in the MPM's own .c file.
> >
> > Great! I'll do something about the remaining patch "before long".
>
> It has been a while :)
>
> The dirwalk_stat hook has now been committed:
> http://svn.apache.org/viewvc?view=revision&revision=1388447
>
> Attached is a patch that adds a hook called just before htaccess is
> opened. See if you can use that to resolve the remaining issue.
>
> >
> >>
> >> /* Steinar */
> >> --
> >> Homepage: http://www.sesse.net/
> >
> >
> >
> > --
> > Born in Roswell... married an alien...
> > http://emptyhammock.com/
>
>
>
> --
> Born in Roswell... married an alien...
> http://emptyhammock.com/
>
Re: mpm-itk and upstream Apache, once again
On Sun, Aug 5, 2012 at 11:05 AM, Jeff Trawick wrote:
> On Sun, Aug 5, 2012 at 11:00 AM, Steinar H. Gunderson
> wrote:
>> On Wed, Aug 01, 2012 at 01:58:16PM -0400, Jeff Trawick wrote:
>>> Your post-perdir-config patch has been committed to trunk with r1368121.
>>
>> Thanks!
>>
>>> Attached is a patch to trunk that allows you to hook in to the stat
>>> calls from directory walk. Call apr_stat() like core_dirwalk_stat()
>>> but check for APR_STATUS_IS_EACCES(rv) and decide whether to run
>>> lingering close and exit. Let us know how that goes.
>>>
>>> You still need the parse-htaccess patch for now.
>>
>> I backported this to 2.4.2, and changed mpm-itk to hook into that function
>> with the following hook:
>>
>> static apr_status_t itk_dirwalk_stat(apr_finfo_t *finfo, request_rec *r,
>>apr_int32_t wanted)
>> {
>> apr_status_t status = apr_stat(finfo, r->filename, wanted, r->pool);
>> if (ap_has_irreversibly_setuid && APR_STATUS_IS_EACCES(status)) {
>>ap_log_rerror(APLOG_MARK, APLOG_WARNING, status, r,
>> "Couldn't read %s, closing connection.",
>> r->filename);
>>ap_lingering_close(r->connection);
>>clean_child_exit(0);
>> }
>> return status;
>> }
>>
>> Seems to work great, from my limited testing. As an extra bonus, I can easily
>> call clean_child_exit() (which runs more cleanup hooks) instead of exit(),
>> since this is in the MPM's own .c file.
>
> Great! I'll do something about the remaining patch "before long".
It has been a while :)
The dirwalk_stat hook has now been committed:
http://svn.apache.org/viewvc?view=revision&revision=1388447
Attached is a patch that adds a hook called just before htaccess is
opened. See if you can use that to resolve the remaining issue.
>
>>
>> /* Steinar */
>> --
>> Homepage: http://www.sesse.net/
>
>
>
> --
> Born in Roswell... married an alien...
> http://emptyhammock.com/
--
Born in Roswell... married an alien...
http://emptyhammock.com/
Index: server/config.c
===
--- server/config.c (revision 1388440)
+++ server/config.c (working copy)
@@ -80,6 +80,7 @@
APR_HOOK_LINK(quick_handler)
APR_HOOK_LINK(optional_fn_retrieve)
APR_HOOK_LINK(test_config)
+ APR_HOOK_LINK(pre_htaccess)
)
AP_IMPLEMENT_HOOK_RUN_ALL(int, header_parser,
@@ -171,6 +172,9 @@
AP_IMPLEMENT_HOOK_RUN_FIRST(int, quick_handler, (request_rec *r, int lookup),
(r, lookup), DECLINED)
+AP_IMPLEMENT_HOOK_RUN_FIRST(int, pre_htaccess, (request_rec *r, const char
*filename),
+(r, filename), DECLINED)
+
/* hooks with no args are implemented last, after disabling APR hook probes */
#if defined(APR_HOOK_PROBES_ENABLED)
#undef APR_HOOK_PROBES_ENABLED
@@ -2078,6 +2082,7 @@
struct htaccess_result *new;
ap_conf_vector_t *dc = NULL;
apr_status_t status;
+int rc;
/* firstly, search cache */
for (cache = r->htaccess; cache != NULL; cache = cache->next) {
@@ -2104,6 +2109,10 @@
*/
filename = ap_make_full_path(r->pool, d,
ap_getword_conf(r->pool, &access_name));
+rc = ap_run_pre_htaccess(r, filename);
+if (rc != DECLINED && rc != OK) {
+return rc;
+}
status = ap_pcfg_openfile(&f, r->pool, filename);
if (status == APR_SUCCESS) {
Index: include/http_config.h
===
--- include/http_config.h (revision 1388440)
+++ include/http_config.h (working copy)
@@ -1322,6 +1322,15 @@
AP_DECLARE_HOOK(void,optional_fn_retrieve,(void))
/**
+ * Allow modules to perform a check immediately prior to opening htaccess.
+ * @param r The current request
+ * @param filename The htaccess file which will be processed
+ * @return HTTP status code to fail the operation, or DECLINED to let later
+ * modules decide
+ */
+AP_DECLARE_HOOK(int,pre_htaccess,(request_rec *r, const char *filename))
+
+/**
* A generic pool cleanup that will reset a pointer to NULL. For use with
* apr_pool_cleanup_register.
* @param data The address of the pointer
Re: mpm-itk and upstream Apache, once again
On 8/5/2012 8:32 AM, Steinar H. Gunderson wrote: > On Sun, Aug 05, 2012 at 11:05:59AM -0400, Jeff Trawick wrote: >> Great! I'll do something about the remaining patch "before long". > > When the time comes, do we have any hopes of getting this back from trunk to > 2.4, or would it need to wait for 2.6/3.0? > > FWIW, the mpm-itk security hardening that was discussed (running with uid != > 0, > and limiting setuid/setgid ranges through seccomp) is starting to come quite > nicely along, although the problem of initgroups() remains (a rogue process > with CAP_SETGID can add any supplementary group it pleases, and seccomp is > unable to check it), and there's been very limited user testing so far. > I guess we can't get fully down to the level of prefork, but it can get > pretty close. Steinar, I solved a very similar problem by spinning off a low-numbered port daemon which accesses resources (in this case, port 21 or similar) and returns the fd to the caller after it evaluates whether that request is permitted by the configuration parsed when it was launched. The solution might give you some ideas on how this mpm could have very limited root privileges with very specific purposes, and not at risk from any remote code execution flaws discovered in the future; http://svn.apache.org/repos/asf/httpd/mod_ftp/trunk/modules/ftp/ftp_lowportd.c Hope it inspires some interesting improvements :) Bill
Re: mpm-itk and upstream Apache, once again
On Sun, Aug 5, 2012 at 11:32 AM, Steinar H. Gunderson wrote: > On Sun, Aug 05, 2012 at 11:05:59AM -0400, Jeff Trawick wrote: >> Great! I'll do something about the remaining patch "before long". > > When the time comes, do we have any hopes of getting this back from trunk to > 2.4, or would it need to wait for 2.6/3.0? 2.4.small-number > > FWIW, the mpm-itk security hardening that was discussed (running with uid != > 0, > and limiting setuid/setgid ranges through seccomp) is starting to come quite > nicely along, although the problem of initgroups() remains (a rogue process > with CAP_SETGID can add any supplementary group it pleases, and seccomp is > unable to check it), and there's been very limited user testing so far. > I guess we can't get fully down to the level of prefork, but it can get > pretty close. > > /* Steinar */ > -- > Homepage: http://www.sesse.net/ -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: mpm-itk and upstream Apache, once again
On Sun, Aug 05, 2012 at 11:05:59AM -0400, Jeff Trawick wrote: > Great! I'll do something about the remaining patch "before long". When the time comes, do we have any hopes of getting this back from trunk to 2.4, or would it need to wait for 2.6/3.0? FWIW, the mpm-itk security hardening that was discussed (running with uid != 0, and limiting setuid/setgid ranges through seccomp) is starting to come quite nicely along, although the problem of initgroups() remains (a rogue process with CAP_SETGID can add any supplementary group it pleases, and seccomp is unable to check it), and there's been very limited user testing so far. I guess we can't get fully down to the level of prefork, but it can get pretty close. /* Steinar */ -- Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
On Sun, Aug 5, 2012 at 11:00 AM, Steinar H. Gunderson
wrote:
> On Wed, Aug 01, 2012 at 01:58:16PM -0400, Jeff Trawick wrote:
>> Your post-perdir-config patch has been committed to trunk with r1368121.
>
> Thanks!
>
>> Attached is a patch to trunk that allows you to hook in to the stat
>> calls from directory walk. Call apr_stat() like core_dirwalk_stat()
>> but check for APR_STATUS_IS_EACCES(rv) and decide whether to run
>> lingering close and exit. Let us know how that goes.
>>
>> You still need the parse-htaccess patch for now.
>
> I backported this to 2.4.2, and changed mpm-itk to hook into that function
> with the following hook:
>
> static apr_status_t itk_dirwalk_stat(apr_finfo_t *finfo, request_rec *r,
>apr_int32_t wanted)
> {
> apr_status_t status = apr_stat(finfo, r->filename, wanted, r->pool);
> if (ap_has_irreversibly_setuid && APR_STATUS_IS_EACCES(status)) {
>ap_log_rerror(APLOG_MARK, APLOG_WARNING, status, r,
> "Couldn't read %s, closing connection.",
> r->filename);
>ap_lingering_close(r->connection);
>clean_child_exit(0);
> }
> return status;
> }
>
> Seems to work great, from my limited testing. As an extra bonus, I can easily
> call clean_child_exit() (which runs more cleanup hooks) instead of exit(),
> since this is in the MPM's own .c file.
Great! I'll do something about the remaining patch "before long".
>
> /* Steinar */
> --
> Homepage: http://www.sesse.net/
--
Born in Roswell... married an alien...
http://emptyhammock.com/
Re: mpm-itk and upstream Apache, once again
On Wed, Aug 01, 2012 at 01:58:16PM -0400, Jeff Trawick wrote:
> Your post-perdir-config patch has been committed to trunk with r1368121.
Thanks!
> Attached is a patch to trunk that allows you to hook in to the stat
> calls from directory walk. Call apr_stat() like core_dirwalk_stat()
> but check for APR_STATUS_IS_EACCES(rv) and decide whether to run
> lingering close and exit. Let us know how that goes.
>
> You still need the parse-htaccess patch for now.
I backported this to 2.4.2, and changed mpm-itk to hook into that function
with the following hook:
static apr_status_t itk_dirwalk_stat(apr_finfo_t *finfo, request_rec *r,
apr_int32_t wanted)
{
apr_status_t status = apr_stat(finfo, r->filename, wanted, r->pool);
if (ap_has_irreversibly_setuid && APR_STATUS_IS_EACCES(status)) {
ap_log_rerror(APLOG_MARK, APLOG_WARNING, status, r,
"Couldn't read %s, closing connection.",
r->filename);
ap_lingering_close(r->connection);
clean_child_exit(0);
}
return status;
}
Seems to work great, from my limited testing. As an extra bonus, I can easily
call clean_child_exit() (which runs more cleanup hooks) instead of exit(),
since this is in the MPM's own .c file.
/* Steinar */
--
Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
On Thu, Jul 19, 2012 at 11:38 AM, Steinar H. Gunderson
wrote:
> On Thu, Jul 19, 2012 at 11:27:04AM -0400, Jeff Trawick wrote:
>> What changes are needed to httpd trunk so that you can build mpm-itk
>> with apxs and enable it via LoadModule, such that mpm-itk is fully
>> functional? As I'm sure you're aware, prefork, worker, and event are
>> all untied from core enough to support that in httpd >= 2.4.
>
> We'd need:
>
> 1. A hook right after merging the perdir config.
> 2. Fixes to get Apache to drop the connection if it detects
> (during .htaccess lookup) that it would need to change the uid.
>
> Both patches are simple, although for #2 to be truly generic (ie. be usable
> by mod_privileges as well) we'd need some sort of signalling mechanism saying
> “we have switched uids and cannot switch back”, which then both
> mod_privileges (in secure mode) and mpm-itk could set.
>
> I've attached the current versions of both patches from my current Apache 2.4
> patch set; you can see the “ap_running_under_mpm_itk” variable which would
> probably need to be replaced by ap_mpm_query() or similar.
>
> /* Steinar */
> --
> Homepage: http://www.sesse.net/
>
Your post-perdir-config patch has been committed to trunk with r1368121.
(http://svn.apache.org/viewvc?view=revision&revision=r1368121)
Attached is a patch to trunk that allows you to hook in to the stat
calls from directory walk. Call apr_stat() like core_dirwalk_stat()
but check for APR_STATUS_IS_EACCES(rv) and decide whether to run
lingering close and exit. Let us know how that goes.
You still need the parse-htaccess patch for now.
--
Born in Roswell... married an alien...
http://emptyhammock.com/
Index: server/core.c
===
--- server/core.c (revision 1368124)
+++ server/core.c (working copy)
@@ -4779,6 +4779,12 @@
return APR_SUCCESS;
}
+static apr_status_t core_dirwalk_stat(apr_finfo_t *finfo, request_rec *r,
+ apr_int32_t wanted)
+{
+return apr_stat(finfo, r->filename, wanted, r->pool);
+}
+
static void core_dump_config(apr_pool_t *p, server_rec *s)
{
core_server_config *sconf = ap_get_core_module_config(s->module_config);
@@ -4855,7 +4861,8 @@
ap_hook_child_status(ap_core_child_status, NULL, NULL, APR_HOOK_MIDDLE);
ap_hook_insert_network_bucket(core_insert_network_bucket, NULL, NULL,
APR_HOOK_REALLY_LAST);
-
+ap_hook_dirwalk_stat(core_dirwalk_stat, NULL, NULL, APR_HOOK_REALLY_LAST);
+
/* register the core's insert_filter hook and register core-provided
* filters
*/
Index: server/request.c
===
--- server/request.c(revision 1368131)
+++ server/request.c(working copy)
@@ -70,6 +70,7 @@
APR_HOOK_LINK(insert_filter)
APR_HOOK_LINK(create_request)
APR_HOOK_LINK(post_perdir_config)
+APR_HOOK_LINK(dirwalk_stat)
)
AP_IMPLEMENT_HOOK_RUN_FIRST(int,translate_name,
@@ -93,6 +94,9 @@
(request_rec *r), (r), OK, DECLINED)
AP_IMPLEMENT_HOOK_RUN_ALL(int, post_perdir_config,
(request_rec *r), (r), OK, DECLINED)
+AP_IMPLEMENT_HOOK_RUN_FIRST(apr_status_t,dirwalk_stat,
+(apr_finfo_t *finfo, request_rec *r, apr_int32_t
wanted),
+(finfo, r, wanted), AP_DECLINED)
static int auth_internal_per_conf = 0;
static int auth_internal_per_conf_hooks = 0;
@@ -609,7 +613,7 @@
* with APR_ENOENT, knowing that the path is good.
*/
if (r->finfo.filetype == APR_NOFILE || r->finfo.filetype == APR_LNK) {
-rv = apr_stat(&r->finfo, r->filename, APR_FINFO_MIN, r->pool);
+rv = ap_run_dirwalk_stat(&r->finfo, r, APR_FINFO_MIN);
/* some OSs will return APR_SUCCESS/APR_REG if we stat
* a regular file but we have '/' at the end of the name;
@@ -675,9 +679,8 @@
* check.
*/
if (!(opts & OPT_SYM_LINKS)) {
-rv = apr_stat(&thisinfo, r->filename,
- APR_FINFO_MIN | APR_FINFO_NAME | APR_FINFO_LINK,
- r->pool);
+rv = ap_run_dirwalk_stat(&thisinfo, r,
+ APR_FINFO_MIN | APR_FINFO_NAME |
APR_FINFO_LINK);
/*
* APR_INCOMPLETE is as fine as result as APR_SUCCESS as we
* have added APR_FINFO_NAME to the wanted parameter of
@@ -1092,9 +1095,8 @@
* the name of its target, if we are fixing the filename
* case/resolving aliases.
*/
-rv = apr_stat(&thisinfo, r->filename,
- APR_FINFO_MIN | APR_FINFO_NAME | APR_FINFO_LINK,
- r->pool);
+rv = ap_run_dirwalk_stat(&thisinfo, r,
+ APR_FINFO_MIN
Re: mpm-itk and upstream Apache, once again
On Thu, Jul 19, 2012 at 04:17:44PM +0200, Steinar H. Gunderson wrote: > Furthermore, Fedora has recently accepted the mpm-itk patch into their Apache > packages. For the record, that is not accurate. The Fedora httpd package does not contain the mpm-itk patch, I have repeatedly refused to add it since it makes changes to core httpd which need to go through upstream review. Using such arguments to "bully" upstream into taking patches is not a great way to build goodwill, IMO. Somebody has apparently added a completely separate httpd-itk package which does include the patch, which is totally dumb IMO, but we (Fedora) apparently don't legislate against that in the review process. Regards, Joe
Re: mpm-itk and upstream Apache, once again
On Sun, Jul 22, 2012 at 09:57:18PM +0200, Stefan Fritsch wrote: > And if it gets secured to where a code execution exploit does not grant > full root rights, I would probably be in favor of including it with httpd. I took a look using seccomp for this, and it would seem it is actually rather hard; you can limit setuid() and setgid() easily enough to a range (so that you'd typically exclude root, daemon and other low-uid stuff), but setgroups() takes in a pointer of supplementary gids to add. I can't find any good ways of looking into that list, so it would seem a rogue process could arbitrarily add any gid (like, 0) to its list. So it seems to be hard to properly restrict gid, but maybe restricting uid would already give a significant win? /* Steinar */ -- Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
On Sunday 22 July 2012, Steinar H. Gunderson wrote: > On Sun, Jul 22, 2012 at 09:57:18PM +0200, Stefan Fritsch wrote: > > On reason may be that (at least in theory), mod_privileges is > > more secure: Under Solaris you cannot get uid 0 unless you > > already have all privileges, so an exploited httpd with > > mod_privileges does not give you root. Under Linux however, > > CAP_SETUID is equivalent to full unrestricted root, because it > > allows you to write to root owned files (like /etc/crontab or > > /root/.ssh/authorized_keys). > > It is possible to set prctl(PR_SET_KEEPCAPS, 1) and then setuid() > to a non-root user and still keep the privileges. This would > remove the write to /etc/crontab; I am unsure what happens in > practice if the process does goes back to uid 0 via setuid(), but > in Linux 3.5 or newer, we can disallow that through a seccomp BPF > filter. (Arguably this does not help FreeBSD, Solaris or any other > non-Linux OS. Also, Linux 3.5 was only released this week, and > thus needs some more time before it gets into the hands of a > typical server admin.) As long as the process has CAP_SETUID, it can switch back to uid 0 and write root owned files. But seccomp BPF should be able to fix that. For FreeBSD, one could use capsicum. > > It also a question of the expected maintenance overhead and the > > available man-power. For example, according to Debian's popcon, > > mod_fcgid has around three times as many users as mpm-itk, but it > > is not part of core httpd either. One of the reasons is that not > > enough active developers are really interested in mod_fcgid. > > Again, this argument does not really make sense considering the > inclusion of mod_privileges, which surely must have a much smaller > user base than mpm-itk. Then, it seems, Jeff is correct: There is no consistent set of criteria wether to include or not to include a module. Or maybe the criteria change over time (mod_privileges has been added 4 years ago). > > Yes, that would be a good thing. It would be even better if it > > could work as normal (non-mpm) module together with mpm-prefork. > > This would probably require some rearchitecting; it's certainly > possible (mod_privileges has already shown that), but it's quite a > lot of work, given that we do expect some cooperation from the MPM > currently (e.g. to keep the scoreboard). What's the primary > advantage -- less code duplication? Yes, that's the primary argument. And maybe being able to use the same hooks in different interesting ways. From a superficial glance, it seems that (in addition to the two new hooks already mentioned) hooking near ap_run_create_connection into mpm-prefork would take care of most of it. But I could be wrong. > I think we should try to get mpm-itk working without patching > Apache first; then we can revisit changing it from an MPM into a > normal module, if there is consensus that it would help its > inclusion. Sure.
Re: mpm-itk and upstream Apache, once again
On Sun, Jul 22, 2012 at 09:57:18PM +0200, Stefan Fritsch wrote: > On reason may be that (at least in theory), mod_privileges is more > secure: Under Solaris you cannot get uid 0 unless you already have all > privileges, so an exploited httpd with mod_privileges does not give > you root. Under Linux however, CAP_SETUID is equivalent to full > unrestricted root, because it allows you to write to root owned files > (like /etc/crontab or /root/.ssh/authorized_keys). It is possible to set prctl(PR_SET_KEEPCAPS, 1) and then setuid() to a non-root user and still keep the privileges. This would remove the write to /etc/crontab; I am unsure what happens in practice if the process does goes back to uid 0 via setuid(), but in Linux 3.5 or newer, we can disallow that through a seccomp BPF filter. (Arguably this does not help FreeBSD, Solaris or any other non-Linux OS. Also, Linux 3.5 was only released this week, and thus needs some more time before it gets into the hands of a typical server admin.) > It also a question of the expected maintenance overhead and the > available man-power. For example, according to Debian's popcon, > mod_fcgid has around three times as many users as mpm-itk, but it is > not part of core httpd either. One of the reasons is that not enough > active developers are really interested in mod_fcgid. Again, this argument does not really make sense considering the inclusion of mod_privileges, which surely must have a much smaller user base than mpm-itk. >> If we can really get mpm-itk compilable out-of-tree without Apache >> patches, that would certainly be a better situation than what we >> have today. > Yes, that would be a good thing. It would be even better if it could > work as normal (non-mpm) module together with mpm-prefork. This would probably require some rearchitecting; it's certainly possible (mod_privileges has already shown that), but it's quite a lot of work, given that we do expect some cooperation from the MPM currently (e.g. to keep the scoreboard). What's the primary advantage -- less code duplication? I think we should try to get mpm-itk working without patching Apache first; then we can revisit changing it from an MPM into a normal module, if there is consensus that it would help its inclusion. /* Steinar */ -- Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
On Friday 20 July 2012, Steinar H. Gunderson wrote: > On Fri, Jul 20, 2012 at 01:48:33PM -0400, Jeff Trawick wrote: > >> Why would you keep mpm-itk separate but mod_privileges not? On reason may be that (at least in theory), mod_privileges is more secure: Under Solaris you cannot get uid 0 unless you already have all privileges, so an exploited httpd with mod_privileges does not give you root. Under Linux however, CAP_SETUID is equivalent to full unrestricted root, because it allows you to write to root owned files (like /etc/crontab or /root/.ssh/authorized_keys). > > IMO it is not a very relevant question given the big picture: > > > > * Most modules written for httpd are not bundled with the server > > or otherwise hosted/developed at the ASF. > > * mod_privileges is for a minority server operating system and is > > not used extensively even there. > > * You won't find much rhyme or reason to why some modules are > > bundled and some are not other than whether or the author has > > commit access, and even then there isn't much consistency. > > I'm not sure if I understand this. Should not “the module is > considered useful” be a better criteria than “the module is > written by someone with commit access”? And how can “the module > has a very small user base” be an argument _for_ keeping it in > trunk, and the more popular one out? > > If nothing else, should not a module that's patched in by a > significant fraction be pulled into the main tree, to lighten the > burden on distributors? It also a question of the expected maintenance overhead and the available man-power. For example, according to Debian's popcon, mod_fcgid has around three times as many users as mpm-itk, but it is not part of core httpd either. One of the reasons is that not enough active developers are really interested in mod_fcgid. > > As far as mpm-itk: A few hooks can be added to httpd core so > > that it can be enabled just like other modules*, whether or not > > anyone here cares about the implementation details. > > > > *Of course that isn't really true of the popular 2.2.x branch, > > but I don't think it is realistic to hope that mpm-itk would > > ever make it to 2.2.x anyway. > > If we can really get mpm-itk compilable out-of-tree without Apache > patches, that would certainly be a better situation than what we > have today. Yes, that would be a good thing. It would be even better if it could work as normal (non-mpm) module together with mpm-prefork. And if it gets secured to where a code execution exploit does not grant full root rights, I would probably be in favor of including it with httpd.
Re: mpm-itk and upstream Apache, once again
On Thursday 19 July 2012, Graham Leggett wrote: > On 19 Jul 2012, at 18:07, Tim Bannister wrote: > > On 19 Jul 2012, at 17:26, Nick Kew wrote: > > > > > >>> 2. Fixes to get Apache to drop the connection if it detects > >>> (during .htaccess lookup) that it would need to change the > >>> uid. > >> > >> > >> > >> Dropping the connection gratuitously breaks HTTP, and so has no > >> place in httpd (of course, a third-party module sets its own > >> rules). Would it need a core patch to return an Internal Server > >> Error (500)? > > > > > > > > Vanilla httpd does this all the time… after a timed-out > > keepalive. The client cannot make any assumptions about the > > configured timeout, and can't tell whether the dropped > > connection is due to a genuine timeout or a UID mismatch between > > the previous and current request. > > I would hate to have to troubleshoot this - two completely > independent behaviors, with the same symptom but completely > different cause. > > Nick is right, a 500 is the right thing to do here. No, mpm itk only closes the connection if a keep-alive request would need to be processed with a different uid than the previous request on the same connection. I.e., it cannot happen for the first request on a connection (which would cause breakage). But dropping the conn for a keep-alive request is allowed, and there are other modules that do it in some cases (including mod_proxy and mpm_event).
Re: mpm-itk and upstream Apache, once again
On Thu, Jul 19, 2012 at 05:26:23PM +0100, Nick Kew wrote:
> How does it protect against such potential attacks as running an
> external program as root through a RewriteMap running earlier
> than the directory walk?
By the way, I actually tried this under prefork. I compiled httpd-2.4.2
with prefork and the following configuration in a vhost:
RewriteEngine on
Rewritemap examplemap prg:/home/sesse/mymapper.pl
RewriteRule /invalid %{examplemap:$1}
and lo and behold, mymapper.pl is started as root. mod_rewrite seems to open
the map programs already when parsing the configuration file, which is before
the MPMs' hooks run (and that's when prefork drops its privileges).
/* Steinar */
--
Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
> If nothing else, should not a module that's patched in by a significant > fraction be pulled into the main tree, to lighten the burden on distributors? Personally, this is not a big consideration in my vote on this project taking responsibility for a module.
Re: mpm-itk and upstream Apache, once again
On Fri, Jul 20, 2012 at 1:56 PM, Steinar H. Gunderson wrote: > On Fri, Jul 20, 2012 at 01:48:33PM -0400, Jeff Trawick wrote: >>> Why would you keep mpm-itk separate but mod_privileges not? >> >> IMO it is not a very relevant question given the big picture: >> >> * Most modules written for httpd are not bundled with the server or >> otherwise hosted/developed at the ASF. >> * mod_privileges is for a minority server operating system and is not >> used extensively even there. >> * You won't find much rhyme or reason to why some modules are bundled >> and some are not other than whether or the author has commit access, >> and even then there isn't much consistency. > > I'm not sure if I understand this. Should not “the module is considered > useful” be a better criteria than “the module is written by someone with > commit access”? And how can “the module has a very small user base” be an > argument _for_ keeping it in trunk, and the more popular one out? I don't think there's much to understand; you can point at the current slate of included and excluded modules that we ended up with after 15 or so years of development and read from that anything you want (or read nothing from it, as I suggest). > If nothing else, should not a module that's patched in by a significant > fraction be pulled into the main tree, to lighten the burden on distributors? Distributors can generally take care of themselves. Some active httpd developers represent distributors of httpd, and they can jump in and help if they think the technical justification and integration burden warrant. >> As far as mpm-itk: A few hooks can be added to httpd core so that it >> can be enabled just like other modules*, whether or not anyone here >> cares about the implementation details. >> >> *Of course that isn't really true of the popular 2.2.x branch, but I >> don't think it is realistic to hope that mpm-itk would ever make it to >> 2.2.x anyway. > > If we can really get mpm-itk compilable out-of-tree without Apache patches, > that would certainly be a better situation than what we have today. > (The situation with 2.2.x will work itself out in time, of course, > as distributions and users slowly migrate to 2.4.x.) > >> By the way, did any other httpd-ers have a look at those patches and >> have suggestions for what hooks could be added? > > I don't know how many have actually looked at the code in detail; there was > some light review around the time of the initial 2.4.x port, but I generally > do not receive a lot of feedback on the httpd integration itself. I guess > it's not a part that make a lot of people excited. I'll try to make some progress with API extensions anyway if nobody speaks up soon. > > /* Steinar */ > -- > Homepage: http://www.sesse.net/ > -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: mpm-itk and upstream Apache, once again
On Fri, Jul 20, 2012 at 01:48:33PM -0400, Jeff Trawick wrote: >> Why would you keep mpm-itk separate but mod_privileges not? > > IMO it is not a very relevant question given the big picture: > > * Most modules written for httpd are not bundled with the server or > otherwise hosted/developed at the ASF. > * mod_privileges is for a minority server operating system and is not > used extensively even there. > * You won't find much rhyme or reason to why some modules are bundled > and some are not other than whether or the author has commit access, > and even then there isn't much consistency. I'm not sure if I understand this. Should not “the module is considered useful” be a better criteria than “the module is written by someone with commit access”? And how can “the module has a very small user base” be an argument _for_ keeping it in trunk, and the more popular one out? If nothing else, should not a module that's patched in by a significant fraction be pulled into the main tree, to lighten the burden on distributors? > As far as mpm-itk: A few hooks can be added to httpd core so that it > can be enabled just like other modules*, whether or not anyone here > cares about the implementation details. > > *Of course that isn't really true of the popular 2.2.x branch, but I > don't think it is realistic to hope that mpm-itk would ever make it to > 2.2.x anyway. If we can really get mpm-itk compilable out-of-tree without Apache patches, that would certainly be a better situation than what we have today. (The situation with 2.2.x will work itself out in time, of course, as distributions and users slowly migrate to 2.4.x.) > By the way, did any other httpd-ers have a look at those patches and > have suggestions for what hooks could be added? I don't know how many have actually looked at the code in detail; there was some light review around the time of the initial 2.4.x port, but I generally do not receive a lot of feedback on the httpd integration itself. I guess it's not a part that make a lot of people excited. /* Steinar */ -- Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
On Fri, Jul 20, 2012 at 12:26 PM, Steinar H. Gunderson wrote: > On Thu, Jul 19, 2012 at 06:54:56PM +0100, Tim Bannister wrote: >> I think there's a case for leaving itk separate, a bit like mod_fcgid. It >> is a bit unusual and troubleshooting won't be straightforward. > > Why would you keep mpm-itk separate but mod_privileges not? IMO it is not a very relevant question given the big picture: * Most modules written for httpd are not bundled with the server or otherwise hosted/developed at the ASF. * mod_privileges is for a minority server operating system and is not used extensively even there. * You won't find much rhyme or reason to why some modules are bundled and some are not other than whether or the author has commit access, and even then there isn't much consistency. As far as mpm-itk: A few hooks can be added to httpd core so that it can be enabled just like other modules*, whether or not anyone here cares about the implementation details. *Of course that isn't really true of the popular 2.2.x branch, but I don't think it is realistic to hope that mpm-itk would ever make it to 2.2.x anyway. By the way, did any other httpd-ers have a look at those patches and have suggestions for what hooks could be added?
Re: mpm-itk and upstream Apache, once again
On Thu, Jul 19, 2012 at 06:54:56PM +0100, Tim Bannister wrote: > I think there's a case for leaving itk separate, a bit like mod_fcgid. It > is a bit unusual and troubleshooting won't be straightforward. Why would you keep mpm-itk separate but mod_privileges not? /* Steinar */ -- Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
On Thu, Jul 19, 2012 at 05:26:23PM +0100, Nick Kew wrote: > Does it run per-dir config as root? Yes, although it has very limited root rights; it can setuid and it can read arbitrary files and directories, but it cannot e.g. load kernel modules or write to arbitrary files. > How does it protect against such potential attacks as running an > external program as root through a RewriteMap running earlier > than the directory walk? If the administrator (who will naturally be root) sets up such a RewriteMap, it will naturally be executed as (limited) root. RewriteMap is not supported in .htaccess. Note that this is very much comparable to mod_privileges, for both cases. With seccomp v2 support (coming in Linux 3.5), we can also restrict the uid ranges to setuid to. > Given that the header_parser runs immediately after directory > config in request.c, are there specific reasons (beyond inelegance) > not to run as header_parser with REALLY_FIRST? As far as I understand, we don't have the per-directory configuration merged at that stage. Supporting separate vhosts in a more fine-grained fashion than was a common request (we now support and , and I'm looking at integrating ap_expr support for 2.4). > Dropping the connection gratuitously breaks HTTP, and so has no > place in httpd (of course, a third-party module sets its own rules). On the contrary, RFC 2616 explicitly allows this behavior; section 8.1.4 states “A client, server, or proxy MAY close the transport connection at any time. [...] This means that clients, servers, and proxies MUST be able to recover from asynchronous close events. Client software SHOULD reopen the transport connection and retransmit the aborted sequence of requests without user interaction so long as the request sequence is idempotent (see section 9.1.2).” All clients that I know of deal with this without any problems; the only problem I've ever seen is with old versions of Varnish, and that has long since been fixed as a bug in Varnish. > Would it need a core patch to return an Internal Server Error (500)? Without any patches, it will return 403. In any case, 500 or other HTTP error messages are not very useful; it breaks several useful use cases, such as a) A user browsing to vhost-with-uid-1001, and then before the timeout follows a link to vhost-with-uid-1002. b) vhost-with-uid-1001 includes an image (or other media) from vhost-with-uid-1002. c) A proxy (reverse or otherwise) or web crawler needs to access vhost-with-uid-1001 and vhost-with-uid-1002 in quick succession (similar to a) ). These are not hypothetical situations; clients actually do this, and they will return hard errors to the user if they are presented with HTTP error messages. /* Steinar */ -- Homepage: http://www.sesse.net/
Re: mpm-itk and upstream Apache, once again
On 19 Jul 2012, at 18:22, Graham Leggett wrote: > I would hate to have to troubleshoot this - two completely independent > behaviors, with the same symptom but completely different cause. > > Nick is right, a 500 is the right thing to do here. I'm really not convinced. I'd expect a user agent to retry in the keepalive-disconnect case, whereas a 500 response usually gets displayed to the user. Very different experiences. I think there's a case for leaving itk separate, a bit like mod_fcgid. It is a bit unusual and troubleshooting won't be straightforward. -- Tim Bannister – is...@jellybaby.net smime.p7s Description: S/MIME cryptographic signature
Re: mpm-itk and upstream Apache, once again
On 19 Jul 2012, at 18:07, Tim Bannister wrote: > On 19 Jul 2012, at 17:26, Nick Kew wrote: > >>> 2. Fixes to get Apache to drop the connection if it detects (during >>> .htaccess lookup) that it would need to change the uid. >> >> Dropping the connection gratuitously breaks HTTP, and so has no place in >> httpd (of course, a third-party module sets its own rules). Would it need a >> core patch to return an Internal Server Error (500)? > > Vanilla httpd does this all the time… after a timed-out keepalive. The client > cannot make any assumptions about the configured timeout, and can't tell > whether the dropped connection is due to a genuine timeout or a UID mismatch > between the previous and current request. I would hate to have to troubleshoot this - two completely independent behaviors, with the same symptom but completely different cause. Nick is right, a 500 is the right thing to do here. Regards, Graham --
Re: mpm-itk and upstream Apache, once again
On 19 Jul 2012, at 17:26, Nick Kew wrote: >> 2. Fixes to get Apache to drop the connection if it detects (during >> .htaccess lookup) that it would need to change the uid. > > Dropping the connection gratuitously breaks HTTP, and so has no place in > httpd (of course, a third-party module sets its own rules). Would it need a > core patch to return an Internal Server Error (500)? Vanilla httpd does this all the time… after a timed-out keepalive. The client cannot make any assumptions about the configured timeout, and can't tell whether the dropped connection is due to a genuine timeout or a UID mismatch between the previous and current request. -- Tim Bannister - +44 7980408788 - is...@jellybaby.net smime.p7s Description: S/MIME cryptographic signature
Re: mpm-itk and upstream Apache, once again
On Thu, Jul 19, 2012 at 11:38 AM, Steinar H. Gunderson wrote: > On Thu, Jul 19, 2012 at 11:27:04AM -0400, Jeff Trawick wrote: >> What changes are needed to httpd trunk so that you can build mpm-itk >> with apxs and enable it via LoadModule, such that mpm-itk is fully >> functional? As I'm sure you're aware, prefork, worker, and event are >> all untied from core enough to support that in httpd >= 2.4. > > We'd need: > > 1. A hook right after merging the perdir config. > 2. Fixes to get Apache to drop the connection if it detects > (during .htaccess lookup) that it would need to change the uid. > > Both patches are simple, although for #2 to be truly generic (ie. be usable > by mod_privileges as well) we'd need some sort of signalling mechanism saying > “we have switched uids and cannot switch back”, which then both > mod_privileges (in secure mode) and mpm-itk could set. > > I've attached the current versions of both patches from my current Apache 2.4 > patch set; you can see the “ap_running_under_mpm_itk” variable which would > probably need to be replaced by ap_mpm_query() or similar. I suspect that an ap_hook_stat() to be called from directory walk would allow itk to keep its odd processing private, and might be useful to other modules as well. For ap_parse_htaccess(), I suspect that some mechanism for plugging into the whole htaccess-is-a-file mess could enable interesting features beyond mpm-itk, but perhaps the minimum is the appropriate solution: Use a hook in place of a direct call to ap_pcfg_openfile(). The post-perdir-config hook looks reasonable to me. -- While the usefulness of a new hook or that by other existing modules (mod_privileges perhaps) to solve a current problem is interesting, it should be sufficient to determine that a new hook can be put to good use by some theoretical module. > > /* Steinar */ > -- > Homepage: http://www.sesse.net/ > -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: mpm-itk and upstream Apache, once again
On Thu, 19 Jul 2012 17:38:31 +0200 "Steinar H. Gunderson" wrote: > On Thu, Jul 19, 2012 at 11:27:04AM -0400, Jeff Trawick wrote: > > What changes are needed to httpd trunk so that you can build mpm-itk > > with apxs and enable it via LoadModule, such that mpm-itk is fully > > functional? As I'm sure you're aware, prefork, worker, and event are > > all untied from core enough to support that in httpd >= 2.4. > > We'd need: > > 1. A hook right after merging the perdir config. Does it run per-dir config as root? How does it protect against such potential attacks as running an external program as root through a RewriteMap running earlier than the directory walk? Given that the header_parser runs immediately after directory config in request.c, are there specific reasons (beyond inelegance) not to run as header_parser with REALLY_FIRST? > 2. Fixes to get Apache to drop the connection if it detects > (during .htaccess lookup) that it would need to change the uid. Dropping the connection gratuitously breaks HTTP, and so has no place in httpd (of course, a third-party module sets its own rules). Would it need a core patch to return an Internal Server Error (500)? -- Nick Kew
Re: mpm-itk and upstream Apache, once again
On Thu, Jul 19, 2012 at 11:27:04AM -0400, Jeff Trawick wrote:
> What changes are needed to httpd trunk so that you can build mpm-itk
> with apxs and enable it via LoadModule, such that mpm-itk is fully
> functional? As I'm sure you're aware, prefork, worker, and event are
> all untied from core enough to support that in httpd >= 2.4.
We'd need:
1. A hook right after merging the perdir config.
2. Fixes to get Apache to drop the connection if it detects
(during .htaccess lookup) that it would need to change the uid.
Both patches are simple, although for #2 to be truly generic (ie. be usable
by mod_privileges as well) we'd need some sort of signalling mechanism saying
“we have switched uids and cannot switch back”, which then both
mod_privileges (in secure mode) and mpm-itk could set.
I've attached the current versions of both patches from my current Apache 2.4
patch set; you can see the “ap_running_under_mpm_itk” variable which would
probably need to be replaced by ap_mpm_query() or similar.
/* Steinar */
--
Homepage: http://www.sesse.net/
Add an extra hook right after merging per-directory configuration.
This makes sure we are able to setuid() as early as possible (that
is, as soon as know what uid/gid to use for this location), so we
won't run all sorts of subrequests and other stuff as root.
Index: httpd-2.4.1/server/request.c
===
--- httpd-2.4.1.orig/server/request.c
+++ httpd-2.4.1/server/request.c
@@ -69,6 +69,7 @@ APR_HOOK_STRUCT(
APR_HOOK_LINK(auth_checker)
APR_HOOK_LINK(insert_filter)
APR_HOOK_LINK(create_request)
+APR_HOOK_LINK(post_perdir_config)
)
AP_IMPLEMENT_HOOK_RUN_FIRST(int,translate_name,
@@ -91,6 +92,21 @@ AP_IMPLEMENT_HOOK_VOID(insert_filter, (r
AP_IMPLEMENT_HOOK_RUN_ALL(int, create_request,
(request_rec *r), (r), OK, DECLINED)
+/**
+ * This hook allows modules to affect the request immediately after the
+ * per-directory configuration for the request has been generated. This allows
+ * modules to make decisions based upon the current directory configuration
+ *
+ * This hook is private to mpm-itk, so it is not exposed in http_request.h.
+ *
+ * @param r The current request
+ * @return OK or DECLINED
+ */
+AP_DECLARE_HOOK(int,post_perdir_config,(request_rec *r))
+
+AP_IMPLEMENT_HOOK_RUN_ALL(int,post_perdir_config,
+ (request_rec *r), (r), OK, DECLINED)
+
static int auth_internal_per_conf = 0;
static int auth_internal_per_conf_hooks = 0;
static int auth_internal_per_conf_providers = 0;
@@ -191,6 +207,13 @@ AP_DECLARE(int) ap_process_request_inter
r->log = d->log;
}
+/* First chance to handle the request after per-directory configuration is
+ * generated
+ */
+if ((access_status = ap_run_post_perdir_config(r))) {
+return access_status;
+}
+
/* Only on the main request! */
if (r->main == NULL) {
if ((access_status = ap_run_header_parser(r))) {
Fix an issue where users can sometimes get spurious 403s on persistent
connections (the description in the comments explains the logic).
This would particularly hit people with reverse proxies, since these
have a higher tendency of accessing things from different vhosts in
the same connection.
Index: httpd-2.4.2/server/config.c
===
--- httpd-2.4.2.orig/server/config.c
+++ httpd-2.4.2/server/config.c
@@ -69,6 +69,8 @@ AP_DECLARE_DATA apr_array_header_t *ap_s
AP_DECLARE_DATA ap_directive_t *ap_conftree = NULL;
+AP_DECLARE_DATA int ap_running_under_mpm_itk = 0;
+
APR_HOOK_STRUCT(
APR_HOOK_LINK(header_parser)
APR_HOOK_LINK(pre_config)
@@ -2129,6 +2131,32 @@ AP_CORE_DECLARE(int) ap_parse_htaccess(a
else {
if (!APR_STATUS_IS_ENOENT(status)
&& !APR_STATUS_IS_ENOTDIR(status)) {
+/*
+ * If we are in a persistent connection, we might end up in a state
+ * where we can no longer read .htaccess files because we have already
+ * setuid(). This can either be because the previous request was for
+ * another vhost (basically the same problem as when setuid() fails in
+ * itk.c), or it can be because a .htaccess file is readable only by
+ * root.
+ *
+ * In any case, we don't want to give out a 403, since the request has
+ * a very real chance of succeeding on a fresh connection (where
+ * presumably uid=0). Thus, we give up serving the request on this
+ * TCP connection, and do a hard close of the socket. As long as we're
+ * in a persistent connection (and there _should_ not be a way this
+ * would happen on the first request in a connection, save for subrequests,
+ * which we
Re: mpm-itk and upstream Apache, once again
On Thu, Jul 19, 2012 at 10:17 AM, Steinar H. Gunderson wrote: > Hi, > > I've asked previously on this list about inclusion of mpm-itk > (http://mpm-itk.sesse.net/) into upstream Apache; previously, the requests > have died down, mostly over discussions on security (mpm-itk does > configuration and request parsing as uid 0, although with very limited > capabilities) and arguments along the lines of “there is no need”, > e.g. various people I've talked to feel that there are other adequate > solutions for the problem, including suexec, multiple Apache instances with > reverse proxying, or some GSoC project. > (http://wiki.apache.org/httpd/PrivilegeSeparation even claims you can > keep administrators from reading each others' sites simply by setting > setting chmod 0640, completely ignoring the case where you can run PHP code > or CGI scripts!) > > However, since then mod_privileges have entered Apache trunk, which gives > similar functionality (contradicting the arguments about “no need”), is very > similar in terms of security model (contradicting the arguments about “the > model is too insecure”), but is Solaris-specific, has less functionality (it > lacks per-vhost nicing and per-vhost client limits), and generally seems to > be less mature (e.g., as far as I can see, it fails to adequately handle the > case where the client goes to a different-uid vhost and .htaccess thus is > not readable). > > Furthermore, Fedora has recently accepted the mpm-itk patch into their Apache > packages. This means that nearly every major distributor of Apache now > supports mpm-itk; in particular, Arch, Debian, Fedora, FreeBSD ports, Gentoo, > Mandriva, openSUSE and Ubuntu all include mpm-itk. I do not know of any > module with a similar status, and having them all integrate the patch > separately instead of simply having it in mainline seems wasteful. > > mpm-itk has, despite its non-mainline status, been in production in large > sites for many years (it has been under development since 2005), and should > at this point be considered mature. What would be needed to get it into > mainline? I personally don't want to think about getting mpm-itk into mainline, but I am interested in the following, which is largely a prerequisite to what you requested: What changes are needed to httpd trunk so that you can build mpm-itk with apxs and enable it via LoadModule, such that mpm-itk is fully functional? As I'm sure you're aware, prefork, worker, and event are all untied from core enough to support that in httpd >= 2.4. > > /* Steinar */ > -- > Homepage: http://www.sesse.net/ > -- Born in Roswell... married an alien... http://emptyhammock.com/
mpm-itk and upstream Apache, once again
Hi, I've asked previously on this list about inclusion of mpm-itk (http://mpm-itk.sesse.net/) into upstream Apache; previously, the requests have died down, mostly over discussions on security (mpm-itk does configuration and request parsing as uid 0, although with very limited capabilities) and arguments along the lines of “there is no need”, e.g. various people I've talked to feel that there are other adequate solutions for the problem, including suexec, multiple Apache instances with reverse proxying, or some GSoC project. (http://wiki.apache.org/httpd/PrivilegeSeparation even claims you can keep administrators from reading each others' sites simply by setting setting chmod 0640, completely ignoring the case where you can run PHP code or CGI scripts!) However, since then mod_privileges have entered Apache trunk, which gives similar functionality (contradicting the arguments about “no need”), is very similar in terms of security model (contradicting the arguments about “the model is too insecure”), but is Solaris-specific, has less functionality (it lacks per-vhost nicing and per-vhost client limits), and generally seems to be less mature (e.g., as far as I can see, it fails to adequately handle the case where the client goes to a different-uid vhost and .htaccess thus is not readable). Furthermore, Fedora has recently accepted the mpm-itk patch into their Apache packages. This means that nearly every major distributor of Apache now supports mpm-itk; in particular, Arch, Debian, Fedora, FreeBSD ports, Gentoo, Mandriva, openSUSE and Ubuntu all include mpm-itk. I do not know of any module with a similar status, and having them all integrate the patch separately instead of simply having it in mainline seems wasteful. mpm-itk has, despite its non-mainline status, been in production in large sites for many years (it has been under development since 2005), and should at this point be considered mature. What would be needed to get it into mainline? /* Steinar */ -- Homepage: http://www.sesse.net/
