[idea] web-application security powered by SELinux

[KaiGai Kohei]

Hello,

Now I have considered the way to work web-applications with restrictive
privileges set based on an identification of the client. It enables to
check and prevent violated actions from (buggy) applications using
features provided by the operating system.

I'm concerned about most of web-application, such as PHP scripts, are
launched as part of web-server process itself. It also means all the
web-application instances share the same privilege set of the server
process, even if these are invoked on the requests come from different
users. In other word, we cannot apply valid access controls on them
(except for ones applied by web-application itself, but it is hard to
ensure they don't have security bugs), because it seems to the operating
system multiple processes/threads with same privileges run simultaneously.

If we can run web-applications with more restrictive privileges set
for each users, groups and so on, the operating system can acquire
any actions from userspace and apply its access control policies.
I assume SELinux as the operating system feature here, but not
limited to SELinux. I guess this discussion can be applied on any
other advanced security features also.

In my opinion, we need the following three facilities:

1. The backend identifies the client and decide what privileges should
   be assigned on the launched web-applications prior to its invocation.
   The existing http-authentication is a candidate, but we don't assume
   a certain method to identify the client.

2. The backend creates a one-time thread or process to handle the given
   requests, and assigns the new privileges, then launches the worker
   thread or process to invoke contents handler.

3. The backend wait for the completion of the worker. The reason why
   we don't use them again is that it is fundamentally danger to allow
   a path to revert the privileges of web-application.

Please note that we don't assume users who want to use this feature
give first priority for performance. It is a kind of security tradeoff.

One idea is to add a security focused MPM which provide above features
and hooks for external modules.
I've actually developed a working example based on the "prefork" MPM.
When it accepts a request from the client, it creates a one-time thread
and assigns a new security context (which is a privileges set in SELinux),
then invokes contents handler.

  http://code.google.com/p/sepgsql/source/browse/misc/httpd-selinux/

However, I don't adhere the current implementation as is.
I would like to have a discussion to brush up the idea to achieve
the goal and to get acceptance in the mainline.

Any comments, questions and others are welcome.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei 

Re: [face2face] BOF Thurs 8:30pm at ApacheCon

[Ruediger Pluem]

On 24.03.2009 17:19, William A. Rowe, Jr. wrote:
> Apache httpd Server 3.0; What Color is your Pony?
> 
> A Birds of a Feather session for developers and users
> 
> 20:30-21:30; Thursday 26-03; Lausanne Room; Moevenpick

Lets make a photo there.

Regards

Rüdiger

Re: SIGQUIT using default signal handler instead of sig_coredump

[Jeff Trawick]

On Tue, Mar 24, 2009 at 11:20 PM, Ruediger Pluem  wrote:

> On 24.03.2009 19:12, Eric Covener wrote:
> > SIGQUIT (3) does not go through sig_coredump(). Is this intentional to
> > short-circuit the sig_coredump handling in the case something is
> > broken there?
>

I dunno about the intentions.


>
>
> IMHO SIGQUIT isn't supposed to cause a coredump. Only "fatal" signals are
> supposed to do this. If you need a coredump of your running process just
> use
> gcore or gdb.


See http://en.wikipedia.org/wiki/SIGQUIT

I think the issue is that SIGQUIT does terminate the process with a core
dump (at least on some systems).  Given that, the question is whether we
should respect any CoredumpDirectory setting on the way out.  Respecting
CoredumpDirectory seems reasonable.

OS X:

$ lsap
 9292 1 ./httpd
 9293  9292 ./httpd
 9294  9292 ./httpd
 9295  9292 ./httpd
 9296  9292 ./httpd
 9297  9292 ./httpd
$ kill -QUIT 9297
$ ls /cores/
core.9297

(/cores/ is the default location here.)

Re: Mod_proxy_http override

[Ruediger Pluem]

On 24.03.2009 19:57, Dingwell, Robert A. wrote:
> Hi,
> 
> Is there a way to override http proxy handling to use a different module
> other then mod_proxy_http?  I¹ve been trying to use apache as a forwarding
> proxy but I have an issue with the way it handles connection refused cases
> from a server. 
> 
> I believe my issue lies within the code section below from the
> proxy_http_handler  function in mod_proxy_http.c but I want to make sure I
> understand this correctly.
> 
> Step One is looking up the address of the server to contact on behalf of the
> client, if it cannot find an address for  the server it fails and goes onto
> cleanup which will send Ostatus¹ back to the client.
> 
> Getting to Step Two means that it found an address and Step Two itself is
> going to attempt to make the initial connection to that address.  My issue
> then comes into play where the address of a server is known but the server
> could refuse the connection on a particular port for some reason such as the
> site is down, or whatever.   If this happens and Apache is configured to
> ProxyRequests, the status is set to HTTP_NOT_FOUND, which really isnt the
> case, the server refused the connection it's not that the page wasn't found.
> 
> 
> Am I mistaken is thinking that r->proxyreq == PROXYREQ_PROXY  is always true
> when ProxyRequests is set to On in the configuration? If so how do I force

Yes. It is only true for forward proxy requests not for reverse proxy requests.

> the HTTP_SERVICE_UNAVAILABLE instead of the HTTP_NOT_FOUND? Do I have to

IMHO it shouldn't send either but a HTTP_BAD_GATEWAY.

Regards

Rüdiger

Patch: mod_lua usage of APR_DECLARE / compiling using MSVC

[Peter Cawley]

The APR_DECLARE and AP_LUA_DECLARE both perform useful (and different)
tasks when compiling under win32/MSVC. Attached is a patch which I
think corrects the usage of these macros in mod_lua - it certainly
cuts down on the number of errors spat out by MSVC. Also included is
moving "apr_table_t *rs;" up a line in req_dispatch so that it is
declared at the top of it's enclosing block (MSVC's C compiler doesn't
allow variables to be declared in the middle of a block).

With these changes made, there are just 4 errors remaining when I try
to compile mod_lua using MSVC (those being unresolved external
symbols: _ap_args_to_table, _ap_body_to_table,
__imp__apr_optional_hook_...@4 and __imp__apr_optional_hook_...@20). I
believe that at least the former two are because my win32 copy of
Apache is 2.2.x, and those functions were added in 2.3.x - are there
prebuilt binary versions of 2.3 available?

Peter


win32compile.patch
Description: Binary data

Re: SIGQUIT using default signal handler instead of sig_coredump

[Ruediger Pluem]

On 24.03.2009 19:12, Eric Covener wrote:
> SIGQUIT (3) does not go through sig_coredump(). Is this intentional to
> short-circuit the sig_coredump handling in the case something is
> broken there?

IMHO SIGQUIT isn't supposed to cause a coredump. Only "fatal" signals are
supposed to do this. If you need a coredump of your running process just use
gcore or gdb.

Regards

Rüdiger

Re: mod_lua check for lua compilation error

[Bertrand Mansion]

On Tue, Mar 24, 2009 at 7:57 PM, Brian McCallister  wrote:
> On Sat, Mar 21, 2009 at 10:54 AM, Bertrand Mansion  
> wrote:
>> In order to be able to detect if a lua file fails to compile and get
>> an informative error message like :
>>  Error!
>>  /web/localhost/htdocs/info.lua:174: '=' expected near '+'
>> instead of :
>>  Error!
>>  attempt to call a nil value
>>
>> I suggest you move the load/pcall function from
>> lua_vmprep:apl_get_lua_state() to mod_lua:lua_handler() and
>> mod_lua:lua_request_rec_hook_harness() like in the attached patch.
>
> I disagree with this change, but agree with the intention. We need a
> good way to get compilation errors back to folks asking for the
> lua_State. We could use a state flag in the vm create callback which
> says what happened -- instead of just calling it with a valid
> lua_State when it is created, it will be called every time, with an
> enumeration of what happened, one of { create, reuse, syntax_error,
> other_error } or such.
>
> I think the lua_State may hold the compilation problems, but am not
> sure, need to go poke around -- I won't have a chance for a while
> still, am dealing with a family medical issue for a couple weeks
> (nothing urgent, just needing attention).

Peter's suggestion is ok for me. I just need to figure out how to do that :)

>> I haven't yet tested how my patch works with hooks but it seems to be
>> ok for lua_handler. I am not sure about the lua_gc() in case pcall
>> fails, it just seem logical and that's how it's done in Lua's lua.c
>> interpreter.
>>
>> PS: I wish mod_lua was under a distributed SCM like Git or Mercurial,
>> it would make things easier for me. It takes me forever to make a
>> patch and I am not even sure it can be used.
>
> Jukka maintains a git mirror of apache at http://jukka.zitting.name/git/

I have setup my own repository to make working on mod_lua easier for me:
http://mamasam.indefero.net/p/modlua/source/changes/mansion/

-- 
Bertrand Mansion
Mamasam

current status of loadable MPM changes

[Jeff Trawick]

http://people.apache.org/~trawick/loadable_mpms.txt

(you'll find additional stuff I forgot/didn't realize before long)

something I'm in a relative hurry to get feedback on is this part:

2. retain data

   How can MPMs retain data across unload of the DSO?
s->process->server_pool userdata
   won't work as-is because there's no server_rec in the pre_config hook.

   Proposal: Add ap_set_retained_data(key, value),
ap_get_retained_data(key) APIs
 (implementation detail: it will set/get userdata on pglobal)

 Although not necessary, provide ap_get_config_count()
(better name!) to
 indicate how many passes of config have completed.  (The
interface could be
 a global variable, but some third-party module would
inevitably clear it.)

status of loadable MPM work

[Jeff Trawick]

See http://people.apache.org/~trawick/loadable_mpms.txt (I'm sure you'll
find some things I've omitted before long)
Something in there I'm in a hurry to hear feedback on is

2. retain data

   How can MPMs retain data across unload of the DSO?
s->process->server_pool userdata
   won't work as-is because there's no server_rec in the pre_config hook.

   Proposal: Add ap_set_retained_data(key, value),
ap_get_retained_data(key) APIs
 (implementation detail: it will set/get userdata on pglobal)

 Although not necessary, provide ap_get_config_count()
(better name!) to
 indicate how many passes of config have completed.  (The
interface could be
 a global variable, but some third-party module would
inevitably clear it.)


-- 
Born in Roswell... married an alien...

Re: Mod_proxy_http override

[Graham Leggett]

Dingwell, Robert A. wrote:


Getting to Step Two means that it found an address and Step Two itself is
going to attempt to make the initial connection to that address.  My issue
then comes into play where the address of a server is known but the server
could refuse the connection on a particular port for some reason such as the
site is down, or whatever.   If this happens and Apache is configured to
ProxyRequests, the status is set to HTTP_NOT_FOUND, which really isnt the
case, the server refused the connection it's not that the page wasn't found.


I ran into this recently, I suspect this is definitely a bug.


Am I mistaken is thinking that r->proxyreq == PROXYREQ_PROXY  is always true
when ProxyRequests is set to On in the configuration? If so how do I force
the HTTP_SERVICE_UNAVAILABLE instead of the HTTP_NOT_FOUND? Do I have to
patch this myself and recompile it?


Potentially you could patch it and submit it for inclusion into the 
server, at which point you won't need to worry about patching it again 
in the future.


Regards,
Graham
--


smime.p7s
Description: S/MIME Cryptographic Signature

status of loadable MPM work

[Jeff Trawick]

See http://people.apache.org/~trawick/loadable_mpms.txt (I'm sure you'll
find some things I've omitted before long)
Something in there I'm in a hurry to hear feedback on is

2. retain data

   How can MPMs retain data across unload of the DSO?
s->process->server_pool userdata
   won't work as-is because there's no server_rec in the pre_config hook.

   Proposal: Add ap_set_retained_data(key, value),
ap_get_retained_data(key) APIs
 (implementation detail: it will set/get userdata on pglobal)

 Although not necessary, provide ap_get_config_count()
(better name!) to
 indicate how many passes of config have completed.  (The
interface could be
 a global variable, but some third-party module would
inevitably clear it.)


-- 
Born in Roswell... married an alien...

Re: mod_lua check for lua compilation error

[Peter Cawley]

On Tue, Mar 24, 2009 at 6:57 PM, Brian McCallister  wrote:
> I think the lua_State may hold the compilation problems, but am not
> sure, need to go poke around -- I won't have a chance for a while
> still, am dealing with a family medical issue for a couple weeks
> (nothing urgent, just needing attention).

The lua_State will maintain the error message on the top of the stack,
although the call stack will have been unwound - you need to append it
onto the error message as the error occurs (i.e. using a lua_pcall
error handler function) if you want to keep it.

Re: [PATCH] mod_dbd with more than one pool

[Ronald Park]

Sorry for coming into this discussion late.

I think this is an excellent set of patches.  I had developed a similar
enhancement to allow mod_dbd to connect with multiple databases at my last
job but never had a chance to make it available here.  Your version is far
more clever and configurable.

I did have one minor nit to pick with the implementation of the DBDInitSQL
command.  A user might presume that the order of commands listed in the
config file would be preserved when the commands are run.  By using a hash
table, you'll likely get a random order causing problems.  For example,
perhaps the user needs to first load one table then choose a subset from
that first table to file another; if you run them in the reserve order, the
second table will likely be empty. :(

If you choose not to change the implementation, then the documentation
should make this clear.

Thanks,
Ron

Principal Software Engineer for Hire
http://www.linkedin.com/in/ronaldparkjr

Re: mod_lua check for lua compilation error

[Brian McCallister]

On Sat, Mar 21, 2009 at 10:54 AM, Bertrand Mansion  wrote:
> In order to be able to detect if a lua file fails to compile and get
> an informative error message like :
>  Error!
>  /web/localhost/htdocs/info.lua:174: '=' expected near '+'
> instead of :
>  Error!
>  attempt to call a nil value
>
> I suggest you move the load/pcall function from
> lua_vmprep:apl_get_lua_state() to mod_lua:lua_handler() and
> mod_lua:lua_request_rec_hook_harness() like in the attached patch.

I disagree with this change, but agree with the intention. We need a
good way to get compilation errors back to folks asking for the
lua_State. We could use a state flag in the vm create callback which
says what happened -- instead of just calling it with a valid
lua_State when it is created, it will be called every time, with an
enumeration of what happened, one of { create, reuse, syntax_error,
other_error } or such.

I think the lua_State may hold the compilation problems, but am not
sure, need to go poke around -- I won't have a chance for a while
still, am dealing with a family medical issue for a couple weeks
(nothing urgent, just needing attention).

>
> I haven't yet tested how my patch works with hooks but it seems to be
> ok for lua_handler. I am not sure about the lua_gc() in case pcall
> fails, it just seem logical and that's how it's done in Lua's lua.c
> interpreter.
>
> PS: I wish mod_lua was under a distributed SCM like Git or Mercurial,
> it would make things easier for me. It takes me forever to make a
> patch and I am not even sure it can be used.

Jukka maintains a git mirror of apache at http://jukka.zitting.name/git/


-Brian

>
> --
> Bertrand Mansion
> Mamasam
>

Mod_proxy_http override

[Dingwell, Robert A.]

Hi,

Is there a way to override http proxy handling to use a different module
other then mod_proxy_http?  I¹ve been trying to use apache as a forwarding
proxy but I have an issue with the way it handles connection refused cases
from a server. 

I believe my issue lies within the code section below from the
proxy_http_handler  function in mod_proxy_http.c but I want to make sure I
understand this correctly.

Step One is looking up the address of the server to contact on behalf of the
client, if it cannot find an address for  the server it fails and goes onto
cleanup which will send Ostatus¹ back to the client.

Getting to Step Two means that it found an address and Step Two itself is
going to attempt to make the initial connection to that address.  My issue
then comes into play where the address of a server is known but the server
could refuse the connection on a particular port for some reason such as the
site is down, or whatever.   If this happens and Apache is configured to
ProxyRequests, the status is set to HTTP_NOT_FOUND, which really isnt the
case, the server refused the connection it's not that the page wasn't found.


Am I mistaken is thinking that r->proxyreq == PROXYREQ_PROXY  is always true
when ProxyRequests is set to On in the configuration? If so how do I force
the HTTP_SERVICE_UNAVAILABLE instead of the HTTP_NOT_FOUND? Do I have to
patch this myself and recompile it? And if I do can I simpley change the
name of the module to something like my_mod_proxy_http and in the apache
config simply comment out the original and add mine?


Thanks 

Rob


static int proxy_http_handler(request_rec *r, proxy_worker *worker,
  proxy_server_conf *conf,
  char *url, const char *proxyname,
  apr_port_t proxyport)
{

...
..
.


  /* Step One: Determine Who To Connect To */
if ((status = ap_proxy_determine_connection(p, r, conf, worker, backend,
uri, &url, proxyname,
proxyport, server_portstr,
sizeof(server_portstr))) !=
OK)
goto cleanup;

/* Step Two: Make the Connection */
if (ap_proxy_connect_backend(proxy_function, backend, worker,
r->server)) {
if (r->proxyreq == PROXYREQ_PROXY)
status = HTTP_NOT_FOUND;
else
status = HTTP_SERVICE_UNAVAILABLE;
goto cleanup;
}


  /* Step Three: Create conn_rec */
if (!backend->connection) {
if ((status = ap_proxy_connection_create(proxy_function, backend,
 c, r->server)) != OK)
goto cleanup;
}

/* Step Four: Send the Request */
if ((status = ap_proxy_http_request(p, r, backend, backend->connection,
conf, uri, url, server_portstr)) !=
OK)
goto cleanup;

/* Step Five: Receive the Response */
if ((status = ap_proxy_http_process_response(p, r, backend,
 backend->connection,
 conf, server_portstr)) !=
OK)
goto cleanup;


.
.
.

SIGQUIT using default signal handler instead of sig_coredump

[Eric Covener]

SIGQUIT (3) does not go through sig_coredump(). Is this intentional to
short-circuit the sig_coredump handling in the case something is
broken there?

I'd like to document for CoreDumpDirectory, so people from Java world
aren't left scratching their heads.

-- 
Eric Covener
cove...@gmail.com

Re: [VOTE] release 2.3.2 as alpha

[R]

Hi guys,

On Mon, Mar 23, 2009 at 4:35 PM, Gregg L. Smith  wrote:
> Hello Devs,
>
> With the talk of removing pcre I've been dreading this day.

I too am having issues with pcre when trying to move a custom module
from 2.0 to 2.2.

The ap_ apis seems to behave differently and doesnt match the way it
used to earlier.

Can anybody point me to a summary of what changed exactly other than
the ap_ prefix.

Thanks
Ryan.

Re: [VOTE] release 2.3.2 as alpha

[Gregg L. Smith]

Yes, hopefully someone will since I'm not there yet and I've tasted 2.3 
on two prior occasions and would love to again..


For the record, the pcre and deftables projects has been removed from 
the Apache workspace, but their cousin seem to still exist in the 
makefile, so command line builds pcre as we all are familiar with yet 
still errors linking the library.


Gregg

Nick Kew wrote:
Sorry, can't help with Windows.  But it's good that you've flagged the 
issue:

hopefully someone will figure it out.

For the record, pcre also caught me out on MacOS.  I had the library, but
no pcre-config, which was required for build.  However, in the Mac case,
installing fink's current pcre fixed it.

Re: moving towards loadable MPMs

[Mladen Turk]

Paul Querna wrote:

I was hoping someone will address the separation of
child process management from the mpm.
Majority of this stuff is common and duplicated across
mpms while there can be an api for that thought.



Yes, the simple mpm inteded to go there, and only uses APR functions
for threadpools/fork/processes, rather than native OS functions like
the existing winnt/worker/prefork MPMs.



Well the problem is that apr isn't enough there.
Bill proposed to add registry (not sure about SCM)
code to the apr, but was rejected. There is
also a problem of sending the sockets to the child.
So one would still need the 'os' layer hooks where
os part of the mpm can do it's work.

Although good, the simple mpm isn't much of the
use solving that.

Like said earlier. You guys should start thinking
like a Windows developer :)


Regards
--
^(TM)

[face2face] BOF Thurs 8:30pm at ApacheCon

[William A. Rowe, Jr.]

Apache httpd Server 3.0; What Color is your Pony?

A Birds of a Feather session for developers and users

20:30-21:30; Thursday 26-03; Lausanne Room; Moevenpick

--

http://wiki.apache.org/httpd/Face2Face/BoFAcEu09

Please feel free to add to the wiki page what you would like to
see us all discuss!

See you there,

Bill

Re: ProxyPassReverse and paths

[Jim Jagielski]

On Mar 24, 2009, at 11:46 AM, William A. Rowe, Jr. wrote:


Jim Jagielski wrote:

On 2nd thought the whole idea of a balancer itself
incorporating a path in conjunction with each ind. members also
possibly having one is a big can of worms. Need to mull this over...


Let me try mind-mapping this out - please sanity check a whole group
of assumptions.  (And I'm not surprised, I had the same reaction after
I thought I had the right patch, and realized this is much more  
tangled.)




BalancerMember http://hash/bang/
BalancerMember http://backup/app2/

ProxyPass /app/ balancer://foo/bar

What does Member
which is on one  balancer block of balancer://foo and not in  
the
another with another URL?  This is impossible today due to the fact  
that

the balancer set struct doesn't contain a URI member at all.

Is this construct interesting for other directives?  I suspect so.

I don't think that  URL's are valid for declaring  
BalancerMember's;

my gut instinct is to WARN in 2.2 and to fail parsing this in 2.3.

Now let's look at the ProxyPass[reverse] - what do the directives  
above

do with a request to /app/login ?  This pretty clearly maps to either
http://hash/bang/bar/login (or http://backup/app2/bar/login) ...  
right?

The reverse must strip the balancer member target, including URL, PLUS
the RHS URL expression of the ProxyPassReverse.

Sanity check this please, because we can't fix it until we understand
what it was meant to do :)



Again looking over all this, the confusion, I think comes about because
historically you were able to pass a full URL as the 2nd arg to
ProxyPass. But when the balancer was added, you are instead passing
a "name" that looks like a URL. And the more I think about it, whether
this is optimal or not, logically it makes sense. Because what,
exactly, does a path in a balancer://foo/ *name* really mean? In the
above example, is the URL passed to the members http://hash/bang/bar ?
Do 2 balancers named balancer://foo/bar and balancer://foo/boo share
the exact same connection pool and other resources?

I think that forcing balancer://.. to always ignore paths and require
users to set paths as member definitions is the easier course of
action. So adding error-log entries which say "You added path
info to balancer://foo/ this is ignored" is likely enuff.

I'm trying to think of scenarios where paths in balancers make sense
and don't make things harder to understand... if anyone can, then
let me know :)

Re: ProxyPassReverse and paths

[William A. Rowe, Jr.]

Jim Jagielski wrote:

On 2nd thought the whole idea of a balancer itself
incorporating a path in conjunction with each ind. members also
possibly having one is a big can of worms. Need to mull this over...


Let me try mind-mapping this out - please sanity check a whole group
of assumptions.  (And I'm not surprised, I had the same reaction after
I thought I had the right patch, and realized this is much more tangled.)



  BalancerMember http://hash/bang/
  BalancerMember http://backup/app2/

ProxyPass /app/ balancer://foo/bar

What does  balancer block of balancer://foo and not in the
another with another URL?  This is impossible today due to the fact that
the balancer set struct doesn't contain a URI member at all.

Is this construct interesting for other directives?  I suspect so.

I don't think that  URL's are valid for declaring BalancerMember's;
my gut instinct is to WARN in 2.2 and to fail parsing this in 2.3.

Now let's look at the ProxyPass[reverse] - what do the directives above
do with a request to /app/login ?  This pretty clearly maps to either
http://hash/bang/bar/login (or http://backup/app2/bar/login) ... right?
The reverse must strip the balancer member target, including URL, PLUS
the RHS URL expression of the ProxyPassReverse.

Sanity check this please, because we can't fix it until we understand
what it was meant to do :)

Re: ProxyPassReverse and paths

[Jim Jagielski]

On 2nd thought the whole idea of a balancer itself
incorporating a path in conjunction with each ind. members also
possibly having one is a big can of worms. Need to mull this over...

Re: ProxyPassReverse and paths

[Ruediger Pluem]

On 24.03.2009 14:57, Jim Jagielski wrote:
> 
> On Mar 24, 2009, at 9:44 AM, Ruediger Pluem wrote:
> 
>> On 24.03.2009 14:28, Jim Jagielski wrote:
>>> There have been a few times when people get caught up when
>>> using ProxyPassReverse with balancers that contain a path...
>>> After all, the normal convention is everywhere you see a
>>> ProxyPass there should be a corresponding ProxyPassReverse
>>> that follows the same format. However in cases where
>>> ProxyPass contains a path, PPR doesn't work correctly...
>>>
>>> This fixes that but is also safe for the vast majority of
>>> existing sites. I plan to commit unless there are objections:
>>>
>>> Index: modules/proxy/proxy_util.c
>>> ===
>>> --- modules/proxy/proxy_util.c(revision 757753)
>>> +++ modules/proxy/proxy_util.c(working copy)
>>> @@ -1080,11 +1080,9 @@
>>>   * or may not be the right one... basically, we need
>>>   * to find which member actually handled this request.
>>>   *
>>> - * TODO: Recover the path from real and use that
>>> - *   for more exact matching
>>>   */
>>>  if ((strncasecmp(real, "balancer:", 9) == 0) &&
>>> -(balancer = ap_proxy_get_balancer(r->pool, sconf, real))) {
>>> +(balancer = ap_proxy_get_balancerwpath(r->pool, sconf,
>>> real))) {
>>
>> Doesn't it make more sense to cut off the path of real here to get the
>> balancer
>> instead of adding a new function to the API (which also requires a
>> minor bump,
>> but thats just nitpicking)?
>>
> 
> Well, cutting off path from real is what ap_proxy_get_balancer() actually
> *does* and is what I'm trying to adjust :)

Hm. I am getting confused now. We do *not* store the path in balancer->name.
So if we do not cut it off in ap_proxy_get_balancer our strcmp will never match
correct?

Regards

Rüdiger

Re: [VOTE] release 2.3.2 as alpha

[Nick Kew]

On 23 Mar 2009, at 20:35, Gregg L. Smith wrote:


Hello Devs,

With the talk of removing pcre I've been dreading this day.


Sorry, can't help with Windows.  But it's good that you've flagged  
the issue:

hopefully someone will figure it out.

For the record, pcre also caught me out on MacOS.  I had the library,  
but

no pcre-config, which was required for build.  However, in the Mac case,
installing fink's current pcre fixed it.

--
Nick Kew

Re: ProxyPassReverse and paths

[Jim Jagielski]

On Mar 24, 2009, at 9:44 AM, Ruediger Pluem wrote:


On 24.03.2009 14:28, Jim Jagielski wrote:

There have been a few times when people get caught up when
using ProxyPassReverse with balancers that contain a path...
After all, the normal convention is everywhere you see a
ProxyPass there should be a corresponding ProxyPassReverse
that follows the same format. However in cases where
ProxyPass contains a path, PPR doesn't work correctly...

This fixes that but is also safe for the vast majority of
existing sites. I plan to commit unless there are objections:

Index: modules/proxy/proxy_util.c
===
--- modules/proxy/proxy_util.c(revision 757753)
+++ modules/proxy/proxy_util.c(working copy)
@@ -1080,11 +1080,9 @@
  * or may not be the right one... basically, we need
  * to find which member actually handled this request.
  *
- * TODO: Recover the path from real and use that
- *   for more exact matching
  */
 if ((strncasecmp(real, "balancer:", 9) == 0) &&
-(balancer = ap_proxy_get_balancer(r->pool, sconf,  
real))) {

+(balancer = ap_proxy_get_balancerwpath(r->pool, sconf,
real))) {


Doesn't it make more sense to cut off the path of real here to get  
the balancer
instead of adding a new function to the API (which also requires a  
minor bump,

but thats just nitpicking)?



Well, cutting off path from real is what ap_proxy_get_balancer()  
actually

*does* and is what I'm trying to adjust :)

Re: moving towards loadable MPMs

[Paul Querna]

On Tue, Mar 24, 2009 at 2:42 PM, Mladen Turk  wrote:
> Jeff Trawick wrote:
>>
>> On Tue, Mar 24, 2009 at 12:41 PM, Mladen Turk > > wrote:
>>
>>    Jeff Trawick wrote:
>>
>>
>>        (no plans here to touch WinNT MPMs, and I don't see any
>>        compelling reason to leave it broken; it should build as before
>>        once the inevitable minor slips are corrected)
>>
>>
>>    The major problem with that is the huge amount of platform
>>    dependent code for managing the service, registry and fork.
>>    (Although -k install and -k uninstall could be a separate
>>     utility app)
>>
>>
>> Maybe its the lack of sleep, but I see that as a separate code reuse
>> issue.  I'm not trying to address code reuse with this effort, except with
>> minor details where the solution to cutting the ties to the MPM involves a
>> shared definition.
>>
>
> Too bad then :)
> I was hoping someone will address the separation of
> child process management from the mpm.
> Majority of this stuff is common and duplicated across
> mpms while there can be an api for that thought.
>

Yes, the simple mpm inteded to go there, and only uses APR functions
for threadpools/fork/processes, rather than native OS functions like
the existing winnt/worker/prefork MPMs.

Mostly just lack of time on my part to fully flesh out the Simple MPM.../.

Re: ProxyPassReverse and paths

[Ruediger Pluem]

On 24.03.2009 14:28, Jim Jagielski wrote:
> There have been a few times when people get caught up when
> using ProxyPassReverse with balancers that contain a path...
> After all, the normal convention is everywhere you see a
> ProxyPass there should be a corresponding ProxyPassReverse
> that follows the same format. However in cases where
> ProxyPass contains a path, PPR doesn't work correctly...
> 
> This fixes that but is also safe for the vast majority of
> existing sites. I plan to commit unless there are objections:
> 
> Index: modules/proxy/proxy_util.c
> ===
> --- modules/proxy/proxy_util.c(revision 757753)
> +++ modules/proxy/proxy_util.c(working copy)
> @@ -1080,11 +1080,9 @@
>* or may not be the right one... basically, we need
>* to find which member actually handled this request.
>*
> - * TODO: Recover the path from real and use that
> - *   for more exact matching
>*/
>   if ((strncasecmp(real, "balancer:", 9) == 0) &&
> -(balancer = ap_proxy_get_balancer(r->pool, sconf, real))) {
> +(balancer = ap_proxy_get_balancerwpath(r->pool, sconf,
> real))) {

Doesn't it make more sense to cut off the path of real here to get the balancer
instead of adding a new function to the API (which also requires a minor bump,
but thats just nitpicking)?

Regards

Rüdiger

Re: moving towards loadable MPMs

[Mladen Turk]

Jeff Trawick wrote:
On Tue, Mar 24, 2009 at 12:41 PM, Mladen Turk > wrote:


Jeff Trawick wrote:


(no plans here to touch WinNT MPMs, and I don't see any
compelling reason to leave it broken; it should build as before
once the inevitable minor slips are corrected)


The major problem with that is the huge amount of platform
dependent code for managing the service, registry and fork.
(Although -k install and -k uninstall could be a separate
 utility app)


Maybe its the lack of sleep, but I see that as a separate code reuse 
issue.  I'm not trying to address code reuse with this effort, except 
with minor details where the solution to cutting the ties to the MPM 
involves a shared definition.




Too bad then :)
I was hoping someone will address the separation of
child process management from the mpm.
Majority of this stuff is common and duplicated across
mpms while there can be an api for that thought.


Regards
--
^(TM)

ProxyPassReverse and paths

[Jim Jagielski]

There have been a few times when people get caught up when
using ProxyPassReverse with balancers that contain a path...
After all, the normal convention is everywhere you see a
ProxyPass there should be a corresponding ProxyPassReverse
that follows the same format. However in cases where
ProxyPass contains a path, PPR doesn't work correctly...

This fixes that but is also safe for the vast majority of
existing sites. I plan to commit unless there are objections:

Index: modules/proxy/proxy_util.c
===
--- modules/proxy/proxy_util.c  (revision 757753)
+++ modules/proxy/proxy_util.c  (working copy)
@@ -1080,11 +1080,9 @@
   * or may not be the right one... basically, we need
   * to find which member actually handled this request.
   *
- * TODO: Recover the path from real and use that
- *   for more exact matching
   */
  if ((strncasecmp(real, "balancer:", 9) == 0) &&
-(balancer = ap_proxy_get_balancer(r->pool, sconf, real))) {
+(balancer = ap_proxy_get_balancerwpath(r->pool, sconf,  
real))) {

  int n;
  proxy_worker *worker;
  worker = (proxy_worker *)balancer->workers->elts;
@@ -1245,9 +1243,10 @@
  return ret;
}

-PROXY_DECLARE(proxy_balancer *) ap_proxy_get_balancer(apr_pool_t *p,
-   
proxy_server_conf *conf,

-  const char *url)
+static proxy_balancer *proxy_find_balancer(apr_pool_t *p,
+   proxy_server_conf *conf,
+   const char *url,
+   int usepath)
{
  proxy_balancer *balancer;
  char *c, *uri = apr_pstrdup(p, url);
@@ -1257,8 +1256,13 @@
  if (c == NULL || c[1] != '/' || c[2] != '/' || c[3] == '\0') {
 return NULL;
  }
-/* remove path from uri */
-if ((c = strchr(c + 3, '/'))) {
+/*
+ * remove path from uri if desired. For compatibility, if
+ * we want to use the path, but there is no real path, just
+ * a trailing '/', strip it and continue
+ */
+c = strchr(c + 3, '/');
+if ( (usepath && c && !*(c+1)) || (!usepath && c)) {
  *c = '\0';
  }
  balancer = (proxy_balancer *)conf->balancers->elts;
@@ -1271,6 +1275,21 @@
  return NULL;
}

+PROXY_DECLARE(proxy_balancer *) ap_proxy_get_balancer(apr_pool_t *p,
+   
proxy_server_conf *conf,

+  const char *url)
+{
+return proxy_find_balancer(p, conf, url, 0);
+}
+
+PROXY_DECLARE(proxy_balancer *) ap_proxy_get_balancerwpath(apr_pool_t  
*p,
+
proxy_server_conf *conf,
+   const char  
*url)

+{
+return proxy_find_balancer(p, conf, url, 1);
+}
+
+
PROXY_DECLARE(const char *) ap_proxy_add_balancer(proxy_balancer  
**balancer,

apr_pool_t *p,
proxy_server_conf  
*conf,

Index: modules/proxy/mod_proxy.h
===
--- modules/proxy/mod_proxy.h   (revision 757753)
+++ modules/proxy/mod_proxy.h   (working copy)
@@ -583,6 +583,17 @@
proxy_server_conf  
*conf,

const char *url);
/**
+ * Get the balancer from proxy configuration without dropping the path
+ * associated with that balancer
+ * @param p memory pool used for finding balancer
+ * @param conf  current proxy server configuration
+ * @param url   url to find the worker from. Has to have balancer://  
prefix

+ * @return  proxy_balancer or NULL if not found
+ */
+PROXY_DECLARE(proxy_balancer *) ap_proxy_get_balancerwpath(apr_pool_t  
*p,
+   
proxy_server_conf *conf,

+  const char *url);
+/**
* Add the balancer to proxy configuration
* @param balancer the new balancer
* @param p  memory pool to allocate balancer from

Re: moving towards loadable MPMs

[Jeff Trawick]

On Tue, Mar 24, 2009 at 12:41 PM, Mladen Turk  wrote:

> Jeff Trawick wrote:
>
>>
>> (no plans here to touch WinNT MPMs, and I don't see any compelling reason
>> to leave it broken; it should build as before once the inevitable minor
>> slips are corrected)
>>
>>
> The major problem with that is the huge amount of platform
> dependent code for managing the service, registry and fork.
> (Although -k install and -k uninstall could be a separate
>  utility app)


Maybe its the lack of sleep, but I see that as a separate code reuse issue.
 I'm not trying to address code reuse with this effort, except with minor
details where the solution to cutting the ties to the MPM involves a shared
definition.


> You could probably consider adding extra hooks that won't
> do anything on *nix but would allow to easily plug the
> NT mpm in this new layout.
> If you go from the WinNT requirements, the *nix would fit
> easily in this schema with just few no-ops for launching the
> child process and sending the sockets, etc...
>


Help me understand why this isn't orthogonal to cutting the hard-coded tie
to the built-inMPM ;)

Re: svn commit: r757427 - /httpd/httpd/trunk/modules/mappers/mod_rewrite.c

[Jim Jagielski]

Would it be useful to put some sort of notice of this
in r->notes? Not just here but everyplace we escape.
We get tripped up on this a lot :/

Just a thought (and maybe not a very good one)

On Mar 23, 2009, at 11:59 AM, jor...@apache.org wrote:


Author: jorton
Date: Mon Mar 23 15:59:36 2009
New Revision: 757427

URL: http://svn.apache.org/viewvc?rev=757427&view=rev
Log:
* modules/mappers/mod_rewrite.c (apply_rewrite_rule): When evaluating
a proxy rule in directory context, do escape the filename by
default, since mod_proxy will not escape in that case due to the
(deliberate) fixup hook ordering.

Thanks to: rpluem
PR: 46428

Modified:
  httpd/httpd/trunk/modules/mappers/mod_rewrite.c

Modified: httpd/httpd/trunk/modules/mappers/mod_rewrite.c
URL: 
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/mappers/mod_rewrite.c?rev=757427&r1=757426&r2=757427&view=diff
=
=
=
=
=
=
=
=
==
--- httpd/httpd/trunk/modules/mappers/mod_rewrite.c (original)
+++ httpd/httpd/trunk/modules/mappers/mod_rewrite.c Mon Mar 23  
15:59:36 2009

@@ -4083,7 +4083,20 @@
* ourself).
*/
   if (p->flags & RULEFLAG_PROXY) {
-/* PR#39746: Escaping things here gets repeated in  
mod_proxy */

+/* For rules evaluated in server context, the mod_proxy fixup
+ * hook can be relied upon to escape the URI as and when
+ * necessary, since it occurs later.  If in directory  
context,

+ * the ordering of the fixup hooks is forced such that
+ * mod_proxy comes first, so the URI must be escaped here
+ * instead.  See PR 39746, 46428, and other headaches. */
+if (ctx->perdir && (p->flags & RULEFLAG_NOESCAPE) == 0) {
+char *old_filename = r->filename;
+
+r->filename = ap_escape_uri(r->pool, r->filename);
+rewritelog((r, 2, ctx->perdir, "escaped URI in per-dir  
context "
+"for proxy, %s -> %s", old_filename, r- 
>filename));

+}
+
   fully_qualify_uri(r);

   rewritelog((r, 2, ctx->perdir, "forcing proxy-throughput with  
%s",

Re: moving towards loadable MPMs

[Mladen Turk]

Jeff Trawick wrote:


(no plans here to touch WinNT MPMs, and I don't see any compelling 
reason to leave it broken; it should build as before once the inevitable 
minor slips are corrected)




The major problem with that is the huge amount of platform
dependent code for managing the service, registry and fork.
(Although -k install and -k uninstall could be a separate
 utility app)

You could probably consider adding extra hooks that won't
do anything on *nix but would allow to easily plug the
NT mpm in this new layout.
If you go from the WinNT requirements, the *nix would fit
easily in this schema with just few no-ops for launching the
child process and sending the sockets, etc...


Regards
--
^(TM)

moving towards loadable MPMs

[Jeff Trawick]

MPMs are tightly bound into the server through several mechanisms:
they define key functions ap_mpm_query and ap_mpm_run
they define global variables like ap_server_conf
build-time symbols in MPMDIR/mpm.h that enable generation of support code or
provide macros for core to use

I'm working on removing these ties.

* key functions are replaced with hooks
* global variables are moved to http_main if necessary or replaced with
ap_mpm_query() calls if possible
* core will provide as much MPM support code as practical (resulting in a
small amount of extra code for users that don't switch MPMs)

stage 1:

reduce the ties but don't change the build process (i.e., leave MPMs as
compiled-in module)

at this stage, WinNT MPM will still provide mpm.h but will indicate which
support code NOT to supply
* only winnt MPM will retain mpm.h, and it will have a bunch of
AP_MPM_NO_foo instead of a small number of AP_MPM_WANT_foo; punt on
mpm_common having a better picture of which functions to generate)

(no plans here to touch WinNT MPMs, and I don't see any compelling reason to
leave it broken; it should build as before once the inevitable minor slips
are corrected)

stage 2:

allow building as shared object

one issue is whether to support the expected --with-mpm=FOO as the way to
select an MPM in the traditional, built-in style

we could have

--with-mpm=FOO still provides built-in MPM;   no --with-mpm=FOO but
--enable-worker=shared --enable-prefork=shared builds the DSOs for switching

or just use normal module enablement but check for multiple
statically-enabled MPMs

(FWIW, MPM hooks run first and declared APR_HOOK_MIDDLE so only one of
multiple loaded MPMs would be used)

Any concerns with the general idea, or comments with either stage?

Any concerns with committing to trunk once stage 1 has been completed for a
few MPMs (and I think there's a reasonable chance of Windows MPM still
working)?

Fwd: The revival of rsync+http

[Martin Langhoff]

I guess people around here are aware of Tridge's old rproxy, which was
mentioned here before:

 http://marc.info/?l=apache-httpd-dev&m=96942551231967&w=2
 http://marc.info/?l=apache-httpd-dev&m=93121563713572&w=2

This post is just to mention that there is a revival of the concep,
being hacked into an apache module. There is a lousy wikipage here
http://wiki.laptop.org/go/Apache_Proxy_CRCsync and some discussions
starting to take shape in
http://lists.laptop.org/listinfo/http-crcsync

Also - the mozilla project  has a GSoC project for "web pages over
rsync", which hopefully will use the same protocol as well --
https://wiki.mozilla.org/Community:SummerOfCode09

cheers,



m
--
 martin.langh...@gmail.com
 mar...@laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff