Re: mpm perchild and mod_php4
On Sun, Dec 08, 2002, Jochen Kächelin <[EMAIL PROTECTED]> wrote: > Can somebody give me hint how to get Apache 2.0.43, perchild > and mod_php4 running to do some testing? perchild is broken right now. Unless you're willing to do some coding, don't expect it to work :/ However, it appears that you're having some other problems as well since the bugs in perchild I know of wouldn't cause a segfault. > error_log: > > [Sun Dec 08 16:03:03 2002] [notice] child pid 21526 exit signal Segmentation fault >(11) > [Sun Dec 08 16:03:03 2002] [notice] child pid 21524 exit signal Segmentation fault >(11) > [Sun Dec 08 16:03:03 2002] [emerg] (13)Permission denied: apr_proc_mutex_lock >failed. Attempting to shutdown process > gracefully. > [Sun Dec 08 16:03:03 2002] [emerg] (13)Permission denied: apr_proc_mutex_unlock >failed. Attempting to shutdown process > gracefully. > [Sun Dec 08 16:03:03 2002] [emerg] (13)Permission denied: apr_proc_mutex_lock >failed. Attempting to shutdown process > gracefully. > [Sun Dec 08 16:03:03 2002] [emerg] (13)Permission denied: apr_proc_mutex_unlock >failed. Attempting to shutdown process > gracefully. JE
Re: trouble w/ perchild MPM
On Wed, Nov 27, 2002, James Ponder <[EMAIL PROTECTED]> wrote: > On Tue, Nov 26, 2002 at 07:44:34PM -0500, Johannes Erdfelt wrote: > > This can include shared (sometimes requiring connections to be passed) > > and non shared (always answered by the child) sockets. > > > > I don't particularly see the non shared case as a concern. The shared > > case can be a problem. > > > > If either are a problem, I suspect that perchild is not the MPM you want > > to use. > > Perhaps perchild can be improved further to have a set of listener > processes/threads, under the generic apache user id or another secured id, > which accepts connections but does not process them. These threads then > pass the connections on to the correct user-owned child id via the named > sockets, allowing these child users to have no listening sockets at all > (except the unix domain socket). It would need to be a process. Mixing uid's among threads is not portable. Switching to a model like is possible, but I'm willing to be that passing fd's is not cheap and would just create overhead. Do you want to do this for security reasons? JE
Re: trouble w/ perchild MPM
On Wed, Nov 27, 2002, James Ponder <[EMAIL PROTECTED]> wrote: > On Tue, Nov 26, 2002 at 07:06:33AM -0500, Jeff Trawick wrote: > > Any debugging you can provide would be much appreciated. perchild > > is one of the potentially cool features of 2.0, but at the moment it > > I'm trying to understand perchild, so I've had a quick look at the source. > Could you confirm my understanding? > > It appears that perchild starts by spawning processes, each process uses the > inherited table to find out if it should change owner via setuid() and does so > accordingly. The children then spawn worker threads, all of those threads > listen simultaneously, serialising on a cross-process mutex. The first process > that got this mutex accepts the connection, processes it until the post_read > hook, passes it to another process if necessary via a named socket, and then > longjmps out of the request so it can abort the connection processing itself. That's how it currently is implemented. > My questions are - why does worker use a listener thread, and perchild doesn't? I suspect because it was originally thought it may not have been needed. The existing implementation still has a design flaw where moving to a listener thread model can help solve. I'm currently in the process of implementing this. > And secondly, surely allowing all these differently-owned processes access to > the listening sockets means that any user with gdb can attach to their Apache > process and intercept requests? or am I missing the purpose of perchild? That's a limitation of the existing implementation. Each child listens on all sockets, even if it can't possibly answer a request for it. This should be relatively easy to fix. To answer your question about gdb, that is a configuration issue. Once the code has been changed to not listen to all sockets, the security implications are limited to sockets that the child is responsible for. This can include shared (sometimes requiring connections to be passed) and non shared (always answered by the child) sockets. I don't particularly see the non shared case as a concern. The shared case can be a problem. If either are a problem, I suspect that perchild is not the MPM you want to use. JE
Re: trouble w/ perchild MPM
On Tue, Nov 26, 2002, Enrico Weigelt <[EMAIL PROTECTED]> wrote: > On Tue, Nov 26, 2002 at 07:06:33AM -0500, Jeff Trawick wrote: > > hi folks, > > > > Please grab perchild.c from current cvs and see how much is still > > broken. Here is one easy place to grab it. Make sure you get the > > latest version. > > > > >http://cvs.apache.org/viewcvs.cgi/httpd-2.0/server/mpm/experimental/perchild/perchild.c > I've tried it yesterday, but there was no new stuff since 2.0.43, > which i'm currently using. > So if you've got something more recent, you're welcomed to post > it to me :) 2.0.43 seems to be tagged at revision 1.132. The latest version is 1.136 which contains numerous important fixes. > (i currently like mailing diffs much than using CVS, since ip access > is quite expensive @ my location, but mail forwarding w/ uucp costs > almost nothing ... or is there an mail gw for CVS ?) Should be as easy as this retrieving this URL: http://cvs.apache.org/viewcvs.cgi/*checkout*/httpd-2.0/server/mpm/experimental/perchild/perchild.c?rev=HEAD&content-type=text/plain That is the latest version of just that file. JE
Re: SV: MPM perchild.
That looks about good. Do you only have one child defined? JE On Tue, Nov 26, 2002, Jonas Eriksson <[EMAIL PROTECTED]> wrote: > > > Hmm sorry! I did a bougus error in http.conf ... > > When i now start the server i get the following in my shell promt and > the server starts. > What does the debug info tell? Is it all ok or do i have to change > anyting? > > [root@mose /usr/local/apache2.0.43/bin]# ./httpd -f > /usr/local/apache2.0.43/conf/httpd.conf > [Tue Nov 26 18:35:41 2002] [debug] perchild.c(2007): filling out > child_info_table; UID: 1096, GID: 1094, SD: 4 4, OUTPUT: 5 5, Child Num: > 0 > [Tue Nov 26 18:35:41 2002] [debug] perchild.c(2007): filling out > child_info_table; UID: 1096, GID: 1094, SD: 4 4, OUTPUT: 5 5, Child Num: > 1 > [Tue Nov 26 18:35:41 2002] [debug] perchild.c(2007): filling out > child_info_table; UID: 1096, GID: 1094, SD: 4 4, OUTPUT: 5 5, Child Num: > 2 > > > > > On Tue, Nov 26, 2002, Jonas Eriksson <[EMAIL PROTECTED]> wrote: > > I Still get error "Unable to find process with matching uid/gid." > > after compileing 2.43 with > > > http://cvs.apache.org/viewcvs.cgi/httpd-2.0/server/mpm/experimental/perc > > hild/perchild.c > > Are you sure you configured your setup correctly? > > Do you have a ChildPerUserId before you use AssignUserId? > > Can you give us the configuration you're using? > > JE > >
Re: MPM perchild.
On Tue, Nov 26, 2002, Jonas Eriksson <[EMAIL PROTECTED]> wrote: > I Still get error "Unable to find process with matching uid/gid." > after compileing 2.43 with > http://cvs.apache.org/viewcvs.cgi/httpd-2.0/server/mpm/experimental/perc > hild/perchild.c Are you sure you configured your setup correctly? Do you have a ChildPerUserId before you use AssignUserId? Can you give us the configuration you're using? JE
Re: MPM perchild againg
On Tue, Nov 26, 2002, Enrico Weigelt <[EMAIL PROTECTED]> wrote: > it seems that under some circumstances the messages for connection > passing between childs are not received at the destination process. > sometimes it also happened, that apr_poll() returned w/o error, > but scan through the listener list does not find the touched socket. > perhaps there's a leak in the listener list. where could it be modified ? > should the pollset better be recreated before each poll ? What version of the perchild MPM are you using? There were some significant bugs earlier that I've fixed, but there's still one significant bug that prevents perchild from being useful. JE
Re: new download page
On Sun, Oct 27, 2002, Thom May <[EMAIL PROTECTED]> wrote: > * Joshua Slive ([EMAIL PROTECTED]) wrote : > > Pier Fumagalli wrote: > > > > >On 27/10/02 0:54, "David Burry" wrote: > > > > > > > > > > > > Right. If we had very reliable mirrors and a good technique for keeping > > them that way, I'd be fine with doing an automatic redirect or fancy DNS > > tricks. But we don't have that at the moment. > > > > >I looked into it back in the days, but the only way would be to go down to > > >RIPE (IANA in the US) to see where that IP is coming from, doing some > > >weirdo > > >WHOIS parsing and stuff... _WAY_ overkilling... Anyhow this is going waaay > > >offtopic! :-) > > > > See: http://maxmind.com/geoip/ > > Or just ask BGP... http://www.supersparrow.org/ Network routes don't necessarily mean that server is "best". Bandwidth varies greatly between different routes as well as server load. Plus, supersparrow is mostly a proof of concept. Dents (the underlying DNS server that I've mostly wrote) is a long way off from being production ready, and the method supersparrow uses doesn't scale well (telnet to a Cisco router). Anyway, it's next to impossible to make a perfect decision about the "best" server to use. IMHO, if you make the decision for the user (by only returning certain servers via DNS, etc) then it should be close to a perfect choice. Otherwise, you may just want to let the user choose themselves by just listing the mirrors and their location and let the user choose. JE
Re: [patch] perchild MPM bug fixes (+ open problem)
On Thu, Oct 24, 2002, Jeff Trawick <[EMAIL PROTECTED]> wrote: > Johannes Erdfelt <[EMAIL PROTECTED]> writes: > > Problem 2: > > pass_request fills out an iovec with the headers and the body of the > > request it wants to pass to another process. It unfortunately uses the > > wrong variable for the length causing it to always be 0. > > > > Solution: > > Use the correct variable name "l". len is never set to anything so I removed > > that and used 0 in the one reference to it. Implemented in the patch below. > > applied, but I made another fix (?) too... see my note below, and let > me know if it is bad :) > > > Index: perchild.c > > @@ -1635,7 +1645,6 @@ > > apr_bucket_brigade *bb = apr_brigade_create(r->pool, c->bucket_alloc); > > apr_bucket_brigade *sockbb; > > char request_body[HUGE_STRING_LEN] = "\0"; > > -apr_off_t len = 0; > > looks to me like len should be initialized to sizeof(request_body) > since on input to apr_brigade_flatten it should have the maximum > length of the char array I'll have to defer to you here. I'm not that familiar with the brigade implementation to know what the appropriate fix for this is. However, it most certainly is better than the current behaviour so I'm not against it. If it's wrong, I'll find out pretty soon :) Thanks for applying these patches! JE
Re: [patch] perchild MPM bug fixes (+ open problem)
On Mon, Oct 21, 2002, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > > > What I was thinking was to add an artificial limitation that you can't > > > > share an IP:port pair across two different uid/gid's since that's the > > > > only case you want to pass a connection. > > > > > > That limitation should already exist, although it may not. > > > > That limitation doesn't exist either in the documentation or in the > > implementation. > > > > I figured that was part of the point of connection passing. I know it's > > also useful for passing between the siblings for a particular uid/gid > > also, but why not just require 1 child per uid/gid and then not worry > > about connection passing at all? > > A couple of reasons. Remember first of all that the more threads per > process, the less stable your server. This is because if a module > seg-faults, the whole process dies. For that reason, many admins will > want to have multiple child processes with the same uid/gid to create some > failover. > > The second reason, however, is that the file descriptor passing is meant > to solve the case of multiple Name-based Vhosts on the same ip:port all > with different uid/gid requirements. In that case, you will have multiple > child processes all listening to the same ip:port, but you will need to do > the file descriptor passing. Well, that's what I said before, but you implied that the limitations should prevent this. :) > Closing the file descriptors of ip:port combinations that can't be served > is a great optimization, but that is all it is, an optimization for the > non-Name-based vhost case. Yes. This brings it back to what I was saying before, you can't pass SSL connections, so as a result, you can't share an SSL port across two uid/gid's. So this should be documented somewhere, or enforced in the server (which may be difficult because of the layering), so people don't do that :) Anyway, I think I understand the path to follow now. I'm going to implement the necessary fixes to make perchild finally usable. JE
Re: [patch] perchild MPM bug fixes (+ open problem)
On Mon, Oct 21, 2002, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > On Thu, 17 Oct 2002, Johannes Erdfelt wrote: > > > Ryan, I've CC'd you on this just to let you see the patch. If you don't > > want me to involve you in this, please accept my apologies and let me > > know and I won't CC you in any further patches. > > I have no problem being CC'ed on patches, although for the most part, I am > unlikely to respond. In this case however, I will. I want to say that > this is great. I have reviewed the patch and the logic, and it all looks > fantastic. I'm really glad that the MPM is making progress. I will also > provide a possible solution to the one problem that you were unable to > solve. :-) Thanks for taking the time to look at the patch. I'll make sure to CC you on any further patches but focus on the rest of the development team to merge it :) > > Problem 4: > > Occasionally, when a request needs to be passed to another process, it will > > hang until another connection is accepted. perchild uses the same process > > accept mutex that the other MPM's seem to need. This results in > > serialization of all accept() calls as is the intent of the mutex. The > > problem is that another process might have acquired the mutex from the > > process that actually needs the mutex to hit accept() and see the passed > > request. This isn't normally a problem because all processes accept > > connections on all listening ports. The problem in this case is the fact > > not all processes accept connections on the per process unix domain socket > > to pass connections around, for obvious reasons. > > > > Solution: > > I don't have one :) I'm not sure what the best way of solving this is. I > > don't fully understand the semantics of the process accept mutex yet. Any > > suggestions? > > There is only one solution that I can see for this, unfortunately, it is a > rather big change. :-( The first step, is to migrate the perchild MPM to > the same model as the worker MPM, where one thread accepts requests and > puts them in a queue, and the rest of the threads remove the requests from > the queue, and actually serve them. I've been meaning to do this for a > long time, but I never got around to it. Ahh, that's a good idea. I can implement this as well. > Once that is done, the solution to this problem is easy. Every child > process needs to have two acceptor threads. The first accepts on the TCP > socket, and the second accepts on the Unix Domain Socket. For the TCP > socket, the thread should just use the same accept_mutex that it is using > now. However for the Unix Domain Socket, each thread that is accepting on > the socket should just use apr_file_lock to lock the socket before > accepting. This should remove the starvation that you are seeing, because > each socket will have it's own lock. If there is only one thread listening on the unix domain socket (because each of those is specific to a child) is a lock even needed? The current lock around the accept() logic is mostly to prevent the thundering hurd syndrome, right? Other than that, this makes perfect sense too, I can implement this as well. Thanks for the reply! JE
Re: [patch] perchild MPM bug fixes (+ open problem)
On Mon, Oct 21, 2002, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > > > As long as you are doing all this work, there is one more thought that I > > > have been meaning to implement, but that I never got around to. Currently > > > perchild doesn't work with SSL, because of when the request is passed off, > > > and how SSL works. The easy solution to this, is to have the child > > > processes close the sockets for any requests that they cannot > > > handle. This will also improve the chance that a request won't be passed > > > if you have vhosts with different ports. Consider the following: > > > > > > > > > AssignChildPerUidGid rbb rbb > > > > > > > > > > > > AssignChildPerUidGid foo foo > > > > > > > > > There is no reason for the foo/foo child process to be listening on port > > > 80. > > > > > > Just a thought for how to get SSL to work. > > > > I actually have a patch for this already :) Although I implemented it > > only as an optimization and not because of the issue with SSL. I hadn't > > tried to do SSL yet. > > > > I'd imagine this SSL limitation will have to be clearly documented since > > it may not be obvious to everyone. > > Actually, as long as you have a patch to do this, then SSL should just > work, and no docs should be necessary. :-) Perhaps I misunderstood. The patch I had developed (which is broken because of the problems with the accept lock) just didn't listen on the socket if it has no chance of answering the query itself. That's what I thought you meant, but reading what you said again I may have misunderstood. Did you mean closing the client socket or the listening socket? Wouldn't closing the client socket just cause the client to treat it as an error? What I was thinking was to add an artificial limitation that you can't share an IP:port pair across two different uid/gid's since that's the only case you want to pass a connection. JE
Re: [patch] perchild MPM bug fixes (+ open problem)
On Mon, Oct 21, 2002, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > On Mon, 21 Oct 2002, Johannes Erdfelt wrote: > > Perhaps I misunderstood. The patch I had developed (which is broken > > because of the problems with the accept lock) just didn't listen on the > > socket if it has no chance of answering the query itself. > > > > That's what I thought you meant, but reading what you said again I may > > have misunderstood. > > Nope, that's exactly what I meant. (see below) > > > Did you mean closing the client socket or the listening socket? Wouldn't > > closing the client socket just cause the client to treat it as an error? > > Just close the listening socket in the child_init phase of the > module. You want to do this before you start accepting requests. You > could also do this by just removing the socket from the accept array. The > reason I like having the child close the socket, is that it eliminates > a potential bad config. > > Consider the case where an admin configures the server to listen on > www.foo.com:8080, but he never assigns a child process to listen to that > port. If you just don't accept the connections, the user will hang > forever. If every child process, however, actively closes the sockets > that it can't serve, then the client won't even be able to connect. That's a good point too. > > What I was thinking was to add an artificial limitation that you can't > > share an IP:port pair across two different uid/gid's since that's the > > only case you want to pass a connection. > > That limitation should already exist, although it may not. That limitation doesn't exist either in the documentation or in the implementation. I figured that was part of the point of connection passing. I know it's also useful for passing between the siblings for a particular uid/gid also, but why not just require 1 child per uid/gid and then not worry about connection passing at all? JE
[patch] perchild MPM bug fixes (+ open problem)
Ryan, I've CC'd you on this just to let you see the patch. If you don't
want me to involve you in this, please accept my apologies and let me
know and I won't CC you in any further patches.
I spent some time debugging the perchild MPM since it wasn't working for
me, nor anyone else it seems. I've found a few problems:
Problem 1:
In worker_thread, there is a variable called csd that is used to get
the new socket from lr->accept_func(). If that variable is NULL, then
the memory for the new socket is allocated in the per-transaction pool.
Unfortunately, the code neglected to reset the variable to NULL after
servicing a request. The result is that the first request for each
thread worked fine, but subsequent request may have the memory
overwritten and resulting in an invalid FD.
Solution:
Set it to NULL before calling lr->accept_func(). Implemented in the patch
below.
Problem 2:
pass_request fills out an iovec with the headers and the body of the
request it wants to pass to another process. It unfortunately uses the
wrong variable for the length causing it to always be 0.
Solution:
Use the correct variable name "l". len is never set to anything so I removed
that and used 0 in the one reference to it. Implemented in the patch below.
Problem 3:
receive_from_other_child assumes that the iovec is the same on read as
it is on write. This isn't true and readmsg() follows readv() semantics.
iovec is a scatter/gather list and as a result, the 2 send buffers are
merged into one received buffer with the second always being untouched.
It also trusted the lengths in iov.iov_len which will be the size of the
original buffer, not the size of the data actually received.
Solution:
Merge the 2 buffer's into 1 and find the null terminators for the 2 strings.
Implemented in the patch below.
Problem 4:
Occasionally, when a request needs to be passed to another process, it will
hang until another connection is accepted. perchild uses the same process
accept mutex that the other MPM's seem to need. This results in
serialization of all accept() calls as is the intent of the mutex. The
problem is that another process might have acquired the mutex from the
process that actually needs the mutex to hit accept() and see the passed
request. This isn't normally a problem because all processes accept
connections on all listening ports. The problem in this case is the fact
not all processes accept connections on the per process unix domain socket
to pass connections around, for obvious reasons.
Solution:
I don't have one :) I'm not sure what the best way of solving this is. I
don't fully understand the semantics of the process accept mutex yet. Any
suggestions?
Anyway, this patch has been tested lightly and makes things work MUCH better
for the perchild MPM now, atleast for me :)
If this patch is acceptable by the Apache development team, can it be
applied?
JE
Index: perchild.c
===
RCS file: /home/cvspublic/httpd-2.0/server/mpm/experimental/perchild/perchild.c,v
retrieving revision 1.135
diff -u -r1.135 perchild.c
--- perchild.c 11 Oct 2002 15:41:52 - 1.135
+++ perchild.c 18 Oct 2002 01:31:22 -
@@ -679,9 +679,9 @@
{
struct msghdr msg;
struct cmsghdr *cmsg;
-char headers[HUGE_STRING_LEN];
-char request_body[HUGE_STRING_LEN];
-struct iovec iov[2];
+char buffer[HUGE_STRING_LEN * 2], *headers, *body;
+int headerslen, bodylen;
+struct iovec iov;
int ret, dp;
apr_os_sock_t sd;
apr_bucket_alloc_t *alloc = apr_bucket_alloc_create(ptrans);
@@ -690,15 +690,13 @@
apr_os_sock_get(&sd, lr->sd);
-iov[0].iov_base = headers;
-iov[0].iov_len = HUGE_STRING_LEN;
-iov[1].iov_base = request_body;
-iov[1].iov_len = HUGE_STRING_LEN;
+iov.iov_base = buffer;
+iov.iov_len = sizeof(buffer);
msg.msg_name = NULL;
msg.msg_namelen = 0;
-msg.msg_iov = iov;
-msg.msg_iovlen = 2;
+msg.msg_iov = &iov;
+msg.msg_iovlen = 1;
cmsg = apr_palloc(ptrans, sizeof(*cmsg) + sizeof(sd));
cmsg->cmsg_len = sizeof(*cmsg) + sizeof(sd);
@@ -715,14 +713,25 @@
APR_BRIGADE_INSERT_HEAD(bb, bucket);
bucket = apr_bucket_socket_create(*csd, alloc);
APR_BRIGADE_INSERT_HEAD(bb, bucket);
-bucket = apr_bucket_heap_create(iov[1].iov_base,
-iov[1].iov_len, NULL, alloc);
+
+body = strchr(iov.iov_base, 0);
+if (!body) {
+return 1;
+}
+
+body++;
+bodylen = strlen(body);
+
+headers = iov.iov_base;
+headerslen = body - headers;
+
+bucket = apr_bucket_heap_create(body, bodylen, NULL, alloc);
APR_BRIGADE_INSERT_HEAD(bb, bucket);
-bucket = apr_bucket_heap_create(iov[0].iov_base,
-iov[0].iov_len, NULL, alloc);
+bucket = apr_bucket_heap_create(headers, headerslen, NULL, alloc);
APR_BRIGADE_INSERT_HEAD(bb, bucket);
apr_pool_userdata_set(bb, "PERCHILD_
Re: perchild mpm
On Thu, Oct 17, 2002, Jonas Eriksson <[EMAIL PROTECTED]> wrote: > Any progress in getting perchild to work ? I've started some debugging. In my case, passing the request off to a different process is causing corruption and the request to fail. That's as far as I have gotten so far :) What problems are you having? JE
Re: Final patch for a long time.
On Tue, Oct 15, 2002, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > The first project is the Perchild MPM. It basically works, but there are > bugs. Can you give some more information? I'm interested in using the Perchild MPM myself, but haven't because of the reports that it's buggy. I'm willing to help debug it and make it usable, but the 3 bug reports I can find in bugzilla aren't very verbose. Any hints or direction where to look? Anything not in bugzilla that you know about? JE
