Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
:) all is not lost. if you assume that you want some form of notification, but you want to leave it unspecified because you're not sure what each apr thread will be used for, then you can make a somewhat generic kill off other threads cleanup. so for example, when an httpd thread is created it would register http_thread_cleanup which would use whatever magic global int die_now_please = 1, and release some condition variable, or throw something into a queue / and so on. that would be registered in the parent thread's pool -- and would only be invoked by the parent thread. pools let you do this, you don't need the mutexes for it, you just have to be explicit about parallelism. (combine that with a root pool per thread and then we can remove alloc_mutex and free lists and push the real gnarly problems into the libc malloc where it's probably best solved.) the main problem i see is that there's no easy way to break a thread out of accept(). but folks may want to look at something such as SIGIO, or having a single acceptor thread per process, or ... using kevent (freebsd), rt signals (linux), /dev/poll (solaris), completion ports (nt) which i believe all have other methods of stuffing events into the queues. for legacy systems (which probably don't have native threads to begin with) you can just use SIGTERM like we did in 1.3 and block it everywhere except during accept(). if i were to write a webserver today i'd probably start with a model such as kevent or rt signals and make the rest of the old legacy world emulate that, because those are the way of the future (and the past actually, vms had this model :). i don't care about performance on legacy operating systems. -dean On Tue, 17 Jul 2001, Aaron Bannert wrote: Uh...you knew that already, didn't you... duh... jeez now i'm the smartass ;) -aaron On Tue, Jul 17, 2001 at 08:43:18AM -0700, Aaron Bannert wrote: Normally this would be done (in POSIX) with pthread_cancel(), passing it the pthread_t from the other thread. Unfortunately, this is not a part of APR because many of the current OS implementations of this mechanism will leak resources (aparently in the kernel), and that is bad. -aaron On Tue, Jul 17, 2001 at 01:32:52AM -0700, dean gaudet wrote: On Sat, 14 Jul 2001, Sander Striker wrote: The way I see it, each process has a single pool instance as the parent for all the threads. Resetting or destroying that pool should effectively kill all threads. What am I missing? how does a thread kill another thread? -dean
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Tue, Jul 17, 2001 at 06:28:37PM -0500, William A. Rowe, Jr. wrote: Yes, yes, yes. Can we please split the concept of a heirarchial parent (the 'creator' thread's or process pool, in this case) from the allocation parent (the actual give me memory for my pool from ... here!) Then we have an OS Knows Best malloc/free mpm for threading, just as you suggest. This solves your thread-specific requirements and our scoping issues, along with fixing the 'walk the chain of pools for a block' problem, both at once. Okay, I will try and commit something into the SMS code that does this. If I understand you, there will be one parent which will handle allocation and another that will handle cleanups. This should handle your concerns about a thread that didn't exit properly and my concern about having a unnecessary locking in the allocation code. Deal? I'll make it default to having both parents be identical, and the apr_thread_create call can set it to be null (or a std. SMS). The only problem is the SMS code is chock-full of subtle race conditions right now. We *really* need more eyes on this code. -- justin
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
[snip] that would be registered in the parent thread's pool -- and would only be invoked by the parent thread. pools let you do this, you don't need the mutexes for it, you just have to be explicit about parallelism. (combine that with a root pool per thread and then we can remove alloc_mutex and free lists and push the real gnarly problems into the libc malloc where it's probably best solved.) Yes, yes, yes. Can we please split the concept of a heirarchial parent (the 'creator' thread's or process pool, in this case) from the allocation parent (the actual give me memory for my pool from ... here!) Then we have an OS Knows Best malloc/free mpm for threading, just as you suggest. This solves your thread-specific requirements and our scoping issues, along with fixing the 'walk the chain of pools for a block' problem, both at once. It's probably just me, but I'm having trouble parsing this (I think I'm getting a cold :( ). Are you saying you want the thread function to have access to both a scope pool as well as an allocator pool, in case they are different? -aaron
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Tue, 17 Jul 2001, Aaron Bannert wrote:
On Tue, Jul 17, 2001 at 01:29:47AM -0700, dean gaudet wrote:
On Sun, 15 Jul 2001, Sander Striker wrote:
Why are we so desperate in opting out the child-pool creation?
I don't really have problems with a child pool for each thread. Actually,
it will make the dynamic locking a lot easier to implement if it stays.
all threads MUST have their own private pool root.
otherwise you're just throwing scalability away in locks. (which is
proved by the claim i saw that ian got better performance by defining
ALLOC_USE_MALLOC on an 8-way.)
I totally agree, but only as a solution in httpd.
no, everywhere.
I also believe that we should provide this [application-specific requirement]
outside of the basic thread support in APR.
Please allow me to use pseudocode:
void * worker_function(void * opaque_application_data) {
apr_pool_t *thread_pool;
create_child_pool(thread_pool, global_root_pool);
now you've got mutexes in global_root_pool. see my performance comment
above.
-dean
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Tue, Jul 17, 2001 at 04:49:04PM -0700, dean gaudet wrote:
Please allow me to use pseudocode:
void * worker_function(void * opaque_application_data) {
apr_pool_t *thread_pool;
create_child_pool(thread_pool, global_root_pool);
now you've got mutexes in global_root_pool. see my performance comment
above.
IMHO, the global_root_pool should be the standard SMS - which is
just a SMS that calls malloc/free - no locking there. It actually
has no free-list/allocation-list. This strategy may or may not
work - depending if you'd ever want to cleanup the global_root_pool
(in Aaron's example). Which I don't think we'd want to do anyway.
-- justin
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
From: Aaron Bannert [EMAIL PROTECTED] Sent: Tuesday, July 17, 2001 6:41 PM [snip] that would be registered in the parent thread's pool -- and would only be invoked by the parent thread. pools let you do this, you don't need the mutexes for it, you just have to be explicit about parallelism. (combine that with a root pool per thread and then we can remove alloc_mutex and free lists and push the real gnarly problems into the libc malloc where it's probably best solved.) Yes, yes, yes. Can we please split the concept of a heirarchial parent (the 'creator' thread's or process pool, in this case) from the allocation parent (the actual give me memory for my pool from ... here!) Then we have an OS Knows Best malloc/free mpm for threading, just as you suggest. This solves your thread-specific requirements and our scoping issues, along with fixing the 'walk the chain of pools for a block' problem, both at once. It's probably just me, but I'm having trouble parsing this (I think I'm getting a cold :( ). Are you saying you want the thread function to have access to both a scope pool as well as an allocator pool, in case they are different? I've officially graduated to the rbb (insert a decent name here) club :-) Thank you, yes, scope pool defines teardowns such as cleanups, while the allocator pool addresses our performance concerns :) An obvious test is that 'allocator' is unique (e.g. thread private), a parent (at any depth) of the 'scope' pool, or the 'scope' pool itself.
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
I totally agree, but only as a solution in httpd.
no, everywhere.
I also believe that we should provide this [application-specific
requirement]
outside of the basic thread support in APR.
Please allow me to use pseudocode:
void * worker_function(void * opaque_application_data) {
apr_pool_t *thread_pool;
create_child_pool(thread_pool, global_root_pool);
now you've got mutexes in global_root_pool. see my performance comment
above.
I'm not advocating this scenario as a real solution, but merely trying
to illustrate how the child-pool creation routines don't really have to
be in apr_thread_create(). The application using APR threads can do this
before calling apr_thread_create() or it can do it in it's worker_function.
Either way, it is better to leave it totally up to the application, no?
-aaron
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Tue, Jul 17, 2001 at 10:12:35AM -0700, Aaron Bannert wrote: - Optionally creating a child-pool for each thread. This provides us two things in my mind: 1) Allowing the application to choose an SMS for this thread-pool (which could very well be a special SMS created just for this purpose). for example, you may wish to create a special 'security' thread. it uses an apr_sms_mlock_t instance. this special thread handles all of your security, keeping private keys and plain-text user passwords in-memory. luke
RE: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Sun, 15 Jul 2001, Sander Striker wrote: Why are we so desperate in opting out the child-pool creation? I don't really have problems with a child pool for each thread. Actually, it will make the dynamic locking a lot easier to implement if it stays. all threads MUST have their own private pool root. otherwise you're just throwing scalability away in locks. (which is proved by the claim i saw that ian got better performance by defining ALLOC_USE_MALLOC on an 8-way.) -dean
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
Normally this would be done (in POSIX) with pthread_cancel(), passing it the pthread_t from the other thread. Unfortunately, this is not a part of APR because many of the current OS implementations of this mechanism will leak resources (aparently in the kernel), and that is bad. -aaron On Tue, Jul 17, 2001 at 01:32:52AM -0700, dean gaudet wrote: On Sat, 14 Jul 2001, Sander Striker wrote: The way I see it, each process has a single pool instance as the parent for all the threads. Resetting or destroying that pool should effectively kill all threads. What am I missing? how does a thread kill another thread? -dean
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
Uh...you knew that already, didn't you... duh... jeez now i'm the smartass ;) -aaron On Tue, Jul 17, 2001 at 08:43:18AM -0700, Aaron Bannert wrote: Normally this would be done (in POSIX) with pthread_cancel(), passing it the pthread_t from the other thread. Unfortunately, this is not a part of APR because many of the current OS implementations of this mechanism will leak resources (aparently in the kernel), and that is bad. -aaron On Tue, Jul 17, 2001 at 01:32:52AM -0700, dean gaudet wrote: On Sat, 14 Jul 2001, Sander Striker wrote: The way I see it, each process has a single pool instance as the parent for all the threads. Resetting or destroying that pool should effectively kill all threads. What am I missing? how does a thread kill another thread? -dean
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Tue, 17 Jul 2001, Aaron Bannert wrote:
On Tue, Jul 17, 2001 at 01:29:47AM -0700, dean gaudet wrote:
On Sun, 15 Jul 2001, Sander Striker wrote:
Why are we so desperate in opting out the child-pool creation?
I don't really have problems with a child pool for each thread. Actually,
it will make the dynamic locking a lot easier to implement if it stays.
all threads MUST have their own private pool root.
otherwise you're just throwing scalability away in locks. (which is
proved by the claim i saw that ian got better performance by defining
ALLOC_USE_MALLOC on an 8-way.)
I totally agree, but only as a solution in httpd.
I also believe that we should provide this [application-specific requirement]
outside of the basic thread support in APR.
Please allow me to use pseudocode:
void * worker_function(void * opaque_application_data) {
apr_pool_t *thread_pool;
create_child_pool(thread_pool, global_root_pool);
do_thread_stuff();
cleanup_child_pool(thread_pool);
if (apr_thread_exit_makes_sense_to_call_from_inside_worker_function) {
int rv = APR_SUCCESS;
apr_thread_exit(rv);
}
return NULL;
}
What I mean by this gibberish is that the pool-handling code can exist
completely outside of our thread_creation-type functions in APR. Is there
any reason for APR threads to do more than this (that is, just simple thread
creation)?
We have hit an impass in my mind. Dean and I are saying that having each
thread have it's own pool is a requirement. Not just for httpd, but for
anything using pools. Dean, if I am mis-interpreting you, then I am
sorry, and please correct me.
Aaron, you disagree. you want each app to make that decision on their
own. I see no way to implement that in a thread-safe manner.
So, I would suggest that we approach this problem in another way. My ONLY
goal for APR is to make a library that is useful for cross-platform
development and eases the pain of cross-platform development for
developers. That was the goal when I created APR, and my original test
case was the Apache web server.
I believe that the problem is that the threaded code is creating the
pool, and not advertising it to the thread itself. This is an easy thing
to fix. I do not agree that every APR app that uses threads should have
to create their own thread pools. That is wasted effort by every APR app.
I would like to see a conclusion to this thread quickly. So, could people
please post their desires quickly, and what they want to see from this.
Ryan
_
Ryan Bloom [EMAIL PROTECTED]
Covalent Technologies [EMAIL PROTECTED]
-
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
From: [EMAIL PROTECTED] Sent: Tuesday, July 17, 2001 11:13 AM I believe that the problem is that the threaded code is creating the pool, and not advertising it to the thread itself. This is an easy thing to fix. I do not agree that every APR app that uses threads should have to create their own thread pools. That is wasted effort by every APR app. I would like to see a conclusion to this thread quickly. So, could people please post their desires quickly, and what they want to see from this. thread-local pools would be a nice improvement, perhaps as an argument to apr_thread_create. They can be flagged such that they avoid locking their own own allocations (of course their 'parent' allocator, malloc, or the parent free-list management, may still need to lock based on platform and so forth). But _unless_ they remain rooted to the top level pool, in apr_process_create they become orphaned, and their cleanups are never executed. That isn't workable. It sounds like (forgive me if I'm stating the obvious) that a pool needs two kinds of parent sms pools. One is the 'heirarchy' for purposes of setup and teardown. The other is the 'allocator', where the blocks will be allocated from. Want to use the conventional 1.3 model? The heirarchial parent is the parent pool, but the allocatation parent is the single top level pool. Want to flip to a malloc, that trusts the clib to 'do the right thing'(sm) with cpu/thread specific free lists? Then we build a very simple malloc/free sms that can be passed as the allocation parent when the pool is created. If I've abused the sms concept here, then correct me, but I don't see any issues remaining beyond semantics and syntax. Bill
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
I'm not sure that the alternative is workable, either. At the time of the fork, when the child process gets a snapshot of the parent's memory, it's possible that some thread other than the one invoking fork could be halfway through registering a new resource (e.g., file descriptor) in its pool. There's no guarantee that it's safe to attempt a cleanup of that other thread's pool in the child process; if the fork has caught the data structures in an intermediate state, attempting to destroy that pool might yield a segv. Correct me if I'm wrong, but that would be a property of a buggy MPM. If the MPM chooses to mix non-synchronized fork()s and thread invocation, than that's what it gets. -aaron
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
We have hit an impass in my mind. Dean and I are saying that having each thread have it's own pool is a requirement. Not just for httpd, but for anything using pools. Dean, if I am mis-interpreting you, then I am sorry, and please correct me. Aaron, you disagree. you want each app to make that decision on their own. I see no way to implement that in a thread-safe manner. (Just as a side note, I don't remember seeing anything refuting the technical aspects of such an implementation. I would be interested in one, of course, if you think it is infeasible.) So, I would suggest that we approach this problem in another way. My ONLY goal for APR is to make a library that is useful for cross-platform development and eases the pain of cross-platform development for developers. That was the goal when I created APR, and my original test case was the Apache web server. I believe that the problem is that the threaded code is creating the pool, and not advertising it to the thread itself. This is an easy thing to fix. I do not agree that every APR app that uses threads should have to create their own thread pools. That is wasted effort by every APR app. I would like to see a conclusion to this thread quickly. So, could people please post their desires quickly, and what they want to see from this. In my mind, there are really only two major points that need addressing: - Optionally creating a child-pool for each thread. This provides us two things in my mind: 1) Allowing the application to choose an SMS for this thread-pool (which could very well be a special SMS created just for this purpose). 2) Allowing an application to refrain from creating a child-pool. - Like you [Ryan] said, it is a problem that the thread creation routines right now are not advertising the pool to the thread itself. This to me can be corrected with a better API. The two things I've brought up here are: 1) void * worker_fn(void *opaque_data) should have an additional pool param. 2) apr_thread_exit() is unacceptable. It currently requires a apr_thread_t structure. I would be happy if either the worker_fn was extended to pass in the apr_thread_t OR if we could architect a way for apr_thread_exit() to not need the apr_thread_t struct. I have a feeling the second point will be more palatable, and as such I'll volunteer to cook up a patch illustrating these changes. As for the first point, which is the reason this thread has been going on so long, I'm willing to compromise on the idea of forcefully creating a childpool always, if it necessary, merely because I see it only as overhead for some applications, but not a functional defect. -aaron
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
From: Justin Erenkrantz [EMAIL PROTECTED] Sent: Tuesday, July 17, 2001 12:00 PM On Tue, Jul 17, 2001 at 11:26:55AM -0500, William A. Rowe, Jr. wrote: But _unless_ they remain rooted to the top level pool, in apr_process_create they become orphaned, and their cleanups are never executed. That isn't workable. I disagree with this. I gave my big long monologue on this on Sunday night. In summary, since you can't cleanup the thread itself, why are we cleaning up its memory? -- justin Because, otherwise, all resources (files, sockets, locks, etc) are not closed when another thread forks. There is a difference between a clean thread exit and cleaning up after a thread. We can't do a clean thread exit. We can clean up after one, if the resources are apr_entities with properly registered cleanups.
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Tue, Jul 17, 2001 at 10:17:01AM -0700, Brian Pane wrote: Aaron Bannert wrote: I'm not sure that the alternative is workable, either. At the time of the fork, when the child process gets a snapshot of the parent's memory, it's possible that some thread other than the one invoking fork could be halfway through registering a new resource (e.g., file descriptor) in its pool. There's no guarantee that it's safe to attempt a cleanup of that other thread's pool in the child process; if the fork has caught the data structures in an intermediate state, attempting to destroy that pool might yield a segv. Correct me if I'm wrong, but that would be a property of a buggy MPM. If the MPM chooses to mix non-synchronized fork()s and thread invocation, than that's what it gets. Synchronizing the forks with pool resource registration isn't a scalable design; it requires locking a process-wide lock every time you want to register a resource in a per-thread pool. It would be a huge mistake to slow down the routine operation of the httpd in order to optimize for fork. I wasn't suggesting that, I was suggesting that it's a bad idea to start fork()ing off processes from an MPM that's also doing threads and you want that fork()ed of process to later do cleanups. My point is it's solely an MPM architecture decision. Are you saying that this is a typical scenario? -aaron
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
Aaron Bannert wrote: On Tue, Jul 17, 2001 at 10:17:01AM -0700, Brian Pane wrote: Aaron Bannert wrote: I'm not sure that the alternative is workable, either. At the time of the fork, when the child process gets a snapshot of the parent's memory, it's possible that some thread other than the one invoking fork could be halfway through registering a new resource (e.g., file descriptor) in its pool. There's no guarantee that it's safe to attempt a cleanup of that other thread's pool in the child process; if the fork has caught the data structures in an intermediate state, attempting to destroy that pool might yield a segv. Correct me if I'm wrong, but that would be a property of a buggy MPM. If the MPM chooses to mix non-synchronized fork()s and thread invocation, than that's what it gets. Synchronizing the forks with pool resource registration isn't a scalable design; it requires locking a process-wide lock every time you want to register a resource in a per-thread pool. It would be a huge mistake to slow down the routine operation of the httpd in order to optimize for fork. I wasn't suggesting that, I was suggesting that it's a bad idea to start fork()ing off processes from an MPM that's also doing threads and you want that fork()ed of process to later do cleanups. I agree. My point is it's solely an MPM architecture decision. Are you saying that this is a typical scenario? I think it's an atypical scenario, and one that can't be implemented very well anyway. (The synchronized approach has a performance problem, and the unsynchronized approach can crash due to a race condition.) Thus my argument is that this scenario (forking from a threaded MPM and then trying to clean up the other threads' resources in the child process) isn't a valid use case for the design of the pools. --Brian
Re: Pools in threads WAS Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Sun, Jul 15, 2001 at 02:33:54PM -0700, [EMAIL PROTECTED] wrote: Guys, before you make comments like this, you should really read the code. First of all, the thread_exit call needs to take in the thread variable. Without it, platforms like OS/2 can't return values from the thread exit routine. We could stipulate in APR that on OS/2 we won't return the result code. Big deal. It's not the end of the world. And, it's only a hack that works in conjunction with OS/2's apr_thread_join. OS/2 threads obviously don't support return codes. APR is only as good as the underlying thread library. BeOS, Unix (pthread), and Win32 don't need the thread structure to destroy the thread or return the result code. That seems to be the consensus on how threads should operate. If you are willing to lose the return code on OS/2, you now have OS/2 not needing the thread structure either. APR is a game of tradeoffs - this is one we should make in order to get a better API at the cost of something that this native OS doesn't provide anyway. We should be seeing red flags everywhere because our own code isn't calling the apr_thread_exit() function. Why? The API is, uh, lacking. Secondly, as long as the thread has access to the apr_thread_t, it also has access to the pool within that value. Well, I disagree that the thread should have access to its underlying apr_thread_t. It's a very awkward thing to do. Doable? Certainly. Clean? No. You now need a pointer into the thread array (or linked-list if that is how you do it - almost all places do it now with arrays) passed to the worker function. Gag. And, once a thread is spawned, it is now on its own for memory. So, it should create a SMS independent of any other SMS. That's the goal and driving force behind SMS - we need a per-thread SMS. There is *zero* benefit to having any relationship between the thread's memory pool and the parent's memory pool. You can't cleanup the thread from the parent anyway, so trust the thread to cleanup itself (as well as its pool and any memory allocations). I fail to see the problem here. Sever the imagined relationship. The code becomes simpler, too. -- justin
Re: Pools in threads WAS Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
There is *zero* benefit to having any relationship between the thread's memory pool and the parent's memory pool. You can't cleanup the thread from the parent anyway, so trust the thread to cleanup itself (as well as its pool and any memory allocations). I fail to see the problem here. Sever the imagined relationship. The code becomes simpler, too. -- justin col. problem-solving by conventions. follow these rules, everything's okay.
Re: Pools in threads WAS Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Sun, 15 Jul 2001, Justin Erenkrantz wrote: On Sun, Jul 15, 2001 at 02:33:54PM -0700, [EMAIL PROTECTED] wrote: Guys, before you make comments like this, you should really read the code. First of all, the thread_exit call needs to take in the thread variable. Without it, platforms like OS/2 can't return values from the thread exit routine. We could stipulate in APR that on OS/2 we won't return the result code. Big deal. It's not the end of the world. And, it's only a hack that works in conjunction with OS/2's apr_thread_join. OS/2 threads obviously don't support return codes. APR is only as good as the underlying thread library. No, because that is utterly bogus. The whole point of APR, is to allow a program to be written that will work on ANY platform. There are very few features that can't be implemented cross platform. If you are willing to lose the return code on OS/2, you now have OS/2 not needing the thread structure either. APR is a game of tradeoffs - this is one we should make in order to get a better API at the cost of something that this native OS doesn't provide anyway. No. APR is not a game of tradeoffs. It has never been a game of tradeoffs. We only make decisions like this when it is absolutely impossible to implement a feature on a given system. As we have already shown, this can be implemented. We should be seeing red flags everywhere because our own code isn't calling the apr_thread_exit() function. Why? The API is, uh, lacking. No, that is not why we don't call apr_thread_exit. We don't call apr_thread_exit, because we don't need it. There are two ways for threads to terminate. 1) Call apr_thread_exit. 2) return from the function that was started when the thread was created. We happen to use option 2, because it fits our model better. Do not misinterpret a choice for a flaw. Even before the APR threading code was written, there was a hybrid thread/process version of Apache. Versions of this can be found in the apache-apr repository. That original hybrid Apache did not EVER call pthread_exit(). This was a conscious choice made by myself, Bill Stoddard, and Manoj Kasichainula. Secondly, as long as the thread has access to the apr_thread_t, it also has access to the pool within that value. Well, I disagree that the thread should have access to its underlying apr_thread_t. It's a very awkward thing to do. Doable? Certainly. Clean? No. You now need a pointer into the thread array (or linked-list if that is how you do it - almost all places do it now with arrays) passed to the worker function. Gag. So it has been implemented poorly. That doesn't mean anything. A poor implementation is very often where things begin. It is relatively simple to fix the implementation. And, once a thread is spawned, it is now on its own for memory. So, it should create a SMS independent of any other SMS. That's the goal and driving force behind SMS - we need a per-thread SMS. SMS has nothing to do with threads. I made a mistake when I wrote APR. I tied it to pools. But pools don't work for all apps. They happen to work really well for the web server, but most apps don't have memory profiles like a web server. The idea of an SMS-like memory allocator was discussed years ago, and the idea was to make APR useful outside of the web server. There is *zero* benefit to having any relationship between the thread's memory pool and the parent's memory pool. You can't cleanup the thread from the parent anyway, so trust the thread to cleanup itself (as well as its pool and any memory allocations). I fail to see the problem here. Sever the imagined relationship. The code becomes simpler, too. -- justin Pools are by their very nature hierarchical. That is why the relationship is there, and why it needs to remain there. You can't just rely on the thread to cleanup itself. Pools (and SMS's) are used for a LOT more than just memory allocation in the server. They make sure that descriptors are closed correctly. In some places, they are used to make sure that anything the child opened that isn't returned to the OS automatically is returned when the child goes away. If you divorce the thread pool from the pool for the child, you WILL break many assumptions in the server. Ryan _ Ryan Bloom [EMAIL PROTECTED] Covalent Technologies [EMAIL PROTECTED] -
Pools in threads WAS Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Sun, Jul 15, 2001 at 07:16:35PM +0200, Sander Striker wrote: Fair enough. It's just that in order to opt-out of the child-pool creating process in apr_thread_create, we're going to have to add a parameter Why are we so desperate in opting out the child-pool creation? I don't really have problems with a child pool for each thread. Actually, it will make the dynamic locking a lot easier to implement if it stays. The problem is that apr_thread_create is completely bogus right now. The child thread has no access to that pool in apr_thread_t-pool. If you want a pool in the thread, you should be creating inside of the thread's function not in apr_thread_create. And, apr_thread_exit call should *not* be taking in an apr_thread_t because the child doesn't have access to that either. We're enforcing the use of global variables to work around this and that's plain wrong. -- justin
Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Sun, Jul 15, 2001 at 07:16:35PM +0200, Sander Striker wrote: Fair enough. It's just that in order to opt-out of the child-pool creating process in apr_thread_create, we're going to have to add a parameter Why are we so desperate in opting out the child-pool creation? I don't really have problems with a child pool for each thread. Actually, it will make the dynamic locking a lot easier to implement if it stays. I have some use cases that just want a thread. Since no child-pool is required in this case, it becomes unnecessary overhead (that is currently broken, as the child-pool is only cleaned in apr_thread_exit() but that whole thing is screwey). So if apr threads didn't know anything about child-pools, then they would just simply create a thread that called my worker_function(). I could pass in my opaque data to that worker_function(), and do the child-pool creations and destruction right there. I just don't see how child-pool creation MUST be in apr_thread_create() -aaron
Thread pools are isolated WAS Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
Ah, ok, I'm getting the picture now. Well, for my purposes, the child pool creation doesn't have to be in apr_thread_create. There doesn't have to be child pool creation at all. But, I do want the worker fn to be able to create a child pool from the pool passed into apr_thread_create. So, we need a way to pass in the data to the worker fn. No. I thought we were going to per-thread pools. And, this is why we were going this way. This removes all of the notions of cleanups from a parent trying to forcibly cleanup the thread. IMHO, there should be no relationship between the parent SMS and its threads' SMS. -- justin
RE: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Sun, 15 Jul 2001, Sander Striker wrote: Fair enough. It's just that in order to opt-out of the child-pool creating process in apr_thread_create, we're going to have to add a parameter Why are we so desperate in opting out the child-pool creation? I don't really have problems with a child pool for each thread. Actually, it will make the dynamic locking a lot easier to implement if it stays. The reality is that it should stay. The problem we have right now, is that httpd creates a thread-pool, and then APR creates one. So, if httpd didn't create one, and just used the one APR created automatically, this problem would go away completely. Ryan _ Ryan Bloom [EMAIL PROTECTED] Covalent Technologies [EMAIL PROTECTED] -
Re: Pools in threads WAS Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
On Sun, 15 Jul 2001, Justin Erenkrantz wrote: On Sun, Jul 15, 2001 at 07:16:35PM +0200, Sander Striker wrote: Fair enough. It's just that in order to opt-out of the child-pool creating process in apr_thread_create, we're going to have to add a parameter Why are we so desperate in opting out the child-pool creation? I don't really have problems with a child pool for each thread. Actually, it will make the dynamic locking a lot easier to implement if it stays. The problem is that apr_thread_create is completely bogus right now. The child thread has no access to that pool in apr_thread_t-pool. If you want a pool in the thread, you should be creating inside of the thread's function not in apr_thread_create. And, apr_thread_exit call should *not* be taking in an apr_thread_t because the child doesn't have access to that either. We're enforcing the use of global variables to work around this and that's plain wrong. -- justin Guys, before you make comments like this, you should really read the code. First of all, the thread_exit call needs to take in the thread variable. Without it, platforms like OS/2 can't return values from the thread exit routine. Secondly, as long as the thread has access to the apr_thread_t, it also has access to the pool within that value. What you all need to realize, is that some of the annoying pieces of APR exist because we tried to do it the way that makes sense to all of you, and we had to change it to make it work on all platforms. Please, look through what ALL platforms do before you decide to go changing API's. You are going to break things that work today. Ryan _ Ryan Bloom [EMAIL PROTECTED] Covalent Technologies [EMAIL PROTECTED] -
RE: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
Right, changed the subject line again, the typo was hurting my eyes and I wouldn't want a thread with that subj. line. Sorry for reposting. Justin: By having the possibility of having other children processes, you now need a mechanism to kill all threads in the same process efficiently. You'd need to kick them out of the accept mutex, but I'm not seeing how that's going to happen quickly. -- justin Ryan: You need to be able to kill all threads in a process efficiently regardless. We will always need to be able to re-spawn processes that die, and we will always have the concept of MaxRequestsPerChild. If a process is going to die and be replaced, then we need to kill all the threads in that process as quickly as possible. Only allowing one process at a time doesn't remove that requirement. The way I see it, each process has a single pool instance as the parent for all the threads. Resetting or destroying that pool should effectively kill all threads. What am I missing? Sander
RE: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded
The way I see it, each process has a single pool instance as the parent for all the threads. Resetting or destroying that pool should effectively kill all threads. What am I missing? As I see it, the problem is: [ Platforms with SAFE_ACCEPT == APR_SUCCESS rather than the lock don't apply here. ] All threads are stuck in the accept mutex except for one. It reads the POD. And, it is time to die. It sets workers_may_exit to 1. It then releases the accept mutex and exits. Another *child process* acquires the accept mutex. Depending upon how many child processes you have, it may take a while for all of the threads in the doomed child process to acquire the mutex, wake up, check workers_may_exit, release the mutex, and then exit. Only when all of the threads in the doomed child process are exited does any cleanup occur. These sibling threads are blocked until they receive the mutex. They aren't going to be going anywhere until the mutex is turned over to them. They can't check the value of workers_may_exit *until* they acquire the mutex. Condition variables help *somewhat*, but the problem is now of scope - you have an interprocess condition and a intraprocess condition to check. Does this work? I don't know. And, you can't kick the thread out of the mutex acquire (pthread_cancel or similar strategies don't cancel a mutex operation), so you are screwed. Err, doesn't destruction of the mutex wake everyone up? Oh, wait, does every process share the same mutex? And, destroying its parent pool does *NOT* destroy the thread. Look at the code again. Well, it should, although it may not do so now. The only person that can call pthread_exit() is the actual thread itself. You can't call pthread_exit on behalf of another thread (i.e. from the thread that knows it is doomed or a cleanup thread). It just doesn't work like that. I just don't agree with Ryan's assessment here. But, it's possible I'm missing some major piece of code. -- justin Could be I'm way off too. Sander
