Re: How Do I Create A Per-Worker Pool?
On Thu, 26 Sep 2002, Charles Reitzel wrote: > Thanks for telling me what I need to know: "you can't get there from here." > > I don't want to start a philosophical debate, but it is a common idiom > (yea, verily a "pattern") in multi-threaded programming to avoid contention > by duplicating singletons in each thread. This can be done either by using > platform-specific thread-local storage or, portably, by passing in the > needed singleton (or a place to put it) into the worker thread. > > A mutex-free, per-thread pool would serve this purpose perfectly. Change > the word "thread" to "worker" in the above, and you have made it portable > across MPM models. > > A per-thread pool would assist a great deal in bringing many Apache 1.x > modules over to 2.x. In essence, you move the globals into the per-thread > pool, and the remaining application semantics are unchanged. > > You can't "just add synchronization" to most > applications/modules. Usually, you need to redesign much of it to make it > work without killing performance. > > Convinced? Unfortunately nope. You are confusing a programming model with pools. Pools are about variable life-time. In the case of what you are trying to do, it is possible already. All you need to do, is do the initalization in the child_init phase, and save each version of the data in a table that is accessible by the whole process. The problem with a qorker_rec, is that it doesn't fit with the Apache model. Yes, it would be possible to add the pointer to the request_Rec, but it isn't necessary. Everything in Apache is centered around the request, not the thread/process that is serving the data. By adding a worker_rec, we are actually encouraging programming practices that we don't want, like copying a bunch of global data into a worker_pool. We would much rather that modules were written with thread-safety in mind, so while it is possible to do, it isn't the right solution. Also, we did have a thread_pool at one time that worked for threaded MPMs. The problem was that it just wasn't ever used. This makes sense, because pools are about lifetime, and in all of the MPMs (except perchild), threads have the same lifetime as processes by definition. I hope that makes sense. Ryan
Re: How Do I Create A Per-Worker Pool?
Thanks for telling me what I need to know: "you can't get there from here." I don't want to start a philosophical debate, but it is a common idiom (yea, verily a "pattern") in multi-threaded programming to avoid contention by duplicating singletons in each thread. This can be done either by using platform-specific thread-local storage or, portably, by passing in the needed singleton (or a place to put it) into the worker thread. A mutex-free, per-thread pool would serve this purpose perfectly. Change the word "thread" to "worker" in the above, and you have made it portable across MPM models. A per-thread pool would assist a great deal in bringing many Apache 1.x modules over to 2.x. In essence, you move the globals into the per-thread pool, and the remaining application semantics are unchanged. You can't "just add synchronization" to most applications/modules. Usually, you need to redesign much of it to make it work without killing performance. Convinced? take it easy, Charlie On Wed, 26 Sep 2002, Ryan Bloom wrote: >You shouldn't be trying to access a per-thread pool. It isn't necessary. >The pool in the request-rec, is by definition, only available within that >request, which is limited to a single thread. The only reason to access a >per-thread pool, is if you want to allocate memory that has the same >lifetime as the thread itself. There is currently no way to do that, but >there also hasn't been a convincing reason to do it. Ryan
Re: How Do I Create A Per-Worker Pool?
On Wed, 25 Sep 2002, Charles Reitzel wrote: > Thanks for the reply. Forget the worker_rec for the time being. How do > you access the per-thread pool from a module? > > Sorry if I need it spelled out. But I don't see a pointer to it on the > request_rec, the server_rec is process-wide and the child_init() callback > is called once-per-process, not once-per-thread, so I am guessing that is > the process pool passed in. What am I missing? > > More info: I am developing with Win32 (mpm_winnt) for now, but I want to > deploy to Linux. I'd like to have things work both places. You shouldn't be trying to access a per-thread pool. It isn't necessary. The pool in the request-rec, is by definition, only available within that request, which is limited to a single thread. The only reason to access a per-thread pool, is if you want to allocate memory that has the same lifetime as the thread itself. There is currently no way to do that, but there also hasn't been a convincing reason to do it. Ryan
Re: How Do I Create A Per-Worker Pool?
Thanks for the reply. Forget the worker_rec for the time being. How do you access the per-thread pool from a module? Sorry if I need it spelled out. But I don't see a pointer to it on the request_rec, the server_rec is process-wide and the child_init() callback is called once-per-process, not once-per-thread, so I am guessing that is the process pool passed in. What am I missing? More info: I am developing with Win32 (mpm_winnt) for now, but I want to deploy to Linux. I'd like to have things work both places. take it easy, Charlie On On Wed, 25 Sep 2002, Ryan Bloom wrote: >On Wed, 25 Sep 2002, Charles Reitzel wrote: > > Objective: to create a mutex-free pool per worker in non-MPM-specific > > way. > > ... > > > > I found this exchange in a June posting: > > > >> Just do: > >> apr_allocator_t *allocator; > >> apr_allocator_create(&allocator); > >> apr_pool_create_ex(&pool, parent_pool, abort_fn, allocator); > >> apr_allocator_owner_set(allocator, pool); > > ... > > > > Looks good. But I have questions. What is the correct place to > > put this code and where do keep the pool pointer afterwards. I.e. > > how do you find the pool from within a module handler? > > ... > >The structures in Apache are designed around an HTTP request, not the >entity that is running the request. Every threaded MPM has a pool for each >thread, which does allow for thread-safe programming. Why exactly would >you want a worker_rec? What problem are you having? Ryan
Re: How Do I Create A Per-Worker Pool?
I was wondering that point (and thatr's what I asked Ian). So, the way is to move to A2, but is it reliable as the 1.3 version, I mean is as stable as the older brother, or is it under development? I heard it is rock solid, but I coudnt check. Thanks in advance. Leo. - Original Message - From: <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Wednesday, September 25, 2002 4:13 PM Subject: Re: How Do I Create A Per-Worker Pool? > This message uses a character set that is not supported by the Internet > Service. To view the original message content, open the attached > message. If the text doesn't display correctly, save the attachment to > disk, and then open it using a viewer that can display the original > character set. <> >
Re: How Do I Create A Per-Worker Pool?
On Wed, 25 Sep 2002, Ian Holsman wrote: > Leonardo Javier Belén wrote: > > I think I have a question to ask associated to this thread: How can I > > implement a "global pool" of objects. Let me say it better, I want a global > > pool of database connection, first because I have a limited number of > > connections and second, because I want to be able to dominate the dbms > > accesses. All developed as an "ansi C" apache module, but I thing is the > > same. I'm running Apache 1.3. Is this the same thing? Leonardo Javier Belen. > > AFIP-AR > > Have a look at apr_reslist.c in the apr-util/misc directory. apr_reslist.c actually won't work in this case. He is trying to use Apache 1.3, which means that the connections would need to be shared across processes. In general, it just isn't worth doing a connection pool with 1.3, because of the overhead of the process model. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: How Do I Create A Per-Worker Pool?
Thanks for the tip, but is this actually Ok if I try to use it on Apache 1.3 Leo. - Original Message - From: "Ian Holsman" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Wednesday, September 25, 2002 4:06 PM Subject: Re: How Do I Create A Per-Worker Pool? Leonardo Javier Bel&alefmaks;n wrote: > I think I have a question to ask associated to this thread: How can I > implement a "global pool" of objects. Let me say it better, I want a global > pool of database connection, first because I have a limited number of > connections and second, because I want to be able to dominate the dbms > accesses. All developed as an "ansi C" apache module, but I thing is the > same. I'm running Apache 1.3. Is this the same thing? Leonardo Javier Belen. > AFIP-AR Have a look at apr_reslist.c in the apr-util/misc directory. > > - Original Message - > From: <[EMAIL PROTECTED]> > To: <[EMAIL PROTECTED]> > Sent: Wednesday, September 25, 2002 3:29 PM > Subject: Re: How Do I Create A Per-Worker Pool? > > > >>On Wed, 25 Sep 2002, Charles Reitzel wrote: >> >> >>>Hi All, >>> >>>This is a thorny (to me) module development question. I have asked on >> >>the >> >>>module list and searched the archives, but have found only a partial >>>answer. Any pointers will be appreciated. >>> >>>Objective: to create a mutex-free pool per worker in non-MPM-specific >> >>way. >> >>>I found this exchange in a June posting: >>> >>>>>Another change we made, as I mentioned in a previous >>>>>Email, was using non-mutexing per-thread memory >>>>>pools (HeapCreate(HEAP_NO_SERIALIZE, ...)). > To get >>>>>best performance with Apache 2 we would really need >>>>>such a memory pool. >>>> >>>>And we already have it! Just do: >>>> >>>>apr_allocator_t *allocator; >>>>apr_allocator_create(&allocator); >>>>apr_pool_create_ex(&pool, parent_pool, abort_fn, allocator); >>>>apr_allocator_owner_set(allocator, pool); >>>> >>>>Now you have a mutexless allocator associated with a pool. All child >> >>pools >> >>>>of this pool will use the same allocator and will therefor also have >> >>no mutex. >> >>>Looks good. But I have questions. What is the correct place to put >> >>this >> >>>code and where do keep the pool pointer afterwards. I.e. how do you >> >>find >> >>>the pool from within a module handler? >>> >>> >>>To my mind, the Apache module API is missing an important data >> >>structure: >> >>>worker_rec. It is obviously redundant (but does no harm) in the >> >>prefork >> >>>MPM, but is a necessary ingredient in _any_ threaded MPM. Such a >> >>structure >> >>>would allow modules to be written as thread-safe - independent of the >> >>MPM >> >>>currently configured by the user. The request record could and should >> >>>contain a pointer to the current worker for easy access. In the >> >>absence of >> >>>a worker_rec, however, some guidance on the above would be helpful. >>> >> >>The structures in Apache are designed around an HTTP request, not the >>entity that is running the request. Every threaded MPM has a pool for >>each thread, which does allow for thread-safe programming. >> >>Why exactly would you want a worker_rec? What problem are you having? >> >>Ryan >> >>__ __ >>___ >>Ryan Bloom[EMAIL PROTECTED] >>550 Jean St >>Oakland CA 94610 >>-- -- >>--- > >
Re: How Do I Create A Per-Worker Pool?
Leonardo Javier Bel&alefmaks;n wrote: > I think I have a question to ask associated to this thread: How can I > implement a "global pool" of objects. Let me say it better, I want a global > pool of database connection, first because I have a limited number of > connections and second, because I want to be able to dominate the dbms > accesses. All developed as an "ansi C" apache module, but I thing is the > same. I'm running Apache 1.3. Is this the same thing? Leonardo Javier Belen. > AFIP-AR Have a look at apr_reslist.c in the apr-util/misc directory. > > - Original Message - > From: <[EMAIL PROTECTED]> > To: <[EMAIL PROTECTED]> > Sent: Wednesday, September 25, 2002 3:29 PM > Subject: Re: How Do I Create A Per-Worker Pool? > > > >>On Wed, 25 Sep 2002, Charles Reitzel wrote: >> >> >>>Hi All, >>> >>>This is a thorny (to me) module development question. I have asked on >> >>the >> >>>module list and searched the archives, but have found only a partial >>>answer. Any pointers will be appreciated. >>> >>>Objective: to create a mutex-free pool per worker in non-MPM-specific >> >>way. >> >>>I found this exchange in a June posting: >>> >>>>>Another change we made, as I mentioned in a previous >>>>>Email, was using non-mutexing per-thread memory >>>>>pools (HeapCreate(HEAP_NO_SERIALIZE, ...)). > To get >>>>>best performance with Apache 2 we would really need >>>>>such a memory pool. >>>> >>>>And we already have it! Just do: >>>> >>>>apr_allocator_t *allocator; >>>>apr_allocator_create(&allocator); >>>>apr_pool_create_ex(&pool, parent_pool, abort_fn, allocator); >>>>apr_allocator_owner_set(allocator, pool); >>>> >>>>Now you have a mutexless allocator associated with a pool. All child >> >>pools >> >>>>of this pool will use the same allocator and will therefor also have >> >>no mutex. >> >>>Looks good. But I have questions. What is the correct place to put >> >>this >> >>>code and where do keep the pool pointer afterwards. I.e. how do you >> >>find >> >>>the pool from within a module handler? >>> >>> >>>To my mind, the Apache module API is missing an important data >> >>structure: >> >>>worker_rec. It is obviously redundant (but does no harm) in the >> >>prefork >> >>>MPM, but is a necessary ingredient in _any_ threaded MPM. Such a >> >>structure >> >>>would allow modules to be written as thread-safe - independent of the >> >>MPM >> >>>currently configured by the user. The request record could and should >> >>>contain a pointer to the current worker for easy access. In the >> >>absence of >> >>>a worker_rec, however, some guidance on the above would be helpful. >>> >> >>The structures in Apache are designed around an HTTP request, not the >>entity that is running the request. Every threaded MPM has a pool for >>each thread, which does allow for thread-safe programming. >> >>Why exactly would you want a worker_rec? What problem are you having? >> >>Ryan >> >> >>___ >>Ryan Bloom[EMAIL PROTECTED] >>550 Jean St >>Oakland CA 94610 >> >>--- > >
Re: How Do I Create A Per-Worker Pool?
I think I have a question to ask associated to this thread: How can I implement a "global pool" of objects. Let me say it better, I want a global pool of database connection, first because I have a limited number of connections and second, because I want to be able to dominate the dbms accesses. All developed as an "ansi C" apache module, but I thing is the same. I'm running Apache 1.3. Is this the same thing? Leonardo Javier Belen. AFIP-AR - Original Message - From: <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Wednesday, September 25, 2002 3:29 PM Subject: Re: How Do I Create A Per-Worker Pool? > On Wed, 25 Sep 2002, Charles Reitzel wrote: > > > Hi All, > > > > This is a thorny (to me) module development question. I have asked on > the > > module list and searched the archives, but have found only a partial > > answer. Any pointers will be appreciated. > > > > Objective: to create a mutex-free pool per worker in non-MPM-specific > way. > > > > I found this exchange in a June posting: > > > > Another change we made, as I mentioned in a previous > > > > Email, was using non-mutexing per-thread memory > > > > pools (HeapCreate(HEAP_NO_SERIALIZE, ...)). > To get > > > > best performance with Apache 2 we would really need > > > > such a memory pool. > > > > > >And we already have it! Just do: > > > > > > apr_allocator_t *allocator; > > > apr_allocator_create(&allocator); > > > apr_pool_create_ex(&pool, parent_pool, abort_fn, allocator); > > > apr_allocator_owner_set(allocator, pool); > > > > > >Now you have a mutexless allocator associated with a pool. All child > pools > > >of this pool will use the same allocator and will therefor also have > no mutex. > > > > Looks good. But I have questions. What is the correct place to put > this > > code and where do keep the pool pointer afterwards. I.e. how do you > find > > the pool from within a module handler? > > > > > > To my mind, the Apache module API is missing an important data > structure: > > worker_rec. It is obviously redundant (but does no harm) in the > prefork > > MPM, but is a necessary ingredient in _any_ threaded MPM. Such a > structure > > would allow modules to be written as thread-safe - independent of the > MPM > > currently configured by the user. The request record could and should > > > contain a pointer to the current worker for easy access. In the > absence of > > a worker_rec, however, some guidance on the above would be helpful. > > > > The structures in Apache are designed around an HTTP request, not the > entity that is running the request. Every threaded MPM has a pool for > each thread, which does allow for thread-safe programming. > > Why exactly would you want a worker_rec? What problem are you having? > > Ryan > > > ___ > Ryan Bloom[EMAIL PROTECTED] > 550 Jean St > Oakland CA 94610 > > ---
Re: How Do I Create A Per-Worker Pool?
On Wed, 25 Sep 2002, Charles Reitzel wrote: > Hi All, > > This is a thorny (to me) module development question. I have asked on the > module list and searched the archives, but have found only a partial > answer. Any pointers will be appreciated. > > Objective: to create a mutex-free pool per worker in non-MPM-specific way. > > I found this exchange in a June posting: > > > Another change we made, as I mentioned in a previous > > > Email, was using non-mutexing per-thread memory > > > pools (HeapCreate(HEAP_NO_SERIALIZE, ...)). > To get > > > best performance with Apache 2 we would really need > > > such a memory pool. > > > >And we already have it! Just do: > > > > apr_allocator_t *allocator; > > apr_allocator_create(&allocator); > > apr_pool_create_ex(&pool, parent_pool, abort_fn, allocator); > > apr_allocator_owner_set(allocator, pool); > > > >Now you have a mutexless allocator associated with a pool. All child pools > >of this pool will use the same allocator and will therefor also have no mutex. > > Looks good. But I have questions. What is the correct place to put this > code and where do keep the pool pointer afterwards. I.e. how do you find > the pool from within a module handler? > > > To my mind, the Apache module API is missing an important data structure: > worker_rec. It is obviously redundant (but does no harm) in the prefork > MPM, but is a necessary ingredient in _any_ threaded MPM. Such a structure > would allow modules to be written as thread-safe - independent of the MPM > currently configured by the user. The request record could and should > contain a pointer to the current worker for easy access. In the absence of > a worker_rec, however, some guidance on the above would be helpful. > The structures in Apache are designed around an HTTP request, not the entity that is running the request. Every threaded MPM has a pool for each thread, which does allow for thread-safe programming. Why exactly would you want a worker_rec? What problem are you having? Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
