RE: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-12 Thread Tair-Shian Chou
Greg wrote: > then the compiler may optimize this as: > > wrlock >test if exists again >allocate element >insert element > >> race is here, prior to filling in element, another thread > may >> read element, and use element that hasn't been filled in yet. >fill in element > unlo

RE: util_ldap [Bug 29217] - Remove references to calloc()and free()

2004-06-12 Thread Tair-Shian Chou
Brad Nicholes wrote: > > In fact, I don't think that these are shared locks at all > > #define LDAP_CACHE_LOCK_CREATE(p) \ > if (!st->util_ldap_cache_lock) > apr_thread_rwlock_create(&st->util_ldap_cache_lock, st->pool) > > which means that in the shared memory cache, it is highly > li

FW: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-11 Thread Mathihalli, Madhusudan
Another mail.. -Madhu >-Original Message- >From: CHOU,TAIR-SHIAN (HP-Cupertino,ex1) >[mailto:[EMAIL PROTECTED] >Sent: Friday, June 11, 2004 7:05 PM >To: Mathihalli, Madhusudan >Subject: FW: util_ldap [Bug 29217] - Remove references to >calloc() and free() > &g

FW: util_ldap [Bug 29217] - Remove references to calloc()and free()

2004-06-11 Thread Mathihalli, Madhusudan
W: util_ldap [Bug 29217] - Remove references to >calloc()and free() > > > >-Original Message- >From: Tair-Shian Chou [mailto:[EMAIL PROTECTED] >Sent: Friday, June 11, 2004 6:38 PM >To: '[EMAIL PROTECTED]'; '[EMAIL PROTECTED]' >Subject: RE: util_

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-11 Thread William A. Rowe, Jr.
At 04:41 PM 6/11/2004, Brad Nicholes wrote: >I am sure that we can take advantage of what has been done in mod_ssl and >other places that have to mutex protect shared memory. It is actually working >great on NetWare at the moment but then we don't use shared memory and >we are multi-threaded only

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-11 Thread Brad Nicholes
>After speaking many times about this problem on #apache-modules, Paul >Querna said that it would be better to port mod_auth_ldap to >mod_authn_ldap, and do module caching auth, for all authentication >method. I am sure this way is really better and i agree with him. I know >he started this cac

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-11 Thread Matthieu Estrade
mod_ssl mutex are totally different. global mutex are used. Actually, the mutex is in the module_conf, so i think when apache fork childs, this mutex is no more valid, and each child will have a value for it. There is also in util_ldap.c, apr_thread_mutex_create(&st->mutex, APR_THREAD_MUTEX_DEFA

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-11 Thread Jim Jagielski
I could be completely confused here, but isn't this (a shared cache) already something we have a framework for in mod_ssl? Not that what's *in* mod_ssl (well, the *scache* files) is the solution for ldap, but the particulars design and logic flow certainly should work, right?

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-11 Thread Matthieu Estrade
Hi, I am not sure about what i will say, but i think all these mutex are broken. when i did that, i think i used apr_thread_mutex_create which use a pool (st->pool) which is the server pool. This function is to use in a child, not between all forked child. When it create the mutex, it do a apr_pc

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-11 Thread Greg Marr
At 12:33 PM 6/11/2004, William A. Rowe, Jr. wrote: The proper logic to add to a cache is wrlock test if exists again add element unlock because there is a race condition in the logic below rdlock test if the element exists >> race is here, prior to wrlocking, another thread may wrlock->inser

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-11 Thread William A. Rowe, Jr.
The proper logic to add to a cache is wrlock test if exists again add element unlock because there is a race condition in the logic below rdlock test if the element exists >> race is here, prior to wrlocking, another thread may wrlock->insert promote to wrlock insert unlock

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-11 Thread Graham Leggett
Brad Nicholes wrote: It appears to me that if it doesn't handle low memory situations or it is giving false positives, those are separate issues from pools vs. calloc/free. I still think we need to implement some better monitoring or logging code in the cache_mgr and enhance the cache-status pa

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-10 Thread Brad Nicholes
In fact, I don't think that these are shared locks at all #define LDAP_CACHE_LOCK_CREATE(p) \ if (!st->util_ldap_cache_lock) apr_thread_rwlock_create(&st->util_ldap_cache_lock, st->pool) which means that in the shared memory cache, it is highly likely that multiple processes could be alter

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-10 Thread Brad Nicholes
It appears to me that if it doesn't handle low memory situations or it is giving false positives, those are separate issues from pools vs. calloc/free. I still think we need to implement some better monitoring or logging code in the cache_mgr and enhance the cache-status pages so that we can tr

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-10 Thread Graham Leggett
Brad Nicholes wrote: Do we even know that there is a problem with this code? I haven't seen any memory leaks so far. I would hate to go to all of the work to redesign and rewrite the ldap_cache manager for little to no gain. It does not seem to handle the "we ran out of memory while trying to add

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-10 Thread Brad Nicholes
>The LDAP cache is a complex memory structure - each cache entry consists >of a struct with a whole lot of malloc'ed blocks. And there are >different kind of cache entries (bind cache, opcache). Deleting an entry >means walking through the struct and relying on a human writing code to >delete e

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-10 Thread Graham Leggett
Brad Nicholes wrote: I have considered using a pool per entry in other caching code that I have written just so that I could have much finer control over allocating and freeing the memory. But in the end what it really comes down to is malloc and free and if that is what you really want, then w

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-10 Thread Brad Nicholes
I have considered using a pool per entry in other caching code that I have written just so that I could have much finer control over allocating and freeing the memory. But in the end what it really comes down to is malloc and free and if that is what you really want, then why not just use mallo

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-10 Thread Graham Leggett
Brad Nicholes wrote: At least on NetWare, switching to pools would make the code much more complex. Rather than simply calling calloc and free in the same way that we are calling apr_rmm_calloc() and apr_rmm_free(), we would have to implement essentially the same model using pools and reslists.

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-10 Thread Brad Nicholes
I agree that the LDAP code needs to do more error checking and between the work that you did, along with the holes that I plugged previously and the shared memory fixes that Mathieu Estrade did, I think the code is much more robust than it has ever been. Many of our NetWare customers use auth_l

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-10 Thread Graham Leggett
Brad Nicholes wrote: I guess that is a possibility but I still don't understand what the problem is with using calloc() and free() for the ldap caching code. This seems to be a common thing to do when global memory needs to be allocated and deallocated constantly. To avoid having the memory gr

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-09 Thread Brad Nicholes
I guess that is a possibility but I still don't understand what the problem is with using calloc() and free() for the ldap caching code. This seems to be a common thing to do when global memory needs to be allocated and deallocated constantly. To avoid having the memory grow uncontrolably, you

Re: util_ldap [Bug 29217] - Remove references to calloc() and free()

2004-06-09 Thread Graham Leggett
Brad Nicholes wrote: But if you are allocating memory for cache entries that are constantly expiring and being purged, the pool will continue to grow until the server is restarted. The pool would end up with stale memory that the system has no way of reclaiming outside of restarting the server.