Hi there,
this is basically a follow-up to r1594729.
Investigating an out-of-memory issue, I discovered
another unfortunate allocation sequence / condition
under which the allocator reuses huge nodes for
small allocations (pseudo-code):
temp_pool = apr_pool_create();
while (many-times)
apr_pool_clear(temp_pool);
long_lived_pool = apr_pool_create();
apr_palloc(temp_pool, 1MB);
If no small nodes (i.e. <= 80 kB) is available, all
the long-living pools will reuse the 1 MB temp. node
for its initial allocation.
The problem is that those huge nodes are kept under
index 0, so the limiter code of r1594729 will not kick
in if the apr_allocator_t.free array is empty because
max_index is 0 in that case.
Also, it would be very nice to port both patches back
to 1.4+.
-- Stefan^2.
[[[
Follow-up on r1594729: Handle the case where the allocator
only contains large nodes (>80k or 20 pages). We don't
want to waste them on small allocations.
However, we will not limit the allocation size for larger
requests, i.e. those may be wasteful. They tend to either
get released quickly and or are followed by similar
allocations from the same pool - reducing the waste.
max_free_index will also limit the recycled block size.
Finally, fragmentation may become an issue with large nodes.
Hence, reusing them as they are may be the less risky option
overall.
* memory/unix/apr_pools.c
(allocator_alloc): Only reuse large nodes for relatively
large allocation requests.
]]]
[[[
Index: memory/unix/apr_pools.c
===================================================================
--- memory/unix/apr_pools.c (revision 1721274)
+++ memory/unix/apr_pools.c (working copy)
@@ -329,9 +329,10 @@
}
/* If we found nothing, seek the sink (at index 0), if
- * it is not empty.
+ * it is not empty. However, don't be to eager to reuse
+ * a large node for a small allocation.
*/
- else if (allocator->free[0]) {
+ else if (allocator->free[0] && (index > MAX_INDEX / 2)) {
#if APR_HAS_THREADS
if (allocator->mutex)
apr_thread_mutex_lock(allocator->mutex);
]]]
