Hi,
> Note that delta following involves patterns something like
>
> allocate (small) space for delta
> for i in (1..depth) {
> allocate large space for base
> allocate large space for result
> .. apply delta ..
> free large space for base
> free small space for delta
> }
>
> so if you have some stupid heap algorithm that doesn't try to merge and
> re-use free'd spaces very aggressively (because that takes CPU time!),
ptmalloc2 (in glibc) _per arena_ is basically best-fit. This is the
best known general strategy, but it certainly cannot be the best in
every case.
> you
> might have memory usage be horribly inflated by the heap having all those
> holes for all the objects that got free'd in the chain that don't get
> aggressively re-used.
It depends how large 'large' is -- if it exceeds the mmap() threshold
(settable with mallopt(M_MMAP_THRESHOLD, ...))
the 'large' spaces will be allocated with mmap() and won't cause
any internal fragmentation.
It might pay to experiment with this parameter if it is hard to
avoid the alloc/free large space sequence.
> Threaded memory allocators then make this worse by probably using totally
> different heaps for different threads (in order to avoid locking), so they
> will *all* have the fragmentation issue.
Indeed.
Could someone perhaps try ptmalloc3
(http://malloc.de/malloc/ptmalloc3-current.tar.gz) on this case?
Thanks,
Wolfram.
