On Wed, 12 Dec 2007, David Miller wrote:
>
> I personally don't think it's unreasonable for GIT to have it's
> own customized allocator at least for certain object types.
Well, we actually already *do* have a customized allocator, but currently
only for the actual core "object descriptor" that really just has the SHA1
and object flags in it (and a few extra words depending on object type).
Those are critical for certain loads, and small too (so using the standard
allocator wasted a _lot_ of memory). In addition, they're fixed-size and
never free'd, so a specialized allocator really can do a lot better than
any general-purpose memory allocator ever could.
But the actual object *contents* are currently all allocated with whatever
the standard libc malloc/free allocator is that you compile for (or load
dynamically). Havign a specialized allocator for them is a much more
involved issue, exactly because we do have interesting allocation patterns
etc.
That said, at least those object allocations are all single-threaded (for
right now, at least), so even when git does multi-threaded stuff, the core
sha1_file.c stuff is always run under a single lock, and a simpler
allocator that doesn't care about threads is likely to be much better than
one that tries to have thread-local heaps etc.
I suspect that is what the google allocator does. It probably doesn't have
per-thread heaps, it just uses locking (and quite possibly things like
per-*size* heaps, which is much more memory-efficient and helps avoid some
of the fragmentation problems).
Locking is much slower than per-thread accesses, but it doesn't have the
issues with per-thread-fragmentation and all the problems with one thread
allocating and another one freeing.
Linus
