I am happy to announce the first preliminary version of
neomimalloc!
neomimalloc is a wrapper to easily use the mimalloc allocator
developed by Microsoft Research, this allocator is having a great
success and it seems that the Rust community is thinking of using
it by default instead of jemalloc
(https://github.com/rust-lang/rust/pull/62340) so I think it is
very useful for the DLang community to have a working wrapper for
this modern allocator.
I created neomimalloc based on two principles that I consider
fundamental for this kind of wrapper:
1) Provide a complete solution easy to use, I think it is
particularly stupid to force the user to install a shared library
to use a third party allocator, doing so would be very difficult
to use in all DLang software.
So I realized a solution: the neomimalloc dub package compiles
the allocator and includes the symbols directly in its static
library, so the user only needs to add neomimalloc as a
dependency to his project in order to have mimalloc included in
his executable.
2) Being able to use the allocator with the high level API
present in phobos
(https://dlang.org/phobos/std_experimental_allocator.html), this
is really very useful because it allows users to use the
allocator as if it were included in the library D standard!
The usage is really the same, to do this I implemented the
wrapper following the instructions in the phobos documentation
(https://dlang.org/phobos/std_experimental_allocator_building_blocks.html)
3) In addition to the high-level interface, provide the bindings
with C to the user to allow the use directly and to provide those
APIs specific to mimalloc that could not be used by the
high-level interface
Currently there are some issues in this wrapper, mainly:
1) It only supports POSIX operating systems, this is not linked
to a lack of mimalloc but it is a problem of my package because
currently it is not able to compile mimalloc itself from dub.json
for Windows operating systems
This is mainly related to my total non knowledge of Windows
operating systems so if someone knows them better than me, please
help me solve the problem, pull requests are welcome!
2) Need for git during the compilation phase of the dub package,
this because dub does not support repository cloning recursively
(https://github.com/dlang/dub/issues/1044) so I have to do it
manually with a bad hack (clone recursively the neomimalloc
repository again in a temporary directory, very bad to do)
3) mimalloc provides many other implementations to allocate
memory in addition to the classic malloc (eg zalloc, calloc
etc.), it would be nice to support all allocators with high-level
interfaces, of course low-level bindings support all mimalllc
functions
The dub package allows 5 types of configuration:
1) standard: is the default configuration of neomimalloc, builda
mimalloc without any particular additional feature
2) secure: builds mimalloc in secure mode (adding guard pages,
randomized allocation, encrypted free lists, etc) to protect
against various heap vulnerabilities with a slight performance
penalty
3) override-malloc: like the standard configuration but
overriding the libc allocator symbols, for example the
core.stdc.stdlib.malloc function instead of calling the classic
libc allocator will use mimalloc (obviously this will also happen
with calloc, realloc etc)
4) override-malloc-secure: like the override-malloc but enable
secure mode
5) manual-link: link to mimalloc as shared library without builds
mimalloc, currently it is not implemented yet so it should not be
used
dub package: http://neomimalloc.dub.pm
neomimalloc git repository:
https://github.com/ErnyTech/neomimalloc.git
neomimalloc docs: https://neomimalloc.dpldocs.info
mimalloc git repository: https://github.com/microsoft/mimalloc.git
mimalloc docs: https://microsoft.github.io/mimalloc
mimalloc publication:
https://www.microsoft.com/en-us/research/uploads/prod/2019/06/mimalloc-tr-v1.pdf
