On 2/14/16 10:22 PM, Jon D wrote:
Is there a way to reserve capacity in associative arrays? In some
programs I've been writing I've been getting reasonable performance up
to about 10 million entries, but beyond that performance is impacted
considerably (say, 30 million or 50 million entries). GC stats (via the
"--DRT-gcopt=profile:1" option) indicate dramatic increases in gc time,
which I'm assuming comes from resizing the underlying hash table. I'm
guessing that by preallocating a large size the performance degradation
would not be quite so dramatic. The underlying implementation of
associative arrays appears to take an initial number of buckets, and
there's a private resize() method, but it's not clear if there's a
public way to use these.
There is not a public way to access these methods unfortunately.
It would be a good addition to druntime I believe.
Recently, I added a clear method to the AA, which does not reduce
capacity. So if you frequently build large AAs, and then throw them
away, you could instead reuse the memory.
I would caution to be sure of this cause, however, before thinking it
would solve the problem. The AA not only uses an array for buckets, but
allocates a memory location for each element as well. I'm often wrong
when I assume what the problem is when it comes to GC issues...
-Steve
