Hi guys, I have a question on how to free large arrays in D after
they're no longer needed.
Let's say I have this:
SomeKey[] getKeys() {
SomeKey[] n;
foreach(batching stuff...) {
SomeKey newstuff = db.select!(SomeKey[])(...); // gets around
6M of these from db
redis.send("newItems", newStuff.length);
n ~= newStuff;
destroy(newStuff);
}
return n;
}
void main() {
SomeKey[] myarray = getKeys();
ulong[string] mappedValues;
foreach(i, item; myArray) {
mappedValues[item.name] = i;
}
// No longer need myarray, and we need the memory back
destory(myarray);
GC.collect;
// Another memory intensive operation below
}
This seems to work for me to a point, but I notice my program
still holds a lot in memory even after certain memory intensive
arrays should have been collected.
Is this just a hit and miss with the GC or am I doing something
wrong? Uses around 40% less memory forcing the GC and cleaning
arrays but seems like a lot is left over, especially for threads
that have already finished(I've expect a thread to free all it's
memory). I'm not sharing anything across threads besides a config
struct.
This is a simplified version of my app. At the moment when at
around 9GB at it's peak instead of 19GB like before.