On Sunday, 22 March 2015 at 09:42:41 UTC, Ozan Süel wrote:
Hi!
I'm working on a Big Data project, where a huge amount of RAM
is needed. Using D I've run into a - let's called it - memory
leak. I tested this with following code:
foreach(i; 0..1000) {
int[] ints;
foreach(j; 0..1000) {
ints ~= uniform(0, 100);
}
}
Without the first foreach this code use only 220KB,
with the first foreach this code needs 2,5MB.
(220KB x 1000 is something around 2,5MB).
220KB x 1000 would be around 200MB.
But why is GC (Garbage Collector) not running? Following the
explanations in http://wiki.dlang.org/Memory_Management memory
usage should be something around 220KB.
The GC is running. 1000 x 1000 ints would be 4MB. That alone is
more than the 2.5MB you're observing.
I used GC.minimize, slow down the loop, replaced
uniform...doesn't work.
(By the way: After a while my LINUX server killed my big data
project because running out of RAM & SWAP space)
GC.minimize alone can't do anything if the memory isn't
collected. Use GC.collect(), too.
Also, if you know the final size of the array beforehand, you can
`reserve` it. That avoids a lot of copying (and garbage creation)
when appending.
int[] ints;
ints.reserve(1000); /* <- */
foreach(j; 0..1000) {
ints ~= uniform(0, 100);
}