Hi all,
After several weeks working on a bunch of mem issues related to the
libgc based garbage collector, we've identified the following issue and
a possible solution (Dick already sent some workarounds to the list):
The libgc garbage collector has a really hard time identifying pointers
to objects since it "guesses" what is a pointer instead of actually
"knowing" by using data passed by the mono runtime.
It means something as simple as introducing a long on the stack (for
instance something like array = new int[1000000]) will block (forever)
the memory at address 1000000. Yes, as incredible as it sounds, it can
cause important mem problems on long living apps (typically servers).
(As a side note, this exact problem is present on sgen, since it also
scans the stack "conservatively").
A small improvement could be made in the current GC with little effort,
and is supplying more class refmaps to libgc.
Libgc is very hard to modify, it contains too many hacks and
optimizations that have made the code a nightmare to understand and
modify, so we don't find useful to make anything here beyond very small
patches.
That said, mono currently can provide reference bitmaps for objects,
it's a matter of providing the right descriptor to the garbage collector.
Libgc supports this kind of descriptors and mono already generates them
for the sgen gc, so it's just a matter of joining those together (which
should beeasy to do). This should improve a great number of scans in the
arking process, leaving only stacks and several minor objects without
precise marking. (Should become similar to the current sgen idea, where
stacks and other roots are scanned conservatively, although not compacting).
Attached is the sample code we use to reproduce the issue on 32 bit
based Linux/Mono systems.
Some notes about the test app below:
=======================================
the program accepts commands like gc, mem, exit, 2, or 1
2 n m creates n arrays of ints with m elements, and put them in an
arraylist. After the call completes, they are no longer referenced.
1 n m same, but waiting for a key press after each new array
gc n performs n gcs
exit exits
So, the case:
mono test.exe
2 2000000 70 creates 2 million int arrays of 70 elements
each (virtual goes up to 777MB)
gc 10 should free everything, but around 33MB
remain allocated acording to pmap:
...
bf4b5000 32K 0K 0K ---p [anon]
bfc9e000 88K 32K 28K rwxp [stack]
ffffe000 4K 0K 0K r-xp [vdso]
Total: 777820K 33852K 29336K
2 20 25000000 creates 20 int arrays of 25 million elements
each (2.7GB)
gc 10 now pmap shows everything is screwed up:
...
b7f2b000 8K 8K 8K rwxp /lib/ld-2.6.1.so
bf4b5000 32K 0K 0K ---p [anon]
bfc9e000 88K 32K 28K rwxp [stack]
ffffe000 4K 0K 0K r-xp [vdso]
Total: 2764356K 1696132K 1691616K
Trying with smaller sizes lets you see that segments are joined and
split, but seems that there is some inability to free everything.
======================================
Regards,
pablo
plain text document attachment (Program.cs)
using System;
using System.Collections;
namespace test
{
class Program
{
static void Main(string[] args)
{
WaitForEnter();
}
private static void WaitForEnter()
{
Console.WriteLine("Command:");
while (true)
{
Console.Write("> ");
string line = Console.ReadLine();
string[] args = line.Split(' ');
if (args.Length <= 0) continue;
switch (args[0].ToLower())
{
case "exit": return;
case "1": Case1(args); break;
case "2": Case2(args); break;
case "mem":
Console.WriteLine("Memory now: {0}", GC.GetTotalMemory(false));
break;
case "gc":
Gcs(args);
break;
default:
Console.WriteLine("Unknown command");
break;
}
}
}
private static void Gcs(string[] args)
{
int loop = (args.Length == 2) ? Int32.Parse(args[1]) : 1;
for (int i = 0; i < loop; ++i)
{
Console.WriteLine("Memory {1} now : {0}", GC.GetTotalMemory(false), i);
Console.WriteLine("Memory {1} after GC: {0}", GC.GetTotalMemory(true), i);
}
}
private const int * 1024;
private static void Case1(string[] args)
{
int loop = (args.Length >= 2) ? Int32.Parse(args[1]) : 5;
int size = (args.Length >= 3) ? Int32.Parse(args[2]) : 10 * OneMeg;
ArrayList container = new ArrayList();
for (int i = 0; i < loop; ++i)
{
int[] s1 = new int[size];
for (int j = 0; j < size; ++j)
{
s1[j] = j;
}
container.Add(s1);
Console.Write("Iteration {0}, press enter for next", i);
Console.ReadLine();
}
// Explicit in case it helps
container = null;
}
private static void Case2(string[] args)
{
int loop = (args.Length >= 2) ? Int32.Parse(args[1]) : 5;
int size = (args.Length >= 3) ? Int32.Parse(args[2]) : 10 * OneMeg;
ArrayList container = new ArrayList();
for (int i = 0; i < loop; ++i)
{
int[] s1 = new int[size];
for (int j = 0; j < size; ++j)
{
s1[j] = j;
}
container.Add(s1);
Console.WriteLine("Iteration {0}", i);
}
// Explicit in case it helps
container = null;
}
}
}
_______________________________________________
Mono-devel-list mailing list
Mono-devel-list@lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list
