Hi all, After several weeks working on a bunch of mem issues related to the libgc based garbage collector, we've identified the following issue and a possible solution (Dick already sent some workarounds to the list):
The libgc garbage collector has a really hard time identifying pointers to objects since it "guesses" what is a pointer instead of actually "knowing" by using data passed by the mono runtime. It means something as simple as introducing a long on the stack (for instance something like array = new int[1000000]) will block (forever) the memory at address 1000000. Yes, as incredible as it sounds, it can cause important mem problems on long living apps (typically servers). (As a side note, this exact problem is present on sgen, since it also scans the stack "conservatively"). A small improvement could be made in the current GC with little effort, and is supplying more class refmaps to libgc. Libgc is very hard to modify, it contains too many hacks and optimizations that have made the code a nightmare to understand and modify, so we don't find useful to make anything here beyond very small patches. That said, mono currently can provide reference bitmaps for objects, it's a matter of providing the right descriptor to the garbage collector. Libgc supports this kind of descriptors and mono already generates them for the sgen gc, so it's just a matter of joining those together (which should beeasy to do). This should improve a great number of scans in the arking process, leaving only stacks and several minor objects without precise marking. (Should become similar to the current sgen idea, where stacks and other roots are scanned conservatively, although not compacting). Attached is the sample code we use to reproduce the issue on 32 bit based Linux/Mono systems. Some notes about the test app below: ======================================= the program accepts commands like gc, mem, exit, 2, or 1 2 n m creates n arrays of ints with m elements, and put them in an arraylist. After the call completes, they are no longer referenced. 1 n m same, but waiting for a key press after each new array gc n performs n gcs exit exits So, the case: mono test.exe > 2 2000000 70 creates 2 million int arrays of 70 elements each (virtual goes up to 777MB) > gc 10 should free everything, but around 33MB remain allocated acording to pmap: ... bf4b5000 32K 0K 0K ---p [anon] bfc9e000 88K 32K 28K rwxp [stack] ffffe000 4K 0K 0K r-xp [vdso] Total: 777820K 33852K 29336K > 2 20 25000000 creates 20 int arrays of 25 million elements each (2.7GB) > gc 10 now pmap shows everything is screwed up: ... b7f2b000 8K 8K 8K rwxp /lib/ld-2.6.1.so bf4b5000 32K 0K 0K ---p [anon] bfc9e000 88K 32K 28K rwxp [stack] ffffe000 4K 0K 0K r-xp [vdso] Total: 2764356K 1696132K 1691616K Trying with smaller sizes lets you see that segments are joined and split, but seems that there is some inability to free everything. ====================================== Regards, pablo
using System; using System.Collections; namespace test { class Program { static void Main(string[] args) { WaitForEnter(); } private static void WaitForEnter() { Console.WriteLine("Command:"); while (true) { Console.Write("> "); string line = Console.ReadLine(); string[] args = line.Split(' '); if (args.Length <= 0) continue; switch (args[0].ToLower()) { case "exit": return; case "1": Case1(args); break; case "2": Case2(args); break; case "mem": Console.WriteLine("Memory now: {0}", GC.GetTotalMemory(false)); break; case "gc": Gcs(args); break; default: Console.WriteLine("Unknown command"); break; } } } private static void Gcs(string[] args) { int loop = (args.Length == 2) ? Int32.Parse(args[1]) : 1; for (int i = 0; i < loop; ++i) { Console.WriteLine("Memory {1} now : {0}", GC.GetTotalMemory(false), i); Console.WriteLine("Memory {1} after GC: {0}", GC.GetTotalMemory(true), i); } } private const int OneMeg = 1024 * 1024; private static void Case1(string[] args) { int loop = (args.Length >= 2) ? Int32.Parse(args[1]) : 5; int size = (args.Length >= 3) ? Int32.Parse(args[2]) : 10 * OneMeg; ArrayList container = new ArrayList(); for (int i = 0; i < loop; ++i) { int[] s1 = new int[size]; for (int j = 0; j < size; ++j) { s1[j] = j; } container.Add(s1); Console.Write("Iteration {0}, press enter for next", i); Console.ReadLine(); } // Explicit in case it helps container = null; } private static void Case2(string[] args) { int loop = (args.Length >= 2) ? Int32.Parse(args[1]) : 5; int size = (args.Length >= 3) ? Int32.Parse(args[2]) : 10 * OneMeg; ArrayList container = new ArrayList(); for (int i = 0; i < loop; ++i) { int[] s1 = new int[size]; for (int j = 0; j < size; ++j) { s1[j] = j; } container.Add(s1); Console.WriteLine("Iteration {0}", i); } // Explicit in case it helps container = null; } } }
_______________________________________________ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list