Duncan Coutts wrote:
As some are aware some Wash modules cause ghc to take a very large
amount of memory. It also generates very large C and assembler files
(the .raw_s file for one module is nearly 50Mb). So unsurprisingly it
also makes gcc take a very large amount of memory.
Unfortunately for people with weaker machines these happen at the same
time. That is at the same time that gcc starts taking 300Mb, ghc is
still taking up 400Mb. Even on machines with 1Gb of ram this pushes
everything else out into swap.
Note that unless constrained, ghc will take even more than 400Mb to
build wash (I noticed it using over 750Mb). The Gentoo ebuild already
limits ghc to 400Mb on 64 bit machines (and 200Mb on 32bit ones).
What I was wondering is if ghc could do a major GC and free most of it's
memory back to the system just before it calls gcc to compile the .hc
code that ghc has generated. That way the memory spike of ghc and gcc
would not coincide and our machines would not be brought to a crawl.
Am I right in thinking that ghc's rts can free memory back to the system
when the heap pressure reduces (at least if it's doing a compacting GC)?
No, not at the moment. One thing we planned to do but never got around
to is to use madvise() to improve swapping behaviour when memory is
tight (see the thread that Bulat refferred to).
In this case if it can do so, it should be quite dramatic. It'd ought to
be able to go from 400Mb back down to just a few Mb or so.
Yes it ought to.
A related problem is that the block allocator is really stupid. It
makes no attempt to reduce fragmentation, and in fact freeing blocks is
O(n) because the freelist is kept sorted in address order. This isn't
usually an issue, although it has been reported to be noticeable with
very large residencies (500M+). It's on my list to fix at some point.
I mention this because freeing all that memory might not be possible if
it is highly fragmented.
Cheers,
Simon
_______________________________________________
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
