On 25/09/2009 14:25, Jason Dagit wrote:
On Fri, Sep 25, 2009 at 4:34 AM, Simon Marlow <marlo...@gmail.com
<mailto:marlo...@gmail.com>> wrote:
On 24/09/2009 04:57, Jason Dagit wrote:
I ran this more with normal builds and just the GC statistics.
Here is
what we see:
unmodified darcs:
469 MB total memory in use (3 MB lost due to fragmentation)
Total time 26.51s ( 29.76s elapsed)
Productivity 85.6% of total user, 76.2% of total elapsed
469 MB total memory in use (3 MB lost due to fragmentation)
Total time 26.59s ( 32.47s elapsed)
Productivity 85.6% of total user, 70.1% of total elapsed
469 MB total memory in use (3 MB lost due to fragmentation)
Total time 26.54s ( 29.95s elapsed)
Productivity 85.5% of total user, 75.8% of total elapsed
With my patch applied:
554 MB total memory in use (4 MB lost due to fragmentation)
Total time 23.30s ( 31.56s elapsed)
Productivity 83.1% of total user, 61.3% of total elapsed
554 MB total memory in use (4 MB lost due to fragmentation)
Total time 22.85s ( 26.33s elapsed)
Productivity 82.9% of total user, 71.9% of total elapsed
554 MB total memory in use (4 MB lost due to fragmentation)
Total time 22.88s ( 26.38s elapsed)
Productivity 82.8% of total user, 71.8% of total elapsed
Now that the profiler is disabled the productivity with my
changes is
less, the run-time is maybe improved by 2-3 seconds, and the memory
usage has increased by almost 100 megs. I can only assume that the
profiling is interfering with my results a fair bit.
The profile graphs in your previous message showed a residency of
around ~30M before your patch, and ~6M after your patch. Which
seems like a worthwhile saving. I presume those graphs were from a
different test case? If not, then something very strange is going
on. If they are from a different test case, then do the numbers
stand up when using the non-profiled darcs?
Thanks for trying to help me understand the profiling.
My test case has not changed at all. The only thing I've been changing
is the darcs source code.
Do you have suggestions on determining which strange thing may be happening?
Well, if the heap profile is showing a ~30M residency and yet the
runtime is using 500MB, there's about 400MB unaccounted for.
Ah - so one problem is that the contents of ByteStrings are not
accounted for in the heap profile, for unpleasant technical reasons to
do with the way they're stored. Unfortunately it's not easy to fix,
because the GC doesn't retain information about which ByteStrings are
live, only that the 4KB block in which they live is still alive.
The underlying problem in your example may be fragmentation, due to the
way that ByteStrings are pinned and hence hold on to the whole 4KB block
in which they were allocated until they die. Duncan has been thinking
about how to improve the situation, but I'm not sure of the current
status - Duncan?
Cheers,
Simon
_______________________________________________
darcs-users mailing list
darcs-users@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-users
