> The thot plickens!
Unforunately...yes. :)
> My results for the above are:
> Interpreter version Time Data size
> CVS 8s 1320kB
> CVS + ML COW 20s 19172kB
> CVS + Grey1 5s 1884kB
> CVS + ML COW - reclaimable check ?? 5952kB
>
> Time is approximate time from entering N to reaching user
> input stage.
> Data size is measured when sleeping at the same point.
> So I saw only a small increase in memory size with grey cows,
> whereas you saw a much larger increase doing the same.
I realized something that might be useful. BASIC uses randomness in
constructing the maze. Changing "time I24" in merged_basic.pasm should
help.
In addition, I took the code in instructions.pasm, down near "I_INPUT",
and changed:
set S3, ""
read S3, 256
clone S4, S3
to be:
save I0
getout: print "\nTook"
time N6
sub N7, N6, N5
print N7
print " seconds. "
interpinfo I0, 1
print "A total of "
print I0
print " bytes were allocated\n"
interpinfo I0, 2
print "A total of "
print I0
print " DOD runs were made\n"
interpinfo I0, 3
print "A total of "
print I0
print " collection runs were made\n"
interpinfo I0, 10
print "Copying a total of "
print I0
print " bytes\n"
interpinfo I0, 5
print "There are "
print I0
print " active Buffer structs\n"
interpinfo I0, 7
print "There are "
print I0
print " total Buffer structs\n"
restore I0
set S3, ""
read S3, 256
clone S4, S3
time N5
This helps grab some stats.
> One culprit is the check in mem_allocate that does not
> bother to compact the pool if the saving is insufficient.
> Because you are under-estimating the reclaimable
> amount, this check actually prevents the pool from
> being compacted at all after a certain point during the
> above test. The last line above reflects the results from
> removing the two checks on reclaimable (threshold and size)
Yep. In one version, I made a guaranteed_reclaimable and
possibly_reclaimable, and updated them appropriately, using
possibly_reclaimable for the compaction check, and guaranteed_reclaimable
to calculate the size.
So, I removed the checks on reclaimable to *always* do a DOD and
compaction whenever we don't have enough memory. I then changed the
total_size calculation to the following, to avoid accumulation errors in
reclaimable causing me memory bloat:
{
struct Memory_Block *cur_block;
total_size = 0;
cur_block = pool->top_block;
while (cur_block) {
total_size += cur_block->size - cur_block->free;
cur_block = cur_block->prev;
}
}
Theoretically, modified-cow should have a non-increasing-bloat memory
usage now. I get the results:
modified-cow: 3088K
grey: 6124K
cvs: 1988K
Which show that while better, this rather low overestimate solution is
*still* quite a bit larger than cvs parrot. And it confuses me.
> However, even with that problem bypassed, there is still a
> major increase in memory usage. I put a few debugging
> statements in resources.c to track block allocs and frees;
> the value of interpreter->memory_allocated when wumpus
> reaches its idle state waiting for user input are as follows:
> CVS = 121,780
> CVS + ML COW = 18,556,096
> CVS + ML COW - reclaimable check = 2,197,359 (peak was about 6.2MB)
If I see what Parrot tells me as the pool size via
interpreter->memory_allocated, (see above pasm code) I get:
modified-cow: 600K
grey: 450K
cvs: 136K
So it's not *directly* related to the memory pool amounts, because
grey and cow have different orderings in memory_allocated and
windows-task-manager.
Another thing that might screw up our testing, is that you have three
extra size_t values in your buffer heaader (the outlawed ones). This
accounts for 12 bytes per header. The above pasm logging code shouws 88K
total buffers, which translates into about 1MB in use on the headers. So
while it doesn't explain all of grey's bloat versus cvs, it can explain
some of it.
> I suspect a combination of slower and fatter may make it
> hard to persuade Dan that we need cows?
Yeah. :)
Dan has given me permission to commit the code, if I'm confident that the
performance issues will be resolvable. At first, I thought they would
be..I figured it to be a small logic error somewhere causing the problems.
By using a possibly and guaranteed reclaimable, combined with a new
method for calculating pool size, I get CPU usage to drop to CVS levels,
with memory usage lower, but still higher than CVS.
It seems that while COW might save memory due to sharing, it also makes
when-to-collect logic break, and break our balance of collection
frequency and new-block-size, leading to an apparant memory usage
increase. I can't really think of any other cause.
Personally, I find that COW logic makes things a bit more complex, and
somewhat harder to debug. And it certainly requires some more discipline
to be sure you copy data before modifying it, etc. So while I've been
pushing for COW for a long time, if it turns out to be horribly broken in
memory usage, I'm going to have to sideline my work on it and continue
with other stuff. :|
Thanks,
Mike Lambert
