Am 26.12.2016 um 10:31 schrieb Alexander Stohr:
Am 2016-12-25 um 21:42 schrieb Bernd Oppolzer:
Thank you for your feedback.
Thank you for your kind answers.
You're welcome; I'm happy to meet someone who is interested in my work :-)
BTW, I had to remove some sort of self check from the
Stanford Pascal runtime; it checked (before), that pointers only pointed
to the valid range of heap addresses; because the "traditional" heap
consisted
of one contiguous segment, this was very easy to implement. But I had to
remove this check, because pointers now can point to auto variables,
static variables, "new" alloc areas etc. as well.
It would still work for some applications. But they would be few.
The vast majority of real use projects probably tend to break those box.
So if it was an option with an enable switch, most would disable it.
Probably for that reason it would never be worth keeping it at all.
The Pascal documents say that they implemented it in a very basic manner,
just to be sufficient for the needs of the compiler; and that future
implementors
are free to replace the storage management by more sophisticated solutions.
The heap elements allocated by the compiler are - for example - all
freed, when
a certain block is completely compiled, that is: the internal lists of
definitions
are freed, because they are no longer needed. I kept the mark/release
logic,
because I wanted to keep this logic. The new/mark/release areas are
completely
different from the alloc/free areas. The whole mark/release area is
allocated
at startup and cannot be enlarged (4 MB minus stack at the moment, can
be configured). The alloc/free area is allocated on an as-needed base in
64 k chunks;
limited to ca. 8 to 10 MB due to address range limitations - may grow much
larger, if 31-bit addressing were possible.
The problem, why I had to add this to the Pascal runtime, was: the
original
Stanford Pascal runtime only had functions new, mark, and release.
Release
releases all storage which was required since the last mark call -
but there
is no invidual "free" of areas. But I wanted to port an application
to Stanford
Pascal which required invidiual allocs and frees (like C
malloc/free). I decided
to add alloc/free and leave new/mark/release untouched for the moment,
because it is used in the compiler.
So it was created more like a stack allocator. The allocations were
local to the
functions or context they were allocated in and at some waypoint
(exit/return)
all of them were released. Thats not a generic universal heap design
but rather
goes to the level where gaps of sometimes named but then unused items
increase over time
and on persistent would need some garbage collection (time loss!) any
now and then.
But for that design as above growth and shrinkage is determined by the
code path.
Under some conditions the growth might be very determined
but the shrinkage is always very fixed whilst beeing much rarer.
(I feel a little similarity to older stack rewinding concepts in C
exception/resume features.)
I see you did wise to keep those items out of yournew codes for the
project
whilst keeping it untouched for the moment in the existing codes that
dont interfere.
Do you see a good chance to use some larger existing code bases and
test suites for verifying the compiler?
Do you have some heap tracking functionality inside so that e.g. "1234
heap Bytes lost" it printed at exit?
Is there a some debug option for stack max size tracking?
Is there something for stack/heap object out of bounds writes/access?
(thinking of magic word fences in between, and of heap sanity checking)
The first test for the compiler is always the compiler itself (first and
second pass);
it should compile itself again and again and yield the same results.
Then I collected
over the time some 30 testcases, which cover different areas;
esspecially the new
features that I added. I am a big fan of test driven development, so I
often added new
statements and features which first lead to a compiler error, and then I
implemented
them, until they worked as the should. Now these test cases are kept for
regression testing.
For the LE heap management, which is a sort of addendum to the Pascal
runtime
(the compiler doesn't need it):
there are functions that give statistics on heap usage at the end of the
process or
at any point in time in between
there are functions that check the heap for integrity (same checks as
suggested by
the IBM paper - the LE heap management technology is a product of IBM
Watson
Research Center, see the presentation link some days ago)
I wrote a program to check for memory leaks (in ANSI C), which works
with the
"normal" LE heap management (as provided by IBM); you call this program
twice
at different points in time, and the program tells you, which areas have
been
allocated and not freed in the meantime; this was very helpful when
finding memory
leaks for my customers (insurance companies with PL/1 code base ... this
tool and
the effort that I had to understand the LE heap management gave me the
idea to
rewrite it in Pascal) - I believe, it should be possible to rewrite that
in Pascal, too.
in fact, it took me some weeks and very much tests to get the heap
management functions
running without errors (hopefully); there are many special cases which
need special
treatment (when combining adjacent free areas etc. and maintaining the
trees of
free areas). This was one of the most difficult pieces of software I
wrote in the last years.
Stack and heap max size is no problem; the values are collected
regularly (and printed
at process end, if you wish).
There are magic words etc. and several security measures regarding
overwrite of
heap control blocks etc., but with normal processing, errors will be
discovered at
free processing, only. And: not all heap areas are regularly checked at
every alloc
and free request. The LE system has a HEAPCHK option, which does exactly
that
(check the entire heap at every request). If needed, it would be easy to
add this, because
the function to check the heap is already available.
If you want more information, see the IBM presentation:
http://bernd-oppolzer.de/stackheap.pdf
or the Source code of the Heap Management:
http://bernd-oppolzer.de/paslibx.pas
or: feel free to ask further questions.
Kind regards
Bernd
Regards, Alex.
_______________________________________________
fpc-other maillist - fpc-other@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-other
_______________________________________________
fpc-other maillist - fpc-other@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-other
