Re: yydestruct leaks

2001-02-07 Thread Alan Burlison

[EMAIL PROTECTED] wrote:

 Hmm, so this is (kind of) akin to the regcomp fix - it is the "new" stuff
 that is in yyl?val that should be free-d. And it is worse than that
 as yyl?val is just the topmost the parser state, so if I understand correctly
 it isn't only their current values, but also anything that is in the
 parser's stack (ysave-yyvs) at the time-of-death that needs to go.
 And all of those use the horrid yacc-ish YYSTYPE union, so we don't know
 what they are. Yacc did, it had a table which mapped tokens to types
 which it used to get union assigns right. But byacc does not put that info
 anywhere for run-time to use, so to get it right we would need to
 re-process perly.y and then look at the state stack as we popped it.

Yup - that's about the size of it.

 Yugh.
 
 The way I usually do this is make YYSTYPE an "object" or something
 like a Pascal variant record - which has a tag.

That was my idea - I just couldn't figure out any clean way of capturing
the type information.  If only byacc had a $$type variable as well as $$
etc...

 This would not be easy to fix for perl5.
 The best I can come up with is to make them all OP *, inventing
 special parse-time-only "ops" which can hold ival/pval/gvval values.
 
 Then yydestruct could just free the ops in yylval and yyvs[],
 freeing a gvalOP or pvalOP would do the right thing.
 
 Almost certainly far more than we want to do to the maint branch.

That seems workable, although as you say, far too radical for the maint
branch :-(

 The other way this mess is handled is to use a "slab allocator"
 e.g. GCC used an "obstack" - this allows all the memory allocated
 since one noted a "checkpoint" to be free-d.
 One could fake that idea by making malloc "plugable" and plugging
 it during parse to build a linked list or some such.

Well, that's kinda what we have with the scope stack, the problem is
that you don't know the type of the thing that needs freeing.

 The down side of that scheme is that auxillary allocators tend to
 upset Purify like tools almost as much as memory leaks do.

I've tried 2 approaches to this.  The first is to add "#ifdef PURIFY"
code to pp_ctl.c along the lines of the following:

S_doeval(...)
{
...
/* Flush any existing leaks to the log */
purify_new_leaks();
...
if (yyparse() == failed) {
...
/* Ignore any leaks */
purify_clear_leaks();
}
...
}

However I'm still suspicious of this because of the number of leaks that
only appear when S_doeval is somewhere in the stack trace.

The other approach is to postprocess the purify log and ignore anything
that has S_doeval or Perl_pp_entereval in the stack.  That's the
approach I'm currently using, but of course it ignores any real leaks
that coincidentally appear within an eval.  I think I'll try getting rid
of as many leaks as possible under this restricted regime - even with
this restriction, and with the bugs I've already fixed the test suite
contains 141 memory errors.

The truth of the matter is that I suspect eval and die will always leak
until it is re-architected in perl6 - whenever that might be.

Alan Burlison



Re: yydestruct leaks

2001-02-07 Thread nick

Alan Burlison [EMAIL PROTECTED] writes:
If an eval{} fails because of a snytax error, yydestroy is called on
leaving the eval scope.  Unfortunately it does this:

yyval   = ysave-oldyyval;
yylval  = ysave-oldyylval;

So anything that is in those 2 vars that hasn't made its way into the
parse tree is lost forever.  I know this is an old problem, but I've
been trying to think of a way to fix it.  Has anybody any suggestions? 

Hmm, so this is (kind of) akin to the regcomp fix - it is the "new" stuff
that is in yyl?val that should be free-d. And it is worse than that
as yyl?val is just the topmost the parser state, so if I understand correctly
it isn't only their current values, but also anything that is in the 
parser's stack (ysave-yyvs) at the time-of-death that needs to go.
And all of those use the horrid yacc-ish YYSTYPE union, so we don't know
what they are. Yacc did, it had a table which mapped tokens to types
which it used to get union assigns right. But byacc does not put that info
anywhere for run-time to use, so to get it right we would need to 
re-process perly.y and then look at the state stack as we popped it.

Yugh.

The way I usually do this is make YYSTYPE an "object" or something
like a Pascal variant record - which has a tag. 

This would not be easy to fix for perl5.
The best I can come up with is to make them all OP *, inventing 
special parse-time-only "ops" which can hold ival/pval/gvval values.

Then yydestruct could just free the ops in yylval and yyvs[],
freeing a gvalOP or pvalOP would do the right thing.

Almost certainly far more than we want to do to the maint branch.

The other way this mess is handled is to use a "slab allocator"
e.g. GCC used an "obstack" - this allows all the memory allocated 
since one noted a "checkpoint" to be free-d.
One could fake that idea by making malloc "plugable" and plugging 
it during parse to build a linked list or some such.

The down side of that scheme is that auxillary allocators tend to 
upset Purify like tools almost as much as memory leaks do.


-- 
Nick Ing-Simmons