In perl.git, the branch smoke-me/require-h-ph has been created
<http://perl5.git.perl.org/perl.git/commitdiff/70794f7b4fe6d2c8f69493d55274dba5e4a20f91?hp=0000000000000000000000000000000000000000>
at 70794f7b4fe6d2c8f69493d55274dba5e4a20f91 (commit)
- Log -----------------------------------------------------------------
commit 70794f7b4fe6d2c8f69493d55274dba5e4a20f91
Author: Father Chrysostomos <[email protected]>
Date: Fri Jun 29 00:50:30 2012 -0700
Make formats close over the right closure
This was brought up in ticket #113812.
Formats that are nested inside closures only work if invoked from
directly inside that closure. Calling the format from an inner sub
call wonât work.
Commit af41786fe57 stopped it from crashing, making it work as well
as 5.8, in that closed-over variables would be undefined, being
unavailable.
This commit adds a variation of the find_runcv function that can check
whether CvROOT matches an argument passed in. So we look not for the
current sub, but for the topmost sub on the call stack that is a clone
of the closure prototype that the formatâs CvOUTSIDE field points to.
M embed.fnc
M embed.h
M pad.c
M pp.h
M pp_ctl.c
M proto.h
M t/comp/form_scope.t
commit 5dff782da0fedb2c813f8a63e1a8df588bb6159e
Author: Father Chrysostomos <[email protected]>
Date: Thu Jun 28 23:12:23 2012 -0700
[perl #113812] Always use find_runcv when cloning a sub
A closure prototypeâs CvOUTSIDE pointer might have been modified if
its containing sub is freed first. When a sub is cloned, the enclos-
ing sub is always the currently-running sub (not so for formats).
So this commit makes subs always use find_runcv, the way they did
before 71f882da828.
So the closure logic which was needed for formats is now moved into an
else branch that is used only for them.
M pad.c
M t/op/closure.t
commit c5fb998ef860f7f66b2a91fa441d43fc2168dda0
Merge: 79c9643 01bfea8
Author: Father Chrysostomos <[email protected]>
Date: Fri Jun 29 00:22:25 2012 -0700
[Merge] CV-based slab allocator for ops
This branch uses per-CV slabs for ops, so that ops can all be freed
after compilation errors, fixing memory leaks and a crash.
See commit 8be227ab5e for how it works.
commit 01bfea8b32d3803399b0a0a7d1a0f3181d552e34
Author: Father Chrysostomos <[email protected]>
Date: Thu Jun 28 09:13:40 2012 -0700
Teach diagnostics.pm about %p
M lib/diagnostics.pm
M lib/diagnostics.t
commit 9b6decb00f21b648fca95404b606619175fbe932
Author: Father Chrysostomos <[email protected]>
Date: Thu Jun 28 09:10:56 2012 -0700
Increase $diagnostics::VERSION to 1.30
M lib/diagnostics.pm
commit efc859fb2266cae5156ec3e6efab319e797708a8
Author: Father Chrysostomos <[email protected]>
Date: Thu Jun 28 08:52:10 2012 -0700
perldiag: Document âSlab leaked from cvâ
M pod/perldiag.pod
commit 5cb52f3093bcf0de944e346eec0a2848f4b10752
Author: Father Chrysostomos <[email protected]>
Date: Wed Jun 27 22:40:48 2012 -0700
When reusing op slots, only zero as much as needed
If an op slot is reused for a smaller op, we only need to zero out the
space used for the op, not the whole slot.
M op.c
commit e73728817537f826e284beffd686bfacb0b604e9
Author: Father Chrysostomos <[email protected]>
Date: Wed Jun 27 22:38:25 2012 -0700
-DS should not invoke warnhook
I was using Perl_warn, both for its convenience, and because the line
numbers were extremely helpful in tracking bugs.
But it invokes the warnhook, if present, and also respects tied
STDERR. We should be using Perl_debug_log.
Changing this also avoids the need for /* diag_listed_as: SKIPME */
all over the place.
M op.c
commit 20429ba0ae8bff3c2402240c7b31f71b813be1dc
Author: Father Chrysostomos <[email protected]>
Date: Tue Jun 26 20:27:37 2012 -0700
op.c: Memory funcs need dVAR
M op.c
commit 07b97a33998a2cba764c68af8de883c1f4be3060
Author: Father Chrysostomos <[email protected]>
Date: Tue Jun 26 14:30:40 2012 -0700
fresh_perl.t: Skip #112312 tests on miniperl
They require attributes.pm.
M t/run/fresh_perl.t
commit 51c777cac7e1ad2a9910a6b33dc5e64d48281952
Author: Father Chrysostomos <[email protected]>
Date: Tue Jun 26 13:24:13 2012 -0700
Enlarge the last slot on an op slab to fit
For simplicity, op slots are never resized once they are allocated.
But they are reused after they are freed, if they are big enough.
When allocating the last op slot that will fit on a slab, we might as
well enlarge the slot to contain whatever space is left over, so this
slot, after being freed, can be reused for a larger op.
M op.c
commit be49fd9f9c5b1a5632988410e5c38c5558a9794c
Author: Father Chrysostomos <[email protected]>
Date: Tue Jun 26 10:08:58 2012 -0700
Define cv_forget_slab under PL_OP_SLAB_ALLOC
Instead of using #ifndef every time we call cv_forget_slab, just
define it as a no-op under PL_OP_SLAB_ALLOC.
M cv.h
M op.c
M pp_ctl.c
commit 7d82c9ce7c2bfa3a12c9a63cdbf316426bfee222
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 23 23:34:13 2012 -0700
Test perl #112312, crash on syntax error
I am having difficulty getting these tests to fail. They crash when
run standalone, but always pass when run via fresh_perl.t. Hopefully,
they will fail somewhere. :-)
Yes, fresh_perl.t does begin with this:
# ** DO NOT ADD ANY MORE TESTS HERE **
But t/comp/parser.t does not (and should not) use test.pl, so it is
very hard to test something like this.
Putting it here seemed slightly better than putting it in
its own file.
M t/run/fresh_perl.t
commit 42440e3c68e8bafb7e2a74763360939de0fad6be
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 23 18:24:52 2012 -0700
Test bug #111462, Safe + %^H + disallowed ops
M dist/Safe/t/safeops.t
commit 7e0f24e59bf40b2be3dc3bcf94ca24a10dac4cae
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 23 17:47:33 2012 -0700
perlhacktips: Update PERL_DEBUG_READONLY_OPS
M pod/perlhacktips.pod
commit 01995fcc964779c368050e699e454ea876ef63f6
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 23 17:44:23 2012 -0700
PERL_IMPLICIT_SYS can use the new slab allocator
M makedef.pl
M perl.h
commit 5cd2f99d38a9a2d64301a8f3e827e4c37a76d1e2
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 23 09:56:53 2012 -0700
Add slab allocation diagnostics (under perl -DS)
These proved extremely useful for getting this slab allocator to work.
We might as well leave them in place for future debugging.
M op.c
commit 56967202fdb3f5cc825b65c20306a5a04a8d73ad
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 23 09:56:07 2012 -0700
-DS option for slab allocation
M perl.c
M perl.h
M pod/perlrun.pod
commit 8be227ab5eaa23f2d21fd15f70190e494496dcbe
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 23 09:54:31 2012 -0700
CV-based slab allocation for ops
This addresses bugs #111462 and #112312 and part of #107000.
When a longjmp occurs during lexing, parsing or compilation, any ops
in C autos that are not referenced anywhere are leaked.
This commit introduces op slabs that are attached to the currently-
compiling CV. New ops are allocated on the slab. When an error
occurs and the CV is freed, any ops remaining are freed.
This is based on Nick Ing-Simmonsâ old experimental op slab implemen-
tation, but it had to be rewritten to work this way.
The old slab allocator has a pointer before each op that points to a
reference count stored at the beginning of the slab. Freed ops are
never reused. When the last op on a slab is freed, the slab itself is
freed. When a slab fills up, a new one is created.
To allow iteration through the slab to free everything, I had to have
two pointers; one points to the next item (op slot); the other points
to the slab, for accessing the reference count. Ops come in different
sizes, so adding sizeof(OP) to a pointer wonât work.
The old slab allocator puts the ops at the end of the slab first, the
idea being that the leaves are allocated first, so the order will be
cache-friendly as a result. I have preserved that order for a dif-
ferent reason: We donât need to store the size of the slab (slabs
vary in size; see below) if we can simply follow pointers to find
the last op.
I tried eliminating reference counts altogether, by having all ops
implicitly attached to PL_compcv when allocated and freed when the CV
is freed. That also allowed op_free to skip FreeOp altogether, free-
ing ops faster. But that doesnât work in those cases where ops need
to survive beyond their CVs; e.g., re-evals.
The CV also has to have a reference count on the slab. Sometimes the
first op created is immediately freed. If the reference count of
the slab reaches 0, then it will be freed with the CV still point-
ing to it.
CVs use the new CVf_SLABBED flag to indicate that the CV has a refer-
ence count on the slab. When this flag is set, the slab is accessible
via CvSTART when CvROOT is not set, or by subtracting two pointers
(2*sizeof(I32 *)) from CvROOT when it is set. I decided to sneak the
slab into CvSTART during compilation, because enlarging the xpvcv
struct by another pointer would make all CVs larger, even though this
patch only benefits few (programs using string eval).
When the CVf_SLABBED flag is set, the CV takes responsibility for
freeing the slab. If CvROOT is not set when the CV is freed or
undeffed, it is assumed that a compilation error has occurred, so the
op slab is traversed and all the ops are freed.
Under normal circumstances, the CV forgets about its slab (decrement-
ing the reference count) when the root is attached. So the slab ref-
erence counting that happens when ops are freed takes care of free-
ing the slab. In some cases, the CV is told to forget about the slab
(cv_forget_slab) precisely so that the ops can survive after the CV is
done away with.
Forgetting the slab when the root is attached is not strictly neces-
sary, but avoids potential problems with CvROOT being written over.
There is code all over the place, both in core and on CPAN, that does
things with CvROOT, so forgetting the slab makes things more robust
and avoids potential problems.
Since the CV takes ownership of its slab when flagged, that flag is
never copied when a CV is cloned, as one CV could free a slab that
another CV still points to, since forced freeing of ops ignores the
reference count (but asserts that it looks right).
To avoid slab fragmentation, freed ops are marked as freed and
attached to the slabâs freed chain (an idea stolen from DBM::Deep).
Those freed ops are reused when possible. I did consider not reusing
freed ops, but realised that would result in significantly higher mem-
ory using for programs with large âif (DEBUG) {...}â blocks.
SAVEFREEOP was slightly problematic. Sometimes it can cause an op to
be freed after its CV. If the CV has forcibly freed the ops on its
slab and the slab itself, then we will be fiddling with a freed slab.
Making SAVEFREEOP a no-op wonât help, as sometimes an op can be
savefreed when there is no compilation error, so the op would never
be freed. It holds a reference count on the slab, so the whole
slab would leak. So SAVEFREEOP now sets a special flag on the op
(->op_savefree). The forced freeing of ops after a compilation error
wonât free any ops thus marked.
Since many pieces of code create tiny subroutines consisting of only
a few ops, and since a huge slab would be quite a bit of baggage for
those to carry around, the first slab is always very small. To avoid
allocating too many slabs for a single CV, each subsequent slab is
twice the size of the previous.
Smartmatch expects to be able to allocate an op at run time, run it,
and then throw it away. For that to work the op is simply mallocked
when PL_compcv hasât been set up. So all slab-allocated ops are
marked as such (->op_slabbed), to distinguish them from mallocked ops.
All of this is kept under lock and key via #ifdef PERL_CORE, as it
should be completely transparent. If it isnât transparent, I would
consider that a bug.
I have left the old slab allocator (PL_OP_SLAB_ALLOC) in place, as
it is used by PERL_DEBUG_READONLY_OPS, which I am not about to
rewrite. :-)
Concerning the change from A to X for slab allocation functions:
Many times in the past, A has been used for functions that were
not intended to be public but were used for public macros. Since
PL_OP_SLAB_ALLOC is rarely used, it didnât make sense for Perl_Slab_*
to be API functions, since they were rarely actually available. To
avoid propagating this mistake further, they are now X.
M embed.fnc
M embed.h
M makedef.pl
M op.c
M op.h
M pad.c
M perl.h
M pp_ctl.c
M proto.h
M sv.c
commit 6e64f32b089786e5be480ddf137d427c003b791b
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 23 09:50:15 2012 -0700
Add OP_FREED op type
This is a dummy op type that should never be seen by any code except
op allocation code (to come).
So it is not in the usual list of opcodes, but is #defined outside the
range valid of opcodes.
M opnames.h
M regen/opcode.pl
commit bb38a9e0f528ce1ecffe4f8c3c76984148803932
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 23 09:48:34 2012 -0700
Flag ops that are on the savestack
This is to allow future commits to free dangling ops after errors.
If an op is on the savestack, then it is going to be freed by scope.c,
and op_free must not be called on it by anyone else.
So we flag such ops new.
M op.h
M scope.h
commit bfbc3ad9dfc28552739737eb87f09552732c0e95
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 23 09:43:33 2012 -0700
dump.c: Dump CVf_SLABBED
M dump.c
commit 58aef2b37ed9a7afbd27c2ee12895eeabf126984
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 23 09:43:10 2012 -0700
Add CVf_SLABBED flag
This will indicate that a CV has a reference count on, and ownership
of, a slab used for allocating ops.
M cv.h
commit 79c9643d871bd6cb08b3dd32a05ba534f13b890a
Author: Jesse Luehrs <[email protected]>
Date: Fri Jun 29 01:56:27 2012 -0500
fix 386a548 for fallback => undef
The default case for non-overloaded classes is fallback => 1, so saying
fallback => 1 on its own shouldn't enable overloading, but saying
fallback => undef on its own should (even though undef is the default
for overloaded classes).
M gv.c
M lib/overload.t
commit 56185adc32a821236821be07b37118f048b8bdad
Author: Jesse Luehrs <[email protected]>
Date: Fri Jun 29 00:40:42 2012 -0500
perldelta for 27c6f44
M pod/perldelta.pod
commit 386a54892347d3298ba7f1e98f207e36bc4f3424
Author: Jesse Luehrs <[email protected]>
Date: Fri Jun 29 00:38:04 2012 -0500
"use overload fallback => 0" should enable overloading [perl #113010]
This makes
package Foo;
use overload fallback => 0;
and
package Bar;
use overload '+' => \&add, fallback => 0;
behave identically when an operator other than '+' is used.
M gv.c
M lib/overload.t
commit af41786fe5732d5ec7932b946eec99a695ac6e43
Author: Father Chrysostomos <[email protected]>
Date: Thu Jun 28 20:28:09 2012 -0700
Formats in closures called outside closures â crash
If a format closing over lexical variables is defined inside a clo-
sure, it must only be called directly inside that closure, not from
any other eval, sub, or format.
Calling it from anywhere else started causing a crash in 5.10.0,
because the format would try to close over variables in the currently-
running sub, using padoffsets intended for a completely unrelated pad.
This commit stops it from crashing by checking whether the currently-
running sub is a clone of the formatâs outer sub (a closure proto-
type). If it is not, the outer closure prototype is used, resulting
in âVariable is not availableâ warnings.
This makes things work as well as they did in 5.8. Ideally, we should
search the call stack for the topmost clone of the formatâs outer sub;
but Iâm saving that for another commit.
M pad.c
M t/comp/form_scope.t
commit cc4aa37cf0381fa64942b6fb2777fcec81dbc43d
Author: Jesse Luehrs <[email protected]>
Date: Thu Jun 28 22:17:23 2012 -0500
fix storing objects with reftype REF [perl #113880]
M dist/Storable/Storable.xs
M dist/Storable/t/blessed.t
commit e013ada0e4202b68c9a3047a37a1e3805672e6ad
Author: Jesse Luehrs <[email protected]>
Date: Thu Jun 28 20:27:07 2012 -0500
perldelta updates for 6728836, d60d201, 82f9620, 7878705
M pod/perldelta.pod
commit e09ac076a1dab8e2c5712775f478fcfb61cb7eb3
Author: Father Chrysostomos <[email protected]>
Date: Thu Jun 28 16:31:17 2012 -0700
Donât let formats outlive their outer subs
This began crashing in 5.11.3:
sub foo {
sub bar {
my ($a,$b,$c,$d,$e,$f,$g,$h,$i,$j,$k,$l,$m,$n,$o,$p,$q,$r,$s,$x);
format =
@||||||
$x
.
}
}
undef *bar;
write;
(On some systems, you need more alphabet soup to make it crash.)
This commit (just the perly.y part shown) caused it to crash:
commit 421f30ed1e95009450bdc7905bf3433ee806ea4f
Author: Zefram <[email protected]>
Date: Tue Dec 15 11:48:31 2009 +0100
[perl #22977] Bug in format/write
diff --git a/perly.y b/perly.y
index 18e5875..a61a6b3 100644
--- a/perly.y
+++ b/perly.y
@@ -511,7 +511,9 @@ peg : PEG
;
format : FORMAT startformsub formname block
- { SvREFCNT_inc_simple_void(PL_compcv);
+ {
+ CV *fmtcv = PL_compcv;
+ SvREFCNT_inc_simple_void(PL_compcv);
#ifdef MAD
$$ = newFORM($2, $3, $4);
prepend_madprops($1->tk_mad, $$, 'F');
@@ -521,6 +523,10 @@ format : FORMAT startformsub formname block
newFORM($2, $3, $4);
$$ = (OP*)NULL;
#endif
+ if (CvOUTSIDE(fmtcv) && !CvUNIQUE(CvOUTSIDE(fmtcv))) {
+ SvREFCNT_inc_simple_void(fmtcv);
+ pad_add_anon((SV*)fmtcv, OP_NULL);
+ }
}
;
Unfortunately, adding the format to the pad like that (to allow
pad_fixup_inner_anons to fix up formats as well as subs) is proble-
matic. It causes the formatâs CvOUTSIDE to be weak. Since the for-
mat does not hold a reference count on its outer sub, that sub can be
freed before the format. When that happens, regular subs are fixed
up by having CvOUTSIDE change to point to the grandparent. If you
do that for formats, you run into a problem: Formats can be cloned
even when the outer sub is not running. Formats are cloned whenever
invoked *by name* via write. If CvOUTSIDE points to a different sub,
then closing over the scalars in specific pad offsets in that sub can
result in reading past the end of the pad. If you donât read past the
end of the pad, you are still making variables close over unrelated
variables, so the inner $x could close over an outer @y, etc. Subrou-
tines donât have that problem, as they can only be cloned when they
have an outer sub. (Even though the outer subâs prototype, if it is a
closure, might have been freed, the outer sub itself is still running
and referenced by the context stack.)
This commit changes the direction of the weak reference between an
outer subâs pad and an inner format, fixing the crash.
To do so, it has to store, not the format itself, but a weak RV point-
ing to the format, in the outer subâs pad.
M pad.c
M t/comp/form_scope.t
commit d142a0767037572181a72aa9b71be2ad3a6e1e98
Author: Father Chrysostomos <[email protected]>
Date: Thu Jun 28 09:24:46 2012 -0700
diagnostics.t: Restore test name removed by f0e510f
M lib/diagnostics.t
commit f9d4ccdc4830e30952502e4f7fa821786d59bfe0
Author: Father Chrysostomos <[email protected]>
Date: Wed Jun 27 21:32:08 2012 -0700
perldiag: Add missing regexp delims
M pod/perldiag.pod
-----------------------------------------------------------------------
--
Perl5 Master Repository
