Author: Armin Rigo <[email protected]>
Branch:
Changeset: r84210:958642dc2cb6
Date: 2016-05-05 08:55 +0200
http://bitbucket.org/pypy/pypy/changeset/958642dc2cb6/
Log: hg merge gc-del-3
Add rgc.FinalizerQueue, documented in pypy/doc/discussion/finalizer-
order.rst. It is a more flexible way to make RPython finalizers.
This branch does not use it in pypy/, it just adds the new way while
keeping the old one valid too.
diff too long, truncating to 2000 out of 2685 lines
diff --git a/pypy/doc/discussion/finalizer-order.rst
b/pypy/doc/discussion/finalizer-order.rst
--- a/pypy/doc/discussion/finalizer-order.rst
+++ b/pypy/doc/discussion/finalizer-order.rst
@@ -1,19 +1,123 @@
-.. XXX armin, what do we do with this?
+Ordering finalizers in the MiniMark GC
+======================================
-Ordering finalizers in the SemiSpace GC
-=======================================
+RPython interface
+-----------------
-Goal
-----
+In RPython programs like PyPy, we need a fine-grained method of
+controlling the RPython- as well as the app-level ``__del__()``. To
+make it possible, the RPython interface is now the following one (from
+May 2016):
-After a collection, the SemiSpace GC should call the finalizers on
+* RPython objects can have ``__del__()``. These are called
+ immediately by the GC when the last reference to the object goes
+ away, like in CPython. However, the long-term goal is that all
+ ``__del__()`` methods should only contain simple enough code. If
+ they do, we call them "destructors". They can't use operations that
+ would resurrect the object, for example. Use the decorator
+ ``@rgc.must_be_light_finalizer`` to ensure they are destructors.
+
+* RPython-level ``__del__()`` that are not passing the destructor test
+ are supported for backward compatibility, but deprecated. The rest
+ of this document assumes that ``__del__()`` are all destructors.
+
+* For any more advanced usage --- in particular for any app-level
+ object with a __del__ --- we don't use the RPython-level
+ ``__del__()`` method. Instead we use
+ ``rgc.FinalizerController.register_finalizer()``. This allows us to
+ attach a finalizer method to the object, giving more control over
+ the ordering than just an RPython ``__del__()``.
+
+We try to consistently call ``__del__()`` a destructor, to distinguish
+it from a finalizer. A finalizer runs earlier, and in topological
+order; care must be taken that the object might still be reachable at
+this point if we're clever enough. A destructor on the other hand runs
+last; nothing can be done with the object any more.
+
+
+Destructors
+-----------
+
+A destructor is an RPython ``__del__()`` method that is called directly
+by the GC when there is no more reference to an object. Intended for
+objects that just need to free a block of raw memory or close a file.
+
+There are restrictions on the kind of code you can put in ``__del__()``,
+including all other functions called by it. These restrictions are
+checked. In particular you cannot access fields containing GC objects;
+and if you call an external C function, it must be a "safe" function
+(e.g. not releasing the GIL; use ``releasegil=False`` in
+``rffi.llexternal()``).
+
+If there are several objects with destructors that die during the same
+GC cycle, they are called in a completely random order --- but that
+should not matter because destructors cannot do much anyway.
+
+
+Register_finalizer
+------------------
+
+The interface for full finalizers is made with PyPy in mind, but should
+be generally useful.
+
+The idea is that you subclass the ``rgc.FinalizerQueue`` class::
+
+* You must give a class-level attribute ``base_class``, which is the
+ base class of all instances with a finalizer. (If you need
+ finalizers on several unrelated classes, you need several unrelated
+ ``FinalizerQueue`` subclasses.)
+
+* You override the ``finalizer_trigger()`` method; see below.
+
+Then you create one global (or space-specific) instance of this
+subclass; call it ``fin``. At runtime, you call
+``fin.register_finalizer(obj)`` for every instance ``obj`` that needs
+a finalizer. Each ``obj`` must be an instance of ``fin.base_class``,
+but not every such instance needs to have a finalizer registered;
+typically we try to register a finalizer on as few objects as possible
+(e.g. only if it is an object which has an app-level ``__del__()``
+method).
+
+After a major collection, the GC finds all objects ``obj`` on which a
+finalizer was registered and which are unreachable, and mark them as
+reachable again, as well as all objects they depend on. It then picks
+a topological ordering (breaking cycles randomly, if any) and enqueues
+the objects and their registered finalizer functions in that order, in
+a queue specific to the prebuilt ``fin`` instance. Finally, when the
+major collection is done, it calls ``fin.finalizer_trigger()``.
+
+This method ``finalizer_trigger()`` can either do some work directly,
+or delay it to be done later (e.g. between two bytecodes). If it does
+work directly, note that it cannot (directly or indirectly) cause the
+GIL to be released.
+
+To find the queued items, call ``fin.next_dead()`` repeatedly. It
+returns the next queued item, or ``None`` when the queue is empty.
+
+It is allowed in theory to cumulate several different
+``FinalizerQueue`` instances for objects of the same class, and
+(always in theory) the same ``obj`` could be registered several times
+in the same queue, or in several queues. This is not tested though.
+
+
+Ordering of finalizers
+----------------------
+
+After a collection, the MiniMark GC should call the finalizers on
*some* of the objects that have one and that have become unreachable.
Basically, if there is a reference chain from an object a to an object b
then it should not call the finalizer for b immediately, but just keep b
alive and try again to call its finalizer after the next collection.
-This basic idea fails when there are cycles. It's not a good idea to
+(Note that this creates rare but annoying issues as soon as the program
+creates chains of objects with finalizers more quickly than the rate at
+which major collections go (which is very slow). In August 2013 we tried
+instead to call all finalizers of all objects found unreachable at a major
+collection. That branch, ``gc-del``, was never merged. It is still
+unclear what the real consequences would be on programs in the wild.)
+
+The basic idea fails in the presence of cycles. It's not a good idea to
keep the objects alive forever or to never call any of the finalizers.
The model we came up with is that in this case, we could just call the
finalizer of one of the objects in the cycle -- but only, of course, if
@@ -33,6 +137,7 @@
detach the finalizer (so that it's not called more than once)
call the finalizer
+
Algorithm
---------
@@ -136,28 +241,8 @@
that doesn't change the state of an object, we don't follow its children
recursively.
-In practice, in the SemiSpace, Generation and Hybrid GCs, we can encode
-the 4 states with a single extra bit in the header:
-
- ===== ============= ======== ====================
- state is_forwarded? bit set? bit set in the copy?
- ===== ============= ======== ====================
- 0 no no n/a
- 1 no yes n/a
- 2 yes yes yes
- 3 yes whatever no
- ===== ============= ======== ====================
-
-So the loop above that does the transition from state 1 to state 2 is
-really just a copy(x) followed by scan_copied(). We must also clear the
-bit in the copy at the end, to clean up before the next collection
-(which means recursively bumping the state from 2 to 3 in the final
-loop).
-
-In the MiniMark GC, the objects don't move (apart from when they are
-copied out of the nursery), but we use the flag GCFLAG_VISITED to mark
-objects that survive, so we can also have a single extra bit for
-finalizers:
+In practice, in the MiniMark GCs, we can encode
+the 4 states with a combination of two bits in the header:
===== ============== ============================
state GCFLAG_VISITED GCFLAG_FINALIZATION_ORDERING
@@ -167,3 +252,8 @@
2 yes yes
3 yes no
===== ============== ============================
+
+So the loop above that does the transition from state 1 to state 2 is
+really just a recursive visit. We must also clear the
+FINALIZATION_ORDERING bit at the end (state 2 to state 3) to clean up
+before the next collection.
diff --git a/rpython/doc/rpython.rst b/rpython/doc/rpython.rst
--- a/rpython/doc/rpython.rst
+++ b/rpython/doc/rpython.rst
@@ -191,6 +191,12 @@
``__setitem__`` for slicing isn't supported. Additionally, using negative
indices for slicing is still not support, even when using ``__getslice__``.
+ Note that the destructor ``__del__`` should only contain `simple
+ operations`__; for any kind of more complex destructor, consider
+ using instead ``rpython.rlib.rgc.FinalizerQueue``.
+
+.. __: garbage_collection.html
+
This layout makes the number of types to take care about quite limited.
diff --git a/rpython/memory/gc/base.py b/rpython/memory/gc/base.py
--- a/rpython/memory/gc/base.py
+++ b/rpython/memory/gc/base.py
@@ -6,6 +6,7 @@
from rpython.memory.support import get_address_stack, get_address_deque
from rpython.memory.support import AddressDict, null_address_dict
from rpython.rtyper.lltypesystem.llmemory import NULL, raw_malloc_usage
+from rpython.rtyper.annlowlevel import cast_adr_to_nongc_instance
TYPEID_MAP = lltype.GcStruct('TYPEID_MAP', ('count', lltype.Signed),
('size', lltype.Signed),
@@ -36,8 +37,15 @@
def setup(self):
# all runtime mutable values' setup should happen here
# and in its overriden versions! for the benefit of test_transformed_gc
- self.finalizer_lock_count = 0
- self.run_finalizers = self.AddressDeque()
+ self.finalizer_lock = False
+ self.run_old_style_finalizers = self.AddressDeque()
+
+ def mark_finalizer_to_run(self, fq_index, obj):
+ if fq_index == -1: # backward compatibility with old-style finalizer
+ self.run_old_style_finalizers.append(obj)
+ return
+ handlers = self.finalizer_handlers()
+ self._adr2deque(handlers[fq_index].deque).append(obj)
def post_setup(self):
# More stuff that needs to be initialized when the GC is already
@@ -60,8 +68,9 @@
def set_query_functions(self, is_varsize, has_gcptr_in_varsize,
is_gcarrayofgcptr,
- getfinalizer,
- getlightfinalizer,
+ finalizer_handlers,
+ destructor_or_custom_trace,
+ is_old_style_finalizer,
offsets_to_gc_pointers,
fixed_size, varsize_item_sizes,
varsize_offset_to_variable_part,
@@ -74,8 +83,9 @@
fast_path_tracing,
has_gcptr,
cannot_pin):
- self.getfinalizer = getfinalizer
- self.getlightfinalizer = getlightfinalizer
+ self.finalizer_handlers = finalizer_handlers
+ self.destructor_or_custom_trace = destructor_or_custom_trace
+ self.is_old_style_finalizer = is_old_style_finalizer
self.is_varsize = is_varsize
self.has_gcptr_in_varsize = has_gcptr_in_varsize
self.is_gcarrayofgcptr = is_gcarrayofgcptr
@@ -136,8 +146,10 @@
the four malloc_[fixed,var]size[_clear]() functions.
"""
size = self.fixed_size(typeid)
- needs_finalizer = bool(self.getfinalizer(typeid))
- finalizer_is_light = bool(self.getlightfinalizer(typeid))
+ needs_finalizer = (bool(self.destructor_or_custom_trace(typeid))
+ and not self.has_custom_trace(typeid))
+ finalizer_is_light = (needs_finalizer and
+ not self.is_old_style_finalizer(typeid))
contains_weakptr = self.weakpointer_offset(typeid) >= 0
assert not (needs_finalizer and contains_weakptr)
if self.is_varsize(typeid):
@@ -323,9 +335,44 @@
callback2, attrname = _convert_callback_formats(callback) # :-/
setattr(self, attrname, arg)
self.root_walker.walk_roots(callback2, callback2, callback2)
- self.run_finalizers.foreach(callback, arg)
+ self.enum_pending_finalizers(callback, arg)
enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)'
+ def enum_pending_finalizers(self, callback, arg):
+ self.run_old_style_finalizers.foreach(callback, arg)
+ handlers = self.finalizer_handlers()
+ i = 0
+ while i < len(handlers):
+ self._adr2deque(handlers[i].deque).foreach(callback, arg)
+ i += 1
+ enum_pending_finalizers._annspecialcase_ = 'specialize:arg(1)'
+
+ def _copy_pending_finalizers_deque(self, deque, copy_fn):
+ tmp = self.AddressDeque()
+ while deque.non_empty():
+ obj = deque.popleft()
+ tmp.append(copy_fn(obj))
+ while tmp.non_empty():
+ deque.append(tmp.popleft())
+ tmp.delete()
+
+ def copy_pending_finalizers(self, copy_fn):
+ "NOTE: not very efficient, but only for SemiSpaceGC and subclasses"
+ self._copy_pending_finalizers_deque(
+ self.run_old_style_finalizers, copy_fn)
+ handlers = self.finalizer_handlers()
+ i = 0
+ while i < len(handlers):
+ h = handlers[i]
+ self._copy_pending_finalizers_deque(
+ self._adr2deque(h.deque), copy_fn)
+ i += 1
+
+ def call_destructor(self, obj):
+ destructor = self.destructor_or_custom_trace(self.get_type_id(obj))
+ ll_assert(bool(destructor), "no destructor found")
+ destructor(obj)
+
def debug_check_consistency(self):
"""To use after a collection. If self.DEBUG is set, this
enumerates all roots and traces all objects to check if we didn't
@@ -364,18 +411,25 @@
def debug_check_object(self, obj):
pass
+ def _adr2deque(self, adr):
+ return cast_adr_to_nongc_instance(self.AddressDeque, adr)
+
def execute_finalizers(self):
- self.finalizer_lock_count += 1
+ if self.finalizer_lock:
+ return # the outer invocation of execute_finalizers() will do it
+ self.finalizer_lock = True
try:
- while self.run_finalizers.non_empty():
- if self.finalizer_lock_count > 1:
- # the outer invocation of execute_finalizers() will do it
- break
- obj = self.run_finalizers.popleft()
- finalizer = self.getfinalizer(self.get_type_id(obj))
- finalizer(obj)
+ handlers = self.finalizer_handlers()
+ i = 0
+ while i < len(handlers):
+ if self._adr2deque(handlers[i].deque).non_empty():
+ handlers[i].trigger()
+ i += 1
+ while self.run_old_style_finalizers.non_empty():
+ obj = self.run_old_style_finalizers.popleft()
+ self.call_destructor(obj)
finally:
- self.finalizer_lock_count -= 1
+ self.finalizer_lock = False
class MovingGCBase(GCBase):
diff --git a/rpython/memory/gc/generation.py b/rpython/memory/gc/generation.py
--- a/rpython/memory/gc/generation.py
+++ b/rpython/memory/gc/generation.py
@@ -355,6 +355,7 @@
scan = beginning = self.free
self.collect_oldrefs_to_nursery()
self.collect_roots_in_nursery()
+ self.collect_young_objects_with_finalizers()
scan = self.scan_objects_just_copied_out_of_nursery(scan)
# at this point, all static and old objects have got their
# GCFLAG_NO_YOUNG_PTRS set again by trace_and_drag_out_of_nursery
@@ -422,6 +423,19 @@
if self.is_in_nursery(obj):
root.address[0] = self.copy(obj)
+ def collect_young_objects_with_finalizers(self):
+ # XXX always walk the whole 'objects_with_finalizers' list here
+ new = self.AddressDeque()
+ while self.objects_with_finalizers.non_empty():
+ obj = self.objects_with_finalizers.popleft()
+ fq_nr = self.objects_with_finalizers.popleft()
+ if self.is_in_nursery(obj):
+ obj = self.copy(obj)
+ new.append(obj)
+ new.append(fq_nr)
+ self.objects_with_finalizers.delete()
+ self.objects_with_finalizers = new
+
def scan_objects_just_copied_out_of_nursery(self, scan):
while scan < self.free:
curr = scan + self.size_gc_header()
diff --git a/rpython/memory/gc/incminimark.py b/rpython/memory/gc/incminimark.py
--- a/rpython/memory/gc/incminimark.py
+++ b/rpython/memory/gc/incminimark.py
@@ -372,10 +372,19 @@
self.gc_state = STATE_SCANNING
#
- # A list of all objects with finalizers (these are never young).
- self.objects_with_finalizers = self.AddressDeque()
- self.young_objects_with_light_finalizers = self.AddressStack()
- self.old_objects_with_light_finalizers = self.AddressStack()
+ # Two lists of all objects with finalizers. Actually they are lists
+ # of pairs (finalization_queue_nr, object). "probably young objects"
+ # are all traced and moved to the "old" list by the next minor
+ # collection.
+ self.probably_young_objects_with_finalizers = self.AddressDeque()
+ self.old_objects_with_finalizers = self.AddressDeque()
+ p = lltype.malloc(self._ADDRARRAY, 1, flavor='raw',
+ track_allocation=False)
+ self.singleaddr = llmemory.cast_ptr_to_adr(p)
+ #
+ # Two lists of all objects with destructors.
+ self.young_objects_with_destructors = self.AddressStack()
+ self.old_objects_with_destructors = self.AddressStack()
#
# Two lists of the objects with weakrefs. No weakref can be an
# old object weakly pointing to a young object: indeed, weakrefs
@@ -609,15 +618,18 @@
# If the object needs a finalizer, ask for a rawmalloc.
# The following check should be constant-folded.
if needs_finalizer and not is_finalizer_light:
+ # old-style finalizers only!
ll_assert(not contains_weakptr,
"'needs_finalizer' and 'contains_weakptr' both specified")
obj = self.external_malloc(typeid, 0, alloc_young=False)
- self.objects_with_finalizers.append(obj)
+ res = llmemory.cast_adr_to_ptr(obj, llmemory.GCREF)
+ self.register_finalizer(-1, res)
+ return res
#
# If totalsize is greater than nonlarge_max (which should never be
# the case in practice), ask for a rawmalloc. The following check
# should be constant-folded.
- elif rawtotalsize > self.nonlarge_max:
+ if rawtotalsize > self.nonlarge_max:
ll_assert(not contains_weakptr,
"'contains_weakptr' specified for a large object")
obj = self.external_malloc(typeid, 0, alloc_young=True)
@@ -641,13 +653,12 @@
obj = result + size_gc_header
self.init_gc_object(result, typeid, flags=0)
#
- # If it is a weakref or has a lightweight finalizer, record it
+ # If it is a weakref or has a lightweight destructor, record it
# (checks constant-folded).
- if is_finalizer_light:
- self.young_objects_with_light_finalizers.append(obj)
+ if needs_finalizer:
+ self.young_objects_with_destructors.append(obj)
if contains_weakptr:
self.young_objects_with_weakrefs.append(obj)
- #
return llmemory.cast_adr_to_ptr(obj, llmemory.GCREF)
@@ -851,6 +862,7 @@
collect_and_reserve._dont_inline_ = True
+ # XXX kill alloc_young and make it always True
def external_malloc(self, typeid, length, alloc_young):
"""Allocate a large object using the ArenaCollection or
raw_malloc(), possibly as an object with card marking enabled,
@@ -1566,6 +1578,13 @@
self.header(shadow).tid |= GCFLAG_VISITED
new_shadow_object_dict.setitem(obj, shadow)
+ def register_finalizer(self, fq_index, gcobj):
+ from rpython.rtyper.lltypesystem import rffi
+ obj = llmemory.cast_ptr_to_adr(gcobj)
+ fq_index = rffi.cast(llmemory.Address, fq_index)
+ self.probably_young_objects_with_finalizers.append(obj)
+ self.probably_young_objects_with_finalizers.append(fq_index)
+
# ----------
# Nursery collection
@@ -1633,6 +1652,11 @@
if self.rrc_enabled:
self.rrc_minor_collection_trace()
#
+ # visit the "probably young" objects with finalizers. They
+ # always all survive.
+ if self.probably_young_objects_with_finalizers.non_empty():
+ self.deal_with_young_objects_with_finalizers()
+ #
while True:
# If we are using card marking, do a partial trace of the arrays
# that are flagged with GCFLAG_CARDS_SET.
@@ -1658,8 +1682,8 @@
# weakrefs' targets.
if self.young_objects_with_weakrefs.non_empty():
self.invalidate_young_weakrefs()
- if self.young_objects_with_light_finalizers.non_empty():
- self.deal_with_young_objects_with_finalizers()
+ if self.young_objects_with_destructors.non_empty():
+ self.deal_with_young_objects_with_destructors()
#
# Clear this mapping. Without pinned objects we just clear the dict
# as all objects in the nursery are dragged out of the nursery and, if
@@ -2221,7 +2245,10 @@
if self.rrc_enabled:
self.rrc_major_collection_trace()
#
- if self.objects_with_finalizers.non_empty():
+ ll_assert(not (self.probably_young_objects_with_finalizers
+ .non_empty()),
+ "probably_young_objects_with_finalizers should be empty")
+ if self.old_objects_with_finalizers.non_empty():
self.deal_with_objects_with_finalizers()
elif self.old_objects_with_weakrefs.non_empty():
# Weakref support: clear the weak pointers to dying objects
@@ -2237,9 +2264,9 @@
self.more_objects_to_trace.delete()
#
- # Light finalizers
- if self.old_objects_with_light_finalizers.non_empty():
- self.deal_with_old_objects_with_finalizers()
+ # Destructors
+ if self.old_objects_with_destructors.non_empty():
+ self.deal_with_old_objects_with_destructors()
# objects_to_trace processed fully, can move on to sweeping
self.ac.mass_free_prepare()
self.start_free_rawmalloc_objects()
@@ -2408,7 +2435,7 @@
#
# If we are in an inner collection caused by a call to a finalizer,
# the 'run_finalizers' objects also need to be kept alive.
- self.run_finalizers.foreach(self._collect_obj, None)
+ self.enum_pending_finalizers(self._collect_obj, None)
def enumerate_all_roots(self, callback, arg):
self.prebuilt_root_objects.foreach(callback, arg)
@@ -2573,41 +2600,45 @@
# ----------
# Finalizers
- def deal_with_young_objects_with_finalizers(self):
- """ This is a much simpler version of dealing with finalizers
- and an optimization - we can reasonably assume that those finalizers
- don't do anything fancy and *just* call them. Among other things
+ def deal_with_young_objects_with_destructors(self):
+ """We can reasonably assume that destructors don't do
+ anything fancy and *just* call them. Among other things
they won't resurrect objects
"""
- while self.young_objects_with_light_finalizers.non_empty():
- obj = self.young_objects_with_light_finalizers.pop()
+ while self.young_objects_with_destructors.non_empty():
+ obj = self.young_objects_with_destructors.pop()
if not self.is_forwarded(obj):
- finalizer = self.getlightfinalizer(self.get_type_id(obj))
- ll_assert(bool(finalizer), "no light finalizer found")
- finalizer(obj)
+ self.call_destructor(obj)
else:
obj = self.get_forwarding_address(obj)
- self.old_objects_with_light_finalizers.append(obj)
+ self.old_objects_with_destructors.append(obj)
- def deal_with_old_objects_with_finalizers(self):
- """ This is a much simpler version of dealing with finalizers
- and an optimization - we can reasonably assume that those finalizers
- don't do anything fancy and *just* call them. Among other things
+ def deal_with_old_objects_with_destructors(self):
+ """We can reasonably assume that destructors don't do
+ anything fancy and *just* call them. Among other things
they won't resurrect objects
"""
new_objects = self.AddressStack()
- while self.old_objects_with_light_finalizers.non_empty():
- obj = self.old_objects_with_light_finalizers.pop()
+ while self.old_objects_with_destructors.non_empty():
+ obj = self.old_objects_with_destructors.pop()
if self.header(obj).tid & GCFLAG_VISITED:
# surviving
new_objects.append(obj)
else:
# dying
- finalizer = self.getlightfinalizer(self.get_type_id(obj))
- ll_assert(bool(finalizer), "no light finalizer found")
- finalizer(obj)
- self.old_objects_with_light_finalizers.delete()
- self.old_objects_with_light_finalizers = new_objects
+ self.call_destructor(obj)
+ self.old_objects_with_destructors.delete()
+ self.old_objects_with_destructors = new_objects
+
+ def deal_with_young_objects_with_finalizers(self):
+ while self.probably_young_objects_with_finalizers.non_empty():
+ obj = self.probably_young_objects_with_finalizers.popleft()
+ fq_nr = self.probably_young_objects_with_finalizers.popleft()
+ self.singleaddr.address[0] = obj
+ self._trace_drag_out1(self.singleaddr)
+ obj = self.singleaddr.address[0]
+ self.old_objects_with_finalizers.append(obj)
+ self.old_objects_with_finalizers.append(fq_nr)
def deal_with_objects_with_finalizers(self):
# Walk over list of objects with finalizers.
@@ -2620,14 +2651,17 @@
marked = self.AddressDeque()
pending = self.AddressStack()
self.tmpstack = self.AddressStack()
- while self.objects_with_finalizers.non_empty():
- x = self.objects_with_finalizers.popleft()
+ while self.old_objects_with_finalizers.non_empty():
+ x = self.old_objects_with_finalizers.popleft()
+ fq_nr = self.old_objects_with_finalizers.popleft()
ll_assert(self._finalization_state(x) != 1,
"bad finalization state 1")
if self.header(x).tid & GCFLAG_VISITED:
new_with_finalizer.append(x)
+ new_with_finalizer.append(fq_nr)
continue
marked.append(x)
+ marked.append(fq_nr)
pending.append(x)
while pending.non_empty():
y = pending.pop()
@@ -2647,22 +2681,26 @@
while marked.non_empty():
x = marked.popleft()
+ fq_nr = marked.popleft()
state = self._finalization_state(x)
ll_assert(state >= 2, "unexpected finalization state < 2")
if state == 2:
- self.run_finalizers.append(x)
+ from rpython.rtyper.lltypesystem import rffi
+ fq_index = rffi.cast(lltype.Signed, fq_nr)
+ self.mark_finalizer_to_run(fq_index, x)
# we must also fix the state from 2 to 3 here, otherwise
# we leave the GCFLAG_FINALIZATION_ORDERING bit behind
# which will confuse the next collection
self._recursively_bump_finalization_state_from_2_to_3(x)
else:
new_with_finalizer.append(x)
+ new_with_finalizer.append(fq_nr)
self.tmpstack.delete()
pending.delete()
marked.delete()
- self.objects_with_finalizers.delete()
- self.objects_with_finalizers = new_with_finalizer
+ self.old_objects_with_finalizers.delete()
+ self.old_objects_with_finalizers = new_with_finalizer
def _append_if_nonnull(pointer, stack):
stack.append(pointer.address[0])
@@ -2815,9 +2853,6 @@
self.rrc_o_list_old = self.AddressStack()
self.rrc_p_dict = self.AddressDict() # non-nursery keys only
self.rrc_p_dict_nurs = self.AddressDict() # nursery keys only
- p = lltype.malloc(self._ADDRARRAY, 1, flavor='raw',
- track_allocation=False)
- self.rrc_singleaddr = llmemory.cast_ptr_to_adr(p)
self.rrc_dealloc_trigger_callback = dealloc_trigger_callback
self.rrc_dealloc_pending = self.AddressStack()
self.rrc_enabled = True
@@ -2887,7 +2922,7 @@
self.rrc_p_dict_nurs.delete()
self.rrc_p_dict_nurs = self.AddressDict(length_estimate)
self.rrc_p_list_young.foreach(self._rrc_minor_trace,
- self.rrc_singleaddr)
+ self.singleaddr)
def _rrc_minor_trace(self, pyobject, singleaddr):
from rpython.rlib.rawrefcount import REFCNT_FROM_PYPY
diff --git a/rpython/memory/gc/minimark.py b/rpython/memory/gc/minimark.py
--- a/rpython/memory/gc/minimark.py
+++ b/rpython/memory/gc/minimark.py
@@ -153,6 +153,8 @@
# ^^^ prebuilt objects may have the flag GCFLAG_HAS_SHADOW;
# then they are one word longer, the extra word storing the hash.
+ _ADDRARRAY = lltype.Array(llmemory.Address, hints={'nolength': True})
+
# During a minor collection, the objects in the nursery that are
# moved outside are changed in-place: their header is replaced with
@@ -309,10 +311,19 @@
self.old_rawmalloced_objects = self.AddressStack()
self.rawmalloced_total_size = r_uint(0)
#
- # A list of all objects with finalizers (these are never young).
- self.objects_with_finalizers = self.AddressDeque()
- self.young_objects_with_light_finalizers = self.AddressStack()
- self.old_objects_with_light_finalizers = self.AddressStack()
+ # Two lists of all objects with finalizers. Actually they are lists
+ # of pairs (finalization_queue_nr, object). "probably young objects"
+ # are all traced and moved to the "old" list by the next minor
+ # collection.
+ self.probably_young_objects_with_finalizers = self.AddressDeque()
+ self.old_objects_with_finalizers = self.AddressDeque()
+ p = lltype.malloc(self._ADDRARRAY, 1, flavor='raw',
+ track_allocation=False)
+ self.singleaddr = llmemory.cast_ptr_to_adr(p)
+ #
+ # Two lists of all objects with destructors.
+ self.young_objects_with_destructors = self.AddressStack()
+ self.old_objects_with_destructors = self.AddressStack()
#
# Two lists of the objects with weakrefs. No weakref can be an
# old object weakly pointing to a young object: indeed, weakrefs
@@ -517,15 +528,18 @@
# If the object needs a finalizer, ask for a rawmalloc.
# The following check should be constant-folded.
if needs_finalizer and not is_finalizer_light:
+ # old-style finalizers only!
ll_assert(not contains_weakptr,
"'needs_finalizer' and 'contains_weakptr' both specified")
obj = self.external_malloc(typeid, 0, alloc_young=False)
- self.objects_with_finalizers.append(obj)
+ res = llmemory.cast_adr_to_ptr(obj, llmemory.GCREF)
+ self.register_finalizer(-1, res)
+ return res
#
# If totalsize is greater than nonlarge_max (which should never be
# the case in practice), ask for a rawmalloc. The following check
# should be constant-folded.
- elif rawtotalsize > self.nonlarge_max:
+ if rawtotalsize > self.nonlarge_max:
ll_assert(not contains_weakptr,
"'contains_weakptr' specified for a large object")
obj = self.external_malloc(typeid, 0, alloc_young=True)
@@ -547,14 +561,14 @@
# Build the object.
llarena.arena_reserve(result, totalsize)
obj = result + size_gc_header
- if is_finalizer_light:
- self.young_objects_with_light_finalizers.append(obj)
self.init_gc_object(result, typeid, flags=0)
- #
- # If it is a weakref, record it (check constant-folded).
- if contains_weakptr:
- self.young_objects_with_weakrefs.append(obj)
#
+ # If it is a weakref or has a lightweight destructor, record it
+ # (checks constant-folded).
+ if needs_finalizer:
+ self.young_objects_with_destructors.append(obj)
+ if contains_weakptr:
+ self.young_objects_with_weakrefs.append(obj)
return llmemory.cast_adr_to_ptr(obj, llmemory.GCREF)
@@ -676,6 +690,7 @@
collect_and_reserve._dont_inline_ = True
+ # XXX kill alloc_young and make it always True
def external_malloc(self, typeid, length, alloc_young):
"""Allocate a large object using the ArenaCollection or
raw_malloc(), possibly as an object with card marking enabled,
@@ -1241,6 +1256,13 @@
self.old_objects_with_cards_set.append(dest_addr)
dest_hdr.tid |= GCFLAG_CARDS_SET
+ def register_finalizer(self, fq_index, gcobj):
+ from rpython.rtyper.lltypesystem import rffi
+ obj = llmemory.cast_ptr_to_adr(gcobj)
+ fq_index = rffi.cast(llmemory.Address, fq_index)
+ self.probably_young_objects_with_finalizers.append(obj)
+ self.probably_young_objects_with_finalizers.append(fq_index)
+
# ----------
# Nursery collection
@@ -1264,6 +1286,11 @@
# 'old_objects_pointing_to_young'.
self.collect_roots_in_nursery()
#
+ # visit the "probably young" objects with finalizers. They
+ # always all survive.
+ if self.probably_young_objects_with_finalizers.non_empty():
+ self.deal_with_young_objects_with_finalizers()
+ #
while True:
# If we are using card marking, do a partial trace of the arrays
# that are flagged with GCFLAG_CARDS_SET.
@@ -1288,8 +1315,8 @@
# weakrefs' targets.
if self.young_objects_with_weakrefs.non_empty():
self.invalidate_young_weakrefs()
- if self.young_objects_with_light_finalizers.non_empty():
- self.deal_with_young_objects_with_finalizers()
+ if self.young_objects_with_destructors.non_empty():
+ self.deal_with_young_objects_with_destructors()
#
# Clear this mapping.
if self.nursery_objects_shadows.length() > 0:
@@ -1613,7 +1640,7 @@
# with a finalizer and all objects reachable from there (and also
# moves some objects from 'objects_with_finalizers' to
# 'run_finalizers').
- if self.objects_with_finalizers.non_empty():
+ if self.old_objects_with_finalizers.non_empty():
self.deal_with_objects_with_finalizers()
#
self.objects_to_trace.delete()
@@ -1621,8 +1648,8 @@
# Weakref support: clear the weak pointers to dying objects
if self.old_objects_with_weakrefs.non_empty():
self.invalidate_old_weakrefs()
- if self.old_objects_with_light_finalizers.non_empty():
- self.deal_with_old_objects_with_finalizers()
+ if self.old_objects_with_destructors.non_empty():
+ self.deal_with_old_objects_with_destructors()
#
# Walk all rawmalloced objects and free the ones that don't
@@ -1745,8 +1772,8 @@
#
# If we are in an inner collection caused by a call to a finalizer,
# the 'run_finalizers' objects also need to be kept alive.
- self.run_finalizers.foreach(self._collect_obj,
- self.objects_to_trace)
+ self.enum_pending_finalizers(self._collect_obj,
+ self.objects_to_trace)
def enumerate_all_roots(self, callback, arg):
self.prebuilt_root_objects.foreach(callback, arg)
@@ -1878,41 +1905,45 @@
# ----------
# Finalizers
- def deal_with_young_objects_with_finalizers(self):
- """ This is a much simpler version of dealing with finalizers
- and an optimization - we can reasonably assume that those finalizers
- don't do anything fancy and *just* call them. Among other things
+ def deal_with_young_objects_with_destructors(self):
+ """We can reasonably assume that destructors don't do
+ anything fancy and *just* call them. Among other things
they won't resurrect objects
"""
- while self.young_objects_with_light_finalizers.non_empty():
- obj = self.young_objects_with_light_finalizers.pop()
+ while self.young_objects_with_destructors.non_empty():
+ obj = self.young_objects_with_destructors.pop()
if not self.is_forwarded(obj):
- finalizer = self.getlightfinalizer(self.get_type_id(obj))
- ll_assert(bool(finalizer), "no light finalizer found")
- finalizer(obj)
+ self.call_destructor(obj)
else:
obj = self.get_forwarding_address(obj)
- self.old_objects_with_light_finalizers.append(obj)
+ self.old_objects_with_destructors.append(obj)
- def deal_with_old_objects_with_finalizers(self):
- """ This is a much simpler version of dealing with finalizers
- and an optimization - we can reasonably assume that those finalizers
- don't do anything fancy and *just* call them. Among other things
+ def deal_with_old_objects_with_destructors(self):
+ """We can reasonably assume that destructors don't do
+ anything fancy and *just* call them. Among other things
they won't resurrect objects
"""
new_objects = self.AddressStack()
- while self.old_objects_with_light_finalizers.non_empty():
- obj = self.old_objects_with_light_finalizers.pop()
+ while self.old_objects_with_destructors.non_empty():
+ obj = self.old_objects_with_destructors.pop()
if self.header(obj).tid & GCFLAG_VISITED:
# surviving
new_objects.append(obj)
else:
# dying
- finalizer = self.getlightfinalizer(self.get_type_id(obj))
- ll_assert(bool(finalizer), "no light finalizer found")
- finalizer(obj)
- self.old_objects_with_light_finalizers.delete()
- self.old_objects_with_light_finalizers = new_objects
+ self.call_destructor(obj)
+ self.old_objects_with_destructors.delete()
+ self.old_objects_with_destructors = new_objects
+
+ def deal_with_young_objects_with_finalizers(self):
+ while self.probably_young_objects_with_finalizers.non_empty():
+ obj = self.probably_young_objects_with_finalizers.popleft()
+ fq_nr = self.probably_young_objects_with_finalizers.popleft()
+ self.singleaddr.address[0] = obj
+ self._trace_drag_out1(self.singleaddr)
+ obj = self.singleaddr.address[0]
+ self.old_objects_with_finalizers.append(obj)
+ self.old_objects_with_finalizers.append(fq_nr)
def deal_with_objects_with_finalizers(self):
# Walk over list of objects with finalizers.
@@ -1925,14 +1956,17 @@
marked = self.AddressDeque()
pending = self.AddressStack()
self.tmpstack = self.AddressStack()
- while self.objects_with_finalizers.non_empty():
- x = self.objects_with_finalizers.popleft()
+ while self.old_objects_with_finalizers.non_empty():
+ x = self.old_objects_with_finalizers.popleft()
+ fq_nr = self.old_objects_with_finalizers.popleft()
ll_assert(self._finalization_state(x) != 1,
"bad finalization state 1")
if self.header(x).tid & GCFLAG_VISITED:
new_with_finalizer.append(x)
+ new_with_finalizer.append(fq_nr)
continue
marked.append(x)
+ marked.append(fq_nr)
pending.append(x)
while pending.non_empty():
y = pending.pop()
@@ -1946,22 +1980,26 @@
while marked.non_empty():
x = marked.popleft()
+ fq_nr = marked.popleft()
state = self._finalization_state(x)
ll_assert(state >= 2, "unexpected finalization state < 2")
if state == 2:
- self.run_finalizers.append(x)
+ from rpython.rtyper.lltypesystem import rffi
+ fq_index = rffi.cast(lltype.Signed, fq_nr)
+ self.mark_finalizer_to_run(fq_index, x)
# we must also fix the state from 2 to 3 here, otherwise
# we leave the GCFLAG_FINALIZATION_ORDERING bit behind
# which will confuse the next collection
self._recursively_bump_finalization_state_from_2_to_3(x)
else:
new_with_finalizer.append(x)
+ new_with_finalizer.append(fq_nr)
self.tmpstack.delete()
pending.delete()
marked.delete()
- self.objects_with_finalizers.delete()
- self.objects_with_finalizers = new_with_finalizer
+ self.old_objects_with_finalizers.delete()
+ self.old_objects_with_finalizers = new_with_finalizer
def _append_if_nonnull(pointer, stack):
stack.append(pointer.address[0])
diff --git a/rpython/memory/gc/semispace.py b/rpython/memory/gc/semispace.py
--- a/rpython/memory/gc/semispace.py
+++ b/rpython/memory/gc/semispace.py
@@ -111,7 +111,9 @@
# self.objects_with_light_finalizers.append(result + size_gc_header)
#else:
if has_finalizer:
+ from rpython.rtyper.lltypesystem import rffi
self.objects_with_finalizers.append(result + size_gc_header)
+ self.objects_with_finalizers.append(rffi.cast(llmemory.Address,
-1))
if contains_weakptr:
self.objects_with_weakrefs.append(result + size_gc_header)
return llmemory.cast_adr_to_ptr(result+size_gc_header, llmemory.GCREF)
@@ -149,6 +151,13 @@
else:
return False
+ def register_finalizer(self, fq_index, gcobj):
+ from rpython.rtyper.lltypesystem import rffi
+ obj = llmemory.cast_ptr_to_adr(gcobj)
+ fq_index = rffi.cast(llmemory.Address, fq_index)
+ self.objects_with_finalizers.append(obj)
+ self.objects_with_finalizers.append(fq_index)
+
def obtain_free_space(self, needed):
# a bit of tweaking to maximize the performance and minimize the
# amount of code in an inlined version of malloc_fixedsize_clear()
@@ -268,8 +277,7 @@
scan = self.free = tospace
self.starting_full_collect()
self.collect_roots()
- if self.run_finalizers.non_empty():
- self.update_run_finalizers()
+ self.copy_pending_finalizers(self.copy)
scan = self.scan_copied(scan)
if self.objects_with_light_finalizers.non_empty():
self.deal_with_objects_with_light_finalizers()
@@ -499,8 +507,7 @@
if self.surviving(obj):
new_objects.append(self.get_forwarding_address(obj))
else:
- finalizer = self.getfinalizer(self.get_type_id(obj))
- finalizer(obj)
+ self.call_destructor(obj)
self.objects_with_light_finalizers.delete()
self.objects_with_light_finalizers = new_objects
@@ -517,12 +524,15 @@
self.tmpstack = self.AddressStack()
while self.objects_with_finalizers.non_empty():
x = self.objects_with_finalizers.popleft()
+ fq_nr = self.objects_with_finalizers.popleft()
ll_assert(self._finalization_state(x) != 1,
"bad finalization state 1")
if self.surviving(x):
new_with_finalizer.append(self.get_forwarding_address(x))
+ new_with_finalizer.append(fq_nr)
continue
marked.append(x)
+ marked.append(fq_nr)
pending.append(x)
while pending.non_empty():
y = pending.pop()
@@ -537,17 +547,21 @@
while marked.non_empty():
x = marked.popleft()
+ fq_nr = marked.popleft()
state = self._finalization_state(x)
ll_assert(state >= 2, "unexpected finalization state < 2")
newx = self.get_forwarding_address(x)
if state == 2:
- self.run_finalizers.append(newx)
+ from rpython.rtyper.lltypesystem import rffi
+ fq_index = rffi.cast(lltype.Signed, fq_nr)
+ self.mark_finalizer_to_run(fq_index, newx)
# we must also fix the state from 2 to 3 here, otherwise
# we leave the GCFLAG_FINALIZATION_ORDERING bit behind
# which will confuse the next collection
self._recursively_bump_finalization_state_from_2_to_3(x)
else:
new_with_finalizer.append(newx)
+ new_with_finalizer.append(fq_nr)
self.tmpstack.delete()
pending.delete()
@@ -627,16 +641,6 @@
self.objects_with_weakrefs.delete()
self.objects_with_weakrefs = new_with_weakref
- def update_run_finalizers(self):
- # we are in an inner collection, caused by a finalizer
- # the run_finalizers objects need to be copied
- new_run_finalizer = self.AddressDeque()
- while self.run_finalizers.non_empty():
- obj = self.run_finalizers.popleft()
- new_run_finalizer.append(self.copy(obj))
- self.run_finalizers.delete()
- self.run_finalizers = new_run_finalizer
-
def _is_external(self, obj):
return (self.header(obj).tid & GCFLAG_EXTERNAL) != 0
diff --git a/rpython/memory/gc/test/test_direct.py
b/rpython/memory/gc/test/test_direct.py
--- a/rpython/memory/gc/test/test_direct.py
+++ b/rpython/memory/gc/test/test_direct.py
@@ -8,7 +8,7 @@
import py
from rpython.rtyper.lltypesystem import lltype, llmemory
-from rpython.memory.gctypelayout import TypeLayoutBuilder
+from rpython.memory.gctypelayout import TypeLayoutBuilder, FIN_HANDLER_ARRAY
from rpython.rlib.rarithmetic import LONG_BIT, is_valid_int
from rpython.memory.gc import minimark, incminimark
from rpython.memory.gctypelayout import zero_gc_pointers_inside,
zero_gc_pointers
@@ -84,7 +84,9 @@
self.gc.set_root_walker(self.rootwalker)
self.layoutbuilder = TypeLayoutBuilder(self.GCClass)
self.get_type_id = self.layoutbuilder.get_type_id
- self.layoutbuilder.initialize_gc_query_function(self.gc)
+ gcdata = self.layoutbuilder.initialize_gc_query_function(self.gc)
+ ll_handlers = lltype.malloc(FIN_HANDLER_ARRAY, 0, immortal=True)
+ gcdata.finalizer_handlers = llmemory.cast_ptr_to_adr(ll_handlers)
self.gc.setup()
def consider_constant(self, p):
diff --git a/rpython/memory/gctransform/framework.py
b/rpython/memory/gctransform/framework.py
--- a/rpython/memory/gctransform/framework.py
+++ b/rpython/memory/gctransform/framework.py
@@ -9,8 +9,10 @@
from rpython.memory import gctypelayout
from rpython.memory.gctransform.log import log
from rpython.memory.gctransform.support import get_rtti, ll_call_destructor
+from rpython.memory.gctransform.support import ll_report_finalizer_error
from rpython.memory.gctransform.transform import GCTransformer
from rpython.memory.gctypelayout import ll_weakref_deref, WEAKREF, WEAKREFPTR
+from rpython.memory.gctypelayout import FIN_TRIGGER_FUNC, FIN_HANDLER_ARRAY
from rpython.tool.sourcetools import func_with_new_name
from rpython.translator.backendopt import graphanalyze
from rpython.translator.backendopt.finalizer import FinalizerAnalyzer
@@ -181,8 +183,11 @@
gcdata.max_type_id = 13 # patched in finish()
gcdata.typeids_z = a_random_address # patched in finish()
gcdata.typeids_list = a_random_address # patched in finish()
+ gcdata.finalizer_handlers = a_random_address # patched in finish()
self.gcdata = gcdata
self.malloc_fnptr_cache = {}
+ self.finalizer_queue_indexes = {}
+ self.finalizer_handlers = []
gcdata.gc = GCClass(translator.config.translation, **GC_PARAMS)
root_walker = self.build_root_walker()
@@ -217,6 +222,7 @@
data_classdef.generalize_attr('max_type_id', annmodel.SomeInteger())
data_classdef.generalize_attr('typeids_z', SomeAddress())
data_classdef.generalize_attr('typeids_list', SomeAddress())
+ data_classdef.generalize_attr('finalizer_handlers', SomeAddress())
annhelper = annlowlevel.MixLevelHelperAnnotator(self.translator.rtyper)
@@ -555,6 +561,12 @@
[s_gc, s_typeid16],
s_gcref)
+ self.register_finalizer_ptr = getfn(GCClass.register_finalizer,
+ [s_gc,
+ annmodel.SomeInteger(),
+ s_gcref],
+ annmodel.s_None)
+
def create_custom_trace_funcs(self, gc, rtyper):
custom_trace_funcs = tuple(rtyper.custom_trace_funcs)
rtyper.custom_trace_funcs = custom_trace_funcs
@@ -681,6 +693,16 @@
ll_instance.inst_typeids_list=
llmemory.cast_ptr_to_adr(ll_typeids_list)
newgcdependencies.append(ll_typeids_list)
#
+ handlers = self.finalizer_handlers
+ ll_handlers = lltype.malloc(FIN_HANDLER_ARRAY, len(handlers),
+ immortal=True)
+ for i in range(len(handlers)):
+ ll_handlers[i].deque = handlers[i][0]
+ ll_handlers[i].trigger = handlers[i][1]
+ ll_instance.inst_finalizer_handlers = llmemory.cast_ptr_to_adr(
+ ll_handlers)
+ newgcdependencies.append(ll_handlers)
+ #
return newgcdependencies
def get_finish_tables(self):
@@ -772,10 +794,8 @@
info = self.layoutbuilder.get_info(type_id)
c_size = rmodel.inputconst(lltype.Signed, info.fixedsize)
fptrs = self.special_funcptr_for_type(TYPE)
- has_finalizer = "finalizer" in fptrs
- has_light_finalizer = "light_finalizer" in fptrs
- if has_light_finalizer:
- has_finalizer = True
+ has_finalizer = "destructor" in fptrs or "old_style_finalizer" in fptrs
+ has_light_finalizer = "destructor" in fptrs
c_has_finalizer = rmodel.inputconst(lltype.Bool, has_finalizer)
c_has_light_finalizer = rmodel.inputconst(lltype.Bool,
has_light_finalizer)
@@ -1498,6 +1518,60 @@
return None
return getattr(obj, '_hash_cache_', None)
+ def get_finalizer_queue_index(self, hop):
+ fq_tag = hop.spaceop.args[0].value
+ assert 'FinalizerQueue TAG' in fq_tag.expr
+ fq = fq_tag.default
+ try:
+ index = self.finalizer_queue_indexes[fq]
+ except KeyError:
+ index = len(self.finalizer_queue_indexes)
+ assert index == len(self.finalizer_handlers)
+ deque = self.gcdata.gc.AddressDeque()
+ #
+ def ll_finalizer_trigger():
+ try:
+ fq.finalizer_trigger()
+ except Exception as e:
+ ll_report_finalizer_error(e)
+ ll_trigger = self.annotate_finalizer(ll_finalizer_trigger, [],
+ lltype.Void)
+ def ll_next_dead():
+ if deque.non_empty():
+ return deque.popleft()
+ else:
+ return llmemory.NULL
+ ll_next_dead = self.annotate_finalizer(ll_next_dead, [],
+ llmemory.Address)
+ c_ll_next_dead = rmodel.inputconst(lltype.typeOf(ll_next_dead),
+ ll_next_dead)
+ #
+ s_deque =
self.translator.annotator.bookkeeper.immutablevalue(deque)
+ r_deque = self.translator.rtyper.getrepr(s_deque)
+ ll_deque = r_deque.convert_const(deque)
+ adr_deque = llmemory.cast_ptr_to_adr(ll_deque)
+ #
+ self.finalizer_handlers.append((adr_deque, ll_trigger,
+ c_ll_next_dead))
+ self.finalizer_queue_indexes[fq] = index
+ return index
+
+ def gct_gc_fq_register(self, hop):
+ index = self.get_finalizer_queue_index(hop)
+ c_index = rmodel.inputconst(lltype.Signed, index)
+ v_ptr = hop.spaceop.args[1]
+ v_ptr = hop.genop("cast_opaque_ptr", [v_ptr],
+ resulttype=llmemory.GCREF)
+ hop.genop("direct_call", [self.register_finalizer_ptr, self.c_const_gc,
+ c_index, v_ptr])
+
+ def gct_gc_fq_next_dead(self, hop):
+ index = self.get_finalizer_queue_index(hop)
+ c_ll_next_dead = self.finalizer_handlers[index][2]
+ v_adr = hop.genop("direct_call", [c_ll_next_dead],
+ resulttype=llmemory.Address)
+ hop.genop("cast_adr_to_ptr", [v_adr],
+ resultvar = hop.spaceop.result)
class TransformerLayoutBuilder(gctypelayout.TypeLayoutBuilder):
@@ -1513,22 +1587,18 @@
self.translator = translator
super(TransformerLayoutBuilder, self).__init__(GCClass, lltype2vtable)
- def has_finalizer(self, TYPE):
+ def has_destructor(self, TYPE):
rtti = get_rtti(TYPE)
return rtti is not None and getattr(rtti._obj, 'destructor_funcptr',
None)
- def has_light_finalizer(self, TYPE):
- fptrs = self.special_funcptr_for_type(TYPE)
- return "light_finalizer" in fptrs
-
def has_custom_trace(self, TYPE):
rtti = get_rtti(TYPE)
return rtti is not None and getattr(rtti._obj, 'custom_trace_funcptr',
None)
- def make_finalizer_funcptr_for_type(self, TYPE):
- if not self.has_finalizer(TYPE):
+ def make_destructor_funcptr_for_type(self, TYPE):
+ if not self.has_destructor(TYPE):
return None, False
rtti = get_rtti(TYPE)
destrptr = rtti._obj.destructor_funcptr
diff --git a/rpython/memory/gctransform/support.py
b/rpython/memory/gctransform/support.py
--- a/rpython/memory/gctransform/support.py
+++ b/rpython/memory/gctransform/support.py
@@ -89,3 +89,11 @@
write(2, " ignoring it\n")
except:
pass
+
+def ll_report_finalizer_error(e):
+ try:
+ write(2, "triggering finalizers raised an exception ")
+ write(2, str(e))
+ write(2, " ignoring it\n")
+ except:
+ pass
diff --git a/rpython/memory/gctypelayout.py b/rpython/memory/gctypelayout.py
--- a/rpython/memory/gctypelayout.py
+++ b/rpython/memory/gctypelayout.py
@@ -17,16 +17,17 @@
OFFSETS_TO_GC_PTR = lltype.Array(lltype.Signed)
- # A custom tracer (CT), enumerates the addresses that contain GCREFs.
- # It is called with the object as first argument, and the previous
- # returned address (or NULL the first time) as the second argument.
- FINALIZER_FUNC = lltype.FuncType([llmemory.Address], lltype.Void)
- FINALIZER = lltype.Ptr(FINALIZER_FUNC)
+ # A CUSTOM_FUNC is either a destructor, or a custom tracer.
+ # A destructor is called when the object is about to be freed.
+ # A custom tracer (CT) enumerates the addresses that contain GCREFs.
+ # Both are called with the address of the object as only argument.
+ CUSTOM_FUNC = lltype.FuncType([llmemory.Address], lltype.Void)
+ CUSTOM_FUNC_PTR = lltype.Ptr(CUSTOM_FUNC)
# structure describing the layout of a typeid
TYPE_INFO = lltype.Struct("type_info",
("infobits", lltype.Signed), # combination of the T_xxx consts
- ("finalizer", FINALIZER),
+ ("customfunc", CUSTOM_FUNC_PTR),
("fixedsize", lltype.Signed),
("ofstoptrs", lltype.Ptr(OFFSETS_TO_GC_PTR)),
hints={'immutable': True},
@@ -80,16 +81,18 @@
def q_cannot_pin(self, typeid):
typeinfo = self.get(typeid)
ANY = (T_HAS_GCPTR | T_IS_WEAKREF)
- return (typeinfo.infobits & ANY) != 0 or bool(typeinfo.finalizer)
+ return (typeinfo.infobits & ANY) != 0 or bool(typeinfo.customfunc)
- def q_finalizer(self, typeid):
- return self.get(typeid).finalizer
+ def q_finalizer_handlers(self):
+ adr = self.finalizer_handlers # set from framework.py or gcwrapper.py
+ return llmemory.cast_adr_to_ptr(adr, lltype.Ptr(FIN_HANDLER_ARRAY))
- def q_light_finalizer(self, typeid):
+ def q_destructor_or_custom_trace(self, typeid):
+ return self.get(typeid).customfunc
+
+ def q_is_old_style_finalizer(self, typeid):
typeinfo = self.get(typeid)
- if typeinfo.infobits & T_HAS_LIGHTWEIGHT_FINALIZER:
- return typeinfo.finalizer
- return lltype.nullptr(GCData.FINALIZER_FUNC)
+ return (typeinfo.infobits & T_HAS_OLDSTYLE_FINALIZER) != 0
def q_offsets_to_gc_pointers(self, typeid):
return self.get(typeid).ofstoptrs
@@ -141,8 +144,9 @@
self.q_is_varsize,
self.q_has_gcptr_in_varsize,
self.q_is_gcarrayofgcptr,
- self.q_finalizer,
- self.q_light_finalizer,
+ self.q_finalizer_handlers,
+ self.q_destructor_or_custom_trace,
+ self.q_is_old_style_finalizer,
self.q_offsets_to_gc_pointers,
self.q_fixed_size,
self.q_varsize_item_sizes,
@@ -170,7 +174,7 @@
T_IS_WEAKREF = 0x080000
T_IS_RPYTHON_INSTANCE = 0x100000 # the type is a subclass of OBJECT
T_HAS_CUSTOM_TRACE = 0x200000
-T_HAS_LIGHTWEIGHT_FINALIZER = 0x400000
+T_HAS_OLDSTYLE_FINALIZER = 0x400000
T_HAS_GCPTR = 0x1000000
T_KEY_MASK = intmask(0xFE000000) # bug detection only
T_KEY_VALUE = intmask(0x5A000000) # bug detection only
@@ -199,11 +203,11 @@
#
fptrs = builder.special_funcptr_for_type(TYPE)
if fptrs:
- if "finalizer" in fptrs:
- info.finalizer = fptrs["finalizer"]
- if "light_finalizer" in fptrs:
- info.finalizer = fptrs["light_finalizer"]
- infobits |= T_HAS_LIGHTWEIGHT_FINALIZER
+ if "destructor" in fptrs:
+ info.customfunc = fptrs["destructor"]
+ if "old_style_finalizer" in fptrs:
+ info.customfunc = fptrs["old_style_finalizer"]
+ infobits |= T_HAS_OLDSTYLE_FINALIZER
#
if not TYPE._is_varsize():
info.fixedsize = llarena.round_up_for_allocation(
@@ -373,21 +377,21 @@
def special_funcptr_for_type(self, TYPE):
if TYPE in self._special_funcptrs:
return self._special_funcptrs[TYPE]
- fptr1, is_lightweight = self.make_finalizer_funcptr_for_type(TYPE)
+ fptr1, is_lightweight = self.make_destructor_funcptr_for_type(TYPE)
fptr2 = self.make_custom_trace_funcptr_for_type(TYPE)
result = {}
if fptr1:
if is_lightweight:
- result["light_finalizer"] = fptr1
+ result["destructor"] = fptr1
else:
- result["finalizer"] = fptr1
+ result["old_style_finalizer"] = fptr1
if fptr2:
result["custom_trace"] = fptr2
self._special_funcptrs[TYPE] = result
return result
- def make_finalizer_funcptr_for_type(self, TYPE):
- # must be overridden for proper finalizer support
+ def make_destructor_funcptr_for_type(self, TYPE):
+ # must be overridden for proper destructor support
return None, False
def make_custom_trace_funcptr_for_type(self, TYPE):
@@ -546,3 +550,9 @@
link = lltype.malloc(WEAKREF, immortal=True)
link.weakptr = llmemory.cast_ptr_to_adr(targetptr)
return link
+
+########## finalizers ##########
+
+FIN_TRIGGER_FUNC = lltype.FuncType([], lltype.Void)
+FIN_HANDLER_ARRAY = lltype.Array(('deque', llmemory.Address),
+ ('trigger', lltype.Ptr(FIN_TRIGGER_FUNC)))
diff --git a/rpython/memory/gcwrapper.py b/rpython/memory/gcwrapper.py
--- a/rpython/memory/gcwrapper.py
+++ b/rpython/memory/gcwrapper.py
@@ -1,7 +1,7 @@
from rpython.translator.backendopt.finalizer import FinalizerAnalyzer
from rpython.rtyper.lltypesystem import lltype, llmemory, llheap
-from rpython.rtyper import llinterp
-from rpython.rtyper.annlowlevel import llhelper
+from rpython.rtyper import llinterp, rclass
+from rpython.rtyper.annlowlevel import llhelper, cast_nongc_instance_to_adr
from rpython.memory import gctypelayout
from rpython.flowspace.model import Constant
@@ -15,6 +15,7 @@
chunk_size = 10,
translated_to_c = False,
**GC_PARAMS)
+ self.translator = translator
self.gc.set_root_walker(LLInterpRootWalker(self))
self.gc.DEBUG = True
self.llinterp = llinterp
@@ -30,6 +31,11 @@
self.llinterp)
self.get_type_id = layoutbuilder.get_type_id
gcdata = layoutbuilder.initialize_gc_query_function(self.gc)
+ self.gcdata = gcdata
+
+ self.finalizer_queue_indexes = {}
+ self.finalizer_handlers = []
+ self.update_finalizer_handlers()
constants = collect_constants(flowgraphs)
for obj in constants:
@@ -187,6 +193,55 @@
def thread_run(self):
pass
+ def _get_finalizer_trigger(self, fq):
+ graph = self.translator._graphof(fq.finalizer_trigger.im_func)
+ def ll_trigger():
+ try:
+ self.llinterp.eval_graph(graph, [None], recursive=True)
+ except llinterp.LLException:
+ raise RuntimeError(
+ "finalizer_trigger() raised an exception, shouldn't
happen")
+ return ll_trigger
+
+ def update_finalizer_handlers(self):
+ handlers = self.finalizer_handlers
+ ll_handlers = lltype.malloc(gctypelayout.FIN_HANDLER_ARRAY,
+ len(handlers), immortal=True)
+ for i in range(len(handlers)):
+ fq, deque = handlers[i]
+ ll_handlers[i].deque = cast_nongc_instance_to_adr(deque)
+ ll_handlers[i].trigger = llhelper(
+ lltype.Ptr(gctypelayout.FIN_TRIGGER_FUNC),
+ self._get_finalizer_trigger(fq))
+ self.gcdata.finalizer_handlers = llmemory.cast_ptr_to_adr(ll_handlers)
+
+ def get_finalizer_queue_index(self, fq_tag):
+ assert 'FinalizerQueue TAG' in fq_tag.expr
+ fq = fq_tag.default
+ try:
+ index = self.finalizer_queue_indexes[fq]
+ except KeyError:
+ index = len(self.finalizer_handlers)
+ self.finalizer_queue_indexes[fq] = index
+ deque = self.gc.AddressDeque()
+ self.finalizer_handlers.append((fq, deque))
+ self.update_finalizer_handlers()
+ return index
+
+ def gc_fq_next_dead(self, fq_tag):
+ index = self.get_finalizer_queue_index(fq_tag)
+ deque = self.finalizer_handlers[index][1]
+ if deque.non_empty():
+ obj = deque.popleft()
+ else:
+ obj = llmemory.NULL
+ return llmemory.cast_adr_to_ptr(obj, rclass.OBJECTPTR)
+
+ def gc_fq_register(self, fq_tag, ptr):
+ index = self.get_finalizer_queue_index(fq_tag)
+ ptr = lltype.cast_opaque_ptr(llmemory.GCREF, ptr)
+ self.gc.register_finalizer(index, ptr)
+
# ____________________________________________________________
class LLInterpRootWalker:
@@ -228,7 +283,7 @@
self.llinterp = llinterp
super(DirectRunLayoutBuilder, self).__init__(GCClass, lltype2vtable)
- def make_finalizer_funcptr_for_type(self, TYPE):
+ def make_destructor_funcptr_for_type(self, TYPE):
from rpython.memory.gctransform.support import get_rtti
rtti = get_rtti(TYPE)
if rtti is not None and hasattr(rtti._obj, 'destructor_funcptr'):
@@ -239,15 +294,17 @@
return None, False
t = self.llinterp.typer.annotator.translator
- light = not FinalizerAnalyzer(t).analyze_light_finalizer(destrgraph)
- def ll_finalizer(addr):
+ is_light = not FinalizerAnalyzer(t).analyze_light_finalizer(destrgraph)
+
+ def ll_destructor(addr):
try:
v = llmemory.cast_adr_to_ptr(addr, DESTR_ARG)
self.llinterp.eval_graph(destrgraph, [v], recursive=True)
except llinterp.LLException:
raise RuntimeError(
- "a finalizer raised an exception, shouldn't happen")
- return llhelper(gctypelayout.GCData.FINALIZER, ll_finalizer), light
+ "a destructor raised an exception, shouldn't happen")
+ return (llhelper(gctypelayout.GCData.CUSTOM_FUNC_PTR, ll_destructor),
+ is_light)
def make_custom_trace_funcptr_for_type(self, TYPE):
from rpython.memory.gctransform.support import get_rtti
diff --git a/rpython/memory/support.py b/rpython/memory/support.py
--- a/rpython/memory/support.py
+++ b/rpython/memory/support.py
@@ -2,6 +2,9 @@
from rpython.rlib.objectmodel import free_non_gc_object, we_are_translated
from rpython.rlib.debug import ll_assert
from rpython.tool.identity_dict import identity_dict
+from rpython.rtyper.rclass import NONGCOBJECTPTR
+from rpython.rtyper.annlowlevel import cast_nongc_instance_to_base_ptr
+from rpython.rtyper.annlowlevel import cast_base_ptr_to_nongc_instance
def mangle_hash(i):
@@ -292,6 +295,9 @@
cur = next
free_non_gc_object(self)
+ def _was_freed(self):
+ return False # otherwise, the __class__ changes
+
cache[chunk_size] = AddressDeque
return AddressDeque
diff --git a/rpython/memory/test/gc_test_base.py
b/rpython/memory/test/gc_test_base.py
--- a/rpython/memory/test/gc_test_base.py
+++ b/rpython/memory/test/gc_test_base.py
@@ -128,7 +128,7 @@
assert res == concat(100)
#assert simulator.current_size - curr < 16000 * INT_SIZE / 4
- def test_finalizer(self):
+ def test_destructor(self):
class B(object):
pass
b = B()
@@ -152,6 +152,98 @@
res = self.interpret(f, [5])
assert res == 6
+ def test_old_style_finalizer(self):
+ class B(object):
+ pass
+ b = B()
+ b.nextid = 0
+ b.num_deleted = 0
+ class A(object):
+ def __init__(self):
+ self.id = b.nextid
+ b.nextid += 1
+ def __del__(self):
+ llop.gc__collect(lltype.Void)
+ b.num_deleted += 1
+ def f(x):
+ a = A()
+ i = 0
+ while i < x:
+ i += 1
+ a = A()
+ llop.gc__collect(lltype.Void)
+ llop.gc__collect(lltype.Void)
+ return b.num_deleted
+ res = self.interpret(f, [5])
+ assert res == 6
+
+ def test_finalizer(self):
+ class B(object):
+ pass
+ b = B()
+ b.nextid = 0
+ b.num_deleted = 0
+ class A(object):
+ def __init__(self):
+ self.id = b.nextid
+ b.nextid += 1
+ fq.register_finalizer(self)
+ class FQ(rgc.FinalizerQueue):
+ Class = A
+ def finalizer_trigger(self):
+ while self.next_dead() is not None:
+ b.num_deleted += 1
+ fq = FQ()
+ def f(x):
+ a = A()
+ i = 0
+ while i < x:
+ i += 1
+ a = A()
+ a = None
+ llop.gc__collect(lltype.Void)
+ llop.gc__collect(lltype.Void)
+ return b.num_deleted
+ res = self.interpret(f, [5])
+ assert res == 6
+
+ def test_finalizer_delaying_next_dead(self):
+ class B(object):
+ pass
+ b = B()
+ b.nextid = 0
+ class A(object):
+ def __init__(self):
+ self.id = b.nextid
+ b.nextid += 1
+ fq.register_finalizer(self)
+ class FQ(rgc.FinalizerQueue):
+ Class = A
+ def finalizer_trigger(self):
+ b.triggered += 1
+ fq = FQ()
+ def g(): # indirection to avoid leaking the result for too long
+ A()
+ def f(x):
+ b.triggered = 0
+ g()
+ i = 0
+ while i < x:
+ i += 1
+ g()
+ llop.gc__collect(lltype.Void)
+ llop.gc__collect(lltype.Void)
+ assert b.triggered > 0
+ g(); g() # two more
+ llop.gc__collect(lltype.Void)
+ llop.gc__collect(lltype.Void)
+ num_deleted = 0
+ while fq.next_dead() is not None:
+ num_deleted += 1
+ return num_deleted + 1000 * b.triggered
+ res = self.interpret(f, [5])
+ assert res in (3008, 4008, 5008), "res == %d" % (res,)
+
def test_finalizer_calls_malloc(self):
class B(object):
pass
@@ -162,18 +254,27 @@
def __init__(self):
self.id = b.nextid
b.nextid += 1
- def __del__(self):
- b.num_deleted += 1
- C()
+ fq.register_finalizer(self)
class C(A):
- def __del__(self):
- b.num_deleted += 1
+ pass
+ class FQ(rgc.FinalizerQueue):
+ Class = A
+ def finalizer_trigger(self):
+ while True:
+ a = self.next_dead()
+ if a is None:
+ break
+ b.num_deleted += 1
+ if not isinstance(a, C):
+ C()
+ fq = FQ()
def f(x):
a = A()
i = 0
while i < x:
i += 1
a = A()
+ a = None
llop.gc__collect(lltype.Void)
llop.gc__collect(lltype.Void)
return b.num_deleted
@@ -190,15 +291,21 @@
def __init__(self):
self.id = b.nextid
b.nextid += 1
- def __del__(self):
- b.num_deleted += 1
- llop.gc__collect(lltype.Void)
+ fq.register_finalizer(self)
+ class FQ(rgc.FinalizerQueue):
+ Class = A
+ def finalizer_trigger(self):
+ while self.next_dead() is not None:
+ b.num_deleted += 1
+ llop.gc__collect(lltype.Void)
+ fq = FQ()
def f(x):
a = A()
i = 0
while i < x:
i += 1
a = A()
+ a = None
llop.gc__collect(lltype.Void)
llop.gc__collect(lltype.Void)
return b.num_deleted
@@ -215,20 +322,29 @@
def __init__(self):
self.id = b.nextid
b.nextid += 1
- def __del__(self):
- b.num_deleted += 1
- b.a = self
+ fq.register_finalizer(self)
+ class FQ(rgc.FinalizerQueue):
+ Class = A
+ def finalizer_trigger(self):
+ while True:
+ a = self.next_dead()
+ if a is None:
+ break
+ b.num_deleted += 1
+ b.a = a
+ fq = FQ()
def f(x):
a = A()
i = 0
while i < x:
i += 1
a = A()
+ a = None
llop.gc__collect(lltype.Void)
llop.gc__collect(lltype.Void)
aid = b.a.id
b.a = None
- # check that __del__ is not called again
+ # check that finalizer_trigger() is not called again
llop.gc__collect(lltype.Void)
llop.gc__collect(lltype.Void)
return b.num_deleted * 10 + aid + 100 * (b.a is None)
@@ -290,7 +406,7 @@
res = self.interpret(f, [])
assert res
- def test_weakref_to_object_with_finalizer(self):
+ def test_weakref_to_object_with_destructor(self):
import weakref
class A(object):
count = 0
@@ -310,6 +426,32 @@
res = self.interpret(f, [])
assert res
+ def test_weakref_to_object_with_finalizer(self):
+ import weakref
+ class A(object):
+ count = 0
+ a = A()
+ class B(object):
+ pass
+ class FQ(rgc.FinalizerQueue):
+ Class = B
+ def finalizer_trigger(self):
+ while self.next_dead() is not None:
+ a.count += 1
+ fq = FQ()
+ def g():
+ b = B()
+ fq.register_finalizer(b)
+ return weakref.ref(b)
+ def f():
+ ref = g()
+ llop.gc__collect(lltype.Void)
+ llop.gc__collect(lltype.Void)
+ result = a.count == 1 and (ref() is None)
+ return result
+ res = self.interpret(f, [])
+ assert res
+
def test_bug_1(self):
import weakref
class B(object):
@@ -329,23 +471,32 @@
res = self.interpret(f, [])
assert res
- def test_cycle_with_weakref_and_del(self):
+ def test_cycle_with_weakref_and_finalizer(self):
import weakref
class A(object):
count = 0
a = A()
class B(object):
- def __del__(self):
- # when __del__ is called, the weakref to c should be dead
- if self.ref() is None:
- a.count += 10 # ok
- else:
- a.count = 666 # not ok
+ pass
+ class FQ(rgc.FinalizerQueue):
+ Class = B
+ def finalizer_trigger(self):
+ while True:
+ b = self.next_dead()
+ if b is None:
+ break
+ # when we are here, the weakref to c should be dead
+ if b.ref() is None:
+ a.count += 10 # ok
+ else:
+ a.count = 666 # not ok
+ fq = FQ()
class C(object):
pass
def g():
c = C()
c.b = B()
+ fq.register_finalizer(c.b)
ref = weakref.ref(c)
c.b.ref = ref
return ref
@@ -365,23 +516,32 @@
a = A()
expected_invalid = self.WREF_IS_INVALID_BEFORE_DEL_IS_CALLED
class B(object):
- def __del__(self):
- # when __del__ is called, the weakref to myself is still valid
+ pass
+ class FQ(rgc.FinalizerQueue):
+ Class = B
+ def finalizer_trigger(self):
+ # when we are here, the weakref to myself is still valid
# in RPython with most GCs. However, this can lead to strange
# bugs with incminimark. https://bugs.pypy.org/issue1687
# So with incminimark, we expect the opposite.
- if expected_invalid:
- if self.ref() is None:
- a.count += 10 # ok
+ while True:
+ b = self.next_dead()
+ if b is None:
+ break
+ if expected_invalid:
+ if b.ref() is None:
+ a.count += 10 # ok
+ else:
+ a.count = 666 # not ok
else:
- a.count = 666 # not ok
- else:
- if self.ref() is self:
- a.count += 10 # ok
- else:
- a.count = 666 # not ok
+ if b.ref() is b:
+ a.count += 10 # ok
+ else:
+ a.count = 666 # not ok
+ fq = FQ()
def g():
b = B()
+ fq.register_finalizer(b)
ref = weakref.ref(b)
b.ref = ref
return ref
@@ -399,10 +559,19 @@
class A(object):
pass
class B(object):
- def __del__(self):
- self.wref().x += 1
+ pass
+ class FQ(rgc.FinalizerQueue):
+ Class = B
+ def finalizer_trigger(self):
+ while True:
+ b = self.next_dead()
+ if b is None:
+ break
+ b.wref().x += 1
+ fq = FQ()
def g(a):
b = B()
+ fq.register_finalizer(b)
b.wref = weakref.ref(a)
# the only way to reach this weakref is via B, which is an
# object with finalizer (but the weakref itself points to
@@ -448,9 +617,14 @@
def __init__(self):
self.id = b.nextid
b.nextid += 1
- def __del__(self):
- b.num_deleted += 1
- b.all.append(D(b.num_deleted))
+ fq.register_finalizer(self)
+ class FQ(rgc.FinalizerQueue):
+ Class = A
+ def finalizer_trigger(self):
+ while self.next_dead() is not None:
+ b.num_deleted += 1
+ b.all.append(D(b.num_deleted))
+ fq = FQ()
class D(object):
# make a big object that does not use malloc_varsize
def __init__(self, x):
@@ -461,6 +635,7 @@
i = 0
all = [None] * x
a = A()
+ del a
while i < x:
d = D(i)
all[i] = d
@@ -481,15 +656,24 @@
def __init__(self):
self.id = b.nextid
b.nextid += 1
- def __del__(self):
- llop.gc__collect(lltype.Void)
- b.num_deleted += 1
- C()
- C()
+ fq.register_finalizer(self)
class C(A):
- def __del__(self):
- b.num_deleted += 1
- b.num_deleted_c += 1
+ pass
+ class FQ(rgc.FinalizerQueue):
+ Class = A
+ def finalizer_trigger(self):
+ while True:
+ a = self.next_dead()
+ if a is None:
+ break
+ llop.gc__collect(lltype.Void)
+ b.num_deleted += 1
+ if isinstance(a, C):
+ b.num_deleted_c += 1
+ else:
+ C()
+ C()
+ fq = FQ()
def f(x, y):
persistent_a1 = A()
persistent_a2 = A()
diff --git a/rpython/memory/test/snippet.py b/rpython/memory/test/snippet.py
--- a/rpython/memory/test/snippet.py
+++ b/rpython/memory/test/snippet.py
@@ -1,5 +1,6 @@
import os, py
from rpython.tool.udir import udir
+from rpython.rlib import rgc
from rpython.rtyper.lltypesystem import lltype
from rpython.rtyper.lltypesystem.lloperation import llop
@@ -52,7 +53,7 @@
def set_age_of(c, newvalue):
# NB. this used to be a dictionary, but setting into a dict
# consumes memory. This has the effect that this test's
- # __del__ methods can consume more memory and potentially
+ # finalizer_trigger method can consume more memory and potentially
# cause another collection. This would result in objects
# being unexpectedly destroyed at the same 'state.time'.
state.age[ord(c) - ord('a')] = newvalue
@@ -61,12 +62,21 @@
def __init__(self, key):
self.key = key
self.refs = []
- def __del__(self):
+ fq.register_finalizer(self)
+
+ class FQ(rgc.FinalizerQueue):
+ Class = A
+ def finalizer_trigger(self):
from rpython.rlib.debug import debug_print
- debug_print("DEL:", self.key)
- assert age_of(self.key) == -1
- set_age_of(self.key, state.time)
- state.progress = True
+ while True:
+ a = self.next_dead()
+ if a is None:
+ break
+ debug_print("DEL:", a.key)
+ assert age_of(a.key) == -1
+ set_age_of(a.key, state.time)
+ state.progress = True
+ fq = FQ()
def build_example(input):
state.time = 0
@@ -150,11 +160,22 @@
class B:
count = 0
class A:
- def __del__(self):
- self.b.count += 1
+ pass
+
+ class FQ(rgc.FinalizerQueue):
+ Class = A
+ def finalizer_trigger(self):
+ while True:
+ a = self.next_dead()
+ if a is None:
+ break
+ a.b.count += 1
+ fq = FQ()
+
def g():
b = B()
a = A()
+ fq.register_finalizer(a)
a.b = b
i = 0
lst = [None]
diff --git a/rpython/memory/test/test_transformed_gc.py
b/rpython/memory/test/test_transformed_gc.py
--- a/rpython/memory/test/test_transformed_gc.py
+++ b/rpython/memory/test/test_transformed_gc.py
@@ -293,7 +293,7 @@
res = run([])
assert res == 42
- def define_finalizer(cls):
+ def define_destructor(cls):
class B(object):
pass
b = B()
@@ -316,6 +316,68 @@
return b.num_deleted
return f
_______________________________________________
pypy-commit mailing list
[email protected]
https://mail.python.org/mailman/listinfo/pypy-commit
