On 2016-10-10 22:14, Larry Hastings wrote:
On 10/10/2016 09:36 PM, Chris Angelico wrote:
Hmm. Here's a naughty, and maybe dangerous, theory. Obtain a "memory
deallocation lock". While it is held (by any thread - it's a guard,
more than a lock), Py_DECREF will not actually deallocate memory -
objects can fall to zero references without being wiped. Once the
lock/guard is freed/cleared, anything that had fallen to zero is now
deallocated. This probably would mean stuffing them onto a list of
"doomed objects", and upon release of the guard, any doomed objects
that still have no refs would get deallocated.
If this worked, this would actually be really easy with my current
"buffered reference counting" approach. I literally already have a list
of doomed objects, which a separate thread queues up for each thread to
process later.
(This was a necessary part of "buffered reference counting", in which
reference count changes are stored in a transaction log and later
committed by a single thread. Since there's only one thread making
reference count changes, there's no contention, so it doesn't have to
use atomic incr/decr, which is a big performance win.)
But I don't think this fixes the problem. Consider:
1. Thread A calls Q = PyList_GetItem(L, 0), which returns a borrowed
reference. Thread A then gets suspended, before it has a chance to
Py_INCREF(Q).
2. Thread B does L.clear(), the reference count of Q goes to 0, Q is
added to the doomed objects list.
3. Thread A gets to run again. It does Py_INCREF(Q); Q's refcount is
now back up to 1, as if Q had been resurrected.
4. At some point in the future the "memory deallocation lock" is
released and Q is deleted.
If you say "well, just look at the reference count, and if it's 1 throw
it off the doomed objects list", keep in mind that I no longer have
accurate real-time reference counts. These hacks where we play games
with the reference count are mostly removed in my branch.
Also, I don't know when it would ever be safe to release the "memory
deallocation lock". Just because it's safe for your thread doesn't mean
it's safe for another thread. And if you do it on a thread-by-thread
basis, in the above example it might be safe from thread B's perspective
to release its "memory deallocation lock", but as illustrated that can
have an effect on thread A.
The deallocation lock could be a counter. Doomed objects would be
collectable when it's zero.
I appreciate the brainstorming but I'm not currently sanguine about this
idea,
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
