PEP-384 talks about defining a Stable ABI by making PyTypeObject opaque. Thus,
banning the use of static PyTypeObjects.
Specifically, I’d like to focus on those static PyTypeObjects that are
initialized as StructSequences. As of today, there are 14 instances of these
types (in timemodule.c, posixmodule.c, etc.) within cpython/Modules. These are
all initialized through PyStructSequence_InitType2. This is very problematic
when trying to make these types conform to PEP-384 as they are used through
static PyTypeObjects.
Problems:
* PyStructSequence_InitType2 overrides the PyTypeObject:
This C-API does a direct memcpy from a “prototype” structure. This effectively
overrides anything set within the PyTypeObject. For example, if we were to
initialize a heap allocated PyTypeObject and pass it on to this function, the
C-API would just get rid of the Py_TPFLAG_HEAPTYPE flag, causing issues with
the GC.
* PyStructSequence_InitType2 does not work with heap allocated
PyTypeObjects:
Even if the function is fixed to preserve the state of the PyTypeObject and
only overriding the specific slots (i.e. tp_new, tp_repr, etc.), it is expected
that PyStructSequence_InitType2 will call PyType_Ready on the object. That
means that the incoming object shouldn’t be initialized by a function such as
PyType_FromSpec, as that would have already called PyType_Ready on it.
Therefore, PyStructSequence_InitType2 will now have the responsibility of
setting all the slots and properties of the PyHeapTypeObject, which is not
feasible.
* PyStructSequence_NewType is non-functional:
This function was meant to be used as a way of creating a heap-allocated
PyTypeObject that be passed to PyStructSequence_InitType2, effectively
returning a heap allocated PyTypeObject. The current implementation doesn’t
work in practice. Given that this struct is created in the heap, the GC has
control over it. Thus, when the GC tries to traverse the type it complains
with: “Error: type_traverse() called for non-heap type”, since it doesn’t have
the Py_TPFLAG_HEAPTYPE flag. If we add the flag, we run into bullet point 1, if
we are able to preserve the flag then we will still run into the problem of
bullet point 2. Extra note: This C-API is not being used anywhere within
CPython itself.
Solution:
* Fix the implementation of PyStructSequence_NewType:
The best solution would be to fix the implementation of this function. This can
easily be done by dynamically creating a PyType_Spec and calling PyType_FromSpec
```
PyObject*
PyStructSequence_NewType(PyStructSequence_Desc *desc)
{
// …
PyType_Spec* spec = PyMem_NEW(PyType_Spec, 1);
spec->name = desc->name;
spec->basicsize = sizeof(PyStructSequence) - sizeof(PyObject *);
spec->itemsize = sizeof(PyObject *);
spec->flags = Py_TPFLAGS_DEFAULT;
spec->slots = PyMem_NEW(PyType_Slot, 6);
spec->slots[0].slot = Py_tp_dealloc;
spec->slots[0].pfunc = (destructor)structseq_dealloc;
// …
bases = PyTuple_Pack(1, &PyTuple_Type);
type = PyType_FromSpecWithBases(spec, bases);
// …
```
This will cleanly create a heap allocated PyStructSequence which can be used
just like any stack allocated PyTypeObject initialized through
PyStructSequence_InitType2. This means that any C-Extension should be using
PyStructSequence_NewType and only built-in types should be calling
PyStructSequence_InitType2. This will enable these types to comply with PEP-384
As an extra, I already have patches for this proposal. They can be found here:
Branch: https://github.com/eduardo-elizondo/cpython/tree/heap-structseq
Reimplement PyStructSequence_NewType:
https://github.com/eduardo-elizondo/cpython/commit/413f8ca5bc008d84b3397ca1c9565c604d54b661
Patch timemodule with NewType:
https://github.com/eduardo-elizondo/cpython/commit/0a35ea263a531cb03c06be9efc9e96d68162b308
Thoughts?
_______________________________________________
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
