New submission from STINNER Victor <vstin...@redhat.com>:
Currently, the Python initialization code base is full of:
static void
pymain_clear_cmdline(_PyMain *pymain, _PyCmdline *cmdline)
{
PyMemAllocatorEx old_alloc;
_PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
...
PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
}
It's a hack that I wrote when I worked on the Python initialization code, to
make sure that we use the same memory allocator to allocate and release memory.
The problem is that Python allocates memory at startup, then the memory
allocator can be changed in Py_Initialize() (ex: by PYTHONMALLOC environment
variable), but at exit, we need to reuse the same memory allocator that we used
to allocate memory early.
Instead of "forcing" the memory allocator to "default allocator", I propose to
add a new "context" structure which contains the memory allocator
(PyMemAllocatorEx structure). Example:
typedef struct {
/* Enable UTF-8 mode?
Set by -X utf8 command line option and PYTHONUTF8 environment variable.
If set to -1 (default), inherit Py_UTF8Mode value. */
int utf8_mode;
PyMemAllocatorEx raw_alloc;
} _PyConfigCtx;
The context should be passed to any function allocating directly or indirectly
memory.
Py_Main(), "core config" and "path config" are good target for these changes.
These functions are used during Python initialization.
I know that Eric Snow would like to pass a "context" to functions of the C API.
I don't want to make a giant change of the C API. I only want to modify the
internal C API, and only modifiy Python internal. The context that I propose is
designed for early Python initialization, and fix the practical problem of
memory allocator.
Later we can discussed how _PyRuntime and the pointer to the current
interpreter can be added into this context, or if a new context should be
created.
Attached PR is a proof-of-concept of my idea, but the giant PR might be
splitted into smaller changes to more incremental changes.
----------
components: Interpreter Core
messages: 330001
nosy: eric.snow, ncoghlan, vstinner
priority: normal
severity: normal
status: open
title: Internal C API: pass the memory allocator in a context
type: enhancement
versions: Python 3.8
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue35265>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
