New submission from Neil Schemenauer <nas-pyt...@arctrix.com>:
In the process of working on some garbage collector/obmalloc experiments, I noticed what seems to be a quirk about PyType_GenericAlloc(). It calls: size = _PyObject_VAR_SIZE(type, nitems+1); Note the "+1" which is documented as "for the sentinel". That code dates back to change "e5c691abe3946ddbaa00730b92f3b96f96903f7d" when Guido added support for heap types. This extra item is not added by _PyObject_GC_NewVar(). Also, the documentation for tp_alloc says that the size of the allocated block should be: tp_basicsize + nitems*tp_itemsize, rounded up to a multiple of sizeof(void*); The "+1" for the sentinel is definitely needed in certain cases. I think it might only be needed if 'type' is a subtype of 'type'. I.e. if Py_TPFLAGS_TYPE_SUBCLASS is set on 'type'. I haven't done enough analysis to fully understand this quirk yet but I think we are allocating extra memory quite regularly. Quite a lot of types use tp_alloc = PyType_GenericAlloc. E.g. the 'list' type or a subclass of the tuple type. It seems with the attached patch, unit tests still pass. Perhaps the +1 could be removed on the non-GC branch of the code as well. ---------- components: Interpreter Core files: generic_alloc_sentinel.txt messages: 345002 nosy: nascheme priority: normal severity: normal status: open title: PyType_GenericAlloc might over-allocate memory type: performance Added file: https://bugs.python.org/file48403/generic_alloc_sentinel.txt _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue37200> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
