Hey Victor First of all, I would like to thank Victor for all his efforts to resolve this issue. And I want to thank the other contributors to this issue as well! I am very happy to hear that we can close bpo-1635741. This has been a long mission requiring patience and I'm proud to finally be able to put an end to it.
Let's move forward to the next step! Warm regards, Dong-hee 2022년 1월 28일 (금) 오전 12:37, Victor Stinner <vstin...@python.org>님이 작성: > Hi, > > tl; dr Python no longer leaks memory at exit on the "python -c pass" > command ;-) > > > == Bug report == > > In 2007, the bpo-1635741 issue was reported on SourceForge: > "Interpreter seems to leak references after finalization". > > This bug is 15 years old. It saw the bugs migration from SourceForge > to Roundup (bugs.python.org) in 2007, the code migration from > Subversion to Mercurial in 2009, and the second code migration from > Mercurial to Git in 2016! > > Link to the main issue: https://bugs.python.org/issue1635741 > > Some people reported similar issues seen by WinCRT debug (bpo-6741), > MSVC Debug C Runtime (bpo-32026), GCC AddressSanitzer (bpo-26888), or > LeakSanitizer and Valgrind (bpo-21387). > > In general, "leaking memory" at Python exit doesn't matter, since the > operating system releases all memory when a process completes. It > matters when Python is embedded in an application. For example, Python > can be used by a plugin which is unloaded and reloaded multiple times. > It also matters for sub-interpreters. > > > == Tedious task == > > In the last 3 years, many people helped fixing this old issue by > converting static types to heap types, adding a module state to C > extension modules, converting extensions to the multi-phase > initialization API (PEP 489), fix many memory leaks, fix bugs, etc. > > When it became possible to cleanly unload an extension module (thanks > for the multi-phase init), tests on sub-intepreters (which load and > unload extension modules) showed many *old* reference leaks: all of > them have been fixed! When a test is run in sub-interpreters, Python > is able to detect leaks, whereas currently it doesn't check for memory > leaks at Python exit. > > > == Python 3.10 regressions == > > During the Python 3.10 development, we identified and fixed 3 major > regressions caused by this work: > > * Converting static types to heap types make them mutable: the > Py_TPFLAGS_IMMUTABLETYPE flag was added; 68 types use it in Python > 3.11 (bpo-43908). > > * It became possible to create uninitialized objects by instanciating > types (which should not be instanciated directly): the > Py_TPFLAGS_DISALLOW_INSTANTIATION was added; 41 types use it in Python > 3.11 (bpo-43916). For example, you cannot create an instance of > type(sys.flags). > > * Heap types must implement the full GC protocol: Py_TPFLAGS_HAVE_GC > flag, traverse and clear functions. Otherwise, the GC is unable to > break reference cycles, whereas a type contains (multiple) strong > references to itself (in the MRO tuple and in methods). All heap types > have been fixed to fully implement the GC protocol (bpo-40217). > > > == PEPs == > > The work relies on multiple PEPs: > > * PEP 489: Multi-phase extension module initialization > * PEP 573: Module State Access from C Extension Methods > * PEP 630: Isolating Extension Modules > > > == Persons who helped fixing the issue == > > Incomplete list of people who helped to fix this issue: > > * Christian Heimes (modules: symtable, _hashlib, ,_random, grp, pwd, > _queue, spwd, _struct, gc, _posixshmem, _posixsubprocess, select, > resource) > * Dong-hee Na (modules: _statistics, itertools, _heapq, _collections, > _uuid, math, _stat, syslog, errno, fcntl, mmap, _dbm, _gdbm, _lzma, > faulthandler, _bisect) > * Eric Snow (PEP 573) > * Erlend Egeberg Aasland (modules: _sqlite3, _sre, _multibytecodec) > * Hai Shi (modules: _json, _codecs, _crypt, _contextvars, _abc, _bz2, > _locale, audioop, _ctypes_test, _weakref, _locale) > * Marcel Plch (PEP 573) > * Martin von Löwis (PEP 3121) > * Mohamed Koubaa (modules: sha256, multiprocessing, _winapi, _blake2, > _sha3, _signal, _sha1, _sha512, _md5, zlib, _opcode, _overlapped, > _curses_panel, termios, _sha256, scproxy, cmath, _lsprof, unicodedata) > * Nick Coghlan (PEP 489, PEP 573) > * Paulo Henrique Silva (modules: time, operator, _functools) > * Petr Viktorin (PEP 489, PEP 573, PEP 630) > * Stefan Behnel (PEP 489) > * Victor Stinner (modules _string, mashal,_imp, _warnings, _thread) > > The work was scatted into many sub-issues, it was hard for me to track > all persons who contributed, sorry about that! I only searched for > "New changeset" in bpo-1635741. > > For me, it was a very pleasant collaborative work :-) Contributors > wrote pull requests, I reviewed and merged them. Slowly, contributors > started to review each others and shared some recipes for these tasks. > > > == Close the very old bpo-1635741 issue == > > Today, in the main development branch, Python no longer leaks memory > at exit for the simplest command: "python3 -c pass"! > > Using a Python debug build, you can check the "python -I -X > showrefcount -c pass" command (if you get a negative reference count, > see bpo-46449), or you can use a memory debugger like Valgrind. > > While the work is not 100% done, it's a great milestone (at least for > me ;-)! 15 years after bpo-1635741 was reported, finally I can close > it! Sadly, this bug will no see the bugs migration from Roundup > (bugs.python.org) to GitHub ;-) > > There are still some static types which should be converted to heap > types and some extensions which should be be ported to the multi-phase > initialization API. But this work can be done in existing or new > specific issues. > > Again, a *big* thanks to every single person who helped directly and > indirectly on fixing this issue! > > > Victor > -- > Night gathers, and now my watch begins. It shall not end until my death. > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/E4C6TDNVDPDNNP73HTGHN5W42LGAE22F/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/52RMBRSPTIN63P6TMJ3JAPYME7NL4INH/ Code of Conduct: http://python.org/psf/codeofconduct/
