[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

Roundup Robot added the comment: New changeset c4c14e34e528 by Victor Stinner in branch 'default': Don't define _PyMem_PymallocEnabled() if pymalloc is disabled https://hg.python.org/cpython/rev/c4c14e34e528 -- ___ Python tracker

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

STINNER Victor added the comment: Buildbots are happy, I close the issue. -- resolution: -> fixed status: open -> closed ___ Python tracker ___

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

Roundup Robot added the comment: New changeset 7b079adb0774 by Victor Stinner in branch 'default': Enhance documentation on malloc debug hooks https://hg.python.org/cpython/rev/7b079adb0774 -- ___ Python tracker

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

Roundup Robot added the comment: New changeset 7534eb7bd57e by Victor Stinner in branch 'default': Issue #26516: Fix test_capi on AIX https://hg.python.org/cpython/rev/7534eb7bd57e -- ___ Python tracker

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

Roundup Robot added the comment: New changeset d8d6ec1333e6 by Victor Stinner in branch 'default': Issue #26516: Fix test_capi on 32-bit system https://hg.python.org/cpython/rev/d8d6ec1333e6 -- ___ Python tracker

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

Roundup Robot added the comment: New changeset 3c3df33f2655 by Victor Stinner in branch 'default': Issue #26516: Fix test_capi on Windows https://hg.python.org/cpython/rev/3c3df33f2655 -- ___ Python tracker

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

STINNER Victor added the comment: test_capi fails on Windows: http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/7348/steps/test/logs/stdio Probably a issue with newline characters. -- ___ Python tracker

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

STINNER Victor added the comment: Note: I just checked PYTHONMALLOCSTATS=1 with -X tracemalloc. It looks like it works as expected. Tracemalloc hooks are unregistered before stats are displayed at exit, _Pymem_PymallocEnabled() returns 1 as expected. --

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

STINNER Victor added the comment: I reworked my patch before pushing it: * it now respects -E and -I command line options * I documented changes in Doc/, Misc/NEWS, What's New in Python 3.6, Misc/README.valgrind, etc. * Debug hooks are now also installed when Python is compiled in debug mode

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

Roundup Robot added the comment: New changeset aa280432e9c7 by Victor Stinner in branch 'default': Add PYTHONMALLOC env var https://hg.python.org/cpython/rev/aa280432e9c7 -- ___ Python tracker

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

STINNER Victor added the comment: Oh, I forgot to update wmain() for Windows. Patch 4 handles wmain() too. I added a few comments and reformatted some parts of the code. -- Added file: http://bugs.python.org/file42119/pymem-4.patch ___ Python

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

STINNER Victor added the comment: Patch 3: - Ooops, I updated pymem_api_misuse(), but I forgot to update the related unit test. It's now fixed. -- Added file: http://bugs.python.org/file42104/pymem-3.patch ___ Python tracker

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

STINNER Victor added the comment: When I wrote the PEP 445, I already proposed to add a new environment variable to install debug hooks on a Python compiled in release mode: https://www.python.org/dev/peps/pep-0445/#add-a-new-pydebugmalloc-environment-variable But the proposition was rejected

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

STINNER Victor added the comment: More complete patch (version 2): * Document PYTHONMALLOC environment variable * Add PYTHONMALLOC=debug to install debug hooks without forcing a specific memory allocator (keep the default memory allocator) * Fix sys._debugmallocstats() for PYTHONMALLOC=malloc:

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

Changes by Serhiy Storchaka : -- stage: -> patch review ___ Python tracker ___ ___

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

Roundup Robot added the comment: New changeset af2fc10f703b by Victor Stinner in branch '3.5': Issue #26516: Enhance Python mem allocators doc https://hg.python.org/cpython/rev/af2fc10f703b -- nosy: +python-dev ___ Python tracker

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

STINNER Victor added the comment: The motivation to support PYTHONMALLOC=malloc (always use malloc) and not just PYTHONMALLOC=debug (enable debug hooks) is to allow to use external memory debugger like Valgrind. Valgrind doesn't like pymalloc (pymalloc raises false alarms), we had to a

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

STINNER Victor added the comment: Example with Python compiled in release mode. By default, a buffer overflow is not detected. It may crash later, in a random place... $ ./python -c 'import _testcapi; _testcapi.pymem_buffer_overflow()' Enabling debug hooks detects the buffer overflow

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

STINNER Victor added the comment: The output of sys._debugmallocstats() contains different info: * Stats of type free lists * pymalloc stats * number of calls to memory allocators The available info depends on: * pymalloc disabled at compilation with ./configure --without-pymalloc * pymalloc

[issue26516] Add PYTHONMALLOC env var and add support for malloc debug hooks in release mode

New submission from STINNER Victor: Attached patch: - Add PYTHONMALLOC env var which accepts 4 values: * pymalloc: use pymalloc for PyObject_Malloc(), malloc for PyMem_Malloc() and PyMem_RawMalloc() * pymalloc_debug: pymalloc + debug hooks * malloc: use malloc for PyObject_Malloc(),